CN106204447A - The super resolution ratio reconstruction method with convolutional neural networks is divided based on total variance - Google Patents
The super resolution ratio reconstruction method with convolutional neural networks is divided based on total variance Download PDFInfo
- Publication number
- CN106204447A CN106204447A CN201610515218.XA CN201610515218A CN106204447A CN 106204447 A CN106204447 A CN 106204447A CN 201610515218 A CN201610515218 A CN 201610515218A CN 106204447 A CN106204447 A CN 106204447A
- Authority
- CN
- China
- Prior art keywords
- image
- convolutional neural
- neural networks
- texture
- total variance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 95
- 238000013527 convolutional neural network Methods 0.000 title claims abstract description 40
- 230000003321 amplification Effects 0.000 claims abstract description 20
- 238000003199 nucleic acid amplification method Methods 0.000 claims abstract description 20
- 230000015556 catabolic process Effects 0.000 claims abstract description 16
- 238000012549 training Methods 0.000 claims description 14
- 238000010606 normalization Methods 0.000 claims description 3
- 208000031481 Pathologic Constriction Diseases 0.000 claims 1
- 238000005259 measurement Methods 0.000 claims 1
- 210000001215 vagina Anatomy 0.000 claims 1
- 230000000694 effects Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 238000005457 optimization Methods 0.000 description 3
- 238000012937 correction Methods 0.000 description 2
- 238000000354 decomposition reaction Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 230000035939 shock Effects 0.000 description 2
- 101100365548 Caenorhabditis elegans set-14 gene Proteins 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000008929 regeneration Effects 0.000 description 1
- 238000011069 regeneration method Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000004513 sizing Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Image Processing (AREA)
Abstract
A kind of divide the super resolution ratio reconstruction method with convolutional neural networks based on total variance, comprise the steps: picture breakdown step, be taken based on the method that total variance divides and the picture breakdown of original low-resolution is become structure division and texture part；Structure division image amplification procedure, first is amplified obtaining initial magnified image to described structure division by linear interpolation, is then sharpened edge with sharpening filter, finally carries out modified result；Texture part image reconstruction step, is amplified for described texture part by linear interpolation, the image input convolutional neural networks after amplifying, the texture image after being rebuild after computing；And image integrating step, the structure division image after described amplification and the texture image after described reconstruction are combined, generates final super-resolution image.The image of super-resolution rebuilding of the present invention keeps edge and the texture structure of image simultaneously, and reduces computational complexity, meets the requirement of real-time.
Description
Technical field
The invention belongs to technical field of video image processing, particularly relate to one and divide and convolutional Neural net based on total variance
The super resolution ratio reconstruction method of network.
Background technology
Along with popularizing of digital product, image obtains the main source of information as the mankind, has obtained the most widely
Application.Meanwhile, digital image processing techniques have also been obtained and develop rapidly.And the collection of video image is digital image processing system
In a crucial step.During digital collection, affected by following factor, image resolution ratio and picture quality
Can decline: sample frequency lack sampling makes the spectral aliasing of image, degrades because of anamorphic effect；Atmospheric perturbation, de-
Burnt, relative motion between size sensor and image capture device and subject, can cause the fuzzy of image；And
In the acquisition of image, transmission and storing process, also can introduce noise, such as Gaussian noise, image also can be made to degrade.
Therefore, resolution and the quality of image how are improved so that it is become in recent years as close as original image
One of study hotspot of image processing field in the world.And along with the development of image processing techniques and computer computation ability not
Disconnected lifting, the reconstruction that super-resolution rebuilding technology is low-resolution image of video image provides good solution.It
The image of a series of low resolution can be amplified according to a certain percentage, final generation one width or several high-resolution figures
Picture, and well keep the structure of artwork.
Existing super resolution ratio reconstruction method is broadly divided into three major types: the first kind is super-resolution technique based on interpolation,
Equations of The Second Kind is based on the super-resolution technique rebuild, and the 3rd class is super-resolution technique based on study.
But simple linear interpolation techniques, such as bilinearity and bicubic interpolation, calculate simple can produce sawtooth effect,
Simultaneously also can fuzzy edge.In order to preferably keep the acutance at edge, the interpolation method much instructed based on edge is carried in succession
Go out.Researcher is had to propose to estimate on low-resolution image the covariance of high-definition picture in calendar year 2001, then with this association side
Difference carries out interpolation.There is researcher to propose a kind of autoregression model based on piecemeal in 2008, once estimate monoblock pixel.Have
Researcher proposed the soft decision interpolation technique of a kind of robust in 2012, in the estimation of parameter and pixel, all uses and weights
Little square law.But, these methods all only considered the reconstruction of marginal portion, does not accounts for the reconstruction of texture part.
Based on the super-resolution technique rebuild, it is the inverse process that degrades of analog image, goes to solve an optimization method.Image drops
Matter process is, a panel height image in different resolution, after fuzzy, down-sampled obtains low-resolution image.There is researcher 2005
The method divided based on image total variance year proposed is in such method the most representational one.In the method, image
Total variance is allocated as into bound term, being added in optimization method, thus the solution of restricted problem.It is permissible while keeping edge sharpness
Suppress artificial effect greatly.Researcher is had to propose to estimate high resolution graphics by the gradient of low-resolution image in 2011
As the gradient at edge, the gradient then estimation obtained, as bound term, joins in optimization method.
In recent years, some super resolution ratio reconstruction methods based on study are the most constantly suggested.Researcher is had to carry in 2010
Go out a kind of super resolution ratio reconstruction method based on rarefaction representation.The method proposes, and image block can be by a super complete dictionary
In element represent by the way of linear combination, wherein, the number of nonzero coefficient can be lacked as far as possible.So, first produce
Two super complete dictionary set, the image block in the two set is one to one, is low-resolution image and height respectively
Image in different resolution.For the arbitrarily low image in different resolution block of input, low-resolution dictionary is found a kind of rarefaction representation, then
In high-resolution dictionary, high-definition picture block is generated with this group is sparse.Researcher is had to propose to use the degree of depth in 2016
The method practised rebuilds high-definition picture.Basic skills is, generates many group low resolution and high-definition picture pair, then
Low-resolution image, as the input of convolutional neural networks, using high-definition picture as the output of convolutional neural networks, is trained
Network.For the network trained, using arbitrarily low image in different resolution as input, produce high-definition picture as rebuilding knot
Really.
Existing several method for reconstructing is respectively present defect: method based on interpolation, and amount of calculation is low, but rebuild
Weak effect；Based on the method rebuild, it is impossible to edge and two parts of texture are the most well rebuild simultaneously；Method based on study
Computer complexity is high, and the selection for training storehouse also has the strongest dependency.
Summary of the invention
It is an object of the invention to realize image super-resolution rebuilding, keep edge and the texture structure of image simultaneously, and drop
Low computational complexity, meets the requirement of real-time.
To achieve these goals, the present invention proposes and a kind of divides the super-resolution with convolutional neural networks based on total variance
Method for reconstructing, in conjunction with based on rebuilding and the super-resolution technique of study, can well tie the marginal texture of image and texture
Structure is rebuild.
Technical scheme is as follows:
A kind of divide the super resolution ratio reconstruction method with convolutional neural networks based on total variance, comprise the steps:
Picture breakdown step, is taken based on the method that total variance divides and the picture breakdown of original low-resolution is become structure division
And texture part；
Structure division image amplification procedure, first is amplified obtaining initial enlarged drawing to described structure division by linear interpolation
Picture, is then sharpened edge with sharpening filter, finally carries out modified result；
Texture part image reconstruction step, is amplified for described texture part by linear interpolation, the figure after amplifying
As input convolutional neural networks, the texture image after being rebuild after computing；And
Image integrating step, combines the structure division image after described amplification and the texture image after described reconstruction, raw
Become final super-resolution image
Total variance is divided, and refers to the intensity of variation sum of signal, for two dimensional image, total variance divide be exactly image gradient it
With.In picture breakdown step, the method divided based on total variance is to solve the following equation that minimizes:
Wherein, f represents the image of original low-resolution described in described picture breakdown step, and u represents described picture breakdown
Structure division described in step,Being the gradient of structure division u, λ is Lagrange multiplier, and its optimum value is 0.85.
Structure division image amplification procedure is carried out when linear interpolation is amplified, use bicubic linear interpolation techniques.
In structure division image amplification procedure, it is by pixel is iterated behaviour that edge is sharpened by sharpening filter
Realize, specific as follows:
Wherein, IunRepresenting that nth iteration operates the image obtained, during n=1, Iu representative structure image carries out linear interpolation
Initial pictures after amplification；T is iteration step length, Δ IunWithCalculate in the following manner,
Wherein, IuxAnd IuyIt is image I respectivelyuFirst derivative both horizontally and vertically.
The optimum value of iteration total degree n is 50, and the optimal value of iteration step length t is 0.1.
In structure division image amplification procedure, the method for modified result is that each pixel uses its periphery S × S window
Interior pixel is weighted averagely obtaining revised gray value.
Wherein, weights use similarity based on gray-scale intensity and the pixel of intensity profile estimate, described based on
The similarity of the pixel of gray-scale intensity and intensity profile passes through gray-scale intensity and the ash of pixel periphery N × N respective image block
The similarity of degree distribution is estimated, according to equation below:
Wherein, (m n) represents pixel y (m, weights n) in imparting SxS window to ω；(i j) is normalization constant, generation to Z
The summation of all weights of table；Parameter σ1And σ2The rate of decay of control characteristic equation, (m n) is pixel y (i, j) periphery N × N to d
Pixel composition image block N (i, j) and pixel y (m, n) periphery N × N pixel composition image block N (m, n) it
Between gray-scale intensity difference, h (m, n) be image block N (i, j) and N (m, n) between intensity distribution difference.
Texture part image reconstruction step is carried out when linear interpolation is amplified, use bicubic linear interpolation techniques.
Convolutional neural networks in texture part image reconstruction step includes input layer, hidden layer and output layer；Texture portion
Computing in partial image reconstruction procedures is based on convolutional neural networks model and rebuilds the image after described amplification, convolution god
Being set up by training through network model, training method is as follows:
Choose some width texture images as training set, to every piece image, artwork is used according to a certain percentage double three
The method of secondary interpolation carries out down-sampled, using the image after down-sampled as low-resolution image, using artwork as target image,
By low-resolution image by bicubic interpolation, being amplified to size consistent with artwork, the image division after amplifying becomes multiple solid
The image block of sizing, corresponding artwork divides the most in the same way, thus constitutes input and output image pair, finally joins
To training.
There is advantages that
The a kind of of the present invention divides the super resolution ratio reconstruction method with convolutional neural networks based on total variance, picture breakdown is become
Structural images and texture image, and rebuild by different methods respectively according to the feature of both images.Wherein, for
The reconstruction of structural images, first uses simple linear interpolation techniques, is amplified to consistent with target sizes, then uses a sharpening
Wave filter, iteration image is sharpened, afterwards, carry out interpolation correction, make the structural images after finally amplifying to have
Sharp keen edge, also can suppress the generation of sawtooth effect.For the reconstruction of texture structure, make full use of convolutional neural networks to height
Frequently the ability of information reconstruction, meanwhile, only processes texture image, enormously simplify computational complexity.This method can be same
Time edge and the texture of image are well rebuild, lay a good foundation for follow-up application, meanwhile, the most relatively low computing
Amount, can meet the requirement of real-time.
Accompanying drawing explanation
Fig. 1 be the present invention divide the super resolution ratio reconstruction method flow chart with convolutional neural networks based on total variance；
Fig. 2 is the picture breakdown schematic diagram of the embodiment of the present invention, and wherein, (a) is original image, and (b) is structure division,
C () is texture part；
Fig. 3 is the sub-process schematic diagram in the embodiment of the present invention being amplified structure division；
Fig. 4 is that the structural images of the embodiment of the present invention carries out comparison diagram before and after modified result, wherein, before (a) is correction
Design sketch, (b) is revised design sketch；
Fig. 5 is the sub-process schematic diagram of the image reconstruction of texture part in the embodiment of the present invention；
Fig. 6 is the training method flow chart of the convolutional neural networks of the embodiment of the present invention；
Fig. 7 is the training process schematic of the convolutional neural networks of the embodiment of the present invention.
Detailed description of the invention
Below in conjunction with accompanying drawing, the present invention is elaborated further.
The super resolution ratio reconstruction method with convolutional neural networks is divided based on total variance, including walking as follows as it is shown in figure 1, a kind of
Rapid:
Carry out picture breakdown in step sl, piece image f is resolved into structure division u and texture part v, f=u+v.
Wherein structure division relative smooth, and there is sharp keen edge, and texture part comprises texture and the details of image.Decomposition is adopted
By the method divided based on total variance.Total variance is divided, and refers to the intensity of variation sum of signal, and for two dimensional image, total variance is just divided
It it is the gradient sum of image.The problem of picture breakdown solves by minimizing equation below solving:
WhereinBeing the gradient of image u, gradient is the least, and explanatory diagram picture is the most smooth, and λ is Lagrange multiplier, is used for balancing
This two-part weight.λ is the biggest, illustrate u closer to f, the most smooth；λ is the least, and the weight of total variance subitem is just
The biggest, illustrate that u is the most smooth.Being found through experiments, when λ takes 0.85, discomposing effect is preferable.Decomposition result is as shown in Figure 2.Its
In, upper images is original image f, and lower left figure is structure division u, and lower right figure is texture part v.
In step s 2, structure division is amplified, the acutance at artwork edge to be kept.Structural images comprises
The edge of image and smooth, to the amplification flow chart of structure division as in figure 2 it is shown, include three below step:
Carry out linear interpolation amplification in the step s 21.First, traditional bicubic interpolation is used to process the structure chart of input
Picture, it is thus achieved that structural images Iu after initial amplification.
Bicubic interpolation is again bi-cubic interpolation, be for " interpolation " (Interpolating) in the picture or increase " as
Element " a kind of method of (Pixel) number/density is to create the image border more smoother than bilinear interpolation.
In step S22, carry out structural images based on impact filtering sharpen.Shock filter is used to carry out sharp to edge
Change operation.Shock filter is as follows to the iterative operation of pixel:
Wherein, t is iteration step length, Δ IunWithCalculate in the following manner:
Wherein, IuxAnd IuyIt is that image is in first derivative both horizontally and vertically.Test result indicate that, when iteration is the most secondary
Number is 50, can obtain and preferably sharpen effect when that iteration step length being 0.1.
Above sharpening operation also can introduce some sawtooth while keeping edge sharpness, so, in step S23, right
The result of interpolation is modified.Revise the mode using non-local mean filtering, for each pixel, use periphery 21 × 21
In window, pixel is weighted averagely obtaining revised gray value.Weights use the similarity between pixel to estimate
Meter, the similarity of pixel is defined by the similarity of image block, the most accurate the most more robustness.
Assume current pixel point be y (i, j), the image block of the pixel of its periphery NxN composition be N (i, j).Assume image
In another pixel be y (m, n), the image block of the pixel of its periphery NxN composition be N (m, n).Pixel y (i, j) and y (m,
N) similarity between is estimated by the gray-scale intensity of respective image block and the similarity of intensity profile.Because structural images master
Will be based on smooth region and marginal area, therefore, add gray-scale intensity and be distributed this, two figures can be estimated more accurately
As the similarity between block.Gray-scale intensity difference between two image blocks is defined by below equation:
Wherein,It it is second normal form operator.
For the intensity profile of image block, the method for perception Hash is used to measure.Assume binary picture block H (i,
J) (m, (i, j) with N (m, cryptographic Hash n) n) to be respectively image block N with H.For image block N, (i j), first calculates its ash
Degree meansigma methods.Then for N (i, j) in each pixel, if its gray value is more than average gray value, in H (i, j) phase
Answer position to be entered as 1, be otherwise entered as 0.In like manner calculate image block N (m, cryptographic Hash n).Gray scale between two image blocks is divided
Cloth difference is defined by below equation:
Wherein,It it is first normal form operator.According to d, (m, n) (m, two class similaritys n) defined, to pixel y with h
(m, n) give a weights ω (m, n), be used for measure similarity, as shown by the following formula:
Wherein, (i is j) normalization constant, represents the summation of all weights, parameter σ Z1And σ2Declining of control characteristic equation
Deceleration.Difference between image block is the biggest, and the weights giving respective pixel point are the least, otherwise, then weights are the biggest.Here, will
Window block size N × N is set to 7 × 7, σ1Size takes the variance of 7 × 7 image blocks, σ2Take 0.1.
Revise before and revise after Comparative result figure as it is shown on figure 3, wherein left hand view be the figure of unmodified, right part of flg is for repairing
Figure after just.It can be seen that the structural images after Xiu Zhenging can have sharp keen edge, the generation of sawtooth effect also can be suppressed.
In step s3 texture image is rebuild.Use method based on convolutional neural networks.Fig. 5 is texture part
The flow chart of image reconstruction, concrete steps include:
In step S31, carry out linear interpolation amplification, use traditional bicubic interpolation to process the texture image of input, obtain
Obtain the texture image after initially amplifying.
Carry out texture image reconstruction based on convolutional neural networks in step s 32.Image input convolution god after amplification
Through network, export the texture image after being rebuild.
Tradition method based on convolutional neural networks is, using low-resolution image as input, high-definition picture conduct
Output, owing to being that the full content to image is rebuild, therefore, the structure of network is relative complex, and rebuilds effect and rely on
The size of image library.This will carry great computational complexity.The convolutional neural networks of the employing in the present invention, after decomposing
Texture image as input, the texture image after reconstruction is as output.Meanwhile, convolutional neural networks only comprises three layers, point
It is not input layer, hidden layer, output layer.Because having only to texture structure is rebuild.
Reconstruction needs to set up convolutional neural networks model, and specific practice is:
Choose in MBT data base 154 width texture images as training set.Training method as shown in Figure 6, in step S321,
The method that artwork uses bicubic interpolation according to a certain percentage carries out down-sampled, is then low point by the image after down-sampled
Resolution image, artwork is target image；In step S322, low-resolution image is amplified to and artwork one by bicubic interpolation
Cause size；In step S323, the image division after amplifying becomes the image block of multiple fixed size, and corresponding artwork is also by identical
Model split；In step S323, carry out image pairing training.
Training process is illustrated below as a example by Fig. 7.Taking a size in image is that the image block of M × M is as input.
Then by being carried out convolution algorithm respectively on image block by n1 convolution kernel.As a example by one of them convolution kernel, it is assumed that convolution
The size of core is c × f1 × f1, and wherein c is the port number of input picture, and f1 × f1 is the space size of convolution kernel.Then image
Any one pixel in block, takes the sub-block of its periphery f1 × f1 size, and f1 × f1 core is weights, is weighted average, obtains
New pixel value, as this convolution kernel eigenvalue in this pixel position.Therefore, a total of n1 convolution kernel, n1 can be produced
Width characteristic pattern.It is expressed as follows by formula:
F1(Y)=max (0, W1*Y+B1),
Wherein, W1Represent convolution kernel, B1Represent deviation.* convolution algorithm is represented.
Then, at phase of regeneration, remaining employing convolution, now, size n1 of convolution kernel × f2 × f2, wherein n1 is
The output characteristic figure number of one convolutional layer, f2 × f2 is the space size of convolution kernel.A total of c convolution kernel, because output
Image is c passage.Convolution algorithm process is consistent with the convolution algorithm process of ground floor.The output of this layer is the result of reconstruction.
It is expressed as follows by formula:
F2(Y)=max (0, W2*F1(Y)+B2),
Wherein, this example is coloured image is transformed into YCbCr space from rgb space, and only to brightness Y at
Reason, therefore, port number c is 1, and M × M takes 33 × 33, and f1 × f1 takes 9 × 9, and n1 takes 96, and f2 × f2 takes 5 × 5.Convolutional neural networks
Use caffe framework.
After obtaining the convolutional neural networks model trained, the texture maps after being amplified by bicubic interpolation
Picture, as the input of convolutional neural networks, output is exactly the high-resolution texture image rebuild.
In step s 4, the structural images after amplifying and texture image combination, generate final super-resolution image.
In order to verify the superiority of the present invention, contrasted, including biography with existing super-resolution rebuilding algorithm
The bicubic interpolation algorithm of system, gradient instructs interpolation algorithm, biased field image multiplication method, and based on rarefaction representation super-resolution
Rate algorithm, evaluation is to carry out on 14 width images in image set Set14, calculates objective commenting by each algorithm is rebuild image
It is worth, including average peak signal to noise ratio (PSNR) and the structural similarity index (SSIM) of 14 width images, can objectively demonstrate,prove
The effect of the bright present invention is best, as shown in table 1.
Table 1 PSNR and SSIM evaluation result
Claims (10)
1. one kind is divided the super resolution ratio reconstruction method with convolutional neural networks based on total variance, it is characterised in that to low resolution
Image carry out super-resolution rebuilding, comprise the steps:
Picture breakdown step, is taken based on the method that total variance divides and the picture breakdown of original low-resolution is become structure division and stricture of vagina
Reason part；
Structure division image amplification procedure, first is amplified obtaining initial magnified image to described structure division by linear interpolation,
Then with sharpening filter, edge is sharpened, finally carries out modified result；
Texture part image reconstruction step, is amplified for described texture part by linear interpolation, and the image after amplifying is defeated
Enter convolutional neural networks, the texture image after being rebuild after computing；And
Image integrating step, combines the structure division image after described amplification and the texture image after described reconstruction, generates
Whole super-resolution image.
The most according to claim 1 a kind of divide the super resolution ratio reconstruction method with convolutional neural networks based on total variance, its
It is characterised by: the method divided based on total variance described in described picture breakdown step is to minimize equation below solving,
Wherein, f represents the image of original low-resolution described in described picture breakdown step, and u represents described picture breakdown step
Described in structure division,Being the gradient of structure division, λ is Lagrange multiplier, and f is the image of original low-resolution.
The most according to claim 2 a kind of divide the super resolution ratio reconstruction method with convolutional neural networks based on total variance, its
It is characterised by: described λ value is 0.85.
The most according to claim 1 a kind of divide the super resolution ratio reconstruction method with convolutional neural networks based on total variance, its
It is characterised by: linear interpolation described in described structure division image amplification procedure amplifies employing bicubic linear interpolation techniques.
The most according to claim 1 a kind of divide the super resolution ratio reconstruction method with convolutional neural networks based on total variance, its
It is characterised by: with sharpening filter, edge is sharpened, by pixel described in described structure division image amplification procedure
It is iterated operation to realize, iterative operation foundation below equation,
Wherein, n is iteration total degree, IunRepresenting that nth iteration operates the image obtained, t is iteration step length, Δ IunWith
Calculate in the following manner,
Wherein, IuxAnd IuyIt is image I respectivelyuFirst derivative both horizontally and vertically.
The most according to claim 5 a kind of divide the super resolution ratio reconstruction method with convolutional neural networks based on total variance, its
It is characterised by: described iteration total degree n is 50, and described iteration step length t is 0.1.
The most according to claim 1 a kind of divide the super resolution ratio reconstruction method with convolutional neural networks based on total variance, its
Being characterised by, described in described structure division image amplification procedure, the method for modified result is that each pixel is used its periphery
In SxS window, pixel is weighted averagely obtaining revised gray value；
Wherein, weights use similarity based on gray-scale intensity and the pixel of intensity profile to estimate, described based on gray scale
The similarity measurement of the pixel of intensity and intensity profile passes through gray-scale intensity and the ash of pixel periphery N × N respective image block
The similarity of degree distribution is estimated, according to equation below:
Wherein, (m, in n) representing imparting SxS window, ((i is j) normalization constant, represents institute Z pixel y ω for m, weights n)
There are the summation of weights, parameter σ1And σ2The rate of decay of control characteristic equation, (m n) is pixel y (i, j) picture of periphery N × N to d
Vegetarian refreshments composition image block N (i, j) and pixel y (m, n) periphery N × N pixel composition image block N (m, n) between
Gray-scale intensity difference, h (m, n) be image block N (i, j) and N (m, n) between intensity distribution difference.
The most according to claim 1 a kind of divide the super resolution ratio reconstruction method with convolutional neural networks based on total variance, its
Being characterised by, linear interpolation described in described texture part image reconstruction step amplifies employing bicubic linear interpolation techniques.
The most according to claim 1 a kind of divide the super resolution ratio reconstruction method with convolutional neural networks based on total variance, its
Being characterised by, described in described texture part image reconstruction step, convolutional neural networks includes input layer, hidden layer and output layer.
The most according to claim 1 a kind of divide the super resolution ratio reconstruction method with convolutional neural networks based on total variance, its
Be characterised by, computing described in described texture part image reconstruction step be based on convolutional neural networks model to described amplification after
Image rebuild,
Convolutional neural networks model is set up by training, and training method is as follows:
Choose some width texture images as training set, to every piece image, artwork uses bicubic insert according to a certain percentage
The method of value carries out down-sampled, using the image after down-sampled as low-resolution image, using artwork as target image, by low
Image in different resolution passes through bicubic interpolation, is amplified to size consistent with artwork, and the image division after amplifying becomes multiple fixing big
Little image block, corresponding artwork divides the most in the same way, thus constitutes input and output image pair, finally carries out pairing instruction
Practice.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610515218.XA CN106204447A (en) | 2016-06-30 | 2016-06-30 | The super resolution ratio reconstruction method with convolutional neural networks is divided based on total variance |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610515218.XA CN106204447A (en) | 2016-06-30 | 2016-06-30 | The super resolution ratio reconstruction method with convolutional neural networks is divided based on total variance |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106204447A true CN106204447A (en) | 2016-12-07 |
Family
ID=57465735
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610515218.XA Pending CN106204447A (en) | 2016-06-30 | 2016-06-30 | The super resolution ratio reconstruction method with convolutional neural networks is divided based on total variance |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106204447A (en) |
Cited By (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107087201A (en) * | 2017-05-10 | 2017-08-22 | 西安万像电子科技有限公司 | Image processing method and device |
| CN107240076A (en) * | 2017-05-31 | 2017-10-10 | 北京小米移动软件有限公司 | Image processing method and device |
| WO2018120723A1 (en) * | 2016-12-30 | 2018-07-05 | 平安科技（深圳）有限公司 | Video compressive sensing reconstruction method and system, and electronic apparatus and storage medium |
| CN108364258A (en) * | 2018-02-07 | 2018-08-03 | 浙江师范大学 | A kind of method and system improving image resolution ratio |
| CN108629733A (en) * | 2017-03-17 | 2018-10-09 | 北京大学 | Obtain the method and apparatus of high-definition picture |
| CN108876756A (en) * | 2017-05-09 | 2018-11-23 | 普天信息技术有限公司 | The measure and device of image similarity |
| WO2018214671A1 (en) * | 2017-05-26 | 2018-11-29 | 杭州海康威视数字技术股份有限公司 | Image distortion correction method and device and electronic device |
| CN108960084A (en) * | 2018-06-19 | 2018-12-07 | 清华大学深圳研究生院 | Target tracking method, system, readable storage medium storing program for executing and electronic equipment |
| CN109087242A (en) * | 2017-06-13 | 2018-12-25 | 上海富瀚微电子股份有限公司 | A kind of cfa interpolation processing method and processing device |
| CN109429030A (en) * | 2017-08-31 | 2019-03-05 | 爱唯秀股份有限公司 | The method for rebuilding video using super-resolution algorithms |
| CN109523584A (en) * | 2018-10-26 | 2019-03-26 | 上海联影医疗科技有限公司 | Image processing method, device, multi-mode imaging system, storage medium and equipment |
| CN109528230A (en) * | 2018-11-21 | 2019-03-29 | 济南浪潮高新科技投资发展有限公司 | A kind of tumor of breast dividing method and device based on multi-stage transformation network |
| CN109859145A (en) * | 2019-02-27 | 2019-06-07 | 长安大学 | It is a kind of that texture method is gone with respect to the image of total variance based on multistage weight |
| WO2019120110A1 (en) * | 2017-12-20 | 2019-06-27 | 华为技术有限公司 | Image reconstruction method and device |
| CN109949332A (en) * | 2017-12-20 | 2019-06-28 | 北京京东尚科信息技术有限公司 | Method and apparatus for handling image |
| CN110555823A (en) * | 2019-04-11 | 2019-12-10 | 江南大学 | Image fusion quality evaluation method based on TVL structure texture decomposition |
| CN110838085A (en) * | 2018-08-15 | 2020-02-25 | Tcl集团股份有限公司 | Super-resolution reconstruction method and device for image and electronic equipment |
| WO2020081770A1 (en) * | 2018-10-18 | 2020-04-23 | Sony Corporation | Enhanced color reproduction for upscaling |
| US10805634B2 (en) | 2017-05-17 | 2020-10-13 | Samsung Electronics Co., Ltd | Super-resolution processing method for moving image and image processing apparatus therefor |
| CN112116526A (en) * | 2019-06-19 | 2020-12-22 | 中国石油化工股份有限公司 | Torch smoke image super-resolution method based on deep convolutional neural network |
| CN112435160A (en) * | 2020-11-06 | 2021-03-02 | 上海航天控制技术研究所 | Depth map super-resolution method based on generative countermeasure network |
| CN113271406A (en) * | 2020-02-14 | 2021-08-17 | 逐点有限公司 | Method and system for image processing with multiple image sources |
| CN116258735A (en) * | 2023-05-16 | 2023-06-13 | 四川省肿瘤医院 | Cervical cancer inter-tissue insertion needle reconstruction system |
| CN116402691A (en) * | 2023-06-05 | 2023-07-07 | 四川轻化工大学 | Image super-resolution method and system based on rapid image feature stitching |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104103052A (en) * | 2013-04-11 | 2014-10-15 | 北京大学 | Sparse representation-based image super-resolution reconstruction method |
-
2016
- 2016-06-30 CN CN201610515218.XA patent/CN106204447A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104103052A (en) * | 2013-04-11 | 2014-10-15 | 北京大学 | Sparse representation-based image super-resolution reconstruction method |
Non-Patent Citations (5)
| Title |
|---|
| E.MCCARTHY 等: "A FRAMEWORK FC3R SOFT HASHING AND ITS APPLICATION TO ROBUST IMAGE HASHING", 《2004 INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP)》 * |
| 李兰兰: "基于偏微分方程的图像复原和增强算法研究", 《万方数据库》 * |
| 梁广顺 等: "基于双边滤波与非局部均值的图像去噪研究", 《光电子·激光》 * |
| 符冉迪 等: "基于TV-L分解的红外云图超分辨率算法", 《光学精密工程》 * |
| 胡传平等: "基于深度学习的图像超分辨率算法研究", 《铁道警察学院学报》 * |
Cited By (44)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018120723A1 (en) * | 2016-12-30 | 2018-07-05 | 平安科技（深圳）有限公司 | Video compressive sensing reconstruction method and system, and electronic apparatus and storage medium |
| US10630995B2 (en) | 2016-12-30 | 2020-04-21 | Ping An Technology (Shenzhen) Co., Ltd. | Video compressed sensing reconstruction method, system, electronic device, and storage medium |
| CN108629733A (en) * | 2017-03-17 | 2018-10-09 | 北京大学 | Obtain the method and apparatus of high-definition picture |
| CN108629733B (en) * | 2017-03-17 | 2021-12-24 | 北京大学 | Method and device for acquiring high-resolution image |
| CN108876756B (en) * | 2017-05-09 | 2020-08-28 | 普天信息技术有限公司 | Image similarity measurement method and device |
| CN108876756A (en) * | 2017-05-09 | 2018-11-23 | 普天信息技术有限公司 | The measure and device of image similarity |
| CN107087201A (en) * | 2017-05-10 | 2017-08-22 | 西安万像电子科技有限公司 | Image processing method and device |
| US10805634B2 (en) | 2017-05-17 | 2020-10-13 | Samsung Electronics Co., Ltd | Super-resolution processing method for moving image and image processing apparatus therefor |
| US11250546B2 (en) | 2017-05-26 | 2022-02-15 | Hangzhou Hikvision Digital Technology Co., Ltd. | Image distortion correction method and device and electronic device |
| WO2018214671A1 (en) * | 2017-05-26 | 2018-11-29 | 杭州海康威视数字技术股份有限公司 | Image distortion correction method and device and electronic device |
| CN107240076A (en) * | 2017-05-31 | 2017-10-10 | 北京小米移动软件有限公司 | Image processing method and device |
| CN109087242A (en) * | 2017-06-13 | 2018-12-25 | 上海富瀚微电子股份有限公司 | A kind of cfa interpolation processing method and processing device |
| CN109087242B (en) * | 2017-06-13 | 2023-05-09 | 上海富瀚微电子股份有限公司 | CFA interpolation processing method and device |
| CN109429030A (en) * | 2017-08-31 | 2019-03-05 | 爱唯秀股份有限公司 | The method for rebuilding video using super-resolution algorithms |
| WO2019120110A1 (en) * | 2017-12-20 | 2019-06-27 | 华为技术有限公司 | Image reconstruction method and device |
| CN109949332A (en) * | 2017-12-20 | 2019-06-28 | 北京京东尚科信息技术有限公司 | Method and apparatus for handling image |
| US11551333B2 (en) | 2017-12-20 | 2023-01-10 | Huawei Technologies Co., Ltd. | Image reconstruction method and device |
| CN108364258B (en) * | 2018-02-07 | 2022-09-27 | 浙江师范大学 | Method and system for improving image resolution |
| CN108364258A (en) * | 2018-02-07 | 2018-08-03 | 浙江师范大学 | A kind of method and system improving image resolution ratio |
| CN108960084A (en) * | 2018-06-19 | 2018-12-07 | 清华大学深圳研究生院 | Target tracking method, system, readable storage medium storing program for executing and electronic equipment |
| CN110838085A (en) * | 2018-08-15 | 2020-02-25 | Tcl集团股份有限公司 | Super-resolution reconstruction method and device for image and electronic equipment |
| CN110838085B (en) * | 2018-08-15 | 2021-09-24 | Tcl科技集团股份有限公司 | Super-resolution reconstruction method and device for image and electronic equipment |
| US10931853B2 (en) | 2018-10-18 | 2021-02-23 | Sony Corporation | Enhanced color reproduction for upscaling |
| CN112868034A (en) * | 2018-10-18 | 2021-05-28 | 索尼公司 | Enhanced color reproduction for magnification |
| CN112868045A (en) * | 2018-10-18 | 2021-05-28 | 索尼公司 | Frame handling for ML-based magnification |
| CN112868034B (en) * | 2018-10-18 | 2024-01-16 | 索尼公司 | Enhanced color reproduction for magnification |
| WO2020081770A1 (en) * | 2018-10-18 | 2020-04-23 | Sony Corporation | Enhanced color reproduction for upscaling |
| CN109523584A (en) * | 2018-10-26 | 2019-03-26 | 上海联影医疗科技有限公司 | Image processing method, device, multi-mode imaging system, storage medium and equipment |
| CN109528230B (en) * | 2018-11-21 | 2021-08-17 | 山东浪潮科学研究院有限公司 | Method and device for segmenting breast tumor based on multistage transformation network |
| CN109528230A (en) * | 2018-11-21 | 2019-03-29 | 济南浪潮高新科技投资发展有限公司 | A kind of tumor of breast dividing method and device based on multi-stage transformation network |
| CN109859145A (en) * | 2019-02-27 | 2019-06-07 | 长安大学 | It is a kind of that texture method is gone with respect to the image of total variance based on multistage weight |
| CN109859145B (en) * | 2019-02-27 | 2022-12-09 | 西安汇智信息科技有限公司 | Image de-texturing method based on multi-level weight relative total variation |
| CN110555823B (en) * | 2019-04-11 | 2023-04-07 | 江南大学 | Image fusion quality evaluation method based on TVL structure texture decomposition |
| CN110555823A (en) * | 2019-04-11 | 2019-12-10 | 江南大学 | Image fusion quality evaluation method based on TVL structure texture decomposition |
| CN112116526A (en) * | 2019-06-19 | 2020-12-22 | 中国石油化工股份有限公司 | Torch smoke image super-resolution method based on deep convolutional neural network |
| CN112116526B (en) * | 2019-06-19 | 2024-06-11 | 中国石油化工股份有限公司 | Super-resolution method of torch smoke image based on depth convolution neural network |
| US11710223B2 (en) | 2020-02-14 | 2023-07-25 | Pixelworks, Inc. | Methods and systems for image processing with multiple image sources |
| CN113271406B (en) * | 2020-02-14 | 2023-12-08 | 逐点有限公司 | Method and system for image processing with multiple image sources |
| CN113271406A (en) * | 2020-02-14 | 2021-08-17 | 逐点有限公司 | Method and system for image processing with multiple image sources |
| CN112435160A (en) * | 2020-11-06 | 2021-03-02 | 上海航天控制技术研究所 | Depth map super-resolution method based on generative countermeasure network |
| CN116258735A (en) * | 2023-05-16 | 2023-06-13 | 四川省肿瘤医院 | Cervical cancer inter-tissue insertion needle reconstruction system |
| CN116258735B (en) * | 2023-05-16 | 2023-07-21 | 四川省肿瘤医院 | Cervical cancer inter-tissue insertion needle reconstruction system |
| CN116402691A (en) * | 2023-06-05 | 2023-07-07 | 四川轻化工大学 | Image super-resolution method and system based on rapid image feature stitching |
| CN116402691B (en) * | 2023-06-05 | 2023-08-04 | 四川轻化工大学 | Image super-resolution method and system based on rapid image feature stitching |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106204447A (en) | The super resolution ratio reconstruction method with convolutional neural networks is divided based on total variance | |
| CN106127688B (en) | A kind of super-resolution image reconstruction method and its system | |
| CN108734659B (en) | Sub-pixel convolution image super-resolution reconstruction method based on multi-scale label | |
| CN106127684B (en) | Image super-resolution Enhancement Method based on forward-backward recutrnce convolutional neural networks | |
| CN103824272B (en) | The face super-resolution reconstruction method heavily identified based on k nearest neighbor | |
| CN103871041B (en) | The image super-resolution reconstructing method built based on cognitive regularization parameter | |
| CN105046672B (en) | A kind of image super-resolution rebuilding method | |
| CN106934766A (en) | A kind of infrared image super resolution ratio reconstruction method based on rarefaction representation | |
| CN104899830B (en) | A kind of image super-resolution method | |
| CN106600538A (en) | Human face super-resolution algorithm based on regional depth convolution neural network | |
| CN106067161A (en) | A kind of method that image is carried out super-resolution | |
| CN106981080A (en) | Night unmanned vehicle scene depth method of estimation based on infrared image and radar data | |
| CN105957063A (en) | CT image liver segmentation method and system based on multi-scale weighting similarity measure | |
| CN106127689B (en) | Image/video super-resolution method and device | |
| CN104835130A (en) | Multi-exposure image fusion method | |
| CN105744256A (en) | Three-dimensional image quality objective evaluation method based on graph-based visual saliency | |
| CN105741252A (en) | Sparse representation and dictionary learning-based video image layered reconstruction method | |
| CN103366347B (en) | Image super-resolution rebuilding method based on rarefaction representation | |
| CN105335929A (en) | Depth map super-resolution method | |
| CN109711401A (en) | A kind of Method for text detection in natural scene image based on Faster Rcnn | |
| CN104992403B (en) | Hybrid operator image redirection method based on visual similarity measurement | |
| CN105631807A (en) | Single-frame image super resolution reconstruction method based on sparse domain selection | |
| CN106886978A (en) | A kind of super resolution ratio reconstruction method of image | |
| CN107845065A (en) | Super-resolution image reconstruction method and device | |
| CN105550989A (en) | Image super-resolution method based on nonlocal Gaussian process regression |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20161207 Assignee: Boya cloud (Beijing) Technology Co., Ltd. Assignor: Peking University Contract record no.: 2017990000367 Denomination of invention: Super resolution reconstruction method based on total variation difference and convolution nerve network License type: Common License Record date: 20170908 |
|
| EE01 | Entry into force of recordation of patent licensing contract | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20161207 |
|
| WD01 | Invention patent application deemed withdrawn after publication |