Background technology
Image is one of mankind's main carriers of obtaining outside objective world information.Along with human society enters the information age, computer technology, modern communications technology and the information processing technology have obtained development fast, and consumption electronic product has spread all over each corner of human lives.Along with the appearance of digital picture and application widely, people are also more and more higher to the requirement of picture quality, in the situation that raising image capture device hardware capabilities has approached the limit, adopting software engineering to improve original image resolution ratio becomes one of hot issue of digital image processing field gradually.
Image interpolation is traditional one of method improving image resolution ratio that is used for.The interpolation method of traditional digital picture has a variety of, wherein fastest is Nearest Neighbor (nearest-neighbor) method, the method by copy be inserted into a little adjacent nearest pixel value as the pixel value of insertion point, this method speed is fast, realize simply, but in interpolation image, often there will be zigzag flaw; Bilinear (bilinearity) interpolation method is by getting each mean value that is inserted into four immediate pixel numerical value a little as insertion point pixel value, this method has been eliminated zigzag phenomenon, but computing time is slightly long, and easily there is blurring effect in the boundary in image.Bicubic (bicubic) convolution method by interpolation point (i+u, j+v) around 16 neighbor pixel points take into account, by following interpolation formula, can obtain the pixel value f (i+u, j+v) of this interpolation point:
f(i+u,j+v)=A*B*C
A=[s(u+1)s(u+0)s(u-1)s(u-2)]
C=[s(v+1)s(v+0)s(v-1)s(v-2)]
T
Compare arest neighbors and bilinear interpolation, bicubic convolution method has greatly improved to image interpolation effect tool, and interpolation place numerical value and derivative are all continuous, and details performance is clearer, has overcome to a certain extent the image flaw of above two kinds of methods.But owing to relating to a cube computing in calculating formula, and have matrix convolution, so calculated amount is compared, and the above two are larger.
Comparatively level and smooth region in image, above classic method as Bilinear and Bicubic can process fine.But for the image-region at boundary, above traditional interpolation method has obvious flaw, as fuzzy, sawtooth etc.And human visual system is very responsive for boundary characteristic, therefore make this flaw cause larger harmful effect to image visual effect.Therefore, a main target that realizes good image interpolation is how to keep better image in the feature of boundary when interpolation, avoids producing the various flaws of traditional interpolation method.In order to realize such target, need to carry out comparatively reasonably modeling to picture signal.The difficult point of picture signal modeling is the unstable state characteristic of himself.Meanwhile, natural image often has geometrical rule, and image changes slowly on boundary direction, in the direction perpendicular to border, changes rapidly.This character has reflected the segmentation statistic steady state characteristic of picture signal.According to such a characteristic, the people such as Li have proposed a kind of adaptive interpolation method that utilizes covariance information to carry out Latent Including boundary information.High resolving power covariance information estimates by low resolution covariance, low resolution covariance information is that in hypothesis window, statistic keeps stable, samples and estimate in a local window.This method can be processed the border condition compared with large scale in image well, but cannot process well some compared with the border of small scale and textural characteristics, simultaneously because its method has only been utilized the neighbor pixel dot information in diagonal, the interpolation sharpness on border is compared original image and is also had a certain distance, and the narrow and small edge effect image visual effect that has some falsenesses in interpolation image, seems to lack nature and the sense of reality.Zhang and Wu introduce into more pixel space correlationship, not only utilized the correlationship of diagonal, and considered the correlationship in horizontal and vertical direction, by as if statistics amount in a local window, keep stable, this correlationship restriction is forced on pixels all in window, by piece optimization, estimate finally to obtain the value of one group of pixel to be inserted.This method takes full advantage of the correlationship between high low-resolution image, for the structural information of natural image, has good adaptability, therefore larger improvement the people's such as Li algorithm.
Above two kinds of algorithms are all that hypothesis image statistics in selected window keeps stable, when image boundary characteristic dimension is less than window size, this hypothesis is also false, therefore the estimation for statistic information in window is inaccurate, cause the error of system model, thereby cause interpolation image to occur flaw at boundary.
Summary of the invention
In order to solve prior art interpolation image, occur the problem of flaw at boundary, the embodiment of the present invention provides a kind of image interpolation method that improves image resolution ratio, comprising:
Similarity probability between the partial structurtes of default reference image vegetarian refreshments in the partial structurtes of each pixel and selected window in the selected window of computed image;
According to the pixel value of known low-resolution pixel point in selected window, the first parameter and the second parameter are estimated, the first parameter is for portraying in window the model parameter of neighbor pixel point regression relation on each pixel and its diagonal, the second parameter be for portray interior each pixel of window perpendicular with horizontal line direction on the model parameter of neighbor pixel point regression relation;
Utilize the estimated result of the pixel value of low-resolution pixel point, the estimated result of the first parameter and the second parameter, set up the pixel value of interpolation full-resolution picture vegetarian refreshments in selected window and the correlationship of matching error;
According to similarity probability and matching error, set up optimization aim function, optimization aim function is carried out to the pixel value that optimum estimate obtains full-resolution picture vegetarian refreshments;
The pixel value of the full-resolution picture vegetarian refreshments obtaining by optimum estimate, the full-resolution picture vegetarian refreshments that similarity probability is greater than to setting threshold inserts selected window.
The embodiment of the present invention also provides a kind of image interpolation device that improves image resolution ratio simultaneously, comprising:
Probability generation module, for the similarity probability between the partial structurtes of the partial structurtes of each pixel in the selected window of computed image and the default reference image vegetarian refreshments of selected window;
The first estimation module, for the first parameter and the second parameter being estimated according to the pixel value of known low-resolution pixel point in selected window, the first parameter is for portraying in window the model parameter of neighbor pixel point regression relation on each pixel and its diagonal, the second parameter be for portray interior each pixel of window perpendicular with horizontal line direction on the model parameter of neighbor pixel point regression relation;
Correlationship is set up module, for utilizing the estimated result of the pixel value of low-resolution pixel point, the estimated result of the first parameter and the second parameter, sets up the pixel value of interpolation full-resolution picture vegetarian refreshments in selected window and the correlationship of matching error;
The second estimation module, for setting up optimization aim function according to similarity probability and matching error, carries out to optimization aim function the pixel value that optimum estimate obtains full-resolution picture vegetarian refreshments;
Insert module, for the pixel value of the full-resolution picture vegetarian refreshments that obtains by optimum estimate, the full-resolution picture vegetarian refreshments that similarity probability is greater than to setting threshold inserts selected window.
By specific embodiments provided by the invention, can be found out, the weight when having introduced similarity probability as high-resolution pixel point estimation, and then evaluated error is minimized, reduced the flaw that boundary occurs.
Embodiment
In order to solve interpolation image in prior art, at boundary, there is the problem of flaw, the embodiment of the present invention provides a kind of image interpolation method that improves image resolution ratio, in the scheme of the present embodiment, without loss of generality, regarding in high-definition picture the pixel in low-resolution image as down-sampling that the factor through rule is 2 obtains, therefore these pixels can be regarded the part in original high resolution image as, and Fig. 1 has provided interpolation schematic diagram twice.From existing low-resolution pixel, select 10 interpolation goes out first's full-resolution picture vegetarian refreshments 11 and second portion full-resolution picture vegetarian refreshments 12 to the task of interpolation exactly so.Whole Interpolation Process is divided into two steps.The first step is that interpolation obtains first's full-resolution picture vegetarian refreshments 11.Second step is that first's full-resolution picture vegetarian refreshments 11 of obtaining according to first step interpolation and existing low-resolution pixel are selected 10 and come interpolation to obtain second portion full-resolution picture vegetarian refreshments 12.The algorithm that two steps are processed is the same, only in direction, has the rotation of 45 degree and the scaling on yardstick.Therefore, the core objective of interpolation algorithm can be regarded as from existing low-resolution pixel and selects the process that 10 interpolation go out first's full-resolution picture vegetarian refreshments 11.
The image interpolation method of raising image resolution ratio provided by the invention, refers to Fig. 2, comprises the steps:
Step 101, the similarity probability P between the partial structurtes of center pixel to be inserted in the partial structurtes of each pixel and selected window in the selected window of computed image
land P
h.
For convenience of explanation, the present embodiment is using center pixel to be inserted default reference image vegetarian refreshments in selected window, but this is not limited, reference image vegetarian refreshments can be also any one pixel that comprises first pixel and last pixel in the selected window of image.In local window centered by the pixel to be inserted of the present embodiment Shi Yi center, the pixel value of high-resolution pixel in this window is estimated.The local window with different similarity probability arranging is like this shown by Fig. 3, comprises first 21, second portion 22 and third part 23.In such a local window, not all pixel all has the similar topography's feature of concentricity pixel to be inserted 20, be partial structurtes dissimilar, there are some may also there is larger difference, the pixel of first 21 and local window center pixel local structure similarity to be inserted wherein, the pixel of second portion 22 and third part 23 and local window center pixel 20 partial structurtes to be inserted are dissimilar, therefore need to have a kind of index to react the similarity difference of partial structurtes between each pixel.This similarity difference can be measured by the similarity probability between them.
In the partial structurtes of j pixel and selected window, the computing formula as the similarity probability between the partial structurtes of d pixel of reference image vegetarian refreshments is in window
Similarity probability between the partial structurtes that are the present embodiment Zhong center pixel to be inserted as d pixel of reference image vegetarian refreshments in the partial structurtes that wherein p (d, j) is j pixel in selected window and selected window,
be the vector of the pixel value formation of order of the neighbours in four diagonals of center pixel to be inserted in selected window, d is positive integer, d≤m+n, n is the number of selected window middle high-resolution pixel, m is the number of low-resolution pixel point, and ε is one and avoids the positive number except zero overflow error
it is the vector of the pixel value formation of ordering of the neighbours in four diagonals of j pixel in selected window, j is positive integer, j≤m+n, h is used for the distribution of shapes parameter of control characteristic function, and Fig. 4 is two right schematic diagram of pixel with different similarity probability.
According to above formula
can by the similarity probability calculation between the concentricity pixel to be inserted of all pixels in window out, form two probability vector P
land P
h, wherein
Specific as follows, utilize the low resolution neighbours' point in four diagonals of low-resolution pixel point, similarity probability in the partial structurtes that in the selected window of calculating, each known low-resolution pixel is selected and selected window between the partial structurtes of default reference image vegetarian refreshments, and form low-resolution pixel point similarity probability vector P
l, wherein
represent the similarity probability between the partial structurtes of reference image vegetarian refreshments default in the 1st partial structurtes that low-resolution pixel is selected and selected window,
represent the similarity probability i≤m between the partial structurtes of reference image vegetarian refreshments default in partial structurtes that i low-resolution pixel selected and selected window,
represent the similarity probability between the partial structurtes of reference image vegetarian refreshments default in partial structurtes that m low-resolution pixel selected and selected window.Utilize the low resolution neighbours' point in four diagonals of full-resolution picture vegetarian refreshments, similarity probability between the partial structurtes of default reference image vegetarian refreshments in the partial structurtes of each full-resolution picture vegetarian refreshments and selected window in the selected window of calculating, and form full-resolution picture vegetarian refreshments probability vector P
h, wherein
represent the similarity probability between the partial structurtes of reference image vegetarian refreshments default in the partial structurtes of the 1st full-resolution picture vegetarian refreshments and selected window,
represent the similarity probability k≤n between the partial structurtes of reference image vegetarian refreshments default in the partial structurtes of k full-resolution picture vegetarian refreshments and selected window,
represent the similarity probability between the partial structurtes of reference image vegetarian refreshments default in the partial structurtes of n full-resolution picture vegetarian refreshments and selected window.When the similarity probability calculating between full-resolution picture vegetarian refreshments and center pixel to be inserted, can directly utilize low resolution neighbours in 4 diagonals of each full-resolution picture vegetarian refreshments to put pixel value and form partial structurtes vector
utilize low resolution neighbours in 4 diagonals of each full-resolution picture vegetarian refreshments to put pixel value and form partial structurtes vector as shown in Figure 5 a, then directly utilize formula
calculate.When the similarity probability calculating between the pixel to be inserted of low-resolution pixel Dian He center, due to low-resolution pixel, selecting that neighbours in diagonal select is unknown full-resolution picture vegetarian refreshments, can be according to the structural similarity hypothesis between high low resolution, utilize low resolution neighbours in its diagonal to put pixel value and form
utilize low resolution neighbours in 4 diagonals of each low-resolution pixel point to put pixel value and form partial structurtes vector as shown in Figure 5 b.
Step 102, according to the pixel value of known low-resolution pixel point in selected window, parameter a and parameter b are estimated, parameter a is for portraying in window the model parameter of neighbor pixel point regression relation on each pixel and its diagonal, parameter b be for portray interior each pixel of window perpendicular with horizontal line direction on the model parameter of neighbor pixel point regression relation.
In the present embodiment, utilize known low-resolution pixel to select to estimate unknown full-resolution picture vegetarian refreshments, need to carry out modeling to the relation between them.Adopt two group model parameter a, b to portray respectively this space correlation relation.With
represent the regression relation in diagonal, use
represent the regression relation in horizontal and vertical direction, with t ∈, { 1,2,3,4} represents t neighbours in respective direction.For example
with
represent respectively pixel x
iand y
it neighbours in diagonal;
with
represent respectively pixel x
iand y
it neighbours' point in the horizontal and vertical directions.By supposing that the parameter between high low-resolution image is consistent, can to parameter a and b, estimate according to the pixel value of low-resolution image pixel in local window.Between high low resolution pixel, in horizontal and vertical direction, have consistent model parameter as shown in Figure 6 a, parameter b can directly be weighted least-squares estimation and try to achieve its optimal value.Between high low resolution pixel, in diagonal, there is consistent model parameter as shown in Figure 6 b, according to the conforming hypothesis of model parameter between high low resolution, yardstick is put and is twice, adopt
to parameter, a estimates.
When parameter a, parameter b are estimated, directly pass through
to parameter, a is weighted least-squares estimation, wherein
represent the first parameter optimal value estimated result, a
trepresent a vectorial composition value with the parameter a of vector representation, t ∈ { 1,2,3,4}, f (x
i) represent i low-resolution pixel point x
ipixel value,
represent pixel x
it low resolution neighbours point in diagonal
pixel value.Directly pass through
the second parameter is weighted to least square method and estimates, wherein
represent the second parameter optimal value estimated result, b
trepresent a vectorial composition value by the second parameter of vector representation,
represent low-resolution pixel x
it neighbours' point in vertical and horizontal line direction
pixel value.
Step 103, utilizes the estimated result of the pixel value of low-resolution pixel point, the estimated result of parameter a and parameter b, sets up the pixel value of interpolation full-resolution picture vegetarian refreshments in selected window and the correlationship of matching error.
Pass through parameter
carry out the correlationship in the diagonal of picture engraving part, can obtain
Pass through parameter
portray the correlationship in the horizontal and vertical direction of topography, can obtain
and
In integrating step 102, describe, adopt two group model parameter a, b to portray respectively this space correlation relation.With
represent the regression relation in diagonal, use
represent the regression relation in horizontal and vertical direction, with t ∈, { 1,2,3,4} represents t neighbours in respective direction, and between interpolation neighbours' point, the configuration schematic diagram of space correlation relation as shown in Figure 7.
The correlationship of carrying out picture engraving local space by use a model parameter a and b, the matching error of above-mentioned expression can be expressed as following form:
expression is passed through
portray full-resolution picture vegetarian refreshments y
k 'with neighbor pixel point in its diagonal
matching error during regression relation, Q represents to have neighbor pixel point in selected window inner opposite angle line direction
full-resolution picture vegetarian refreshments y
k 'number,
expression is passed through
portray low-resolution pixel point x
i 'with neighbor pixel point in its diagonal
matching error during regression relation, P represents to have neighbor pixel point in selected window inner opposite angle line direction
low-resolution pixel point x
i 'number,
expression is passed through
portray full-resolution picture vegetarian refreshments y
k "with neighbor pixel point in its diagonal
matching error during regression relation, K represents to have neighbor pixel point in vertical in selected window and horizontal line direction
full-resolution picture vegetarian refreshments y
k "number,
expression is passed through
portray low-resolution pixel point x
i "with neighbor pixel point in its diagonal
matching error during regression relation, S represents to have neighbor pixel point in vertical in selected window and horizontal line direction
low-resolution pixel point x
i "number.
Step 104, according to similarity probability P
land P
hset up optimization aim function F with matching error, optimization aim function F is carried out to the best estimate that optimum estimate obtains the full-resolution picture vegetarian refreshments in a group window.
Set up optimization aim function
meet following constraint condition simultaneously
wherein F is optimization aim function, and f (y) represents the pixel value of full-resolution picture vegetarian refreshments, and w represents selected window.By solving this Estimation Optimization problem, finally can obtain the best estimate of the full-resolution picture vegetarian refreshments in a group window.
The best estimate of full-resolution picture vegetarian refreshments carries out to optimization aim function the pixel value that optimum estimate obtains full-resolution picture vegetarian refreshments.
Step 105, the pixel value of the full-resolution picture vegetarian refreshments obtaining by optimum estimate, the best estimate of output center pixel to be inserted.
Wherein, center pixel to be inserted is the main interpolation point of paying close attention in the present embodiment, therefore only exports the numerical value of this point.In order to reduce the time complexity of total algorithm, can consider the value of the higher point of those and center pixel similarity to be inserted probability also to export as its final pixel value, be about to the full-resolution picture vegetarian refreshments that similarity probability is greater than setting threshold and insert selected window.
By local window being carried out on image to the slip in horizontal direction and vertical direction, we can, in the hope of the optimal estimation value of all pixels to be inserted, finally complete the process of image interpolation.
The second embodiment provided by the invention is a kind of image interpolation device that improves image resolution ratio, and its structure as shown in Figure 8, comprising:
Probability generation module 201, for the similarity probability between the partial structurtes of the partial structurtes of each pixel in the selected window of computed image and the default reference image vegetarian refreshments of selected window;
The first estimation module 202, for the first parameter and the second parameter being estimated according to the pixel value of known low-resolution pixel point in selected window, the first parameter is for portraying in window the model parameter of neighbor pixel point regression relation on each pixel and its diagonal, the second parameter be for portray interior each pixel of window perpendicular with horizontal line direction on the model parameter of neighbor pixel point regression relation;
Correlationship is set up module 203, for utilizing the estimated result of the pixel value of low-resolution pixel point, the estimated result of the first parameter and the second parameter, sets up the pixel value of interpolation full-resolution picture vegetarian refreshments in selected window and the correlationship of matching error;
The second estimation module 204, for setting up optimization aim function according to similarity probability and matching error, carries out to optimization aim function the pixel value that optimum estimate obtains full-resolution picture vegetarian refreshments;
Insert module 205, for the pixel value of the full-resolution picture vegetarian refreshments that obtains by optimum estimate, the full-resolution picture vegetarian refreshments that similarity probability is greater than to setting threshold inserts selected window.
Further,
probability generation module 201, also for passing through
carry out similarity probability calculation, in the partial structurtes that wherein p (d, j) is j pixel in selected window and selected window as the similarity probability between the partial structurtes of d pixel of reference image vegetarian refreshments,
be the vector of the pixel value formation of ordering of the neighbours in four diagonals of d pixel in selected window, d is positive integer, d≤m+n, n is the number of selected window middle high-resolution pixel, m is the number of low-resolution pixel point, and ε is one and avoids the positive number except zero overflow error
be the vector of the pixel value formation of ordering of the neighbours in four diagonals of j pixel in selected window, j is positive integer, j≤m+n, and h is used for the distribution of shapes parameter of control characteristic function.
Further,
probability generation module 201, also for utilizing the low resolution neighbours' point in four diagonals of low-resolution pixel point, similarity probability in the partial structurtes that in the selected window of calculating, each known low-resolution pixel is selected and selected window between the partial structurtes of default reference image vegetarian refreshments, and form low-resolution pixel point similarity probability vector P
l, wherein
represent the similarity probability between the partial structurtes of reference image vegetarian refreshments default in the 1st partial structurtes that low-resolution pixel is selected and selected window,
represent the similarity probability i≤m between the partial structurtes of reference image vegetarian refreshments default in partial structurtes that i low-resolution pixel selected and selected window,
represent the similarity probability between the partial structurtes of reference image vegetarian refreshments default in partial structurtes that m low-resolution pixel selected and selected window; And
Utilize the low resolution neighbours' point in four diagonals of full-resolution picture vegetarian refreshments, similarity probability between the partial structurtes of default reference image vegetarian refreshments in the partial structurtes of each full-resolution picture vegetarian refreshments and selected window in the selected window of calculating, and form full-resolution picture vegetarian refreshments probability vector P
h, wherein
represent the similarity probability between the partial structurtes of reference image vegetarian refreshments default in the partial structurtes of the 1st full-resolution picture vegetarian refreshments and selected window,
represent the similarity probability k≤n between the partial structurtes of reference image vegetarian refreshments default in the partial structurtes of k full-resolution picture vegetarian refreshments and selected window,
represent the similarity probability between the partial structurtes of reference image vegetarian refreshments default in the partial structurtes of n full-resolution picture vegetarian refreshments and selected window.
Further, the
first estimation module 202, also for directly passing through
the first parameter is weighted to least-squares estimation, wherein
represent the first parameter optimal value estimated result, a
trepresent a vectorial composition value by the first parameter of vector representation, t ∈ { 1,2,3,4}, f (x
i) represent i low-resolution pixel point x
ipixel value,
represent pixel x
it low resolution neighbours point in diagonal
pixel value;
Directly pass through
the second parameter is weighted to least square method and estimates, wherein
represent the second parameter optimal value estimated result, b
trepresent a vectorial composition value by the second parameter of vector representation,
represent low-resolution pixel x
it neighbours' point in vertical and horizontal line direction
pixel value.
Further, the
second estimation module 204, also for setting up optimization aim function
meet following constraint condition simultaneously
wherein F is optimization aim function, and f (y) represents the pixel value of full-resolution picture vegetarian refreshments, and w represents selected window,
expression is passed through
portray full-resolution picture vegetarian refreshments y
k 'with neighbor pixel point in its diagonal
matching error during regression relation, Q represents to have neighbor pixel point in selected window inner opposite angle line direction
full-resolution picture vegetarian refreshments y
k 'number,
expression is passed through
portray low-resolution pixel point x
i 'with neighbor pixel point in its diagonal
matching error during regression relation, P represents to have neighbor pixel point in selected window inner opposite angle line direction
low-resolution pixel point x
i 'number,
expression is passed through
portray full-resolution picture vegetarian refreshments y
k "with neighbor pixel point in its diagonal
matching error during regression relation, K represents to have neighbor pixel point in vertical in selected window and horizontal line direction
full-resolution picture vegetarian refreshments y
k "number,
expression is passed through
portray low-resolution pixel point x
i "with neighbor pixel point in its diagonal
matching error during regression relation, S represents to have neighbor pixel point in vertical in selected window and horizontal line direction
low-resolution pixel point x
i "number.
Further, insert module 205, if be also full-resolution picture vegetarian refreshments for reference image vegetarian refreshments, inserts selected window using the full-resolution picture vegetarian refreshments as reference image vegetarian refreshments.
Obviously, those skilled in the art can carry out various changes and modification and not depart from the spirit and scope of the present invention the present invention.Like this, if within of the present invention these are revised and modification belongs to the scope of the claims in the present invention and equivalent technologies thereof, the present invention is also intended to comprise these changes and modification interior.