CN103985107A - Self-adaptive image super-resolution reconstruction method based on visual perception - Google Patents

Abstract

The invention relates to a self-adaptive image super-resolution reconstruction method based on visual perception. The self-adaptive image super-resolution reconstruction method comprises the steps that firstly, block matching is conducted on a plurality of low-resolution images, and registration information of the images is obtained; secondly, a super-resolution image is reconstructed through a maximum posterior probability estimation method, during iteration solving, the gradient information of the reconstructed image is evaluated through human eye visual characteristics, so that regularizing operators are adjusted in a self-adaptive mode, and self-adaptive image reconstruction is conducted. By the adoption of the self-adaptive image super-resolution reconstruction method based on visual perception, the defect that according to a traditional method, a single regularizing operator is applied to a whole image, so that the reconstructed image is not clear is overcome, and the quality and the definition of reconstructed super-resolution images are effectively improved.

Description

A kind of adapting to image super resolution ratio reconstruction method based on visually-perceptible

Technical field

The present invention relates to a kind of adapting to image super resolution ratio reconstruction method based on visually-perceptible, belong to image processing field.

Background technology

Along with the development of multimedia information, people are more and more higher to the requirement of image resolution ratio.But the spatial resolution that improves image is subject to the restriction of sensor density and the size of imaging system, because making image resolution ratio, the factor in the image acquisition process such as illumination declines in addition, and the sharpness of image and resolution are difficult to meet people's needs.Image super-resolution rebuilding technology is not change under the prerequisite of original imaging system hardware, adopts the software algorithm based on signal processing, utilizes the image of several inferior quality, low resolution to recover a panel height quality, high-resolution image.

Image Super-resolution Reconstruction is an ill indirect problem, introduces Image Priori Knowledge model and can effectively provide constraint to image reconstruction.The proportion of Image Priori Knowledge model in image reconstruction is regulated by regularizing operator, existing method is rebuild image to view picture and is adopted single regularizing operator, do not consider human eye vision apperceive characteristic, the size that can not effectively adjust regularizing operator according to the local message self-adaptation of image, quality and the sharpness of image rebuild in impact.

Summary of the invention

For fear of the deficiencies in the prior art, the present invention proposes a kind of adapting to image super resolution ratio reconstruction method based on visually-perceptible, first several low-resolution images are carried out to piece coupling, obtain the registration information of image, then utilize the method that maximum a posteriori probability is estimated to rebuild high-definition picture, in iterative process, utilize human-eye visual characteristic to assess to the gradient information of rebuilding image, thereby adjust regularizing operator adaptively, carry out adaptive image reconstruction.Described method comprises step:

(1) utilize the sample data of low-resolution image, several low-resolution images and image to be reconstructed are carried out to piece and mate, obtain the registration information of image;

(2) utilize the registration parameter obtaining in step (1), set up the maximum a posteriori probability estimation model of rebuilding image, obtain rebuilding Image estimation wherein n is iterations;

(3) calculate the gradient information of rebuilding image, obtain the gradient variance matrix of each image block;

(4) utilize Human Perception characteristic, self-adaptation is adjusted the size of regularizing operator;

(5) method of estimating according to maximum a posteriori probability, by gradient variance matrix and regularizing operator substitution gradient optimal method, rebuilds high-definition picture

(6) judgement with relation whether meet iterated conditional, if meet output otherwise will substitution step (5) is until meet stopping criterion for iteration.

In step (1), described method for registering images is: the M width low-resolution image of supposing input is G={g ⁽¹⁾, g ⁽²⁾... g ^(M), for low-resolution image, be divided into 8 × 8 image block, adopt block matching algorithm and minimum mean square error criterion to obtain the registration parameter of i width low-resolution image:

R ⁽ⁱ⁾＝min|f-g ⁽ⁱ⁾| (1)

Wherein f is image to be reconstructed.

The method of setting up maximum a posteriori probability estimation model described in step (2) is:

p(f|G)∝p(G|f)p(f) (2)

Wherein the posterior probability of low-resolution image is:

$p\left(G\right|f)\∝\exp \{\underset{i=1}{\overset{M}{\mathrm{\Σ}}}-|f-R^{\left(i\right)}{g}^{\left(i\right)}\left|\right\}---\left(3\right)$

The marginal probability of image to be reconstructed is:

p(f)∝λexp{|f|} (4)

Wherein λ is regularization parameter.

The concrete grammar that calculates the gradient information of rebuilding image in step (3) is: the gradient variance matrix that calculates each image block

$C=\left(\begin{array}{cc}\underset{W}{\mathrm{\Σ}}{I}_r^2(r,c)& \underset{W}{\mathrm{\Σ}}{I}_r(r,c)I_c(r,c)\\ \underset{W}{\mathrm{\Σ}}{I}_r(r,c)I_c(r,c)& \underset{W}{\mathrm{\Σ}}{I}_c^2(r,c)\end{array}\right)---\left(5\right)$

Wherein I _r(r, c) and I _c(r, c) difference presentation video along the gradient of row and column both direction, then decomposes and obtains gradient variance matrix on (r, c) position

$C={\mathrm{VDV}}^T=\left(\begin{array}{cc}{V}_1& V_2\end{array}\right)\left(\begin{array}{cc}{\mathrm{\λ}}_1& 0\\ 0& {\mathrm{\λ}}_2\end{array}\right)\left(\begin{array}{c}{V}_1^T\\ V_2^T\end{array}\right)---\left(6\right)$

Wherein V is by proper vector V ₁and V ₂2 × 2 matrix of composition, λ ₁and λ ₂it is corresponding eigenwert.

In step (4), utilize the phase equalization of the gradient of each image block to adjust the big or small formula of regularizing operator as follows:

$\mathrm{\λ}(r,c)=-\cos (\mathrm{\θ}(r,c)-\stackrel{\‾}{\mathrm{\θ}}(r,c))---\left(7\right)$

Wherein θ (r, c) and be respectively the average of image gradient in (r, c) locational phase place and neighborhood phase place.

In step (5), the method for image reconstruction is specially, will the optimized algorithm of substitution based on gradient rebuild image:

${\hat{f}}^{(n+1)}={\hat{f}}^{\left(n\right)}-\mathrm{\β}\{\mathrm{sign}({\hat{f}}^{\left(n\right)}-G)+\underset{i=-1}{\overset{1}{\mathrm{\Σ}}}\underset{j=-1}{\overset{1}{\mathrm{\Σ}}}\mathrm{\λ}(I-C^{-j}{R}^{-i})\mathrm{sign}({\hat{f}}^{\left(n\right)}-R^i{C}^j{\hat{f}}^{\left(n\right)})---\left(8\right)$

Wherein β is the iteration step length in the optimized algorithm based on gradient, and I is unit matrix, R ⁱrepresent image along i pixel of line direction translation, C ^jrepresent image along j pixel of column direction translation, R ^-iand C ^-jrespectively R ⁱand C ^jtransposed matrix.

In step (6), iterative process end condition is, when with meet in time, stops.

Brief description of the drawings

Fig. 1. the basic flow sheet of the inventive method.

Fig. 2. use the inventive method to complete the example of image super-resolution rebuilding

(a) original image

(b) utilize the image reconstruction result of the full variational method

(c) utilize the image reconstruction result of the bilateral variational method

(d) the image reconstruction result of the inventive method

Embodiment

Below the embodiment of technical scheme of the present invention is described in further detail

(1) as shown in Figure 1, input image to be reconstructed and low-resolution image, utilize the sample data of low-resolution image, several low-resolution images and image to be reconstructed are carried out to piece and mate, obtain the registration information of image;

(2) utilize the registration parameter obtaining in step (1), set up the maximum a posteriori probability estimation model of rebuilding image, obtain rebuilding Image estimation wherein n is iterations;

(3) calculate the gradient information of rebuilding image, obtain the gradient variance matrix of each image block;

(4) utilize Human Perception characteristic, self-adaptation is adjusted the size of regularizing operator;

(5) method of estimating according to maximum a posteriori probability, by gradient variance matrix and regularizing operator substitution gradient optimal method, rebuilds high-definition picture

(6) judgement with relation whether meet iterated conditional, if meet output otherwise will substitution step (5) is until meet stopping criterion for iteration.

In step (1), described method for registering images is: the M width low-resolution image of supposing input is G={g ⁽¹⁾, g ⁽²⁾... g ^(M), for low-resolution image, be divided into 8 × 8 image block, adopt block matching algorithm and minimum mean square error criterion to obtain the registration parameter of i width low-resolution image:

R ⁽ⁱ⁾＝min|f-g ⁽ⁱ⁾| (9)

Wherein f is image to be reconstructed.

The method of setting up maximum a posteriori probability estimation model described in step (2) is:

p(f|G)∝p(G|f)p(f) (10)

Wherein the posterior probability of low-resolution image is:

$p\left(G\right|f)\∝\exp \{\underset{i=1}{\overset{M}{\mathrm{\Σ}}}-|f-R^{\left(i\right)}{g}^{\left(i\right)}\left|\right\}---\left(11\right)$

The marginal probability of image to be reconstructed is:

p(f)∝λexp{|f|} (12)

Wherein λ is regularization parameter.

The concrete grammar that calculates the gradient information of rebuilding image in step (3) is: the gradient variance matrix that calculates each image block

$C=\left(\begin{array}{cc}\underset{W}{\mathrm{\Σ}}{I}_r^2(r,c)& \underset{W}{\mathrm{\Σ}}{I}_r(r,c)I_c(r,c)\\ \underset{W}{\mathrm{\Σ}}{I}_r(r,c)I_c(r,c)& \underset{W}{\mathrm{\Σ}}{I}_c^2(r,c)\end{array}\right)---\left(13\right)$

Wherein I _r(r, c) and I _c(r, c) difference presentation video along the gradient of row and column both direction, then decomposes and obtains gradient variance matrix on (r, c) position

$C={\mathrm{VDV}}^T=\left(\begin{array}{cc}{V}_1& V_2\end{array}\right)\left(\begin{array}{cc}{\mathrm{\λ}}_1& 0\\ 0& {\mathrm{\λ}}_2\end{array}\right)\left(\begin{array}{c}{V}_1^T\\ V_2^T\end{array}\right)---\left(14\right)$

Wherein V is by proper vector V ₁and V ₂2 × 2 matrix of composition, λ ₁and λ ₂it is corresponding eigenwert.

In step (4), utilize the phase equalization of the gradient of each image block to adjust the big or small formula of regularizing operator as follows:

$\mathrm{\λ}(r,c)=-\cos (\mathrm{\θ}(r,c)-\stackrel{\‾}{\mathrm{\θ}}(r,c))---\left(15\right)$

Wherein θ (r, c) and be respectively the average of image gradient in (r, c) locational phase place and neighborhood phase place.

In step (5), the method for image reconstruction is specially, will the optimized algorithm of substitution based on gradient rebuild image:

${\hat{f}}^{(n+1)}={\hat{f}}^{\left(n\right)}-\mathrm{\β}\{\mathrm{sign}({\hat{f}}^{\left(n\right)}-G)+\underset{i=-1}{\overset{1}{\mathrm{\Σ}}}\underset{j=-1}{\overset{1}{\mathrm{\Σ}}}\mathrm{\λ}(I-C^{-j}{R}^{-i})\mathrm{sign}({\hat{f}}^{\left(n\right)}-R^i{C}^j{\hat{f}}^{\left(n\right)})---\left(16\right)$

Wherein β is the iteration step length in the optimized algorithm based on gradient, and I is unit matrix, R ⁱrepresent image along i pixel of line direction translation, C ^jrepresent image along j pixel of column direction translation, R ^-iand C ^-jrespectively R ⁱand C ^jtransposed matrix.

In step (6), iterative process end condition is, when with meet in time, stops.

The reconstruction image that the inventive method is obtained and existing full variation image reconstruction and the comparison of bilateral variation image reconstruction algorithm, reconstructed results as shown in Figure 1.Reconstruction image and original image comparison Y-PSNR PSNR index are respectively to 25.64dB (full variation image reconstruction algorithm), 25.37dB (bilateral variation image reconstruction algorithm) and 26.12dB (method of this patent).The PSNR index of the image that the inventive method obtains is larger, and the quality that shows to rebuild image is high.

The present invention proposes a kind of adapting to image super resolution ratio reconstruction method based on visually-perceptible, first several low-resolution images are carried out to piece coupling, obtain the registration information of image, then utilize the method that maximum a posteriori probability is estimated to rebuild high-definition picture, in iterative process, utilize human-eye visual characteristic to assess to the gradient information of rebuilding image, thereby adjust adaptively regularizing operator, carry out adaptive image reconstruction, having taken classic method adopts single regularizing operator to cause the image blurring deficiency of reconstruction for entire image, effectively improve quality and the sharpness of super-resolution image reconstruction.

The foregoing is only the bright most preferred embodiment of this law, in order to limit the present invention, not all in the spirit and principles in the present invention, any amendment of making, be equal to replacement, improvement etc., all should be included in protection scope of the present invention.

Claims (7)

1. the adapting to image super resolution ratio reconstruction method based on visually-perceptible, is characterized in that said method comprising the steps of:

(1) utilize the sample data of low-resolution image, several low-resolution images and image to be reconstructed are carried out to piece and mate, obtain the registration information of image;

(2) utilize the registration parameter obtaining in step (1), set up the maximum a posteriori probability estimation model of rebuilding image, obtain rebuilding Image estimation wherein n is iterations;

(3) calculate the gradient information of rebuilding image, obtain the gradient variance matrix of each image block;

(4) utilize Human Perception characteristic, self-adaptation is adjusted the size of regularizing operator;

(5) method of estimating according to maximum a posteriori probability, by gradient variance matrix and regularizing operator substitution gradient optimal method, rebuilds high-definition picture

(6) judgement with relation whether meet iterated conditional, if meet output otherwise will substitution step (5) is until meet stopping criterion for iteration.

2. the adapting to image super resolution ratio reconstruction method based on visually-perceptible according to claim 1, is characterized in that in step (1), described method for registering images is: the M width low-resolution image of supposing input is G={g ⁽¹⁾, g ⁽²⁾... g ^(M), for low-resolution image, be divided into 8 × 8 image block, adopt block matching algorithm and minimum mean square error criterion to obtain the registration parameter of i width low-resolution image:

R ⁽ⁱ⁾＝min|f-g ⁽ⁱ⁾| (1)

Wherein f is image to be reconstructed.

3. the adapting to image super resolution ratio reconstruction method based on visually-perceptible according to claim 1, is characterized in that, the method for setting up maximum a posteriori probability estimation model described in step (2) is:

p(f|G)∝p(G|f)p(f) (2)

Wherein the posterior probability of low-resolution image is:

$p\left(G\right|f)\∝\exp \{\underset{i=1}{\overset{M}{\mathrm{\Σ}}}-|f-R^{\left(i\right)}{g}^{\left(i\right)}\left|\right\}---\left(3\right)$

The marginal probability of image to be reconstructed is:

p(f)∝λexp{|f|} (4)

Wherein λ is regularization parameter.

4. the adapting to image super resolution ratio reconstruction method based on visually-perceptible according to claim 1, is characterized in that the concrete grammar that calculates the gradient information of rebuilding image in step (3) is: the gradient variance matrix that calculates each image block

$C=\left(\begin{array}{cc}\underset{W}{\mathrm{\Σ}}{I}_r^2(r,c)& \underset{W}{\mathrm{\Σ}}{I}_r(r,c)I_c(r,c)\\ \underset{W}{\mathrm{\Σ}}{I}_r(r,c)I_c(r,c)& \underset{W}{\mathrm{\Σ}}{I}_c^2(r,c)\end{array}\right)---\left(5\right)$

Wherein I _r(r, c) and I _c(r, c) difference presentation video along the gradient of row and column both direction, then decomposes and obtains gradient variance matrix on (r, c) position

$C={\mathrm{VDV}}^T=\left(\begin{array}{cc}{V}_1& V_2\end{array}\right)\left(\begin{array}{cc}{\mathrm{\λ}}_1& 0\\ 0& {\mathrm{\λ}}_2\end{array}\right)\left(\begin{array}{c}{V}_1^T\\ V_2^T\end{array}\right)---\left(6\right)$

Wherein V is by proper vector V ₁and V ₂2 × 2 matrix of composition, λ ₁and λ ₂it is corresponding eigenwert.

5. the adapting to image super resolution ratio reconstruction method based on visually-perceptible according to claim 1, is characterized in that in step (4), utilizes the phase equalization of the gradient of each image block to adjust the big or small formula of regularizing operator as follows:

$\mathrm{\λ}(r,c)=-\cos (\mathrm{\θ}(r,c)-\stackrel{\‾}{\mathrm{\θ}}(r,c))---\left(7\right)$

Wherein θ (r, c) and be respectively the average of image gradient in (r, c) locational phase place and neighborhood phase place.

6. the adapting to image super resolution ratio reconstruction method based on visually-perceptible according to claim 1, is characterized in that the method for image reconstruction in step (5) is specially, will the optimized algorithm of substitution based on gradient rebuild image:

${\hat{f}}^{(n+1)}={\hat{f}}^{\left(n\right)}-\mathrm{\β}\{\mathrm{sign}({\hat{f}}^{\left(n\right)}-G)+\underset{i=-1}{\overset{1}{\mathrm{\Σ}}}\underset{j=-1}{\overset{1}{\mathrm{\Σ}}}\mathrm{\λ}(I-C^{-j}{R}^{-i})\mathrm{sign}({\hat{f}}^{\left(n\right)}-R^i{C}^j{\hat{f}}^{\left(n\right)})---\left(8\right)$

Wherein β is the iteration step length in the optimized algorithm based on gradient, and I is unit matrix, R ⁱrepresent image along i pixel of line direction translation, C ^jrepresent image along j pixel of column direction translation, R ^-iand C ^-jrespectively R ⁱand C ^jtransposed matrix.

7. the adapting to image super resolution ratio reconstruction method based on visually-perceptible according to claim 1, is characterized in that in step (6), iterative process end condition is, when with meet in time, stops.