CN105069825A - Image super resolution reconstruction method based on deep belief network - Google Patents

Abstract

The invention discloses an image super resolution reconstruction method based on a deep belief network, relating to the image processing. The image super resolution reconstruction method based on the deep belief network comprises steps of obtaining a low resolution image, performing interpolation amplification on the low resolution image to the needed size, using a repeated partitioning sampling method to obtain a low resolution brightness image block, inputting the low resolution image block, using the deep belief network which is trained in advance to predict the high resolution image block, performing neighborhood regularization optimization solution on an obtained fitting result, combining all high resolution brightness image blocks to obtain a high resolution and brightness image, combining the high resolution and brightness image with the values of the other two channels which are obtained in advance, and converting high resolution and brightness image to the image expressed by color RGB to obtain a predicted high resolution image. The invention realizes the super-resolution reconstruction of the single-frame image, improves the peak signal-to-noise ratio, obtains a clear rim and a rich texture of a constructed image, and can be used for the video safety monitoring, medical digital imaging and spaceflight detection.

Description

Based on the image super-resolution rebuilding method of degree of depth confidence network

Technical field

The present invention relates to image procossing, especially relate to the image super-resolution rebuilding method based on degree of depth confidence network that can be used for Video security monitoring, medical digital image, space flight detection etc.

Background technology

Image super-resolution rebuilding, because its application prospect and actual application value, is paid close attention to widely in recent years, and has been emerged a large amount of outstanding algorithms.These algorithms roughly can be divided into three major types: the image super-resolution rebuilding algorithm based on interpolation, the Super-Resolution of Images Based based on reconstruct and the image super-resolution rebuilding algorithm based on study.Compare other two classes algorithms, the attention rate that the image super-resolution rebuilding algorithm based on study is subject to is higher.It is by a large amount of learning training, adds suitable prior-constrained, makes it compared to the method for traditional interpolation and reconstruct, has more outstanding performance.

In the super-resolution reconstruction method based on study, the Super-resolution Reconstruction based on rarefaction representation is the important oversubscription method for reconstructing of a class, is paid close attention to widely, and is proved to be the oversubscription method for reconstructing that a class gets a good chance of.The method that an other class obtains paying close attention to is the method based on returning, and oversubscription problem is converted into regression problem by these class methods, design regression optimization model.These class methods are also proved to be a kind of method that effective oversubscription is rebuild.But these two large class methods are all carry out modelling by top-down mode, namely need to provide model of fit.Currently solve the less discussion of mode to bottom-up.Degree of depth study is a kind of natural bottom-up network architecture, and in recent years, degree of depth study all achieves breakthrough good effect in speech recognition, text identification and Images Classification.

Chinese patent CN104778659A disclose a kind of based on the degree of depth study single-frame image super-resolution reconstruction method, comprise the following steps: 1, first by training two autocoders for obtaining the feature of low resolution and corresponding high-definition picture block; 2, based on the feature obtaining high resolving power and low-resolution image block, retraining monolayer neuronal networking learns the Nonlinear Mapping relation of two features; 3, based on two autocoders and monolayer neural networks, build the degree of depth network of three layers, using low-resolution image block as input, high-definition picture block, as output, finely tunes the parameter of three layer depth networks; The three layer depth networks that step 4, foundation obtain do single-frame images super-resolution rebuilding, with the gray-scale value of low-resolution image block for input, obtain exporting the gray-scale value being corresponding high-definition picture block.

Summary of the invention

The object of the present invention is to provide the Super-resolution Reconstruction that can realize single-frame images, improve the Y-PSNR of image, obtain the image super-resolution rebuilding method based on degree of depth confidence network rebuild clear picture edge and enrich texture.

The present invention includes following steps:

(1) the image Y of low resolution is inputted;

(2) the image Y of low resolution is first carried out bi-cubic interpolation amplification, be amplified to required size;

(3) by image from red, green, the RGB color space conversion that blue three Color Channels represent to brightness and blue red-color concentration side-play amount (YCbCr) color space, and only carries out follow-up process to the brightness value of image i.e. brightness (Y) passage;

(4) with moving window in the enterprising line slip of image, step-length is 1 get image block, obtains image block (patch) Y of low resolution _i, i represents the sequence number of image block;

(5) by the training image blocks of low resolution, and the high-definition picture block of correspondence, network parameter { W is obtained by training 3 limited Boltzmann machines (RestrictedBoltzmamnMachines, RBM) _i, { c _i, wherein, { W _irepresent the weight vector that the limit in network connects, { c _irepresent being biased next time to last layer in network, with training { the W obtained _i, { c _ithe parameter of initialization degree of depth confidence network (DeepBeliefNetwork, DBN);

(6) train DBN network (TrainedDBN), 3 RBM are gathered into folds, obtains DBN, utilize training image blocks obtained above, finely tune parameter { W with the back propagation of stochastic gradient descent method and error _i, { c _i, until e<tol or iterations t>T, e represents error, tol and T is constant threshold given in algorithm;

(7) by image block (patch block) Y of the low resolution of image to be tested _ias input, the DBN utilizing step (6) to train predicts and obtains corresponding high-resolution patch block X _i;

(8) the high-resolution patch block X will obtained _i, the pixel assignment of getting in the middle of image block is the pixel value of full resolution pricture correspondence position coordinate, to the patch block Y of all low resolution _iprocess, obtain required high-resolution image brightness picture X _{dBN_SR};

(9) add non local similarity constraint, local similarity constraint, degree of depth confidence network matching constraint, set up the super-resolution Optimized model based on neighborhood relationships regularization and fit correlation regularization;

(10) Optimized model in solution procedure (9) is carried out with the graceful iterative algorithm of Donald Bragg be separated;

(11) value (CbCr) of the luminance channel obtained (Y) Super-resolution Reconstruction image with other two Color Channels is combined, then convert the image of RGB to, obtain predicted high-definition picture X.

In step (5), the method for the Boltzmann machine that described training 3 is limited can be:

(5.1) energy function of known RBM is E (υ, h; θ)=-h ^tw υ-b ^th-c ^tυ, θ=and W, b, c}, joint probability is defined as, wherein Z=Σ _vΣ _hexp (-E (υ, h; θ)), v is RBM input, and h is that RBM hidden layer exports, and W represents that the weight vector that input layer limit in a network connects, c represent input layer being biased last layer in network, and b represents last layer being biased lower one deck in network; Setting iteration total degree T, the parameter θ that random initializtion is to be trained ₀=(W ₀, b ₀, c ₀) and its assignment to θ _t, t represents current iteration number of times;

(5.2) by proper vector to be entered/image block assignment to v ₀, and utilizing P (v|h), P (h|v) iterates and obtains h n time ₀, v _nand h _n, v ₀represent each component of the 0th iteration input vector, h ₀represent each component of the 0th iteration hidden layer, conditional probability is calculated as follows:

$P\left(h\right|v)=\underset{i}{\&Pi;}P\left(h_i\right|v),$

P(h _i＝1|v,θ)＝σ(c _i+w _iv)，

P(v|h)＝Π _jP(v _i|h)，

P(v _j＝1|h,θ)＝σ(b _j+w' _jh)，

Wherein σ () is activation function;

(5.3) utilize the joint probability distribution Grad that stochastic gradient descent method and error back propagation method obtain through n gibbs sampler, and upgrade θ _tthree parameter W of the inside _t, b _t, c _t, obtain θ _t+1;

(5.4) if iterations t=T, or difference value reaches necessarily little degree, then EOP (end of program); Otherwise by θ _t+1assignment is to θ _t, and return (5.2) step.

In step (6), the method for described training DBN network can be:

(6.1) stacked by 3 RBM, namely first RBM is with low-resolution image block for input, and the input that the output that computing obtains obtains as second RBM, the output of second RBM is as the input of the 3rd RBM, and the output finally obtained is X _{dBN_SR};

(6.2) stochastic gradient descent method and error back propagation method training network is utilized, fine setting θ _tthree parameter W of the inside _t, b _t, c _t, obtain θ _t+1;

(6.3) if iterations t=T, or difference value reaches necessarily little degree, then EOP (end of program); Otherwise by θ _t+1assignment is to θ _t, and return (6.1).

In step (9), described non-local similarity constraint is: in the neighborhood that certain pixel is larger, utilize similar pixel, retrain current point, namely represent current point by similitude weighted mean, computing formula is:

R _NL＝||(I-W _NL)X|| ₁

Wherein X is the vector form of input picture; I is unit matrix; W _nLfor weight matrix, if represent, block i is more similar to block j, then weighted value is larger, and the value of each element is provided by formula below:

$W_{NL}(i,j)=\left\{\begin{array}{cc}\frac{w(i,j)}{\underset{j\&Element;S_i}{\&Sigma;}w(i,j)},& j\&Element;S_i\\ 0,& j\&NotElement;S_i\end{array}\right.$

Wherein, x _irepresent the image block centered by i-th pixel, G _αrepresent that standard deviation is the Gaussian function of α, its effect distributes larger weights the closer to the position at center, and less weights are distributed in deep position, and a ° expression step-by-step is multiplied, and h is attenuation parameter;

Described local similarity is constrained to: intuitively, in an image, each pixel and the pixel around it are extremely similar, so this phenomenon can be made full use of, current point is retrained with neighbouring pixel, discontinuous, rough phenomenon that image can be made so more smoothly can to overcome non local similarity constraint cause, both are called complementary constraint, and calculate this constraint by the method for controlled kernel regression, this constraint is from optimization problem:

$\underset{{\left\{{\mathrm{\&beta;}}_n\right\}}_0^N}{min}\underset{i=1}{\overset{P}{\&Sigma;}}{\&lsqb;x_i-{\mathrm{\&beta;}}_0-{\mathrm{\&beta;}}_1^T(l_i-l)-{\mathrm{\&beta;}}_2^{T}vech\left\{\right(l_i-l\left){\left(l_i-l\right)}^T\right\}-...\&rsqb;}^2*K_H(l_i-l),$

Wherein $K_H(l_i-l)=\frac{\sqrt{\det \left(C_i\right)}}{2{\mathrm{\&pi;h}}_k^2}\exp (-\frac{{(l_i-l)}^T{C}_i(l_i-l)}{2h_k^2}),$ K _h(l _i-l) be weight core, position l _inearer with l, weight is larger, C _ifor gradient covariance matrix, h _kfor the smoothing parameter of controlled core;

This problem is converted into matrix form: $\widehat{\mathrm{\&beta;}}=\underset{\mathrm{\&beta;}}{\mathrm{argmin}}{(X_l-\mathrm{\&Psi;}\mathrm{\&beta;})}^{T}K(X_l-\mathrm{\&Psi;}\mathrm{\&beta;})$

$\mathrm{\&beta;}=\&lsqb;{\mathrm{\&beta;}}_0,{\mathrm{\&beta;}}_1^T,...{\mathrm{\&beta;}}_N^T\&rsqb;$

X _l＝[x _i,x ₂,…,x _P] ^T

K＝diag[K _H(l ₁-l),K _H(l ₂-l),…,K _H(l _P-l)]

Wherein $\mathrm{\&Psi;}=\left[\begin{array}{cccc}1& {(l_1-l)}^T& vech\left\{(l_1-l){(l_1-l)}^T\right\}& \mathrm{...}\\ 1& {(l_2-l)}^T& vech\left\{(l_2-l){(l_2-l)}^T\right\}& \mathrm{...}\\ .& .& .& .\\ .& .& .& .\\ .& .& .& .\\ 1& {(l_P-l)}^T& vech\left\{(l_P-l){(l_P-l)}^T\right\}& \mathrm{...}\end{array}\right],$

$vech\left(\left[\begin{array}{ccc}a& b& c\\ b& e& f\\ c& f& i\end{array}\right]\right)={\left[\begin{array}{cccccc}a& b& c& e& f& i\end{array}\right]}^T$

The regularization constraint computing formula obtained is:

R _L＝||(I-W _L)X|| ₁

Wherein I is unit vector, and W _lhave:

$W_L(i,j)=\left\{\begin{array}{cc}{\left(e_1^T{\left({\mathrm{\&Psi;}}^{T}K\mathrm{\&Psi;}\right)}^{-1}{\mathrm{\&Psi;}}^{T}K\right)}_{i,j},& j\&Element;X_i\\ 0,& j\&NotElement;X_i\end{array}\right.$

X _irepresent the image block centered by i pixel, e ₁for only having first element to be 1, other elements are all the column vector of 0, and Ψ is the distance matrix of pixel.

The matching of described degree of depth confidence network is constrained to:

Described super-resolution Optimized model is expressed as:

$\begin{array}{c}\hat{X}=\underset{X}{\mathrm{argmin}}\left\{\frac{1}{2}{R}_R+{\mathrm{\&lambda;}}_{NL}{R}_{NL}+{\mathrm{\&lambda;}}_L{R}_L+\frac{1}{2}{\mathrm{\&lambda;}}_{DBN\_SR}{R}_{DBN\_SR}\right\}\\ =\underset{X}{\arg \min }\left\{\begin{array}{c}\frac{1}{2}\left|\right|DHX-Y|{|}_2^2+{\mathrm{\&lambda;}}_{NL}|\left|\right(I-W_{NL}\left)X\right|{|}_1+{\mathrm{\&lambda;}}_L\left|\right|(I-W_L)X|{|}_1\\ +\frac{1}{2}{\mathrm{\&lambda;}}_{DBN\_SR}\left|\right|X-X_{DBN\_SR}|{|}_2^2\end{array}\right\}\end{array}$

Wherein, D represents the matrix of down-sampling, and H represents fuzzy matrix, λ _nL, λ _land λ _{dBN_SR}be respectively the weighted value of three regular terms above, be used for regulating their proportions.

The method that the degree of depth learns is incorporated in image super-resolution rebuilding by the present invention, gives full play to the learning ability that degree of depth study is outstanding, carrys out the high-frequency information of Recovery image, then in conjunction with suitable regularization constraint, propose super-resolution rebuilding algorithm.

The present invention has following outstanding advantages:

1. first traditional degree of depth confidence network model for classification problem design is transformed by the present invention, makes its constrained input into real-valued data, be finally transformed into the model of applicable super-resolution rebuilding from binaryzation data.Utilize the stratification of degree of depth confidence network, the structure of nonlinearity, play the processing power that it is powerful to data, recover the high-frequency information that image is a large amount of.

2. design regularization term, set up image super-resolution rebuilding model.Owing to doing super-resolution rebuilding with DBN, do not consider the information of neighborhood, make the super-resolution rebuilding algorithm based on DBN have certain limitation.For this shortcoming, the present invention adds the regularization constraint that can make full use of neighborhood information, constructs the Super-resolution reconstruction established model based on regularization constraint and DBN.

3. design carrys out the numerical algorithm of rapid solving based on the graceful algorithm of separation Donald Bragg.

Accompanying drawing explanation

Fig. 1 is the image super-resolution rebuilding algorithm frame based on degree of depth confidence network of the present invention;

Fig. 2 is the reconstructed results of image only after degree of depth confidence network processes and the comparison of other 3 kinds of methods that use the present invention to amplify twice;

Fig. 3 uses the reconstructed results of the Nonlinear magnify twice of the present invention and existing 5 kinds of methods to compare.

Embodiment

1., with reference to Fig. 1, framework of the present invention is:

Step 1, obtains low resolution interpolation image Y.

From internet, a random panel height of downloading differentiates luminance picture X, and the bi-cubic interpolation utilizing the imresize function in Matlab software this low resolution luminance picture to be carried out 2 times amplifies, and obtains low-resolution image Y.

Step 2, obtains network output map X _{dBN_SR}.

(2a) by image from RGB color space conversion to YCbCr color space, and only follow-up process is carried out to the brightness value of image i.e. Y passage;

(2b) with moving window in the enterprising line slip of image, obtain the patch block Y of low resolution _i;

(2c) using the patch block of low resolution that obtains as input, utilize the good DBN of training in advance (TrainedDBN) to predict and obtain high-resolution patch block X _i;

(2d) the high-resolution patch block X will obtained _i, recombinate according to order originally, obtain wanted high-resolution image brightness picture X _{dBN_SR};

Step 3, rebuilds and obtains high-definition picture.

The value of Y passage and other two passages combined, then convert the image of RGB to, this just obtains predicted high-definition picture.

2. experimental result and interpretation of result:

Experiment one, carries out super-resolution image reconstruction result with the present invention through degree of depth confidence network.

In order to the validity of verification algorithm, on test library set9, compare with other three kinds of classic algorithm.The five width images of Fig. 2 are respectively: Bilinear is bilinear interpolation, and SCSR represents the super-resolution rebuilding algorithm based on rarefaction representation, and Bicubic represents segmentation cubic interpolation method.Conveniently, remember that algorithm of the present invention be DBN_SR, Original is original image.Table 1 is that the structural similarity (StructuralSimilarityIndexMeasurement, SSIM) of Fig. 2 reconstructed results and other algorithms and Y-PSNR (PeakSignaltoNoiseRatio, PSNR) compare.

The experimental result of Fig. 2 illustrates: from intuitive visual, and algorithm of the present invention processes in detail must be well a lot, on the texture of parrot eyes, more clear, careful than other three kinds of algorithms.Although bilinear interpolation speed is fast, sawtooth, blooming are serious, although bicubic interpolation than bilinear interpolation at edge smoothly a bit, still have serious blooming, SCSR can recover a lot of details, but still has ringing.Generally speaking, the algorithm that the present invention proposes, not only in visual effect, and all achieves effect more significant than other three kinds of typical algorithms, presents the super-resolution rebuilding performance that it is outstanding in objective evaluation standard.

Experiment two, adds the super-resolution image reconstruction result after regular terms constraint with the present invention through degree of depth confidence network.

In order to the validity of verification algorithm, on test library set9, compare with other three kinds of classic algorithm.It is original image that the seven width images of Fig. 3 are respectively Original, and Bicubic is segmentation cubic interpolation method, and SCSR is the super-resolution rebuilding algorithm based on rarefaction representation, ANR is that anchor point neighbour returns, SRCNN is degree of depth convolutional neural networks, and LLE is that local linear embeds, and Ours is the present invention.Table 2 compares with SSIM and PSNR of other algorithms for reconstructed results.

The experimental result of Fig. 3 and table 2 illustrates: after constantly adding three regular terms, that structural similarity (SSIM) and Y-PSNR (PSNR) all increase in test set, after adding all regular terms, obtain the highest effect.This experiment shows, three kinds of regular terms that the present invention proposes, and all increase to the effect of high-definition picture, illustrate that each is all useful, the combination of three effectively can improve the quality of final high-definition picture.

Table 1

Table 2

Test by when bicubic down-sampling when being input as former figure, enable algorithm and other algorithms in same starting point.Table 2 can be found out, all higher than other several classic algorithm on Y-PSNR is similar with structure, the oversubscription result SRCNN wherein obtained based on degree of depth convolutional neural networks is super-resolution rebuilding effect the best way in the paper delivered for 2014, but the present invention is after having added three regularization constraints, and image reconstruction effect is more than SRCNN's.Illustrate that algorithm of the present invention has very strong reconstruction ability.

Claims (7)

1., based on the image super-resolution rebuilding method of degree of depth confidence network, it is characterized in that comprising the following steps:

(1) the image Y of low resolution is inputted;

(2) the image Y of low resolution is first carried out bi-cubic interpolation amplification, be amplified to required size;

(3) by image from red, green, the RGB color space conversion that blue three Color Channels represent to brightness and blue red-color concentration side-play amount (YCbCr) color space, and only carries out follow-up process to the brightness value of image i.e. brightness (Y) passage;

(4) with moving window in the enterprising line slip of image, step-length is 1 get image block, obtains image block (patch) Y of low resolution _i, i represents the sequence number of image block;

(5) by the training image blocks of low resolution, and the high-definition picture block of correspondence, network parameter { W is obtained by training 3 limited Boltzmann machines _i, { c _i, wherein, { W _irepresent the weight vector that the limit in network connects, { c _irepresent being biased next time to last layer in network, with training { the W obtained _i, { c _ithe parameter of initialization degree of depth confidence network (DeepBeliefNetwork, DBN);

(6) train DBN network (TrainedDBN), 3 RBM are gathered into folds, obtains DBN, utilize training image blocks obtained above, finely tune parameter { W with the back propagation of stochastic gradient descent method and error _i, { c _i, until e<tol or iterations t>T, e represents error, tol and T is constant threshold given in algorithm;

(7) by image block (patch block) Y of the low resolution of image to be tested _ias input, the DBN utilizing step (6) to train predicts and obtains corresponding high-resolution patch block X _i;

(8) the high-resolution patch block X will obtained _i, the pixel assignment of getting in the middle of image block is the pixel value of full resolution pricture correspondence position coordinate, to the patch block Y of all low resolution _iprocess, obtain required high-resolution image brightness picture X _{dBN_SR};

(9) add non local similarity constraint, local similarity constraint, degree of depth confidence network matching constraint, set up the super-resolution Optimized model based on neighborhood relationships regularization and fit correlation regularization;

(10) Optimized model in solution procedure (9) is carried out with the graceful iterative algorithm of Donald Bragg be separated;

(11) value (CbCr) of the luminance channel obtained (Y) Super-resolution Reconstruction image with other two Color Channels is combined, then convert the image of RGB to, obtain predicted high-definition picture X.

2. as claimed in claim 1 based on the image super-resolution rebuilding method of degree of depth confidence network, it is characterized in that in step (5), the method for the Boltzmann machine that described training 3 is limited is:

(5.1) energy function of known RBM is E (υ, h; θ)=-h ^tw υ-b ^th-c ^tυ, θ=and W, b, c}, joint probability is defined as, wherein Z=Σ _vΣ _hexp (-E (υ, h; θ)), v is RBM input, and h is that RBM hidden layer exports, and W represents that the weight vector that input layer limit in a network connects, c represent input layer being biased last layer in network, and b represents last layer being biased lower one deck in network; Setting iteration total degree T, the parameter θ that random initializtion is to be trained ₀=(W ₀, b ₀, c ₀) and its assignment to θ _t, t represents current iteration number of times;

(5.2) by proper vector to be entered/image block assignment to v ₀, and utilizing P (v|h), P (h|v) iterates and obtains h n time ₀, v _nand h _n, v ₀represent each component of the 0th iteration input vector, h ₀represent each component of the 0th iteration hidden layer, conditional probability is calculated as follows:

$P\left(h\right|v)=\underset{i}{\&Pi;}P\left(h_i\right|v),$

P(h _i＝1|v,θ)＝σ(c _i+w _iv)，

P(v|h)＝Π _jP(v _i|h)，

P(v _j＝1|h,θ)＝σ(b _j+w' _jh)，

Wherein σ () is activation function;

(5.3) utilize the joint probability distribution Grad that stochastic gradient descent method and error back propagation method obtain through n gibbs sampler, and upgrade θ _tthree parameter W of the inside _t, b _t, c _t, obtain θ _t+1;

(5.4) if iterations t=T, or difference value reaches necessarily little degree, then EOP (end of program); Otherwise by θ _t+1assignment is to θ _t, and return (5.2) step.

3. as claimed in claim 1 based on the image super-resolution rebuilding method of degree of depth confidence network, it is characterized in that in step (6), the method for described training DBN network is:

(6.1) stacked by 3 RBM, namely first RBM is with low-resolution image block for input, and the input that the output that computing obtains obtains as second RBM, the output of second RBM is as the input of the 3rd RBM, and the output finally obtained is X _{dBN_SR};

(6.2) stochastic gradient descent method and error back propagation method training network is utilized, fine setting θ _t3 parameter W of the inside _t, b _t, c _t, obtain θ _t+1;

(6.3) if iterations t=T, or difference value reaches necessarily little degree, then EOP (end of program); Otherwise by θ _t+1assignment is to θ _t, and return (6.1).

4. as claimed in claim 1 based on the image super-resolution rebuilding method of degree of depth confidence network, it is characterized in that in step (9), described non-local similarity constraint is: in the neighborhood that certain pixel is larger, utilize similar pixel, retrain current point, namely represent current point by similitude weighted mean, computing formula is:

R _NL＝||(I-W _NL)X|| ₁

Wherein X is the vector form of input picture; I is unit matrix; W _nLfor weight matrix, if represent, block i is more similar to block j, then weighted value is larger, and the value of each element is provided by formula below:

$W_{NL}(i,j)=\left\{\begin{array}{cc}\frac{w(i,j)}{\underset{j\&Element;S_i}{\&Sigma;}w(i,j)},& j\&Element;S_i\\ 0,& j\&NotElement;S_i\end{array}\right.$

Wherein, x _irepresent the image block centered by i-th pixel, G _αrepresent that standard deviation is the Gaussian function of α, its effect distributes larger weights the closer to the position at center, and less weights are distributed in deep position, and ο represents that step-by-step is multiplied, and h is attenuation parameter.

5. as claimed in claim 1 based on the image super-resolution rebuilding method of degree of depth confidence network, it is characterized in that in step (9), described local similarity is constrained to: intuitively, in an image, each pixel and the pixel around it are extremely similar, so this phenomenon can be made full use of, current point is retrained with neighbouring pixel, image can be made so more smoothly can to overcome, and that non local similarity constraint causes is discontinuous, rough phenomenon, both are called complementary constraint, this constraint is calculated by the method for controlled kernel regression, this constraint is from optimization problem:

$\underset{{\left\{{\mathrm{\&beta;}}_n\right\}}_0^N}{min}\underset{i=1}{\overset{P}{\&Sigma;}}{\&lsqb;x_i-{\mathrm{\&beta;}}_0-{\mathrm{\&beta;}}_1^T(l_i-l)-{\mathrm{\&beta;}}_2^{T}vech\left\{\right(l_i-l\left){\left(l_i-l\right)}^T\right\}-...\&rsqb;}^2*K_H(l_i-l),$

Wherein $K_H(l_i-l)=\frac{\sqrt{\det \left(C_i\right)}}{2{\mathrm{\&pi;h}}_k^2}\exp (-\frac{{(l_i-l)}^T{C}_i(l_i-l)}{2h_k^2}),$ K _h(l _i-l) be weight core, position l _inearer with l, weight is larger, C _ifor gradient covariance matrix, h _kfor the smoothing parameter of controlled core;

This problem is converted into matrix form: $\widehat{\mathrm{\&beta;}}=\underset{\mathrm{\&beta;}}{\mathrm{argmin}}{(X_l-\mathrm{\&Psi;}\mathrm{\&beta;})}^{T}K(X_l-\mathrm{\&Psi;}\mathrm{\&beta;})$

$\mathrm{\&beta;}=\&lsqb;{\mathrm{\&beta;}}_0,{\mathrm{\&beta;}}_1^T,...{\mathrm{\&beta;}}_N^T\&rsqb;$

X _l＝[x _i,x ₂,…,x _P] ^T

K＝diag[K _H(l ₁-l),K _H(l ₂-l),…,K _H(l _P-l)]

Wherein $\mathrm{\&Psi;}=\left[\begin{array}{cccc}1& {(l_1-l)}^T& vech\left\{(l_1-l){(l_1-l)}^T\right\}& \mathrm{...}\\ 1& {(l_2-l)}^T& vech\left\{(l_2-l){(l_2-l)}^T\right\}& \mathrm{...}\\ .& .& .& .\\ .& .& .& .\\ .& .& .& .\\ 1& {(l_P-l)}^T& vech\left\{(l_P-l){(l_P-l)}^T\right\}& \mathrm{...}\end{array}\right],$

$vech\left(\left[\begin{array}{ccc}a& b& c\\ b& e& f\\ c& f& i\end{array}\right]\right)={\left[\begin{array}{cccccc}a& b& c& e& f& i\end{array}\right]}^T$

The regularization constraint computing formula obtained is:

R _L＝||(I-W _L)X|| ₁

Wherein I is unit vector, and W _lhave:

$W_L(i,j)=\left\{\begin{array}{cc}{\left(e_1^T{\left({\mathrm{\&Psi;}}^{T}K\mathrm{\&Psi;}\right)}^{-1}{\mathrm{\&Psi;}}^{T}K\right)}_{i,j},& j\&Element;X_i\\ 0,& j\&NotElement;X_i\end{array}\right.$

X _irepresent the image block centered by i pixel, e ₁for only having first element to be 1, other elements are all the column vector of 0, and Ψ is the distance matrix of pixel.

6. as claimed in claim 1 based on the image super-resolution rebuilding method of degree of depth confidence network, it is characterized in that in step (9), the matching of described degree of depth confidence network is constrained to:

7., as claimed in claim 1 based on the image super-resolution rebuilding method of degree of depth confidence network, it is characterized in that, in step (9), described super-resolution Optimized model is expressed as:

$\begin{array}{c}\hat{X}=\underset{X}{\mathrm{argmin}}\left\{\frac{1}{2}{R}_R+{\mathrm{\&lambda;}}_{NL}{R}_{NL}+{\mathrm{\&lambda;}}_L{R}_L+\frac{1}{2}{\mathrm{\&lambda;}}_{DBN\_SR}{R}_{DBN\_SR}\right\}\\ =\underset{X}{\arg \min }\left\{\begin{array}{c}\frac{1}{2}\left|\right|DHX-Y|{|}_2^2+{\mathrm{\&lambda;}}_{NL}|\left|\right(I-W_{NL}\left)X\right|{|}_1+{\mathrm{\&lambda;}}_L\left|\right|(I-W_L)X|{|}_1\\ +\frac{1}{2}{\mathrm{\&lambda;}}_{DBN\_SR}\left|\right|X-X_{DBN\_SR}|{|}_2^2\end{array}\right\}\end{array}$

Wherein, D represents the matrix of down-sampling, and H represents fuzzy matrix, λ _nL, λ _land λ _{dBN_SR}be respectively the weighted value of three regular terms above, be used for regulating their proportions.