CN105787899A - Rapid image super-resolution method based on self-adaptive regression - Google Patents

Abstract

The invention discloses a rapid image super-resolution method based on self-adaptive regression. The method comprises the steps of: in a training stage, extracting a training sample set from existing high quality images, using the training sample set to carry out training, obtaining a dictionary and calculating a mapping model; in a dictionary base reordering stage, using a statistical law to reorder the dictionary bases to obtain a new dictionary; and in a super-resolution stage, using the reordered dictionary and the mapping model to perform the super-solution on an input low-resolution image, determining the number of the dictionary bases to be used so as to shrink the searching space and increase a super-resolution speed, and obtaining an output high-solution image through the mapping model for each low-solution characteristic. According to the invention, additional parameters are not needed in the training process, so that the high-resolution characteristics can be restored more precisely in the super-resolution process; in addition, statistical information is used to reorder the dictionary bases and the more effective dictionary bases are used in the super-resolution process, so that the super-resolution speed of the image is increased effectively.

Description

A kind of rapid image ultra-resolution method returned based on self adaptation

Technical field

Present patent application relates to computer vision and image processing field, particularly to a kind of rapid image ultra-resolution method returned based on self adaptation.

Background technology

Image Super-resolution belongs to computer vision and image processing field, is a classical image processing problem, has important science and industrial research to be worth.The target of Image Super-resolution is exactly, and by the low-resolution image given, reconstructs its corresponding high-definition picture so that when reconstructed error is little as far as possible, and visual effect is good as much as possible.The image super-resolution method of current main flow can be divided into three major types: based on the method for interpolation；Method based on reconstruct；Method based on study.

Based on the method for interpolation, being the basic ultra-resolution method of a class, its processing procedure would generally use local covariance coefficient, fixing function core or adaptive structure core, the feature simple and quick because of it and be widely used.But, in a lot of situations, the result that this kind of method produces can produce visual artifact along with the increase of amplification, such as: sawtooth effect and blurring effect.Method based on reconstruct, it is assumed that low-resolution image is obtained through several degeneration factors by high-definition picture, such as: down-sampled and obfuscation.This kind of method emphasizes the importance of Reconstruction Constraints in super-resolution process, thus, its high-definition picture obtained often has excessively smooth and factitious edge, and can produce ringing effect near image border.Based on the method for study, because utilizing machine learning techniques from training set learning to a large amount of prioris, thus obtain better result.But, this kind of method typically requires and solves based on L₀Norm or L₁The optimization problem of norm, its processing speed is generally very slow.

Summary of the invention

It is an object of the invention to propose a kind of rapid image ultra-resolution method returned based on self adaptation, overcome the technological deficiency that the processing speed of prior art existence is slow.

For solving the technical problem of above-mentioned prior art, the present invention is by the following technical solutions.

A kind of rapid image ultra-resolution method returned based on self adaptation, it is characterised in that comprise the steps:

The A1 training stage: extract training sample set from existing high quality graphic, utilize described training sample set train dictionary and calculate mapping model M；Described mapping model M includes: PCA dimensionality reduction matrix P, low-resolution dictionary D and mapping matrix F_k；

A2 dictionary basic weight phase sorting: utilize statistical law that the base of dictionary is resequenced, it may be assumed that utilize described training sample set and the mapping model M obtained, the base of described low-resolution dictionary D is reordered, obtain new dictionary D_R；

The A3 super-resolution stage: the low resolution image of input is carried out super-resolution by the dictionary obtained after utilizing sequence and mapping model, obtain the high-definition picture of output: namely: first determine the number K ' of the dictionary base that will use, for reducing search volume, and then accelerate super-resolution speed, for each low resolution featureThe full resolution pricture I of output is obtained by mapping model M_H。

The implementation process of described A1 training stage is: obtain high-definition picture from described training sample set；Described high-definition picture is carried out down-sampling by minification, obtains low-resolution image；High-resolution features and the low resolution feature of correspondence is extracted respectively from corresponding high-definition picture, low-resolution image；High-resolution features, low resolution features training described in utilizing go out dictionary, and calculate mapping matrix, and then obtain mapping model.

The implementation process of described A2 dictionary basic weight phase sorting is: extract more low resolution characteristic set from described training sample set；To each low resolution feature in described low resolution characteristic set, utilize similarity measurement, find dictionary base immediate with it, and think that this dictionary base neighbour's feature includes this low resolution feature；All low resolution features are all added up one time, obtains the size of neighbour's characteristic set of all dictionary bases；According to the size of described neighbour's characteristic set, dictionary base is resequenced.

The implementation process in described A3 super-resolution stage is: for each described low resolution feature, find dictionary base immediate with it, then recover high-resolution features with corresponding mapping matrix in low-resolution dictionary；By all high-resolution features recovered plus the image block comprising low-frequency information, obtain corresponding high-definition picture block, all high-definition picture blocks are fused into a high-definition picture.

Described mapping model includes low-resolution dictionary, PCA dimensionality reduction matrix and mapping matrix.

The present invention compared with prior art, including advantages below and beneficial effect:

1. the present invention proposes a kind of rapid image ultra-resolution method returned based on self adaptation, can directly learn to the mapping relations between low resolution feature and high-resolution features, and because using the low resolution feature dependency with dictionary base as mark in mapping process, not only do not need additional parameter in the training process, and high-resolution features can be recovered in super-resolution process more accurately；Utilize statistical information that dictionary base is reordered, super-resolution process uses more effective dictionary base, thus effectively accelerating super-resolution speed.

2. the present invention does it is assumed that but directly train low resolution to the mapping relations one by one between high-resolution features without the coefficient of expressing that height is differentiated feature.

3. the present invention adopts more effective character representation form.During training mapping model, it does not have be confined in single cluster, but carry out in the neighbour space of dictionary base.Obtain the self adaptation mark value that each low resolution feature is corresponding, in regression model, but not do not treat with making any distinction between.

Accompanying drawing explanation

Fig. 1 is the flow chart of a kind of embodiment of the rapid image ultra-resolution method that the present invention returns based on self adaptation.

Detailed description of the invention

It is described in further detail below in conjunction with drawings and Examples and to the present invention.

As it is shown in figure 1, be the flow chart of a kind of embodiment of rapid image ultra-resolution method of the present invention.The method comprises the following steps:

The A1 training stage: extract training sample set from existing high quality graphic, utilize described training sample set train dictionary and calculate mapping model M；Described mapping model M includes: PCA (PrincipalComponentAnalysis) dimensionality reduction matrix P, low-resolution dictionary D and mapping matrix F_k；

A2 dictionary basic weight phase sorting: utilize statistical law that the base of dictionary is resequenced, it may be assumed that utilize described training sample set and the mapping model M obtained, the base of described low-resolution dictionary D is reordered, obtain new dictionary D_R；

The A3 super-resolution stage: the low resolution image of input is carried out super-resolution by the dictionary obtained after utilizing sequence and mapping model, obtain the high-definition picture of output: namely: first determine the number K' of the dictionary base that will use, for reducing search volume, and then accelerate super-resolution speed, for each low resolution featureThe full resolution pricture of output is obtained by mapping model M.

The implementation process of described A1 training stage is: obtain high-definition picture from described training sample set；Described high-definition picture is carried out down-sampling by minification, obtains low-resolution image；High-resolution features and the low resolution feature of correspondence is extracted respectively from corresponding high-definition picture, low-resolution image；High-resolution features, low resolution features training described in utilizing go out dictionary, and calculate mapping matrix, and then obtain mapping model.

The implementation process of described A2 dictionary basic weight phase sorting is: extract more low resolution characteristic set from described training sample set；To each low resolution feature in described low resolution characteristic set, utilize similarity measurement, find dictionary base immediate with it, and think that this dictionary base neighbour's feature includes this low resolution feature；All low resolution features are all added up one time, obtains the size of neighbour's characteristic set of all dictionary bases；According to the size of described neighbour's characteristic set, dictionary base is resequenced.

The implementation process in described A3 super-resolution stage is: for each described low resolution feature, find dictionary base immediate with it, then recover high-resolution features with corresponding mapping matrix in low-resolution dictionary；By all high-resolution features recovered plus the image block comprising low-frequency information, obtain corresponding high-definition picture block, all high-definition picture blocks are fused into a high-definition picture.

Described mapping model includes low-resolution dictionary, PCA dimensionality reduction matrix and mapping matrix.

Below it is described in further detail.

The A1 training stage: using natural image common data sets (such as ImageNet data set) as training set, therefrom obtain high-definition pictureIts down-sampling is obtained low-resolution imageMinification is s, extracts low resolution characteristic set from the image pair of high-resolution and low-resolutionAnd the bigger high-resolution and low-resolution feature of quantity is to setWithUtilizeAcquire low-resolution dictionaryAnd PCA (PrincipalComponentAnalysis) dimensionality reduction matrix P.UtilizeWithAnd D calculates the mapping matrix of correspondence

A2 dictionary basic weight sorts: utilize the low-resolution dictionary D training out in A1 and low resolution characteristic setObtain the statistical property about dictionary base further, utilize this statistical property to dictionary basic weight new sort, obtain new dictionary D_R。

The A3 quick super-resolution stage: first choose dictionary base number K', the D to use_RIn front K' dictionary base just constitute little dictionaryFor each low resolution featureIn low-resolution dictionaryIn find dictionary base immediate with it(dictionary base is exactly the column vector in dictionary), then use corresponding mapping matrixRecover high-resolution featuresBy all high-resolution features recoveredPlus the image block comprising low-frequency informationObtain corresponding high-definition picture blockAll full resolution pricture blocks are fused into the full resolution pricture I of an output_H。

In particular embodiments, can operate by following mode.

The A1 training stage: using natural image common data sets (such as ImageNet data set) as training set.First use the first interpolation algorithm, from high-definition pictureIn obtain low-resolution imageThese low-resolution images utilize the second interpolation algorithm up-sampling again, and in said process, the minification of down-sampling and the amplification of up-sampling are all s times.Second interpolation algorithm and the first interpolation algorithm can be identical, it is also possible to different.FromMiddle extraction high-definition picture set of blocksWith characteristic set?In relevant position extract two pairs of low-resolution image set of blocksWith characteristic setWherein, high-resolution features y_HFor:

y_H=p_H-p_L(1)

Low resolution feature y_LFor:

y_L=[f₁*p_L；f₂*p_L；f₃*p_L；f₄*p_L](2)

Wherein, f₁And f₂It is the gradient high pass filter of level and vertical direction, f₃And f₄Being level and vertical direction Laplacian high-pass filter, symbol * represents convolution operation.In order to reduce computational complexity, adopting PCA dimensionality reduction, PCA transformation matrix is P.In order to obtain the dictionary of low resolution, it is possible to use the low resolution characteristic set after dimensionality reductionOptimize following objective functions:

$\begin{array}{c}\underset{D,X}{\min }\left|\right|Y_{Ls}-DX|{|}_F^2\\ s.t.\left|\right|d_k|{|}_2=1,and,|\left|x_i\right|{|}_0\≤L\end{array}---\left(3\right)$

Wherein,X={x_iIt is low resolution characteristic set, dictionary and coefficient respectively.Symbol min represents the minima seeking object function, and D and X is exactly the result that solving-optimizing problem to export, the condition met when symbol s.t. represents solving-optimizing problem.||||_F、||||₂With | | | |₀Represent F-norm, 2-norm and 0-norm respectively.L is used to the positive integer of restriction degree of rarefication.Utilize high-resolution and low-resolution feature pairWithBase d for any low-resolution dictionary_k,Its max neighbour of middle searching, composition low resolution neighbour set { N_L,k}.And calculate low resolution feature in neighbour's setWith dictionary base d_kBetween self adaptation degree of associationAnd form matrix $L_k={\left\{l_k^i\right\}}_{i=1}^{\max },$ Wherein,

$l_k^i=e^{c_{k,i}}---\left(4\right)$

By high-resolution features setThe feature of middle relevant position is used for forming high-resolution neighbour set { N_H,k}.Assuming that a low resolution feature y_L, the low-resolution dictionary base nearest with it and neighbour's set are d respectively_kAnd N_L,k, in order to obtain reconstruct high-resolution features y_HRequired mapping matrix F_k, first solve following optimization objective function:

$\underset{F_k}{min}\left|\right|N_{H,k}-F_k\left[\begin{array}{c}{N}_{L,k}\\ L_k\end{array}\right]|{|}_F^2---\left(5\right)$

In formula (5), symbol min represents the minima seeking object function, N_H,kAnd N_L,kIt is dictionary base d_kCorresponding high-resolution and low-resolution neighbour set, L_kIt is corresponding adaptive matrix, F_kIt is by the mapping matrix of low resolution Feature Mapping to high-resolution features, is also the result to export of this optimization problem.There are analytic solutions in this problem, concrete form is:

$F_k=N_{H,k}{M}_k^T{\left(M_k{M}_k^T\right)}^{-1}---\left(6\right)$

In this training stage, for any one dictionary base d_k, we train and obtain a mapping model F_k, by above-mentioned formula (6):Wherein, ()^TThe transposition of representing matrix, ()^-1Representing matrix inverse, $M_k=\left[\begin{array}{c}{N}_{L,k}\\ L_k\end{array}\right]=\left[\begin{array}{cccc}{y}_{L,k}^1& y_{L,k}^2& L& y_{L,k}^{\max }\\ l_k^1& l_k^2& L& l_k^{\max }\end{array}\right].$ It will be seen that in the training process, different low resolution featureThe mark value having correspondence participates inReturn.These mark value are different along with the difference of low resolution feature, have adaptive characteristic, can more accurately by low resolution Feature Mapping to corresponding high-resolution features.At test phase, i.e. the super-resolution stage, for arbitrarily low resolution characteristicsIf the dictionary base of its correspondence is d_k, then self adaptation mark value is exactlyBy formula (7), it may be assumed that $y_H^i=F_k\left[\begin{array}{c}{y}_L^i\\ l_k^i\end{array}\right],$ Just can obtain the high-resolution features of correspondence.Therefore, this regression model has adaptive characteristic, returns so being referred to as self adaptation.

In formula (6), ()^TThe transposition of representing matrix, ()^-1Representing matrix inverse, $M_k=\left[\begin{array}{c}{N}_{L,k}\\ L_k\end{array}\right].$ So corresponding high-resolution features can be tried to achieve by following formula:

$y_H^i=F_k\left[\begin{array}{c}{y}_L^i\\ l_k^i\end{array}\right]---\left(7\right)$

In formula (7),It is dictionary base d_kWith low resolution featureBetween self adaptation mark value..The F of above-mentioned formula_kIt is and the y of input_LUnrelated, thus can calculated off line out, F_kIt is exactly dictionary base d_kCorresponding mapping matrix.Therefore for each low-resolution dictionary base, its corresponding mapping matrix can be tried to achieve.The algorithm specifically solving above-mentioned dictionary learning optimization problem can adopt KSVD algorithm etc., and the scope that present patent application contains is not limited to exemplified method.

A2 dictionary basic weight sorts: utilize the low-resolution dictionary D training out in A1 and low resolution characteristic setObtain the statistical property about dictionary base further.For each low resolution featureCalculate degree of associationThe base d that degree of association is maximum_k, it is calledArest neighbors base, we also referred to asFor d_kArest neighbors feature (d_kArest neighbors feature can have multiple), find each dictionary base d_kCorresponding arest neighbors set omega_k, utilize this statistical property to dictionary basic weight new sort, obtain new dictionaryMake

$\left|{\mathrm{\Ω}}_{R_1}\right|\≥\left|{\mathrm{\Ω}}_{R_2}\right|\≥L\≥\left|{\mathrm{\Ω}}_{R_K}\right|---\left(8\right)$

Wherein, | | represent the number of element in set.In dictionary after sequence, the position that more important base is put is more forward.In rapid super-resolution, as long as we choose the mapping matrix of previous section dictionary base and correspondence thereof, just can obtain good effect.

The A3 rapid super-resolution stage: first choose dictionary base number K', the D to use_RIn front K' dictionary base just constitute little dictionaryFor the low-resolution image I arbitrarily inputted_L, therefrom extract overlapped low resolution characteristic set.For each low resolution featureIn low-resolution dictionaryIn find dictionary base immediate with itAnd corresponding mapping matrixCalculateWithBetween self adaptation mark valueThen pass through formula (7) and recover high-resolution featuresBy all high resoluting characteristics recoveredPlus the image block comprising low-frequency informationObtain corresponding high-definition picture blockAll full resolution pricture blocks are fused into a full resolution pricture I_H。

In a word, the inventive method can directly learn to low resolution feature to high-resolution features between mapping relations, and because using the low resolution feature dependency with dictionary base as mark in mapping process, not only do not need additional parameter in the training process, and high-resolution features can be recovered in super-resolution process more accurately；Utilize statistical information that dictionary base is reordered, super-resolution process uses more effective dictionary base, thus accelerating super-resolution speed.

Claims (5)

1. the rapid image ultra-resolution method returned based on self adaptation, it is characterised in that comprise the steps:

The A1 training stage: extract training sample set from existing high quality graphic, utilize described training sample set train dictionary and calculate mapping model M；Described mapping model M includes: PCA dimensionality reduction matrix P, low-resolution dictionary D and mapping matrix F_k；

A2 dictionary basic weight phase sorting: utilize statistical law that the base of dictionary is resequenced, it may be assumed that utilize described training sample set and the mapping model M obtained, the base of described low-resolution dictionary D is reordered, obtain new dictionary D_R；

The A3 super-resolution stage: the low resolution image of input is carried out super-resolution by the dictionary obtained after utilizing sequence and mapping model, obtain the high-definition picture of output: namely: first determine the number K' of the dictionary base that will use, for reducing search volume, and then accelerate super-resolution speed, for each low resolution featureThe full resolution pricture I of output is obtained by mapping model M_H。

2. a kind of rapid image ultra-resolution method returned based on self adaptation according to claim 1, it is characterised in that the implementation process of described A1 training stage is:

High-definition picture is obtained from described training sample set；Described high-definition picture is carried out down-sampling by minification, obtains low-resolution image；High-resolution features and the low resolution feature of correspondence is extracted respectively from corresponding high-definition picture, low-resolution image；High-resolution features, low resolution features training described in utilizing go out dictionary, and calculate mapping matrix, and then obtain mapping model.

3. a kind of rapid image ultra-resolution method returned based on self adaptation according to claim 1, it is characterised in that the implementation process of described A2 dictionary basic weight phase sorting is:

More low resolution characteristic set is extracted from described training sample set；To each low resolution feature in described low resolution characteristic set, utilize similarity measurement, find dictionary base immediate with it, and think that this dictionary base neighbour's feature includes this low resolution feature；All low resolution features are all added up one time, obtains the size of neighbour's characteristic set of all dictionary bases；According to the size of described neighbour's characteristic set, dictionary base is resequenced.

4. a kind of rapid image ultra-resolution method returned based on self adaptation according to claim 1, it is characterised in that the implementation process in described A3 super-resolution stage is:

For each described low resolution feature, low-resolution dictionary finds dictionary base immediate with it, then recovers high-resolution features with corresponding mapping matrix；By all high-resolution features recovered plus the image block comprising low-frequency information, obtain corresponding high-definition picture block, all high-definition picture blocks are fused into a high-definition picture.

5. a kind of rapid image ultra-resolution method returned based on self adaptation according to claim 2, it is characterised in that: described mapping model includes low-resolution dictionary, PCA dimensionality reduction matrix and mapping matrix.