CN110648392A - Iterative initial image generation method based on learning and sparse representation - Google Patents

Iterative initial image generation method based on learning and sparse representation Download PDF

Info

Publication number
CN110648392A
CN110648392A CN201910845539.XA CN201910845539A CN110648392A CN 110648392 A CN110648392 A CN 110648392A CN 201910845539 A CN201910845539 A CN 201910845539A CN 110648392 A CN110648392 A CN 110648392A
Authority
CN
China
Prior art keywords
resolution
image
image block
block
low
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910845539.XA
Other languages
Chinese (zh)
Other versions
CN110648392B (en
Inventor
曾向荣
刘衍
周典乐
龙鑫
孙博良
钟志伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
National University of Defense Technology
Original Assignee
National University of Defense Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by National University of Defense Technology filed Critical National University of Defense Technology
Priority to CN201910845539.XA priority Critical patent/CN110648392B/en
Publication of CN110648392A publication Critical patent/CN110648392A/en
Application granted granted Critical
Publication of CN110648392B publication Critical patent/CN110648392B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/005—General purpose rendering architectures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Graphics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses an iterative initial image generation method based on learning and sparse representation, which specifically comprises the following steps: inputting a low-resolution image block sequence; dictionary D for calculating low-resolution image blocklAnd a dictionary D of high-resolution image blocks corresponding theretoh(ii) a Acquiring a high-resolution initial interpolation image by a non-uniform interpolation method; based on the learning dictionary DlAnd to calculate a sparse representation sparsity a, then based on said learning dictionary DhAnd calculating a high-frequency prior image block hiAnd finally, overlapping the obtained down-sampled high-frequency prior image and the high-resolution initial interpolation image to generate a high-resolution iterative initial image.

Description

Iterative initial image generation method based on learning and sparse representation
Technical Field
The invention relates to an iterative initial image generation method based on learning and sparse representation, in particular to a method based on learning and sparse representation, which is used for further improving the quality of an initial image so as to be more beneficial to later image reconstruction.
Background
At present, the reconstruction-based method still occupies an important position in the super-resolution reconstruction field, and the reasons are mainly complete algorithm theory, moderate numerical solution computation amount, strong stability of the computation method and high robustness. The MAP method and the variation regularization method still receive much attention. The method generally utilizes the prior knowledge to construct a regular term, construct a minimum energy functional equation and then obtain a feasible solution by utilizing an iterative solution method. However, many methods at present mainly focus on the optimization of a regularization term and the optimization of a numerical solution method, and in the iterative solution, a high-resolution iterative initial image is generated only by interpolating a reference image in a low-resolution frame. Since it only contains information of one image, there is a certain adverse effect on the quality of the reconstructed image. If the iteration initial image can contain the information of each frame in the low-resolution image sequence, better reconstruction effect can be obtained. To address this problem, Lukin et al employ a New Edge-directed interpolation method (NEDI) to improve the quality of the initial high-resolution iterative image. SongRui et al estimate the blur parameters of the low-resolution image sequence by training using an analog analysis method of the blur parameters, then calculate analogism between polynomials and linear correlation of estimation error change according to the high-resolution image and the reference image, fuse all low-resolution image information into a high-resolution initial image using a training estimation result, and estimate the final high-resolution image using a MAP-based method.
Disclosure of Invention
In order to satisfy the requirement that the iterative initial image can contain the information of each block of the low-resolution image so as to obtain a better reconstruction effect, the invention provides an iterative initial image generation method based on learning and sparse representation, which comprises the following steps:
1) obtaining a set TrI ═ F of high and low resolution image block pairsh,GlIn which Fh＝{f1,f2,…fiIs the set of high resolution image blocks, Gl＝{g1,g2,…giIs FhSet of corresponding low resolution image blocks, fiFor the ith block of high resolution image block, giThe image block is the ith block and the first resolution image block, and i is a natural number;
2) computing a corresponding learning dictionary D using high resolution image blocks and low resolution image blockshAnd DlAnd have them the same sparse representation;
3) based on the learning dictionary DhAnd DlObtaining a high-resolution initial interpolation image R through a non-uniform interpolation method, and dividing the high-resolution initial interpolation image R into R1,r2…riBlocks, leaving a certain pixel overlap between blocks, where riRepresenting the ith high-resolution initial interpolation image block, wherein i is a natural number;
4) based on the learning dictionary DlAnd calculating a sparse representation coefficient alpha by using formula (1), and then based on the learning dictionary DhAnd calculating a high-frequency prior image block h by using a formula (2)iAll the high-frequency prior image blocks hiRecombining on HR grid, averaging the overlapped area to obtain high-frequency prior image H,
hi＝Dhα (2)
wherein
Figure BDA0002196961560000022
Parameter beta is used to control low resolution image blocks and high frequency prior matching with adjacent image blocksThe method comprises the following steps that weight among image block estimation values is obtained, Q is a gradient operator, lambda is a regularization coefficient, U is used for extracting an overlapping area of a currently calculated high-frequency prior image block and the calculated high-frequency prior image block, and y represents a value of the calculated high-frequency prior image block in the current overlapping area;
5) and finally, superposing the downsampled high-resolution prior image and the high-resolution initial interpolation image R to generate a high-resolution iterative initial image.
Preferably, the step 2 is a specific process of obtaining the learning dictionary D with the same sparse representation by solving the formula (3)hAnd Dl
Figure BDA0002196961560000031
Wherein
Figure BDA0002196961560000032
Alpha represents the sparse representation coefficient of the sample X under D, chi is the regularization coefficient, N and M respectively represent the dimensionality of the high-resolution image block vector form and the dimensionality of the low-resolution image block vector form, and 1/N and 1/M are used for balancing the influence of different sizes of the high-resolution image block and the low-resolution image block on the whole representation frame.
Preferably, the gradient operator Q is in a specific form as follows:
Q1＝[-1,0,1],Q2＝Q1 T
Q3＝[1,0,-2,0,1],Q4＝Q3 T
wherein Q1，Q2，Q3And Q4Respectively representing a first order gradient operator, a second order gradient operator, a third order gradient operator and a fourth order gradient operator.
The invention has the beneficial effects that: the quality of the iteration initial image is further improved by using a method based on learning and sparse representation, and the subsequent reconstruction is facilitated.
Drawings
FIG. 1 is a detailed flow chart of the present invention.
Detailed Description
In order to more clearly describe the technical contents of the present invention, the following further description is given in conjunction with specific embodiments.
The invention uses a non-uniform interpolation method to fuse the information of each frame of the low-resolution image sequence into the high-resolution initial image, uses the high-frequency image corresponding to the initial image generated by the learning-based method as prior information, and then superposes the prior information and the high-frequency image as an iterative initial image. And finally, estimating a high-resolution reconstructed image by using an anisotropic space self-adaptive regularization processing method and a self-adaptive bilateral total variation model and a self-adaptive trilateral filtering variation model so as to enhance the edge contour and texture structure holding capacity of a reconstruction algorithm and improve the subjective visual effect and objective evaluation index of the reconstructed image. Referring specifically to fig. 1, the method comprises the following steps:
1) obtaining a set TrI ═ F of high and low resolution image block pairsh,GlIn which Fh＝{f1,f2,…fiIs the set of high resolution image blocks, Gl＝{g1,g2,…giIs FhSet of corresponding low resolution image blocks, fiFor the ith block of high resolution image block, giThe image block is the ith block and the first resolution image block, and i is a natural number;
2) computing a corresponding learning dictionary D using high resolution image blocks and low resolution image blockshAnd DlAnd have them the same sparse representation;
3) based on the learning dictionary DhAnd DlObtaining a high-resolution initial interpolation image R through a non-uniform interpolation method, and dividing the high-resolution initial interpolation image R into R1,r2…riSo that a certain pixel overlap is preserved from block to block, where riRepresenting the ith high-resolution initial interpolation image block, wherein i is a natural number;
4) based on the learning dictionary DlAnd calculates a sparse representation coefficient alpha using formula (1) and then based thereonThe learning dictionary DhAnd calculating a high-frequency prior image block h by using a formula (2)iAll the high-frequency prior image blocks hiRecombining on HR grid, averaging the overlapped area to obtain high-frequency prior image H
Figure BDA0002196961560000041
hi＝Dhα (2)
Wherein
Figure BDA0002196961560000042
The parameter beta is used for controlling the weight between the low-resolution image block and the high-frequency prior image block estimation value matched with the adjacent image block, Q is a gradient operator, lambda is a regularization coefficient, U is used for extracting the overlapping area of the currently calculated high-frequency prior image block and the calculated high-frequency prior image block, and y represents the value of the calculated high-frequency prior image block in the current overlapping area;
5) and finally, superposing the downsampled high-resolution prior image and the high-resolution initial interpolation image R to generate a high-resolution iterative initial image.
After the registration parameters are acquired, the information of each frame of the low-resolution image sequence is fused into the high-resolution initial image, and after the high-resolution initial interpolation image is acquired by using the non-uniform interpolation method, if the quality of the initial image can be further improved by using a learning-based method, the method is more beneficial to the subsequent image reconstruction. Based on the thought, learning is carried out by means of a high-resolution training image in a learning-based image sparse representation method sample library proposed by Yang and the like. Firstly, a sparse representation overcomplete dictionary is generated, then a sparse representation model is utilized, a high-frequency prior image corresponding to an initial interpolation image is obtained through the overcomplete dictionary obtained through learning, and then the interpolation image and the prior image are overlapped to generate a high-resolution initial image.
Learning of a sparse representation over-complete dictionary is a construction process for finding an optimal basis under a sparse representation. The overcomplete dictionary generated based on the learning method can not only meet the constraint condition of the uniqueness of the sparse representation, but also obtain finer dictionary representation. The basic principle of overcomplete sparse dictionary generation is described first below.
The learning dictionary is generated by matching a set of training samples X ═ X1,x2,…xnThe learning of (c) to obtain an overcomplete dictionary D, which can be summarized as the optimization problem as follows:
here, α represents the sparse representation coefficient of sample X under D, χ is the regularization parameter, | ·| survival1For enhancing the sparsity of the solution.
Equation (4) is a non-convex problem when solving for both D and α. However, when one of these two quantities is fixed, it can be solved by converting it into a convex problem. Therefore, it is usually performed iteratively in two steps. The solving step of the overcomplete dictionary D is as follows:
initializing D as a Gaussian random matrix, and normalizing each column of the matrix;
fix D, update α by the above equation, i.e.:
Figure BDA0002196961560000052
fix α, update D by the above equation, i.e.:
Figure BDA0002196961560000061
and returning to the step 2 until convergence.
Acquisition of a learning-based high-frequency prior model requires simultaneous use of a learning dictionary DLAnd DhI.e. a dictionary describing the low-resolution image block and a dictionary of the high-resolution image block corresponding thereto. Since the representation of the test data on the two dictionaries is usually inconsistent, an effective local feature extraction and representation method is established and the height is establishedIsomorphic description among local features of a resolution image block is a problem that must be solved. Aiming at the problem, Yang and the like uniformly carry out joint sparse coding on the high-resolution image blocks and the low-resolution image blocks in a joint solving mode so as to ensure DLAnd DhWith isomorphic sparse representations in between.
Let TrI be Fh,GlIn which Fh＝{f1,f2,…fiIs the set of high resolution image blocks, Gl＝{g1,g2,…giIs FhA set of corresponding low resolution image blocks. To estimate the corresponding learning dictionary D by using the high and low resolution image blocksLAnd DhAnd to have the same sparse representation, this can be achieved by solving the following equation.
Figure BDA0002196961560000062
Here, N and M represent the dimensions in the form of high and low resolution image block vectors, respectively. 1/N and 1/M are used to balance the effect of different sizes of high and low resolution image blocks on the overall representation frame.
Formula (5) can be abbreviated as follows:
Figure BDA0002196961560000063
here, the first and second liquid crystal display panels are,
Figure BDA0002196961560000064
for this equation, the solution can be obtained by following the K-SVD method, and the detailed description is omitted here.
In training the dictionary, the set TrI of high and low resolution image block pairs is acquired. In the Yang method, a low-resolution image block g in a training setiBy high resolution image blocks fiAnd (4) obtaining the down-sampling operation. This approach is simple and easy to implement, but adds to the blur in the actual imaging processDegradation factors such as affine transformation effects are not considered. The use of such a training set can negatively impact the effectiveness of acquiring high frequency prior images. Therefore, when the low-resolution image blocks are generated, the high-resolution image blocks are subjected to degradation processing based on the degradation model to generate the corresponding low-resolution image blocks, so that the dictionary learning result is more in line with the actual requirement. Furthermore, to extract feature information from the low-resolution image blocks more efficiently, the high-frequency feature information of the low-resolution image blocks is extracted using first and second order gradient operators Q of the image, the gradient operators being as follows.
Q1＝[-1,0,1],Q2＝Q1 T
Q3＝[1,0,-2,0,1],Q4＝Q3 T
After the learning-based sparse representation dictionary D is created, a high-frequency prior image H can be obtained according to an initial image R generated by non-uniform interpolation. Partitioning an input image R into R1,r2…riWherein r isiRepresenting the ith high-resolution initial interpolation image block, i being a natural number, overlapping blocks with certain pixels, and performing image segmentation on each image blockiUsing a dictionary DlThe sparse representation coefficient α is estimated according to equation (6), and then the dictionary D is usedhIt is possible to estimate the high frequency prior image block hi。
Figure BDA0002196961560000071
hi＝Dhα (2)
Equations (6) and (2) describe the basic flow of generating high-frequency prior image blocks using sparse representation dictionaries. However, in practical application, a problem must be noted: because the method only processes each low-resolution image block independently and does not consider the matching problem between the estimation result of the corresponding high-frequency prior image block and the estimated image blocks around the high-frequency prior image block, the situation that the obtained high-frequency prior image block is not the best matching block may exist. Therefore, the above formula is improved to the following form:
here, U is used to extract the overlapping area of the currently estimated high frequency a priori image block and the aforementioned estimated image block, and y represents the value of the estimated image block in the current overlapping area. The above formula can be summarized in the following form:
here, the first and second liquid crystal display panels are,
Figure BDA0002196961560000074
wherein λ is a regularization coefficient, and the parameter β is used to control the weight between the low-resolution image block and the estimated value of the high-frequency prior image block matched with the adjacent image block.
Through the above description of the embodiments, it is obvious for those skilled in the art that the embodiments may be implemented by software plus a necessary general hardware platform, and may also be implemented by hardware. With this understanding in mind, portions of the above-described technical solutions that substantially or otherwise contribute to the prior art may be embodied in the form of a software product that can be stored on a computer readable and writable medium, such as a usb-disk, a removable hard disk, a ROM, a RAM, a magnetic or optical disk, or the like. Including instructions for causing a computing device (e.g., a personal computer, server, or network device, etc.) to perform the methods described in the method embodiments or portions of the method embodiments above.
The foregoing is a more detailed description of the invention in connection with specific/preferred embodiments and is not intended to limit the practice of the invention to those descriptions. It will be apparent to those skilled in the art that the present invention may be practiced without these specific details.

Claims (4)

1. An iterative initial image generation method based on learning and sparse representation is characterized by comprising the following steps:
1) obtaining a set TrI ═ F of high and low resolution image block pairsh,GlIn which Fh＝{f1,f2,…fiIs the set of high resolution image blocks, Gl＝{g1,g2,…giIs FhSet of corresponding low resolution image blocks, fiFor the ith block of high resolution image block, giThe image block is the ith block and the first resolution image block, and i is a natural number;
2) computing a corresponding learning dictionary D using high resolution image blocks and low resolution image blockshAnd DlAnd have them the same sparse representation;
3) on the basis of the low-resolution image block, a high-resolution initial interpolation image R is obtained through a non-uniform interpolation method, and the high-resolution initial interpolation image R is divided into R1,r2…riSo that a certain pixel overlap is preserved from block to block, where riRepresenting the ith high-resolution initial interpolation image block, wherein i is a natural number;
4) based on the learning dictionary DlAnd calculating a sparse representation coefficient alpha by using formula (1), and then based on the learning dictionary DhAnd calculating a high-frequency prior image block h by using a formula (2)iAll the high-frequency prior image blocks hiRecombining on a high-resolution HR grid, carrying out average processing on an overlapped area to obtain a high-frequency prior image H,
Figure FDA0002196961550000011
hi＝Dhα (2)
wherein
Figure FDA0002196961550000012
The parameter beta is used for controlling the weight between the low-resolution image block and the high-frequency prior image block estimated value matched with the adjacent image block, Q is a gradient operator, and lambda is a regularization systemThe number of the high-frequency prior image blocks is U, the overlapping area of the current calculated high-frequency prior image blocks and the calculated high-frequency prior image blocks is extracted, and y represents the value of the calculated high-frequency prior image blocks in the current overlapping area;
5) and finally, superposing the downsampled high-resolution prior image and the high-resolution initial interpolation image R to generate a high-resolution iterative initial image.
2. The iterative initial image generation method according to claim 1, wherein the step 2 specifically includes: obtaining a learning dictionary D with the same sparse representation by solving equation (3)hAnd Dl
Figure FDA0002196961550000021
WhereinAlpha represents the sparse representation coefficient of the sample X under D, chi is the regularization coefficient, N and M respectively represent the dimensionality of the high-resolution image block vector form and the dimensionality of the low-resolution image block vector form, and 1/N and 1/M are used for balancing the influence of different sizes of the high-resolution image block and the low-resolution image block on the whole representation frame.
3. The iterative initial image generation method of claim 1, wherein the gradient operator Q is of the following specific form:
Q1＝[-1,0,1],Q2＝Q1 T
Q3＝[1,0,-2,0,1],Q4＝Q3 T
wherein Q1，Q2，Q3And Q4Respectively representing a first order gradient operator, a second order gradient operator, a third order gradient operator and a fourth order gradient operator.
4. A computer-readable medium, on which a computer program is stored, characterized in that the program, when executed, carries out the steps of the method according to any one of claims 1-3.
CN201910845539.XA 2019-09-10 2019-09-10 Iterative initial image generation method based on learning and sparse representation Active CN110648392B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910845539.XA CN110648392B (en) 2019-09-10 2019-09-10 Iterative initial image generation method based on learning and sparse representation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910845539.XA CN110648392B (en) 2019-09-10 2019-09-10 Iterative initial image generation method based on learning and sparse representation

Publications (2)

Publication Number Publication Date
CN110648392A true CN110648392A (en) 2020-01-03
CN110648392B CN110648392B (en) 2022-11-22

Family

ID=69010414

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910845539.XA Active CN110648392B (en) 2019-09-10 2019-09-10 Iterative initial image generation method based on learning and sparse representation

Country Status (1)

Country Link
CN (1) CN110648392B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105550988A (en) * 2015-12-07 2016-05-04 天津大学 Super-resolution reconstruction algorithm based on improved neighborhood embedding and structure self-similarity
CN105844590A (en) * 2016-03-23 2016-08-10 武汉理工大学 Image super-resolution reconstruction method and system based on sparse representation
US20170293825A1 (en) * 2016-04-08 2017-10-12 Wuhan University Method and system for reconstructing super-resolution image

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105550988A (en) * 2015-12-07 2016-05-04 天津大学 Super-resolution reconstruction algorithm based on improved neighborhood embedding and structure self-similarity
CN105844590A (en) * 2016-03-23 2016-08-10 武汉理工大学 Image super-resolution reconstruction method and system based on sparse representation
US20170293825A1 (en) * 2016-04-08 2017-10-12 Wuhan University Method and system for reconstructing super-resolution image

Also Published As

Publication number Publication date
CN110648392B (en) 2022-11-22

Similar Documents

Publication Publication Date Title
Wang et al. Edge-directed single-image super-resolution via adaptive gradient magnitude self-interpolation
US9652830B2 (en) Method and apparatus for performing hierarchical super-resolution of an input image
CN110210524B (en) Training method of image enhancement model, image enhancement method and device
Gong et al. Weighted mean curvature
CN103136727B (en) Based on the super resolution image reconstruction method of gradient consistency and anisotropy regularization
Li et al. FilterNet: Adaptive information filtering network for accurate and fast image super-resolution
CN106127688B (en) A kind of super-resolution image reconstruction method and its system
CN106952228A (en) The super resolution ratio reconstruction method of single image based on the non local self-similarity of image
Kudo et al. Virtual thin slice: 3D conditional GAN-based super-resolution for CT slice interval
CN110675318B (en) Sparse representation image super-resolution reconstruction method based on main structure separation
Laghrib et al. An edge preserving high-order PDE for multiframe image super-resolution
CN111724307B (en) Image super-resolution reconstruction method based on maximum posterior probability and non-local low-rank prior, terminal and readable storage medium
CN114494015A (en) Image reconstruction method based on blind super-resolution network
CN109741258B (en) Image super-resolution method based on reconstruction
CN113379647B (en) Multi-feature image restoration method for optimizing PSF estimation
Xiao et al. Detail enhancement of image super-resolution based on detail synthesis
CN112837220B (en) Method for improving resolution of infrared image and application thereof
Rashid et al. Single MR image super-resolution using generative adversarial network
Fang et al. Learning explicit smoothing kernels for joint image filtering
Singh et al. Single image super-resolution using adaptive domain transformation
CN110648392B (en) Iterative initial image generation method based on learning and sparse representation
Zefreh et al. Super-resolution of license plate images using algebraic reconstruction technique
Yang et al. Example-based image super-resolution via blur kernel estimation and variational reconstruction
Tuli et al. Structure preserving loss function for single image super resolution
CN115953299A (en) Infrared remote sensing image super-resolution method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant