CN108564544A - Image Blind deblurring based on edge perception combines sparse optimization method - Google Patents

Abstract

The invention discloses a kind of Image Blind deblurrings based on edge perception to combine sparse optimization method, and opposite total variation regular terms is introduced on the basis of image L0 sparse priors, carries out blind deblurring to natural image, carries out in accordance with the following steps：Blurred picture y, fuzzy core k are inputted, clear image to be solved is x；It is blurred picture y, the parameters such as initiation parameter λ, σ to initialize image x to be solved；The method for intersecting estimation using fuzzy core and image solves fuzzy core k from thick to thin；Non-blind deblurring is carried out to blurred picture y according to the fuzzy core k that step 3) is finally calculated, finds out clearly image x^(L)；Last processing is done to clearly image, obtains final clear image.

Description

Image blind deblurring combined sparse optimization method based on edge perception

The technical field is as follows:

the invention relates to an image blind deblurring combined sparse optimization method based on edge perception, and belongs to the technical field of image processing.

Background art:

the importance of information processing technology is recognized by the rapid development of science and technology, particularly the development of computer and multimedia technology, in modern society. Since images are widely and easily available as carriers of information transmission in the spread of video cameras and still cameras in ordinary households, it is becoming more and more important to ensure high definition of the obtained images simply and efficiently. However, in the daily shooting process, the obtained images are often blurred due to the imaging equipment, weather factors, processing and transmission modes and the like in the processes of acquisition, recording, processing and transmission. A sharp image can be estimated directly using degraded image information, but since many parameters are unknown during imaging, we need to estimate both the blur kernel of the image and the sharp digital image. This estimation of sharp images without knowledge of the blur kernel information is a blind restoration of the image. In many practical situations, the blur kernel is often unknown, where efficient restoration of a degraded blurred image is a challenging problem.

The theoretical basis of the image deblurring technology is as follows: performing modeling processing on the image according to the reason of image degradation, and restoring or reconstructing the degraded image into an original image, which can be specifically expressed as: where y denotes an observed blurred image (degraded image), k denotes a blur kernel (which may also be referred to as a blur function, a point spread function, a degradation factor, or the like), and x denotes a convolution operation, x is a sharp image (original image) to be restored, and n is additive noise (generally assumed to be gaussian noise), so the image deblurring problem is how to obtain a sharp image x from the blurred image y mathematically, which requires deconvolution. The blind deconvolution problem is a more challenging inversion of the pathology. Because the blur kernel is unknown in the blind deblurring problem, its algorithm generally employs separate alternating estimation steps for the blur kernel and the sharp image: 1) firstly, estimating an image blur kernel k, namely estimating the blur kernel of an image by using an initially recovered sharp image; 2) and estimating the clear image x, and performing non-blind estimation on the image by using the fuzzy core obtained by the previous estimation to obtain a deblurred image. The two processes are sequentially alternated, and finally, a clear image is obtained.

In the early stage of deblurring work, the success of blind deblurring of an image is to utilize the prior knowledge of the image and the edge detection for kernel estimation as efficiently as possible, provide a regularization term constraint solution space and solve a clear image in an iteration mode. The conventional method generally implements blind deblurring by using the L0 norm of the gradient of an image as a priori knowledge, and the method is good for most natural image effects. Recently, Pan et al achieved blind deblurring of text images by combining the L0 norm prior of image intensity and image gradient, which is derived from observing different properties of the text image, and based on this prior, yielded reliable intermediate results of kernel estimation. There is no need to detect a protruding edge. In the final image restoration step, artifacts (artifacts, noise) are removed and deblurred, further optimizing the natural image deblurring effect. The method basically achieves the currently known optimal result on the basis of the blind recovery problem of the text image, and can actually obtain a good deblurring effect on images of natural images and low-illumination scenes. According to the statistical result of the text image, the text image is mainly bicolor (two gradient values), the gradient statistics of the heavy tail distribution of the natural image is not met, if the image is blurred, other gradient values can appear, and the Pan method utilizes the priori knowledge of the text image to carry out regularization constraint to obtain a clear image result. However, this statistical characteristic does not exist in the natural image because the luminance values of the natural image are rich and varied, and the statistical characteristic (only two values) of the text image cannot be used as a constraint condition. There are of course many other ways of blind deblurring an image, but the computational complexity is relatively high. Another recent work by Pan has attracted considerable attention: the combination of the image gradient L0 norm and the image dark channel L0 norm realizes the blind deblurring effect with stronger robustness and higher accuracy, and the work brings a new thinking direction for researching the blind deblurring problem: whether the problem of image deblurring can be solved by using a method for processing other image problems or not and whether a better image prior representation method exists or not.

The information disclosed in this background section is only for enhancement of understanding of the general background of the invention and should not be taken as an acknowledgement or any form of suggestion that this information forms the prior art already known to a person skilled in the art.

The invention content is as follows:

the invention aims to provide an image blind deblurring combination sparse optimization method based on edge perception, which improves the results of an algorithm on fuzzy kernel and clear image estimation, thereby overcoming the defects in the prior art.

In order to achieve the purpose, the scheme provided by the invention is as follows: a relative total variation regularization term is introduced on the basis of sparse prior of an image L0 to carry out blind deblurring on a natural image, and the method better retains the edge information of the image, thereby obtaining a good result. According to the definition of the relative total variation, the value of the relative total variation item in the blurred image is larger than that of the sharp image, and the image is constrained by using the value as a priori when blind deblurring is carried out, specifically, the method comprises the following steps:

s1, inputting a fuzzy image y, wherein a fuzzy kernel is k, and a clear image to be solved is x;

s2, initializing an image x to be solved as a blurred image y, and initializing parameters such as lambda, sigma and the like;

s3, solving a fuzzy kernel k from coarse to fine by using a fuzzy kernel and image cross estimation method;

s4, performing non-blind deblurring on the blurred image y according to the blur kernel k finally calculated in the step 3), and solving a clear image x^(L)；

And S5, performing final processing on the clear image to obtain a final clear image.

The technical scheme further defined in the aspect is as follows:

preferably, in the above technical solution, step3 specifically is:

(ii) the coarse estimate of the blur kernel is k⁽⁰⁾(ii) a Initializing the number of times of non-blind deblurring as 0;

secondly, according to the fuzzy kernel k roughly estimated above, a clear intermediate image is solved by using the following formula 1:

wherein x^(l)The clear image to be solved obtained after the lambda iteration is solved is shown,representing the values of the minimum values x, u and g of the objective function, u and g being auxiliary variables introduced manually and corresponding to the structural image S and the image gradient respectivelyx^(l-1)Representing the sharp image to be solved, k, after the 1 st iterative solution^(l-1)The fuzzy kernel after the l-1 iteration solution is shown, y represents a fuzzy image, represents convolution operation,which represents the square of the 2-norm,representing an image x^(l-1)Gradient of (1) | · | | non-conducting phosphor₀Denotes a norm of 0, S^(l-1)Denotes x^(l-1)in the structural image of (1), λ and σ are regular term coefficients, μ and β are coefficients of an introduced variable, β is initialized to 2 λ σ and μ to 2 λ,representing an image x^(l-1)Relative total variation regularization term of, wherein D_h(p)、D_v(p) is the window total variation, L_h(p)、L_v(p) is the inherent variation of the window, p is a pixel point on the image, h and v respectively represent the horizontal and vertical directions, epsilon is a very small positive number, and the denominator L is avoided_h(p) and L_v(p) is 0;

③, clear image x estimated according to the second step^(l)The blur kernel can be solved directly with the following equation 2:

k^(l)the blur kernel obtained after the first solution is shown,denotes the value, x, taken by the objective function to take the minimum value k^(l)Clear image after the first solution of representation, k^(l-1)The l-1 solved blur kernel is shown, y is the blurred image, x is the convolution operation,represents the square of the 2 norm, γ is the regularization term parameter;

updating lambda value, judging whether L is less than maximum circulation times L, updating L to L +1, repeating (c) and (c) to obtain final fuzzy kernel k to k^(L)。

Preferably, in the above technical solution, the L0+ RTV regularization model constructed in step 3;

the regularization model for significant edge prior and relative total variation based on the image gradient L0 norm is as follows:

in the formula 3, the first and second phases,

let u be S or S,when μ and β tend to infinity, equation 3 changes to equation 4:

let the values of u, g approach zero, equation 5 is calculated:

a clear image x can be obtained by the following equation 6:

in the formula: f (-) and F^-1(. cndot.) represents a fourier transform and an inverse fourier transform,is a complex conjugate form of the fourier transform, respectively representing differential operators in the horizontal direction and the vertical direction;

setting a clear image x, and calculating values of u and g through formulas 7, 8 and 9 respectively:

then:

calculating u and g by the above formula, and combining formula 6 to obtain a clear intermediate image x in the first iteration^(l)。

Preferably, in the above technical solution, the added RTV regular term calculates formula 10 of the structural image:

wherein ,is the value of the minimum value S taken by the objective function, S is the structural image, I is the input image,summing the difference values of pixel points of output and input images to ensure the accuracy of the output structural image, η is a regularization parameter and a regularization termIs a relative total variation, D_x(p)、D_y(p) is the window total variation, concretelyL_x(p)、L_y(p) is a window inherent variation which has no relation with the gradient direction, and specifically shows g_p,qis defined according to spatial correlation, wherein α controls the window scale, p is the central pixel point of the variation region R (p), q is any point in the variation region R (p), epsilon_SIs a very small positive number.

The specific calculation for the relative total variation subentry is as follows equation 11:

where equations 12 and 13 are:

U_xpindicating that each pixel point combines the gradient information, W, in the field of the point_xpIndicating that it is only relevant to the gradient at that point; similarly, the expression in the y direction is in the form of equations 14 and 15 as follows:

by splitting the relative total variation terms, equation 10 can be written in the form of a matrix 16 as follows:

wherein ,C_x、C_yIs a Toplitz matrix obtained by approximating a discrete gradient operator by forward scoring_S、v_IVector representation of S and I, respectively, U_x、U_y、W_x、W_yIs a diagonal matrix with values on the diagonal of U_x[i,i]＝u_xi，U_y[i,i]＝u_yi，W_x[i,i]＝w_xi，W_y[i,i]＝w_yi(ii) a The euler-lagrange equation is used to calculate equation 12 and convert the minimization problem into linear problem equation 17:

wherein E is a matrix of units and E is a matrix of units,is based on a structure vectora calculated weight matrix, (E + η L^t) Is a non-negatively symmetric laplace matrix;

the structural image S can be obtained by a plurality of iterative calculations 12, 13, 14, 15, 17.

Compared with the prior art, the invention has the following beneficial effects:

first, the method combines L0 sparseness and relative total variation based on edge perception, better retains edge detail information of the image, and more accurately estimates the fuzzy kernel, thereby improving the clear image effect of estimation.

Secondly, a large number of experiments show that compared with the original method, the deblurring effect of the method is good on the natural image, and the effect on the Levin data set is basically higher than that of the existing known blind deblurring method.

Description of the drawings:

FIG. 1 is a flow chart of an implementation of the present invention;

FIG. 2 is a blurred image to be processed;

FIG. 3a is the deblurring result of FIG. 2 using the deblurring method disclosed in the article "mage smoothening via L0 GradientMinification" by Li Xu, Cewu Lu et al;

FIG. 3b is the Deblurring result of FIG. 2 using the Deblurring method disclosed in the article "Deblurring Text Images via L0-regulated Intensity and GradientPrior" by Jinshan Pany, Zhe Hu et al;

FIG. 3c is the deblurring result of FIG. 2 using the method of the present invention;

FIG. 4a blurred text image;

FIG. 4b is the deblurring result of FIG. 4a using the method of Li Xu, Cewu Lu et al;

FIG. 4c is the deblurring of FIG. 4a using the method of Jinshan Pan, Zhe Hu et al;

FIG. 4d is the deblurring result of FIG. 4a using the method of the present invention.

The specific implementation mode is as follows:

the following detailed description of specific embodiments of the invention is provided, but it should be understood that the scope of the invention is not limited to the specific embodiments.

Throughout the specification and claims, unless explicitly stated otherwise, the word "comprise", or variations such as "comprises" or "comprising", will be understood to imply the inclusion of a stated element or component but not the exclusion of any other element or component.

Detailed Description

The experiments will be described in further detail below with reference to the accompanying drawings.

Referring to fig. 1, the specific steps of the present invention are as follows:

step 1: inputting a blurred image:

inputting a fuzzy image y, wherein a fuzzy kernel is k, and a clear image to be solved is x;

step 2: initializing parameters:

initializing an image x to be solved as an original blurred image y, and initializing parameters such as lambda, sigma and the like;

and step 3: calculating a fuzzy kernel:

solving the fuzzy kernel from coarse to fine by using a fuzzy kernel and image cross estimation method:

firstly, initializing a fuzzy kernel, roughly estimating to be k⁽⁰⁾Initializing the number of times of non-blind deblurring to be 0;

② initializing beta-2 lambda sigma and initializing mu_max、β_maxThe value of the auxiliary variable u is calculated using equation (8) based on the fuzzy kernel k roughly estimated above.

Wherein the structural image S, the specific calculation can be implemented by the following code:

step1, inputting an image I, parameters η and alpha;

step 2: initialization t is 0, S⁰＝I；

Step 3: the weights U are calculated using equations (12), (13), (14), (15)_x、U_y、W_x、W_y；

Step 4: solving linear equation (17);

step 5: iterating step3, step4 as many times as required;

step 6: the structural image S is output.

③, the value of the auxiliary variable g is calculated by using the formula (9)

fourthly, calculating a clear image by using the formula (6)

x^(l)The clear image to be solved is obtained after the first lambda cycle solution, u and g are auxiliary variables corresponding to S and G respectivelyk^(l-1)Showing the blur kernel after the l-1 cycle solution, y showing the blurred image, F (-) and F^-1(. cndot.) represents a fourier transform and an inverse fourier transform,is a complex conjugate form of the fourier transform, representing the differential operators in the horizontal and vertical directions, respectively.

utilizing clear image x estimated from third step^(l)Solving a fuzzy kernel:

k^(l)the blur kernel obtained after the first solution is shown,denotes the value, x, taken by the objective function to take the minimum value k^(l)Clear image after the first solution of representation, k^(l-1)The l-1 solved blur kernel is shown, y is the blurred image, x is the convolution operation,represents the square of the 2 norm, γ is the regularization term parameter;

updating lambda value, judging whether L is greater than maximum circulation times L, updating L +1, updating image structure S, repeating the above steps to obtain final fuzzy kernel k^(L)；

And 4, step 4: calculating sharp images

According to the fuzzy kernel k iteratively calculated in the step 3), carrying out non-blind deblurring on the original fuzzy image y, and calculating a clear image x^(L)；

And 5: obtaining a sharp image

Performing some necessary processing on the image obtained in step4, for example, removing artificial artifacts to obtain

And finally, obtaining a clear image.

The deblurring effect of the present invention is further described below in conjunction with experimental conditions.

1. Conditions of the experiment

The experimental operation system is an Inter (R) Core (TM) i7CPU @3.4GHz 64-bit Windows operating system, and the used simulation software is MATLAB (R2014 a). The source of blurred images used in the experiment was a database of blurred natural images in the article "Understanding and evaluating blur reduction algorithms" published by a.levin, y.weiss, f.durand, w.t.freeman et al, as shown in fig. 2.

2. Content of the experiment

The blind deblurring process is performed on fig. 2 by using the present invention and three existing deblurring methods, and the result is shown in fig. 3.

FIG. 3a is the deblurring result of FIG. 2 using the deblurring method disclosed in the article "mage smoothening via L0 GradientMinimization" by Li Xu, Cewu Lu et al;

FIG. 3b is the Deblurring result of FIG. 2 using the Deblurring method disclosed in the article "Deblurring Text Images via L0-regulated Intensity and Gradient primer" by Jinshan Pany, Zhe Hu et al;

FIG. 3c is the deblurring result of FIG. 2 using the method of the present invention;

3. analysis of simulation results

In the simulation experiment, a peak signal-to-noise ratio (PSNR) index is adopted to evaluate an experiment result, wherein the PSNR is respectively defined as:

wherein: x represents the original sharp image and Z represents the recovered deblurred image. A larger peak signal-to-noise ratio indicates a better deblurring performance.

PSNR value of FIG. 3a is 32.13 dB;

the PSNR value of FIG. 3b is 31.66 dB;

the PSNR value of fig. 3c is 33.08 dB. The partial data is the result of the experiment of fig. 2 by three methods, fig. 2 is a specific fuzzy picture in a Levin data set, the Levin data set comprises eight different fuzzy kernels, and the experimental result of the table is the result of blind deblurring of the fuzzy picture corresponding to the eight fuzzy kernels respectively to the original clear image.

To better compare the deblurring performance of these three methods, we performed experiments on a standard set of blurred pictures on the Levin dataset, with the results shown in table 1.

Table 1 shows the fuzzy picture experiment results:

from the obtained data, it can be known that the PSNR value of the deblurring result of the method of the present invention is higher than that of the deblurring results of other methods, i.e. the present invention has better deblurring effect than the prior art.

In the experimental process, we find that the method of the present invention can achieve good results for deblurring the text image, and the result is shown in fig. 4 d. The source of the blurred text image is the database in the article "deblurring text Images via L0-regulated Intensity and Gradient printer" by jin shan Pan, Zhe Hu et al.

FIG. 4a blurred text image;

FIG. 4b is the deblurring result of FIG. 4(a) using the method of Li Xu, Cewu Lu et al;

FIG. 4c is the deblurring result of FIG. 4(a) using the method of Jinshan Pan, Zhe Hu et al;

FIG. 4d is the deblurring result of FIG. 4a using the method of the present invention.

The foregoing descriptions of specific exemplary embodiments of the present invention have been presented for purposes of illustration and description. It is not intended to limit the invention to the precise form disclosed, and obviously many modifications and variations are possible in light of the above teaching. The exemplary embodiments were chosen and described in order to explain certain principles of the invention and its practical application to enable one skilled in the art to make and use various exemplary embodiments of the invention and various alternatives and modifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the claims and their equivalents.

Claims (4)

1. The image blind deblurring combination sparse optimization method based on edge perception is characterized in that a relative total variation regularization term is introduced on the basis of image L0 sparse prior, blind deblurring is carried out on a natural image, and the method comprises the following steps:

s1, inputting a fuzzy image y, wherein a fuzzy kernel is k, and a clear image to be solved is x;

s2, initializing an image x to be solved as a blurred image y, and initializing parameters such as lambda, sigma and the like;

s3, solving a fuzzy kernel k from coarse to fine by using a fuzzy kernel and image cross estimation method;

s4, performing non-blind deblurring on the blurred image y according to the blur kernel k finally calculated in the step3, and solving a clear image x^(L)；

And S5, performing final processing on the clear image to obtain a final clear image.

2. The image blind deblurring combined sparse optimization method based on edge perception according to claim 1, wherein the step3 specifically comprises:

(ii) the coarse estimate of the blur kernel is k⁽⁰⁾(ii) a Initializing the number of times of non-blind deblurring as 0;

secondly, according to the fuzzy kernel k roughly estimated above, a clear intermediate image is solved by using the following formula 1:

wherein x^(l)The clear image to be solved obtained after the lambda-th iterative solution is shown,representing the values of the minimum values x, u and g of the objective function, u and g being auxiliary variables introduced manually and corresponding to the structural image S and the image gradient respectivelyx^(l-1)Representing the sharp image to be solved, k, after the 1 st iterative solution^(l-1)The fuzzy kernel after the l-1 iteration solution is shown, y represents a fuzzy image, represents convolution operation,which represents the square of the 2-norm,representing an image x^(l-1)Gradient of (1) | | | the non-calculation₀Represents a norm of 0，S^(l-1)Denotes x^(l-1)in the structural image of (1), λ and σ are regular term coefficients, μ and β are coefficients of an introduced variable, β is initialized to 2 λ σ and μ to 2 λ,representing an image x^(l-1)Relative total variation regularization term of, wherein D_h(p)、D_v(p) is the window total variation, L_h(p)、L_v(p) is the inherent variation of the window, p is a pixel point on the image, h and v respectively represent the horizontal and vertical directions, epsilon is a very small positive number, and the denominator L is avoided_h(p) and L_v(p) is 0;

③, clear image x estimated according to the second step^(l)The blur kernel can be solved directly with the following equation 2:

k^(l)the blur kernel obtained after the first solution is shown,denotes the value, x, taken by the objective function to take the minimum value k^(l)Clear image after the first solution of representation, k^(l-1)The l-1 solved blur kernel is shown, y is the blurred image, x is the convolution operation,represents the square of the 2 norm, γ is the regularization term parameter;

updating lambda value, judging whether L is less than maximum circulation times L, updating L to L +1, repeating (c) and (c) to obtain final fuzzy kernel k to k_(L)。l＝0。

3. The image blind deblurring combined sparse optimization method based on edge perception according to claim 1, characterized in that the L0+ RTV regularization model constructed in step 3;

the regularization model for significant edge prior and relative total variation based on the image gradient L0 norm is as follows:

in the formula 3, the first and second phases,

let u be S or S,when μ and β tend to infinity, equation 3 changes to equation 4:

let the values of u, g approach zero, equation 5 is calculated:

a clear image x can be obtained by the following equation 6:

in the formula: f (-) and F^-1(. cndot.) represents a fourier transform and an inverse fourier transform,is a complex conjugate form of the fourier transform, respectively representing differential operators in the horizontal direction and the vertical direction;

setting a clear image x, and calculating values of u and g through formulas 7, 8 and 9 respectively:

then:

calculating u and g by the above formula, and combining formula 6 to obtain a clear intermediate image x in the first iteration^(l)。

4. The image blind deblurring combined sparse optimization method based on edge perception according to claim 1, wherein the added RTV regularization term is used to calculate formula 10 of the structural image:

wherein ,is the value of the minimum value S taken by the objective function, S is the structural image, I is the input image,summing the difference values of pixel points of output and input images to ensure the accuracy of the output structural image, η is a regularization parameter and a regularization termIs a relative total variation, D_x(p)、D_y(p) is the window total variation, concretelyL_x(p)、L_y(p) is a window inherent variation which has no relation with the gradient direction, and specifically shows g_p,qis defined according to spatial correlation, wherein α controls the window scale, p is the central pixel point of the variation region R (p), q is any point in the variation region R (p), epsilon_SIs a very small positive number.

The specific calculation for the relative total variation subentry is as follows equation 11:

where equations 12 and 13 are:

U_xpindicating that each pixel point combines the gradient information, W, in the field of the point_xpIndicating that it is only relevant to the gradient at that point; similarly, the expression in the y direction is in the form of equations 14 and 15 as follows:

by splitting the relative total variation terms, the equation (10) can be written in the form of the following matrix 16:

wherein ,C_x、C_yIs a Toplitz matrix obtained by approximating a discrete gradient operator by forward scoring_S、v_IVector representation of S and I, respectively, U_x、U_y、W_x、W_yIs a diagonal matrix with values on the diagonal of U_x[i,i]＝u_xi，U_y[i,i]＝u_yi，W_x[i,i]＝w_xi，W_y[i,i]＝w_yi(ii) a The euler-lagrange equation is used to calculate equation 12 and convert the minimization problem into linear problem equation 17:

wherein E is a matrix of units and E is a matrix of units,is based on a structure vectora calculated weight matrix, (E + η L^t) Is a non-negatively symmetric laplace matrix;

the structural image S can be obtained by a plurality of iterative calculations 12, 13, 14, 15, 17.