CN102509281A - Double-image planar motion blur eliminating method based on transparency - Google Patents

Abstract

The invention discloses a method for eliminating planar motion blur by using two ordinary images. The method comprises the following specific steps of: 1, computing a foreground image, a background image and transparency; 2, modeling motion within exposure time; 3, establishing a target function; 4, registering two input images to obtain initial motion estimation; 5, evaluating a fuzzy kernel to minimize a target function in the step 3; 6, estimating a binary transparency image by using a Markovian random field; 7, re-estimating the fuzzy kernel to obtain a fuzzy kernel energy function; 8, modifying a Richardson-Lucy algorithm, and combining the modified Richardson-Lucy algorithm with the motion modeling in the step 2 to obtain a clear motion object foreground image; and 9, combining a deblurred foreground with a deblurred background by using deblurred transparency to obtain a deblurred image. In the obtained deblurred image, a boundary is kept clear, and a ''ringing'' effect is suppressed simultaneously.

Description

Remove the plane motion blur method based on the dual image of transparency

Technical field:

The present invention relates to technical field of image processing, relate in particular to a kind of plane motion blur method that is applicable to, specifically a kind of dual image based on transparency remove the plane motion blur method.The present invention is applicable to that the relevant image of two frames goes plane motion fuzzy.

Background technology:

Because in obtaining the process of image information, relative motion has appearred in camera (camera or video camera) and object, exists in the at this moment resulting image and blurs to a certain degree.In order to obtain distinct image, people have proposed the image deblurring algorithm.Image restoration technology in recent years becomes the focus of domestic and international image processing field research, has been applied to many aspects such as astronomical sight, remote sensing, military affairs, medical image, biological study, cracking of cases, traffic, industrial vision, video restoration.The recovery of image can be regarded the problem of a deconvolution as, and the recovery that it belongs to a type of indirect problem with pathosis, particularly overall inconsistent blurred picture in the mathematical problem is a challenging difficult problem especially.Although existing many at present comparatively ripe image deblurring methods still have many technical barriers not solve, such as asking for of fuzzy core, the research of problems such as the modeling of irregular ambiguity and recovery is still in the primary research stage.Can be divided into by fuzzy core: space invariance fuzzy core (the fuzzy and object overall situation translation that the camera translation causes etc.) and spatial variations fuzzy core (object different piece motion vector is inconsistent).The space invariance fuzzy core can be tried to achieve preferably at present.

To the image restoration of spatial variations fuzzy core, mainly contain three trend at present: 1, use ancillary hardware equipment: add sensor record camera motion path and the camera that uses the time shutter able to programme.2, use multiple image.3, man-machine interactively is by auxiliary some movable information that provides of manual work.But to common blurred picture (video), existing algorithm is not effective especially.

Summary of the invention:

What the present invention will solve is the problems referred to above that prior art exists, and aims to provide a kind of two width of cloth normal image of utilizing and removes the plane motion blur method.

The technical scheme that addresses the above problem employing is: remove the plane motion blur method based on the dual image of transparency, it is characterized in that comprising following concrete steps:

1) calculates foreground image, background image and transparency;

2) modeling is carried out in the motion in the time shutter; Set up the relation between existing blurred picture and the picture rich in detail to be asked: with the anglec of rotation five equilibrium in the time shutter (20 five equilibriums are enough); Blurred picture equals the superposed average of the picture rich in detail in each time slice, the problem of asking motor nuclei is converted into asks translation and the time corresponding weight in each timeslice;

3) use IRLS (being the heavy weighted least-squares method of iteration) to set up objective function;

4) two width of cloth input pictures are carried out registration, obtain initial estimation;

5) ask fuzzy core to make the objective function in the step 3 minimum, suppose that other amounts are known, ask weight and global translation amount in the anglec of rotation and each timeslice respectively;

6) use markov random file to estimate two-value transparency image;

7) the two-value transparency image feedback that obtains of step 6) reappraises fuzzy core, obtains the fuzzy core energy function again;

8) revising the Richardson-Lucy algorithm makes it and step 2) motion modeling combine, obtain motion object foreground image clearly;

9) prospect after the deblurring and background are used transparency after the deblurring to combine to obtain the image after the deblurring.

Dual image based on transparency of the present invention is removed the plane motion blur method, and the de-blurred image that adopts this method to obtain keeps sharpness of border, suppresses " ring " effect simultaneously.Plane motion can comprise rotatablely moving and adds translation motion.Through the image degradation process is carried out modeling, use IRLS (being the heavy weighted least-squares method of iteration) to come the establishing target function, last, revise the Richardson-Lucy algorithm and obtain motion object foreground image clearly.This method is applicable to that the relevant image of two frames goes plane motion fuzzy.

Description of drawings:

Below in conjunction with accompanying drawing and embodiment the present invention is described further.

The dual image that Fig. 1 is based on transparency is removed the process flow diagram of plane motion blur method.

Embodiment:

With reference to accompanying drawing, the dual image based on transparency of the present invention is removed the plane motion blur method, carries out according to the following steps:

1. ask transparency

Use the matting method to obtain the transparency of foreground image, background image and input picture.

2. modeling is carried out in image degradation

Set up the relation between fuzzy transparency image and the corresponding Clear & Transparent degree image: fuzzy transparency image equals the summation that the interior Clear & Transparent degree image of each time slice multiply by time weighting.Remember that original fuzzy transparency image is B, f is a Clear & Transparent degree image to be asked, and N is the time slice sum, Δ θ _i, Δ x _i, Δ y _iBe respectively the interior anglec of rotation of timeslice i and the global displacement on the x/y direction, w _iFor timeslice i (being generally 20) at the shared proportion of total exposure time, Tr (x; F, Δ θ, Δ x _t, Δ y _t) be the corresponding picture rich in detail of timeslice i, then

$\hat{B}\left(y\right)=\underset{i=0}{\overset{N-1}{\mathrm{\Σ}}}\mathrm{Tr}(x;f,\mathrm{\Δ\θ},\mathrm{\Δ}{x}_i,\mathrm{\Δ}{y}_i)*w_i$

In the present invention, find the solution conveniently, by anglec of rotation cutting, make the anglec of rotation in each time slice identical total exposure time in order to optimize.Therefore, has only a rotation parameter Δ θ.

Describe for convenient, introduce motion and describe operator Q, Q={ (q _i, w _i) | i ∈ 0,1 ..., N-1}, wherein, q _i=(Δ θ, Δ x _i, Δ y _i).

To fuzzy transparency image B ₁/ B ₂, can be modeled as respectively:

$B_1\left(x\right)=f_1\left(x\right)\⊗K_1(x;Q_1)+n_1,$

Q ₁＝{(q _1i，w _1i)|i∈0，1，...，N-1}，

q _1i＝(Δθ ₁，Δx _1i，Δy _1i)，

$B_2\left(x\right)=f_2\left(x\right)\⊗K_2(x;Q_2)+n_2,$

Q ₂＝{(q _2i，w _2i)|i∈0，1，...，N-1}，

q _2i＝(Δθ ₂，Δx _2i，Δy _2i).

3. wherein, K ₁The fuzzy transparency image B of expression ₁The spatial variations fuzzy core, K ₂The fuzzy transparency image B of expression ₂The spatial variations fuzzy core.Use IRLS (the heavy weighted least-squares method of iteration) to set up objective function

With picture rich in detail f ₁, B ₁Rotation, translation make it and f ₂Overlap, obtain B ₁', correspondingly, fuzzy core K ₁The rotation translation is K ₁', then use IRLS (the heavy weighted least-squares method of iteration) to set up following objective function:

$E_D(k_2,k_1^{\′})=\mathrm{Wr}.*{\left|\right|B_1^{\′}\left(x\right)\⊗K_2(x;Q_2)-B_2\left(x\right)\⊗K_1^{\′}(x;Q_1^{\′})\left|\right|}^2,$

Wherein, Wr is a matrix; Matrix element

r is a residual error

$r=B_1^{\′}\left(x\right)\⊗K_2(x;Q_2){-B}_2\left(x\right)\⊗K_1^{\′}(x;Q_1^{\′})$

4. the initialization of fuzzy core

Two width of cloth input pictures are carried out registration, obtain total anglec of rotation, translation.The anglec of rotation, the translation initial value that each time slice is corresponding is that the total anglec of rotation, translation are divided by total timeslice sum.Time weighting is 1/N.

5. asking fuzzy core to make the objective function in the step 3 minimum below, is example in the hope of the fuzzy core of second two field picture, and the fuzzy core of the 1st frame in like manner can get.

Estimate the anglec of rotation: adopt direct search method (Nelder-Mead simplex) to find out optimum solution at interval [minrt, maxrt].

Estimated time the sheet weight: the loop iteration formula is following:

$W_2^{r+1}=W_2^r-{\mathrm{\λ}}_w\frac{\∂E_D(k_2,k_1^{\′})}{\∂W_2}$

$=W_2^r-{\mathrm{\λ}}_w{M}_1^{\′T}(W_2^r{M}_1^{\′}-W_1{M}_2),$ Each pixel in the matrix M is the dot product of the Wr in fuzzy transparency and the step 3.

Estimate translational movement: set up gaussian pyramid and successively find the solution.

6. estimate two-value transparency image

The deconvolution energy function is following:

$E\left(f_2\right)=\underset{x}{\mathrm{\Σ}}E\left(f_2\left(x\right)\right)$

$=\underset{x}{\mathrm{\Σ}}\{{(B_2\left(x\right)-\underset{i=0}{\overset{N-1}{\mathrm{\Σ}}}\mathrm{Tr}(x;f_2,q_{2i})*w_{2i})}^2$

$+{(B_1^{\′}\left(x\right)-\underset{i=0}{\overset{N-1}{\mathrm{\Σ}}}\mathrm{Tr}(x;f_2,q_{1i}^{\′})*w_{1i})}^2\}$

Adopt between each pixel of markov random file modeling to concern, the long-pending energy function minimization problem of hai roll is converted into found the solution the linear system problem of constraint.

7. the two-value transparency image feedback that obtains of step 6 is used to reappraise fuzzy core, obtains following fuzzy core energy function:

$E_d(k_1,k_2)=$

$\mathrm{Wr}.*{\left|\right|B_1^{\′}\left(x\right)\⊗K_2(x;Q_2)-B_2\left(x\right)\⊗K_1^{\′}(x;Q_1^{\′})\left|\right|}^2$

$+{\mathrm{\λ}}_1{\left|\right|B_2\left(x\right)-f_2\left(x\right)\⊗K_2(x;Q_2)\left|\right|}^2$

$+{\mathrm{\λ}}_2{\left|\right|B_1^{\′}\left(x\right)-f_2\left(x\right)\⊗K_1^{\′}(x;Q_1^{\′})\left|\right|}^2$

8. revise the Richardson-Lucy algorithm and make it to combine, obtain motion object foreground image clearly with our motion modeling

$P\left(I_{B_y}\right|I_{f_x})=\left\{\begin{array}{cc}{w}_i& \mathrm{ify}=H_{i}x\\ 0& \mathrm{otherwise}\end{array}\right.$

Wherein, IB is an ambiguous prospect,

be clear prospect to be asked.

As follows iterative:

$I_{f_x}^{t+1}=I_{f_x}^t\×\underset{i=0}{\overset{N-1}{\mathrm{\Σ}}}{w}_i{E}^t\left(H_i^{-1}x\right)$

Wherein, $E^t\left(x\right)=\frac{I_{B_x}}{{\mathrm{\Σ}}_{i=0}^{N-1}{w}_i{I}_{f_{\left(H_{i}x\right)}}^t}.$

9. the transparency after prospect after the deblurring and the background use deblurring is combined and obtain picture rich in detail.

What should be understood that is: the foregoing description is just to explanation of the present invention, rather than limitation of the present invention, and any innovation and creation that do not exceed in the connotation scope of the present invention all fall within protection scope of the present invention.

Claims (10)

1. remove the plane motion blur method based on the dual image of transparency, it is characterized in that comprising following concrete steps:

1) calculates foreground image, background image and transparency;

2) modeling is carried out in the motion in the time shutter; Set up the relation between existing blurred picture and the picture rich in detail to be asked: with the anglec of rotation five equilibrium in the time shutter; Blurred picture equals the superposed average of the picture rich in detail in each time slice, the problem of asking motor nuclei is converted into asks translation and the time corresponding weight in each timeslice;

3) use IRLS to set up objective function;

4) two width of cloth input pictures are carried out registration, obtain initial estimation;

5) ask fuzzy core to make the objective function in the step 3 minimum, suppose that other amounts are known, ask weight and global translation amount in the anglec of rotation and each timeslice respectively;

6) use markov random file to estimate two-value transparency image;

7) the two-value transparency image feedback that obtains of step 6) reappraises fuzzy core, obtains the fuzzy core energy function again;

8) revising the Richardson-Lucy algorithm makes it and step 2) motion modeling combine, obtain motion object foreground image clearly;

9) prospect after the deblurring and background are used transparency after the deblurring to combine to obtain the image after the deblurring.

2. the dual image based on transparency as claimed in claim 1 is removed the plane motion blur method, it is characterized in that described step 1) use matting method obtains the transparency of foreground image, background image and input picture.

3. the dual image based on transparency as claimed in claim 2 is removed the plane motion blur method, it is characterized in that step 2) adopt following method to set up the relation between fuzzy transparency image and the corresponding Clear & Transparent degree image:

Remember that original fuzzy transparency image is B, f is a Clear & Transparent degree image to be asked, and N is the time slice sum, Δ θ _i, Δ x _i, Δ y _iBe respectively the global displacement on the anglec of rotation, x and the y direction in the timeslice i, w _iFor timeslice i at the shared proportion of total exposure time, Tr (x; F, Δ θ, Δ x _t, Δ y _t) be the corresponding picture rich in detail of timeslice i, then

By anglec of rotation cutting, make the anglec of rotation in each time slice identical total exposure time, therefore, have only a rotation parameter Δ θ;

Introduce motion simultaneously and describe operator Q, Q={ (q _i, w _i) | i ∈ 0,1 ..., N-1}, wherein, q _i=(Δ θ, Δ x _i, Δ y _i);

To fuzzy transparency image B ₁And B ₂, be modeled as respectively:

Q ₁＝{(q _1i，w _1i)|i∈0，1，...，N-1}，

q _1i＝(Δθ ₁，Δx _1i，Δy _1i)，

Q ₂＝{(q _2i，w _2i)|i∈0，1，...，N-1}，

q _2i＝(Δθ ₂，Δx _2i，Δy _2i).

Wherein, K ₁The fuzzy transparency image B of expression ₁The spatial variations fuzzy core, K ₂The fuzzy transparency image B of expression ₂The spatial variations fuzzy core.

4. the dual image based on transparency as claimed in claim 3 is removed the plane motion blur method, it is characterized in that step 3) use IRLS sets up objective function according to the following steps:

With picture rich in detail f ₁, B ₁Rotation, translation make it and f ₂Overlap, obtain B ₁', correspondingly, fuzzy core K ₁The rotation translation is K ₁', then use IRLS to set up following objective function:

Wherein, Wr is a matrix; Matrix element

r is a residual error

。

5. the dual image based on transparency as claimed in claim 4 is removed the plane motion blur method, it is characterized in that step 4) undertaken by following method:

Two width of cloth input pictures are carried out registration, obtain total anglec of rotation and translation, the anglec of rotation, the translation initial value that each time slice is corresponding be the total anglec of rotation, translation divided by total timeslice sum, time weighting is 1/N.

6. the dual image based on transparency as claimed in claim 5 is removed the plane motion blur method, it is characterized in that in the step 5) obtaining by following method the fuzzy core of second two field picture, and the fuzzy core of the 1st two field picture obtains in an identical manner:

5.1) estimate the anglec of rotation: adopt the direct search method to find out optimum solution at interval [maxrt, maxrt];

5.2) estimated time the sheet weight: the loop iteration formula is following:

Each pixel in

matrix M is the dot product of the Wr in fuzzy transparency and the step 3);

5.3) estimate translational movement: set up gaussian pyramid and successively find the solution.

7. the dual image based on transparency as claimed in claim 6 is removed the plane motion blur method, it is characterized in that step 6) adopts following method to estimate two-value transparency image:

The deconvolution energy function is following:

Adopt between each pixel of markov random file modeling to concern, the long-pending energy function minimization problem of hai roll is converted into found the solution the linear system problem of constraint.

8. the dual image based on transparency as claimed in claim 7 is removed the plane motion blur method, it is characterized in that step 7) reappraises fuzzy core with the two-value transparency image feedback that step 6) obtains, and obtains following fuzzy core energy function:

。

9. the dual image based on transparency as claimed in claim 8 is removed the plane motion blur method; It is characterized in that step 8) adopts following method to revise the Richardson-Lucy algorithm and makes it step 2) motion modeling combine, obtain motion object foreground image clearly:

Wherein, IB is an ambiguous prospect,

be clear prospect to be asked;

As follows iterative:

Wherein, .

10. remove the plane motion blur method like any one described dual image of claim 1-9, it is characterized in that step 2 based on transparency) in the anglec of rotation in the time shutter is divided into smaller or equal to 20 five equilibriums.

Images