CN102800078A - Non-local mean image detail restoration method - Google Patents

Abstract

The invention discloses a non-local mean image detail restoration method which mainly solves the problem of relatively great error of the detail part restoration in the existing sample-based non-local mean image restoration method. The method comprises the following steps: (1) determining a to-be-restored area omega and the boundary delta thereof for a to-be-restored image I; (2) finding a to-be-restored block with the highest priority on the boundary; (3) calculating the similarity between a sample block and the restoration block according to the correlation and similarity distance of pixels; (4) searching for m sample blocks the most similar to the restoration block, and taking the weighted mean of the m sample blocks as a filling block psi0 of the restoration block; and (5) updating the confidence item and the to-be-restored area, and repeating the steps (1) to (5) until all points in the to-be-restored area are restored. The method disclosed by the invention can be used for restoring the image-damaged area and removing the target and the text in image.

Description

Non-local mean image detail restorative procedure

Technical field

The invention belongs to technical field of image processing, relate to image repair, can be used for repairing the damaged zone of image, target removes the removal with the image Chinese version.

Background technology

The image repair technology is an important content in the image restoration research, and its purpose is the information of coming recovery automatically to lose according to the existing information of image, and it can be used for recovery, video text removal and the video error concealing etc. of old photo drop-out.Existing image repair method roughly can be divided into based on the restorative procedure of structure with based on two big types of the restorative procedures of texture.Wherein the restorative procedure based on structure all is a kind of restorative procedure based on PDE in essence; Propose by people such as Bertalmio the earliest; The restorative procedure that proposes by people such as Chan subsequently based on overall variation TV model, and the curvature Driven Diffusion CDD model restorative procedure that is produced by the inspiration of TV repairing model all belongs to the restorative procedure based on structure.These methods all are that the diffusion through information realizes, are only applicable to the damaged image repair of non-texture image and small scale.

In addition; The restorative procedure based on sample that people such as Criminisi propose is a kind of restorative procedure based on texture; This method used for reference the thought in the texture synthesis method seek sample block and the coupling duplicate; Made full use of simultaneously based on the diffusion way in the restorative procedure of structure and defined the priority of repairing piece; Make that being near the reparation piece in edge with more structural information has higher reparation priority, thereby when repairing texture information, structural information is also had certain maintenance.This method adopts single sample block directly to fill the area to be repaired; Owing to be difficult to make in the reality sample block and to be repaired to reach Optimum Matching; Therefore when filling to be repaired, can have certain error, along with the carrying out of repair process, this way can cause the accumulation of error.

Afterwards; Alexander Wong and JeffOrchar have proposed a kind of non-local mean based on sample and have repaired algorithm; Adopt the weighted mean of a plurality of sample block to synthesize the filling block that is used to fill the area to be repaired, improved defective to a certain extent based on the sample restorative procedure.But this method is owing to use an attenuation coefficient to calculate sample block and to be repaired similarity weights as the negative exponential function of constant; And the information that is comprised in different to be repaired is different; Do like this and will certainly cause the calculating of similarity weights not accurate enough; And then causing repairing the well detail textures in the connection layout picture of result, and above-mentioned existing method has error to the reparation of detail section in the image; The edge as a result and the texture that cause repairing are wrong, and then make the whole figure as a result nature that becomes.

Summary of the invention

The objective of the invention is to deficiency, proposed a kind of non-local mean image detail restorative procedure, make the detailed information among the image repair result more accurate, thereby improve repairing effect to above-mentioned prior art.

The technical thought that realizes the object of the invention is; On basis based on the non-local mean restorative procedure of sample, utilize the PPB model, smoothing parameter is revised; Constructed a new weights computing formula; The similarity computation rule of two image blocks of having laid equal stress on redetermination can be searched for similar, more accurately better to be repaired the result.

Implementation step comprises as follows:

(1), confirms the border δ of area to be repaired Ω and area to be repaired for the image I to be repaired of input;

(2) utilize following formula, calculate the priority P (p) that central point all on the δ of the border of area to be repaired are repaired piece:

P(p)＝C(p)·D(p)，

Wherein, D (p) is a data item, and C (p) is the degree of confidence item, the credibility of presentation video pixel, and C (p) is initialized as C (p)=0, p ∈ Ω, C (p)=1, p ∈ I-Ω;

(3) central point

with the highest reparation piece of priority

is the center; Choose size and be the neighborhood of the M * M region of search as this reparation piece, definition is that the piece

at center is a sample block with point

in should the zone;

(3.1) utilize following formula, the correlativity of the central point

of calculating reparation piece

and the central point

of sample block

:

$\mathrm{\ϵ}=\exp (-\frac{{(\hat{p}-\hat{q})}^2}{2{\mathrm{\σ}}^2})$

Wherein, σ ²For repairing piece

Variance;

(3.2) correlativity that obtains above the basis; Find out in the region of search all and be in the point of homogeneous region, as new region of search with the central point

of repairing piece

;

(4) Calculate the new repair block within the search area

and the sample block

similarity distance:

$d({\mathrm{\ψ}}_{\hat{p}},{\mathrm{\ψ}}_{\hat{q}})={\left|\right|{\mathrm{\ψ}}_{\hat{p}}-{\mathrm{\ψ}}_{\hat{q}}\left|\right|}_2^2$

Wherein,

is 2 norms;

(5) according to similarity distance

Obeying degree of freedom is card side's distribution x of n ²(n) characteristic, when n>=25, quantile

Choose m the most similar sample block that similarity distance is positioned at quantile β left side, as repairing piece

Set;

(6) according to similarity distance

Calculate quantile α ₀Difference with the similarity distance average:

Wherein, Be the value of its quantile,

Average for similarity distance;

(7), calculate the similarity weights of interior sample block

of set and reparation piece according to following formula:

$\mathrm{\ω}=\frac{1}{Z}\exp (-\frac{d({\mathrm{\ψ}}_{\hat{p}},{\mathrm{\ψ}}_{\hat{q}})}{4{\mathrm{\σ}}^2\mathrm{Nt}})$

Wherein, Z is a normalized parameter, σ ²For repairing piece

Variance, N is for repairing piece

The number of the point that middle pixel value is known;

(8) according to the similarity weights, with the weighted mean of the whole sample block in the set, as filling block Ψ ₀, and with this filling block to repairing piece

Fill reparation;

(9) after repairing piece

completion reparation; Upgrade the area to be repaired, and upgrade the degree of confidence C (p) of the point of having accomplished reparation with the degree of confidence of repairing piece

central point

:

$C\left(p\right)=C\left(\hat{p}\right),p\∈{\mathrm{\ψ}}_{\hat{p}}\∩\mathrm{\Ω},$

Wherein,

for repairing the degree of confidence of piece central point , and ∩ representes ' with ' relation;

(10) repeating step (1) ~ (9), have a few in the area to be repaired repaired.

The present invention compared with prior art has following advantage:

(1) the present invention adds the correlativity ε between two pixels in the existing similarity determination methods, makes similar of obtaining more accurate.

(2) the present invention is according to quantile α ₀With the difference t of similarity distance average, existing similarity weights computing formula ω is revised, make weights can estimate of the contribution of different sample block more accurately to filling block, make the detail section among the image repair result more accurate.

Description of drawings

Fig. 1 is a process flow diagram of the present invention;

Fig. 2 is the breakage image that the present invention tests use;

Fig. 3 is with the reparation result of the present invention to Fig. 2;

Fig. 4 is that the target that the present invention tests use removes image;

Fig. 5 is with the reparation result of the present invention to Fig. 4;

Fig. 6 is that the text that experiment is used among the present invention is removed image;

Fig. 7 is with the reparation result of the present invention to Fig. 6.

Embodiment

With reference to Fig. 1, performing step of the present invention is following:

Step 1 is read in image I to be repaired, Fig. 2 for example, and Fig. 4 or Fig. 6 confirm area to be repaired Ω and border δ thereof;

Step 2, the priority of all pieces of computing center's point on the δ of border:

(2.1) definition D (p) is a data item, and C (p) is the degree of confidence item, and the credibility of presentation video pixel carries out initialization: C (p)=0 to C (p), p ∈ Ω, C (p)=1, p ∈ I-Ω.

(2.2) utilize following formula, calculate degree of confidence item C (p) and data item D (p):

$C\left(p\right)=\frac{{\mathrm{\Σ}}_{q\∈{\mathrm{\ψ}}_{\mathrm{pI}(I-\mathrm{\Ω})}}C\left(q\right)}{\left|{\mathrm{\ψ}}_p\right|},$

$D\left(p\right)=\frac{|\▿{I_p}^{\⊥}\·n_p|}{\mathrm{\α}},$

Wherein, q is for repairing piece Ψ _pThe middle known point of pixel value, C (q) is the degree of confidence of some q, | Ψ _p| for repairing piece Ψ _PArea, n _pBe the vector of unit length vertical with the border, area to be repaired at the p place,

Be the p point place vector of unit length vertical with gradient, i.e. isophote direction, α is a normalizing parameter, for 8 gray level image α=255;

(2.3) utilize following formula, calculate the priority P (p) of all pieces of computing center's point on the δ of border: P (p)=C (p) D (p).

Step 3, according to the correlativity of the central point of the central point of repairing piece and sample block, carry out preliminary election and get:

(3.1) central point

with the highest reparation piece Ψ of priority

is the center; Choose size and be the neighborhood of the M * M region of search as this reparation piece, definition is that the piece

at center is a sample block with point

in should the zone.

(3.2) utilize following formula, the correlativity of the central point

of calculating reparation piece

and the central point

of sample block :

$\mathrm{\ϵ}=\exp (-\frac{{(\hat{p}-\hat{q})}^2}{2{\mathrm{\σ}}^2})$

Wherein, σ ²For repairing piece

Variance.

(3.3) correlativity that calculates above the basis; Find out in the region of search all and be in the point of homogeneous region, as new region of search with the central point

of repairing piece

;

Step 4 uses non-local mean image detail restorative procedure that it is repaired for

.

(4.1) calculate the similarity distance of sample block

and reparation piece

in the new region of search:

wherein,

is 2 norms;

(4.2) according to similarity distance

Obeying degree of freedom is card side's distribution x of n ²(n) characteristic, when n>=25, quantile

Choose m the most similar sample block that similarity distance is positioned at quantile β left side, as repairing piece

Set;

(4.3) according to similarity distance

Calculate quantile α ₀Difference with reparation piece average:

Wherein,

Be the value of its quantile,

For repairing the average of piece;

(4.4) utilize following formula, respectively the similarity weights of sample block

and reparation piece in the set of computations:

$\mathrm{\ω}=\frac{1}{Z}\exp (-\frac{{\left|\right|{\mathrm{\ψ}}_{\hat{p}}-{\mathrm{\ψ}}_{\hat{q}}\left|\right|}_2^2}{4{\mathrm{\σ}}^2\mathrm{Nt}})$

Wherein, Z is a normalized parameter, σ ²For repairing piece

Variance, N is for repairing piece The number of the point that middle pixel value is known;

(4.5) according to the similarity weights, with the weighted mean of the whole sample block in the set, as filling block Ψ ₀, and with this filling block to repairing piece

Fill reparation;

Step 5; After repairing piece

completion reparation; Upgrade the area to be repaired, and upgrade the degree of confidence C (p) of the point of having accomplished reparation with the degree of confidence of repairing piece

central point :

$C\left(p\right)=C\left(\hat{p}\right),$ $p\∈{\mathrm{\ψ}}_{\hat{p}}\∩\mathrm{\Ω},$

Wherein,

for repairing the degree of confidence of piece central point

, and ∩ representes ' with ' relation;

Repeat above five steps, have a few in the area to be repaired repaired.

Effect of the present invention can further confirm through following experiment:

1. experiment condition:

The Criminisi method is used in this experiment respectively, compares test based on non-local mean restorative procedure and the inventive method of sample, and the reparation block size gets 7 * 7 in the experiment, and the region of search size gets 41 * 41.This experiment is divided into three parts: experiment is repaired in the damaged zone of (1) image, and its experiment uses figure to be Fig. 2 (b), and the experimental result contrast uses figure to be Fig. 2 (a); (2) text is removed experiment, and its experiment uses figure to be Fig. 4 (b), and the experimental result contrast uses figure to be Fig. 4 (a); (3) text is removed experiment, and its experiment uses figure to be Fig. 6 (b), and the experimental result contrast uses figure to be Fig. 6 (a).Various control methodss all are to use the MATLAB Programming with Pascal Language to realize in this experiment.

2. experiment content and result:

Under above-mentioned experiment condition, carry out the experiment of three parts respectively.

Experiment (1): utilize the present invention and existing two kinds of methods; Repair process is carried out in damaged zone in Fig. 2 (b) image; Result such as Fig. 3; Wherein Fig. 3 (a) is for using the figure as a result of Criminisi method reparation, and Fig. 3 (b) is for using the figure as a result that repairs based on the non-local mean restorative procedure of sample, the figure as a result that Fig. 3 (c) repairs for use the present invention.

Respectively with experimental result Fig. 3 (a) of top three kinds of methods, Fig. 3 (b) and Fig. 3 (c) with compare with original image Fig. 2 (a) respectively, see from visual effect, use the texture information that the inventive method can better the connection layout picture, more approach former figure.

Experiment (2) utilizes the present invention and existing two kinds of methods; Target in Fig. 4 (b) image is removed; Result such as Fig. 5; Wherein Fig. 5 (a) is for using the figure as a result of Criminisi method reparation, and Fig. 5 (b) is for using the figure as a result that repairs based on the non-local mean restorative procedure of sample, the figure as a result that Fig. 5 (c) repairs for use the present invention.

With experimental result Fig. 5 (a) of top three kinds of methods, Fig. 5 (b) and Fig. 5 (c) compare respectively, see from visual effect, use reparation that the inventive method obtains grain details clear and natural more as a result, more near former figure.

Experiment (3) utilizes the present invention and existing two kinds of methods; Fig. 6 (b) image Chinese version is removed; Result such as Fig. 7; Wherein Fig. 7 (a) is for using the figure as a result of Criminisi method reparation, and Fig. 7 (b) is for using the figure as a result that repairs based on the non-local mean restorative procedure of sample, the figure as a result that Fig. 7 (c) repairs for use the present invention.

Respectively with experimental result Fig. 7 (a) of top three kinds of methods; Fig. 7 (b) and Fig. 7 (c) with compare with original image Fig. 6 (a) respectively; See from visual effect; Use reparation that the inventive method obtains grain details clear and natural more as a result, borderline region and texture region all more near with former figure.

Calculate above-mentioned three kinds of method reparation results' Y-PSNR PSNR respectively, its result is as shown in table 1:

Table 1 uses distinct methods to repair result's PSNR value contrast

Visible from table 1, the Y-PSNR PSNR of the inventive method has raising than existing two kinds of methods.

Above experimental result shows that the present invention is superior to existing two kinds of methods on overall performance, can repair the detailed information that better keeps edge and texture on the result, and image is clear and natural more.

Claims (2)

1. a non-local mean image detail restorative procedure comprises the steps:

(1), confirms the border δ of area to be repaired Ω and area to be repaired for the image I to be repaired of input;

(2) utilize following formula, calculate the priority P (p) that central point all on the δ of the border of area to be repaired are repaired piece:

P(p)＝C(p)·D(p)，

Wherein, D (p) is a data item, and C (p) is the degree of confidence item, the credibility of presentation video pixel, and C (p) is initialized as C (p)=0, p ∈ Ω, C (p)=1, p ∈ I-Ω;

(3) central point

with the highest reparation piece of priority

is the center; Choose size and be the neighborhood of the M * M region of search as this reparation piece, definition is that the piece at center is a sample block with point

in should the zone;

(3.1) utilize following formula, the correlativity of the central point

of calculating reparation piece and the central point

of sample block

:

$\mathrm{\ϵ}=\exp (-\frac{{(\hat{p}-\hat{q})}^2}{2{\mathrm{\σ}}^2})$

Wherein, σ ²For repairing piece

Variance;

(3.2) correlativity that obtains above the basis; Find out in the region of search all and be in the point of homogeneous region, as new region of search with the central point of repairing piece

;

(4) Calculate the new repair block within the search area

and the sample block

similarity distance:

$d({\mathrm{\ψ}}_{\hat{p}},{\mathrm{\ψ}}_{\hat{q}})={\left|\right|{\mathrm{\ψ}}_{\hat{p}}-{\mathrm{\ψ}}_{\hat{q}}\left|\right|}_2^2,$

Wherein,

is 2 norms;

(5) according to similarity distance

Obeying degree of freedom is card side's distribution x of n ²(n) characteristic, when n>=25, quantile

Choose m the most similar sample block that similarity distance is positioned at quantile β left side, as repairing piece

Set;

(6) according to similarity distance

Calculate quantile α ₀Difference with the similarity distance average:

Wherein,

Be the value of its quantile,

Average for similarity distance;

(7), calculate the similarity weights of interior sample block

of set and reparation piece

according to following formula:

$\mathrm{\ω}=\frac{1}{Z}\exp (-\frac{d({\mathrm{\ψ}}_{\hat{p}},{\mathrm{\ψ}}_{\hat{q}})}{4{\mathrm{\σ}}^2\mathrm{Nt}}),$

Wherein, Z is a normalized parameter, σ ²For repairing piece

Variance, N is for repairing piece

The number of the point that middle pixel value is known;

(8) according to the similarity weights, with the weighted mean of the whole sample block in the set, as filling block Ψ ₀, and with this filling block to repairing piece Fill reparation;

(9) after repairing piece

completion reparation; Upgrade the area to be repaired, and upgrade the degree of confidence C (p) of the point of having accomplished reparation with the degree of confidence of repairing piece

central point :

$C\left(p\right)=C\left(\hat{p}\right),p\∈{\mathrm{\ψ}}_{\hat{p}}\∩\mathrm{\Ω},$

Wherein,

for repairing the degree of confidence of piece central point

, and ∩ representes ' with ' relation;

(10) repeating step (1) ~ (9), have a few in the area to be repaired repaired.

2. according to claims 1 described bayesian non-local mean image repair method, it is characterized in that the computing method of step (2) described degree of confidence item C (p) and data item D (p) are:

$C\left(p\right)=\frac{{\mathrm{\Σ}}_{x\∈{\mathrm{\ψ}}_{p\∩(I-\mathrm{\Ω})}}C\left(x\right)}{\left|{\mathrm{\ψ}}_p\right|}$ $D\left(p\right)=\frac{|\▿{I_p}^{\⊥}\·n_p|}{\mathrm{\α}},$

Wherein, x is for repairing piece Ψ _pThe middle known point of pixel value, C (q) is the degree of confidence of some q, | Ψ _p| for repairing piece Ψ _PArea, n _pBe the vector of unit length vertical with the border, area to be repaired at the p place,

For with the vertical vector of unit length of gradient at p point place, i.e. the vector of unit length of p point place isophote direction, α is a normalizing parameter, for 8 gray level images, α=255.

Images