CN102800077B - Bayes non-local mean image restoration method - Google Patents

Abstract

The invention discloses a Bayes non-local image restoration method which mainly solves the problems that the search of similar blocks is inaccurate and a parameter value is determined by experience in the existing sample-based non-local mean image restoration method. The method comprises the following steps: (1) determining a to-be-restored area omega and the boundary delta thereof for a to-be-restored image I; (2) finding a restoration block psi<hat(p)> with the highest priority on the boundary, and modeling the psi<hat(p)> by use of a Bayesian framework; (3) pre-selecting a search area by use of an adaptive threshold; (4) searching for m sample blocks the most similar to the restoration block psi<hat(p)>, and taking the weighted mean of the m sample blocks as a filling block psi<p'> of the restoration block; and (5) updating the confidence item and the to-be-restored area, and repeating the steps (1) to (5) until all points in the to-be-restored area are restored. The method disclosed by the invention can be used for restoring the image-damaged area, restoring the image scratch and removing the text in the image.

Description

Bayesian non-local mean image repair method

Technical field

The invention belongs to technical field of image processing, relate to image repair, can be used for repairing damaged area in image, the removal of image scratch and image Chinese version.

Background technology

Image information with its contain much information, the important means that advantage becomes the important sources of mankind's obtaining information and utilize information such as transmission speed is fast, operating distance is far away, and image in reality can cause the loss of image information for various reasons, at this moment will use image repair technology.

The object of image repair is automatically to recover the information of losing according to the existing information of image, and they can be for the recovery of drop-out in old photo, video text removal and video error concealing etc.Existing image repair method roughly can be divided into the restorative procedure based on structure and the large class of the restorative procedure based on texture two.Wherein the restorative procedure based on structure is all a kind of restorative procedure based on partial differential equation in essence, proposed by people such as Bertalmio the earliest, the restorative procedure based on total variation TV model being proposed by people such as Chan subsequently, and inspire the Curvature-driven diffusion CDD model restorative procedure producing all to belong to the restorative procedure based on structure by TV repairing model.These methods are all to realize by the diffusion of information, are only applicable to the image repair of non-texture image and small scale breakage.

In addition, the restorative procedure based on sample that the people such as Criminisi propose is a kind of restorative procedure based on texture, the method used for reference the thought in texture synthesis method find sample block and coupling copy, the diffusion way simultaneously taking full advantage of in the restorative procedure based on structure defines the priority of repairing piece, make to be near the reparation piece having the edge of more structural information and there is higher reparation priority, thereby in repairing texture information, structural information is also had to certain maintenance.The method adopts single sample block directly to fill area to be repaired, owing to being difficult to make sample block and multiblock to be repaired reach Optimum Matching in reality, therefore in the time filling multiblock to be repaired, can have certain error, along with the carrying out of repair process, this way can cause the accumulation of error.

Afterwards, Alexander Wong and Jeff Orchar have proposed a kind of non-local mean based on sample and have repaired algorithm, adopt the weighted mean of multiple sample block to synthesize the filling block for filling area to be repaired, improved to a certain extent the defect based on sample restorative procedure.But the method is owing to calculating the similarity weights of sample block and multiblock to be repaired with the negative exponential function that an attenuation coefficient is constant, and the information comprising in different multiblocks to be repaired is different, do like this and will certainly cause the calculating of similarity weights not accurate enough, and then cause repairing the well detail textures in connection layout picture of result.

Summary of the invention

The object of the invention is to the deficiency for the above-mentioned non-local mean restorative procedure based on sample, propose a kind of bayesian non-local mean image repair method, make the clean mark in image repair result, thereby improve repairing effect.

The technical thought that realizes the object of the invention is, on the basis of the non-local mean restorative procedure based on sample, utilize Bayesian Estimation theory, point in region of search is chosen in advance, and based on Bayesian Estimation belfry a new weights computing formula, utilize it to calculate the similarity weights of repairing between piece and sample block, can search for more accurately similar, better to be repaired result.Implementation step comprises as follows:

(1), for the image I to be repaired of input, determine the border δ of area to be repaired Ω and area to be repaired;

(2) utilize following formula, calculate the priority P (p) of all reparation pieces of central point on the δ of the border of area to be repaired:

P(p)＝C(p)·D(p)，

Wherein, D (p) is data item, and C (p) is degree of confidence item, the credibility of presentation video pixel, and C (p) is initialized as C (p)=0, p ∈ Ω, C (p)=1, p ∈ I-Ω;

(3) with the highest reparation piece of priority central point centered by, choose the region of search of the big or small neighborhood for M × M as this reparation piece, define in this region with point centered by piece for sample block;

(3.1) to repairing piece with sample block Ψ utilize Bayesian frame modeling, calculate respectively the average of repairing piece average with sample block and calculate their equal value difference ?

(3.2) according to equal value difference obey the characteristic of Gaussian distribution, define an adaptive threshold: t=λ σ ₀, wherein, σ ₀for standard deviation, λ=1.65, u ₀for average, for variance;

(3.3) select to own in region of search point, as choosing in advance rear new region of search;

(4) calculate the reparation piece in new region of search with sample block similarity distance:

$d({\mathrm{\&psi;}}_{\hat{p}},{\mathrm{\&psi;}}_{\hat{q}})={\left|\right|{\mathrm{\&psi;}}_{\hat{p}}-{\mathrm{\&psi;}}_{\hat{q}}\left|\right|}_2^2,$

Wherein, be 2 norms;

(5) according to similarity distance card side's distribution X that obedience degree of freedom is n ²(n) characteristic, in the time of n>=25, quantile choose m the most similar sample block that similarity distance is positioned at quantile β left side, as repairing piece set;

(6) according to following formula, the sample block in set of computations with reparation piece similarity weights:

$\mathrm{\&omega;}=\frac{1}{Z}\exp (-\frac{d({\mathrm{\&psi;}}_{\hat{p}},{\mathrm{\&psi;}}_{\hat{q}})}{4{\mathrm{\&sigma;}}^{2}N}),$

Wherein, Z is normalized parameter, σ ²for repairing piece variance, N for repair piece the number of the point that middle pixel value is known;

(7) according to similarity weights, by the weighted mean of the whole sample block in set, as filling block Ψ ₀, and with this filling block to repair piece fill reparation;

(8) when repairing piece complete after reparation, upgrade area to be repaired, and with repairing piece central point degree of confidence upgrade the degree of confidence C (p) of point that has completed reparation:

$C\left(p\right)=C\left(\hat{p}\right),$ $p\&Element;{\mathrm{\&psi;}}_{\hat{p}}\&cap;\mathrm{\&Omega;},$

Wherein, for repairing piece central point degree of confidence, ∩ represents ' with ' relation;

(9) repeating step (1) ~ (8), until being repaired a little in area to be repaired;

The present invention compared with prior art has following advantage:

(1) the present invention chooses in advance by average is carried out in region of search, gives up the different point of distributing, and only retains the identical point that distributes, and similar of making to search is more accurate, thereby improves repairing effect.

(2) the present invention is according to sample block and the similarity distance of repairing piece distribution character, set a most similar adaptive sample block number m, make the sample block number that searches more accurate, thereby improve repairing effect.

(3) the present invention, taking Bayesian Estimation theory as basis, has constructed a new adaptive weight, and these weights can estimate that different sample block are for the contribution of repairing piece more accurately, thereby improve repairing effect.

Brief description of the drawings

Fig. 1 is process flow diagram of the present invention;

Fig. 2 is the breakage image that the present invention tests use;

Fig. 3 is the reparation result to Fig. 2 with the present invention;

Fig. 4 is the cut image that the present invention tests use;

Fig. 5 is the reparation result to Fig. 4 with the present invention;

Fig. 6 is that the text that in the present invention, experiment is used is removed image;

Fig. 7 is the reparation result to Fig. 6 with the present invention.

Embodiment

With reference to Fig. 1, performing step of the present invention is as follows:

Step 1, reads in image I to be repaired, for example Fig. 2, and Fig. 4 or Fig. 6, determine area to be repaired Ω and border δ thereof.

Step 2, the priority of all piece of computing center's point on the δ of border:

(2.1) definition D (p) is data item, and C (p) is degree of confidence item, the credibility of its presentation video pixel, C (p) is carried out to initialization: C (p)=0, p ∈ Ω, C (p)=1, p ∈ I-Ω;

(2.2) utilize following formula, calculate degree of confidence item C (p) and data item D (p):

$C\left(p\right)=\frac{{\mathrm{\&Sigma;}}_{q\&Element;{\mathrm{\&psi;}}_{\mathrm{pI}(I-\mathrm{\&Omega;})}}C\left(q\right)}{\left|{\mathrm{\&psi;}}_p\right|},$

$D\left(p\right)=\frac{|{{\&dtri;I}_p}^{\&perp;}\&CenterDot;n_p|}{\mathrm{\&alpha;}},$

Wherein, q is for repairing piece Ψ _pthe point that middle pixel value is known, C (q) is the degree of confidence of a q, | Ψ _p| for repairing piece Ψ _parea, n _pfor at the p place vector of unit length vertical with border, area to be repaired, for the vector of unit length of p point place and gradient vertical, i.e. isophote direction, α is normalizing parameter, for 8 gray level image α=255;

(2.3) utilize following formula, calculate the priority P (p) of all piece of computing center's point on the δ of border:

P(p)＝C(p)D(p)。

Step 3, by the distribution character of equal value difference, region of search is chosen in advance:

(3.1) with the highest reparation piece of priority central point centered by, choose the field of search of the big or small neighborhood for M × M as this reparation piece, define in this region with point centered by piece for sample block;

(3.2) to repairing piece and sample block utilize Bayesian frame modeling, calculate respectively the average of repairing piece average with sample block and calculate their equal value difference ?

(3.3) according to equal value difference obey the characteristic of Gaussian distribution, define an adaptive threshold: t=λ σ ₀, wherein, σ ₀for standard deviation, λ=1.65, u ₀for average, for variance;

(3.4) select to own in region of search point, as choosing in advance rear new region of search.

Step 4, for use bayesian non-local method to repair it.

(4.1) calculate the sample block in new region of search with reparation piece similarity distance:

$d({\mathrm{\&psi;}}_{\hat{p}},{\mathrm{\&psi;}}_{\hat{q}})={\left|\right|{\mathrm{\&psi;}}_{\hat{p}}-{\mathrm{\&psi;}}_{\hat{q}}\left|\right|}_2^2,$ Wherein, be 2 norms;

(4.2) according to similarity distance card side's distribution X that obedience degree of freedom is n ²(n) characteristic, in the time of n>=25, quantile choose m the most similar sample block that similarity distance is positioned at quantile β left side, as repairing piece set;

(4.3) utilize following formula, respectively sample block in set of computations with reparation piece similarity weights:

$\mathrm{\&omega;}=\frac{1}{Z}\exp (-\frac{{\left|\right|{\mathrm{\&psi;}}_{\hat{p}}-{\mathrm{\&psi;}}_{\hat{q}}\left|\right|}_2^2}{4{\mathrm{\&sigma;}}^{2}N}),$

Wherein, Z is normalized parameter, σ ²for repairing piece variance, N for repair piece the number of the point that middle pixel value is known;

(4.4) according to similarity weights, by the weighted mean of the whole sample block in set, as filling block Ψ ₀, and with this filling block to repair piece fill reparation.

Step 5, when repairing piece complete after reparation, upgrade area to be repaired, and with repairing piece central point degree of confidence upgrade the degree of confidence C (p) of point that has completed reparation:

$C\left(p\right)=C\left(\hat{p}\right),$ $p\&Element;{\mathrm{\&psi;}}_{\hat{p}}\&cap;\mathrm{\&Omega;},$

Wherein, for repairing piece central point degree of confidence, ∩ represents ' with ' relation; Repeat above five steps, until being repaired a little in area to be repaired.

Effect of the present invention can further confirm by following experiment:

1. experiment condition:

This experiment is used respectively Criminisi method, and non-local mean restorative procedure and the inventive method based on sample are carried out contrast test, repair block size and get 7 × 7 in experiment, and region of search size gets 41 × 41.This experiment is divided into three parts: (1) image damaged area reparative experiment, and its experiment is Fig. 2 (b) with figure, experimental result contrast is Fig. 2 (a) with figure; (2) image scratch reparative experiment, its experiment is Fig. 4 (b) with figure, experimental result contrast is Fig. 4 (a) with figure; (3) text is removed experiment, and its experiment is Fig. 6 (b) with figure, and experimental result contrast is Fig. 6 (a) with figure.In this experiment, various control methodss are all to use MATLAB Programming with Pascal Language to realize.

2. experiment content and result:

Under above-mentioned experiment condition, carry out respectively the experiment of three parts.

Experiment (1): utilize the present invention and existing two kinds of methods, repair process is carried out in damaged area in Fig. 2 (b) image, result is as Fig. 3, the wherein result figure of Fig. 3 (a) for using Criminisi method to repair, the result figure that Fig. 3 (b) repairs for the non-local mean restorative procedure using based on sample, the result figure of Fig. 3 (c) for using the present invention to repair.

Respectively by experimental result Fig. 3 of three kinds of methods (a) above, Fig. 3 (b) and Fig. 3 (c) and original image Fig. 2 (a) contrast, from visual effect, use the texture information that the inventive method can better connection layout picture, more approach former figure.

Experiment (2) utilizes the present invention and existing two kinds of methods, cut in Fig. 4 (b) image is carried out to repair process, result is as Fig. 5, the wherein result figure of Fig. 5 (a) for using Criminisi method to repair, the result figure that Fig. 5 (b) repairs for the non-local mean restorative procedure using based on sample, the result figure of Fig. 5 (c) for using the present invention to repair.

Respectively by experimental result Fig. 5 of three kinds of methods (a) above, Fig. 5 (b) and Fig. 5 (c) and original image Fig. 4 (a) contrast, from visual effect, the reparation result grain details clear and natural more that uses the inventive method to obtain, more approaches former figure.

Experiment (3) utilizes the present invention and existing two kinds of methods, Fig. 6 (b) image Chinese version is removed, result is as Fig. 7, the wherein result figure of Fig. 7 (a) for using Criminisi method to repair, the result figure that Fig. 7 (b) repairs for the non-local mean restorative procedure using based on sample, the result figure of Fig. 7 (c) for using the present invention to repair.

Respectively by experimental result Fig. 7 of three kinds of methods (a) above, Fig. 7 (b) and Fig. 7 (c) and original image Fig. 6 (a) contrast, from visual effect, the reparation result grain details clear and natural more that uses the inventive method to obtain, all more approaches former figure at borderline region and texture region.

Calculate respectively above-mentioned three kinds of methods and repair the Y-PSNR PSNR of result, its result is as shown in table 1:

Table 1 uses distinct methods to repair the PSNR value contrast of result

As seen from Table 1, the Y-PSNR PSNR of the inventive method is improved than existing two kinds of methods.

Above experimental result shows, the present invention is better than existing two kinds of methods on overall performance, can repair in result better keep the edge information and texture information, and image is clear and natural more.

Claims (2)

1. a bayesian non-local mean image repair method, comprises the steps:

(1), for the image I to be repaired of input, determine the border δ of area to be repaired Ω and area to be repaired;

(2) utilize following formula, calculate the priority P (p) of all reparation pieces of central point on the δ of the border of area to be repaired:

P(p)＝C(p)·D(p)，

Wherein, D (p) is data item, and C (p) is degree of confidence item, the credibility of presentation video pixel, and C (p) is initialized as C (p)=0, p ∈ Ω, C (p)=1, p ∈ I-Ω;

(3) with the highest reparation piece of priority central point centered by, choose the region of search of the big or small neighborhood for M × M as this reparation piece, define in this region with point centered by piece for sample block;

(3.1) to repairing piece and sample block utilize Bayesian frame modeling, calculate respectively the average of repairing piece average with sample block and calculate their equal value difference ?

(3.2) according to equal value difference obey the characteristic of Gaussian distribution, define an adaptive threshold: t=λ σ ₀, wherein, σ ₀for standard deviation, λ=1.65, u ₀for average, for variance;

(3.3) select to own in region of search point, as choosing in advance rear new region of search;

(4) calculate the reparation piece in new region of search with sample block similarity distance:

Wherein, be 2 norms;

(5) according to similarity distance card side's distribution X that obedience degree of freedom is n ²(n) characteristic, in the time of n>=25, quantile choose m the most similar sample block that similarity distance is positioned at quantile β left side, as repairing piece set;

(6) according to following formula, the sample block in set of computations with reparation piece similarity weights:

Wherein, Z is normalized parameter, σ ²for repairing piece variance, N for repair piece the number of the point that middle pixel value is known;

(7) according to similarity weights, by the weighted mean of the whole sample block in set, as filling block Ψ ₀, and with this filling block to repair piece fill reparation;

(8) when repairing piece complete after reparation, upgrade area to be repaired, and with repairing piece central point degree of confidence upgrade the degree of confidence C (p) of point that has completed reparation:

Wherein, for repairing piece central point degree of confidence, ∩ represents ' with ' relation;

(9) repeating step (1) ~ (8), until being repaired a little in area to be repaired.

2. according to the bayesian non-local mean image repair method described in claims 1, it is characterized in that degree of confidence item C (p) that step (2) is described and the computing method of data item D (p) are:

Wherein, x is for repairing piece Ψ _pthe point that middle pixel value is known, C (x) is the degree of confidence of an x, | Ψ _p| for repairing piece Ψ _parea, n _pfor at the p place vector of unit length vertical with border, area to be repaired, for with the vector of unit length of the gradient vertical at p point place, i.e. the vector of unit length of p point place isophote direction, α is normalizing parameter, for 8 gray level images, α=255.