CN103886561B - Criminisi image inpainting method based on mathematical morphology - Google Patents

Abstract

A Criminisi image inpainting method based on mathematical morphology comprises the steps of inputting images to be restored, adopting specific colors to pre-mark areas to be restored of the images to be restored and preprocessing the images to be restored based on the mathematical morphology; calculating the priority of all of pixel points o the edges of the areas to be restored and confirming preferential restored pixel points; searching and filling optimum matching blocks in good areas of the preferential restored pixel points; updating the edges of the preferential restored pixel points, re-processing the edges till the areas to be restored are restored, and obtaining image inpainting results. The Criminisi image inpainting method based on the mathematical morphology s wide in image inpainting range, has high inpainting quality on different images to be restored, meet the vision demands of people and has important practical significance.

Description

Criminisi image repair methods based on mathematical morphology

Technical field

The present invention relates to image restoration field, the Criminisi image repairs based on mathematical morphology are especially related to Method.

Background technology

Image repair is the important content of image restoration, is a research heat of Digital Image Processing and computer vision Point, has been widely applied to the aspects such as reparation, removal word, the removal jamming target of incomplete photo in recent years.Its essence is with Information present in complex pattern to be repaired makes image repair whole structure meet the visual demand of people come the information for recovering to lack.

Current image repair method is broadly divided into two major classes：Image repair method based on partial differential equation and based on texture Image repair method.And the image that big area information is lacked is directed to, generally using the image repair method based on texture.Its essence is pin Block of pixels in image is repaired, centered on a pixel on damaged edge, with block of pixels existing in image Matched, come the impaired region of filling information, reached the purpose of image repair, Criminisi image repair algorithms are to be based on The representative of the image repair method of texture.

Criminisi image repairs algorithm was proposed that it was repaired and is referred to as by Criminisi et al. in 2004：Preferentially Power calculating, best match block search and filling, renewal confidence level.But the priority in Criminisi image repair algorithms Priority (p) is calculated and the search of best matching blocks has self-defect with filling, i.e.,：Confidence level can be with repairing number of times The information for increasing artwork in causing is reduced, and has the appearance influence priority of the difference of the order of magnitude, is occurred in data item vertical Phenomenon, it is zero to cause priority, the size of confidence level to priority will nonsensical and best match module more than one, System can be random selection etc..Therefore nearly ten years, there are the priority of different emphasis and the searching for best matching blocks of many Rope and the improvement filled：It is interim in electronics technology volume 23 the 1st in 2010《A kind of improved image repair side based on master drawing Method》In one text, using graded, block size selection is dynamically carried out, eliminate exploratory artificial setting module size Bother, improve operating efficiency, improve image repair effect；In CAD in 2011 and figure journal volume 23 2nd is interim《The complex region reparation of the Dunhuang frescoes based on priority innovatory algorithm》In one text, for complicated damaged area Peripheral information obscures the more Dunhuang frescoes of insincere and number, is taking into full account Dunhuang frescoes self information complexity, mural painting Repair visual effect and its repair on the basis of reasonability easily the factor such as influenceed by pixel reparation order, a kind of D-S evidences of proposition The image repair algorithm that gross data fusion method is improved to the prioritization functions of restoring area filling algorithm, repairing effect Relative to the raising that former algorithm has conspicuousness；It is interim in computer application in 2012 and software volume 29 the 9th《One kind is improved Criminisi image repair algorithms》In one text, introduce curvature to determine the choosing of the fill order and best matching blocks of object block Select, and it is every weighted sum to improve priority, and more preferable repairing effect can be obtained by changing weights, at the same avoid due to The wrong fill order that the rapid attenuation band of confidence level comes, achieves gratifying repairing effect.Above modified hydrothermal process is pin To the architectural feature of certain area to be repaired, what the calculating defect of certain Criminisi image repair algorithms was carried out, can only The accumulation of error message is reduced to a certain extent.

The content of the invention

Priority calculating and best match block search and filling in Criminisi image repair algorithms are in restoring area Carried out on the basis of mark, therefore the first step is carried out, and is the basis of follow-up work.Present invention introduces the corrosion in mathematical morphology With expansion, pre-processed, it is ensured that it is reasonable that area to be repaired marks, and having for Criminisi image repair algorithms is ensured with this Effect is carried out, and reduces the accumulation of error message, it is adaptable to the reparation of the area to be repaired of different features.

Technical solution of the present invention provides a kind of Criminisi image repair methods based on mathematical morphology, including following Step, step 1 is input into complex pattern to be repaired, and the area to be repaired of the complex pattern to be repaired is marked, treated in advance using particular color Reparation image carries out pretreatment and is shown below,

Pretreatment expression formula is shown below：

Wherein, A is complex pattern to be repaired, and B, C are not less than C for the size of the consistent structural element of shape and B, and B is tied for expansion Constitutive element, C is corrosion structure element；

Step 2, calculates priority priority (p) of each pixel at area to be repaired edge, it is determined that preferentially repairing picture Vegetarian refreshments；

Step 3, for preferential repairing pixel point, the search and filling of best matching blocks is carried out in intact region；

Step 4, updates area to be repaired edge, and return to step 2 carries out repetitive cycling operation, until area to be repaired is repaired Complete, obtain image repair result.

And, according to expansion structure element B, expansion expression formula is shown below,

Wherein, x is the pixel of the image after expansion, and a is the pixel of complex pattern A to be repaired, and b is expansion structure element B In element.

And, according to corrosion structure Elements C, corrosion expression formula is shown below,

Wherein, x is the pixel of the image after expansion, and a is the pixel of complex pattern A to be repaired, and c is expansion structure Elements C In element.

The present invention is applicable the wide range of image repair, has reparation very high for the complex pattern to be repaired of different emphasis Quality, meets the visual demand of people, with important practical significance.There is advantages below compared with prior art：

（1）The present invention need not improve the decision formula of priority and matching principle, it is only necessary in the base of artificial mark in advance On plinth, pre-processed using mathematical morphology, Criminisi image repairs algorithm is carried out on the basis of rational, can The accumulation of error message is greatly reduced, and reduces amount of calculation, realize easy, effect is significant.

（2）Improvement mathematical morphology of the invention cannot only be applied to classical Criminisi image repairs algorithm, equally In may apply to other improved Criminisi image repairs algorithms, for example with other confidence calculations modes Criminisi image repair algorithms, can equally lift repairing effect, reduce the accumulation of error message.

Brief description of the drawings

Fig. 1 is the flow chart of the embodiment of the present invention.

Specific embodiment

Technical solution of the present invention can support automatic running flow using computer software mode.Below in conjunction with accompanying drawing and implementation Example describes technical solution of the present invention in detail.

Referring to Fig. 1, the flow of embodiment is as follows：

Step 1, is input into complex pattern to be repaired, and the area to be repaired of the complex pattern to be repaired is marked in advance using particular color, Image to be repaired is pre-processed.The region that wherein φ is represented is unlabelled region, i.e., intact region；The region that Ω is represented It is the region of mark, i.e. area to be repaired；δ Ω represent the border in region to be repaired.

Specific implementation is artificial mark in advance can be carried out to the area to be repaired of image to be repaired using a kind of specific color Note, can voluntarily be selected area to be repaired to carry out handmarking, to be carried out in follow-up technical solution of the present invention by user The rim detection of area to be repaired lays the first stone.After marked the complex pattern to be repaired input of area to be repaired in advance, using this hair The flow that bright embodiment is provided is processed.Complex pattern to be repaired in the embodiment of the present invention is using white.For example, the photograph of certain personage There is supernumerary group in piece, in background, using white covering supernumerary group region.

The structural information and texture information at image edge to be repaired can be to a certain extent influenceed due to advance handmarking, because This is pre-processed using corrosion and expansion, reaches the effect of reasonable mark.Pretreatment expression formula is shown below：

Wherein, A is complex pattern to be repaired, and B, C are for the consistent structural element of shape and the size of B is not less than C, structure in principle Element is the set of element.

B is expansion structure element, and expansion expression formula is shown below：

Wherein, x is that the pixel of image after expansion, a are during the pixel of complex pattern to be repaired, b are expansion structure element B Element.

It is thicker that morphological dilation plays lines, and hole disappears, the effect of expanded images.

C is corrosion expansion structure element, and corrosion expansion expression formula is shown below：

Wherein, x is that the pixel of image after expansion, a are during the pixel of complex pattern to be repaired, c are expansion structure Elements C Element.

Morphological erosion plays lines and attenuates, expansion of pores, the effect of contractible graph picture.

Pretreatment expression formula combines expansion expression formula and corrodes expression formula, and effect is ensuring that the reasonability of mark, Ensure area to be repaired with specific color mark area to be repaired can guarantee that simultaneously, again the texture at area to be repaired edge with Structural information is interference-free, such that it is able to reduce the accumulation of error message from root.

During specific implementation, those skilled in the art can voluntarily preset the size radius of B, C, and the size of C is not more than B, 0 even can be set to, i.e., use corrosion, for example, be respectively the collar plate shape structural element that radius is 4 and 0 from B, C.

Step 2, calculates priority priority (p) of each pixel at area to be repaired edge, it is determined that preferential reparation Pixel.

Embodiment calculates each pixel at area to be repaired edge according to formula priority (p)=C (p) × D (p) Priority, choose the maximum pixel of priority and enter row major reparation.

Wherein：

Wherein, the definition of variable C (i) is C (i)=0,C (i)=1,|ψ_p| represent ψ_pArea, ψ_pTable Show object block to be repaired.

Wherein, n_pIt is certain pixel p at area to be repaired edge on borderThe outer normal vector of the unit of Ω；Represent picture The direction and the intensity that wait radiation at vegetarian refreshments p, its expression formula isI_x, I_yPixel p is represented respectively in x, y Partial differential on direction；α is normalized parameter, and value is 255 in gray-scale map.

C (p) be confidence level, numerical value is bigger, and priority is higher, reflect containing artwork region information it is many, should give excellent First repair；D (p) is data item, and numerical value is bigger, and priority is higher, and it is high to reflect evolution surface linear structural strength, should give excellent First repair.

During specific implementation, C (p) and D (p) can also use other calculations.

Step 3, it is determined that after preferential repairing pixel point, the search and filling of best matching blocks are carried out in intact region φ.Can Determine that best match template is filled according to SSD matching principle global searches.

Embodiment set the preferential repairing pixel point of step 2 gained asAccordingly object block to be repaired isMatching principle is such as Shown in following formula：

Wherein,Represent object block to be repaired；ψ_qRepresent the sample block in intact region；Represent two blocksWith ψ_qKnown pixels point color difference quadratic sum；SSD matching principles are represented when object block to be repairedWith sample in intact region This block ψ_qKnown element quadratic sum it is minimum when, this sample block is best matching blocksAnd then according to best matching blocksTo object block to be repairedIt is filled.

During specific implementation, it would however also be possible to employ other matching principles.

Step 4, updates area to be repaired edge, finally repetitive cycling operation is constantly carried out from return to step 2, until treating Restoring area Ω is repaired and completed, then image repair is completed.During specific implementation, can first judge whether area to be repaired Ω has repaired Into, it is to terminate flow, otherwise return to area to be repaired edge, return to step 2.

The block of reparation is all on the edge of area to be repaired, therefore often one reparation of texture block of completion is just carried out more to edge Newly.Based on each pixel at remaining area to be repaired edge, new preferential repairing pixel point is selected equally to be processed.

Specific embodiment described herein is only to the spiritual explanation for example of the present invention.Technology neck belonging to of the invention The technical staff in domain can be made various modifications or supplement to described specific embodiment or be replaced using similar mode Generation, but without departing from spirit of the invention or surmount scope defined in appended claims.

Claims (1)

1. a kind of Criminisi image repair methods based on mathematical morphology, it is characterised in that comprise the following steps,

Step 1, is input into complex pattern to be repaired, and the area to be repaired of the complex pattern to be repaired is marked, treated in advance using particular color Reparation image carries out pretreatment and is shown below,

$D=A\⊕B\mathrm{\Θ}C$

Wherein, A is complex pattern to be repaired, and B, C are not less than C for the size of the consistent structural element of shape and B, and B is expansion structure unit Element, C is corrosion structure element；

According to expansion structure element B, expansion expression formula is shown below,

$A\⊕B=\left\{x\right|\∃a\∈A,b\∈B:x=a+b\}$

Wherein, x is the pixel of the image after expansion, and a is the pixel of complex pattern A to be repaired, and b is in expansion structure element B Element；

According to corrosion structure Elements C, corrosion expression formula is shown below,

$A\mathrm{\Θ}C=\left\{x\right|\∀c\∈C,\∃a\∈A:x=a-c\}$

Wherein, x is the pixel of the image after expansion, and a is the pixel of complex pattern A to be repaired, and c is in expansion structure Elements C Element；

Step 2, calculates priority priority (p) of each pixel at area to be repaired edge, it is determined that preferential repairing pixel point； Implementation is as follows,

According to formula priority (p)=C (p) × D (p), the priority of each pixel at area to be repaired edge is calculated, selected Take the maximum pixel of priority and enter row major reparation,

Wherein, C (p) be confidence level, numerical value is bigger, and priority is higher, reflect containing artwork region information it is many, should give Preferential repairing；D (p) is data item, and numerical value is bigger, and priority is higher, and it is high to reflect evolution surface linear structural strength, should give Preferential repairing；

$C\left(p\right)=\frac{{\mathrm{\Σ}}_{i\∈{\mathrm{\ψ}}_p\∩\mathrm{\φ}}C\left(i\right)}{\left|{\mathrm{\ψ}}_p\right|}$

Wherein, the definition of variable C (i) is|ψ_p| represent ψ_pArea, ψ_pRepresent to be repaired Object block；The region that φ is represented is unlabelled region, represents intact region；

$D\left(p\right)=\frac{|\▿I_p^{\⊥}\·n_p|}{\mathrm{\α}}$

Wherein, n_pIt is certain pixel p at area to be repaired edge on borderThe outer normal vector of unit；Represent pixel p The direction and the intensity that wait radiation at place, expression formula isI_x、I_yPixel p is represented respectively on x, y direction Partial differential；α is normalized parameter；

Step 3, for preferential repairing pixel point, the search and filling of best matching blocks is carried out in intact region；

Step 4, updates area to be repaired edge, and return to step 2 carries out repetitive cycling operation, until area to be repaired has been repaired Into obtaining image repair result.