CN101847255B - Structural information synthesis-based image completion method - Google Patents
Structural information synthesis-based image completion method Download PDFInfo
- Publication number
- CN101847255B CN101847255B CN 201010152234 CN201010152234A CN101847255B CN 101847255 B CN101847255 B CN 101847255B CN 201010152234 CN201010152234 CN 201010152234 CN 201010152234 A CN201010152234 A CN 201010152234A CN 101847255 B CN101847255 B CN 101847255B
- Authority
- CN
- China
- Prior art keywords
- node
- image
- expression
- target area
- cost
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention belongs to the technical field of image inpainting, and relates to a structural information synthesis-based image completion method, which comprises the following steps of: for an image, determining an inpainting border area; calculating the cost energy of nodes per se; calculating the correlation energy of the nodes; determining an integral target function; initializing and updating message values between every two adjacent nodes; and calculating the optimal solution of the nodes. The method has the advantages of achieving the effect of simultaneously reserving structural information and texture information without human intervention and greatly improving image inpainting effect.
Description
Technical field
The invention belongs to the image repair technical field, relate to a kind of image completion method of compages information.
Background technology
The background technology that relates among the present invention has:
(1) TV image repair algorithm: the inpainting model based on the total variational method is by Chan﹠amp the earliest; Shen is called the TV model based on Variational Principle, and it is one of best model that keeps structural information based on partial differential equation, namely utilize diffusion equation in the physics with the Information Communication of known region in zone to be repaired.Breakage has reasonable effect to TV image repair algorithm to small scale, can effectively keep its structural information.If but the area to be repaired area more greatly then can produce significantly fuzzyly, not too meets people's visual sensory.
(2) based on the synthetic image completion algorithm of texture: the image completion algorithm that Criminisi has proposed patch-based texture synthesis solves the problem that removes the large tracts of land object from image.Utilize this algorithm, can fill the zone that object removes with a kind of visually reasonable manner.This algorithm combines the advantage based on the ﹠ Future Opportunities of Texture Synthesis of sample and small scale image repair technology effectively.This algorithm has proposed the concept of confidence level, and it is similar to the information propagation pattern in the small scale image repair technology.Although this algorithm has been eliminated the blurring effect in the former algorithm, but easily cause matching error, therefore for the abundant Incomplete image of structural information, repairing effect is not good.
Summary of the invention
The object of the invention is to overcome the above-mentioned deficiency of prior art, a kind of more efficient image completion method is provided.Do not need human intervention to reach the effect that keeps simultaneously structural information and texture information.
For this reason, the present invention adopts following technical scheme: a kind of image completion method of compages information comprises the following steps:
1. determine the target area of image, read in the image of the target area excessively to be repaired of mark.Judge the target area according to color value, draw mask images, represent with a logic matrix, wherein element value is that 1 this pixel of expression needs to repair in the matrix; Element value is 0 to be expressed as known pixels.Then this logic matrix and Laplace operator are done convolution, find out the border of this target area.
2. the cost energy of computing node self, E
1(xi)=C (i) d (x
i, φ) wherein,
The expression node credibility, and when overlapping with known region, d (x
i, φ) expression piece x
iWith the matching degree of known region, it is that SSD weighs with the difference quadratic sum of color value between the pixel of overlapping region.
3. the relevance energy function of computing node, E
2(x
i, x
j)=λ
1E
2 t(x
i, x
j)+λ
2E
2 s(x
i, x
j), E
2 t(x
i, x
j)=d (x
i, x
j), wherein, d (x
i, x
j) expression adjacent block x
iAnd x
jThe SSD of overlapping region.
D wherein
Gx(x
i, x
j) and d
Gy(x
i, x
j) represent respectively adjacent block x
iAnd x
jGradient difference in X and Y-direction.
4. determine the overall goals function
5. the message value between the initialization adjacent node, and upgrade.With any two node n among the bayesian network structure figure G
iAnd n
jBetween message be initially 0, namely
Upgrade the internodal value of information.During each iteration, for separating x
i, from n
iPropagate into n
jThe computing formula of information be:
Wherein N (j) i represent n
jNon-n
iThe field collection.m
J → i t(x
i) expression n
jNode is to x
iBe n
iDegree of belief.After T iteration, match block is x
iNode n
iThe reliability computing formula be:
6. the optimum solution of computing node finds to make b
i(x
i) minimized x
i *Be exactly node n
iBest matching blocks.That is:
The present invention is on the basis of traditional image completion algorithm that synthesizes based on texture and TV image repair algorithm, has proposed the image completion algorithm of compages information, has greatly improved the effect of image completion.This fundamental idea of the invention is to propose corresponding energy object function for characteristics of image, then uses the reliability propagation algorithm with its optimization, and the match block that chooses is copied to image after defect area draws reparation.This paper has proposed new energy function, and it has comprised structural energy, texture energy and correlativity energy.Combine structure and texture information just because of new energy function, so that this invention has remedied the deficiency of two class classic algorithm, can avoid blurring effect again so that can keep structural information, and proved superiority of the present invention from theoretical and experiment two aspects.
Description of drawings
Fig. 1 is based on the image completion method overview flow chart of compages information.
Fig. 2 chooses target.
The initial pictures that Fig. 3 is to be repaired.
Fig. 4 has marked the image that needs removing objects.
Fig. 5 (a), (b), (c) are respectively the image completion arithmetic result figure of the compages information that proposes with the as a result figure of the as a result figure of TV image repair algorithm reparation, Criminisi method, the present invention.
Embodiment
The present invention proposes the piece disappearance image repair algorithm frame based on structural information, this algorithm utilizes Bayesian Network will be applicable to the TV algorithm of repair structure information and the Texture Synthesis combination that is applicable to repair texture information, its essence is to propose corresponding energy object function for characteristics of image, then with its optimization, the coupling texture block that then will choose copies to defect area, draws the image after the reparation.This invention does not need human intervention to reach the effect that keeps simultaneously structural information and texture information.
The present invention is based on the image completion method of compages information, and Fig. 1 is overview flow chart, specifically may further comprise the steps:
1. determine the target area of image.
As shown in Figure 4, read in the image of the target area excessively to be repaired of color mark.Judge the target area according to color value, draw mask images, represent with a logic matrix, wherein element value is that 1 this pixel of expression needs to repair in the matrix; Element value is 0 to be expressed as known pixels.Then this logic matrix and Laplace operator are done convolution, find out the border of this target area.Judge boundary value, if be non-zero, expression also has the pixel that needs reparation, then down carries out; If 0, expression reparation is finished, and withdraws from.
2. the cost energy of computing node self
If I represents whole image, Ω is target area (namely treating the completion zone).The same with ﹠ Future Opportunities of Texture Synthesis, need the first size of definite sample window Ψ, default value is 9 * 9 pixels.Fig. 2 shows the algorithm pattern that mates energy function.In order to define the node in the algorithm, the image lattice is by horizontal space pixel gap
xWith vertical space pixel gap
yForm.Known observation label is comprised of all w of regional φ * h piece, is designated as X={x
1, x
2... x
k.And
N block of pixels in the expression target area is node.
According to the cost energy of each node self in formula (1) calculating chart 2 target areas, the i.e. rehabilitation cost of node.Wherein, C (i) represents node credibility, and its computing formula is (2), and C (i) value that the more outer field pixel in target area contains is larger, namely needs more early to fill; And it is less in C (i) value of the pixel at the center of target area.In the formula | Ψ
i| be piece | Ψ
i| area.
E
1(x
i)=C(i)d(x
i,φ) (1)
And when overlapping with known region, d (x
i, φ) expression piece x
iWith the matching degree of known region, it is that SSD weighs with the difference quadratic sum of color value between the pixel of overlapping region.Work as x
iAnd there is not when overlapping E between known region
1(x
i)=0.As shown in Figure 2, E
1(x
i) then be that the P piece is put into x
iThe SSD value of time domain 1 (red line mark) on the position.
3. the relevance energy function of computing node
Such as publicity (3), E
2(x
i, x
j) expression (x
i, x
j) the consistance cost.Wherein, E
2 t(x
i, x
j) the using texture homogeneity cost of adjacent block in the presentation video, E
2 s(x
i, x
j) expression adjacent block the structural integrity cost.λ
1And λ
2Weight coefficient for balance texture cost and infrastructure cost.When containing more significant structural information in the target area, λ
2Then larger; When texture information in the target area relatively enriches, λ
1Then larger.
E
2(x
i,x
j)=λ
1E
2 t(x
i,x
j)+λ
2E
2 s(x
i,x
j) (3)
E
2 t(x
i, x
j) be defined as:
E
2 t(x
i,x
j)=d(x
i,x
j) (4)
Wherein, d (x
i, x
j) expression adjacent block x
iAnd x
jThe SSD of overlapping region.As shown in Figure 2: E
2 t(x
i, x
j) be that the piece P in any known region and P ' are put into respectively x
iAnd x
jThe time, lap is the SSD value in Green Marker zone.
E
2 s(x
i, x
j) be defined as:
D wherein
Gx(x
i, x
j) and d
Gy(x
i, x
j) represent respectively adjacent block x
iAnd x
jGradient difference in X and Y-direction.
4. determine the overall goals function.
The overall goals function definition is (6).The overall goals function has represented the Global Information in the entire image, such as: structure, texture, global consistency etc.The overall goals function be all nodes of target area energy and.In this algorithm, the energy function of node comprises the cost ENERGY E of self
1With with the relevance energy function E of field node
2Determining of overall goals function is the key of whole algorithm, and it is related to whether Global Information can keep effectively in the repair process.
Wherein, E
1(x
i) expression node rehabilitation cost, E
2(x
i, x
j) expression (x
i, x
j) the consistance cost.
5. the message value between the initialization adjacent node, and upgrade.
With any two node n among the bayesian network structure figure G
iAnd n
jBetween message be initially 0, namely
Upgrade the internodal value of information
During each iteration, for separating x
i, from n
iPropagate into n
jThe computing formula of information be:
Wherein N (j) i represent n
jNon-n
iThe field collection.m
J → i t(x
i) expression n
jNode is to x
iBe n
iDegree of belief.After T iteration, match block is x
iNode n
iThe reliability computing formula be:
6. the optimum solution of computing node.
Find and make b
i(x
i) minimized x
i *Be exactly node n
iBest matching blocks.That is:
Fig. 3 is original image, Fig. 4 is for marking the image that needs removing objects, and Fig. 5 (a), (b), (c) are respectively the image completion arithmetic result figure of the compages information that proposes with the as a result figure of the as a result figure of TV image repair algorithm reparation, Criminisi method, the present invention.Difference to three width of cloth figure is carried out mark with red rectangle, can find out: (a) occur large-area fuzzy among the figure; (b) more serious matching error has appearred, the roof line in the red rectangle of mark and discontinuous among the figure; (d) figure has remedied the defective among front two width of cloth figure significantly.The color on roof is very complete in two rectangles, and lake surface completion effect is fine.
Claims (1)
1. the image completion method of a compages information comprises the following steps:
(1) determine the target area of image: read in the image that is labeled, judge the target area according to color value, draw mask images, represent with a logic matrix, wherein element value is that 1 expression needs the pixel of repairing in the matrix; Element value is 0 to be expressed as known pixels, then this logic matrix and Laplace operator is done convolution, finds out the border of this target area;
(2) the cost energy of computing node self, E
1(x
i)=C (i) d (x
i, φ) wherein,
The expression node credibility, and when overlapping with known region, d (x
i, φ) expression piece x
iWith the matching degree of known region, it is that SSD weighs with the difference quadratic sum of color value between the pixel of overlapping region, and Ω is the target area;
(3) the relevance energy function E of computing node
2(x
i, x
j)=λ
1E
2 t(x
i, x
j)+λ
2E
2 s(x
i, x
j), E
2 t(x
i, x
j)=d (x
i, x
j), wherein, d (x
i, x
j) expression adjacent block x
iAnd x
jThe SSD of overlapping region;
Wherein, d
Gx(x
i, x
j) and d
Gy(x
i, x
j) represent respectively adjacent block x
iAnd x
jGradient difference in X and Y-direction; λ
1And λ
2Weight coefficient for balance texture cost and infrastructure cost;
(4) determine that overall goals function E (x) is the rehabilitation cost E of all nodes of target area
1(x
i) and consistance cost E
2(x
i, x
j) energy and;
(5) message value between the initialization adjacent node, and upgrade: with any two node n among the bayesian network structure figure
iAnd n
jBetween message be initially 0, namely
Upgrade the internodal value of information, during each iteration, for separating x
i, from n
iPropagate into n
jThe computing formula of information be:
Wherein, N (j) i represent n
jNon-n
iThe neighborhood collection;
Expression n
jNode is to x
iBe n
iDegree of belief; After t iteration, match block is x
iNode n
iThe confidence level computing formula be:
(6) optimum solution of computing node finds to make b
i(x
i) minimized
Be exactly node n
iBest matching blocks, that is:
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN 201010152234 CN101847255B (en) | 2010-04-21 | 2010-04-21 | Structural information synthesis-based image completion method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN 201010152234 CN101847255B (en) | 2010-04-21 | 2010-04-21 | Structural information synthesis-based image completion method |
Publications (2)
Publication Number | Publication Date |
---|---|
CN101847255A CN101847255A (en) | 2010-09-29 |
CN101847255B true CN101847255B (en) | 2013-01-30 |
Family
ID=42771867
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN 201010152234 Expired - Fee Related CN101847255B (en) | 2010-04-21 | 2010-04-21 | Structural information synthesis-based image completion method |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN101847255B (en) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102063705B (en) * | 2010-12-02 | 2012-08-08 | 天津大学 | Method for synthesizing large-area non-uniform texture |
CN106408546A (en) * | 2016-09-05 | 2017-02-15 | 广东欧珀移动通信有限公司 | Image restoration method, apparatus and terminal device |
CN109685724B (en) * | 2018-11-13 | 2020-04-03 | 天津大学 | Symmetric perception face image completion method based on deep learning |
CN109741268B (en) * | 2018-12-05 | 2023-05-09 | 天津大学 | Damaged image complement method for wall painting |
CN111815543B (en) * | 2020-08-04 | 2024-02-09 | 北京惠朗时代科技有限公司 | Image restoration-oriented multi-scale feature matching method |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1731449A (en) * | 2005-07-14 | 2006-02-08 | 北京航空航天大学 | A method of image restoration |
-
2010
- 2010-04-21 CN CN 201010152234 patent/CN101847255B/en not_active Expired - Fee Related
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1731449A (en) * | 2005-07-14 | 2006-02-08 | 北京航空航天大学 | A method of image restoration |
Non-Patent Citations (3)
Title |
---|
jian Sun et al..Image Completion with Structure Propagation.《ACM Transactions on Graphics》.2005,第24卷(第3期),全文. * |
刘忠艳 等.一种基于置信度传播的立体匹配算法.《自动化与仪器仪表》.2010,(第1期),全文. * |
张红英 等.数字图像修复技术综述.《中国图象图形学报》.2007,第12卷(第1期),全文. * |
Also Published As
Publication number | Publication date |
---|---|
CN101847255A (en) | 2010-09-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN101847255B (en) | Structural information synthesis-based image completion method | |
CN103578085B (en) | Image cavity region based on variable-block method for repairing and mending | |
CN102999887B (en) | Sample based image repairing method | |
CN101246545B (en) | Possion method for removing cloud from optical remote sensing image | |
CN101661613A (en) | Image restoration method based on image segmentation, and system therefor | |
CN103914863B (en) | A kind of coloured image abstract method for drafting | |
CN104616286B (en) | Quick semi-automatic multi views depth restorative procedure | |
CN103971338B (en) | Variable-block image repair method based on saliency map | |
CN103377462B (en) | The method and apparatus that scan image is processed | |
CN104376535A (en) | Rapid image repairing method based on sample | |
CN103927717A (en) | Depth image recovery method based on improved bilateral filters | |
CN104680492B (en) | Image repair method based on composition of sample uniformity | |
CN102509327B (en) | Multiscale global sampling method for filling image void | |
CN104715451A (en) | Seamless image fusion method based on consistent optimization of color and transparency | |
CN104809698A (en) | Kinect depth image inpainting method based on improved trilateral filtering | |
CN104159093A (en) | Time-domain-consistent cavity region repairing method for static scene video shot in motion | |
CN107369204A (en) | A kind of method for recovering the basic three-dimensional structure of scene from single width photo based on deep learning | |
CN107240073A (en) | A kind of 3 d video images restorative procedure merged based on gradient with clustering | |
CN101674397A (en) | Repairing method of scratch in video sequence | |
CN103971354A (en) | Method for reconstructing low-resolution infrared image into high-resolution infrared image | |
CN102957936B (en) | Virtual viewpoint generation method from video single viewpoint to multiple viewpoints | |
CN106504294A (en) | RGBD image vector methods based on diffusion profile | |
CN106898048A (en) | A kind of undistorted integration imaging 3 D displaying method for being suitable for complex scene | |
CN104008547A (en) | Method for visible serial segmentation of human body slice images based on skeleton angular points | |
CN104881850B (en) | A kind of spissatus automatic removal method of multi-temporal remote sensing image |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C14 | Grant of patent or utility model | ||
GR01 | Patent grant | ||
CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20130130 |
|
CF01 | Termination of patent right due to non-payment of annual fee |