CN109829867B - Spherical convergence sample block repairing method for stable filling - Google Patents
Spherical convergence sample block repairing method for stable filling Download PDFInfo
- Publication number
- CN109829867B CN109829867B CN201910111792.2A CN201910111792A CN109829867B CN 109829867 B CN109829867 B CN 109829867B CN 201910111792 A CN201910111792 A CN 201910111792A CN 109829867 B CN109829867 B CN 109829867B
- Authority
- CN
- China
- Prior art keywords
- repaired
- block
- image
- confidence
- priority
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 85
- 230000008439 repair process Effects 0.000 claims description 45
- 230000011218 segmentation Effects 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 3
- 125000004432 carbon atom Chemical group C* 0.000 claims 1
- 230000000007 visual effect Effects 0.000 abstract description 8
- 238000004422 calculation algorithm Methods 0.000 description 50
- 230000000694 effects Effects 0.000 description 45
- 238000010586 diagram Methods 0.000 description 31
- 238000004088 simulation Methods 0.000 description 9
- 238000009792 diffusion process Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 238000009825 accumulation Methods 0.000 description 3
- 230000015572 biosynthetic process Effects 0.000 description 3
- 230000007423 decrease Effects 0.000 description 3
- 238000003786 synthesis reaction Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000001427 coherent effect Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000035876 healing Effects 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000003909 pattern recognition Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000002238 attenuated effect Effects 0.000 description 1
- OSGAYBCDTDRGGQ-UHFFFAOYSA-L calcium sulfate Chemical compound [Ca+2].[O-]S([O-])(=O)=O OSGAYBCDTDRGGQ-UHFFFAOYSA-L 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 239000010813 municipal solid waste Substances 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
Images
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses a spherical convergence sample block repairing method aiming at stable filling, which comprises the following steps: determining a region to be repaired and initializing a reliability value; then distinguishing whether the image is located in a structural region or a texture region; determining the priority of the boundary sample block of the area to be repaired, and adopting a structure priority ratio to calculate the priority or determining the filling sequence according to a priority method of spherical convergence; then, searching an optimal matching block according to a matching criterion based on the Manhattan distance, and filling a missing part corresponding to the current block to be filled with the optimal matching block; and finally, updating the confidence value of the filling edge by adopting a confidence updating criterion based on the Stirling theory. And circulating the steps until the area to be repaired is repaired. The invention can obtain stable filling sequence, maintain the continuity of the image structure information and the definition of the texture after the restoration, obtain the restored image meeting the visual demand of human eyes, and particularly can better restore the image with complex structure and texture information.
Description
Technical Field
The invention relates to the technical field of image restoration, in particular to an image restoration technology based on a sample block, and particularly relates to a spherical convergence sample block restoration method aiming at stable filling.
Background
Digital image restoration is also called image completion or image de-occlusion, and aims to fill a defective area by using known information around the damaged area according to a certain algorithm or rule so that the restored image looks coherent and natural. The image restoration technology is widely applied to the fields of ancient cultural relic protection, old photo and old movie restoration, movie and television special effect production, error concealment in video communication and the like, and is a research hotspot in the fields of computer vision and image processing.
Current image restoration algorithms can be divided into two broad categories: representative methods of the type suitable for repairing small-scale damage are Partial Differential Equation (PDE) based methods and sparse-based methods. The most representative method includes BSCB (simulation spread events and Bellleter) model (simulation M, Sapiro G, shells V, et. Image information [ C ]. Proceedings of the 27th annular Conference Computer Graphics and Interactive technologies, New York, USA,2000:417 @), integral part (TV) model (Shell J H and Chan T. physical models for localization information [ J. Journal application of simulation J. 3. physical models for Diffusion of local texture [ J. 3. Journal ] and Diffusion of Curvature J. 9. Journal J. 3. Diffusion of the model [ D ] 3. Journal J. 3. Journal ] and Diffusion of Curvature J. sub.1043. J. 3. Journal. J. 3. Diffusion of the model [ D ] 3. Journal. J. 3. Journal. J. 3. and Diffusion of the model [ D. Journal ] and Diffusion of the model [ D. sub.1049. K. Reproduction, 2001,12(4):436- & 449.), and the method is only suitable for repairing small-scale damage such as scratches, stains and the like. Due to the limitation of a mathematical model, the image restoration algorithm based on the structure needs to assume that a damaged area is smooth when establishing a partial differential equation, and when a missing area is large, the method can introduce a smoothing effect into the restored area to cause blurring, and the restoration time is exponentially increased.
The sparsity-based image restoration technique of document 1 (Liu J, Musialski P, Wonka P, et al. sensor completion for evaluating values in visual data [ J ]. IEEE Transactions on Pattern Analysis and Machine Analysis, 2013,35(1): 208-.
Another class is suitable for repairing large area broken images, including texture synthesis techniques based on sample block matching and image inpainting methods based on deep learning. The method based on deep learning (Yang CH, Lu X, Lin ZH, et al, high-resolution image interpolating using multi-scale neural patch synthesis [ C ]. Procedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition, Honolulu,2017:4076-4084.) utilizes neural network training data to generate missing parts according to image surrounding information, but the method has longer training time, greatly depends on Computer performance, and does not well utilize data on the whole data set.
The matching-based sample block repairing algorithm of document 2 (crimizi a, Perez P, Toyama k. region filtering and object removal by empirical-based image inpainting [ J ]. IEEE transactions on image processing, 2004,13(9): 1200) fills a region to be repaired in units of blocks, the main idea is to select a pixel point with the highest priority on the boundary of an image lost region, set a template with a certain size with the point as the center, then find a block which is the closest to the template according to a certain criterion in the whole known region, and finally fill the template with the best matching block. The method can well keep the continuity and the definition of the structure and the texture information in the repair area, but the mismatching phenomenon is easy to generate in the repair process, and the visual effect is influenced to a certain extent.
For this reason, researchers have studied sample block-based image inpainting techniques and proposed a number of improved algorithms. In a document (a chain optimization image restoration research [ J ] under the constraint of dynamic scale block matching, an electronic report, 2015,43(3): 529-; document 3 (zhang quan, shigao. an image restoration algorithm for block matching [ J ]. photoelectron, laser, 2012,23(04): 805-; the method comprises the following steps of (Lee J, Lee DK, Park RH. Robust extension-based encoding using region segmentation [ J ]. IEEE Transactions on Consumer Electronics,2012,58(2):553- > 561.) dividing an image to be repaired according to structure and texture information, and adaptively selecting a sample block size and a search range; literature (He K M and Sun J. statistics of batch offsets for image completion [ C ]. Proceedings of the 12th European Conference on Computer Vision, Florence,2012:16-29.) optimally combines a stack of shifted images to fill in missing areas using the offset of matching similar blocks; the literature (Heasabi S and Mahdavi-Amiri N.A modified patch propagation-based image mapping using patch space [ J ]. Iranian Journal of Science and Technology Engineering,2012,37(E2):43-48.) introduces gradient and divergence information and combines the Sum of the squares of the least differences (Sum of Squared differences, SSD) distances to find the best matching block. Document 4(Lee J H, Choi IC, and Kim M H. Laplacian Pattern-Based Image Synthesis [ C ] Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition, Seattle,2016: 2727-.
But the algorithm repair result is easy to generate the phenomena of structural fracture and wrong extension. The algorithm of document 1 estimates missing information based on tensor, and when an image with abundant structure and texture information is repaired, stability and robustness are insufficient. The filling order of the algorithms of documents 2,3 is unstable and the matching criterion is not reasonable. The laplacian pyramid employed by the algorithm of document 4 is not scalable, resulting in a limited search area. In the sample block-based image restoration process, a stable filling sequence is a precondition for maintaining structural continuity after image restoration. The reasonable matching criterion is the basis for finding a suitable matching block. Therefore, the reasonable filling order, the matching criterion and the confidence term updating mode are key factors for improving the quality of the sample block-based image restoration algorithm.
Disclosure of Invention
In order to keep the continuous consistency of the repaired image structure and the texture information, the invention provides a spherical convergence sample block repairing method aiming at stable filling. The method can obtain a stable filling sequence, effectively reduce the phenomena of mismatching and error accumulation, keep the structural continuity and the texture definition of the repaired image, obtain the repaired image meeting the vision of human eyes, and particularly better repair the image with complex structure and texture information.
The technical scheme adopted by the invention for solving the technical problems is as follows:
a spherical convergence sample block repairing method aiming at stable filling comprises the following steps:
step 1: initializing a region to be repaired:
marking I as the whole image, marking phi as the known area of the image, and marking the area omega to be repaired of the image with the same color. δ Ω is the boundary of the region to be repaired, the confidence value c (p) of the pixel point p in the known region Φ in the whole image i is initialized to 1, and the confidence value c (p) of the pixel point p in the unknown region Ω is initialized to 0.
Step 2: setting a segmentation threshold t for the maximum confidence value max (c):
forming a 9 x 9 block to be filled Ψ centered on each point p on the boundary δ Ω p And calculating the confidence term value; defining all blocks psi to be repaired at the current boundary p The maximum confidence term value of (2) is max (c), a threshold value t is set for the maximum confidence term value of (2), and t is 0.6; when all blocks Ψ p When the maximum confidence value max (c) is greater than a certain threshold t, the damaged area of the image is considered to be located in the structural area, and if the maximum confidence value max (c) is less than the threshold t, the area to be repaired of the image is considered to be located in the texture area, that is, the value of the maximum confidence term max (c) is used to distinguish whether the image is located in the structural area or the texture area.
And 3, step 3: determining the priority of the boundary sample block of the area to be repaired:
and determining the priority of each boundary sample block of the area to be filled according to a priority method based on spherical convergence.
When the maximum value max (C) of the confidence term is larger than the threshold value t, the priority is calculated by adopting the structure priority ratio for repairing the structural area of the image preferentially:
P(p)＝a×C(p)+b×D(p)
where a and b represent the factors that the data item dominates in priority, and a is 0.3, and b is 0.7. And C (p) is a confidence term and represents the proportion of the known pixels in the block to be repaired, and the more the known information is, the larger the value of C (p) is, the pixel block can be repaired preferentially. D (p) is a data item, the value of which depends on the angle between the direction of the isophote and the normal vector of the boundary, if the smaller the angle, the stronger the structural information of the point p is, the larger d (p) is. Pixel blocks with equal illumination lines perpendicular to the boundary of the repair area have larger values of d (p), and can be repaired preferentially:
wherein: l Ψ p I denotes Ψ p I.e. the block Ψ to be filled p The number of pixel points in. n is p A normal vector representing the p points is shown,
expressing the direction and intensity of the p-point isolux line, and the expression is
Alpha is a normalization factor and has a value of 255 for a typical gray scale image.
When the maximum value max (c) of the confidence term is smaller than the threshold t, then the filling order is determined according to the priority method of spherical convergence:
therein, Ψ p Indicates the current block to be repaired, Ψ pi Is shown at the boundary of the area to be repaired
According to the maximum geometrical distance
And selecting the next block to be repaired.
And 4, step 4: finding the best matching block:
selecting one block to be filled with the highest priority as the current block Ψ to be repaired p Searching according to the matching criterion based on the Manhattan distance in the known region phi of the whole image IFinding and repairing block psi p Most similar filling block Ψ q 。
The matching criterion based on manhattan distance is:
wherein, d M (Ψ p ,Ψ q ) Representing the Manhattan distance, m, n representing the block Ψ to be repaired p P, q denote the block Ψ to be repaired p And matching block Ψ q The pixel value of (2).
And 5: filling corresponding pixel values of the matching blocks:
the current block to be filled psi p Using the best matching block Ψ q Filling corresponding pixel values;
and 6: update fill edge confidence value:
block Ψ to be filled at the present time p After the pixel block is filled with a new pixel block, updating the confidence coefficient C (p) value of the filling edge by adopting a confidence coefficient updating criterion based on the Stirling theory;
and 7: and (5) circulating the step 3 to the step 6 until the omega repairing of the region to be repaired is completed.
Further, the priority method based on spherical convergence in step 3 is specifically:
wherein, c (p) is a confidence term representing the proportion of known pixels in the block to be repaired, and d (p) is a data term representing the intensity of the structural information in the image. The ratio a is 0.3, b is 0.7, a and b represent the data items that are dominant in priority, and the threshold t is 0.6.
When the maximum value max (c) of the confidence term is greater than the threshold t, in order to preferentially repair the structural area of the image, the data item needs to be a dominant factor in the priority, so that the priority of the block to be repaired is determined by adopting the structural priority proportion.
When the maximum value max (c) of the confidence term is smaller than the threshold t, determining the filling sequence according to the priority method of spherical convergence, specifically:
a. selecting a block psi to be repaired with p' as a central point with the highest priority in a region omega to be repaired p 'selecting psi taking p' as a central point in a region omega to be repaired according to the farthest geometric distance rule for a sample block needing to be repaired firstly p "is the next block to be repaired, i.e. such that the geometric distance between the central points p' and p" is the farthest.
b. Finding out the block psi to be repaired according to the matching criterion based on the Manhattan distance p ' best match Block Ψ q ' and fill psi p '。
c. By psi p "is the current block to be repaired, and finds the next block Ψ to be repaired in the region Ω to be repaired according to the farthest geometric distance p "', such that the geometric distance between the center points p" and p' "is the farthest.
d. Finding out the block psi to be repaired according to the matching criterion based on the Manhattan distance p "best matching Block Ψ q ", simultaneously fill Ψ p ”。
e. And repeating the process until the omega repairing of the region to be filled is completed.
For large-area and small-scale damaged images, the structural part of the image can be repaired preferentially by adopting a spherical convergence priority method. And when the confidence value is smaller than the set threshold value, the residual texture and the smooth area in the large-area damaged image are repaired according to the spherical convergence repairing process. For small-scale damaged images, after the structural information is repaired preferentially, a plurality of unconnected blocks can be formed in the residual damaged area, and all the blocks are repaired as a whole by adopting the spherical convergence priority method.
Thus, the priority of the blocks to be filled is determined by the priority method of spherical convergence, and more reasonable filling sequence can be obtained. Because the spherical convergence priority calculation method can preferentially repair the structural part of the image, the integrity of the structural information of the image is maintained. Meanwhile, the texture and the smooth information of the damaged area are gradually repaired inwards according to the spherical sequence, so that the consistency of the repaired image and neighborhood information is ensured, the phenomenon of error extension is reduced, and a better repairing result is obtained.
The confidence level updating criterion based on the stirling theory in the step 6 is as follows:
wherein, c (p) is the confidence term of the current block to be filled, and c (q) is the updated confidence term.
The Stirling equation is to estimate n! Approximate mathematical formula:
let lambda n 0.08C (p), n! The confidence term update criterion proposed based on stirling theory, therefore, is:
the edge confidence value of the damaged area is updated through the confidence updating criterion based on the Stirling theory, the rapid attenuation of the confidence term can be effectively inhibited, a reasonable filling sequence is obtained, the structural part of the image is repaired preferentially, the structural integrity and the texture clearness of the repaired image are maintained, and the better repairing quality is obtained.
Compared with the prior art, the invention has the technical effects that:
firstly, the invention determines the priority by using a priority method based on spherical convergence:
when the maximum value of the confidence item is greater than a set threshold value, the structural area of the image is repaired preferentially, so that the data item is the dominant factor in the priority to repair the structural part of the image preferentially;
and when the maximum value of the confidence term is smaller than a set threshold value, determining a filling sequence according to a priority method of spherical convergence, determining the next block to be repaired according to a farthest geometric distance rule, gradually repairing the damaged area inwards, and properly extending texture information to ensure the continuity and reasonability of the repaired image in vision.
The confidence value of the filling edge is updated by using the confidence updating criterion based on the Stirling theory, so that the rapid decline of the confidence item is effectively inhibited, a more stable filling order is obtained, the structural part of the image is preferentially repaired, the structural continuity and integrity are better maintained, and a better repairing result is obtained.
Therefore, the method of the invention adopts a priority method based on spherical convergence to determine the priority of the blocks to be repaired, can obtain a stable filling sequence, repairs the structural part of the image by priority, simultaneously repairs the damaged area inwards gradually according to the spherical sequence, and extends the texture information moderately to ensure the continuity and the texture definition of the structural information of the repaired image: and the confidence term is updated by adopting the confidence criterion based on the Stirling theory, so that the attenuation speed of the confidence term is slowed down, the consistency of the repaired image and neighborhood information is well maintained, and the repaired image which is natural and continuous and meets the visual requirements of human eyes better is obtained.
Drawings
FIG. 1 is a schematic diagram of a spherical convergence repair process used by embodiments of the present invention;
FIG. 2 is a graph comparing the confidence update function of Stirling theory with that of the algorithm of reference 2;
FIG. 3 is a schematic diagram of the healing effect of the confidence term update algorithm based on Stirling theory used in an embodiment of the present invention;
wherein, column a of fig. 3 and column b of fig. 3 are an original image and an image to be repaired, respectively, column c of fig. 3 is an effect diagram of the algorithm of document 2 (the following documents are all references mentioned in the background art) after the column b of fig. 3 is repaired, and column d of fig. 3 is an effect diagram of the method of the present invention after the column b of fig. 3 is repaired;
FIG. 4 is a graph of confidence values for all points on line 140 of FIGS. 3c and 3 d;
FIG. 5 is a test chart of t, a, b;
FIG. 6 is a diagram illustrating the effect of t on PSNR;
FIG. 7 is a schematic diagram illustrating the effect of a value on PSNR;
FIG. 8 is a diagram showing the results of repairing small-scale damaged images (scratches and blocks) by using the algorithms of documents 1, 2 and 3 and the method of the present invention;
wherein, the column a of fig. 8 and the column b of fig. 8 are an original image and an image to be repaired, respectively, the small rectangular frame in the figure is a local repair area, and the rectangular frame at the corner is an enlarged image of the small rectangular frame in the figure;
column c of fig. 8 is a graph showing the effect of the algorithm of document 1 after repairing column b of fig. 8;
column d of fig. 8 is a graph showing the effect of the algorithm of reference 2 on the column b of fig. 8 after the repair, column e of fig. 8 is a graph showing the effect of the algorithm of reference 3 on the column b of fig. 8 after the repair, and column f of fig. 8 is a graph showing the effect of the method of the present invention on the column b of fig. 8 after the repair;
FIG. 9 is a diagram illustrating the results of repairing a large-area damaged image by using the algorithms of references 2,3 and 4 and the method of the present invention; wherein, the column a of fig. 9 and the column b of fig. 9 are the original image and the image to be restored, respectively. Fig. 9 c is a graph showing the effect of the algorithm of reference 2 on the column b of fig. 9 after the repair, fig. 9d is a graph showing the effect of the algorithm of reference 3 on the column b of fig. 9 after the repair, fig. 9 e is a graph showing the effect of the algorithm of reference 4 on the column b of fig. 9 after the repair, and fig. 9 f is a graph showing the effect of the method of the present invention on the column b of fig. 9 after the repair;
FIG. 10a is an enlarged contrast view of the image of FIG. 8 taken at position 5. The column a of fig. 10a and the column b of fig. 10a are the original image and the image to be repaired, respectively, the small rectangular frame in the figure is the local repair area, the rectangular frame at the corner is an enlarged view of the small rectangular frame in the figure, the column c of fig. 10a is the effect diagram after the column b of fig. 10a is repaired by using the algorithm of document 1, the column d of fig. 10a is the effect diagram after the column b of fig. 10a is repaired by using the algorithm of document 2, the column e of fig. 10a is the effect diagram after the column b of fig. 10a is repaired by using the algorithm of document 3, and the column f of fig. 10a is the effect diagram after the column b of fig. 10a is repaired by using the method of the present invention;
FIG. 10b is an enlarged contrast view of the 3 rd image taken from FIG. 9. Wherein, the column a of fig. 10b and the column b of fig. 10b are the original image and the image to be repaired, respectively. Fig. 10b, c, d, e and f show the effect of the algorithm of document 2 after the b-column of fig. 10b is repaired, the algorithm of document 3 after the b-column of fig. 10b is repaired, the algorithm of document 4 after the b-column of fig. 10b is repaired, and the method of the present invention after the b-column of fig. 10b is repaired.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
In a specific embodiment of the present invention, a spherical convergence sample block repairing method for stable padding includes the following steps:
step 1: initializing a region to be repaired:
marking I as the whole image, marking phi as the known area of the image, and marking the area omega to be repaired of the image with the same color. δ Ω is the boundary of the region to be repaired, the confidence value c (p) of the pixel point p in the known region Φ in the whole image i is initialized to 1, and the confidence value c (p) of the pixel point p in the unknown region Ω is initialized to 0.
Step 2: setting a segmentation threshold t for the maximum confidence value max (c):
forming a 9 x 9 block to be filled Ψ centered on each point p on the boundary δ Ω p And calculating the confidence term value thereof; defining all blocks psi to be repaired at the current boundary p Has a maximum confidence term value of max (c) and sets a threshold value t, t being 0.6; when all blocks Ψ p When the maximum confidence value max (c) is greater than a certain threshold t, the damaged area of the image is considered to be located in the structure area, and if the maximum confidence value max (c) is less than the threshold t, the area to be repaired of the image is considered to be located in the texture area, that is, the value of the maximum confidence term max (c) is used to distinguish whether the image is located in the structure area or the texture area.
And step 3: determining the priority of the boundary sample block of the area to be repaired:
determining the priority of each boundary sample block of the area to be filled according to a priority method based on spherical convergence, which is specifically implemented by the following steps:
when the maximum value max (C) of the confidence term is larger than a threshold value t, the priority is calculated by adopting a structure priority ratio for repairing the structural area of the image preferentially:
P(p)＝a×C(p)+b×D(p)
where a is 0.3 and b is 0.7, a and b represent the dominant factors in the priority of the data item. And C (p) is a confidence term and represents the proportion of the known pixels in the block to be repaired, and the more the known information is, the larger the value of C (p) is, the pixel block can be repaired preferentially. D (p) is a data item, the value of which depends on the angle between the direction of the isophote and the normal vector of the boundary, if the smaller the angle, the stronger the structural information of the point p is, the larger d (p) is. Pixel blocks with equal illumination lines perpendicular to the boundary of the repair area have larger values of d (p), and can be repaired preferentially:
wherein: l Ψ p I denotes Ψ p I.e. the block Ψ to be filled p The number of pixel points in. n is p A normal vector representing the p points is shown,
expressing the direction and intensity of the p-point isolux line, and the expression is
α is a normalization factor, pairIn a typical gray scale image, the value is 255.
When the maximum value max (c) of the confidence term is smaller than the threshold t, then the filling order is determined according to the priority method of spherical convergence:
therein, Ψ p Indicates the current block to be repaired,
is shown at the boundary of the region to be repaired
According to the farthest geometric distance
And selecting the next block to be repaired.
The specific method comprises the following steps:
a. selecting a block psi to be repaired with p' as a central point with the highest priority in a region omega to be repaired p 'selecting psi taking p' as a central point in a region omega to be repaired according to the farthest geometric distance rule for a sample block needing to be repaired firstly p "is the next block to be repaired, i.e. such that the geometric distance between the central points p' and p" is the farthest.
b. Finding out the block psi to be repaired according to the matching criterion based on the Manhattan distance p Best match block Ψ of ` q ', and fill psi p '。
c. By psi p "is the current block to be repaired, and finds the next block Ψ to be repaired in the region Ω to be repaired according to the farthest geometric distance p "', such that the geometric distance between the center points p" and p' "is the farthest.
d. Finding out the block psi to be repaired according to the matching criterion based on the Manhattan distance p "best matching Block Ψ q ", simultaneously fill Ψ p ”。
e. And repeating the process until the omega repairing of the region to be filled is completed.
Fig. 1 is a schematic diagram of a spherical convergence process. In FIG. 1a, Ω denotes the irregular region to be repaired, Ψ, remaining after spherical convergence to preferentially repair the image structure p ' represents a block to be repaired with p ' as a central point, and psi with p ' as a central point is selected from omega according to a furthest geometric distance rule p "for the next block to be repaired, i.e. the geometric distance between the central points p' and p" is the farthest, find the best matching block to fill psi according to the SSD criterion p '; then by Ψ p "is the current block to be repaired, and finds the next block to be repaired psi with the farthest geometric distance p "', filling Ψ simultaneously p And repeating the process until the area to be filled is repaired. For different types of damaged images, the structural part of the image can be repaired preferentially by adopting a spherical convergence priority method, and when the confidence value is smaller than a set threshold value, the residual texture and smooth area in the large-area damaged image is repaired according to the spherical convergence repair process shown in fig. 1. For small-scale damaged images, after the structural information is repaired preferentially, a plurality of unconnected blocks are formed in the residual damaged area, and all the blocks are repaired as a whole by the priority method of spherical convergence.
And 4, step 4: finding the best matching block:
selecting one block to be filled with the highest priority as the current block Ψ to be repaired p Finding and repairing block psi in the known area phi of the whole image I according to the matching criterion based on the Manhattan distance p Most similar filling block Ψ q 。
The matching criterion based on manhattan distance is:
wherein, d M (Ψ p ,Ψ q ) Representing the Manhattan distance, m, n representing the block Ψ to be repaired p P, q denote the block Ψ to be repaired p And matching block Ψ q The pixel value of (2).
And 5: filling corresponding pixel values of the matching blocks:
will be currently filled block Ψ p Using the best matching block Ψ q Filling the corresponding pixel value;
step 6: update fill edge confidence value:
block Ψ to be filled at the present time p After being filled by a new pixel block, the confidence coefficient C (p) value of the filling edge is updated by adopting the confidence coefficient updating criterion based on the Stirling theory:
wherein, c (p) is the confidence term of the current block to be filled, and c (q) is the updated confidence term.
A conventional matching-based sample block image inpainting algorithm uses the formula c (q) ═ c (p) to update the confidence term, and replaces the value of c (p) in the formula with x, and its corresponding confidence update function can be regarded as f (x) ═ x, x ∈ [0,1 ]. When x changes from 1 to 0, it is a straight line with a slope that decreases by 45 °.
Stirling theory is based on estimating n! Mathematical formula of approximation:
let lambda be n 0.08C (p), n! The confidence term update criterion proposed based on stirling theory is therefore:
the confidence update function based on Stirling theory can be considered as x replacing the value of C (p) in the update criterion
The confidence level update function of document 2 is f (x) x, where x ∈ [0,1 ∈ x]. Comparing the function images of the two shows that h (x) monotonically increases over the interval (0,1) and both values are greater than f (x), indicating that the rate of decrease of h (x) is slower than f (x), and the rate of decay of the confidence term can be suppressed. As shown in fig. 2.
Fig. 3 is a schematic diagram of the repairing effect of the confidence term updating rule based on the stirling theory, and it can be seen from fig. 4 that the confidence values of 225 th column to 275 th column in the 140 th row of the image repaired by the algorithm of document 2 all approach to 0, resulting in that the structural part in fig. 3c cannot be completely repaired. By utilizing the confidence updating function based on the Stirling theory, the confidence value of the 140 th row of the repaired image is not close to 0 any more. Comparing fig. 3c and fig. 3d, it can be seen that: compared with the algorithm of the document 2, the method effectively inhibits the phenomenon that the confidence term is rapidly attenuated to 0, obtains more stable filling sequence, maintains structural continuity and integrity, and obtains more excellent repairing effect.
And 7: and (6) circulating the step 3 to the step 6 until the omega repair of the region to be repaired is completed.
In particular, the values of t in step 2 and a and b in step 3 are empirical data obtained from experiments and verified by a large number of experiments, as follows:
FIG. 5 is a test chart for discussing the effect of threshold t and proportional values a, b on repair performance, and repairing small-scale damage, i.e. scratches in FIGS. 5 a-c, blocks in FIG. 5d, and text in FIG. 5 e;
the impact of the test t varying from 0.1 to 0.9 on the repair effect of the broken image in fig. 5 is shown in fig. 6 for PSNR values. As can be seen from fig. 6a, the PSNR value of the repaired image varies little as t gradually increases. As t continues to increase, the PSNR value shows a downward trend. As can be seen from fig. 6b, the average PSNR takes a larger value when t is 0.6. Therefore, the threshold value t is 0.6.
The effect of varying a from 0 to 1 on the repair effect of the damaged images 5a to e when a is different was tested, and PSNR values are shown in fig. 7. As can be seen from fig. 7a, the PSNR of the restored image takes a large value at a e (0.3,0.4), and the PSNR values in this interval do not differ much. As can be seen from the change curve of the average PSNR value in fig. 7b, the average PSNR value is maximum when t is 0.6. Thus, if a is 0.3, b is 0.7.
To sum up, t is 0.6, a is 0.3, and b is 0.7 to maintain structural coherence, texture clarity, and consistency with neighborhood information.
Simulation experiment:
the following is a simulation experiment of image inpainting. Simulation experiments can fully show that the repairing effect of the method is superior to that of other repairing methods. In simulation experiment, setting parameter a to 0.3, b to 0.7, t to 0.6, and block Ψ to be filled p And the best matching block Ψ q Is 9 x 9. In the following simulation experiments, the repair results obtained by the method of the present invention were compared with repair results obtained by other repair methods.
The repair result pair for small-scale damaged images (scratches and blocks) is shown in fig. 8, in which a small rectangular frame is a local repair area, and rectangular frames at the corners are enlarged views of the small rectangular frame. Wherein, column a of fig. 8 and column b of fig. 8 are the original image and the image to be repaired, column c of fig. 8 is the effect diagram after the algorithm of document 1 is used for repairing column b of fig. 8, column d of fig. 8 is the effect diagram after the algorithm of document 2 is used for repairing column b of fig. 8, column e of fig. 8 is the effect diagram after the algorithm of document 3 is used for repairing column b of fig. 8, and column f is the effect diagram after the algorithm of the invention is used for repairing column b of fig. 8.
Comparing the various graphs in fig. 8 can be seen: compared with the algorithms in documents 1, 2 and 3, the method obtains a more stable filling sequence, maintains the structural continuity, the texture definition and the consistency with neighborhood information, and obtains a restored image which meets the visual requirements of human eyes better.
The calculation shows that the repair result of document 1, i.e. the peak signal-to-noise ratio of column c of fig. 8, is: 31.40dB, 31.44dB, 36.75dB, 31.96dB and 43.94 dB. The repair results of document 2, i.e., the peak snr of column d of fig. 8, are: 40.06 dB, 32.73dB, 38.57dB, 34.01dB and 50.91 dB. The restoration results of document 3, i.e., the peak signal-to-noise ratios in column e of fig. 8 are: 38.04dB, 33.04dB, 37.30dB, 33.65dB and 49.29 dB. The repairing result of the method of the invention, namely the peak signal-to-noise ratio of the f column of fig. 8, is respectively as follows: 40.85dB, 39.85dB, 39.22dB, 35.43dB, 54.81 dB. The peak signal-to-noise ratio value of the method is at least 2.47dB higher than that of the algorithm of the document 1, at least 0.65dB higher than that of the algorithm of the document 2, and at least 1.78dB higher than that of the algorithm of the document 3.
Therefore, the method is superior to the schemes provided by the documents 1, 2 and 3 in the aspects of subjective visual effect and objective evaluation index. The solutions provided in documents 1, 2 and 3 are advanced in the industry.
The repair result pair for a large area broken image is shown in fig. 9. Wherein, the column a of fig. 9 and the column b of fig. 9 are the original image and the image to be restored, respectively. Column c of fig. 9 is an effect diagram after the column b of fig. 9 is repaired by the algorithm of document 2, column d of fig. 9 is an effect diagram after the column b of fig. 9 is repaired by the algorithm of document 3, column e of fig. 9 is an effect diagram after the column b of fig. 9 is repaired by the algorithm of document 4, and column f of fig. 9 is an effect diagram after the column b of fig. 9 is repaired by the method of the present invention.
Comparing the various graphs in fig. 9 can be seen: compared with the algorithms of documents 2,3 and 4, the method of the invention obtains a more reasonable filling sequence of the blocks to be repaired, better maintains the integrity of the structural part and properly extends the texture information, thereby obtaining a better-quality repaired image.
An enlarged comparative graph of the repair result of the 5 th image in fig. 8 is shown in fig. 10 a. The column a of fig. 10a and the column b of fig. 10a are the original image and the image to be repaired, respectively, the small rectangular frame in the figure is the local repair area, the rectangular frame at the corner is an enlarged view of the small rectangular frame in the figure, the column c of fig. 10a is the effect diagram after the column b of fig. 10a is repaired by using the algorithm of document 1, the column d of fig. 10a is the effect diagram after the column b of fig. 10a is repaired by using the algorithm of document 2, the column e of fig. 10a is the effect diagram after the column b of fig. 10a is repaired by using the algorithm of document 3, and the column f of fig. 10a is the effect diagram after the column b of fig. 10a is repaired by using the method of the present invention.
From the enlarged view at the corners in fig. 10a it can be seen that: when the scratches on the back of the person in the image are repaired, the repairing results of documents 1, 2, and 3 all show texture extension phenomena of different degrees, fail to maintain the integrity of the curve, and show a blurring phenomenon. The method of the invention keeps the continuity of the curve and the clearness of the texture. The repairing result of the method is superior to that of documents 1, 2 and 3, and the image with clear texture and coherent structure is obtained.
An enlarged contrast of the healing effect of the 3 rd image in fig. 9 is shown in fig. 10 b. Wherein, the column a of fig. 10b and the column b of fig. 10b are the original image and the image to be restored, respectively. Column c of fig. 10b is an effect diagram after the column b of fig. 10b is repaired by the algorithm of document 2, column d of fig. 10b is an effect diagram after the column b of fig. 10b is repaired by the algorithm of document 3, column e of fig. 10b is an effect diagram after the column b of fig. 10b is repaired by the algorithm of document 4, and column f of fig. 10b is an effect diagram after the column b of fig. 10b is repaired by the method of the present invention.
As can be seen from the block diagram of fig. 10 b: the repair result of document 2 includes "trash" due to accumulation of errors, and does not satisfy the visual requirement of human eyes. The mismatch phenomenon occurs in the repair effect of document 3, resulting in the structure extending into the texture portion. The experimental result of document 4 shows a phenomenon of structural fracture. The method of the invention keeps the integrity and the continuity of the structural part, obtains the repairing effect of clear texture and consistency with neighborhood information, and obtains a better result compared with the algorithms of documents 2,3 and 4.
The simulation experiment results show that the method is superior to the existing method in the aspects of visual effect and objective evaluation index, can obtain more reasonable filling sequence and better matching blocks, effectively reduces the mismatching rate and error accumulation phenomenon, keeps the structural integrity and texture clearness of the repaired image, and has feasibility and superiority in the field of image repair.
Furthermore, it should be understood that although the present description refers to embodiments, not every embodiment may contain only a single embodiment, and such description is for clarity only, and those skilled in the art should integrate the description, and the embodiments may be combined as appropriate to form other embodiments understood by those skilled in the art.
Claims (6)
1. A spherical convergence sample block repairing method aiming at stable filling is characterized by comprising the following steps:
step S1: initializing a region to be repaired:
step S2: setting a segmentation threshold t for the maximum confidence value max (c):
step S3: determining the priority of the boundary sample block of the area to be repaired:
step S4: finding the best matching block:
step S5: filling corresponding pixel values of the matching blocks:
step S6: update fill edge confidence value:
step S7: looping the step S3 to the step S6 until the omega repair of the region to be repaired is completed;
in step S3, the specific steps are as follows:
determining the priority of each boundary sample block of the area to be filled according to a priority method based on spherical convergence, wherein the priority method based on spherical convergence is as follows:
wherein the values of a and b are: a is 0.3, b is 0.7, a and b represent the data item dominant factor in priority, and threshold t is 0.6;
c (p) is a confidence term and represents the proportion of known pixels in the block to be repaired, and the more the known information is, the larger the value of C (p) is, the pixel block can be repaired preferentially;
d (p) is a data item, the value of the data item depends on the included angle between the direction of the isolux line and the normal vector of the boundary, if the included angle is smaller, the structural information of the point p is stronger, the D (p) is larger, and the pixel block of the isolux line vertical to the boundary of the repair area has a larger value D (p), so that the repair is carried out preferentially;
Ψ p representing a current block to be repaired;
when the maximum value max (C) of the confidence item is larger than the threshold value t, calculating the priority by adopting the structure priority ratio for repairing the structure area of the image preferentially, so that the data item is the dominant factor in the priority;
when the maximum value max (C) of the confidence term is smaller than the threshold value t, determining the filling sequence according to the priority method of spherical convergence;
in the above formula, the first and second carbon atoms are,
in the formula: l Ψ p I denotes Ψ p I.e. the block Ψ to be filled p The number of middle pixel points;
n p a normal vector representing the p points is shown,
α is a normalization factor, which is 255 for a typical grayscale image;
the method for determining the filling order according to the priority method of spherical convergence comprises the following steps:
a. selecting a block psi to be repaired with p' as a central point with the highest priority in a region omega to be repaired p 'selecting psi taking p' as a central point in a region omega to be repaired according to the farthest geometric distance rule for a sample block needing to be repaired firstly p Is the nextThe block to be repaired is the block with the longest geometric distance between the central points p 'and p';
b. finding out the block psi to be repaired according to the matching criterion based on the Manhattan distance p Best match block Ψ of ` q ', and fill psi p '；
c. By psi p "is the current block to be repaired, and finds the next block Ψ to be repaired in the region Ω to be repaired according to the farthest geometric distance p "', such that the geometric distance between the center points p" and p' "is farthest;
d. finding out the block psi to be repaired according to the matching criterion based on the Manhattan distance p "best matching Block Ψ q ", simultaneously fill Ψ p ”；
e. Repeating the process until the omega repair of the region to be filled is completed;
for different types of damaged images, adopting a priority method of spherical convergence to repair the structural part of the image preferentially, and when the maximum value max (C) of the confidence term is smaller than a set threshold t, performing the residual texture and smooth area in the large-area damaged image according to the spherical convergence repairing process; and for small-scale damaged images, after the structural information is repaired preferentially, a plurality of unconnected blocks are formed in the residual damaged area, and all the blocks are repaired as a whole by using the spherical convergence priority method.
2. The spherical convergence sample block repairing method for stable filling according to claim 1, wherein the step S1 is as follows: marking I as the whole image, phi as the known region of the image, and marking the region omega to be repaired of the image with the same color; δ Ω is the boundary of the region to be repaired, the confidence value c (p) of the pixel point p in the known region Φ in the whole image i is initialized to 1, and the confidence value c (p) of the pixel point p in the unknown region Ω is initialized to 0.
3. The spherical convergence sample block repairing method for stable filling according to claim 1, wherein the step S4 is as follows:
selecting the highest priorityOne block to be filled is the current block psi to be repaired p Finding out and repairing block psi in the known area phi of the whole image I according to the matching criterion based on the Manhattan distance p Most similar filling block Ψ q ；
The matching criterion based on manhattan distance is:
wherein d is M (Ψ p ,Ψ q ) Representing the Manhattan distance, m, n representing the block Ψ to be repaired p P, q denote the block Ψ to be repaired p And matching block Ψ q The pixel value of (2).
4. The spherical convergence sample block repairing method for stable filling according to claim 1, wherein the step S5 is as follows: the current block to be filled psi p Using the best matching block Ψ q Is filled in with the corresponding pixel value.
5. The spherical convergence sample block repairing method for stable filling according to claim 1, wherein the step S6 is as follows: block Ψ to be filled at the present time p After being filled with a new pixel block, the confidence c (p) value of the filled edge is updated using the confidence update criterion based on stirling theory.
6. The spherical convergent sample block repairing method for stable filling according to claim 1, wherein the confidence updating criterion based on the Stirling theory in the step S6 is as follows:
wherein, c (p) is the confidence term of the current block to be filled, and c (q) is the updated confidence term;
the Stirling equation is to estimate n! Approximate mathematical formula:
let lambda n 0.08C (p), n! The confidence term update criterion proposed based on stirling theory is therefore:
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910111792.2A CN109829867B (en) | 2019-02-12 | 2019-02-12 | Spherical convergence sample block repairing method for stable filling |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910111792.2A CN109829867B (en) | 2019-02-12 | 2019-02-12 | Spherical convergence sample block repairing method for stable filling |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109829867A CN109829867A (en) | 2019-05-31 |
| CN109829867B true CN109829867B (en) | 2022-09-09 |
Family
ID=66863509
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910111792.2A Expired - Fee Related CN109829867B (en) | 2019-02-12 | 2019-02-12 | Spherical convergence sample block repairing method for stable filling |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109829867B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110298798B (en) * | 2019-06-20 | 2021-02-19 | 浙江工业大学 | Image restoration method based on low-rank tensor completion and discrete total variation |
| CN111754426B (en) * | 2020-06-10 | 2022-11-29 | 天津大学 | Automatic restoration method for mural shedding disease based on genetic algorithm |
| CN111724320B (en) * | 2020-06-19 | 2021-01-08 | 北京波谱华光科技有限公司 | Blind pixel filling method and system |
| CN112070696A (en) * | 2020-09-07 | 2020-12-11 | 上海大学 | Image restoration method and system based on texture and structure separation, and terminal |
| CN117274148A (en) * | 2022-12-05 | 2023-12-22 | 魅杰光电科技(上海)有限公司 | Unsupervised wafer defect detection method based on deep learning |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103210301A (en) * | 2010-09-13 | 2013-07-17 | Mks仪器股份有限公司 | Monitoring, detecting and quantifying chemical compounds in sample |
| CN103577707A (en) * | 2013-11-15 | 2014-02-12 | 上海交通大学 | Robot failure diagnosis method achieved by multi-mode fusion inference |
| CN104282000A (en) * | 2014-09-15 | 2015-01-14 | 天津大学 | Image repairing method based on rotation and scale change |
| WO2013186543A3 (en) * | 2012-06-11 | 2015-02-26 | University Of Bath | Method of coding video transmissions |
| CN104484866A (en) * | 2014-12-15 | 2015-04-01 | 天津大学 | Image inpainting method based on rotation and scale space expansion |
| EP3343516A1 (en) * | 2017-01-03 | 2018-07-04 | Thomson Licensing | Method and device for applying an effect of an augmented or mixed reality application |
Family Cites Families (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6987520B2 (en) * | 2003-02-24 | 2006-01-17 | Microsoft Corporation | Image region filling by exemplar-based inpainting |
| US8019171B2 (en) * | 2006-04-19 | 2011-09-13 | Microsoft Corporation | Vision-based compression |
| CN101635047A (en) * | 2009-03-25 | 2010-01-27 | 湖南大学 | Texture synthesis and image repair method based on wavelet transformation |
| CN101661613B (en) * | 2009-08-27 | 2011-11-09 | 北京交通大学 | Image restoration method based on image segmentation, and system therefor |
| US9145140B2 (en) * | 2012-03-26 | 2015-09-29 | Google Inc. | Robust method for detecting traffic signals and their associated states |
| CN102760285A (en) * | 2012-05-31 | 2012-10-31 | 河海大学 | Image restoration method |
| US20140272914A1 (en) * | 2013-03-15 | 2014-09-18 | William Marsh Rice University | Sparse Factor Analysis for Learning Analytics and Content Analytics |
| CN103176088B (en) * | 2013-03-18 | 2015-09-02 | 北京航空航天大学 | The defining method in the weak path of electromagnetic coupled between a kind of multitone jamming pair |
| CN104376535B (en) * | 2014-11-04 | 2018-02-23 | 徐州工程学院 | A kind of rapid image restorative procedure based on sample |
| CN104680492B (en) * | 2015-03-11 | 2017-07-28 | 浙江工业大学 | Image repair method based on composition of sample uniformity |
| CN106023089B (en) * | 2016-01-19 | 2018-11-13 | 河南理工大学 | A kind of image repair method based on Block- matching |
| FR3053509B1 (en) * | 2016-06-30 | 2019-08-16 | Fittingbox | METHOD FOR OCCULATING AN OBJECT IN AN IMAGE OR A VIDEO AND ASSOCIATED AUGMENTED REALITY METHOD |
| CN106204503B (en) * | 2016-09-08 | 2018-11-09 | 天津大学 | Based on the image repair algorithm for improving confidence level renewal function and matching criterior |
-
2019
- 2019-02-12 CN CN201910111792.2A patent/CN109829867B/en not_active Expired - Fee Related
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103210301A (en) * | 2010-09-13 | 2013-07-17 | Mks仪器股份有限公司 | Monitoring, detecting and quantifying chemical compounds in sample |
| WO2013186543A3 (en) * | 2012-06-11 | 2015-02-26 | University Of Bath | Method of coding video transmissions |
| CN103577707A (en) * | 2013-11-15 | 2014-02-12 | 上海交通大学 | Robot failure diagnosis method achieved by multi-mode fusion inference |
| CN104282000A (en) * | 2014-09-15 | 2015-01-14 | 天津大学 | Image repairing method based on rotation and scale change |
| CN104484866A (en) * | 2014-12-15 | 2015-04-01 | 天津大学 | Image inpainting method based on rotation and scale space expansion |
| EP3343516A1 (en) * | 2017-01-03 | 2018-07-04 | Thomson Licensing | Method and device for applying an effect of an augmented or mixed reality application |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109829867A (en) | 2019-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109829867B (en) | Spherical convergence sample block repairing method for stable filling | |
| CN109712067B (en) | Virtual viewpoint drawing method based on depth image | |
| CN107967673B (en) | Self-adaptive sample block image restoration method | |
| US7379583B2 (en) | Color segmentation-based stereo 3D reconstruction system and process employing overlapping images of a scene captured from viewpoints forming either a line or a grid | |
| Jiao et al. | Local stereo matching with improved matching cost and disparity refinement | |
| CN110866924A (en) | Line structured light center line extraction method and storage medium | |
| CN109584282B (en) | Non-rigid image registration method based on SIFT (scale invariant feature transform) features and optical flow model | |
| CN108346135A (en) | A kind of improved Criminisi image repair methods | |
| CN109345525B (en) | Ghost-removed high-dynamic-range image quality evaluation method | |
| CN110751680A (en) | Image processing method with fast alignment algorithm | |
| CN113947538A (en) | Multi-scale efficient convolution self-attention single image rain removing method | |
| Zarif et al. | Image completion: Survey and comparative study | |
| Ling et al. | Quality assessment for synthesized view based on variable-length context tree | |
| Wu et al. | A novel hybrid image inpainting model | |
| Wang et al. | Image inpainting based on structural tensor edge intensity model | |
| Fan et al. | Collaborative three-dimensional completion of color and depth in a specified area with superpixels | |
| CN111179190B (en) | Image restoration method based on Criminisi | |
| Lin et al. | Effective exemplar-based image inpainting using low-rank matrix completion | |
| CN109064425B (en) | Self-adaptive non-local total variation image denoising method | |
| CN110059651B (en) | Real-time tracking and registering method for camera | |
| CN111311527A (en) | Image restoration method based on content perception generation model | |
| CN112070696A (en) | Image restoration method and system based on texture and structure separation, and terminal | |
| Kawai et al. | Image Inpainting Considering Brightness Change and Spatial Locality of Textures. | |
| CN105809635A (en) | Image block matching restoration method based on line scanning | |
| Amintoosi et al. | Precise image registration with structural similarity error measurement applied to superresolution |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20220909 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |