CN113570524B - Repairing method for high reflection noise depth image - Google Patents
Repairing method for high reflection noise depth image Download PDFInfo
- Publication number
- CN113570524B CN113570524B CN202110899091.7A CN202110899091A CN113570524B CN 113570524 B CN113570524 B CN 113570524B CN 202110899091 A CN202110899091 A CN 202110899091A CN 113570524 B CN113570524 B CN 113570524B
- Authority
- CN
- China
- Prior art keywords
- depth image
- image
- repaired
- formula
- depth
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 34
- 238000005260 corrosion Methods 0.000 claims description 3
- 230000007797 corrosion Effects 0.000 claims description 3
- 230000008961 swelling Effects 0.000 claims description 3
- 229910052721 tungsten Inorganic materials 0.000 claims description 3
- 230000002159 abnormal effect Effects 0.000 abstract description 4
- 238000001514 detection method Methods 0.000 description 5
- 229910052751 metal Inorganic materials 0.000 description 3
- 239000002184 metal Substances 0.000 description 3
- 229910018487 Ni—Cr Inorganic materials 0.000 description 2
- VNNRSPGTAMTISX-UHFFFAOYSA-N chromium nickel Chemical compound [Cr].[Ni] VNNRSPGTAMTISX-UHFFFAOYSA-N 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000009792 diffusion process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/30—Assessment of water resources
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the technical field of image restoration, and particularly relates to a restoration method for a depth image with high reflection noise. The technical scheme comprises the following steps: firstly, performing an open operation on a depth image to be repaired, and subtracting an open operation result from the depth image to be repaired to obtain a depth image with background influence eliminated; then performing binarization operation through the set threshold percentage, wherein a binarization result is used as a mask of the depth image to be repaired; and finally, realizing the accurate restoration of the low-quality depth image by using an image block matching method. The method can automatically identify the abnormal depth information in the depth image, and then can effectively restore the low-quality depth image caused by the high reflection factor by an image restoration method.
Description
Technical Field
The invention belongs to the technical field of image restoration, and particularly relates to a restoration method for a depth image with high reflection noise.
Background
The three-dimensional reconstruction method based on the image is widely applied to the quality detection field of precise instruments and industrial products due to the characteristics of low hardware dependency and easy operation. However, such reconstruction methods based on optical principles are often affected by the material of the object to be detected, such as a plurality of regions with high reflection inevitably generated in the three-dimensional reconstruction process of metal and glass products, and the regions will cause abnormal reconstruction results, which seriously affects the judgment of subsequent quality inspection personnel on the quality of the product. Therefore, how to accurately eliminate these interferences and obtain high-quality three-dimensional reconstruction results for such low-quality depth images affected by high reflection is a difficult problem in the field of industrial quality detection.
The existing image restoration method mainly comprises a restoration method based on diffusion, a restoration method based on image block matching and a deep learning restoration method. The diffusion-based image restoration method mainly utilizes pixels at the periphery of the edge of the area to be restored gradually from outside to inside, the method can effectively restore a small-range missing area, but cannot effectively restore the internal structure of a larger area, and a high-reflection area usually appears in a large area form in a depth image, so that the method cannot be directly used for restoring the depth image affected by high reflection; the restoration method based on image block matching replaces the image blocks similar to the area to be restored in the original image, but the judgment standard of the similarity can influence the restoration result; the image restoration method based on the deep learning class realizes the restoration of the image through a large number of marked sample training, and although the method can obtain a good restoration effect, the method needs a large number of data samples, has higher time complexity of an algorithm and can not meet the timeliness requirement of the industrial quality detection field.
The three types of image restoration methods generally restore the traditional natural images, and the areas to be restored generally need manual marking, and for the depth images generated in the field of industrial quality detection, the manual marking can not only generate misjudgment, but also seriously affect the quality detection efficiency. Therefore, how to automatically identify the abnormal region in the low-quality depth image is a key for realizing high-quality restoration of the depth image, and the method for automatically identifying the abnormal region provided by the patent can accurately lock the region with problems in the low-quality depth image, so that the accurate restoration of the low-quality depth image is realized.
Disclosure of Invention
In order to eliminate the region of the depth image where the depth information is wrong due to high reflection, the invention aims to provide a restoration method for the depth image with high reflection noise.
The technical scheme adopted by the invention is as follows: a repair method for a high reflection noise depth image, comprising the steps of:
step 1, setting the structure of the structural element B according to the formula (1), wherein the size of the structural element B is w multiplied by w, setting the threshold selection percentage P of the depth image, setting the size of the repair image block as S multiplied by S,
wherein ,is an element in the structural element, and the value of the element is 1;
step 2, the depth image to be repaired is processed according to the formula (2)D S Carrying out opening operation on the structural element B to obtain a depth image D after the opening operation Open ，
Wherein-! Anda corrosion and swelling operator representing an image;
step 3, using depth image D to be repaired according to formula (3) S Subtracting the depth image D after the opening operation Open Obtaining a depth image D for eliminating the background N ；
D N ＝D S -D Open (3)
Step 4, statistics of depth image D in step 3 N Is a histogram of (a)A binarization threshold T is calculated according to equation (4),
wherein ,Hi Representing depth image D N The total number of pixels with a mid-gray value of i, W, H respectively represent the depth image D N Is the width and height of (2);
step 5, the depth image D is processed according to the formula (5) N Performing binarization processing according to the threshold T obtained in the step 4 to obtain a mask M of the depth image to be repaired;
step 6, obtaining a depth image D of the mask M to be repaired by utilizing the step 5 S The image block with the size of S multiplied by S is found at other positions of the image block and repaired according to the formula (6) to obtain a repaired depth image D P ，
D P ＝Inpaint(D S ,M,S×S) (6)
Wherein Inpaint (·) represents an image restoration function.
Compared with the prior art, the method and the device can automatically identify the area to be repaired in the low-quality depth image caused by high reflection, and further realize accurate repair of the high-reflection noise depth image.
Drawings
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a depth image (left) and corresponding gray scale image (right) of a nickel chromium metal sample affected by high reflectance;
FIG. 3 is an on operation of a depth image to be repaired;
FIG. 4 is a depth image to be repaired with background interference eliminated;
FIG. 5 is a mask of a percent binarization method to obtain a depth image to be repaired;
fig. 6 is a restored depth image (left) and a corresponding gray image (right).
Detailed Description
As shown in fig. 1, the repairing method for the depth image of the high reflection noise according to the embodiment includes the following steps:
step 1, setting the structure of the structural element B according to the formula (1), wherein the size of the structural element B is w multiplied by w, setting the threshold selection percentage P of the depth image, setting the size of the repair image block as S multiplied by S,
wherein ,is an element in the structural element, and the value of the element is 1;
step 2, the depth image D to be repaired is processed according to the formula (2) S Carrying out opening operation on the structural element B to obtain a depth image D after the opening operation Open ，
Wherein-! Anda corrosion and swelling operator representing an image;
step 3, using depth image D to be repaired according to formula (3) S Subtracting the depth image D after the opening operation Open Obtaining a depth image D for eliminating the background N ；
D N ＝D S -D Open (3)
Step 4, statistics of depth image D in step 3 N Is a histogram of (a)A binarization threshold T is calculated according to equation (4),
wherein ,Hi Representing depth image D N The total number of pixels with a mid-gray value of i, W, H respectively represent the depth image D N Is the width and height of (2);
step 5, the depth image D is processed according to the formula (5) N Performing binarization processing according to the threshold T obtained in the step 4 to obtain a mask M of the depth image to be repaired;
step 6, obtaining a depth image D of the mask M to be repaired by utilizing the step 5 S The image block with the size of S multiplied by S is found at other positions of the image block and repaired according to the formula (6) to obtain a repaired depth image D P ，
D P ＝Inpaint(D S ,M,S×S) (6)
Wherein Inpaint (·) represents an image restoration function.
Taking a nickel-chromium metal sample as an example object, fig. 2 is a depth image and a gray level image of the sample affected by high reflection, and according to the sample, a low-quality depth image restoration model is established, wherein the key steps are as follows:
the method in the step 2 obtains the result of the open operation of the depth image to be repaired, as shown in fig. 3;
step 3, obtaining a depth image to be repaired for eliminating background interference by the method, as shown in fig. 4;
step 4, selecting a percentage to calculate the threshold value of the depth image in the step 3 according to a preset threshold value;
step 5, obtaining a mask of the depth image to be repaired by the method, as shown in fig. 5;
and (6) obtaining a repaired depth image by the method in the step (6), as shown in fig. 6.
Sample results show that the method can well overcome the problem of depth image distortion caused by high reflection, and the restored depth image has good homogeneity, and the texture transition in the corresponding gray level image is natural.
Claims (1)
1. The repairing method for the high reflection noise depth image is characterized by comprising the following steps of:
step 1, setting the structure of the structural element B according to the formula (1), wherein the size of the structural element B is w multiplied by w, setting the threshold selection percentage P of the depth image, setting the size of the repair image block as S multiplied by S,
wherein ,is an element in the structural element, and the value of the element is 1;
step 2, the depth image D to be repaired is processed according to the formula (2) S Carrying out opening operation on the structural element B to obtain a depth image D after the opening operation Open ，
Wherein-! Anda corrosion and swelling operator representing an image;
step 3, using depth image D to be repaired according to formula (3) S Subtracting the depth image D after the opening operation Open Obtaining a depth image D for eliminating the background N ；
D N ＝D S -D Open (3)
Step 4, statistics of depth image D in step 3 N Is a histogram of (a)A binarization threshold T is calculated according to equation (4),
wherein ,Hi Representing depth image D N The total number of pixels with a mid-gray value of i, W, H respectively represent the depth image D N Is the width and height of (2);
step 5, the depth image D is processed according to the formula (5) N Performing binarization processing according to the threshold T obtained in the step 4 to obtain a mask M of the depth image to be repaired;
step 6, obtaining a depth image D of the mask M to be repaired by utilizing the step 5 S Is repaired according to the formula (6) to obtainTo the restored depth image D P ，
D P ＝Inpaint(D S ,M,S×S) (6)
Wherein Inpaint (·) represents an image restoration function.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110899091.7A CN113570524B (en) | 2021-08-06 | 2021-08-06 | Repairing method for high reflection noise depth image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110899091.7A CN113570524B (en) | 2021-08-06 | 2021-08-06 | Repairing method for high reflection noise depth image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113570524A CN113570524A (en) | 2021-10-29 |
| CN113570524B true CN113570524B (en) | 2023-09-22 |
Family
ID=78170597
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110899091.7A Active CN113570524B (en) | 2021-08-06 | 2021-08-06 | Repairing method for high reflection noise depth image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113570524B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114331915B (en) * | 2022-03-07 | 2022-08-05 | 荣耀终端有限公司 | Image processing method and electronic device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6028953A (en) * | 1995-08-22 | 2000-02-22 | Kabushiki Kaisha Toshiba | Mask defect repair system and method which controls a dose of a particle beam |
| CN110675346A (en) * | 2019-09-26 | 2020-01-10 | 武汉科技大学 | Image acquisition and depth map enhancement method and device suitable for Kinect |
| CN111311515A (en) * | 2020-02-13 | 2020-06-19 | 山西大学 | Depth image fast iterative restoration method for automatic detection of error area |
| CN113188474A (en) * | 2021-05-06 | 2021-07-30 | 山西大学 | Image sequence acquisition system for imaging of high-light-reflection material complex object and three-dimensional shape reconstruction method thereof |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW201028964A (en) * | 2009-01-23 | 2010-08-01 | Ind Tech Res Inst | Depth calculating method for two dimension video and apparatus thereof |
-
2021
- 2021-08-06 CN CN202110899091.7A patent/CN113570524B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6028953A (en) * | 1995-08-22 | 2000-02-22 | Kabushiki Kaisha Toshiba | Mask defect repair system and method which controls a dose of a particle beam |
| CN110675346A (en) * | 2019-09-26 | 2020-01-10 | 武汉科技大学 | Image acquisition and depth map enhancement method and device suitable for Kinect |
| CN111311515A (en) * | 2020-02-13 | 2020-06-19 | 山西大学 | Depth image fast iterative restoration method for automatic detection of error area |
| CN113188474A (en) * | 2021-05-06 | 2021-07-30 | 山西大学 | Image sequence acquisition system for imaging of high-light-reflection material complex object and three-dimensional shape reconstruction method thereof |
Non-Patent Citations (1)
| Title |
|---|
| 采用曲率扩散和边缘重建的深度图像空洞修复;牟琦;夏蕾;李占利;李洪安;;西安科技大学学报(第02期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113570524A (en) | 2021-10-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108961217B (en) | Surface defect detection method based on regular training | |
| CN111815601B (en) | Texture image surface defect detection method based on depth convolution self-encoder | |
| CN109767422B (en) | Pipeline detection and identification method based on deep learning, storage medium and robot | |
| CN116758491B (en) | Printing monitoring image analysis method and system applied to 3D printing | |
| Jing et al. | Fabric defect detection based on golden image subtraction | |
| CN114418899B (en) | Self-adaptive repairing method and system for self-color printing and readable storage medium | |
| CN115456956B (en) | Method, equipment and storage medium for detecting scratches of liquid crystal display | |
| CN111161260A (en) | Hot-rolled strip steel surface defect detection method and device based on deep learning | |
| CN110751606B (en) | Foam image processing method and system based on neural network algorithm | |
| CN113570524B (en) | Repairing method for high reflection noise depth image | |
| CN111681213A (en) | Light guide plate line scratch defect detection method based on deep learning | |
| CN114332008A (en) | Unsupervised defect detection and positioning method based on multi-level feature reconstruction | |
| CN111311515B (en) | Depth image rapid iterative restoration method for automatic detection of error region | |
| CN115471463A (en) | Solar cell defect detection method based on countermeasure type generation network | |
| CN117115161B (en) | Plastic defect inspection method | |
| CN117333467B (en) | Image processing-based glass bottle body flaw identification and detection method and system | |
| CN112381794B (en) | Printing defect detection method based on deep convolution generation network | |
| CN111275687B (en) | Fine-grained image stitching detection method based on connected region marks | |
| CN117314826A (en) | Performance detection method of display screen | |
| CN115797314A (en) | Part surface defect detection method, system, equipment and storage medium | |
| Liu et al. | Texture smoothing quality assessment via information entropy | |
| CN112508862B (en) | Method for enhancing magneto-optical image of crack by improving GAN | |
| Xiao et al. | 3D center segregation reconstruction of steel continuous casting slab | |
| CN117132592B (en) | Industrial defect detection method based on entropy fusion | |
| CN116681628B (en) | Business license data processing method and system based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |