CN113570524A - Restoration method for high-reflection noise depth image - Google Patents

Restoration method for high-reflection noise depth image Download PDF

Info

Publication number
CN113570524A
CN113570524A CN202110899091.7A CN202110899091A CN113570524A CN 113570524 A CN113570524 A CN 113570524A CN 202110899091 A CN202110899091 A CN 202110899091A CN 113570524 A CN113570524 A CN 113570524A
Authority
CN
China
Prior art keywords
depth image
image
restored
depth
formula
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110899091.7A
Other languages
Chinese (zh)
Other versions
CN113570524B (en
Inventor
闫涛
杜亮
钱宇华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanxi University
Original Assignee
Shanxi University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanxi University filed Critical Shanxi University
Priority to CN202110899091.7A priority Critical patent/CN113570524B/en
Publication of CN113570524A publication Critical patent/CN113570524A/en
Application granted granted Critical
Publication of CN113570524B publication Critical patent/CN113570524B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/77—Retouching; Inpainting; Scratch removal
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/30—Assessment of water resources

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

The invention belongs to the technical field of image restoration, and particularly relates to a restoration method for a high-reflection noise depth image. The technical scheme comprises the following steps: firstly, performing an opening operation on a depth image to be restored, and subtracting an opening operation result from the depth image to be restored to obtain a depth image without background influence; then, carrying out binarization operation according to the set threshold percentage, wherein a binarization result is used as a mask of the depth image to be restored; and finally, realizing the accurate restoration of the low-quality depth image by using an image block matching method. The method can automatically identify the abnormal depth information in the depth image, and then can effectively restore the low-quality depth image caused by high-reflectivity factors through the image restoration method.

Description

Restoration method for high-reflection noise depth image
Technical Field
The invention belongs to the technical field of image restoration, and particularly relates to a restoration method for a high-reflection noise depth image.
Background
The three-dimensional reconstruction method based on the image is widely applied to the field of quality detection of precision instruments and industrial products due to the characteristics of low hardware dependence and easiness in operation. However, such reconstruction methods based on optical principles are often influenced by the material of the object to be detected, for example, some high-light-reflection regions are inevitably generated in the three-dimensional reconstruction process of metal and glass products, and these regions cause the reconstruction result to be abnormal, thereby seriously influencing the judgment of subsequent quality inspectors on the product quality. Therefore, for such low-quality depth images affected by high light reflection, how to accurately eliminate the interference and obtain a high-quality three-dimensional reconstruction result is a difficult problem in the field of industrial quality detection.
The existing image restoration method mainly comprises a restoration method based on diffusion, a restoration method based on image block matching and a deep learning type restoration method. The image restoration method based on diffusion mainly utilizes pixels on the periphery of the edge of a region to be restored from outside to inside to restore gradually, the method can effectively restore a small-range missing region, but cannot effectively restore the internal structure of a large region, and a high-light-reflection region usually appears in a large-block region form in a depth image, so the method cannot be directly used for restoring the depth image affected by high light reflection; the restoration method based on image block matching is used for searching image blocks similar to the area to be restored in an original image for replacement, but the judgment standard of the similarity possibly influences the restoration result; the image restoration method based on the deep learning class realizes the restoration of the image through a large amount of marked samples, and although the method can obtain a good restoration effect, the method needs a large amount of data samples and has high time complexity of algorithm, so that the timeliness requirement in the field of industrial quality detection cannot be met.
The three types of image restoration methods usually restore a traditional natural image, and an area to be restored usually needs to be marked manually, and for a depth image generated in the field of industrial quality detection, the manual marking may not only cause misjudgment, but also seriously affect the quality detection efficiency. Therefore, how to automatically identify the abnormal region in the low-quality depth image is the key to realize the high-quality restoration of the depth image, and the method for automatically identifying the abnormal region can accurately lock the region with problems in the low-quality depth image, so as to realize the accurate restoration of the low-quality depth image.
Disclosure of Invention
In order to eliminate the area of the depth image with wrong depth information caused by high light reflection, the invention aims to provide a repairing method for the depth image with high light reflection noise.
The technical scheme adopted by the invention is as follows: a restoration method for a high reflection noise depth image comprises the following steps:
step 1, setting the structure of a structural element B according to the formula (1), wherein the size of the structural element B is w multiplied by w, setting the threshold selection percentage P of a depth image, setting the size of a repair image block to be S multiplied by S,
Figure BDA0003199040930000021
wherein ,
Figure BDA0003199040930000022
is an element in the structural element, and the value of the element is 1;
step 2, according to the formula (2), the depth image D to be restoredSOpening operation is carried out on the structural element B to obtain a depth image D after opening operationOpen，
Figure BDA0003199040930000023
Wherein! And
Figure BDA0003199040930000024
erosion and dilation operators representing images;
step 3, using the depth image D to be restored according to the formula (3)SSubtracting the on-operated depth image DOpenObtaining a depth image D with background eliminatedN；
DN＝DS-DOpen (3)
Step 4, counting the depth image D in the step 3NHistogram of (1)
Figure BDA0003199040930000025
A binarization threshold value T is calculated according to equation (4),
Figure BDA0003199040930000026
wherein ,HiRepresenting a depth image DNThe total number of pixels having a middle gray value of i, W and H each representing depthImage DNWidth and height of (d);
step 5, the depth image D is processed according to the formula (5)NCarrying out binarization processing according to the threshold value T obtained in the step 4 to obtain a mask M of the depth image to be restored;
Figure BDA0003199040930000027
step 6, obtaining the depth image D of the mask M to be repaired in the step 5SThe image blocks with the size of S multiplied by S are searched for other positions and are repaired according to the formula (6), and the repaired depth image D is obtainedP，
DP＝Inpaint(DS,M,S×S) (6)
Here, Inpaint (·) represents an image repair function.
Compared with the prior art, the method can automatically identify the area to be repaired in the low-quality depth image caused by high reflection, so that the high-reflection noise depth image can be accurately repaired.
Drawings
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a depth image (left) and corresponding grayscale image (right) of a nickel-chromium metal specimen affected by high reflectance;
FIG. 3 is an opening operation of a depth image to be restored;
FIG. 4 is a depth image to be restored with background interference removed;
FIG. 5 is a mask of a depth image to be restored obtained by the percentage binarization method;
fig. 6 shows the restored depth image (left) and the corresponding grayscale image (right).
Detailed Description
As shown in fig. 1, the method for repairing a depth image with high reflective noise according to this embodiment includes the following steps:
step 1, setting the structure of a structural element B according to the formula (1), wherein the size of the structural element B is w multiplied by w, setting the threshold selection percentage P of a depth image, setting the size of a repair image block to be S multiplied by S,
Figure BDA0003199040930000031
wherein ,
Figure BDA0003199040930000032
is an element in the structural element, and the value of the element is 1;
step 2, according to the formula (2), the depth image D to be restoredSOpening operation is carried out on the structural element B to obtain a depth image D after opening operationOpen，
Figure BDA0003199040930000033
Wherein! And
Figure BDA0003199040930000037
erosion and dilation operators representing images;
step 3, using the depth image D to be restored according to the formula (3)SSubtracting the on-operated depth image DOpenObtaining a depth image D with background eliminatedN；
DN＝DS-DOpen (3)
Step 4, counting the depth image D in the step 3NHistogram of (1)
Figure BDA0003199040930000034
A binarization threshold value T is calculated according to equation (4),
Figure BDA0003199040930000035
wherein ,HiRepresenting a depth image DNThe total number of pixels having a middle gray value of i, W and H respectively represent the depth image DNWidth and height of (d);
step 5, the depth image D is processed according to the formula (5)NAccording toCarrying out binarization processing on the threshold value T obtained in the step 4 to obtain a mask M of the depth image to be restored;
Figure BDA0003199040930000036
step 6, obtaining the depth image D of the mask M to be repaired in the step 5SThe image blocks with the size of S multiplied by S are searched for other positions and are repaired according to the formula (6), and the repaired depth image D is obtainedP，
DP＝Inpaint(DS,M,S×S) (6)
Here, Inpaint (·) represents an image repair function.
Taking a nickel-chromium metal sample as an example object, and fig. 2 is a depth image and a gray image of the sample affected by high reflection, establishing a low-quality depth image restoration model according to the sample, wherein the key steps are as follows:
2, obtaining a result of the opening operation of the depth image to be repaired by the method, as shown in fig. 3;
obtaining a depth image to be repaired without background interference by the method in the step 3, as shown in fig. 4;
4, calculating the threshold of the depth image in the step 3 according to the preset threshold selection percentage;
step 5, obtaining a mask of the depth image to be repaired by the method, as shown in fig. 5;
and 6, obtaining the repaired depth image by the method in the step 6, as shown in fig. 6.
Sample results show that the method can well overcome the problem of depth image distortion caused by high light reflection, the repaired depth image has better homogeneity, and the corresponding gray level image has natural texture transition.

Claims (1)

1. A method for repairing a high-reflection noise depth image is characterized by comprising the following steps:
step 1, setting the structure of a structural element B according to the formula (1), wherein the size of the structural element B is w multiplied by w, setting the threshold selection percentage P of a depth image, setting the size of a repair image block to be S multiplied by S,
Figure FDA0003199040920000011
wherein ,
Figure FDA0003199040920000012
is an element in the structural element, and the value of the element is 1;
step 2, according to the formula (2), the depth image D to be restoredSOpening operation is carried out on the structural element B to obtain a depth image D after opening operationOpen，
Figure FDA0003199040920000013
Wherein! And
Figure FDA0003199040920000014
erosion and dilation operators representing images;
step 3, using the depth image D to be restored according to the formula (3)SSubtracting the on-operated depth image DOpenObtaining a depth image D with background eliminatedN；
DN＝DS-DOpen (3)
Step 4, counting the depth image D in the step 3NHistogram of (1)
Figure FDA0003199040920000015
A binarization threshold value T is calculated according to equation (4),
Figure FDA0003199040920000016
wherein ,HiRepresenting a depth image DNThe total number of pixels with a medium gray value of i,w and H respectively represent depth images DNWidth and height of (d);
step 5, the depth image D is processed according to the formula (5)NCarrying out binarization processing according to the threshold value T obtained in the step 4 to obtain a mask M of the depth image to be restored;
Figure FDA0003199040920000017
step 6, obtaining the depth image D of the mask M to be repaired in the step 5SThe image blocks with the size of S multiplied by S are searched for other positions and are repaired according to the formula (6), and the repaired depth image D is obtainedP，
DP＝Inpaint(DS,M,S×S) (6)
Here, Inpaint (·) represents an image repair function.
CN202110899091.7A 2021-08-06 2021-08-06 Repairing method for high reflection noise depth image Active CN113570524B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110899091.7A CN113570524B (en) 2021-08-06 2021-08-06 Repairing method for high reflection noise depth image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110899091.7A CN113570524B (en) 2021-08-06 2021-08-06 Repairing method for high reflection noise depth image

Publications (2)

Publication Number Publication Date
CN113570524A true CN113570524A (en) 2021-10-29
CN113570524B CN113570524B (en) 2023-09-22

Family

ID=78170597

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110899091.7A Active CN113570524B (en) 2021-08-06 2021-08-06 Repairing method for high reflection noise depth image

Country Status (1)

Country Link
CN (1) CN113570524B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114331915A (en) * 2022-03-07 2022-04-12 荣耀终端有限公司 Image processing method and electronic device

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6028953A (en) * 1995-08-22 2000-02-22 Kabushiki Kaisha Toshiba Mask defect repair system and method which controls a dose of a particle beam
US20100188584A1 (en) * 2009-01-23 2010-07-29 Industrial Technology Research Institute Depth calculating method for two dimensional video and apparatus thereof
CN110675346A (en) * 2019-09-26 2020-01-10 武汉科技大学 Image acquisition and depth map enhancement method and device suitable for Kinect
CN111311515A (en) * 2020-02-13 2020-06-19 山西大学 Depth image fast iterative restoration method for automatic detection of error area
CN113188474A (en) * 2021-05-06 2021-07-30 山西大学 Image sequence acquisition system for imaging of high-light-reflection material complex object and three-dimensional shape reconstruction method thereof

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6028953A (en) * 1995-08-22 2000-02-22 Kabushiki Kaisha Toshiba Mask defect repair system and method which controls a dose of a particle beam
US20100188584A1 (en) * 2009-01-23 2010-07-29 Industrial Technology Research Institute Depth calculating method for two dimensional video and apparatus thereof
CN110675346A (en) * 2019-09-26 2020-01-10 武汉科技大学 Image acquisition and depth map enhancement method and device suitable for Kinect
CN111311515A (en) * 2020-02-13 2020-06-19 山西大学 Depth image fast iterative restoration method for automatic detection of error area
CN113188474A (en) * 2021-05-06 2021-07-30 山西大学 Image sequence acquisition system for imaging of high-light-reflection material complex object and three-dimensional shape reconstruction method thereof

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
牟琦;夏蕾;李占利;李洪安;: "采用曲率扩散和边缘重建的深度图像空洞修复", 西安科技大学学报, no. 02 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114331915A (en) * 2022-03-07 2022-04-12 荣耀终端有限公司 Image processing method and electronic device

Also Published As

Publication number Publication date
CN113570524B (en) 2023-09-22

Similar Documents

Publication Publication Date Title
CN111815601B (en) Texture image surface defect detection method based on depth convolution self-encoder
CN111161243B (en) Industrial product surface defect detection method based on sample enhancement
WO2021143343A1 (en) Method and device for testing product quality
CN109242791B (en) Batch repair method for damaged plant leaves
CN109580630B (en) Visual inspection method for defects of mechanical parts
WO2021027135A1 (en) Cell detection model training method and apparatus, computer device and storage medium
CN107228860B (en) Gear defect detection method based on image rotation period characteristics
CN115100221A (en) Glass defect segmentation method
CN111951284B (en) Optical remote sensing satellite image refined cloud detection method based on deep learning
CN116777907A (en) Sheet metal part quality detection method
CN108830857B (en) Self-adaptive Chinese character copy label image binarization segmentation method
CN111681213A (en) Light guide plate line scratch defect detection method based on deep learning
CN114549446A (en) Cylinder sleeve defect mark detection method based on deep learning
CN111754538A (en) Threshold segmentation method for USB surface defect detection
CN114331869A (en) Dam face crack semantic segmentation method
CN113570524A (en) Restoration method for high-reflection noise depth image
CN114841992A (en) Defect detection method based on cyclic generation countermeasure network and structural similarity
CN112802022B (en) Method for intelligently detecting defective glass image, electronic device and storage medium
CN117115161B (en) Plastic defect inspection method
WO2024045963A1 (en) Appearance defect detection method, electronic device, and storage medium
CN115880683B (en) Urban waterlogging ponding intelligent water level detection method based on deep learning
CN115797314A (en) Part surface defect detection method, system, equipment and storage medium
CN114937167A (en) Bridge pavement crack image crack detection method under complex background
CN114548250A (en) Mobile phone appearance detection method and device based on data analysis
CN112287933B (en) Method and system for removing character interference of X-ray image of automobile hub

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant