CN112595262B - Binocular structured light-based high-light-reflection surface workpiece depth image acquisition method - Google Patents
Binocular structured light-based high-light-reflection surface workpiece depth image acquisition method Download PDFInfo
- Publication number
- CN112595262B CN112595262B CN202011421291.3A CN202011421291A CN112595262B CN 112595262 B CN112595262 B CN 112595262B CN 202011421291 A CN202011421291 A CN 202011421291A CN 112595262 B CN112595262 B CN 112595262B
- Authority
- CN
- China
- Prior art keywords
- workpiece
- depth image
- infrared camera
- depth
- structured light
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
- G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
- G01B11/2545—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object with one projection direction and several detection directions, e.g. stereo
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/22—Measuring arrangements characterised by the use of optical techniques for measuring depth
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention relates to a binocular structured light-based method for acquiring a depth image of a high-reflection surface workpiece. The scheme comprises the steps that a high-reflection image acquisition device on the surface of a workpiece and a high-reflection workpiece are adjusted to be in an initial state; adjusting the high-reflectivity image acquisition equipment on the surface of the workpiece and the high-reflectivity workpiece from the initial state to an exposure state; shooting the workpiece with the high light-reflecting surface to obtain a workpiece structured light pattern and a workpiece color picture; acquiring five depth images through depth processing and moving the moving platform according to the workpiece structured light pattern and the workpiece color picture; and obtaining a target depth image of the high-light-reflection surface workpiece through a five-image fusion algorithm. The scheme overcomes the defect that the three-dimensional size information of the workpiece with the high-reflection surface cannot be accurately measured by the 3D structured light through multi-exposure adjustment and multi-image fusion of the multi-eye structured light, and has the advantages of small calculation amount, low cost and strong universality.
Description
Technical Field
The invention relates to the technical field of 3D structured light measurement, in particular to a binocular structured light-based method for acquiring a depth image of a high-reflection surface workpiece.
Background
Although 2D vision is the mainstream at present, with the requirement for measurement accuracy becoming higher and higher, the conditions of the measured object become more and more complex, and the defects of the 2D system become more and more prominent, while the 3D vision technology is getting a breakthrough, is 2D incomparable in terms of accuracy and flexibility, and has been applied in multiple fields in recent years. The 3D structured light measurement technology is an important branch of the three-dimensional vision technology, and the principle of the technology is that a projection device is adopted to project a structured light pattern with coded information on a measured object, and a camera records the information of the structured light pattern to decode, so that three-dimensional information of an object is restored. The 3D structured light measurement technology has the advantages of high precision, non-contact, high detection speed, simplicity in operation and the like, and is applied more and more widely in industrial scenes. At present, the 3D structured light measurement technology is applied more in the field of industrial workpiece dimension measurement. However, the existing 3D structured light measurement technology has the following disadvantages: the measuring effect on the workpiece with the high-reflection surface is poor, and partial three-dimensional data is easy to lose.
Disclosure of Invention
In order to overcome the problems in the related art, the invention provides a binocular-structured-light-based method for acquiring a depth image of a high-reflection surface workpiece, so that the defects that the high-reflection surface workpiece is poor in measurement effect and part of three-dimensional data is easily lost are overcome.
According to the embodiment of the invention, a binocular-structured-light-based method for acquiring a depth image of a high-reflection surface workpiece is provided. The method comprises the following steps:
adjusting a workpiece surface high-reflection image acquisition device and a high-reflection surface workpiece to an initial state, wherein the workpiece surface high-reflection image acquisition device comprises a first infrared camera, a second infrared camera, an infrared projection device, a color camera and a mobile platform;
adjusting the high-reflectivity image acquisition equipment on the surface of the workpiece and the high-reflectivity workpiece from the initial state to an exposure state;
shooting through the high-reflection image acquisition equipment on the surface of the workpiece in the exposure state and the high-reflection surface workpiece to obtain a structured light pattern of the workpiece and a color picture of the workpiece;
obtaining a first depth image through depth processing according to the workpiece structured light pattern and the workpiece color picture, and saving the first depth image as a depth image P 1 ；
Obtaining a depth image P by moving the mobile platform 2 Depth image P 3 Depth image P 4 And depth image P 5 ；
The depth image P is fused by a five-image fusion algorithm 1 The depth image P 2 The depth image P 3 The depth image P 4 And the depth image P 5 And fusing to obtain a target depth image of the high-reflection surface workpiece.
In one or more embodiments, preferably, the adjusting the workpiece surface highly reflective image acquiring apparatus and the highly reflective workpiece to the initial state specifically includes:
mounting the infrared projection device, the first infrared camera, the second infrared camera, and the color camera on the mobile platform in a vertical plane;
mounting the infrared projection device at the middle position of the mobile platform;
mounting said color camera in a position alongside said infrared projection device;
installing the first infrared camera and the second infrared camera on two sides of the infrared projection device;
moving the mobile platform to a central location.
In one or more embodiments, preferably, the adjusting the high-reflectivity image acquiring apparatus for the workpiece surface and the high-reflectivity workpiece surface from the initial state to the exposure state specifically includes:
setting an exposure time of the first infrared camera and the second infrared camera to half of a maximum exposure time;
setting gains of the first infrared camera and the second infrared camera to be half of a maximum gain;
and the infrared projection device projects a structured light pattern to the surface of the workpiece with the high light reflection surface.
In one or more embodiments, preferably, the obtaining of the workpiece structured light pattern and the workpiece color picture by shooting through the workpiece surface highly reflective image obtaining apparatus and the highly reflective surface workpiece in the exposure state specifically includes:
the first color camera and the second color camera shoot the workpiece structured light pattern of the workpiece with the high light-reflecting surface;
and the color camera shoots the surface of the workpiece with the high light-reflecting surface to obtain a color picture of the workpiece.
In one or more embodiments, preferably, the obtaining a first depth image through depth processing according to the workpiece structured light pattern and the workpiece color picture specifically includes:
converting the color picture of the workpiece into a gray-scale image, and extracting the surface contour of the workpiece from the gray-scale image to obtain gray-scale information of a workpiece area;
adjusting the exposure time and the gain of the first infrared camera and the second infrared camera according to the gray information of the workpiece area and exposure time and gain adjustment rules;
taking a surface structured light pattern of the workpiece color picture by using the first infrared camera and the second infrared camera;
decoding the surface structure light patterns acquired by the first infrared camera and the second infrared camera, and calculating a plurality of workpiece surface point cloud pictures by combining calibration parameters of the first infrared camera and the second infrared camera;
matching and fusing a plurality of workpiece surface point cloud images calculated by the first infrared camera and the second infrared camera, and calculating a plurality of fused point cloud depth images;
and fusing the plurality of cloud point depth maps through a multi-depth image fusion algorithm to obtain a first depth image.
In one or more embodiments, preferably, the method for extracting the surface profile of the workpiece is embodied as follows: and (4) adopting a canny algorithm to extract the image edge of the gray image, and screening the workpiece outline through edge length information.
In one or more embodiments, preferably, the adjusting the exposure time and the gain of the first infrared camera and the second infrared camera according to the exposure time and gain adjustment rule includes:
wherein exp i The exposure time, gain, of the first infrared camera and the second infrared camera after the ith adjustment i The gains of the first infrared camera and the second infrared camera after the ith adjustment are obtained, i is the adjustment times, mean is the mean value of the gray scale of the workpiece area, exp max Is the maximum exposure time, gain, of the infrared camera max Is the maximum gain of the infrared camera.
In one or more embodiments, preferably, the multi-depth image fusion algorithm specifically includes:
wherein k is the number of the point cloud depth map, im g is the first depth image after fusion, im g i And the point cloud depth map is the ith point cloud depth map, wherein i is one of integers which are more than or equal to 0 and less than k.
In one or more embodiments, preferably, the obtaining the depth image by moving the mobile platform specifically includes:
moving the mobile platform upwards, and saving the first depth image as a depth image P by adopting a depth image extraction method 2 ；
Moving the mobile platform to a central position, then moving downwards, and storing the first depth image as a depth image P by adopting a depth image extraction method 3 ；
Moving the mobile platform to a central position, moving the mobile platform to the left, and storing the first depth image as a depth image P by adopting a depth image extraction method 4 ；
Moving the mobile platform to the central position, then moving the mobile platform to the right, and storing the first depth image as a depth image P by adopting a depth image extraction method 5 ；
The depth image extraction method specifically comprises the following steps:
the high-reflectivity image acquisition equipment on the surface of the workpiece and the high-reflectivity workpiece are adjusted to an exposure state from the initial state;
the high-light-reflection image acquisition equipment on the surface of the workpiece in the exposure state and the workpiece with the high-light-reflection surface are shot to obtain a structured light pattern of the workpiece and a color picture of the workpiece;
and obtaining a first depth image through depth processing according to the workpiece structured light pattern and the workpiece color picture.
In one or more embodiments, preferably, the five image fusion algorithms are specifically:
wherein P is the final fused depth image, P i For the depth image P 1 The depth image P 2 The depth image P 3 The depth image P 4 And the depth image P 5 One of them.
The technical scheme provided by the embodiment of the invention can have the following beneficial effects:
1. according to the scheme, the infrared camera with a binocular structure is used for shooting, multiple images are obtained through multiple exposure time and gain adjustment, and then the depth images are obtained through image depth processing, so that the defect that the three-dimensional size information of the high-reflection surface workpiece cannot be accurately measured by the existing 3D structured light is overcome, and the high-reflection surface workpiece can be clearly and accurately measured.
2. According to the scheme, only 2 infrared cameras and 1 color camera are used for image acquisition, camera adjustment and picture depth fusion are carried out through preset rules, a general acquisition mode for various high-reflection surface workpieces can be realized, and due to the fact that repeated operation is few and the operation amount is small, the scheme is used for obtaining the depth images of the workpieces, and the characteristics of economy and reliability are achieved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the description below are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without creative efforts.
Fig. 1 is a flowchart of a method for acquiring a depth image of a highly reflective surface workpiece based on binocular structured light according to an embodiment of the present invention.
Fig. 2 is a flowchart illustrating adjusting of a highly reflective image acquisition device on a workpiece surface and a highly reflective workpiece to an initial state in a binocular structured light-based highly reflective surface workpiece depth image acquisition method according to an embodiment of the present invention.
Fig. 3 is a schematic diagram of a highly reflective image acquiring apparatus for a workpiece surface and a highly reflective workpiece in a binocular structured light-based method for acquiring a depth image of a highly reflective workpiece according to an embodiment of the present invention.
Fig. 4 is a flowchart of adjusting from the initial state to an exposure state in a binocular structured light-based method for acquiring a depth image of a highly reflective surface workpiece according to an embodiment of the present invention.
Fig. 5 is a flowchart illustrating photographing of a highly reflective surface workpiece in a binocular structured light-based highly reflective surface workpiece depth image acquisition method according to an embodiment of the present invention.
Fig. 6 is a flowchart of obtaining a first depth image through depth processing in a method for obtaining a depth image of a highly reflective surface workpiece based on binocular structured light according to an embodiment of the present invention.
Fig. 7 is a flowchart of obtaining a depth image by moving the moving platform in the binocular structured light-based depth image obtaining method for a highly reflective surface workpiece according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention.
In some of the flows described in the present specification and claims and in the above figures, a number of operations are included that occur in a particular order, but it should be clearly understood that these operations may be performed out of order or in parallel as they occur herein, with the order of the operations being indicated as 101, 102, etc. merely to distinguish between the various operations, and the order of the operations by themselves does not represent any order of performance. Additionally, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first", "second", etc. in this document are used for distinguishing different messages, devices, modules, etc., and do not represent a sequential order, nor limit the types of "first" and "second" to be different.
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Although 2D vision is the mainstream at present, with the higher and higher requirement for measurement precision, the more and more complex conditions of the measured object and the more and more prominent defects of the 2D system are also achieved, and the 3D vision technology is continuously broken through, is incomparable with 2D in terms of precision and flexibility, and has gradually started to show its advantages in various fields at present.
The 3D structured light measurement technology is an important branch of the three-dimensional vision technology, and the principle of the technology is that a projection device is adopted to project a structured light pattern with coded information on a measured object, and a camera records the information of the structured light pattern to decode, so that three-dimensional information of an object is restored. The 3D structured light measurement technology has the advantages of high precision, non-contact, high detection speed, simplicity in operation and the like, and is applied more and more widely in industrial scenes. At present, the 3D structured light measurement technology is applied more in the field of industrial workpiece dimension measurement.
However, the existing 3D structured light measurement technology has the following disadvantages: the measuring effect on the workpiece with the high light-reflecting surface is poor, and partial three-dimensional data is easy to lose; this is caused by the fact that the light reflects slightly at different angles and exposure intensities, and therefore may reflect strongly under certain conditions, which may result in an inability to confirm the measured dimensions of the workpiece.
The technical scheme of the invention provides a binocular structured light-based depth image acquisition method for a highly reflective surface workpiece, which enables multiple groups of image data to be acquired through binocular structured light, the image data are mutually supplemented, and a target depth image is finally acquired through an image depth fusion method, so that the measurement accuracy of the highly reflective surface workpiece measured by 3D structured light cannot be influenced due to image information loss caused by high reflection.
Fig. 1 is a flowchart of a method for acquiring a depth image of a highly reflective surface workpiece based on binocular structured light according to an embodiment of the present invention.
As shown in fig. 1, the present invention provides a method for obtaining a depth image of a highly reflective surface workpiece based on binocular structured light. The method comprises the following steps:
s101, adjusting a high-reflection image acquisition device on the surface of a workpiece and a high-reflection workpiece to an initial state, wherein the high-reflection image acquisition device on the surface of the workpiece comprises a first infrared camera, a second infrared camera, an infrared projection device, a color camera and a mobile platform;
s102, adjusting the high-reflectivity image acquisition equipment on the surface of the workpiece and the high-reflectivity workpiece on the surface of the workpiece from the initial state to an exposure state;
s103, shooting through the high-reflection image acquisition equipment on the surface of the workpiece in the exposure state and the high-reflection surface workpiece to obtain a structured light pattern of the workpiece and a color picture of the workpiece;
s104, obtaining a first depth image through depth processing according to the workpiece structured light pattern and the workpiece color picture, and saving the first depth image as a depth image P 1 ；
S105, obtaining a depth image P by moving the mobile platform 2 Depth image P 3 Depth image P 4 And depth image P 5 ；
S106, fusing the depth image P through a five-image fusion algorithm 1 The depth image P 2 The depth image P 3 The depth image P 4 And the depth image P 5 Fusion to obtainA target depth image of the highly reflective surface workpiece.
In the embodiment of the invention, the full shooting of the workpiece with the high-reflection surface is completed through a series of steps of initialization, exposure, shooting, image depth processing, obtaining of multiple groups of images by a mobile platform, fusion of multiple groups of image data and the like of the workpiece with the high-reflection surface and the workpiece state of the high-reflection surface, so that a binocular structured light target depth image without information loss is obtained, and the workpiece measurement is accurately completed.
Further, in fig. 2 to 7, the steps of adjusting the initial state to the exposure state, obtaining the first depth image through the depth processing, and the like will be described in detail.
Fig. 2 is a flowchart illustrating adjusting of a highly reflective image acquisition device on a workpiece surface and a highly reflective workpiece to an initial state in a binocular structured light-based highly reflective surface workpiece depth image acquisition method according to an embodiment of the present invention.
As shown in fig. 2, in one or more embodiments, preferably, the adjusting the workpiece surface highly reflective image acquiring apparatus and the highly reflective workpiece to the initial state specifically includes:
s201, mounting the infrared projection device, the first infrared camera, the second infrared camera and the color camera on the mobile platform on a vertical plane;
s202, installing the infrared projection device in the middle of the mobile platform;
s203, installing the color camera at a position beside the infrared projection device;
s204, mounting the first infrared camera and the second infrared camera on two sides of the infrared projection device;
s205, moving the mobile platform to a central position.
In the embodiment of the present invention, the initialization of the device state is performed by configuring the infrared projection device, the first infrared camera, the second infrared camera, and the color camera at corresponding positions and by adjusting the mobile platform to a central position.
Fig. 3 is a schematic diagram of a highly reflective image acquiring apparatus for a workpiece surface and a highly reflective workpiece in a binocular structured light-based method for acquiring a depth image of a highly reflective workpiece according to an embodiment of the present invention.
In one or more embodiments, the apparatus for acquiring highly reflective images of a surface of a workpiece comprises a first infrared camera 301, a second infrared camera 302, an infrared projection device 303, a color camera 304, a mobile platform 305; the projection range of the infrared projection device 303 is marked by a dashed line below the highly reflective surface piece 306. The first infrared camera 301 and the second infrared camera 302 are two infrared cameras with 200 ten thousand pixels, the infrared projection device 303 is an infrared projection device, the resolution is 100 ten thousand pixels, and the color camera 304 is a color camera with 200 ten thousand pixels. Through the configuration mode in the embodiment, the shooting of the surface of the highly reflective workpiece can be performed, but the shooting can also be performed by adopting camera equipment which is more than pixels in the embodiment.
Fig. 4 is a flowchart of adjusting from the initial state to an exposure state in a method for acquiring a depth image of a highly reflective surface workpiece based on binocular structured light according to an embodiment of the present invention.
As shown in fig. 4, in one or more embodiments, preferably, the adjusting the workpiece surface highly reflective image acquiring apparatus and the highly reflective surface workpiece from the initial state to the exposure state specifically includes:
s401, setting the exposure time of the first infrared camera and the second infrared camera to be half of the maximum exposure time;
s402, setting gains of the first infrared camera and the second infrared camera to be half of a maximum gain;
and S403, projecting a structured light pattern to the surface of the workpiece with the high light-reflecting surface by the infrared projection device.
In the embodiment of the invention, the exposure time and the gain are set before exposure, and the exposure time and the gain are set by half of the maximum value, so that the reliability in the whole exposure process is ensured, specifically, the adjustment increase and decrease ranges of the exposure time and the gain can be maximized, and the adjustment decrease and increase degrees can be ensured to be the same.
Fig. 5 is a flowchart of a photographing process of a highly reflective surface workpiece in a binocular structured light-based method for acquiring a depth image of the highly reflective surface workpiece according to an embodiment of the present invention.
As shown in fig. 5, in one or more embodiments, preferably, the obtaining of the workpiece structured light pattern and the workpiece color picture by shooting the workpiece surface highly reflective image obtaining apparatus and the highly reflective surface workpiece in the exposure state specifically includes:
s501, shooting a workpiece structure light pattern of the workpiece with the high light reflecting surface by the first color camera and the second color camera;
s502, the color camera shoots the surface of the workpiece with the high light-reflecting surface to obtain a color picture of the workpiece.
In the embodiment of the invention, the color camera is used for shooting the color picture of the surface of the workpiece, and further used for carrying out gray scale division by utilizing color change, and the workpiece structured light pattern shot by the first color camera and the second color camera can realize the data input of the dual-purpose 3D structured light.
Fig. 6 is a flowchart of obtaining a first depth image through depth processing in a method for obtaining a depth image of a highly reflective surface workpiece based on binocular structured light according to an embodiment of the present invention.
As shown in fig. 6, in one or more embodiments, preferably, the obtaining a first depth image through depth processing according to the workpiece structured light pattern and the workpiece color picture specifically includes:
s601, converting the color picture of the workpiece into a gray-scale image, extracting the surface contour of the workpiece from the gray-scale image, and obtaining gray-scale information of a workpiece area;
s602, adjusting the exposure time and the gain of the first infrared camera and the second infrared camera according to the gray information of the workpiece area and an exposure time and gain adjustment rule;
s603, shooting a surface structured light pattern on the workpiece color picture by using the first infrared camera and the second infrared camera;
s604, decoding the surface structured light patterns acquired by the first infrared camera and the second infrared camera, and calculating a plurality of workpiece surface point cloud pictures by combining calibration parameters of the first infrared camera and the second infrared camera;
s605, matching and fusing a plurality of workpiece surface point cloud images calculated by the first infrared camera and the second infrared camera, and calculating a plurality of fused point cloud depth images;
and S606, fusing the plurality of cloud point depth maps through a multi-depth image fusion algorithm to obtain a first depth image.
In the embodiment of the invention, a plurality of pictures are obtained by adjusting the exposure time and the gain of the first infrared camera and the second infrared camera according to a first rule, every time the exposure time and the gain are adjusted, 2 infrared cameras shoot the structural light pattern on the surface of the workpiece, then the structural light pattern collected by the 2 infrared cameras is decoded, the point cloud images on the surface of the workpiece are calculated by combining the calibration parameters of the cameras, the point cloud images calculated by the left camera and the right camera are matched and fused, the depth image of the fused point cloud is calculated, and then the depth images are fused according to the following rule to obtain the depth image.
In one or more embodiments, preferably, the method for extracting the surface profile of the workpiece is embodied as follows: and (4) adopting a canny algorithm to extract the image edge of the gray image, and screening the workpiece outline through edge length information.
In one or more embodiments, preferably, the adjusting the exposure time and the gain of the first infrared camera and the second infrared camera according to the exposure time and the gain adjustment rule includes specifically:
wherein exp i The exposure time, gain, of the first infrared camera and the second infrared camera after the ith adjustment i The gains of the first infrared camera and the second infrared camera after the ith adjustment are obtained, i is the adjustment times, mean is the mean value of the gray scale of the workpiece area, exp max Is the maximum exposure time, gain, of the infrared camera max Is the maximum gain of the infrared camera.
In one or more embodiments, preferably, the multi-depth image fusion algorithm specifically includes:
wherein k is the number of the point cloud depth images, im g is the fused first depth image, and Im g i And the point cloud depth map is the ith point cloud depth map, wherein i is one of integers which are more than or equal to 0 and less than k.
In the embodiment of the invention, k point cloud depth images can be acquired, multiple times of adjustment are adopted in the acquisition process, each time of adjustment is used for acquiring the gain and the exposure time of a new first infrared camera and a new second infrared camera, images with uniform relative gray level change are acquired through repeated correction, each image is ensured to have enough accurate 3D structured light information, and finally a first depth image is acquired by utilizing a multi-depth image fusion algorithm.
Fig. 7 is a flowchart of obtaining a depth image by moving the moving platform in the binocular structured light-based depth image obtaining method for a highly reflective surface workpiece according to an embodiment of the present invention.
As shown in fig. 7, in one or more embodiments, preferably, the obtaining the depth image by moving the moving platform specifically includes:
s701, moving the mobile platform upwards, and storing the first depth image as a depth image P by adopting a depth image extraction method 2 ；
S702, moving the mobile platform to a central position, then moving downwards, and storing the first depth image as a depth image P by adopting a depth image extraction method 3 ；
S703, moving the mobile platform to a central position, moving the mobile platform to the left, and storing the first depth image as a depth image P by adopting a depth image extraction method 4 ；
S704, moving the mobile platform to the center position, then moving the mobile platform to the right, and storing the first depth image as a depth image P by adopting a depth image extraction method 5 ；
The depth image extraction method specifically comprises the following steps:
the high-reflectivity image acquisition equipment on the surface of the workpiece and the high-reflectivity workpiece are adjusted to an exposure state from the initial state;
the high-light-reflection image acquisition equipment on the surface of the workpiece in the exposure state and the workpiece with the high-light-reflection surface are shot to obtain a structured light pattern of the workpiece and a color picture of the workpiece;
and obtaining a first depth image through depth processing according to the workpiece structured light pattern and the workpiece color picture.
In the embodiment of the present invention, the first depth image is saved as the depth image P in step S104 1 However, the first depth image is saved as different depth images in steps S701, S702, S703 and S704, which are depth images P 2 Depth image P 3 Depth image P 4 Depth image P 5 . Therefore, only the first depth image is obtained in the depth image extraction method, and the first processed image is used as a data bridge for storage of subsequent data analysis.
In one or more embodiments, preferably, the five image fusion algorithms are specifically:
wherein P is the final fused depth image, P i For the depth image P 1 The depth image P 2 The depth image P 3 The depth image P 4 And the depth image P 5 One of them.
In the embodiment of the invention, the depth image P is fused by a five-image fusion algorithm 1 Depth image P 2 Depth image P 3 And a depth image P 4 Depth image P 5 The image is finally processed into a single target depth image. The target depth image can display the measured size of the workpiece, and meanwhile, the influence of different positions, different exposure time and gain on the depth image is reduced through the averaging and fusion of 5 pictures, so that the depth image of the workpiece with the high light-reflecting surface is acquired.
In the embodiment of the invention, a binocular structure light-based method for acquiring a depth image of a high-reflection surface workpiece is provided, so that infrared camera data of a binocular structure are subjected to multiple adjustment and data fusion. The following effects can be produced:
1. according to the scheme, the infrared camera with a binocular structure is used for shooting, multiple images are obtained through multiple exposure time and gain adjustment, and then the depth images are obtained through image depth processing, so that the defect that the three-dimensional size information of the high-reflection surface workpiece cannot be accurately measured by the existing 3D structured light is overcome, and the high-reflection surface workpiece can be clearly and accurately measured.
2. According to the scheme, only 2 infrared cameras and 1 color camera are used for image acquisition, camera adjustment and picture depth fusion are carried out through preset rules, a general acquisition mode for various high-reflection surface workpieces can be realized, and due to the fact that repeated operation is few and the operation amount is small, the scheme is used for obtaining the depth images of the workpieces, and the characteristics of economy and reliability are achieved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (8)
1. A binocular structured light-based method for obtaining a depth image of a high-reflection surface workpiece is characterized by comprising the following steps:
adjusting a workpiece surface high-reflection image acquisition device and a workpiece with a high-reflection surface to an initial state, wherein the workpiece surface high-reflection image acquisition device comprises a first infrared camera, a second infrared camera, an infrared projection device, a color camera and a mobile platform;
adjusting the high-reflectivity image acquisition equipment on the surface of the workpiece and the high-reflectivity workpiece from the initial state to an exposure state;
shooting through the high-reflection image acquisition equipment on the surface of the workpiece in the exposure state and the high-reflection surface workpiece to obtain a structured light pattern of the workpiece and a color picture of the workpiece;
obtaining a first depth image through depth processing according to the workpiece structured light pattern and the workpiece color picture, and saving the first depth image as a depth image P 1 ；
Obtaining a depth image P by moving the mobile platform 2 And a depth image P 3 Depth image P 4 And depth image P 5 ；
The depth image P is fused by a five-image fusion algorithm 1 The depth image P 2 The depth image P 3 The depth image P 4 And the depth image P 5 Fusing to obtain a target depth image of the high-reflection surface workpiece;
wherein, obtaining a first depth image through depth processing according to the workpiece structured light pattern and the workpiece color picture specifically includes:
converting the color picture of the workpiece into a gray image, and extracting the surface contour of the workpiece from the gray image to obtain gray information of a workpiece area;
adjusting the exposure time and the gain of the first infrared camera and the second infrared camera according to the gray scale information of the workpiece area and an exposure time and gain adjustment rule;
taking a surface structured light pattern of the workpiece color picture by using the first infrared camera and the second infrared camera;
decoding the surface structure light patterns acquired by the first infrared camera and the second infrared camera, and calculating a plurality of workpiece surface point cloud pictures by combining calibration parameters of the first infrared camera and the second infrared camera;
matching and fusing a plurality of workpiece surface point cloud images calculated by the first infrared camera and the second infrared camera, and calculating a plurality of fused point cloud depth images;
fusing the point cloud depth maps through a multi-depth image fusion algorithm to obtain a first depth image;
wherein the adjusting the exposure time and the gain of the first infrared camera and the second infrared camera according to the exposure time and the gain adjustment rule comprises:
wherein exp i The exposure time, gain, of the first infrared camera and the second infrared camera after the ith adjustment i The gains of the first infrared camera and the second infrared camera after the ith adjustment are obtained, i is the adjustment times, mean is the mean value of the gray scale of the workpiece area, exp max Is the maximum exposure time, gain, of the infrared camera max Is the maximum gain of the infrared camera.
2. The binocular structured light-based depth image acquisition method for the highly reflective surface workpiece, according to claim 1, wherein the adjusting of the workpiece surface highly reflective image acquisition device and the highly reflective surface workpiece to an initial state specifically comprises:
mounting the infrared projection device, the first infrared camera, the second infrared camera, and the color camera on the mobile platform in a vertical plane;
mounting the infrared projection device at the middle position of the mobile platform;
mounting said color camera in a position alongside said infrared projection device;
installing the first infrared camera and the second infrared camera on two sides of the infrared projection device;
moving the mobile platform to a central location.
3. The binocular structured light-based depth image acquisition method for the highly reflective surface workpiece according to claim 2, wherein the adjusting the workpiece surface highly reflective image acquisition device and the highly reflective surface workpiece from the initial state to the exposure state specifically comprises:
setting an exposure time of the first infrared camera and the second infrared camera to half of a maximum exposure time;
setting gains of the first infrared camera and the second infrared camera to be half of a maximum gain;
and the infrared projection device projects a structured light pattern to the surface of the workpiece with the high light reflection surface.
4. The binocular structured light-based depth image acquisition method for the highly reflective surface workpiece according to claim 3, wherein the acquiring of the structured light pattern and the color image of the workpiece by the highly reflective image acquisition device for the workpiece surface in the exposure state and the highly reflective surface workpiece comprises:
a first color camera and a second color camera shoot the workpiece structure light pattern of the workpiece with the high light reflecting surface;
and the color camera shoots the surface of the workpiece with the high light-reflecting surface to obtain a color picture of the workpiece.
5. The binocular structured light-based high-reflectivity surface workpiece depth image acquisition method according to claim 4, wherein the method for extracting the workpiece surface contour specifically comprises the following steps: and (5) screening the outline of the workpiece through edge length information after extracting the image edge of the gray level image by adopting a canny algorithm.
6. The binocular structured light-based depth image acquisition method for the high-reflectivity surface workpiece, according to claim 4, wherein the multi-depth image fusion algorithm specifically comprises:
wherein k is the number of the point cloud depth maps, img is the fused first depth image, and Img i And the point cloud depth map is the ith point cloud depth map, wherein i is one of integers which are more than or equal to 0 and less than k.
7. The binocular structured light-based depth image obtaining method for the high-reflectivity surface workpiece, according to claim 4, wherein the obtaining of the depth image by moving the moving platform specifically comprises:
moving the mobile platform upwards, and saving the first depth image as a depth image P by adopting a depth image extraction method 2 ；
Moving the mobile platform to a central position, then moving downwards, and storing the first depth image as a depth image P by adopting a depth image extraction method 3 ；
Moving the mobile platform to the central position and then to the left,saving the first depth image as a depth image P by adopting a depth image extraction method 4 ；
Moving the mobile platform to the central position, then moving the mobile platform to the right, and storing the first depth image as a depth image P by adopting a depth image extraction method 5 ；
The depth image extraction method specifically comprises the following steps:
the high-reflectivity image acquisition equipment for the surface of the workpiece and the high-reflectivity workpiece are adjusted to an exposure state from the initial state;
the high-light-reflection image acquisition equipment on the surface of the workpiece in the exposure state and the workpiece with the high-light-reflection surface are shot to obtain a structured light pattern of the workpiece and a color picture of the workpiece;
and obtaining a first depth image through depth processing according to the workpiece structured light pattern and the workpiece color picture.
8. The binocular structured light-based high-reflectivity surface workpiece depth image acquisition method according to claim 1, wherein the five image fusion algorithms specifically comprise:
wherein P is the final fused depth image, P i For the depth image P 1 The depth image P 2 The depth image P 3 The depth image P 4 And the depth image P 5 One of them.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011421291.3A CN112595262B (en) | 2020-12-08 | 2020-12-08 | Binocular structured light-based high-light-reflection surface workpiece depth image acquisition method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011421291.3A CN112595262B (en) | 2020-12-08 | 2020-12-08 | Binocular structured light-based high-light-reflection surface workpiece depth image acquisition method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112595262A CN112595262A (en) | 2021-04-02 |
| CN112595262B true CN112595262B (en) | 2022-12-16 |
Family
ID=75188711
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011421291.3A Active CN112595262B (en) | 2020-12-08 | 2020-12-08 | Binocular structured light-based high-light-reflection surface workpiece depth image acquisition method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112595262B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113052898B (en) * | 2021-04-08 | 2022-07-12 | 四川大学华西医院 | Point cloud and strong-reflection target real-time positioning method based on active binocular camera |
| CN114820430B (en) * | 2022-02-18 | 2023-10-03 | 成都飞机工业（集团）有限责任公司 | Multi-light source collaborative exposure 3D printing nondestructive testing method |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4884063B2 (en) * | 2006-04-19 | 2012-02-22 | ディスク・テック株式会社 | Depth measuring device |
| CN106504284B (en) * | 2016-10-24 | 2019-04-12 | 成都通甲优博科技有限责任公司 | A kind of depth picture capturing method combined based on Stereo matching with structure light |
| CN206921118U (en) * | 2017-03-09 | 2018-01-23 | 深圳奥比中光科技有限公司 | Double-wavelength images acquisition system |
| CN106934394B (en) * | 2017-03-09 | 2024-01-19 | 奥比中光科技集团股份有限公司 | Dual wavelength image acquisition system and method |
| CN108267097B (en) * | 2017-07-17 | 2020-07-10 | 先临三维科技股份有限公司 | Three-dimensional reconstruction method and device based on binocular three-dimensional scanning system |
-
2020
- 2020-12-08 CN CN202011421291.3A patent/CN112595262B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN112595262A (en) | 2021-04-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111179358B (en) | Calibration method, device, equipment and storage medium | |
| US20210192761A1 (en) | Image depth estimation method and device, readable storage medium, and electronic apparatus | |
| EP2568253B1 (en) | Structured-light measuring method and system | |
| CN108225216B (en) | Structured light system calibration method and device, structured light system and mobile device | |
| CN112161997B (en) | Online precise visual measurement method and system for three-dimensional geometric dimension of semiconductor chip pin | |
| CN111192235B (en) | Image measurement method based on monocular vision model and perspective transformation | |
| CN112595262B (en) | Binocular structured light-based high-light-reflection surface workpiece depth image acquisition method | |
| CN110310331B (en) | Pose estimation method based on combination of linear features and point cloud features | |
| CN102376089A (en) | Target correction method and system | |
| CN110827360B (en) | Photometric stereo measurement system and method for calibrating light source direction thereof | |
| JP2011182397A (en) | Method and apparatus for calculating shift length | |
| Ben-Artzi et al. | Camera calibration from dynamic silhouettes using motion barcodes | |
| CN113513981B (en) | Multi-target parallel measurement method, system, equipment and storage medium based on binocular stereo vision | |
| CN109974623A (en) | 3 D information obtaining method and device based on line laser and binocular vision | |
| CN109636857B (en) | Alignment method and calibration system | |
| CN112489193B (en) | Three-dimensional reconstruction method based on structured light | |
| CN107504917B (en) | Three-dimensional size measuring method and device | |
| CN112648935A (en) | Image processing method and device and three-dimensional scanning system | |
| CN102901490B (en) | Image matching method based on dynamic threshold, and system | |
| CN117132583A (en) | Wafer defect detection method and device, electronic equipment and nonvolatile storage medium | |
| CN116958218A (en) | Point cloud and image registration method and equipment based on calibration plate corner alignment | |
| CN114596355A (en) | High-precision pose measurement method and system based on cooperative target | |
| CN112729246B (en) | Black surface object depth image measuring method based on binocular structured light | |
| CN114170321A (en) | Camera self-calibration method and system based on distance measurement | |
| CN112734721A (en) | Optical axis deflection angle detection method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |