CN117346693A - Binocular vision and PMD-based surface measurement method, system, equipment and medium thereof - Google Patents
Binocular vision and PMD-based surface measurement method, system, equipment and medium thereof Download PDFInfo
- Publication number
- CN117346693A CN117346693A CN202311616971.4A CN202311616971A CN117346693A CN 117346693 A CN117346693 A CN 117346693A CN 202311616971 A CN202311616971 A CN 202311616971A CN 117346693 A CN117346693 A CN 117346693A
- Authority
- CN
- China
- Prior art keywords
- parallax
- image
- view field
- binocular
- pmd
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000004441 surface measurement Methods 0.000 title claims abstract description 37
- 238000004364 calculation method Methods 0.000 claims abstract description 32
- 238000012937 correction Methods 0.000 claims abstract description 32
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 29
- 239000013598 vector Substances 0.000 claims description 50
- 101100412093 Schizosaccharomyces pombe (strain 972 / ATCC 24843) rec16 gene Proteins 0.000 claims description 9
- 101100468275 Caenorhabditis elegans rep-1 gene Proteins 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 4
- 238000013507 mapping Methods 0.000 claims description 4
- 238000012360 testing method Methods 0.000 claims 2
- 238000005259 measurement Methods 0.000 abstract description 14
- 238000012545 processing Methods 0.000 description 10
- 230000005540 biological transmission Effects 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007123 defense Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000004377 microelectronic Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000010363 phase shift Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
- G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
- G01B11/254—Projection of a pattern, viewing through a pattern, e.g. moiré
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T7/85—Stereo camera calibration
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention relates to the technical field of three-dimensional morphology measurement, in particular to a binocular vision and PMD-based surface measurement method, a binocular vision and PMD-based surface measurement system, equipment and a medium thereof, wherein the method comprises the following steps: preparing shooting hardware comprising a display screen and a binocular camera, wherein the display screen displays a stripe image and maps the stripe image to the surface of a sample to be detected; correcting the calibrated binocular camera polar lines; shooting a reflected image of the display screen mapped on the surface of the tested sample by a binocular camera after calibration and correction; the method comprises the steps of obtaining first parallax information by stereo matching of reflected images based on a left-right method phasor matching algorithm, and obtaining a first parallax image; obtaining second parallax information by three-dimensional matching of the reflected image based on a binocular three-dimensional matching algorithm to obtain a second parallax image; and the first parallax image is used as a reference image, the first parallax image is corrected through the second parallax image, a third parallax image is obtained, point cloud coordinate calculation is carried out based on the third parallax image, three-dimensional reconstruction is carried out on the measured surface according to the calculated point cloud coordinate, and the surface measurement accuracy is high.
Description
Technical Field
The invention relates to the technical field of three-dimensional morphology measurement, in particular to a binocular vision and PMD-based surface measurement method, a binocular vision and PMD-based surface measurement system, binocular vision and PMD-based surface measurement equipment and a medium thereof.
Background
The mirror surface, mirror-like surface and other high-reflection curved surface parts are increasingly widely applied to the high-tech fields of aerospace, national defense, biomedicine, communication and microelectronic lamps, become key parts in the photoelectric and communication industries, accurately measure and evaluate the surface morphology of the high-reflection curved surface parts, and have important significance for researching the relationship between the surface geometric characteristics and the service performance and improving the quality and the product performance of the processed surface.
The basic principle of the phase deflection measurement (PMD, phase Measuring Deflectometry) is that a screen is utilized to project grating fringes onto the surface of a part with a high reflection curved surface to be measured, a camera captures fringe images reflected by the surface of the part, and three-dimensional morphology information of the surface of the part to be measured is calculated according to the fringe information. Specifically, each reflection point of a high-reflection curved surface part has different deflection effects on projected light, after the camera shoots the fringe image, the phase corresponding to each pixel point is calculated based on a phase shift algorithm, and three-dimensional point cloud coordinates can be calculated by using the phase, so that the surface of the high-reflection curved surface part is subjected to three-dimensional reconstruction.
The measurement precision of the phase deflection measurement is related to the calibration precision of the system and the precision of the three-dimensional reconstruction algorithm. The accuracy of the three-dimensional reconstruction algorithm is related to the accuracy of phase stereo matching, and the existing phase stereo matching algorithm needs to depend on the strict establishment of incident-reflected light rays and is determined according to the uniqueness of the normal vectors of the left and right cameras at the searching surface. And the surface of the measured part is a high-reflection mirror surface or a mirror-like surface as much as possible. However, in the actual measurement process, the high-reflection curved surface part is often complicated, even the mirror surface is easy to receive pollution caused by carrying, moving and the like, when the mirror surface is no longer guaranteed to be the mirror surface, the uniqueness matching principle of the incident-reflection normal vector based on the phase is destroyed, so that the stereo matching error is increased, the accuracy of three-dimensional measurement reconstruction is influenced, and the measurement accuracy rate is low.
Disclosure of Invention
The invention aims to solve the technical problems that: in order to solve the technical problem that the surface measurement accuracy is low in the existing phase deflection measurement, the invention provides a binocular vision and PMD-based surface measurement method, and the surface measurement accuracy is improved.
The technical scheme adopted for solving the technical problems is as follows: a binocular vision and PMD based surface measurement method comprising:
s1, preparing shooting hardware, wherein the shooting hardware comprises the following steps: the device comprises a display screen and a binocular camera, wherein the display screen displays a stripe image and maps the stripe image to the surface of a sample to be detected, and the binocular camera is arranged at the outer side of the display screen;
s2, binocular stereo correction is carried out on the calibrated binocular camera to carry out polar correction, so that coplanarity and row alignment of left and right view field images can be realized after correction;
s3, acquiring images, wherein the binocular camera after calibration and correction shoots the display screen and maps the display screen on the surface of the tested sample to acquire left and right view field reflection images;
s4, phase three-dimensional matching calculation is carried out, three-dimensional matching is carried out on the left and right view field reflected images based on a left and right method phasor matching algorithm, first parallax information of the images is obtained, and a first parallax image is obtained;
s5, binocular stereo matching calculation is carried out, stereo matching is carried out on the left and right view field reflection images based on a binocular stereo matching algorithm, second parallax information of the images is obtained, and a second parallax image is obtained;
s6, correcting the parallax map, namely correcting the first parallax map through the second parallax map by taking the first parallax map as a reference map, wherein the corrected first parallax map is a third parallax map;
and S7, calculating point cloud coordinates based on the third parallax map, and carrying out three-dimensional reconstruction on the surface of the sample to be measured according to all calculated point cloud coordinates.
Further, specifically, the step S4 of stereo matching the left and right field of view reflection images based on the left and right method phasor matching algorithm specifically includes the following steps:
s41, calculating a left angle bisector vector by taking a pixel point of the left view field reflection image as a point to be matched and using an incident ray vector and a reflection ray vector of a left camera, wherein the left angle bisector vector is a left surface normal vector;
s42, searching on a corresponding row of the right view field reflection image, and calculating right surface normal vectors of all pixel points on the corresponding row of the right view field reflection image;
s43, calculating cosine values of included angles between the left surface normal vector and all right surface normal vectors, traversing the whole row of the right view field reflection image to obtain an array of cosine values of all included angles, selecting an array corner mark with the largest cosine value in the array, and determining the position of the largest data corner mark in the right view field reflection image, wherein the position is a matching point corresponding to a point to be matched on the left view field reflection image on the right view field reflection image;
s44, performing parallax calculation on coordinates of a point to be matched on the left view field reflection image and coordinates of a corresponding matching point on the right view field reflection image, wherein a parallax calculation formula of the first parallax information is as follows:
disp1 = abs(yleft1 - yright1）；
wherein yleft1 is the column coordinate of the point to be matched on the left view field reflection image, ydight 1 is the column coordinate of the corresponding matching point on the right view field reflection image, and abs () is expressed as absolute value calculation;
s45, traversing all matching points on the left view field reflection image to acquire first parallax information.
Further, specifically, the stereo matching algorithm is a semi-global stereo matching algorithm and a global stereo matching algorithm.
Further, specifically, the step S5 specifically includes the following steps:
s51, carrying out gradient calculation on the left and right view field reflection images based on a Sobel operator, and setting a cost window;
s52, calculating matching cost by using the absolute value of the phase difference as a cost body;
s53, dynamically determining the mobile offset of the cost window by using dynamic programming selection;
s54, traversing the left and right view field reflected images, obtaining second parallax information, and performing parallax optimization to obtain a second parallax image; the parallax calculation formula of the second parallax information is as follows:
disp2 = abs(yleft2 - yright2）；
wherein yleft2 is the column coordinate of the point to be matched on the left view field reflection image, ydight 2 is the column coordinate of the corresponding matching point on the right view field reflection image, and abs () is expressed as an absolute value calculation.
Further specifically, in the step S6, a correction condition is set:
；
wherein disp1 is a parallax value in the first parallax information, disp2 is a parallax value in the second parallax information, th1 and th2 are preset thresholds, the value range of the thresholds is determined by the image size and resolution of the binocular camera, and w1 and w2 are set weight parameters;
correcting the first parallax map according to the second parallax map and the correction condition by taking the first parallax map as a reference map;
when (when)When disp=disp1, the third disparity map is the first disparity map, and is not corrected;
if it isWith disp= =>Modifying a parallax value at a corresponding position on the first parallax map to be the correction value;
if it isSetting weight parameters w1 and w2, wherein w1+w2=1, wherein the weight parameters w1 and w2 are determined by calibration errors of the system, and the weight parameters can be set when leaving a factoryRespectively giving out calibration reprojection errors rep1 and rep2 of the binocular camera and the display screen;
when rep1 is larger than rep2, the binocular camera is determined to be accurate in calibration, and at the moment, w1= (rep 1/rep 2) ×w2;
when rep1 is larger than rep2, determining that the calibration of the display screen is accurate, wherein w2= (rep 2/rep 1) ×w1;
with Disp =And modifying the parallax value at the corresponding position on the first parallax map to be the correction value.
Further, specifically, in the step S41 and the step S42, the calculation formula of the surface normal vector is:
wherein, the vector w is the incident ray vector, and the vector u is the reflected ray vector;
in the step S43, the calculation formula of the cosine value is:
where nl is the left surface normal vector and nr is the right surface normal vector.
Further specifically, in step S54, the parallax optimization method is to use uniqueness and left-right uniqueness to reject the unqualified parallax value, and use the nearest neighbor value to replace the unqualified parallax value.
A binocular vision and PMD based surface measurement system comprising:
the display screen is used for displaying the stripe image and mapping the stripe image to the surface of the sample to be detected;
a binocular camera arranged outside the display screen, wherein the binocular camera comprises a left camera and a right camera, the left camera shoots a left view field reflection image of the surface of the tested sample, and the right camera shoots a right view field reflection image of the surface of the tested sample;
and the terminal equipment acquires the left view field reflection image and the right view field reflection image and adopts the binocular vision and PMD surface measurement method.
A computer device, comprising:
a processor;
a memory for storing executable instructions;
wherein the processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the binocular vision and PMD surface measurement based method as described above.
A computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to implement a binocular vision and PMD surface measurement based method as described above.
The beneficial effects of the invention are as follows:
(1) The binocular vision and PMD surface measurement method provides verification dimensions and correction dimensions for the PMD matching parameters through the binocular vision matching parameters, improves the stereoscopic matching effect of the PMD, and improves the accuracy of surface measurement.
(2) The invention combines binocular vision and PMD, avoids damaging the phase-based incident-reflection normal vector uniqueness matching principle, reduces stereo matching error, improves the accuracy of three-dimensional measurement reconstruction, improves the sample detection capability of the system, and widens the testable range of samples.
Drawings
The invention will be further described with reference to the drawings and examples.
Fig. 1 is a schematic flow chart of a method according to a first embodiment of the invention.
Fig. 2 is a schematic diagram of a system structure according to a second embodiment of the present invention.
Fig. 3 is a schematic structural diagram of a computer device according to a third embodiment of the present invention.
1, a display screen; 2. a binocular camera; 3. a sample to be tested; 10. a computer device; 1002. a processor; 1004 a memory; 1006. and a transmission device.
Detailed Description
The invention will now be described in further detail with reference to the accompanying drawings. The drawings are simplified schematic representations which merely illustrate the basic structure of the invention and therefore show only the structures which are relevant to the invention.
Example 1: the embodiment of the application provides a binocular vision and PMD (PMD-based surface measurement) method, which comprises the following steps of:
s1, preparing shooting hardware, wherein the shooting hardware comprises: a display screen 1 and a binocular camera 2, wherein the display screen 1 displays a stripe image and maps the stripe image to the surface of a tested sample 3, and the binocular camera 2 is arranged at the outer side of the display screen 1; wherein the stripes are sinusoidal stripes.
S2, binocular stereo correction is carried out on the calibrated binocular camera 2, so that coplanarity and row alignment can be achieved after left and right view field images are corrected.
After the shooting hardware leaves the factory, the display screen 1 and the binocular camera 2 are calibrated, calibration results are stored in the system, and the display screen 1 and the binocular camera 2 are calibrated, so that the display screen 1 and the binocular camera 2 can work in an ideal state.
In step S2, the binocular structure with intersecting optical axes is converted into the standard binocular structure with parallel optical axes through limit correction, and matching points on left and right field images photographed later are located at the same level, so that feature point matching only needs to perform one-dimensional search on the same row of pixels, calculation complexity is reduced, and measurement time and measurement efficiency are improved. Further, the epipolar correction is as follows: the epipolar correction is realized by adopting a Bouguet algorithm, and the algorithm processes the left camera coordinate system and the right camera coordinate system of the binocular camera 2 to coplanar and parallel positions by solving a rotation matrix, so that the same-name points can be ensured to be positioned on the same row of pixels. And the polar line correction can be realized by adopting a Hartley binocular stereo correction algorithm, and compared with a Bouguet algorithm, the polar line correction method has low precision, but does not need parameters calibrated by a camera, and has high correction speed.
S3, image acquisition, wherein the binocular camera 2 after calibration and correction is used for shooting and displaying the image 1 mapped on the surface of the sample 3 to be measured, and left and right view field reflection images are obtained.
And S4, performing phase stereo matching calculation, namely performing stereo matching on the left and right view field reflected images based on a left and right method phasor matching algorithm, and obtaining first parallax information of the images to obtain a first parallax image.
In this embodiment, the stereo matching of the left and right view field reflection images based on the left and right method phasor matching algorithm specifically includes the following steps:
s41, calculating a left angle bisector vector by using a pixel point of the left view field reflection image as a point to be matched and using an incident ray vector and a reflection ray vector of a left camera, wherein the left angle bisector vector is a left surface normal vector;
s42, searching on a corresponding row (pixels in the same row) of the right view field reflection image, and calculating right surface normal vectors of all pixel points on the corresponding row of the right view field reflection image; because the left and right view field images can be coplanar and line aligned after being corrected in the step S2, only the corresponding matching points on the right view field reflection image are required to be matched and searched for on the right view field reflection image by the corresponding line of the point to be matched, and the measurement efficiency is improved.
S43, calculating cosine values of included angles between the normal vectors of the left surface and all right surfaces, wherein a calculation formula of the cosine values is as follows:
where nl is the left surface normal vector and nr is the right surface normal vector;
traversing the whole row of the right view field reflection image to obtain an array of cosine values of all included angles, selecting an array corner mark with the largest cosine value in the array, and determining the position of the largest data corner mark on the right view field reflection image, wherein the position is a matching point (homonymy point) corresponding to a point to be matched on the left view field reflection image on the right view field reflection image;
s44, performing parallax calculation according to the coordinates of the point to be matched on the left view field reflection image and the coordinates of the corresponding matching point on the right view field reflection image, wherein the parallax calculation formula of the first parallax information is as follows:
disp1 = abs(yleft1 - yright1）；
wherein yleft1 is the column coordinate of the point to be matched on the left view field reflection image, ydight 1 is the column coordinate of the corresponding matching point on the right view field reflection image, and abs () is expressed as absolute value calculation;
s45, traversing all matching points on the left view field reflection image to acquire first parallax information.
In step S41 and step S42, the calculation formula of the surface normal vector is:
the vector w is an incident ray vector, and the vector u is a reflected ray vector, and it should be noted that the incident ray vector and the reflected ray vector are determined by coordinates of a camera image pixel point and a calibration result stored in a system.
And S5, binocular stereo matching calculation is performed, stereo matching is performed on the left and right view field reflected images based on a binocular stereo matching algorithm, and second parallax information of the images is obtained, so that a second parallax image is obtained.
In this embodiment, the stereo matching algorithm employs, but is not limited to, a Semi-global stereo matching algorithm (SGBM, semi-Global Matching And Mutual Information) and a global stereo matching algorithm (GBM, global Matching And Mutual Information).
Taking a semi-global stereo matching algorithm as an example, the step S5 specifically includes the following steps:
s51, carrying out gradient calculation on left and right view field reflection images based on a Sobel operator, and setting a cost window;
s52, calculating matching cost by using the absolute value of the phase difference as a cost body;
s53, dynamically determining the mobile offset of the cost window by using dynamic programming selection;
s54, traversing the left and right view field reflected images to obtain second parallax information, and performing parallax optimization to obtain a second parallax image; the parallax calculation formula of the second parallax information is:
disp2 = abs(yleft2 - yright2）；
wherein yleft2 is the column coordinate of the point to be matched on the left view field reflection image, ydight 2 is the column coordinate of the corresponding matching point on the right view field reflection image, and abs () is expressed as the absolute value calculation.
In step S54, the parallax optimization method uses uniqueness and left-right uniqueness to reject unqualified parallax values, and uses nearest neighbor values to replace unqualified parallax values, so as to improve matching accuracy and obtain a second parallax map with better effect.
S6, correcting the parallax map, namely correcting the first parallax map through the second parallax map by taking the first parallax map as a reference map, wherein the corrected first parallax map is a third parallax map.
In the present embodiment, in step S6, a correction condition is set:
；
wherein disp1 is a parallax value in the first parallax information, disp2 is a parallax value in the second parallax information, th1 and th2 are preset thresholds, the value range of the thresholds is determined by the image size and resolution of the binocular camera, and w1 and w2 are set weight parameters; the threshold is generally defined as 1-time minimum target resolution and 2.5-time minimum target resolution, so as to control the difference of the parallax values in the two parallax information, and the threshold can be flexibly adjusted according to requirements.
Correcting the first parallax map according to the second parallax map and the correction condition by taking the first parallax map as a reference map:
(1) When (when)When disp=disp1, the third disparity map is the first disparity map, and is not corrected;
(2) If it isWith disp= =>Modifying the parallax value at the corresponding position on the first parallax map to be the correction value;
(3) If it isSetting weight parameters w1 and w2, wherein w1+w2=1, the weight parameters w1 and w2 are determined by the calibration error degree of the system, and calibration reprojection errors rep1 and rep2 of the binocular camera 2 and the display screen 1 are respectively given when the factory is set; when rep1 is larger than rep2, the binocular camera 2 is determined to be accurate in calibration, and at the moment, w1= (rep 1/rep 2) ×w2; when rep1 is larger than rep2, the calibration of the display screen 1 is determined to be accurate, and at the moment, w2= (rep 2/rep 1) ×w1; with Disp =And modifying the parallax value at the corresponding position on the first parallax map to be the correction value.
And S7, calculating point cloud coordinates (X, Y and Z) based on the third parallax map, and carrying out three-dimensional reconstruction on the surface of the sample 3 to be measured according to all calculated point cloud coordinates.
Coordinates (X, Y, Z) are calculated using a triangular ranging formula, the calculation formula being:
；
；
；
wherein b is the polar difference, that is, the coordinate difference of the optical centers of the two cameras after the stereo correction, disp is the parallax value at the corresponding image coordinate in the third parallax map, xleft is the left camera image horizontal axis coordinate, and f is the camera focal length.
In summary, the binocular vision and PMD-based surface measurement method provides verification dimensions and correction dimensions for PMD matching parameters through binocular vision matching parameters, improves the stereoscopic matching effect of the PMD, and improves the accuracy of surface measurement. In addition, the invention combines binocular vision and PMD, avoids damaging the phase-based incident-reflection normal vector uniqueness matching principle, reduces stereo matching error, improves the accuracy of three-dimensional measurement reconstruction, improves the sample detection capability of the system, and widens the testable range of samples.
Example 2: the embodiment of the application provides a binocular vision and PMD-based surface measurement system, which comprises the following components:
a display screen 1 for displaying the stripe image and mapping it to the surface of the sample 3 to be measured;
a binocular camera 2 disposed outside the display screen, the binocular camera 2 including a left camera and a right camera, the left camera capturing a left field of view reflected image of the surface of the sample 3 to be measured, the right camera capturing a right field of view reflected image of the surface of the sample 3 to be measured;
and a terminal device 4, which acquires the left view field reflection image and the right view field reflection image, and adopts the binocular vision and PMD surface measurement method. The terminal equipment is PC computer or notebook computer or other processing equipment.
The foregoing various modifications and embodiments of the method for measuring a surface based on binocular vision and PMD in the first embodiment of fig. 1 are equally applicable to the system for measuring a surface based on binocular vision and PMD in the present embodiment, and those skilled in the art will clearly recognize that the method for measuring a surface based on binocular vision and PMD in the present embodiment is implemented by the foregoing detailed description of the method for measuring a surface based on binocular vision and PMD, so that the detailed description thereof will not be repeated herein for brevity.
Example 3: the present application provides a computer device comprising a processor and a memory having stored therein at least one instruction or at least one program loaded and executed by the processor to implement a binocular vision and PMD based surface measurement method as provided by the above method embodiments.
Fig. 3 shows a schematic diagram of a hardware architecture of an apparatus for implementing a binocular vision and PMD surface measurement method provided by embodiments of the present application, which may be involved in constructing or incorporating an apparatus or system provided by embodiments of the present application. As shown in fig. 3, the computer device 10 may include one or more processors 1002 (the processors may include, but are not limited to, processing means such as a microprocessor MCU or a programmable logic device FPGA), memory 1004 for storing data, and transmission means 1006 for communication functions. In addition, the method may further include: a display, an input/output interface (I/O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of the I/O interface), a network interface, a power supply, and/or a camera. It will be appreciated by those of ordinary skill in the art that the configuration shown in fig. 3 is merely illustrative and is not intended to limit the configuration of the electronic device described above. For example, computer device 10 may also include more or fewer components than shown in FIG. 3, or have a different configuration than shown in FIG. 3.
It should be noted that the one or more processors and/or other data processing circuits described above may be referred to herein generally as "data processing circuits. The data processing circuit may be embodied in whole or in part in software, hardware, firmware, or any other combination. Furthermore, the data processing circuitry may be a single stand-alone processing module, or incorporated in whole or in part into any of the other elements in the computer device 10 (or mobile device). As referred to in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of the path of the variable resistor termination to interface).
The memory 1004 may be used to store software programs and modules of application software, such as a program instruction/data storage device corresponding to a binocular vision and PMD surface measurement method in the embodiments of the present application, and the processor executes the software programs and modules stored in the memory 1004 to perform various functional applications and data processing, i.e., implement a method as described above. Memory 1004 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 1004 may further include memory located remotely from the processor, which may be connected to computer device 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission means 1006 is for receiving or transmitting data via a network. Specific examples of the network described above may include a wireless network provided by a communications provider of the computer device 10. In one example, the transmission means 1006 includes a network adapter (Network Interface Controller, NIC) that can be connected to other network devices via a base station to communicate with the internet. In one example, the transmission device 1006 may be a Radio Frequency (RF) module for communicating with the internet wirelessly.
The display may be, for example, a touch screen type Liquid Crystal Display (LCD) that may enable a user to interact with a user interface of the computer device 10 (or mobile device).
Example 4: the embodiment of the application further provides a computer readable storage medium, which can be set in a server to store at least one instruction or at least one program related to the binocular vision-based PMD surface measurement method in the method embodiment, where the at least one instruction or the at least one program is loaded and executed by the processor to implement the binocular vision-based PMD surface measurement method provided in the method embodiment.
Alternatively, in this embodiment, the storage medium may be located in at least one network server among a plurality of network servers of the computer network. Alternatively, in the present embodiment, the storage medium may include, but is not limited to: a U-disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Example 5: embodiments of the present invention also provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The computer instructions are read from a computer-readable storage medium by a processor of a computer device, and executed by the processor, cause the computer device to perform a binocular vision and PMD surface measurement based method provided in the various alternative embodiments described above.
It should be noted that: the foregoing sequence of the embodiments of the present application is only for describing, and does not represent the advantages and disadvantages of the embodiments. And the foregoing description has been directed to specific embodiments of this application. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
All embodiments in the application are described in a progressive manner, and identical and similar parts of all embodiments are mutually referred, so that each embodiment mainly describes differences from other embodiments. In particular, for apparatus, devices and storage medium embodiments, the description is relatively simple as it is substantially similar to method embodiments, with reference to the description of method embodiments in part.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program for instructing relevant hardware, where the program may be stored in a computer readable storage medium, and the storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
With the above-described preferred embodiments according to the present invention as an illustration, the above-described descriptions can be used by persons skilled in the relevant art to make various changes and modifications without departing from the scope of the technical idea of the present invention. The technical scope of the present invention is not limited to the description, but must be determined according to the scope of claims.
Claims (10)
1. A binocular vision and PMD based surface measurement method comprising:
s1, preparing shooting hardware, wherein the shooting hardware comprises the following steps: a display screen (1) and a binocular camera (2), wherein the display screen (1) displays a stripe image and maps the stripe image to the surface of a tested sample (3), and the binocular camera (2) is arranged on the outer side of the display screen (1);
s2, carrying out binocular stereo correction, namely carrying out polar correction on the calibrated binocular camera (2), so that coplanarity and row alignment can be realized after left and right view field images are corrected;
s3, acquiring images, wherein the binocular camera (2) after calibration and correction is used for shooting the display screen (1) and mapping the display screen on the surface of the tested sample (3) to acquire left and right view field reflection images;
s4, phase three-dimensional matching calculation is carried out, three-dimensional matching is carried out on the left and right view field reflected images based on a left and right method phasor matching algorithm, first parallax information of the images is obtained, and a first parallax image is obtained;
s5, binocular stereo matching calculation is carried out, stereo matching is carried out on the left and right view field reflection images based on a binocular stereo matching algorithm, second parallax information of the images is obtained, and a second parallax image is obtained;
s6, correcting the parallax map, namely correcting the first parallax map through the second parallax map by taking the first parallax map as a reference map, wherein the corrected first parallax map is a third parallax map;
and S7, performing point cloud computing, namely performing point cloud coordinate computing on the basis of the third parallax map, and performing three-dimensional reconstruction on the surface of the tested sample (3) according to all the computed point cloud coordinates.
2. The binocular vision and PMD based surface measurement method of claim 1, wherein in the step S4, the stereo matching of the left and right field of view reflection images based on a left and right phasor matching algorithm specifically comprises the steps of:
s41, calculating a left angle bisector vector by taking a pixel point of the left view field reflection image as a point to be matched and using an incident ray vector and a reflection ray vector of a left camera, wherein the left angle bisector vector is a left surface normal vector;
s42, searching on a corresponding row of the right view field reflection image, and calculating right surface normal vectors of all pixel points on the corresponding row of the right view field reflection image;
s43, calculating cosine values of included angles between the left surface normal vector and all right surface normal vectors, traversing the whole row of the right view field reflection image to obtain an array of cosine values of all included angles, selecting an array corner mark with the largest cosine value in the array, and determining the position of the largest data corner mark in the right view field reflection image, wherein the position is a matching point corresponding to a point to be matched on the left view field reflection image on the right view field reflection image;
s44, performing parallax calculation on coordinates of a point to be matched on the left view field reflection image and coordinates of a corresponding matching point on the right view field reflection image, wherein a parallax calculation formula of the first parallax information is as follows:
disp1 = abs(yleft1 - yright1）；
wherein yleft1 is the column coordinate of the point to be matched on the left view field reflection image, ydight 1 is the column coordinate of the corresponding matching point on the right view field reflection image, and abs () is expressed as absolute value calculation;
s45, traversing all matching points on the left view field reflection image to acquire first parallax information.
3. The binocular vision and PMD surface measurement method of claim 2, wherein the stereo matching algorithm is a semi-global stereo matching algorithm and a global stereo matching algorithm.
4. The binocular vision and PMD surface measuring method of claim 3, wherein the step S5 specifically comprises the steps of:
s51, carrying out gradient calculation on the left and right view field reflection images based on a Sobel operator, and setting a cost window;
s52, calculating matching cost by using the absolute value of the phase difference as a cost body;
s53, dynamically determining the mobile offset of the cost window by using dynamic programming selection;
s54, traversing the left and right view field reflected images, obtaining second parallax information, and performing parallax optimization to obtain a second parallax image;
wherein, the parallax calculation formula of the second parallax information is:
disp2 = abs(yleft2 - yright2）；
wherein yleft2 is the column coordinate of the point to be matched on the left view field reflection image, ydight 2 is the column coordinate of the corresponding matching point on the right view field reflection image, and abs () is expressed as an absolute value calculation.
5. The binocular vision and PMD based surface measuring method of claim 4, wherein in the step S6, a correction condition is set:
；
wherein disp1 is a parallax value in the first parallax information, disp2 is a parallax value in the second parallax information, th1 and th2 are preset thresholds, the value range of the thresholds is determined by the image size and resolution of the binocular camera (2), and w1 and w2 are set weight parameters;
correcting the first parallax map according to the second parallax map and the correction condition by taking the first parallax map as a reference map;
when (when)When disp=disp1, the third disparity map is the first disparity map, and is not corrected;
if it isWith disp= =>Modifying a parallax value at a corresponding position on the first parallax map to be the correction value;
if it isSetting weight parameters w1 and w2, wherein w1+w2=1, the weight parameters w1 and w2 are determined by calibration errors of a system, and calibration re-projection errors rep1 and rep2 of the binocular camera (2) and the display screen (1) are respectively given when the camera leaves a factory;
when rep1 is larger than rep2, the binocular camera (2) is determined to be accurate in calibration, and at the moment, w1= (rep 1/rep 2) ×w2;
when rep1 is larger than rep2, determining that the calibration of the display screen (1) is accurate, wherein w2= (rep 2/rep 1) ×w1;
with Disp =And modifying the parallax value at the corresponding position on the first parallax map to be the correction value.
6. The binocular vision and PMD based surface measurement method of claim 2, wherein in the step S41 and the step S42, the surface normal vector is calculated as:
wherein, the vector w is the incident ray vector, and the vector u is the reflected ray vector;
in the step S43, the calculation formula of the cosine value is:
where nl is the left surface normal vector and nr is the right surface normal vector.
7. The binocular vision and PMD surface measuring method of claim 4, wherein the parallax optimization method is to reject the unacceptable parallax value using uniqueness and left-right uniqueness and replace the unacceptable parallax value with the nearest neighbor value in step S54.
8. A binocular vision and PMD based surface measurement system comprising:
a display screen (1) for displaying the fringe image and mapping it to the surface of the sample (3) to be measured;
a binocular camera (2) disposed outside the display screen, the binocular camera (2) including a left camera and a right camera, the left camera capturing a left field of view reflected image of the surface of the sample (3) under test, the right camera capturing a right field of view reflected image of the surface of the sample (3) under test;
-a terminal device (4) acquiring the left field of view reflection image and the right field of view reflection image, employing the binocular vision and PMD surface measurement based method according to any of claims 1 to 7.
9. A computer device, comprising:
a processor;
a memory for storing executable instructions;
wherein the processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the binocular vision and PMD surface measurement based method of any of claims 1-7.
10. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program, which when executed by a processor causes the processor to implement the binocular vision and PMD surface measurement based method according to any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311616971.4A CN117346693B (en) | 2023-11-30 | 2023-11-30 | Binocular vision and PMD-based surface measurement method, system, equipment and medium thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311616971.4A CN117346693B (en) | 2023-11-30 | 2023-11-30 | Binocular vision and PMD-based surface measurement method, system, equipment and medium thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117346693A true CN117346693A (en) | 2024-01-05 |
| CN117346693B CN117346693B (en) | 2024-02-23 |
Family
ID=89365262
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311616971.4A Active CN117346693B (en) | 2023-11-30 | 2023-11-30 | Binocular vision and PMD-based surface measurement method, system, equipment and medium thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117346693B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104346829A (en) * | 2013-07-29 | 2015-02-11 | 中国农业机械化科学研究院 | Three-dimensional color reconstruction system and method based on PMD (photonic mixer device) cameras and photographing head |
| CN204392458U (en) * | 2015-02-05 | 2015-06-10 | 吉林纪元时空动漫游戏科技股份有限公司 | Binocular stereo vision method is adopted to obtain the system of three-dimensional stereoscopic visual |
| CN114119747A (en) * | 2021-11-23 | 2022-03-01 | 四川大学 | Three-dimensional flow field flow display method based on PMD wavefront detection |
| CN114111637A (en) * | 2021-11-25 | 2022-03-01 | 天津工业大学 | Stripe structured light three-dimensional reconstruction method based on virtual dual-purpose |
| CN115345822A (en) * | 2022-06-08 | 2022-11-15 | 南京航空航天大学 | Automatic three-dimensional detection method for surface structure light of aviation complex part |
| CN116793247A (en) * | 2023-03-27 | 2023-09-22 | 西安交通大学 | Stripe projection profilometry stripe series correction method based on region statistics |
-
2023
- 2023-11-30 CN CN202311616971.4A patent/CN117346693B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104346829A (en) * | 2013-07-29 | 2015-02-11 | 中国农业机械化科学研究院 | Three-dimensional color reconstruction system and method based on PMD (photonic mixer device) cameras and photographing head |
| CN204392458U (en) * | 2015-02-05 | 2015-06-10 | 吉林纪元时空动漫游戏科技股份有限公司 | Binocular stereo vision method is adopted to obtain the system of three-dimensional stereoscopic visual |
| CN114119747A (en) * | 2021-11-23 | 2022-03-01 | 四川大学 | Three-dimensional flow field flow display method based on PMD wavefront detection |
| CN114111637A (en) * | 2021-11-25 | 2022-03-01 | 天津工业大学 | Stripe structured light three-dimensional reconstruction method based on virtual dual-purpose |
| CN115345822A (en) * | 2022-06-08 | 2022-11-15 | 南京航空航天大学 | Automatic three-dimensional detection method for surface structure light of aviation complex part |
| CN116793247A (en) * | 2023-03-27 | 2023-09-22 | 西安交通大学 | Stripe projection profilometry stripe series correction method based on region statistics |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117346693B (en) | 2024-02-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Luhmann et al. | Sensor modelling and camera calibration for close-range photogrammetry | |
| CN108734744B (en) | Long-distance large-view-field binocular calibration method based on total station | |
| CN105066962B (en) | A kind of high-precision photogrammetric apparatus of the big angle of visual field of multiresolution | |
| CN111210468A (en) | Image depth information acquisition method and device | |
| CN111220126A (en) | Space object pose measurement method based on point features and monocular camera | |
| CN104111059A (en) | Distance measuring and locating device and method and terminal | |
| Ma et al. | Line-scan CCD camera calibration in 2D coordinate measurement | |
| CN111750804A (en) | Object measuring method and device | |
| US20240087167A1 (en) | Compensation of three-dimensional measuring instrument having an autofocus camera | |
| CN114299156A (en) | Method for calibrating and unifying coordinates of multiple cameras in non-overlapping area | |
| CN111311671B (en) | Workpiece measuring method and device, electronic equipment and storage medium | |
| CN112229323A (en) | Six-degree-of-freedom measurement method of checkerboard cooperative target based on monocular vision of mobile phone and application of six-degree-of-freedom measurement method | |
| CN109674443B (en) | Pupil distance measuring method and terminal | |
| CN108399640A (en) | A kind of speculum relative pose measurement method based on camera calibration | |
| CN114037768A (en) | Method and device for joint calibration of multiple sets of tracking scanners | |
| CN117346693B (en) | Binocular vision and PMD-based surface measurement method, system, equipment and medium thereof | |
| CN111754584A (en) | Remote large-field-of-view camera parameter calibration system and method | |
| CN110470216B (en) | Three-lens high-precision vision measurement method and device | |
| Sergiyenko et al. | Multi-view 3D data fusion and patching to reduce Shannon entropy in Robotic Vision | |
| CN107783310B (en) | Calibration method and device of cylindrical lens imaging system | |
| EP4386677A2 (en) | Compensation of three-dimensional measuring instrument having an autofocus camera | |
| CN114693807A (en) | Method and system for reconstructing mapping data of power transmission line image and point cloud | |
| CN111623960B (en) | Method and device for measuring optical axis of structured light module | |
| CN109916341B (en) | Method and system for measuring horizontal inclination angle of product surface | |
| CN115018922A (en) | Distortion parameter calibration method, electronic device and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |