CN112710276A - Binocular ranging method and system based on pixel frequency domain calibration correction CCD/CMOS - Google Patents
Binocular ranging method and system based on pixel frequency domain calibration correction CCD/CMOS Download PDFInfo
- Publication number
- CN112710276A CN112710276A CN202011481812.4A CN202011481812A CN112710276A CN 112710276 A CN112710276 A CN 112710276A CN 202011481812 A CN202011481812 A CN 202011481812A CN 112710276 A CN112710276 A CN 112710276A
- Authority
- CN
- China
- Prior art keywords
- frequency domain
- ccd
- cmos
- binocular
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C3/00—Measuring distances in line of sight; Optical rangefinders
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/02—Measuring arrangements characterised by the use of optical techniques for measuring length, width or thickness
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Electromagnetism (AREA)
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention relates to a binocular distance measurement method and system based on pixel frequency domain calibration correction CCD/CMOS, and belongs to the technical field of optical imaging, image processing and signal processing. The system comprises a Young dynamic interference module and a binocular ranging system consisting of a pair of cameras with similar parameters. The Young dynamic interference module is mainly used for calibrating parameters of a frequency domain response function of a CCD/CMOS sensor used in the binocular ranging system, storing the parameters in a computer, acquiring frequency domain distribution of an incident light field by using the calibrated frequency domain response function after acquiring an image from a camera each time, acquiring space distribution of the incident light field after carrying out inverse Fourier transform, resampling the image distribution according to ideal CCD/CMOS sensor pixel distribution and required resolution to acquire a more accurate image, and inputting the more accurate image into a matching algorithm of the binocular ranging system to acquire a more accurate and higher-resolution measurement result.
Description
Technical Field
The invention relates to the technical field of optical imaging, image processing and signal processing, in particular to a binocular distance measuring method and system based on pixel frequency domain calibration correction CCD/CMOS.
Background
At present, CCD/CMOS is widely applied in the fields of digital photography, aerospace, astronomical detection, spectrum telescope and the like, and is industrially used for measuring size, collecting images and detecting high temperature. The CCD/CMOS can directly convert the optical signal into a digital electric signal to realize the acquisition, storage, transmission, processing and reproduction of the image.
The binocular ranging is a depth information acquisition mode based on a triangular ranging principle, and compared with other depth information acquisition systems, the binocular ranging system has the advantages of simple structure, good environmental adaptability, dense depth information and the like, and is widely applied to the fields of autonomous navigation, obstacle detection and the like.
The binocular ranging system mainly comprises a pair of CCD/CMOS cameras with fixed pose relationship and an operation platform for processing images. The measurement principle is as follows: the same object point in the scene is imaged at different positions of a CCD/CMOS image surface due to the optical geometric relation, the position difference is called parallax, the distance of the object point can be calculated by calculating the parallax and combining the parameters of the pair of cameras, and the depth information can be obtained. According to the measurement principle of the binocular ranging system, the measurement accuracy thereof depends on the accuracy of parallax.
The current measurement algorithm is based on the assumption that CCD/CMOS pixels are uniformly distributed, and does not consider the fact that the actual CCD/CMOS pixels have a certain micro-deviation of a geometric position from the theoretical position, so that a certain error is introduced; in addition, the measurement resolution of the binocular ranging system is limited by the resolution of parallax, and the resolution of parallax is determined by the pixel size of the CCD/CMOS sensor, so the most direct method for improving the measurement resolution of the binocular ranging system is to replace an image sensor with a smaller pixel size.
However, in the case of the CCD/CMOS sensor, the sensitivity of the pixel is easily reduced due to the gradual reduction of the size of the pixel, and at the same time, the CCD/CMOS sensor poses a great challenge to the manufacturing process and is easy to create a technical barrier. In view of this situation, researchers expect that the method of dynamic young's interference correction and pixel frequency domain response function reconstruction CCD/CMOS can be applied to image preprocessing of the binocular ranging system to improve the measurement accuracy of the binocular ranging system.
The CCD/CMOS technology for dynamic Young's interference correction mainly aims at calculating the tiny deviation of the position of a pixel set, and is limited by a CCD/CMOS processing technology, so that the position of a pixel of a CCD/CMOS usually has a certain deviation from a preset integer coordinate position, and the position of an output image is easy to be inaccurate.
Disclosure of Invention
The invention aims to provide a binocular distance measurement method and system based on pixel frequency domain calibration correction CCD/CMOS, which utilizes a dynamic Young's interference method to calibrate the frequency domain response function of a CCD/CMOS sensor and combines a binocular distance measurement technology to ensure that the measurement precision is better and the resolution is higher.
In order to achieve the above object, in a first aspect, the present invention provides a binocular ranging method based on pixel frequency domain calibration correction CCD/CMOS, comprising the steps of:
1) the laser output by the light source is divided into two beams of light by the beam splitter, and after the two beams of light are respectively subjected to frequency modulation by the two acousto-optic modulators, the two beams of light have certain frequency difference and are respectively coupled into the optical fibers;
2) fixing an optical fiber bearing two beams of light on a multi-channel optical fiber fixing base, wherein the two beams of light interfere with each other to form a dynamic interference fringe at a far distance;
3) using a CCD/CMOS sensor to be calibrated to acquire continuous fringe images of two optical fibers at different transverse intervals and longitudinal intervals, wherein the different transverse intervals and the longitudinal intervals correspond to different wave vector directions of an incident light field;
4) calculating each parameter of the pixel frequency domain response function through fitting of a plurality of groups of collected continuous images in different wave vector directions, and storing the parameters for later use;
5) selecting optical lenses with the same parameters from a pair of CCD/CMOS sensors calibrated in the mode to form a binocular camera, performing Fourier transform on images acquired by the CCD/CMOS sensors, combining each parameter of the frequency domain response function in the step 4), solving the frequency domain distribution of an incident light field, obtaining the spatial distribution of the incident light field after inverse Fourier transform, and then performing resampling to obtain an image with ideal pixel position and higher resolution;
6) and taking the processed image as the input of a binocular ranging system to further acquire the measured distance.
In the above technical solution, the frequency domain response function of each pixel of the CCD/CMOS sensor can be calibrated by the dynamic young's interference correction technique, and then the frequency domain response function of the CCD/CMOS sensor is obtained, and according to the fourier optical theory, the output of the CCD/CMOS sensor is the convolution of the incident optical field distribution and the response function in the spatial domain, and is the product of the incident optical field distribution and the response function in the frequency domain. Therefore, after knowing the frequency domain response function of the CCD/CMOS sensor, the Fourier transform result of the incident light field can be obtained from the output of the CCD/CMOS sensor, and after inverse Fourier transform, the accurate incident light field distribution can be recovered, and the sampling is carried out again according to the mode of uniform distribution of pixels and the required resolution (pixel size), thereby improving the measurement precision and resolution of the binocular ranging system.
In step 1), a HeNe laser is used as a light source.
In the step 3), the transverse interval and the longitudinal interval of the two bundles of optical fibers can be changed by replacing the positions of the two bundles of optical fibers in the multichannel optical fiber fixing base.
And 4) in the step 4), fitting and calculating parameters of the pixel frequency domain response function by using a four-step phase method.
In step 5), the image collected by the CCD/CMOS sensor is the product of the frequency domain distribution of the incident light field and the frequency domain response function on the frequency domain.
In step 6), according to the basic formula of the binocular ranging system
d is depth, f is cameraThe focal length of the lens, L is the base line distance of the used camera pair, D is the parallax, after the image processed in the step 5) is used, the obtained parallax is more accurate, and meanwhile, the resolution ratio is higher, so that the measurement result is more accurate and the resolution ratio is higher.
In a second aspect, the binocular ranging system based on the pixel frequency domain calibration correction CCD/CMOS provided by the present invention is used for implementing the binocular ranging method based on the pixel frequency domain calibration correction CCD/CMOS, and includes:
the CCD/CMOS sensors are respectively used for acquiring images in front of two optical lenses of the binocular camera;
the Young dynamic interference module is used for calibrating parameters of a frequency domain response function of the CCD/CMOS sensor;
and the computer is used for fitting and calculating each parameter of the pixel frequency domain response function and storing the parameters.
The Young dynamic interference module comprises a HeNe laser, a beam splitter, a pair of light shields, a pair of polarization controllers, a pair of acousto-optic modulators and an optical fiber base; the beam splitter divides laser emitted by the HeNe laser into two beams of light which respectively enter the light chopper, the polarization controller and the acousto-optic modulator in sequence along two paths.
Compared with the prior art, the invention has the advantages that:
the invention can calibrate the frequency domain response function of the CCD/CMOS sensor, thereby recovering the accurate incident light field distribution, and resampling the incident light field distribution with the required resolution (pixel size) according to the mode of uniform distribution of the pixels, thereby improving the measurement precision and resolution of the binocular ranging system.
Drawings
FIG. 1 is a diagram of a binocular ranging system based on a pixel frequency domain calibration correction CCD/CMOS in an embodiment of the present invention;
FIG. 2 is a schematic diagram of a binocular ranging system according to an embodiment of the present invention;
FIG. 3 is a flowchart of a binocular ranging method based on pixel frequency domain calibration correction CCD/CMOS in the embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described with reference to the following embodiments and accompanying drawings. It is to be understood that the embodiments described are only a few embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the described embodiments without any inventive step, are within the scope of protection of the invention.
Unless defined otherwise, technical or scientific terms used herein shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention belongs. The use of the word "comprise" or "comprises", and the like, in the context of this application, is intended to mean that the elements or items listed before that word, in addition to those listed after that word, do not exclude other elements or items. The terms "connected" or "coupled" and the like are not restricted to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", and the like are used merely to indicate relative positional relationships, and when the absolute position of the object being described is changed, the relative positional relationships may also be changed accordingly.
Examples
The binocular distance measuring system based on the pixel frequency domain calibration correction CCD/CMOS comprises a pair of CCD/CMOS sensors 7, a pair of optical lenses 8 with the same parameters and a CCD/CMOS dynamic Young's interference calibration pixel deviation and frequency domain response function reconstruction module. The CCD/CMOS dynamic Young's interference calibration pixel shift and frequency domain response function reconstruction module comprises a HeNe laser 1, a beam splitter 2, a pair of light shields 3, a pair of polarization controllers 4, a pair of acousto-optic modulators (AOMs) 5 and an optical fiber base 6.
Firstly, calibrating a CCD/CMOS sensor in a binocular ranging system. As shown in figure 1, a CCD/CMOS dynamic Young interference is used for calibrating a pixel offset and frequency domain response function reconstruction module, laser generated by a HeNe laser 1 is coupled into an optical fiber, and then split into two beams of light beams 2, the two beams of light beams are respectively modulated by an acousto-optic modulator 5 to generate a certain frequency difference, and then the two beams of optical fibers are respectively fixed at different positions of a multichannel optical fiber base, so that the two beams of light interfere in space. The emergent light intensity of the two optical fibers is measured by using the light chopper 3 to shade one optical fiber light beam and measuring the emergent light intensity of the other optical fiber light beam. The shutter 3 is opened, the CCD/CMOS sensor 7 to be calibrated is used for recording an interference fringe image for a period of time, and the interference fringe image is stored. The fixed position of the fiber is changed, the interference fringes within a period of time are recaptured, and the same operation is repeated for a plurality of times. And all the captured interference fringe images, the CCD/CMOS sensor 7 and the relevant parameters of the laser used are input into a computer program to obtain the frequency domain response function of the CCD/CMOS sensor 7.
Referring to fig. 2, a pair of calibrated CCD/CMOS sensors and optical lenses with the same parameters are used to form a binocular camera, and a scene to be measured is photographed to obtain an image. The image collected by the sensor is subjected to Fourier transform, and then the frequency domain distribution of the incident light field can be obtained by combining with the calibrated CCD/CMOS frequency domain response function, after the Fourier inverse transformation, the spatial distribution of the incident light field can be obtained, then the resampling is carried out, the image with ideal pixel position and higher resolution is obtained, the image is input into a binocular matching algorithm for calculation, the depth of the scene to be measured is obtained, and the overall flow is shown in figure 3. After the CCD/CMOS sensor is calibrated, a more accurate imaging position of the object point to be measured can be obtained, and therefore a more accurate distance measurement result of the object point to be measured is obtained.
Claims (8)
1. A binocular distance measurement method based on pixel frequency domain calibration correction CCD/CMOS is characterized by comprising the following steps:
1) the laser output by the light source is divided into two beams of light by the beam splitter, and after the two beams of light are respectively subjected to frequency modulation by the two acousto-optic modulators, the two beams of light have certain frequency difference and are respectively coupled into the optical fibers;
2) fixing an optical fiber bearing two beams of light on a multi-channel optical fiber fixing base, wherein the two beams of light interfere with each other to form a dynamic interference fringe at a far distance;
3) using a CCD/CMOS sensor to be calibrated to acquire continuous fringe images of two optical fibers at different transverse intervals and longitudinal intervals, wherein the different transverse intervals and the longitudinal intervals correspond to different wave vector directions of an incident light field;
4) calculating each parameter of the pixel frequency domain response function through fitting of a plurality of groups of collected continuous images in different wave vector directions, and storing the parameters for later use;
5) selecting optical lenses with the same parameters from a pair of CCD/CMOS sensors calibrated in the mode to form a binocular camera, performing Fourier transform on images acquired by the CCD/CMOS sensors, combining each parameter of the frequency domain response function in the step 4), solving the frequency domain distribution of an incident light field, obtaining the spatial distribution of the incident light field after inverse Fourier transform, and then performing resampling to obtain an image with ideal pixel position and higher resolution;
6) and taking the processed image as the input of a binocular ranging system to further acquire the measured distance.
2. The binocular ranging method based on the pixel frequency domain calibration correction CCD/CMOS as claimed in claim 1, wherein in step 1), HeNe laser is used as light source.
3. The binocular ranging method based on the pixel frequency domain calibration correction CCD/CMOS as claimed in claim 1, wherein in step 3), the lateral spacing and the longitudinal spacing of the two optical fibers can be changed by replacing the positions of the two optical fibers in the multi-channel optical fiber fixing base.
4. The binocular distance measuring method based on the pixel frequency domain calibration correction CCD/CMOS as claimed in claim 1, wherein in the step 4), the parameters of the pixel frequency domain response function are calculated by using a four-step phase method in a fitting manner.
5. The binocular distance measuring method based on the pixel frequency domain calibration correction CCD/CMOS as claimed in claim 1, wherein in step 5), the image collected by the CCD/CMOS sensor is the product of the frequency domain distribution of the incident light field and the frequency domain response function in the frequency domain.
6. The binocular distance measuring method based on the pixel frequency domain calibration correction CCD/CMOS as claimed in claim 1, wherein in step 6), according to the basic formula of the binocular distance measuring system
D is the depth, f is the focal length of the camera lens, L is the baseline distance of the used camera pair, D is the parallax, after the image processed in the step 5) is used, the acquired parallax is more accurate, and meanwhile, the resolution ratio is higher, so that the measurement result is more accurate and the resolution ratio is higher.
7. A binocular ranging system based on a pixel frequency domain calibration correction CCD/CMOS is used for realizing the binocular ranging method based on the pixel frequency domain calibration correction CCD/CMOS as claimed in any one of claims 1-6, and is characterized by comprising the following steps:
the CCD/CMOS sensors are respectively used for acquiring images in front of two optical lenses of the binocular camera;
the Young dynamic interference module is used for calibrating parameters of a frequency domain response function of the CCD/CMOS sensor;
and the computer is used for fitting and calculating each parameter of the pixel frequency domain response function and storing the parameters.
8. The binocular ranging system based on the pixel frequency domain calibration rectification CCD/CMOS as claimed in claim 7, wherein the Young's dynamic interference module comprises a HeNe laser, a beam splitter, a pair of shutters, a pair of polarization controllers, a pair of acousto-optic modulators and a fiber base;
the beam splitter divides laser emitted by the HeNe laser into two beams of light which respectively enter the light chopper, the polarization controller and the acousto-optic modulator in sequence along two paths.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011481812.4A CN112710276A (en) | 2020-12-15 | 2020-12-15 | Binocular ranging method and system based on pixel frequency domain calibration correction CCD/CMOS |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011481812.4A CN112710276A (en) | 2020-12-15 | 2020-12-15 | Binocular ranging method and system based on pixel frequency domain calibration correction CCD/CMOS |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112710276A true CN112710276A (en) | 2021-04-27 |
Family
ID=75542178
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011481812.4A Pending CN112710276A (en) | 2020-12-15 | 2020-12-15 | Binocular ranging method and system based on pixel frequency domain calibration correction CCD/CMOS |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112710276A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20000040404A (en) * | 1998-12-18 | 2000-07-05 | 이구택 | Method and system for compensating measured error when measuring width using ccd camera |
| CN104320598A (en) * | 2014-11-06 | 2015-01-28 | 中国科学院光电研究院 | Aberration-free image reconstruction method based on CCD array pixel response function frequency domain calibration |
| CN105841925A (en) * | 2016-03-22 | 2016-08-10 | 中国科学院国家空间科学中心 | Detector pixel response Fourier spectrum acquisition-based image reconstruction method |
| CN108245788A (en) * | 2017-12-27 | 2018-07-06 | 苏州雷泰医疗科技有限公司 | A kind of binocular range unit and method, the accelerator radiotherapy system including the device |
| US20190098283A1 (en) * | 2017-09-15 | 2019-03-28 | Elphel, Inc. | Method for the FPGA-Based Long Range Multi-View Stereo with Differential Image Rectification |
| CN111664870A (en) * | 2020-06-04 | 2020-09-15 | 北京控制工程研究所 | Dynamic Young laser interference fringe calibration system and detector pixel geometric position deviation calibration method |
-
2020
- 2020-12-15 CN CN202011481812.4A patent/CN112710276A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20000040404A (en) * | 1998-12-18 | 2000-07-05 | 이구택 | Method and system for compensating measured error when measuring width using ccd camera |
| CN104320598A (en) * | 2014-11-06 | 2015-01-28 | 中国科学院光电研究院 | Aberration-free image reconstruction method based on CCD array pixel response function frequency domain calibration |
| CN105841925A (en) * | 2016-03-22 | 2016-08-10 | 中国科学院国家空间科学中心 | Detector pixel response Fourier spectrum acquisition-based image reconstruction method |
| US20190098283A1 (en) * | 2017-09-15 | 2019-03-28 | Elphel, Inc. | Method for the FPGA-Based Long Range Multi-View Stereo with Differential Image Rectification |
| CN108245788A (en) * | 2017-12-27 | 2018-07-06 | 苏州雷泰医疗科技有限公司 | A kind of binocular range unit and method, the accelerator radiotherapy system including the device |
| CN111664870A (en) * | 2020-06-04 | 2020-09-15 | 北京控制工程研究所 | Dynamic Young laser interference fringe calibration system and detector pixel geometric position deviation calibration method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102494609B (en) | Three-dimensional photographing process based on laser probe array and device utilizing same | |
| Lindner et al. | Calibration of the intensity-related distance error of the PMD ToF-camera | |
| KR970001140B1 (en) | Distance measuring method and apparatus | |
| CN102365859A (en) | Imaging apparatus and imaging method | |
| Zhu et al. | Camera calibration method based on optimal polarization angle | |
| CN113205592B (en) | Light field three-dimensional reconstruction method and system based on phase similarity | |
| CN103727891A (en) | Synchronous three-dimensional speckle interferometric measurement system and method | |
| CN115775303B (en) | Three-dimensional reconstruction method for high-reflection object based on deep learning and illumination model | |
| CN104568963A (en) | Online three-dimensional detection device based on RGB structured light | |
| Reulke | Combination of distance data with high resolution images | |
| CN113643436B (en) | Depth data splicing and fusion method and device | |
| CN104279981A (en) | Mirror surface/mirror-surface-like object absolute surface shape measuring method and device based on stripe reflection | |
| JP2006322853A (en) | Distance measuring device, distance measuring method and distance measuring program | |
| Gao et al. | Stereo camera calibration for large field of view digital image correlation using zoom lens | |
| CN111971956B (en) | Method and system for dynamic stereo calibration | |
| CN103712569A (en) | Single image rapid phase displacement system and phase detection method based on deflection angles | |
| JPH102722A (en) | Three-dimensional position-measuring apparatus | |
| US7133570B1 (en) | Calibrated sensor and method for calibrating same | |
| Chen et al. | Electrically tunable lens assisted absolute phase unwrapping for large depth-of-field 3D microscopic structured-light imaging | |
| CN112710276A (en) | Binocular ranging method and system based on pixel frequency domain calibration correction CCD/CMOS | |
| JP2006323693A (en) | Processor, and method and program for processing image | |
| de Souza et al. | Combining 3D models with 2D infrared images for medical applications | |
| CN112393694B (en) | Measurement method for improving precision of photoelectric autocollimator based on pixel frequency domain calibration | |
| JP4651550B2 (en) | Three-dimensional coordinate measuring apparatus and method | |
| CN212030748U (en) | Transient temperature measuring device based on snapshot spectral imaging technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210427 |
|
| RJ01 | Rejection of invention patent application after publication |