CN111179355A - Binocular camera calibration method combining point cloud and semantic recognition - Google Patents
Binocular camera calibration method combining point cloud and semantic recognition Download PDFInfo
- Publication number
- CN111179355A CN111179355A CN201911325330.7A CN201911325330A CN111179355A CN 111179355 A CN111179355 A CN 111179355A CN 201911325330 A CN201911325330 A CN 201911325330A CN 111179355 A CN111179355 A CN 111179355A
- Authority
- CN
- China
- Prior art keywords
- binocular camera
- marker
- point cloud
- semantic recognition
- coordinate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a binocular camera calibration method combining point cloud and semantic recognition, which relates to the technical field of binocular calibration and comprises the following steps: selecting a marker, and training an image recognition machine learning model; outputting point clouds and pictures by a binocular camera, identifying markers in the pictures by an image identification machine, finding corresponding point clouds from pixels of the binocular camera, positioning the markers, recording coordinates of the markers in a coordinate system of the binocular camera, and simultaneously recording the coordinates of the markers in a world coordinate system to obtain a coordinate combination; changing the relative position of the binocular camera and the marker; repeating for multiple times to obtain multiple groups of coordinate combinations; and (4) inputting the multiple groups of coordinate combinations into a statistical model to obtain the accurate positions of the markers in the world coordinate system. The method realizes calibration by combining the binocular camera point cloud and machine learning image semantic identification, has the advantages of simplicity and strong reliability, can simultaneously correct the possible distortion of the binocular camera point cloud, and greatly reduces the threshold of visual positioning application.
Description
Technical Field
The invention relates to the technical field of binocular calibration, in particular to a binocular camera calibration method combining point cloud and semantic recognition.
Background
The binocular camera has good stability and strong functions due to the binocular vision slam (instant positioning and map construction), and is widely applied to various robots.
However, the existing binocular camera positioning method has the condition that the binocular point cloud is possibly distorted, the calibration method is complex, the accuracy is poor, other existing binocular calibration methods all involve complex data calculation, and the threshold of the vision positioning application is high.
Based on the above problems, there is a need for a binocular camera positioning method combining point cloud and semantic recognition, which is simple and highly reliable, and greatly reduces the threshold of visual positioning application.
Disclosure of Invention
Aiming at the problem in practical application, the invention aims to provide a binocular camera calibration method combining point cloud and semantic recognition, and the specific scheme is as follows:
a binocular camera calibration method combining point cloud and semantic recognition comprises the following steps:
1) selecting a marker, training a pixel-level image semantic recognition machine learning model (M), and enabling the semantic recognition machine learning model (M) to recognize the marker through transfer learning of the model (M);
2) the semantic recognition machine learning model (M) recognizes the marker in the picture, finds the position of the corresponding pixel in the point cloud from the binocular camera pixels, positions the marker, records the coordinate (x) of the marker under a binocular camera coordinate system, and records the coordinate (y) of the marker under a world coordinate system to obtain a coordinate combination (x, y);
3) changing the relative positions of the binocular camera and the marker;
4) repeating the step 2) and the step 3) for a plurality of times to obtain a plurality of groups of coordinate combinations (x, y);
5) and inputting a plurality of groups of coordinate combinations (x, y) into a statistical model f to obtain f (x) - > y.
Further, the statistical model in the step 5) is a mixed model formed by combining a plurality of models;
the statistical model f in the step 5) comprises a conversion matrix, a linear regression or a polynomial regression model.
Further, the mode of changing the relative position in the step 3) includes changing the position of the marker and changing the position of the binocular camera.
Further, in the step 5, the statistical model f is used to realize the conversion from the coordinates of the marker in the binocular camera coordinate system to the coordinates of the marker in the world coordinate system.
Compared with the prior art, the invention has the following beneficial effects: according to the method, through combination of binocular camera point cloud and machine learning image semantic recognition, the relative positions of the markers and the binocular cameras are changed, the coordinates of a plurality of groups of markers under a binocular camera coordinate system and the coordinates under a world coordinate system are recorded, and a calibration method is obtained through calculation of a statistical model.
Drawings
Fig. 1 is a flowchart of a method for calibrating a binocular camera according to the present invention.
Detailed Description
The present invention will be described in further detail with reference to examples and drawings, but the present invention is not limited to these examples.
As shown in fig. 1, a binocular camera calibration method combining point cloud and semantic recognition includes the following steps:
1) selecting a marker, training a pixel level image semantic recognition machine learning model (M), and enabling the semantic recognition machine learning model (M) to recognize the marker through transfer learning of the model (M);
2) outputting a point cloud and a picture by a binocular camera, identifying a marker in the picture by a semantic identification machine learning model (M), finding the position of a corresponding pixel in the point cloud from pixels of the binocular camera, positioning the marker, recording the coordinate (x) of the marker under a coordinate system of the binocular camera, and simultaneously recording the coordinate (y) of the marker under a world coordinate system to obtain a coordinate combination (x, y);
3) changing the relative position of the binocular camera and the marker;
4) repeating the step 2) and the step 3) for multiple times to obtain multiple groups of coordinate combinations (x, y);
5) and inputting the multiple groups of coordinate combinations (x, y) into a statistical model f to obtain the method of f (x) - > y.
The statistical model in the step 5) is a mixed model formed by combining a plurality of models;
the statistical model f in the step 5) comprises a conversion matrix, a linear regression or a polynomial regression model.
The mode of relative position change in the step 3) comprises changing the position of the marker and changing the position of the binocular camera.
And 5, converting the coordinates of the marker in the binocular camera coordinate system to the coordinates of the marker in the world coordinate system by using the statistical model f.
The specific implementation principle of the invention is as follows: the method is suitable for two binocular camera deployment modes, is simple and high in reliability, can simultaneously correct possible distortion of the point cloud of the binocular camera, and greatly reduces a visual positioning application threshold compared with other binocular calibration methods involving complex data calculation.
The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments, and all technical solutions belonging to the idea of the present invention belong to the protection scope of the present invention. It should be noted that modifications and embellishments within the scope of the invention may occur to those skilled in the art without departing from the principle of the invention, and are considered to be within the scope of the invention.
Claims (4)
1. A binocular camera calibration method combining point cloud and semantic recognition is characterized by comprising the following steps:
1) selecting a marker, training a pixel-level image semantic recognition machine learning model (M), and enabling the semantic recognition machine learning model (M) to recognize the marker through transfer learning of the model (M);
2) the semantic recognition machine learning model (M) recognizes the marker in the picture, finds the position of the corresponding pixel in the point cloud from the binocular camera pixels, positions the marker, records the coordinate (x) of the marker under a binocular camera coordinate system, and records the coordinate (y) of the marker under a world coordinate system to obtain a coordinate combination (x, y);
3) changing the relative positions of the binocular camera and the marker;
4) repeating the step 2) and the step 3) for a plurality of times to obtain a plurality of groups of coordinate combinations (x, y);
5) and inputting a plurality of groups of coordinate combinations (x, y) into a statistical model f to obtain f (x) - > y.
2. The binocular camera calibration method combining point cloud and semantic recognition according to claim 1, wherein the statistical model in the step 5) is a hybrid model formed by combining a plurality of models;
the statistical model f in the step 5) comprises a conversion matrix, a linear regression or a polynomial regression model.
3. A binocular camera calibration method combining point cloud and semantic recognition according to claim 1, wherein the manner of relative position change in step 3) includes changing the marker position and changing the binocular camera position.
4. A binocular camera calibration method combining point cloud and semantic recognition according to claim 2, wherein the statistical model f in the step 5 is used to realize the transformation of the coordinates of the marker in the binocular camera coordinate system to the coordinates of the marker in the world coordinate system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911325330.7A CN111179355A (en) | 2019-12-20 | 2019-12-20 | Binocular camera calibration method combining point cloud and semantic recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911325330.7A CN111179355A (en) | 2019-12-20 | 2019-12-20 | Binocular camera calibration method combining point cloud and semantic recognition |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111179355A true CN111179355A (en) | 2020-05-19 |
Family
ID=70652143
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911325330.7A Pending CN111179355A (en) | 2019-12-20 | 2019-12-20 | Binocular camera calibration method combining point cloud and semantic recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111179355A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112581542A (en) * | 2020-12-24 | 2021-03-30 | 北京百度网讯科技有限公司 | Method, device and equipment for evaluating automatic driving monocular calibration algorithm |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105469418A (en) * | 2016-01-04 | 2016-04-06 | 中车青岛四方机车车辆股份有限公司 | Photogrammetry-based wide-field binocular vision calibration device and calibration method |
| CN106709955A (en) * | 2016-12-28 | 2017-05-24 | 天津众阳科技有限公司 | Space coordinate system calibrate system and method based on binocular stereo visual sense |
| CN107367229A (en) * | 2017-04-24 | 2017-11-21 | 天津大学 | Free binocular stereo vision rotating shaft parameter calibration method |
| CN109086785A (en) * | 2017-06-14 | 2018-12-25 | 北京图森未来科技有限公司 | A kind of training method and device of image calibration model |
| CN109118545A (en) * | 2018-07-26 | 2019-01-01 | 深圳市易尚展示股份有限公司 | 3-D imaging system scaling method and system based on rotary shaft and binocular camera |
| CN109215487A (en) * | 2018-08-24 | 2019-01-15 | 宽凳（北京）科技有限公司 | A kind of high-precision cartography method based on deep learning |
| CN109977253A (en) * | 2019-03-29 | 2019-07-05 | 哈尔滨工业大学 | A kind of fast image retrieval method and device based on semanteme and content |
-
2019
- 2019-12-20 CN CN201911325330.7A patent/CN111179355A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105469418A (en) * | 2016-01-04 | 2016-04-06 | 中车青岛四方机车车辆股份有限公司 | Photogrammetry-based wide-field binocular vision calibration device and calibration method |
| CN106709955A (en) * | 2016-12-28 | 2017-05-24 | 天津众阳科技有限公司 | Space coordinate system calibrate system and method based on binocular stereo visual sense |
| CN107367229A (en) * | 2017-04-24 | 2017-11-21 | 天津大学 | Free binocular stereo vision rotating shaft parameter calibration method |
| CN109086785A (en) * | 2017-06-14 | 2018-12-25 | 北京图森未来科技有限公司 | A kind of training method and device of image calibration model |
| CN109118545A (en) * | 2018-07-26 | 2019-01-01 | 深圳市易尚展示股份有限公司 | 3-D imaging system scaling method and system based on rotary shaft and binocular camera |
| CN109215487A (en) * | 2018-08-24 | 2019-01-15 | 宽凳（北京）科技有限公司 | A kind of high-precision cartography method based on deep learning |
| CN109977253A (en) * | 2019-03-29 | 2019-07-05 | 哈尔滨工业大学 | A kind of fast image retrieval method and device based on semanteme and content |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112581542A (en) * | 2020-12-24 | 2021-03-30 | 北京百度网讯科技有限公司 | Method, device and equipment for evaluating automatic driving monocular calibration algorithm |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109270534B (en) | Intelligent vehicle laser sensor and camera online calibration method | |
| CN109931939B (en) | Vehicle positioning method, device, equipment and computer readable storage medium | |
| CN108388641B (en) | Traffic facility map generation method and system based on deep learning | |
| CN106875437B (en) | RGBD three-dimensional reconstruction-oriented key frame extraction method | |
| CN111311689A (en) | Method and system for calibrating relative external parameters of laser radar and camera | |
| CN111551167B (en) | Global navigation auxiliary method based on unmanned aerial vehicle shooting and semantic segmentation | |
| CN102609941A (en) | Three-dimensional registering method based on ToF (Time-of-Flight) depth camera | |
| CN113409459B (en) | Method, device and equipment for producing high-precision map and computer storage medium | |
| CN103886107A (en) | Robot locating and map building system based on ceiling image information | |
| CN104657982A (en) | Calibration method for projector | |
| CN112819903A (en) | Camera and laser radar combined calibration method based on L-shaped calibration plate | |
| CN110378250B (en) | Training method and device for neural network for scene cognition and terminal equipment | |
| CN101702233A (en) | Three-dimension locating method based on three-point collineation marker in video frame | |
| Cvišić et al. | Recalibrating the KITTI dataset camera setup for improved odometry accuracy | |
| CN112907573B (en) | Depth completion method based on 3D convolution | |
| CN110634138A (en) | Bridge deformation monitoring method, device and equipment based on visual perception | |
| CN110736472A (en) | indoor high-precision map representation method based on fusion of vehicle-mounted all-around images and millimeter wave radar | |
| CN112132900A (en) | Visual repositioning method and system | |
| CN110348351B (en) | Image semantic segmentation method, terminal and readable storage medium | |
| CN105513074A (en) | Badminton robot camera calibration method | |
| Sanberg et al. | Extending the stixel world with online self-supervised color modeling for road-versus-obstacle segmentation | |
| CN111179355A (en) | Binocular camera calibration method combining point cloud and semantic recognition | |
| CN105335959A (en) | Quick focusing method and device for imaging apparatus | |
| CN111724432B (en) | Object three-dimensional detection method and device | |
| CN112288821B (en) | Method and device for calibrating external parameters of camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |