CN111427020A - Combined calibration method, device and system for environmental information data acquisition equipment - Google Patents
Combined calibration method, device and system for environmental information data acquisition equipment Download PDFInfo
- Publication number
- CN111427020A CN111427020A CN202010526251.9A CN202010526251A CN111427020A CN 111427020 A CN111427020 A CN 111427020A CN 202010526251 A CN202010526251 A CN 202010526251A CN 111427020 A CN111427020 A CN 111427020A
- Authority
- CN
- China
- Prior art keywords
- camera
- information data
- millimeter wave
- wave radar
- laser radar
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
- G01S7/40—Means for monitoring or calibrating
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/497—Means for monitoring or calibrating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
- G06T2207/10044—Radar image
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Radar Systems Or Details Thereof (AREA)
Abstract
The invention discloses a combined calibration method, a device and a system for environmental information data acquisition equipment, wherein the method comprises the following steps: respectively acquiring data sets of a camera, a millimeter wave radar and a laser radar under a plurality of time nodes, and extracting camera information data, millimeter wave radar information data and laser radar information data under the same time node; matrix sets under corresponding coordinate systems are respectively obtained through matrix transformation based on the data sets, and then the obtained matrix sets are processed to obtain calibration values Tcl of the camera and the laser radar, calibration values Trl of the millimeter wave radar and the laser radar and calibration values Tcr of the camera and the millimeter wave radar, so that combined calibration under the three coordinate systems is completed, and the technical problems of poor data acquisition precision and low uniformity caused by the lack of a method for performing combined calibration on the camera, the millimeter wave radar and the laser radar are solved.
Description
Technical Field
The invention relates to the technical field of camera imaging, in particular to a combined calibration method, a device and a system for environmental information data acquisition equipment.
Background
In manual driving, visual information is mainly used by a driver during driving, wherein the visual information mainly comprises traffic signs, road signs, traffic lights and the like, and people use the visual information as a main basis for decision making. In automatic driving, the camera replaces human eyes to acquire signal data around the vehicle body, and the data acquired by the camera is large in size, low in cost and wide in action range.
However, the camera has inevitable disadvantages that the camera is easily affected by light and has poor shooting effect in places where light is not good or the light is suddenly changed. In order to overcome the influence of light on data acquisition precision, millimeter wave radar detection is added to an imaging system in automatic driving. And because the point cloud imaging of the millimeter wave radar is insensitive to the target edge and the large-angle target, the laser radar is often introduced into a data acquisition system in automatic driving.
Therefore, a joint calibration method is provided, so that a camera, a millimeter wave radar and a laser radar are subjected to joint calibration to improve the accuracy and the uniformity of data acquisition, and the problem to be solved by technical personnel in the field is urgently solved.
Disclosure of Invention
Therefore, the embodiment of the invention provides a combined calibration method, a device and a system for environmental information data acquisition equipment, so as to solve at least part of the technical problems of poor data acquisition precision and low uniformity caused by the lack of a combined calibration method for a camera, a millimeter wave radar and a laser radar.
In order to achieve the above object, the embodiments of the present invention provide the following technical solutions:
the invention provides a combined calibration method of environmental information data acquisition equipment, wherein the environmental information data acquisition equipment comprises the following steps: the camera of the camera is arranged in a preset range away from the rearview mirror;
the millimeter wave radar is arranged at the position which is 40-50cm away from the ground in front of the vehicle;
the laser radar is arranged above the vehicle and 15-25cm away from the top of the vehicle;
the combined calibration method comprises the following steps:
respectively acquiring data sets of a camera, a millimeter wave radar and a laser radar under a plurality of time nodes;
extracting camera information data, millimeter wave radar information data and laser radar information data under the same time node;
respectively obtaining matrix sets under corresponding coordinate systems through matrix transformation based on the data sets;
and processing the obtained matrix sets to obtain calibration values Tcl of the camera and the laser radar, calibration values Trl of the millimeter wave radar and the laser radar and calibration values Tcr of the camera and the millimeter wave radar.
Further, under a plurality of time nodes, the data sets of the camera, the millimeter wave radar and the laser radar are respectively acquired, and the method specifically includes:
respectively acquiring environmental information data which are acquired by a camera at a plurality of time nodes and are based on a calibration board to obtain a camera information data set, and recording the camera information data set as a data set A;
respectively acquiring environment information data based on a calibration plate, which are acquired by a millimeter wave radar at a plurality of time nodes, to obtain a millimeter wave radar information data set, and recording the millimeter wave radar information data set as a data set B;
and respectively acquiring environmental information data which are acquired by the laser radar at a plurality of time nodes and are based on the calibration plate, obtaining laser radar information data, and recording the laser radar information data as a data set C.
Further, the obtaining of the matrix set in the corresponding coordinate system through matrix transformation based on each data set specifically includes: calculating to obtain a transformation matrix set based on a camera coordinate system under a corresponding time node through the data set A, and recording as TA;
calculating to obtain a transformation matrix set based on a millimeter wave radar coordinate system under the corresponding time node through the data set B, and recording as TB;
and calculating to obtain a transformation matrix set based on a laser radar coordinate system under the corresponding time node through the data set C, and recording as TC.
Further, the processing of the obtained matrix sets to obtain calibration values Tcl of the camera and the laser radar, calibration values Trl of the millimeter wave radar and the laser radar, and calibration values Tcr of the camera and the millimeter wave radar specifically includes: converting TA, TB and TC from an Euler coordinate system representation method into a representation form of a lie algebra;
calculating an average value MA of the transformation matrix set TA, calculating an average value MB of the transformation matrix set TB, and calculating an average value MC of the transformation matrix set TC.
Respectively calculating the covariance of any two of the transformation matrix set TA, TB and TC;
by utilizing a calculation method of a hand-eye coordinate system, solving is carried out on a rotation matrix and a translation matrix of a camera and a laser radar to obtain Tcl, solving is carried out on the rotation matrix and the translation matrix of a millimeter wave radar and the laser radar to obtain Trl, and solving is carried out on the rotation matrix and the translation matrix of the camera and the millimeter wave radar to obtain Tcr.
Further, the method further comprises:
calculating to obtain a verification error of the transformation matrix through a formula e = Tcr-Tcl × Trl;
judging whether the verification error e is smaller than a threshold value;
and if the verification error e is larger than the threshold value, optimizing the transformation matrixes TA, TB and TC by using a least square method.
The invention also provides a combined calibration device based on the environmental information data acquisition equipment, which is used for implementing the method, and the device comprises: the data set acquisition unit is used for respectively acquiring data sets of the camera, the millimeter wave radar and the laser radar under a plurality of time nodes;
the data extraction unit is used for extracting camera information data, millimeter wave radar information data and laser radar information data under the same time node;
the set acquisition unit is used for respectively obtaining matrix sets under corresponding coordinate systems through matrix transformation based on the data sets;
and the calibration value output unit is used for processing the obtained matrix sets to obtain the calibration values Tcl of the camera and the laser radar, the calibration values Trl of the millimeter wave radar and the laser radar and the calibration values Tcr of the camera and the millimeter wave radar.
Further, an error verification unit is included, the error verification unit being configured to:
calculating to obtain a verification error of the transformation matrix through a formula e = Tcr-Tcl × Trl;
judging whether the verification error e is smaller than a threshold value;
and if the verification error e is larger than the threshold value, optimizing the transformation matrixes TA, TB and TC by using a least square method.
The invention also provides a combined calibration system based on the environmental information data acquisition equipment, which comprises: a processor and a memory; the memory is to store one or more program instructions;
the processor is configured to execute one or more program instructions to perform the method as described above.
The present invention also provides a computer readable storage medium having one or more program instructions embodied therein for executing the control method of the joint calibration system as described above.
According to the environmental information data acquisition equipment and the combined calibration method, device and system thereof, data sets of a camera, a millimeter wave radar and a laser radar are respectively acquired under a plurality of time nodes, and camera information data, millimeter wave radar information data and laser radar information data under the same time node are extracted; and then respectively obtaining matrix sets under corresponding coordinate systems through matrix transformation based on the data sets, and further processing the obtained matrix sets to obtain calibration values Tcl of the camera and the laser radar, calibration values Trl of the millimeter wave radar and the laser radar and calibration values Tcr of the camera and the millimeter wave radar, thereby completing combined calibration under three coordinate systems, and solving the technical problems of poor data acquisition precision and low uniformity caused by the lack of a method for performing combined calibration on the camera, the millimeter wave radar and the laser radar.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It should be apparent that the drawings in the following description are merely exemplary, and that other embodiments can be derived from the drawings provided by those of ordinary skill in the art without inventive effort.
The structures, ratios, sizes, and the like shown in the present specification are only used for matching with the contents disclosed in the specification, so as to be understood and read by those skilled in the art, and are not used to limit the conditions that the present invention can be implemented, so that the present invention has no technical significance, and any structural modifications, changes in the ratio relationship, or adjustments of the sizes, without affecting the effects and the achievable by the present invention, should still fall within the range that the technical contents disclosed in the present invention can cover.
FIG. 1 is a flow chart of an embodiment of a joint calibration method provided by the present invention;
FIG. 2 is a block diagram of a combined calibration apparatus according to an embodiment of the present invention;
fig. 3 is a block diagram of a combined calibration system according to an embodiment of the present invention.
Detailed Description
The present invention is described in terms of particular embodiments, other advantages and features of the invention will become apparent to those skilled in the art from the following disclosure, and it is to be understood that the described embodiments are merely exemplary of the invention and that it is not intended to limit the invention to the particular embodiments disclosed. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention provides an environmental information data acquisition device, and a combined calibration method, a combined calibration device and a combined calibration system based on the environmental information data acquisition device, so that multiple data detection of a camera, a millimeter wave radar and a laser radar is utilized, the accuracy of environmental information data is improved, the influence of illumination on data acquisition is avoided, and the detection sensitivity of a target edge and a large-angle target is ensured. Meanwhile, when multiple data are detected, the combined calibration of the camera, the millimeter wave radar and the laser radar is realized through the combined calibration method, and the combined calibration method is provided for multiple information data acquisition equipment, so that the calibration accuracy is ensured, and the unified calibration of each coordinate system is realized.
In one embodiment, the present invention provides an environmental information data acquisition device comprising a camera, a millimeter wave radar, and a laser radar; the camera of the camera is arranged in a preset range away from a rearview mirror, the millimeter wave radar is arranged at a position which is 40-50cm away from the ground in front of the vehicle, and the laser radar is arranged at a position which is 15-25cm away from the roof of the vehicle above the vehicle. When the equipment is installed, specifically, the laser radar can be installed above the vehicle at a position about 20cm from the roof of the vehicle, the millimeter wave radar is installed at a height of about 45cm from the ground in front of the vehicle, and finally the camera is installed at a position near the rearview mirror on the front windshield, so that the front target object can be covered.
In the above embodiment, the environmental information data acquisition device provided by the invention utilizes multiple data detection of the camera, the millimeter wave radar and the laser radar, improves the accuracy of environmental information data, avoids the influence of illumination on data acquisition, and ensures the detection sensitivity of target edges and large-angle targets.
Further, in order to match the above multiple data detection device and implement joint calibration under multiple data detection, the present invention further provides a joint calibration method based on the above environment information data acquisition device, as shown in fig. 1, the method includes:
s1: and under a plurality of time nodes, respectively acquiring data sets of the camera, the millimeter wave radar and the laser radar. Specifically, environmental information data which are acquired by a camera at a plurality of time nodes and are based on a calibration board are respectively acquired, a camera information data set is obtained and recorded as a data set A; respectively acquiring environment information data based on a calibration plate, which are acquired by a millimeter wave radar at a plurality of time nodes, to obtain a millimeter wave radar information data set, and recording the millimeter wave radar information data set as a data set B; and respectively acquiring environmental information data which are acquired by the laser radar at a plurality of time nodes and are based on the calibration plate, obtaining laser radar information data, and recording the laser radar information data as a data set C.
Namely, in the calibration process, a fixed combined system is installed, the shooting function of the industrial camera is started, and the laser radar and the millimeter wave radar are started to output point cloud data; the commercial data acquisition software is used for synchronously acquiring industrial camera data, millimeter wave radar data and laser radar data, and the data can be guaranteed to be under the same timestamp, so that the data accuracy is guaranteed. A corner reflector with high reflectivity is arranged at the center of the visual calibration board. And the camera calibration plate with the corner reflector is placed at a position of about 40-50m in front of the vehicle. The vehicle is started, the vehicle runs at a constant speed in the direction of the calibration plate, the data of the industrial camera, the millimeter wave radar and the laser radar are recorded, in the process, the commercial software is used for synchronously recording the multi-source data, and the data of the industrial camera, the millimeter wave radar and the laser radar under the same timestamp are extracted. And then, carrying out data processing, recording the information of the position of the center of the calibration plate detected by the industrial camera as a data set A, recording the position information of the corner reflector at the center of the calibration plate identified by the millimeter wave radar data as a data set B, and similarly recording the position information of the corner reflector at the center of the calibration plate identified by the laser radar as a data set C.
Respectively acquiring environment information data of the camera at a plurality of different time nodes to obtain the environment information data of the camera corresponding to each time node, and forming a set by the environment information data, namely the data set A; respectively acquiring environment information data of the millimeter wave radar at a plurality of different time nodes to obtain the environment information data of each time node corresponding to the millimeter wave radar, and forming a set by the environment information data, namely the data set B; and respectively acquiring environmental information data of the laser radar at a plurality of different time nodes to obtain the environmental information data of each time node corresponding to the laser radar, and forming a set by the environmental information data, namely the data set C.
The environment information data may include traffic environment identifiers such as traffic signs, road signs, traffic lights, and the like.
S2: extracting camera information data, millimeter wave radar information data and laser radar information data under the same time node;
s3: and respectively obtaining matrix sets under corresponding coordinate systems through matrix transformation based on the data sets. Specifically, a transformation matrix set based on a camera coordinate system under a corresponding time node is obtained through calculation of the data set A and is recorded as TA; calculating to obtain a transformation matrix set based on a millimeter wave radar coordinate system under the corresponding time node through the data set B, and recording as TB; and calculating to obtain a transformation matrix set based on a laser radar coordinate system under the corresponding time node through the data set C, and recording as TC.
S4: and processing the obtained matrix sets to obtain calibration values Tcl of the camera and the laser radar, calibration values Trl of the millimeter wave radar and the laser radar and calibration values Tcr of the camera and the millimeter wave radar.
The method specifically comprises the following steps: converting TA, TB and TC from an Euler coordinate system representation method into a representation form of a plum algebra, and in order to calculate the mean value and covariance of a transformation matrix, converting TA, TB and TC from the Euler coordinate system representation method into the representation form of the plum algebra;
calculating an average value MA of a transformation matrix set TA, an average value MB of a transformation matrix set TB and an average value MC of the transformation matrix set TC;
respectively calculating the covariance of any two of the transformation matrix sets TA, TB and TC;
by utilizing a calculation method (AX = XB) of a hand-eye coordinate system, solving is carried out on a rotation matrix and a translation matrix of a camera and a laser radar to obtain Tcl, solving is carried out on the rotation matrix and the translation matrix of a millimeter wave radar and the laser radar to obtain Trl, and solving is carried out on the rotation matrix and the translation matrix of an industrial camera and the millimeter wave radar to obtain Tcr.
In order to improve the detection accuracy, the calibration result may be subjected to error verification, and for this purpose, the method further includes:
calculating to obtain a verification error of the transformation matrix through a formula e = Tcr-Tcl × Trl;
judging whether the verification error e is smaller than a threshold value;
and if the verification error e is larger than the threshold value, optimizing the transformation matrixes TA, TB and TC by using a least square method.
In the above specific embodiment, the joint calibration method provided by the present invention obtains data sets of the camera, the millimeter wave radar and the laser radar respectively at a plurality of time nodes, and extracts camera information data, millimeter wave radar information data and laser radar information data at the same time node; and then respectively obtaining matrix sets under corresponding coordinate systems through matrix transformation based on the data sets, and further processing the obtained matrix sets to obtain calibration values Tcl of the camera and the laser radar, calibration values Trl of the millimeter wave radar and the laser radar and calibration values Tcr of the camera and the millimeter wave radar, thereby completing combined calibration under three coordinate systems, and solving the technical problems of poor data acquisition precision and low uniformity caused by the lack of a method for performing combined calibration on the camera, the millimeter wave radar and the laser radar.
In addition to the above method, the present invention further provides a combined calibration apparatus based on the above environmental information data acquisition device, for implementing the above method, as shown in fig. 2, the apparatus includes:
the system comprises a data set acquisition unit 100, wherein the data set acquisition unit 100 is used for respectively acquiring data sets of a camera, a millimeter wave radar and a laser radar under a plurality of time nodes; specifically, environmental information data which are acquired by a camera at a plurality of time nodes and are based on a calibration board are respectively acquired, a camera information data set is obtained and recorded as a data set A; respectively acquiring environment information data based on a calibration plate, which are acquired by a millimeter wave radar at a plurality of time nodes, to obtain a millimeter wave radar information data set, and recording the millimeter wave radar information data set as a data set B; and respectively acquiring environmental information data which are acquired by the laser radar at a plurality of time nodes and are based on the calibration plate, obtaining laser radar information data, and recording the laser radar information data as a data set C.
Namely, in the calibration process, a fixed combined system is installed, the shooting function of the industrial camera is started, and the laser radar and the millimeter wave radar are started to output point cloud data; the commercial data acquisition software is used for synchronously acquiring industrial camera data, millimeter wave radar data and laser radar data, and the data can be guaranteed to be under the same timestamp, so that the data accuracy is guaranteed. A corner reflector with high reflectivity is arranged at the center of the visual calibration board. And the camera calibration plate with the corner reflector is placed at a position of about 40-50m in front of the vehicle. The vehicle is started, the vehicle runs at a constant speed in the direction of the calibration plate, the data of the industrial camera, the millimeter wave radar and the laser radar are recorded, in the process, the commercial software is used for synchronously recording the multi-source data, and the data of the industrial camera, the millimeter wave radar and the laser radar under the same timestamp are extracted. And then, carrying out data processing, recording the information of the position of the center of the calibration plate detected by the industrial camera as a data set A, recording the position information of the corner reflector at the center of the calibration plate identified by the millimeter wave radar data as a data set B, and similarly recording the position information of the corner reflector at the center of the calibration plate identified by the laser radar as a data set C.
Respectively acquiring environment information data of the camera at a plurality of different time nodes to obtain the environment information data of the camera corresponding to each time node, and forming a set by the environment information data, namely the data set A; respectively acquiring environment information data of the millimeter wave radar at a plurality of different time nodes to obtain the environment information data of each time node corresponding to the millimeter wave radar, and forming a set by the environment information data, namely the data set B; and respectively acquiring environmental information data of the laser radar at a plurality of different time nodes to obtain the environmental information data of each time node corresponding to the laser radar, and forming a set by the environmental information data, namely the data set C.
The environment information data may include traffic environment identifiers such as traffic signs, road signs, traffic lights, and the like.
The data extraction unit 200, the data extraction unit 200 is used for extracting the camera information data, the millimeter wave radar information data and the laser radar information data at the same time node;
a set obtaining unit 300, where the set obtaining unit 300 is configured to obtain matrix sets in corresponding coordinate systems through matrix transformation based on the data sets; specifically, a transformation matrix set based on a camera coordinate system under a corresponding time node is obtained through calculation of the data set A and is recorded as TA; calculating to obtain a transformation matrix set based on a millimeter wave radar coordinate system under the corresponding time node through the data set B, and recording as TB; and calculating to obtain a transformation matrix set based on a laser radar coordinate system under the corresponding time node through the data set C, and recording as TC.
And the calibration value output unit 400 is used for processing the obtained matrix sets to obtain calibration values Tcl of the camera and the laser radar, calibration values Trl of the millimeter wave radar and the laser radar and calibration values Tcr of the camera and the millimeter wave radar.
The calibration value output unit 400 is specifically configured to:
converting TA, TB and TC from an Euler coordinate system representation method into a representation form of a plum algebra, and converting TA, TB and TC from the Euler coordinate system representation method into the representation form of the plum algebra in order to calculate the mean value and covariance of a transformation matrix;
calculating an average value MA of a transformation matrix set TA, an average value MB of a transformation matrix set TB and an average value MC of the transformation matrix set TC;
respectively calculating the covariance of any two of the transformation matrix sets TA, TB and TC;
by utilizing a calculation method (AX = XB) of a hand-eye coordinate system, solving is carried out on a rotation matrix and a translation matrix of a camera and a laser radar to obtain Tcl, solving is carried out on the rotation matrix and the translation matrix of a millimeter wave radar and the laser radar to obtain Trl, and solving is carried out on the rotation matrix and the translation matrix of an industrial camera and the millimeter wave radar to obtain Tcr.
In order to improve the detection precision, the device can carry out error verification on the calibration result, and for this purpose, the device further comprises an error verification unit 500, wherein the error verification unit 500 is used for obtaining the verification error of the transformation matrix through calculation of a formula e = Tcr-Tcl × Trl;
and if the verification error e is larger than the threshold value, optimizing the transformation matrixes TA, TB and TC by using a least square method.
In the above specific embodiment, the joint calibration apparatus provided by the present invention respectively obtains data sets of the camera, the millimeter wave radar and the laser radar at a plurality of time nodes, and extracts camera information data, millimeter wave radar information data and laser radar information data at the same time node; and then respectively obtaining matrix sets under corresponding coordinate systems through matrix transformation based on the data sets, and further processing the obtained matrix sets to obtain calibration values Tcl of the camera and the laser radar, calibration values Trl of the millimeter wave radar and the laser radar and calibration values Tcr of the camera and the millimeter wave radar, thereby completing combined calibration under three coordinate systems, and solving the technical problems of poor data acquisition precision and low uniformity caused by the lack of a method for performing combined calibration on the camera, the millimeter wave radar and the laser radar.
According to a third aspect of the embodiments of the present invention, the present invention further provides a combined calibration system based on the environmental information data acquisition device, as shown in fig. 3, the system includes: a processor 201 and a memory 202;
the memory is to store one or more program instructions;
the processor is configured to execute one or more program instructions to perform the method as described above.
The embodiment of the invention discloses a computer-readable storage medium, in which computer program instructions are stored, and when the computer program instructions are run on a computer, the computer is caused to execute the control method of the joint calibration system.
In an embodiment of the invention, the processor may be an integrated circuit chip having signal processing capability. The Processor may be a general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete Gate or transistor logic device, discrete hardware component.
The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The processor reads the information in the storage medium and completes the steps of the method in combination with the hardware.
The storage medium may be a memory, for example, which may be volatile memory or nonvolatile memory, or which may include both volatile and nonvolatile memory.
The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory.
By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous link DRAM (Synchlink DRAM, S L DRAM), and direct Memory bus RAM (DRTrampbus RAM).
The storage media described in connection with the embodiments of the invention are intended to comprise, without being limited to, these and any other suitable types of memory.
Those skilled in the art will appreciate that the functionality described in the present invention may be implemented in a combination of hardware and software in one or more of the examples described above. When software is applied, the corresponding functionality may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
The above embodiments are only for illustrating the embodiments of the present invention and are not to be construed as limiting the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made on the basis of the embodiments of the present invention shall be included in the scope of the present invention.
Claims (9)
1. A combined calibration method for environmental information data acquisition equipment is characterized in that the environmental information data acquisition equipment comprises the following steps:
the camera of the camera is arranged in a preset range away from the rearview mirror;
the millimeter wave radar is arranged at the position which is 40-50cm away from the ground in front of the vehicle;
the laser radar is arranged above the vehicle and 15-25cm away from the top of the vehicle;
the combined calibration method comprises the following steps:
respectively acquiring data sets of a camera, a millimeter wave radar and a laser radar under a plurality of time nodes;
extracting camera information data, millimeter wave radar information data and laser radar information data under the same time node;
respectively obtaining matrix sets under corresponding coordinate systems through matrix transformation based on the data sets;
and processing the obtained matrix sets to obtain calibration values Tcl of the camera and the laser radar, calibration values Trl of the millimeter wave radar and the laser radar and calibration values Tcr of the camera and the millimeter wave radar.
2. The joint calibration method according to claim 1, wherein the acquiring data sets of the camera, the millimeter wave radar, and the laser radar at the plurality of time nodes respectively specifically comprises:
respectively acquiring environmental information data which are acquired by a camera at a plurality of time nodes and are based on a calibration board to obtain a camera information data set, and recording the camera information data set as a data set A;
respectively acquiring environment information data based on a calibration plate, which are acquired by a millimeter wave radar at a plurality of time nodes, to obtain a millimeter wave radar information data set, and recording the millimeter wave radar information data set as a data set B;
and respectively acquiring environmental information data which are acquired by the laser radar at a plurality of time nodes and are based on the calibration plate, obtaining laser radar information data, and recording the laser radar information data as a data set C.
3. The joint calibration method according to claim 2, wherein the obtaining of the matrix sets in the corresponding coordinate systems based on the data sets through matrix transformation respectively comprises:
calculating to obtain a transformation matrix set based on a camera coordinate system under a corresponding time node through the data set A, and recording as TA;
calculating to obtain a transformation matrix set based on a millimeter wave radar coordinate system under the corresponding time node through the data set B, and recording as TB;
and calculating to obtain a transformation matrix set based on a laser radar coordinate system under the corresponding time node through the data set C, and recording as TC.
4. The joint calibration method according to claim 3, wherein the processing of the obtained matrix sets to obtain the calibration values Tcl of the camera and the lidar, the calibration values Trl of the millimeter-wave radar and the lidar, and the calibration values Tcr of the camera and the millimeter-wave radar specifically comprises:
converting TA, TB and TC from an Euler coordinate system representation method into a representation form of a lie algebra;
calculating an average value MA of a transformation matrix set TA, an average value MB of a transformation matrix set TB and an average value MC of the transformation matrix set TC;
respectively calculating the covariance of any two of the transformation matrix set TA, TB and TC;
by utilizing a calculation method of a hand-eye coordinate system, solving is carried out on a rotation matrix and a translation matrix of a camera and a laser radar to obtain Tcl, solving is carried out on the rotation matrix and the translation matrix of a millimeter wave radar and the laser radar to obtain Trl, and solving is carried out on the rotation matrix and the translation matrix of the camera and the millimeter wave radar to obtain Tcr.
5. The joint calibration method according to claim 1, further comprising:
calculating to obtain a verification error of the transformation matrix through a formula e = Tcr-Tcl × Trl;
judging whether the verification error e is smaller than a threshold value;
and if the verification error e is larger than the threshold value, optimizing the transformation matrixes TA, TB and TC by using a least square method.
6. A joint calibration arrangement for an ambient information data acquisition device for carrying out the method according to any one of claims 1 to 5, the arrangement comprising:
the data set acquisition unit is used for respectively acquiring data sets of the camera, the millimeter wave radar and the laser radar under a plurality of time nodes;
the data extraction unit is used for extracting camera information data, millimeter wave radar information data and laser radar information data under the same time node;
the set acquisition unit is used for respectively obtaining matrix sets under corresponding coordinate systems through matrix transformation based on the data sets;
and the calibration value output unit is used for processing the obtained matrix sets to obtain the calibration values Tcl of the camera and the laser radar, the calibration values Trl of the millimeter wave radar and the laser radar and the calibration values Tcr of the camera and the millimeter wave radar.
7. The joint calibration apparatus according to claim 6, further comprising an error verification unit, wherein the error verification unit is configured to:
calculating to obtain a verification error of the transformation matrix through a formula e = Tcr-Tcl × Trl;
judging whether the verification error e is smaller than a threshold value;
and if the verification error e is larger than the threshold value, optimizing the transformation matrixes TA, TB and TC by using a least square method.
8. A joint calibration system for environmental information data collection devices, the system comprising: a processor and a memory; the memory is to store one or more program instructions;
the processor, configured to execute one or more program instructions to perform the method of any of claims 1-5.
9. A computer readable storage medium, characterized in that the computer readable storage medium contains one or more program instructions for executing the control method of the joint calibration system according to claim 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010526251.9A CN111427020A (en) | 2020-06-11 | 2020-06-11 | Combined calibration method, device and system for environmental information data acquisition equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010526251.9A CN111427020A (en) | 2020-06-11 | 2020-06-11 | Combined calibration method, device and system for environmental information data acquisition equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111427020A true CN111427020A (en) | 2020-07-17 |
Family
ID=71555285
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010526251.9A Pending CN111427020A (en) | 2020-06-11 | 2020-06-11 | Combined calibration method, device and system for environmental information data acquisition equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111427020A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111796248A (en) * | 2020-09-08 | 2020-10-20 | 奥特酷智能科技(南京)有限公司 | Combined calibration method for laser radar and millimeter wave radar |
| CN112083387A (en) * | 2020-09-02 | 2020-12-15 | 南京慧尔视智能科技有限公司 | Radar calibration method and device |
| CN112859022A (en) * | 2021-01-22 | 2021-05-28 | 上海西井信息科技有限公司 | Multi-radar and camera combined calibration method, system, equipment and storage medium |
| CN112907681A (en) * | 2021-02-26 | 2021-06-04 | 北京中科慧眼科技有限公司 | Combined calibration method and system based on millimeter wave radar and binocular camera |
| CN113095154A (en) * | 2021-03-19 | 2021-07-09 | 西安交通大学 | Three-dimensional target detection system and method based on millimeter wave radar and monocular camera |
| CN113256740A (en) * | 2021-06-29 | 2021-08-13 | 湖北亿咖通科技有限公司 | Calibration method of radar and camera, electronic device and storage medium |
| CN113325381A (en) * | 2021-05-28 | 2021-08-31 | 北京百度网讯科技有限公司 | Method, apparatus, device and storage medium for processing data |
| CN114779188A (en) * | 2022-01-24 | 2022-07-22 | 南京慧尔视智能科技有限公司 | Method, device, equipment and medium for evaluating calibration effect |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108536154A (en) * | 2018-05-14 | 2018-09-14 | 重庆师范大学 | Low speed automatic Pilot intelligent wheel chair construction method based on bioelectrical signals control |
| US20200145569A1 (en) * | 2017-10-19 | 2020-05-07 | DeepMap Inc. | Lidar to camera calibration for generating high definition maps |
-
2020
- 2020-06-11 CN CN202010526251.9A patent/CN111427020A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200145569A1 (en) * | 2017-10-19 | 2020-05-07 | DeepMap Inc. | Lidar to camera calibration for generating high definition maps |
| CN108536154A (en) * | 2018-05-14 | 2018-09-14 | 重庆师范大学 | Low speed automatic Pilot intelligent wheel chair construction method based on bioelectrical signals control |
Non-Patent Citations (2)
| Title |
|---|
| 李梦洁: ""基于激光点云与图像结合的行人检测与跟踪技术研究"", 《中国优秀硕士学位论文全文数据库（电子期刊） 信息科技辑》 * |
| 邹博维: ""基于多传感器的车辆局部交通环境感知"", 《中国优秀硕士学位论文全文数据库（电子期刊） 工程科技II辑》 * |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112083387B (en) * | 2020-09-02 | 2022-03-18 | 南京慧尔视智能科技有限公司 | Radar calibration method and device |
| CN112083387A (en) * | 2020-09-02 | 2020-12-15 | 南京慧尔视智能科技有限公司 | Radar calibration method and device |
| CN111796248A (en) * | 2020-09-08 | 2020-10-20 | 奥特酷智能科技(南京)有限公司 | Combined calibration method for laser radar and millimeter wave radar |
| CN112859022A (en) * | 2021-01-22 | 2021-05-28 | 上海西井信息科技有限公司 | Multi-radar and camera combined calibration method, system, equipment and storage medium |
| CN112859022B (en) * | 2021-01-22 | 2023-08-18 | 上海西井科技股份有限公司 | Method, system, equipment and storage medium for joint calibration of multiple radars and cameras |
| WO2022156176A1 (en) * | 2021-01-22 | 2022-07-28 | 上海西井信息科技有限公司 | Multi-radar and camera joint calibration method, system and device, and storage medium |
| CN112907681A (en) * | 2021-02-26 | 2021-06-04 | 北京中科慧眼科技有限公司 | Combined calibration method and system based on millimeter wave radar and binocular camera |
| CN113095154A (en) * | 2021-03-19 | 2021-07-09 | 西安交通大学 | Three-dimensional target detection system and method based on millimeter wave radar and monocular camera |
| CN113325381A (en) * | 2021-05-28 | 2021-08-31 | 北京百度网讯科技有限公司 | Method, apparatus, device and storage medium for processing data |
| CN113325381B (en) * | 2021-05-28 | 2023-08-22 | 北京百度网讯科技有限公司 | Method, apparatus, device and storage medium for processing data |
| CN113256740A (en) * | 2021-06-29 | 2021-08-13 | 湖北亿咖通科技有限公司 | Calibration method of radar and camera, electronic device and storage medium |
| CN114779188A (en) * | 2022-01-24 | 2022-07-22 | 南京慧尔视智能科技有限公司 | Method, device, equipment and medium for evaluating calibration effect |
| CN114779188B (en) * | 2022-01-24 | 2023-11-03 | 南京慧尔视智能科技有限公司 | Method, device, equipment and medium for evaluating calibration effect |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111427020A (en) | Combined calibration method, device and system for environmental information data acquisition equipment | |
| US11380089B1 (en) | All-weather target detection method based on vision and millimeter wave fusion | |
| CN112560972B (en) | Target detection method based on millimeter wave radar prior positioning and visual feature fusion | |
| CN109849930B (en) | Method and device for calculating speed of adjacent vehicle of automatic driving automobile | |
| CN113658256A (en) | Target detection method and device based on laser radar and electronic equipment | |
| CN109683170A (en) | A kind of image traveling area marking method, apparatus, mobile unit and storage medium | |
| CN109741241B (en) | Fisheye image processing method, device, equipment and storage medium | |
| CN109828250B (en) | Radar calibration method, calibration device and terminal equipment | |
| US20210174549A1 (en) | Object-based short range measurement method, device and system, and storage medium | |
| US20220276360A1 (en) | Calibration method and apparatus for sensor, and calibration system | |
| CN109657638A (en) | Barrier localization method, device and terminal | |
| CN113391305B (en) | False target suppression method and device for multi-radar fusion and terminal equipment | |
| CN111476099A (en) | Target detection method, target detection device and terminal equipment | |
| CN112907681A (en) | Combined calibration method and system based on millimeter wave radar and binocular camera | |
| CN113492829B (en) | Data processing method and device | |
| US11244473B2 (en) | Positioning method, positioning apparatus of mobile device and electronic device | |
| CN114839611A (en) | Self-calibration method and device of millimeter wave radar | |
| CN114279453A (en) | Automatic driving vehicle positioning method and device based on vehicle-road cooperation and electronic equipment | |
| CN114966632A (en) | Laser radar calibration method and device, electronic equipment and storage medium | |
| CN112781893A (en) | Spatial synchronization method and device for vehicle-mounted sensor performance test data and storage medium | |
| CN116990776A (en) | Laser radar point cloud compensation method and device, electronic equipment and storage medium | |
| CN115170679A (en) | Calibration method and device for road side camera, electronic equipment and storage medium | |
| CN116168357A (en) | Foreground target machine vision extraction system and method for intelligent vehicle | |
| EP3879301A1 (en) | Radar target simulation system and method | |
| US20220343532A1 (en) | Object-based short range measurement method, device and system, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200717 |