CN117523105B

CN117523105B - Three-dimensional scene reconstruction method for laser radar and multi-camera data fusion

Info

Publication number: CN117523105B
Application number: CN202311583868.4A
Authority: CN
Inventors: 王珂; 范君国; 孙祺淏; 许润泽
Original assignee: Zhengzhou Research Institute Of Harbin Institute Of Technology; Harbin Institute of Technology
Current assignee: Zhengzhou Research Institute Of Harbin Institute Of Technology; Harbin Institute of Technology
Priority date: 2023-11-24
Filing date: 2023-11-24
Publication date: 2024-05-28
Anticipated expiration: 2043-11-24
Also published as: CN117523105A

Abstract

A three-dimensional scene reconstruction method for laser radar and multi-camera data fusion relates to the technical field of computer vision. The method solves the problems of small single scanning scene range and low reconstruction efficiency in the three-dimensional reconstruction method combining a single camera and a single radar in the prior art. The invention can realize three-dimensional reconstruction by adopting a data fusion technology of a single laser radar or multiple laser radars and a plurality of cameras, and the method comprises the following steps: calibrating the laser radar and each camera to obtain an optimal external parameter matrix between the laser radar and each camera; dividing the three-dimensional scene to be detected into areas, so that two adjacent scanning areas are partially overlapped; and scanning and data fusion are carried out on each scanning area in the three-dimensional scene to be detected by using the laser radar and all cameras, sub-images corresponding to the scanning areas are obtained, and data fusion is carried out on all the sub-images, so that three-dimensional scene reconstruction is completed. When multiple radars are used, calibration between radars is also included. The method is mainly used for reconstructing indoor and outdoor three-dimensional scenes.

Description

Three-dimensional scene reconstruction method for laser radar and multi-camera data fusion

Technical Field

The invention relates to the technical field of computer vision, and in particular can be used for building reconstruction of ancient cultural relics, industrial factory building reconstruction and the like.

Background

The three-dimensional reconstruction technology refers to a digital three-dimensional modeling technology for realizing a real world by using image information of a camera or point cloud information of a radar. The generated digital model can be used in subsequent motion simulation and in the currently popular virtual reality technology. The common reconstruction method is to reconstruct by using only a camera or reconstructing by using only a radar, both the reconstruction methods have problems, the camera has a small visual angle, and the estimated depth is inaccurate, so that the problems are caused in the aspects of reconstruction efficiency and accuracy. The disadvantage of using radar alone for reconstruction is that only depth information is available, and no surface color information of the object is available. In recent years, along with the popularity of RGBD cameras, a three-dimensional reconstruction method combines depth and color information to reconstruct, but the combined reconstruction method uses a single camera and a single radar to reconstruct three-dimensionally, and the combined method has the defects of small single-scanning scene range and low reconstruction efficiency, so the problems are needed to be solved.

Disclosure of Invention

The invention aims to solve the problems of small single scanning scene range and low reconstruction efficiency in the three-dimensional reconstruction method combining a single camera and a single radar in the prior art; the invention provides a three-dimensional scene reconstruction method for laser radar and multi-camera data fusion, which specifically comprises two methods, wherein one method is realized by utilizing a single laser radar and a plurality of cameras, and the other method is realized by utilizing two laser radars and a plurality of cameras.

(One) first scheme:

A three-dimensional scene reconstruction method for single laser radar and multi-camera data fusion comprises the following steps:

Fixing a single laser radar on a rotary platform, wherein the rotary platform and a multi-camera are fixed on the same holder, and the acquisition angles of the multi-camera are different;

calibrating the laser radar and each camera to obtain an optimal external parameter matrix between the laser radar and each camera;

dividing the three-dimensional scene to be detected into a plurality of scanning areas, wherein two adjacent scanning areas are partially overlapped;

Scanning and data fusion are carried out on each scanning area in the three-dimensional scene to be detected by using a laser radar and all cameras to obtain subgraphs corresponding to each scanning area, and data fusion is carried out on all subgraphs to finish three-dimensional scene reconstruction;

the specific process for obtaining the subgraph corresponding to the current scanning area comprises the following steps:

s1, a rotating platform drives a laser radar to move, a cradle head drives all cameras to move, a current scanning area is scanned, multi-frame cloud data are collected by the laser radar in the current scanning process, and multi-frame images are collected by each camera;

S2, converting each frame of point cloud data obtained in the current scanning process into first frame of point cloud data under a laser radar coordinate system where the first frame of point cloud data is located, and removing repeated points in the first frame of point cloud data to obtain corrected first frame of point cloud data;

Converting the multi-frame images acquired by each camera in the current scanning process into a first frame image of the camera under a camera coordinate system where the first frame image of the camera is positioned, and removing repeated points in the first frame image to obtain a corrected first frame image corresponding to each camera;

S3, transforming the corrected first frame point cloud data to the current camera coordinate system through an optimal external reference matrix between the laser radar and each camera, and transforming the internal reference matrix of the current camera to the pixel coordinate system, so that the position information of the point in the corrected first frame point cloud data in the pixel coordinate system corresponding to the current camera is obtained;

S4, extracting colors from the first frame images of the corresponding cameras according to the position information of the points in the corrected first frame point cloud data under the pixel coordinates corresponding to the corresponding cameras, giving the extracted colors to the corresponding points in the corrected first frame point cloud data, obtaining first frame point cloud data with color information corresponding to the current scanning area, and taking the first frame point cloud data as a subgraph.

Preferably, the step S2 of converting each frame of point cloud data obtained in the current scanning process into the first frame of point cloud data in the laser radar coordinate system where the first frame of point cloud data is located includes:

In the current scanning process, according to the rotation angle of the motion model of the rotating platform and the cradle head between each frame of point cloud data and the first frame of point cloud data, a rotation matrix between a laser radar coordinate system where each frame of point cloud data is located and a laser radar coordinate system where the first frame of point cloud data is located is obtained;

Multiplying each frame of point cloud data by a corresponding rotation matrix, so that the frame of point cloud data is converted into first frame of point cloud data under a laser radar coordinate system where the first frame of point cloud data is located;

S2, acquiring multi-frame images by each camera in the current scanning process, and converting the multi-frame images into the first frame images in the camera coordinate system of the first frame images of the camera, wherein the specific process comprises the following steps:

in the current scanning process, according to the rotation angle of the motion model of the cradle head between each frame of image and the first frame of image, a rotation matrix between a camera coordinate system where each frame of image is positioned and a camera coordinate system where the first frame of image is positioned is obtained;

And multiplying each frame of image by a corresponding rotation matrix, so that the frame of image is converted into a first frame of image under the camera coordinates of the first frame of image.

Preferably, all sub-graphs are data fused by GICP algorithm.

Preferably, the specific process of calibrating the laser radar and each camera to obtain the optimal external parameter matrix between the laser radar and each camera comprises a coarse positioning process and a fine positioning process;

the coarse positioning process is used for acquiring an initial external parameter matrix from the camera to the laser radar;

and the fine positioning process is used for correcting the initial external parameter matrix to obtain an optimal external parameter matrix.

Preferably, the coarse positioning process comprises:

A1, acquiring point cloud data and images at each acquisition time point: the position of the polygonal object at each acquisition time is adjusted, and the laser radar and the current camera acquire data of the polygonal object at the same visual angle at each acquisition time to obtain point cloud data and images corresponding to each acquisition time;

A2, acquiring plane data and pixel range of the polygonal area: carrying out data extraction on the point cloud data at each acquisition time to obtain plane data; selecting pixel coordinates of each vertex of the polygonal object in the image at the acquisition time to obtain pixel coordinates of each pixel point in a polygonal area surrounded by the pixel coordinates of each vertex;

A3, coordinate transformation: traversing each pixel in the polygonal area, converting all pixel coordinates in the polygonal area into a camera coordinate system by using an internal reference matrix of a camera, and converting pixel coordinates of each pixel point in the polygonal area under the camera coordinate system into a laser radar coordinate system according to a given preset external reference matrix to obtain position information of each pixel point in the polygonal area under the laser radar coordinate system;

A4, constructing a first total residual error function: constructing a point-to-plane distance residual function according to the position information and the point cloud data in the polygonal area under the laser radar coordinate system corresponding to the image at each acquisition time; superposing the point-to-plane distance residual functions constructed at each acquisition time to obtain a first total residual function;

A5, obtaining an initial external parameter matrix: taking a given preset external reference matrix as an initial value of a Gaussian Newton iteration method, solving a first total residual function, and obtaining an initial external reference matrix from a camera to a laser radar;

The fine positioning process comprises the following steps:

b1, acquiring dense point clouds and images:

according to the current positions of the laser radar and the camera, the laser radar and the camera synchronously move to scan a scene with side line information to realize data acquisition, and the laser radar moves towards the same direction in the moving process;

processing the point cloud data acquired in the movement process of the laser radar by utilizing a synchronous positioning and mapping algorithm A-loam to obtain dense point cloud, and simultaneously, reserving the last image acquired by a camera in the movement process;

B2, 2D image edge extraction: extracting edge characteristics of the reserved image by using an edge characteristic algorithm to obtain a 2D image edge;

B3, coordinate transformation: traversing pixel coordinates of all pixel points on the 2D image edge, converting the pixel coordinates of all pixel points on the 2D image edge into a camera coordinate system by utilizing an internal reference matrix of a camera, and then converting the pixel coordinates of all pixel points on the 2D image edge under the camera coordinate system into a laser radar coordinate system according to the initial external reference matrix obtained in the step A5 to obtain position information of all pixel points on the 2D image edge under the laser radar coordinate system;

b4, 3D point cloud edge extraction: voxelized is carried out on the dense point clouds extracted under the laser radar coordinate system, planes in the dense point clouds are extracted, and plane edges intersecting each other pairwise are used as 3D point cloud edges;

B5, matching the 2D image edge with the 3D point cloud edge: matching a 3D image edge obtained by transforming the 2D image edge with a 3D point cloud edge under a laser radar coordinate system by utilizing knn algorithm, and finding a 3D point cloud edge corresponding to the 3D image edge obtained by transforming the 2D image edge;

B6, constructing a second total residual function: constructing a point-to-line distance residual function by utilizing the successfully matched 3D image edge position information obtained by 2D image edge transformation and the corresponding 3D point cloud data on the 3D point cloud edge; superposing the point-to-line distance residual functions corresponding to all the successful matching results to obtain a second total residual function;

And B7, obtaining an optimal external parameter matrix: and taking the initial external parameter matrix as an initial value of a Gaussian Newton iteration method, and solving a second total residual function to obtain an optimal external parameter matrix from the camera to the laser radar.

(II) second scheme:

the three-dimensional scene reconstruction method for multi-laser radar and multi-camera data fusion comprises the following steps:

Two laser radars are fixed on a rotary platform, the rotary platform and a multi-camera are fixed on the same cloud platform, the acquisition angles of the multi-camera are different, and the directions of the two laser radars are different;

Calibrating each laser radar and each camera to obtain an optimal external parameter matrix between each laser radar and each camera;

calibrating the two laser radars to obtain an external parameter matrix between the two laser radars;

Scanning and data fusion are carried out on each scanning area in the three-dimensional scene to be detected by utilizing two laser radars and all cameras, sub-images corresponding to the scanning areas are obtained, and data fusion is carried out on all the sub-images, so that three-dimensional scene reconstruction is completed;

S1, a rotating platform drives laser radars to move, a cradle head drives all cameras to move, a current scanning area is scanned, multiple frames of cloud data are collected by each laser radar in the current scanning process, and multiple frames of images are collected by each camera;

S2, converting each frame of point cloud data obtained by each laser radar in the current scanning process into first frame of point cloud data of the laser radar under a laser radar coordinate system where the first frame of point cloud data of the laser radar is located, eliminating repeated points in the first frame of point cloud data, carrying out data fusion on the two first frames of point cloud data with the repeated points eliminated, eliminating the repeated points, and obtaining corrected first frame of point cloud data;

s3, transforming the corrected first frame point cloud data to the current camera coordinate system through an optimal external parameter matrix between any laser radar and each camera, and transforming the internal parameter matrix of the current camera to the pixel coordinate system, so that the position information of the point in the corrected first frame point cloud data in the pixel coordinate system corresponding to the current camera is obtained;

Preferably, the specific process of calibrating the two laser radars to obtain the external parameter matrix between the two laser radars comprises the following steps:

The two laser radars synchronously move, any object is scanned to realize data acquisition, and the point cloud data acquired in the moving process of each laser radar is processed by utilizing a synchronous positioning and mapping algorithm A-loam to obtain a frame of dense point cloud; and processing dense point clouds corresponding to the two laser radars by using a fast robust variant nearest neighbor iterative algorithm, and solving an extrinsic matrix between the two laser radars.

Preferably, the coarse positioning process comprises:

a3, coordinate transformation: traversing each pixel in the polygonal area, converting all pixel coordinates in the polygonal area into a camera coordinate system by using an internal reference matrix of a camera, and converting the pixel coordinates of each pixel point in the polygonal area under the camera coordinate system into a laser radar coordinate system according to a given preset external reference matrix (which can be said to be a ruler), so as to obtain the position information of each pixel point in the polygonal area under the laser radar coordinate system;

The fine positioning process comprises the following steps:

b1, acquiring dense point clouds and images:

Preferably, S2, converting the multi-frame image acquired by each camera in the current scanning process into the first frame image of the camera under the camera coordinate system where the first frame image of the camera is located includes:

In the current scanning process, according to the rotation angle of the motion model of the cradle head between each image and the first frame image, a rotation matrix between a camera coordinate system where each frame image is positioned and a camera coordinate system where the first frame image is positioned is obtained;

The beneficial effects brought by the invention are as follows:

Therefore, the invention provides a data fusion technology of a single laser radar or multiple laser radars and a plurality of cameras to realize three-dimensional reconstruction, has wide scanning range and greatly improves the reconstruction efficiency. The radar is used for improving the accuracy of depth, the camera is used for providing richer object surface information, and the multi-camera is used for color-imparting, so that the surface information is richer, and the scene information is increased. Besides, the invention also provides a novel coarse-to-fine calibration method, which realizes the calibration between the radar and each camera, further improves the accuracy of reconstruction and improves the reconstruction precision.

The laser radar and each camera perform a calibration process, and specifically comprises coarse positioning and fine positioning.

Coarse positioning is performed by a rectangular checkerboard or other object that provides sufficient constraint information. The object can be seen by the view angles of the radar and the camera, the characteristics of the object are extracted from the acquired image and the point cloud respectively, and the external parameters of the radar and the camera are obtained by aligning the extracted same characteristics (usually line characteristics and surface characteristics) in the two data. The obtained external parameter matrix depends on the characteristics extracted by the algorithm, is unstable and has low precision.

And in the fine positioning process, the radar is used for mapping surrounding scenes by a synchronous positioning mapping method to obtain dense scene point clouds, and a camera is used for collecting a scene image oriented in the radar mapping process. And extracting features, aligning features of the two point clouds and the images, and optimally solving external parameters except radar and cameras by using a Gaussian Newton method. The method has the advantages of rich extracted characteristics and high calibration precision, but because Gauss Newton easily falls into a local extremum, the method can complete the solution by needing a better initial external parameter.

The laser radar and each camera perform a calibration process, so that the laser radar has higher-precision calibration.

The invention provides a whole set of three-dimensional reconstruction method from calibration to data fusion, which has higher calibration precision and reconstruction efficiency compared with the previous method. The method is mainly used for reconstructing indoor and outdoor three-dimensional scenes.

Drawings

FIG. 1 is a flow chart of a three-dimensional scene reconstruction process according to an embodiment;

FIG. 2 is a schematic diagram of the calibration of the laser radar with each camera;

FIG. 3 is a flow chart of a three-dimensional scene reconstruction process according to a second embodiment;

FIG. 4 is a hardware device of the lidar and camera on which the present invention relies;

fig. 5 is a top view of the laser radar and camera portion of fig. 4.

In fig. 4 and 5, reference numeral 1 denotes a lidar, and reference numeral 2 denotes a camera.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

It should be noted that, without conflict, the embodiments of the present invention and features of the embodiments may be combined with each other.

The invention relates to a set of three-dimensional reconstruction method based on laser radar and multi-camera calibration and point cloud and image data fusion, in particular to a method which can take laser radar three-dimensional point cloud data and continuous RGB image data of a monocular camera as scene information input sources and fuse the two types of data to obtain scene subgraph. And constructing a plurality of scene subgraphs, and fusing all subgraphs based on a generalized nearest neighbor iteration GICP algorithm to obtain a three-dimensional model of the reconstructed scene.

The invention discloses a three-dimensional scene reconstruction method for laser radar and multi-camera data fusion, which comprises two implementation modes, wherein the first implementation mode is realized by utilizing a single laser radar and a plurality of cameras, and the second implementation mode is realized by utilizing two laser radars and a plurality of cameras, and the first implementation mode and the second implementation mode are respectively referred to.

In a first embodiment, referring to fig. 1, a three-dimensional scene reconstruction method for fusing single laser radar and multi-camera data according to the present embodiment is described, where the method includes the following steps:

Fixing a single laser radar on a rotary platform, wherein the rotary platform and a multi-camera are fixed on the same holder, and the acquisition angles of the multi-camera are different; where may depend on the specific hardware implementation;

Scanning and data fusion are carried out on each scanning area in the three-dimensional scene to be detected by using a laser radar and all cameras to obtain subgraphs corresponding to each scanning area, and data fusion is carried out on all subgraphs to finish three-dimensional scene reconstruction; all subgraphs are data fused using GICP algorithm as an example, but not limited to this method.

S1, a rotating platform drives a laser radar to move, a cradle head drives all cameras to move, a current scanning area is scanned, multi-frame cloud data are collected by the laser radar in the current scanning process, and multi-frame images are collected by each camera; as an example, when a plurality of cameras are arranged in the same plane, and the divided scanning areas are scanned, the laser radar can be rotated 360 degrees around the horizontal axis through the rotating platform, and the cradle head controls all cameras to rotate 90 degrees along the vertical axis; after moving to the designated position, the radar camera turns off the acquisition function and returns to the original position.

Further, the step S2 of converting each frame of point cloud data obtained in the current scanning process into the first frame of point cloud data under the laser radar coordinate system where the first frame of point cloud data is located includes:

Further, S2, converting the multi-frame image collected by each camera in the current scanning process into the first frame image of the camera under the camera coordinate system where the first frame image of the camera is located, which comprises the following specific processes:

In the current scanning process, according to the rotation angle of the motion model of the cradle head between each frame of image data and the first frame of image, a rotation matrix between a camera coordinate system where each frame of image is positioned and a camera coordinate system where the first frame of image is positioned is obtained;

Furthermore, the specific process of calibrating the laser radar and each camera to obtain the optimal external parameter matrix between the laser radar and each camera comprises a coarse positioning process and a fine positioning process;

the coarse positioning process is used for acquiring an initial external parameter matrix from the camera to the laser radar; the coarse positioning process may be performed by polygonal objects or other objects capable of providing sufficient constraint information, such as rectangular checkerboards, for example. The object can be seen by the view angles of the radar and the camera, the characteristics of the object are extracted from the acquired image and the point cloud respectively, and the external parameters of the radar and the camera are obtained by aligning the extracted same characteristics (usually line characteristics and surface characteristics) in the two data. The obtained external parameter matrix depends on the characteristics extracted by the algorithm, is unstable and low in precision, and further depends on precision calibration to ensure the precision. Specifically, the coarse positioning process includes the following steps:

And the fine positioning process is used for correcting the initial external parameter matrix to obtain an optimal external parameter matrix. The radar and the camera can build the images of surrounding scenes by a synchronous positioning and image building method to obtain dense scene point clouds, and the camera collects a scene image oriented in the radar image building process. And extracting features, and optimizing and solving parameters except radar and cameras for the features of Ji Dian cloud and images by using a Gauss Newton method. The method has the advantages of rich extracted characteristics and high calibration precision, but because Gauss Newton easily falls into a local extremum, the method can complete the solution by needing a better initial external parameter. The following fine positioning procedure is provided, which specifically comprises:

b1, acquiring dense point clouds and images:

B2, 2D image edge extraction: extracting edge characteristics of the reserved image by using an edge characteristic algorithm to obtain a 2D image edge; specifically, as an example, edge features on the reserved image can be extracted by using an edge feature algorithm provided by opencv to obtain pixel coordinates of all edges;

In the embodiment, in the fine positioning process, edge features in the point cloud and edge features in the image are matched, a residual function is constructed, gaussian Newton iteration solution is used, and at the moment, an initial external parameter matrix between a camera and a radar obtained in the coarse calibration process is used as an initial value of Gaussian Newton algorithm solution to be optimized, so that the problem of falling into a local extremum is avoided.

When the method is implemented based on one laser radar and 3 cameras, coarse calibration and fine calibration of the cameras and the radar are required to be carried out for 3 times. And after the calibration is finished, starting to carry out three-dimensional reconstruction of the large scene. The calibration process is used as a basis for the subsequent three-dimensional reconstruction.

In a second embodiment, referring to fig. 3, a three-dimensional scene reconstruction method for fusion of multiple lidars and multi-camera data according to the present embodiment is described, where the method includes the following steps:

Two laser radars are fixed on a rotary platform, the rotary platform and a multi-camera are fixed on the same cloud platform, the acquisition angles of the multi-camera are different, and the directions of the two laser radars are different; the method can be realized by adopting two laser radars and two cameras, and is particularly shown in fig. 4 and 5;

S1, a rotating platform drives laser radars to move, a cradle head drives all cameras to move, a current scanning area is scanned, multiple frames of cloud data are collected by each laser radar in the current scanning process, and multiple frames of images are collected by each camera; as an example, when a plurality of cameras are arranged in the same plane, and the divided scanning areas are scanned, the two laser radars can be rotated 360 degrees around the horizontal axis through the rotating platform, and the cloud deck controls all cameras to rotate 90 degrees along the vertical axis; after moving to the designated position, the radar camera turns off the acquisition function and returns to the original position.

S2, converting each frame of point cloud data obtained by each laser radar in the current scanning process into first frame of point cloud data of the laser radar under a laser radar coordinate system where the first frame of point cloud data of the laser radar is located, eliminating repeated points in the first frame of point cloud data, carrying out data fusion on the two first frames of point cloud data with the repeated points eliminated, eliminating the repeated points, and obtaining corrected first frame of point cloud data; when the maps are combined, namely: the two first frame point cloud data are subjected to data fusion, and the point clouds are directly added for non-overlapping point clouds, but if overlapping point clouds appear, the direct addition does not provide too much useful information at the moment, but greatly increases the point cloud storage and calculation consumption, so that the related weight reduction operation is performed for the overlapping points;

Furthermore, the specific process of calibrating the two laser radars to obtain the external parameter matrix between the two laser radars comprises the following steps:

Further, referring to fig. 2, the specific process of calibrating the laser radar and each camera to obtain the optimal external parameter matrix between the laser radar and each camera includes a coarse positioning process and a fine positioning process;

The coarse positioning process is used for acquiring an initial external parameter matrix from the camera to the laser radar; the coarse positioning process may be performed by polygonal objects or other objects capable of providing sufficient constraint information, such as rectangular checkerboards, for example. The object can be seen by the view angles of the radar and the camera, the characteristics of the object are extracted from the acquired image and the point cloud respectively, and the external parameters of the radar and the camera are obtained by aligning the extracted same characteristics (usually line characteristics and surface characteristics) in the two data. The obtained external parameter matrix depends on the characteristics extracted by the algorithm, is unstable and low in precision, and further depends on precision calibration to ensure the precision. Specifically, referring to fig. 2, the coarse positioning process includes the steps of:

And the fine positioning process is used for correcting the initial external parameter matrix to obtain an optimal external parameter matrix. The radar can build the map of surrounding scenes by a synchronous positioning map building method to obtain dense scene point clouds, and the camera collects a scene image oriented in the radar map building process. And extracting features, and optimizing and solving parameters except radar and cameras for the features of Ji Dian cloud and images by using a Gauss Newton method. The method has the advantages of rich extracted characteristics and high calibration precision, but because Gauss Newton easily falls into a local extremum, the method can complete the solution by needing a better initial external parameter. The following fine positioning procedure is provided, see fig. 2, specifically comprising:

b1, acquiring dense point clouds and images:

When the method is implemented based on 2 laser radars and 3 cameras, coarse calibration and fine calibration of the cameras and the radars and calibration between the 2 laser radars are required to be carried out for 3 times. And after the calibration is finished, starting to carry out three-dimensional reconstruction of the large scene. The calibration process is used as a basis for the subsequent three-dimensional reconstruction.

In the current scanning process, according to the rotation angle of the motion model of the cradle head between each frame of camera data and the first frame of camera data, a rotation matrix between a camera coordinate system where each frame of image is positioned and a camera coordinate system where the first frame of image is positioned is obtained;

The three-dimensional scene reconstruction method based on multi-laser radar and multi-camera data fusion in the second embodiment is not limited to 2-laser radar, and is also applicable to multi-radar modes, and the subsequent three-dimensional scene reconstruction process only needs to be operated in a unified laser radar coordinate system.

Although the invention herein has been described with reference to particular embodiments, it is to be understood that these embodiments are merely illustrative of the principles and applications of the present invention. It is therefore to be understood that numerous modifications may be made to the illustrative embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that the different dependent claims and the features described herein may be combined in ways other than as described in the original claims. It is also to be understood that features described in connection with separate embodiments may be used in other described embodiments.

Claims

1. The three-dimensional scene reconstruction method for the fusion of the single laser radar and the multi-camera data is characterized by comprising the following steps of:

2. The three-dimensional scene reconstruction method based on single-laser radar and multi-camera data fusion according to claim 1, wherein the specific process of converting each frame of point cloud data obtained in the current scanning process into the first frame of point cloud data under the laser radar coordinate system where the first frame of point cloud data is located comprises the following steps:

3. The three-dimensional scene reconstruction method for single-laser radar and multi-camera data fusion according to claim 1, wherein the data fusion of all sub-images is realized by adopting GICP algorithm.

4. The three-dimensional scene reconstruction method for single-laser radar and multi-camera data fusion according to claim 1, wherein the specific process of calibrating the laser radar and each camera to obtain the optimal external parameter matrix between the laser radar and each camera comprises a coarse positioning process and a fine positioning process;

5. The three-dimensional scene reconstruction method for single lidar and multi-camera data fusion of claim 4, wherein the coarse positioning process comprises:

The fine positioning process comprises the following steps:

b1, acquiring dense point clouds and images:

6. The three-dimensional scene reconstruction method based on multi-laser radar and multi-camera data fusion is characterized by comprising the following steps of:

7. The three-dimensional scene reconstruction method for fusion of multiple laser radars and multi-camera data according to claim 6, wherein the specific process of calibrating the two laser radars to obtain the external reference matrix between the two laser radars comprises the following steps:

8. The three-dimensional scene reconstruction method for multi-laser radar and multi-camera data fusion according to claim 6, wherein the specific process of calibrating the laser radar and each camera to obtain the optimal external parameter matrix between the laser radar and each camera comprises a coarse positioning process and a fine positioning process;

9. The method for reconstructing a three-dimensional scene by fusion of multi-lidar and multi-camera data according to claim 8, wherein the coarse positioning process comprises:

The fine positioning process comprises the following steps:

b1, acquiring dense point clouds and images:

10. The three-dimensional scene reconstruction method based on multi-laser radar and multi-camera data fusion according to claim 6, wherein the specific process of converting each frame of point cloud data obtained in the current scanning process into the first frame of point cloud data under the laser radar coordinate system where the first frame of point cloud data is located comprises the following steps: