CN110189379B - Method and system for calibrating external parameters of camera - Google Patents
Method and system for calibrating external parameters of camera Download PDFInfo
- Publication number
- CN110189379B CN110189379B CN201910455877.2A CN201910455877A CN110189379B CN 110189379 B CN110189379 B CN 110189379B CN 201910455877 A CN201910455877 A CN 201910455877A CN 110189379 B CN110189379 B CN 110189379B
- Authority
- CN
- China
- Prior art keywords
- vehicle
- camera
- moving direction
- parallel directions
- coordinate system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
- G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
- G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
- G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S19/00—Satellite radio beacon positioning systems; Determining position, velocity or attitude using signals transmitted by such systems
- G01S19/38—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system
- G01S19/39—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system the satellite radio beacon positioning system transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO
- G01S19/42—Determining position
- G01S19/48—Determining position by combining or switching between position solutions derived from the satellite radio beacon positioning system and position solutions derived from a further system
- G01S19/49—Determining position by combining or switching between position solutions derived from the satellite radio beacon positioning system and position solutions derived from a further system whereby the further system is an inertial position system, e.g. loosely-coupled
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Automation & Control Theory (AREA)
- Studio Devices (AREA)
- Image Analysis (AREA)
Abstract
A calibration method and system for external parameters of a camera are provided, the method comprises: when a vehicle moves in any two non-parallel directions on a plane, acquiring the moving direction of the vehicle, and controlling a camera arranged on the vehicle to shoot a plurality of images; determining the direction of the plane according to the moving direction of the vehicle; determining the directions of all coordinate axes of a vehicle coordinate system of the vehicle according to the directions of the planes; determining camera positions respectively corresponding to the plurality of images shot by the camera, and fitting the direction of each coordinate axis of a camera coordinate system by using the plurality of camera positions; determining an angular offset of each coordinate axis of the camera coordinate system with respect to a respective coordinate axis of the vehicle coordinate system. By implementing the embodiment of the invention, a specific marker is not used, so that the calibration efficiency of the external parameters of the camera can be improved.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to a method and a system for calibrating external parameters of a camera.
Background
The external parameter calibration of the camera (external parameter of the camera for short) refers to determining the conversion relation between the coordinate system of the camera and other coordinate systems (such as a vehicle coordinate system or a world coordinate system).
Taking the conversion relationship between the calibration camera coordinate system and the vehicle coordinate system as an example, the traditional camera external reference calibration method comprises the following steps: firstly, arranging a fixed marker (such as a calibration plate or calibration cloth printed with a chessboard) on a calibration site, parking a vehicle at a position where a camera arranged on the vehicle can shoot the marker, and then shooting the marker by using the camera; identifying the image position of the marker in the image shot by the camera, determining the relative relationship between the marker and the camera, and obtaining the representation of the marker in a camera coordinate system; and then measuring the relative relation between the marker and the vehicle to obtain the representation of the marker in a vehicle coordinate system. Based on the representation of the same marker in both coordinate systems, a translation between the camera coordinate system and the vehicle coordinate system can be obtained.
However, this calibration method needs to be performed at a specific site, and needs to erect a specific marker, which is cumbersome to operate and inefficient.
Disclosure of Invention
The embodiment of the invention discloses a method and a system for calibrating external parameters of a camera, which can improve the calibration efficiency of the external parameters of the camera.
The first aspect of the embodiments of the present invention discloses a method for calibrating external parameters of a camera, where the method includes:
when a vehicle moves in any two non-parallel directions on a plane, acquiring the moving direction of the vehicle, and controlling a camera arranged on the vehicle to shoot a plurality of images;
determining the direction of the plane according to the moving direction of the vehicle;
determining the directions of all coordinate axes of a vehicle coordinate system of the vehicle according to the directions of the planes;
determining camera positions respectively corresponding to the plurality of images shot by the camera, and fitting the direction of each coordinate axis of a camera coordinate system by using the plurality of camera positions;
determining an angular offset of each coordinate axis of the camera coordinate system with respect to a respective coordinate axis of the vehicle coordinate system.
As an optional implementation manner, in the first aspect of the embodiment of the present invention, the determining the camera positions corresponding to the plurality of images captured by the camera respectively, and fitting the direction of each coordinate axis of a camera coordinate system using the plurality of camera positions includes:
determining a relative position of the camera when capturing a subsequent frame image with respect to when capturing a first frame image; the subsequent frame image is an image which is shot after the first frame image in the plurality of images;
fitting the direction of each coordinate axis of the camera coordinate system with a plurality of said relative positions.
As an alternative implementation manner, in the first aspect of the embodiment of the present invention, an included angle between any two non-parallel directions ranges from 60 ° to 120 °.
As an optional implementation manner, in the first aspect of the embodiment of the present invention, the acquiring the moving direction of the vehicle when the vehicle moves in any two non-parallel directions on a plane includes:
measuring a plurality of vehicle positions of a vehicle in a moving process when the vehicle moves in any two non-parallel directions on a certain plane;
and fitting the plurality of vehicle positions to obtain the corresponding vehicle moving direction when the vehicle moves according to the any two non-parallel directions.
As an optional implementation manner, in the first aspect of the embodiment of the present invention, the acquiring the moving direction of the vehicle when the vehicle moves in any two non-parallel directions on a plane includes:
when the vehicle moves in any two non-parallel directions on a certain plane, the corresponding vehicle moving direction when the vehicle moves in any two non-parallel directions on a certain plane is detected through an inertia measuring unit of the vehicle.
As an optional implementation manner, in the first aspect of the embodiment of the present invention, the acquiring the moving direction of the vehicle when the vehicle moves in any two non-parallel directions on a plane includes:
when a vehicle moves in any two non-parallel directions on a plane, detecting whether the satellite positioning signal intensity of the vehicle exceeds a preset signal intensity threshold value;
if the vehicle moving direction exceeds the preset moving direction, measuring a plurality of vehicle positions of the vehicle in the moving process, and fitting the plurality of vehicle positions to obtain the corresponding vehicle moving directions when the vehicle moves according to the any two non-parallel directions;
and if the vehicle moving direction does not exceed the preset value, detecting the corresponding vehicle moving direction when the vehicle moves in two non-parallel directions on a certain plane through an inertia measuring unit of the vehicle.
A second aspect of the embodiments of the present invention discloses a system for calibrating external parameters of a camera, including:
the device comprises an acquisition unit, a control unit and a display unit, wherein the acquisition unit is used for acquiring the moving direction of a vehicle when the vehicle moves along any two non-parallel directions on a certain plane;
the control unit is used for controlling a camera arranged on the vehicle to shoot a plurality of images when the vehicle moves in any two non-parallel directions on a certain plane;
the fitting unit is used for determining the direction of the plane according to the moving direction of the vehicle;
a first determination unit configured to determine directions of respective coordinate axes of a vehicle coordinate system of the vehicle according to the direction of the plane;
the second determining unit is used for determining camera positions respectively corresponding to the plurality of images shot by the camera and fitting the direction of each coordinate axis of a camera coordinate system by using the plurality of camera positions;
a third determination unit for determining an angular offset of each coordinate axis of the camera coordinate system with respect to a respective coordinate axis of the vehicle coordinate system.
As an optional implementation manner, in a second aspect of the embodiment of the present invention, the second determining unit includes:
a first determining subunit operable to determine a relative position of the camera when a subsequent frame image is captured with respect to when a first frame image is captured; the subsequent frame image is an image which is shot after the first frame image in the plurality of images;
and the fitting subunit is used for fitting the direction of each coordinate axis of the camera coordinate system by using the relative positions.
As an alternative implementation, in the second aspect of the embodiment of the present invention, an included angle between any two non-parallel directions ranges from 60 ° to 120 °.
As an alternative implementation, in the second aspect of the embodiment of the present invention:
the acquiring unit is specifically used for measuring a plurality of vehicle positions of the vehicle in a moving process when the vehicle moves in any two non-parallel directions on a certain plane; and fitting the plurality of vehicle positions to obtain the corresponding vehicle moving direction when the vehicle moves according to the any two non-parallel directions.
As an alternative implementation, in the second aspect of the embodiment of the present invention:
the obtaining unit is specifically configured to detect, by an inertial measurement unit of the vehicle, a vehicle moving direction corresponding to a case where the vehicle moves in any two non-parallel directions on a certain plane when the vehicle moves in any two non-parallel directions on the certain plane.
As an alternative implementation, in the second aspect of the embodiment of the present invention:
the acquisition unit is specifically used for detecting whether the satellite positioning signal intensity of the vehicle exceeds a preset signal intensity threshold value or not when the vehicle moves in any two non-parallel directions on a certain plane; if the vehicle moving direction exceeds the preset moving direction, measuring a plurality of vehicle positions of the vehicle in the moving process, and fitting the plurality of vehicle positions to obtain the corresponding vehicle moving directions when the vehicle moves according to the any two non-parallel directions; and if the vehicle moving direction does not exceed the preset value, detecting the corresponding vehicle moving direction when the vehicle moves in any two non-parallel directions on a certain plane through an inertia measuring unit of the vehicle.
A third aspect of the embodiments of the present invention discloses a mobile terminal, including:
a memory storing executable program code;
a processor coupled with the memory;
the processor calls the executable program code stored in the memory to execute any one of the methods disclosed in the first aspect of the embodiments of the present invention.
A fourth aspect of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program causes a computer to execute any one of the methods disclosed in the first aspect of the embodiments of the present invention.
A fifth aspect of the embodiments of the present invention discloses a computer program product, which, when running on a computer, causes the computer to execute any one of the methods disclosed in the first aspect of the embodiments of the present invention.
A sixth aspect of embodiments of the present invention discloses a vehicle including any one of the systems disclosed in the second aspect of embodiments of the present invention.
Compared with the prior art, the embodiment of the invention has the following beneficial effects:
when the vehicle moves in any two non-parallel directions on a plane, acquiring the moving direction of the vehicle, and controlling a camera arranged on the vehicle to shoot a plurality of images; determining the direction of a plane where the vehicle is located according to the moving direction of the vehicle, so as to obtain the directions of all coordinate axes of a vehicle coordinate system; according to the images shot by the camera, the corresponding camera positions when the camera shoots a plurality of images can be determined; the directions of all coordinate axes of the camera coordinate system can be fitted by utilizing a plurality of camera positions so as to obtain the angle deviation of all coordinate axes of the camera coordinate system relative to all coordinate axes of the vehicle coordinate system, and the external parameters of the camera are calibrated. Therefore, in the embodiment of the invention, a specific marker is not needed, the vehicle is not needed to be parked in a fixed place, and external reference calibration can be completed in the driving process of the vehicle, so that the calibration efficiency of the external parameters of the camera is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a schematic flowchart of a calibration method for external parameters of a camera according to an embodiment of the present invention;
FIG. 2 is an exemplary diagram of a vehicle coordinate system disclosed in an embodiment of the present invention;
FIG. 3 is an exemplary diagram of a vehicle coordinate system and camera coordinate system as disclosed in an embodiment of the present invention;
FIG. 4 is a schematic flowchart of another method for calibrating external parameters of a camera according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a system for calibrating external parameters of a camera disclosed in the embodiment of the present invention;
fig. 6 is a schematic structural diagram of a system for calibrating external parameters of a camera disclosed in the embodiment of the present invention;
fig. 7 is a schematic structural diagram of another system for calibrating extrinsic parameters of a camera according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It is to be noted that the terms "comprises" and "comprising" and any variations thereof in the embodiments and drawings of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.
The embodiment of the invention discloses a method and a system for calibrating external parameters of a camera, which can improve the calibration efficiency of the external parameters of the camera. The following are detailed below.
Example one
Referring to fig. 1, fig. 1 is a schematic flow chart illustrating a method for calibrating external parameters of a camera according to an embodiment of the present invention. The calibration system applicable to the calibration method for the external parameters of the camera described in fig. 1 may be operated in a vehicle-mounted electronic device such as a vehicle central control device, a vehicle-mounted computer, a vehicle-mounted industrial control computer, or in a computer, a cloud server, etc. separated from the vehicle, which is not limited in the embodiment of the present invention. As shown in fig. 1, the calibration method of the camera external parameters may include the following steps:
101. when the vehicle moves in any two non-parallel directions on a plane, the calibration system acquires the moving direction of the vehicle and controls a camera arranged on the vehicle to shoot a plurality of images.
In the embodiment of the invention, the vehicle moves on a plane according to any two non-parallel directions, namely the vehicle needs to turn at least once in the moving process, and the angle of each turn can be unlimited. Further, when the angle of the turn is too small (e.g., 10 °) or the angle of the turn is too large (e.g., 170 °), the effect of the plane fitting is poor. Thus, the included angle between any two non-parallel directions may be further defined to be in the range of 60 ° to 120 °. Preferably, the included angle between every two directions can be limited to be 90 degrees, namely the turning angle is 90 degrees, and then a better plane fitting effect can be achieved. Still further optionally, the moving distance of the vehicle on the plane may be limited to be not less than a certain distance threshold, so as to increase the data volume collected by the vehicle, thereby improving the accuracy of the camera external parameter calibration.
When the vehicle moves in any two non-parallel directions on a plane, the user can control the vehicle to run according to a preset straight line. At this time, the specific manner of acquiring the moving direction of the vehicle by the calibration system may be as follows:
the calibration system acquires a vehicle moving direction input by a user, wherein the vehicle moving direction is a linear direction when the vehicle runs according to a preset straight line. By implementing the embodiment, the calibration system can detect the moving direction of the vehicle without using a sensor such as a satellite positioning system and an inertial measurement unit.
As another alternative, the calibration system may also measure a plurality of vehicle positions of the vehicle during movement; and obtaining the corresponding vehicle moving direction when the vehicle moves according to the any two non-parallel directions by fitting the plurality of vehicle positions.
As a further alternative, the calibration system may further detect, through an inertial measurement unit of the vehicle, a vehicle moving direction corresponding to the vehicle moving in any two non-parallel directions on a certain plane.
102. The calibration system determines the direction of the plane according to the moving direction of the vehicle.
In the embodiment of the invention, when two or more vehicle moving directions are acquired, the plane where the vehicle is located can be fitted, so that the direction of the plane is determined. The larger the number of measured moving directions of the vehicle, the closer the fitted plane is to the plane where the vehicle actually lies. The direction of the plane may be represented by a normal perpendicular to the plane.
103. And the calibration system determines the directions of all coordinate axes of the vehicle coordinate system of the vehicle according to the directions of the planes.
In the embodiment of the present invention, please refer to fig. 2, and fig. 2 is an exemplary diagram of a vehicle coordinate system according to the embodiment of the present invention. Generally, a certain portion of the vehicle may be selected as the origin of the vehicle coordinate system, and may be set as the rear axle center point of the vehicle. After the origin of the vehicle coordinate system is set, the vehicle advancing direction is the x-axis of the vehicle coordinate system, the left side of the advancing direction is the y-axis of the vehicle coordinate system, and the direction perpendicular to the plane of the vehicle is the z-axis of the vehicle coordinate system.
Therefore, when step 102 is executed to determine the direction of the plane of the vehicle, the normal direction of the plane may be taken as the direction of the z-axis of the vehicle coordinate system, that is, the directions of the y-axis and the x-axis of the vehicle coordinate system may be further determined according to the direction of the z-axis of the vehicle coordinate system.
For example, assuming that the plane in which the vehicle is currently located is parallel to the horizontal plane, i.e., there is no slope in the plane in which the vehicle is currently located, the vehicle coordinate system may be represented as [ 100; 010; 001].
104. The calibration system determines camera positions corresponding to the multiple images shot by the camera respectively, and fits the directions of all coordinate axes of the camera coordinate system by using the multiple camera positions.
In the embodiment of the invention, the camera poses respectively corresponding to a plurality of images shot by the camera can be calculated through a Simultaneous Localization and Mapping (SLAM) algorithm.
Specifically, the manner of executing step 104 may specifically be:
the calibration system determines the relative position of the camera when shooting the subsequent frame image relative to the first frame image; wherein the subsequent frame image is an image captured after the first frame image among the plurality of images;
the calibration system fits the orientation of each coordinate axis of the camera coordinate system with a plurality of relative positions.
For example, taking the first three frames of images captured by the camera as an example, the camera position where the first frame of image is captured is defaulted to (0, 0, 0) with reference to the captured first frame of image. Based on the SLAM algorithm, the relative position of the camera when the second frame image is captured with respect to the first frame image can be calculated, and for example, the camera position when the second frame image is captured can be represented as (0, 1, 2), and the camera position when the third frame image is captured can be represented as (0, 2, 4). One unit direction (0, 0.447, 0.894) can be fitted using the three camera positions described above.
Referring to fig. 3, fig. 3 is an exemplary diagram of a vehicle coordinate system and a camera coordinate system according to an embodiment of the disclosure. Generally, the camera coordinate system uses the camera optical center as the origin of the coordinate system, the plane formed by the x-axis and the y-axis of the camera coordinate system is the imaging plane of the camera, and the z-axis of the camera coordinate system is perpendicular to the imaging plane. It can be seen that the installation position of the camera on the vehicle can affect the optical center of the camera and the orientation of the imaging plane, so that the relative relationship between the camera coordinate system and the vehicle coordinate system can be affected.
105. The calibration system determines the angular offset of each coordinate axis of the camera coordinate system with respect to the respective coordinate axis of the vehicle coordinate system.
In the embodiment of the present invention, it is assumed that the camera coordinate system is represented by O-xyz, and the vehicle coordinate system is represented by O ' -x ' y ' z ', then step 105 may specifically represent that the angle offset of the x-axis of the camera coordinate system with respect to each coordinate axis of the vehicle coordinate system is [ x 'x，y’x，z’x]The angle offset of the y-axis of the camera coordinate system to each coordinate axis of the vehicle coordinate system is [ x'y，y’y，z’y]The angular offset of the z-axis of the camera coordinate system with respect to each coordinate axis of the vehicle coordinate system is [ x'z，y’z，z’z]. That is, the conversion relationship between the camera coordinate system and the vehicle coordinate system can be expressed as
Using the vehicle coordinate system [ 100; 010; 001]Assuming that the unit direction fitted with the above-described three camera positions is (0, 0.447, 0.894), the angle of the x-axis of the camera coordinate system with respect to each coordinate axis of the vehicle coordinate system is shifted by [ x'x，y’x，z’x]＝[0，0.447，0.894]。
It is understood that the extrinsic parameters of the camera may include a position offset and an angle offset, and in the method described in fig. 1, the angle offset in the extrinsic parameters of the camera, i.e., the transformation relationship between the camera coordinate system and the vehicle coordinate system in the direction (orientation), may be calibrated. After the conversion relation between the camera coordinate system and the vehicle coordinate system is determined, data collected by the camera can be unified to the vehicle coordinate system for representing, or data represented by the vehicle coordinate system can be unified to the camera coordinate system for representing, so that data collected by different sensors of the vehicle can be conveniently fused.
It can be seen that in the method described in fig. 1, the conversion relationship between the camera coordinate system and the vehicle coordinate system in the direction can be calibrated only by the vehicle traveling a distance in different moving directions, and no specific marker needs to be set or limited in a fixed calibration site, so that the camera calibration efficiency can be improved. In addition, the method described in fig. 1 reduces the technical threshold for calibrating the camera external parameter, and the user of the vehicle can calibrate the camera external parameter by himself without the assistance of a special technician.
Example two
Referring to fig. 4, fig. 4 is a schematic flow chart illustrating another method for calibrating external parameters of a camera according to an embodiment of the present invention. As shown in fig. 4, the calibration method of the camera external parameters may include:
401. when the vehicle moves on a plane according to any two non-parallel directions, the calibration system detects whether the satellite positioning signal intensity of the vehicle exceeds a preset signal intensity threshold value; if yes, executing step 402-step 403; if not, step 404-step 405 are executed.
In the embodiment of the invention, due to the problem of the installation stability of the camera, when the vehicle runs on a bumpy road, the optical center of the camera and the orientation of the imaging plane can be changed. Since the calibration method shown in fig. 4 may not require a specific marker nor assistance from a professional, the vehicle may immediately trigger the calibration system to perform the method shown in fig. 4 to recalibrate the camera external reference. However, at this time, the vehicle may need to travel in a tunnel, an underground garage, or other places where satellite positioning signals are poor for a long time, and the accuracy of the measured vehicle position is low, so that a certain deviation exists between a plane fitted according to the vehicle position and a plane where the vehicle is actually located.
Therefore, step 401 is executed first, and when the satellite positioning signal strength is determined to be strong (exceeding the preset signal strength threshold), step 402 to step 403 are executed again, so as to fit the plane where the vehicle is located according to the measured vehicle position.
In addition, as an optional real-time manner, before performing step 401, the calibration system may further perform the following steps:
the calibration system detects whether the installation position of the camera changes; if so, step 401 is performed.
Specifically, the position of the vehicle body in the image currently captured by the camera and the position of the vehicle body in the reference image may be compared at regular intervals; the reference image can be an image shot by a camera after the camera is installed on the vehicle according to a preset standard; when the positions of the vehicle bodies in the two images do not coincide with each other, it is considered that the installation position of the camera has changed.
402. When the vehicle moves in any two non-parallel directions on a plane, the calibration system measures a plurality of vehicle positions of the vehicle in the moving process and controls a camera arranged on the vehicle to shoot a plurality of images.
403. The calibration system fits the plurality of vehicle positions to obtain the corresponding vehicle movement direction when the vehicle moves in any two non-parallel directions, and continues to perform step 405.
404. The calibration system detects the moving direction of the vehicle when the vehicle moves in any two non-parallel directions on a plane through the inertial measurement unit of the vehicle, and simultaneously controls the camera mounted on the vehicle to take a plurality of images, and continues to execute step 405.
In the embodiment of the invention, the inertial measurement unit IMU of the vehicle can measure the direction of the acceleration when the vehicle moves, and the moving speed of the vehicle can be obtained by integrating the measured acceleration with respect to time, so that the moving direction of the vehicle can be obtained according to the direction of the moving speed. The IMU is not affected by external factors and thus may be used when satellite positioning signals are poor.
Preferably, the included angle between every two directions of any two non-parallel directions can be limited to 90 degrees.
405. The direction of movement of the vehicle of the calibration system determines the direction of the plane.
In the embodiment of the invention, when two or more moving directions of the vehicle are measured, the plane where the vehicle is located can be determined according to the moving directions of the vehicle, so that the direction of the plane is determined.
406. And the calibration system determines the directions of all coordinate axes of the vehicle coordinate system of the vehicle according to the directions of the planes.
407. The calibration system determines camera positions corresponding to the multiple images shot by the camera respectively, and fits the directions of all coordinate axes of the camera coordinate system by using the multiple camera positions.
In the embodiment of the present invention, the manner of executing step 407 may specifically be:
the calibration system determines the relative position of the camera when shooting the subsequent frame image relative to the first frame image; wherein the subsequent frame image is an image captured after the first frame image among the plurality of images;
the calibration system fits the orientation of each coordinate axis of the camera coordinate system with a plurality of relative positions.
That is, the SLAM algorithm can be used to calculate the camera positions corresponding to the plurality of images captured by the camera.
408. The calibration system determines the angular offset of each coordinate axis of the camera coordinate system with respect to the respective coordinate axis of the vehicle coordinate system.
It can be seen that in the method described in fig. 4, the calibration system can automatically calibrate the external parameters of the camera without specific markers, thereby improving the efficiency of external parameter calibration of the camera. Furthermore, the adaptability can be adjusted according to different driving places with stronger or weaker satellite positioning signals, so that the accuracy of external reference calibration is improved.
EXAMPLE III
Referring to fig. 5, fig. 5 is a schematic structural diagram of a system for calibrating external parameters of a camera according to an embodiment of the present invention. As shown in fig. 5, the system for calibrating the external parameters of the camera may include:
an acquiring unit 501, configured to acquire a moving direction of a vehicle when the vehicle moves in any two non-parallel directions on a certain plane;
a control unit 502 for controlling a camera mounted on a vehicle to take a plurality of images when the vehicle moves in any two non-parallel directions on a plane;
in embodiments of the present invention, the included angle between any two non-parallel directions may be defined to be in the range of 60 ° to 120 °. Preferably, an included angle between every two directions of any two non-parallel directions can be further limited to 90 °, so that a better plane fitting effect can be achieved. Optionally, the moving distance of the vehicle on the plane may be limited to be not less than a certain distance threshold, so as to increase the data amount acquired by the vehicle, thereby improving the accuracy of the camera external parameter calibration.
The obtaining unit 501 may be specifically configured to, when the user controls the vehicle to travel according to a preset straight line, obtain a straight line direction in which the user inputs that the vehicle travels according to the preset straight line as a vehicle moving direction; the preset straight line may include at least two straight lines with non-parallel directions.
A fitting unit 503 for determining the direction of the plane according to the moving direction of the vehicle;
in the embodiment of the present invention, when the obtaining unit 501 measures two or more moving directions of the vehicle, the fitting unit 503 may fit a plane where the vehicle is located;
a first determining unit 504 for determining directions of respective coordinate axes of a vehicle coordinate system of the vehicle according to the directions of the planes;
a second determining unit 505, configured to determine camera positions corresponding to the multiple images captured by the camera, and fit a direction of each coordinate axis of the camera coordinate system with the multiple camera positions;
a third determination unit 506 for determining the angular offset of each coordinate axis of the camera coordinate system with respect to the respective coordinate axis of the vehicle coordinate system.
It can be seen that, by implementing the calibration system for the external parameters of the camera as shown in fig. 5, the conversion relationship between the camera coordinate system and the vehicle coordinate system in the direction can be calibrated only by driving the vehicle in different moving directions for a certain distance, without setting a specific marker or limiting the vehicle coordinate system in a fixed calibration site, thereby improving the efficiency of camera calibration. In addition, the method described in fig. 1 reduces the technical threshold for calibrating the camera external parameter, and the user of the vehicle can calibrate the camera external parameter by himself without the assistance of a special technician.
Example four
Referring to fig. 6, fig. 6 is a schematic structural diagram of a system for calibrating external parameters of a camera according to an embodiment of the present invention. The system for calibrating the camera extrinsic parameters shown in fig. 6 is optimized by the system for calibrating the camera extrinsic parameters shown in fig. 5. As shown in fig. 6, in the calibration system for camera extrinsic parameters:
the second determining unit 505 may include:
a first determination sub-unit 5051 that determines the relative position of the camera when a subsequent frame image is captured with respect to when the first frame image is captured; wherein the subsequent frame image is an image captured after the first frame image among the plurality of images;
a fitting subunit 5052 is configured to fit the direction of each coordinate axis of the camera coordinate system with a plurality of relative positions.
Optionally, in the calibration system for the external parameters of the camera shown in fig. 6:
the acquiring unit 501 may be specifically configured to measure a plurality of vehicle positions of the vehicle in a moving process when the vehicle moves in any two non-parallel directions on a certain plane; and fitting the positions of the vehicles to obtain the corresponding moving directions of the vehicles when the vehicles move according to the any two non-parallel directions.
Alternatively, the obtaining unit 501 may specifically detect, by an inertial measurement unit of the vehicle, a vehicle moving direction corresponding to the vehicle moving in any two non-parallel directions on a certain plane when the vehicle moves in any two non-parallel directions on the certain plane.
Further optionally, the obtaining unit 501 may specifically detect whether the satellite positioning signal strength of the vehicle exceeds a preset signal strength threshold when the vehicle moves in any two non-parallel directions on a certain plane; if the vehicle moving direction exceeds the preset moving direction, measuring a plurality of vehicle positions of the vehicle in the moving process, and fitting the plurality of vehicle positions to obtain the corresponding vehicle moving direction when the vehicle moves according to any two non-parallel directions; if the vehicle moving direction does not exceed the preset range, the corresponding vehicle moving direction is detected when the vehicle moves in any two non-parallel directions on a certain plane through an inertia measuring unit of the vehicle.
In addition, as an alternative embodiment, the above-mentioned obtaining unit 501 may also detect whether the installation position of the camera changes before performing the operation of detecting whether the satellite positioning signal strength of the vehicle exceeds a preset signal strength threshold; when the change of the installation position is detected, the operation of detecting whether the satellite positioning signal intensity of the vehicle exceeds a preset signal intensity threshold value is executed.
Specifically, the obtaining unit 501 may compare the position of the vehicle body in the image currently captured by the camera with the position of the vehicle body in the reference image at regular intervals; the reference image can be an image shot by a camera after the camera is installed on the vehicle according to a preset standard; when the positions of the vehicle body in the two images are not consistent, the installation position of the camera is determined to be changed.
It can be seen that the calibration system for camera extrinsic parameters shown in fig. 6 is implemented to automatically calibrate the extrinsic parameters of the camera without specific markers, thereby improving the efficiency of camera extrinsic parameter calibration. Furthermore, the adaptability can be adjusted according to different driving places with stronger or weaker satellite positioning signals, so that the accuracy of external reference calibration is improved.
EXAMPLE five
Referring to fig. 7, fig. 7 is a schematic structural diagram of another system for calibrating external parameters of a camera according to an embodiment of the present invention. As shown in fig. 7, the system for calibrating the external parameters of the camera may include:
a memory 701 in which executable program code is stored;
a processor 702 coupled to the memory 701;
a camera 703;
the processor 702 receives an image captured by the camera 703, and calls an executable program code stored in the memory 701 to execute any one of the calibration methods for external parameters of the camera shown in fig. 1 or fig. 2.
It should be noted that the calibration system for external parameters of a camera shown in fig. 7 may further include components, which are not shown, such as a satellite positioning system, an inertial measurement unit, and the like, which are not described in detail in this embodiment.
The embodiment of the invention discloses a vehicle which comprises a camera external parameter calibration system shown in any one of figures 5-7.
The embodiment of the invention discloses a computer-readable storage medium which stores a computer program, wherein the computer program enables a computer to execute any one of the calibration methods of the external parameters of the camera shown in fig. 1 or fig. 2.
An embodiment of the invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to make a computer execute any one of the calibration methods for camera external parameters shown in fig. 1 or fig. 2.
It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. Those skilled in the art should also appreciate that the embodiments described in this specification are exemplary and alternative embodiments, and that the acts and modules illustrated are not required in order to practice the invention.
In various embodiments of the present invention, it should be understood that the sequence numbers of the above-mentioned processes do not imply an inevitable order of execution, and the execution order of the processes should be determined by their functions and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated units, if implemented as software functional units and sold or used as a stand-alone product, may be stored in a computer accessible memory. Based on such understanding, the technical solution of the present invention, which is a part of or contributes to the prior art in essence, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for causing a computer device (which may be a personal computer, a server, a network device, or the like, and may specifically be a processor in the computer device) to execute part or all of the steps of the above-described method of each embodiment of the present invention.
It will be understood by those skilled in the art that all or part of the steps in the methods of the embodiments described above may be implemented by hardware instructions of a program, and the program may be stored in a computer-readable storage medium, where the storage medium includes Read-Only Memory (ROM), Random Access Memory (RAM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), One-time Programmable Read-Only Memory (OTPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-Only Memory (CD-ROM), or other Memory, such as a magnetic disk, or a combination thereof, A tape memory, or any other medium readable by a computer that can be used to carry or store data.
The method and system for calibrating external parameters of a camera disclosed in the embodiments of the present invention are described in detail above, and specific examples are applied herein to explain the principles and embodiments of the present invention. Meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (12)
1. A calibration method for external parameters of a camera is characterized by comprising the following steps:
when a vehicle moves in any two non-parallel directions on a plane, acquiring the moving direction of the vehicle, and controlling a camera arranged on the vehicle to shoot a plurality of images;
determining the direction of the plane according to the moving direction of the vehicle;
determining the directions of all coordinate axes of a vehicle coordinate system of the vehicle according to the directions of the planes;
determining camera positions respectively corresponding to the plurality of images shot by the camera through a SLAM algorithm, and fitting the direction of each coordinate axis of a camera coordinate system by using the plurality of camera positions;
determining an angular offset of each coordinate axis of the camera coordinate system with respect to a respective coordinate axis of the vehicle coordinate system.
2. The method of claim 1, wherein determining the camera positions corresponding to the plurality of images taken by the camera and fitting the directions of each coordinate axis of the camera coordinate system with the plurality of camera positions comprises:
determining a relative position of the camera when capturing a subsequent frame image with respect to when capturing a first frame image; the subsequent frame image is an image which is shot after the first frame image in the plurality of images;
fitting the direction of each coordinate axis of the camera coordinate system with a plurality of said relative positions.
3. A method according to claim 1 or 2, wherein the angle between any two non-parallel directions is in the range 60 ° to 120 °.
4. The method of claim 1, wherein the obtaining the moving direction of the vehicle when the vehicle moves in any two non-parallel directions on a plane comprises:
measuring a plurality of vehicle positions of a vehicle in a moving process when the vehicle moves in any two non-parallel directions on a certain plane;
and fitting the plurality of vehicle positions to obtain the corresponding vehicle moving direction when the vehicle moves according to the any two non-parallel directions.
5. The method of claim 1, wherein the obtaining the moving direction of the vehicle when the vehicle moves in any two non-parallel directions on a plane comprises:
when the vehicle moves in any two non-parallel directions on a certain plane, the corresponding vehicle moving direction when the vehicle moves in any two non-parallel directions on a certain plane is detected through an inertia measuring unit of the vehicle.
6. The method of claim 1, wherein the obtaining the moving direction of the vehicle when the vehicle moves in any two non-parallel directions on a plane comprises:
when a vehicle moves in any two non-parallel directions on a plane, detecting whether the satellite positioning signal intensity of the vehicle exceeds a preset signal intensity threshold value;
if the vehicle moving direction exceeds the preset moving direction, measuring a plurality of vehicle positions of the vehicle in the moving process, and fitting the plurality of vehicle positions to obtain the corresponding vehicle moving directions when the vehicle moves according to the any two non-parallel directions;
and if the vehicle moving direction does not exceed the preset value, detecting the corresponding vehicle moving direction when the vehicle moves in two non-parallel directions on a certain plane through an inertia measuring unit of the vehicle.
7. A calibration system for external parameters of a camera is characterized by comprising:
the device comprises an acquisition unit, a control unit and a display unit, wherein the acquisition unit is used for acquiring the moving direction of a vehicle when the vehicle moves along any two non-parallel directions on a certain plane;
the control unit is used for controlling a camera arranged on the vehicle to shoot a plurality of images when the vehicle moves in any two non-parallel directions on a certain plane;
the fitting unit is used for determining the direction of the plane according to the moving direction of the vehicle;
a first determination unit configured to determine directions of respective coordinate axes of a vehicle coordinate system of the vehicle according to the direction of the plane;
the second determining unit is used for determining camera positions corresponding to the plurality of images shot by the camera through a SLAM algorithm and fitting the direction of each coordinate axis of a camera coordinate system by using the plurality of camera positions;
a third determination unit for determining an angular offset of each coordinate axis of the camera coordinate system with respect to a respective coordinate axis of the vehicle coordinate system.
8. The system of claim 7, wherein the second determining unit comprises:
a first determining subunit operable to determine a relative position of the camera when a subsequent frame image is captured with respect to when a first frame image is captured; the subsequent frame image is an image which is shot after the first frame image in the plurality of images;
and the fitting subunit is used for fitting the direction of each coordinate axis of the camera coordinate system by using the relative positions.
9. The system of claim 7 or 8, wherein the angle between any two non-parallel directions is in the range of 60 ° to 120 °.
10. The system of claim 7, wherein:
the acquiring unit is specifically used for measuring a plurality of vehicle positions of the vehicle in a moving process when the vehicle moves in any two non-parallel directions on a certain plane; and fitting the plurality of vehicle positions to obtain the corresponding vehicle moving direction when the vehicle moves according to the any two non-parallel directions.
11. The system of claim 7, wherein:
the obtaining unit is specifically configured to detect, by an inertial measurement unit of the vehicle, a vehicle moving direction corresponding to a case where the vehicle moves in any two non-parallel directions on a certain plane when the vehicle moves in any two non-parallel directions on the certain plane.
12. The system of claim 7, wherein:
the acquisition unit is specifically used for detecting whether the satellite positioning signal intensity of the vehicle exceeds a preset signal intensity threshold value or not when the vehicle moves in any two non-parallel directions on a certain plane; if the vehicle moving direction exceeds the preset moving direction, measuring a plurality of vehicle positions of the vehicle in the moving process, and fitting the plurality of vehicle positions to obtain the corresponding vehicle moving directions when the vehicle moves according to the any two non-parallel directions; and if the vehicle moving direction does not exceed the preset value, detecting the corresponding vehicle moving direction when the vehicle moves in any two non-parallel directions on a certain plane through an inertia measuring unit of the vehicle.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910455877.2A CN110189379B (en) | 2019-05-28 | 2019-05-28 | Method and system for calibrating external parameters of camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910455877.2A CN110189379B (en) | 2019-05-28 | 2019-05-28 | Method and system for calibrating external parameters of camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110189379A CN110189379A (en) | 2019-08-30 |
| CN110189379B true CN110189379B (en) | 2021-09-03 |
Family
ID=67718409
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910455877.2A Active CN110189379B (en) | 2019-05-28 | 2019-05-28 | Method and system for calibrating external parameters of camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110189379B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112950726B (en) * | 2021-03-25 | 2022-11-11 | 深圳市商汤科技有限公司 | Camera orientation calibration method and related product |
| CN113223076B (en) * | 2021-04-07 | 2024-02-27 | 东软睿驰汽车技术（沈阳）有限公司 | Coordinate system calibration method, device and storage medium for vehicle and vehicle-mounted camera |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103529838A (en) * | 2013-09-23 | 2014-01-22 | 南京航空航天大学 | Multi-objective visual guide driving device of automated guided vehicle and collaborative calibration method of multi-objective visual guide driving device |
| CN105205459A (en) * | 2015-09-16 | 2015-12-30 | 东软集团股份有限公司 | Method and device for identifying type of image feature point |
| CN105588563A (en) * | 2016-01-15 | 2016-05-18 | 武汉光庭科技有限公司 | Joint calibration method of binocular camera and inertial navigation unit in automatic driving |
| CN106558080A (en) * | 2016-11-14 | 2017-04-05 | 天津津航技术物理研究所 | Join on-line proving system and method outside a kind of monocular camera |
| CN107388967A (en) * | 2017-08-14 | 2017-11-24 | 上海汽车集团股份有限公司 | A kind of outer parameter compensation method of vehicle-mounted three-dimensional laser sensor and device |
| CN109767475A (en) * | 2018-12-28 | 2019-05-17 | 广州小鹏汽车科技有限公司 | A kind of method for calibrating external parameters and system of sensor |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107895375B (en) * | 2017-11-23 | 2020-03-31 | 南京莱斯电子设备有限公司 | Complex road route extraction method based on visual multi-features |
| CN108288291A (en) * | 2018-06-07 | 2018-07-17 | 北京轻威科技有限责任公司 | Polyphaser calibration based on single-point calibration object |
-
2019
- 2019-05-28 CN CN201910455877.2A patent/CN110189379B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103529838A (en) * | 2013-09-23 | 2014-01-22 | 南京航空航天大学 | Multi-objective visual guide driving device of automated guided vehicle and collaborative calibration method of multi-objective visual guide driving device |
| CN105205459A (en) * | 2015-09-16 | 2015-12-30 | 东软集团股份有限公司 | Method and device for identifying type of image feature point |
| CN105588563A (en) * | 2016-01-15 | 2016-05-18 | 武汉光庭科技有限公司 | Joint calibration method of binocular camera and inertial navigation unit in automatic driving |
| CN106558080A (en) * | 2016-11-14 | 2017-04-05 | 天津津航技术物理研究所 | Join on-line proving system and method outside a kind of monocular camera |
| CN107388967A (en) * | 2017-08-14 | 2017-11-24 | 上海汽车集团股份有限公司 | A kind of outer parameter compensation method of vehicle-mounted three-dimensional laser sensor and device |
| CN109767475A (en) * | 2018-12-28 | 2019-05-17 | 广州小鹏汽车科技有限公司 | A kind of method for calibrating external parameters and system of sensor |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110189379A (en) | 2019-08-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109767475B (en) | External parameter calibration method and system for sensor | |
| CN106981082B (en) | Vehicle-mounted camera calibration method and device and vehicle-mounted equipment | |
| KR20200125667A (en) | In-vehicle camera self-calibration method and device, and vehicle driving method and device | |
| JP5820787B2 (en) | Image processing apparatus and program | |
| US11340071B2 (en) | Calibration system and calibration apparatus | |
| JP4636346B2 (en) | Car camera calibration apparatus, method, and program | |
| US10384612B2 (en) | Dynamic camera view to aid with trailer attachment | |
| JP6354425B2 (en) | In-vehicle camera mounting attitude detection method and apparatus | |
| CN109131067B (en) | Tripod self-propelled vehicle and obstacle avoidance method thereof | |
| JP5240527B2 (en) | Car camera calibration apparatus, method, and program | |
| CN106814753B (en) | Target position correction method, device and system | |
| CN108932737B (en) | Vehicle-mounted camera pitch angle calibration method and device, electronic equipment and vehicle | |
| US20190118712A1 (en) | Mounting angle detection device for in-vehicle camera, mounting angle calibration device, and mounting angle detection method | |
| CN110189379B (en) | Method and system for calibrating external parameters of camera | |
| WO2015029443A1 (en) | Turning angle correction method, turning angle correction device, image-capturing device, and turning angle correction system | |
| CN112489136B (en) | Calibration method, position determination device, electronic equipment and storage medium | |
| CN113850867A (en) | Camera parameter calibration method, camera parameter calibration device control method, camera parameter calibration device control device, and storage medium | |
| CN112823112A (en) | Detecting damage angles of vehicles and trailers | |
| WO2016129704A1 (en) | Onboard camera calibration apparatus for identifying mounting orientation | |
| CN109345591B (en) | Vehicle posture detection method and device | |
| JP2019028665A (en) | Calibration apparatus and method for on-vehicle camera | |
| CN110207715B (en) | Correction method and correction system for vehicle positioning | |
| CN110796604A (en) | Image correction method and device | |
| US9562772B2 (en) | Method for determining initial data for determining position data of a vehicle | |
| CN109087361B (en) | Monocular camera-based method for calibrating transverse distance of forward object |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |