CN113643358B - External parameter calibration method, device, storage medium and system of camera - Google Patents
External parameter calibration method, device, storage medium and system of camera Download PDFInfo
- Publication number
- CN113643358B CN113643358B CN202110914889.4A CN202110914889A CN113643358B CN 113643358 B CN113643358 B CN 113643358B CN 202110914889 A CN202110914889 A CN 202110914889A CN 113643358 B CN113643358 B CN 113643358B
- Authority
- CN
- China
- Prior art keywords
- coordinate system
- camera
- calibration
- calibration reference
- reference object
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 230000007613 environmental effect Effects 0.000 claims description 25
- 239000011159 matrix material Substances 0.000 claims description 12
- 238000004891 communication Methods 0.000 claims description 2
- 230000004927 fusion Effects 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000012545 processing Methods 0.000 description 7
- 230000002093 peripheral effect Effects 0.000 description 6
- 230000009466 transformation Effects 0.000 description 4
- 230000004044 response Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The application relates to an external parameter calibration method, device, storage medium and system of a camera, belonging to the technical field of computers, wherein the method comprises the following steps: responding to an instruction for calibrating the camera, and controlling the camera to acquire images of the current calibration environment to obtain an environment image; the ground clearance of the calibration reference object in the calibration environment is the same as the equipment height of the self-mobile equipment; determining the position of a marked reference object in the environment image under a camera coordinate system; determining an external parameter calibration result of the camera based on the position of the calibration reference object under the camera coordinate system and the position of the calibration reference object under the world coordinate system; the problem that errors exist when the world coordinate system of the camera is inconsistent with the world coordinate system of the self-mobile device and the environment data collected by the camera and the self-mobile device are fused or compared can be solved; the position under the camera coordinate system can be ensured to be converted into the world coordinate system of the self-mobile device, and the accuracy of subsequent data fusion or data comparison can be improved.
Description
[ field of technology ]
The application relates to an external parameter calibration method, device, storage medium and system of a camera, and belongs to the technical field of computers.
[ background Art ]
The existing self-mobile equipment often has the functions of environment recognition, path planning, map construction and the like. At this time, the self-mobile device generally collects environmental data of the working area, and analyzes and processes the environmental data to implement corresponding functions.
However, the range of gathering environmental data from mobile devices is limited. Based on this, a camera for monitoring the self-mobile device may be additionally provided in the work area. At this time, the environmental data collected from the mobile device is fused with the environmental data collected in the camera monitoring process, so as to analyze the fused environmental data, and the accuracy of data analysis can be improved.
Before the camera is used, external parameter calibration is needed in advance. The traditional external parameter calibration mode comprises the following steps: marking a position under a world coordinate system on the ground; and calculating a homography matrix according to pixel positions of the positions under the world coordinate systems under the camera coordinate systems to obtain an external parameter calibration result of the camera.
However, since the self-mobile device has a certain height, the calculated external parameter calibration results are obtained by converting the data in the camera coordinate system to the ground, which causes a problem that the position in the world coordinate system of the self-mobile device is in error with the actual position.
[ invention ]
The application provides an external parameter calibration method, device, storage medium and system of a camera, which can solve the problem that camera coordinates acquired by the camera are converted into a world coordinate system established based on the ground, and the world coordinate system of a self-mobile device is established based on a plane where the device height of the self-mobile device is located, and errors exist when environmental data acquired by the camera and the world coordinate system are fused or compared. The application provides the following technical scheme:
in a first aspect, a method for calibrating an external parameter of a camera is provided, the method comprising:
responding to an instruction for calibrating the camera, and controlling the camera to acquire an image of a current calibration environment to obtain an environment image of the calibration environment; at least one calibration reference object is placed in the calibration environment, and the ground clearance of the calibration reference object is the same as the equipment height of the self-moving equipment;
determining the position of the calibration reference object in the environment image under a camera coordinate system;
determining an external parameter calibration result of the camera based on the position of the calibration reference object under a camera coordinate system and the position of the calibration reference object under a world coordinate system; the external parameter calibration result is used for converting the position of the calibration reference object under a camera coordinate system into the position under a world coordinate system.
Optionally, the determining the position of the calibration reference in the environment image under a camera coordinate system includes:
displaying the environmental image in a user interface;
and responding to the selection operation of the image position in the environment image, and acquiring the position of the calibration reference object indicated by the selection operation under a camera coordinate system.
Optionally, the number of positions in the camera coordinate system is at least two; the method further comprises the steps of:
determining a corresponding relation between the position of the calibration reference object under a camera coordinate system and the position of the calibration reference object under a world coordinate system according to the execution sequence of the selection operation; the execution order corresponds to the positions in the world coordinate system one by one.
Optionally, the determining the position of the calibration reference in the environment image under a camera coordinate system includes:
identifying a specified position of a calibration reference object in the environment image by using an image identification algorithm, and determining the position of the specified position under a camera coordinate system as the position of the calibration reference object under the camera coordinate system; accordingly, in the calibration environment, the position of the calibration reference in the world coordinate system is the position of the specified position in the world coordinate system.
Optionally, the number of the positions of the calibration reference object under the camera coordinate system is 4, and each position of the calibration reference object under the camera coordinate system corresponds to one position under the world coordinate system;
the determining the external parameter calibration result of the camera based on the position of the calibration reference object under the camera coordinate system and the position of the calibration reference object under the world coordinate system comprises the following steps:
and calculating a homography matrix of the camera by using each position of the calibration reference object under a camera coordinate system and each position of the calibration reference object under a world coordinate system to obtain the external parameter calibration result.
In a second aspect, there is provided an external reference calibration device for a camera, the device comprising a processor and a memory; the memory stores a program that is loaded and executed by the processor to implement the method for calibrating an external parameter of the camera provided in the first aspect.
In a third aspect, a computer readable storage medium is provided, in which a program is stored, which program, when being executed by a processor, is adapted to implement the method for calibrating an external parameter of a camera provided in the first aspect.
In a fourth aspect, there is provided an external reference calibration system for a camera, the system comprising:
at least one calibration reference placed within a calibration environment, the calibration reference having a same elevation above ground as the device of the self-moving device;
the camera to be calibrated comprises the calibration reference object in the acquisition range;
an external parameter calibration device for a camera in communication with said camera to be calibrated, said external parameter calibration device for a camera comprising the external parameter calibration device for a camera of claim 6.
Optionally, the specified position of the calibration reference is located at a position under a world coordinate system marked in advance.
Optionally, the system further comprises a laser ground wire meter; the calibration reference object is a cube;
for each position under the world coordinate system marked in advance, when the calibration reference object is placed at the position under the world coordinate system, the laser ground wire instrument emits two mutually intersected laser lines which are respectively parallel to the x axis and the y axis of the world coordinate system;
when the designated position of the calibration reference is placed at a position in the world coordinate system, the edge of the calibration reference is tangent to the two laser lines, and the designated position overlaps with the position in the world coordinate system.
The beneficial effects of this application include at least: the camera is controlled to acquire images of the current calibration environment by responding to the instruction for calibrating the camera, so that an environment image of the calibration environment is obtained; at least one calibration reference object is placed in the calibration environment, and the ground clearance of the calibration reference object is the same as the equipment height of the self-moving equipment; determining the position of a marked reference object in the environment image under a camera coordinate system; determining an external parameter calibration result of the camera based on the position of the calibration reference object under the camera coordinate system and the position of the calibration reference object under the world coordinate system; the external parameter calibration result is used for converting the position of the calibration reference object under the camera coordinate system into the position under the world coordinate system; the problem that the camera coordinates acquired by the camera are converted into a world coordinate system established based on the ground, the world coordinate system of the self-mobile device is established based on a plane where the device height of the self-mobile device is located, and errors exist when the environment data acquired by the camera and the environment data acquired by the device height of the self-mobile device are fused or compared; because the calibration reference object with the same ground clearance as the equipment height of the self-mobile equipment is arranged in the calibration environment, camera calibration is carried out according to the calibration reference object, so that the position under the camera coordinate system can be ensured to be converted into the position under the world coordinate system determined based on the equipment height of the self-mobile equipment, namely, the position under the world coordinate system of the self-mobile equipment can be converted, and the accuracy of subsequent data fusion or data comparison can be improved.
The foregoing description is only an overview of the technical solutions of the present application, and in order to make the technical means of the present application more clearly understood, it can be implemented according to the content of the specification, and the following detailed description of the preferred embodiments of the present application will be given with reference to the accompanying drawings.
[ description of the drawings ]
FIG. 1 is a schematic diagram of a working system of a self-mobile device according to one embodiment of the present application;
FIG. 2 is a schematic diagram of the external parameter calibration system of the camera according to one embodiment of the present application;
FIG. 3 is a flow chart of a method for calibrating external parameters of a camera according to one embodiment of the present application;
FIG. 4 is a schematic illustration of an environmental image captured by a camera provided in one embodiment of the present application;
FIG. 5 is a block diagram of an external parameter calibration device for a camera provided in one embodiment of the present application;
fig. 6 is a block diagram of an external parameter calibration device for a camera according to another embodiment of the present application.
[ detailed description ] of the invention
The detailed description of the present application is further described in detail below with reference to the drawings and examples. The following examples are illustrative of the present application, but are not intended to limit the scope of the present application.
First, several terms referred to in this application are described.
Pixel coordinates: refers to the location of the pixel in the image. The vertex of the upper left corner of the general pixel coordinate system is the origin, the horizontal right is the u-axis, and the vertical down is the v-axis. In the pixel coordinate system, the coordinates of each pixel are represented by pixels. However, the pixel representation method cannot reflect the physical size of an object in an image, and therefore, it is necessary to convert pixel coordinates into image coordinates.
Image coordinate system: translating an origin of a pixel coordinate system to the center of an image to obtain the origin of the image coordinate system, wherein the x-axis of the image coordinate system is parallel to the u-axis of the pixel coordinate system, and the directions are the same; the y-axis of the image coordinate system is parallel to the v-axis of the pixel coordinate system and is in the same direction.
Camera coordinate system: the optical axis of the camera is taken as the Z axis, and the central position of the light ray in the optical system of the camera is the origin Oc (namely the center of the lens of the camera). The horizontal axis Xc of the camera coordinate system is parallel to the x-axis of the image coordinate system, and the vertical axis Yc of the camera coordinate system is parallel to the y-axis of the image coordinate system. The distance OcOi between the origin of the camera coordinate system and the origin of the image coordinate system is f (i.e., focal length).
World coordinate system: refers to a mapping relationship between an image and a real object. The origin of the world coordinate system is Ow, while the three-dimensional axes Xw, yw, zw of the world coordinate system are not necessarily parallel to the other camera coordinate system, or the image coordinate system, or the pixel coordinate system, but are at an angle and have a certain translation. When the camera coordinate system is translated and rotated around X, Y, Z axis according to certain parameters, the coordinates in the world coordinate system can be obtained. The mode of translating and rotating around the X, Y, Z axis is the external parameter calibration result of the camera.
In other words, the external parameters of the camera are used to indicate the rotational and translational transformation relationship of the camera with respect to the world coordinate system.
Conventional self-moving devices are typically provided with sensors to collect environmental data of the current work area. The sensor includes, but is not limited to, a lidar sensor, an obstacle avoidance sensor, a ranging sensor, a vision sensor, etc., and the type of the sensor is not specifically exemplified in the present embodiment. The self-mobile device can realize the functions of map construction, obstacle avoidance, path planning, target identification and the like of a working area based on the environmental data acquired by the sensor.
However, the sensing range of the sensor mounted on the self-mobile device is often small, and the posture of the collected data is fixed, so when the corresponding function is implemented by using the environmental data collected by the sensor, the analysis of the environmental data may be inaccurate, resulting in a problem of poor effect of implementing the corresponding function.
Based on this, referring to fig. 1, a working system of a self-mobile device is proposed in the present application, which comprises a self-mobile device 10 and a camera 20, wherein the self-mobile device 10 and the camera 20 are both disposed in the same working area. The camera 20 is installed at a higher level than the device level of the self-mobile device 10 so that, when a sensor is installed on the self-mobile device 10, the camera 20 can collect environmental data of a work area in a different posture from the collection posture of the sensor. Then, the environmental data collected by the camera 20 is fused with the environmental data collected by itself, and the fused environmental data is used for analysis, thereby improving the realization effect of the corresponding function.
Before the camera 20 is used, external parameter calibration is needed, that is, rotation and translation transformation relation of the camera 20 relative to the world coordinate system is determined. In the conventional camera calibration method, the planes of Xw axis and Yw axis of the world coordinate system are located on the ground, and at this time, the camera coordinates collected by the camera 20 are converted into the world coordinate system established based on the ground. The world coordinate system of the self-mobile device 10 is established based on the plane of the device height of the self-mobile device 10, so that when the environment data are fused, the problem that the fusion of the environment data has errors is caused.
Based on the above technical problems, the present embodiment provides a system and a method for calibrating an external parameter of a camera, and the system and the method are respectively described below.
Fig. 2 is a schematic structural diagram of an external parameter calibration system of a camera according to an embodiment of the present application, where, as can be seen from fig. 2, the system at least includes: calibration reference 30, camera 20 to be calibrated, and external reference calibration device 40.
The calibration reference 30 is placed within a calibration environment. The number of calibration references 30 is at least one, and the ground clearance h of each calibration reference 30 is the same as the device height of the self-moving device.
In this embodiment, the device height is the height between the ground and the plane in which the Xw axis and the Yw axis are located in the world coordinate system established from the mobile device (i.e., the world coordinate system of the mobile device). The plane in which the Xw axis and the Yw axis lie is typically the plane in which the upper surface of the self-moving device lies. In practical implementation, the device height may also be the installation height of the sensor installed on the self-mobile device, that is, the distances between the Xw axis and the Yw axis in the world coordinate system of the self-mobile device and the ground are the installation heights of the sensor, and the method of determining the device height of the self-mobile device is not limited in this embodiment. The world coordinate system in the application refers to the world coordinate system of the self-mobile device, and the height between the plane where the Xw axis and the Yw axis of the world coordinate system are located and the ground is the device height.
Wherein, the ground clearance of the calibration reference 30 is the distance between the ground and the plane of the calibration reference 30 farthest from the ground.
In one example, the specified location of the calibration reference 30 is located at a position on the ground under a pre-marked world coordinate system. At this time, the height above the ground of the specified position of the calibration reference 30 is the same as the device height of the self-moving device.
Wherein the designated location is distinguished from other locations on the calibration reference 30. Such as: referring to fig. 2, the calibration reference 30 is a cube, and the upper surface of the cube has a 2×2 checkerboard, and the designated position is the center position of the 2×2 checkerboard.
Optionally, there are one or at least two designated locations on each calibration reference 30.
When there is one designated location on each calibration reference 30, the number of calibration references 30 is the same as the number of designated locations. Such as: in fig. 2 there are 4 calibration references, each having a designated position that is the center position of the checkerboard of 2 x 2.
The number of calibration references 30 is less than the number of designated locations when at least two designated locations are on each calibration reference 30. Such as: the calibration reference 30 is a cube, the upper surface of the cube has a checkerboard of 3*3, the designated positions are 3*3 the 4 vertices of one lattice in the middle of the checkerboard, and the implementation of the calibration reference 30 and the implementation of the designated positions are not limited in this embodiment.
In the present embodiment, the specific position is taken as an example of the checkered intersection, and the calibration reference 30 may be further provided with a light emitting device in actual implementation, and the specific position may also be a light emitting position of the light emitting device.
Optionally, the location under world coordinate system marked on the ground is marked using a laser ground finder and a tape measure. Such as: two mutually intersected laser lines are emitted by a laser ground wire instrument and are respectively parallel to the x axis and the y axis of a world coordinate system, four points with the world coordinates of (0, 0), (0, 1), (1, 0) and (1, 1) are drawn on the ground by using a tape measure, and the positions under the world coordinate system are obtained.
To ensure accuracy in placing the specified position of the calibration reference 30 at a location in the world coordinate system, the external reference calibration system also includes a laser ground level (not shown). Illustratively, the calibration reference is a cube. At this time, for each position under the world coordinate system marked in advance, when a calibration reference is placed at the position under the world coordinate system, the laser ground wire instrument emits two mutually intersected laser lines which are respectively parallel to the x axis and the y axis of the world coordinate system; when the designated position of the calibration reference is placed at a position in the world coordinate system, the edge of the calibration reference is tangent to the two laser lines, and the designated position overlaps with the position in the world coordinate system.
In other examples, the calibration reference 30 may be placed on other objects instead of directly on the ground, so as to ensure that the ground clearance of the calibration reference 30 is the same as the device height of the self-moving device, and the placement of the calibration reference 30 is not limited in this embodiment.
The acquisition range of the camera 20 includes a calibration reference 30. In this embodiment, the height of the camera 20 is higher than the height of the self-moving device, so that the camera 20 can acquire the plane of the calibration reference 30 farthest from the ground, thereby acquiring the image of the designated location.
The external parameter calibration device 40 is communicatively connected to the camera 20, and the external parameter calibration device 40 may be a computer, a tablet computer, a notebook computer, a mobile phone, or the like, and the embodiment does not limit the device type of the external parameter calibration device.
In this embodiment, the external parameter calibration device 40 is configured to control the camera to perform image acquisition on a current calibration environment in response to an instruction for calibrating the camera, so as to obtain an environment image of the calibration environment; determining the position of a marked reference object in the environment image under a camera coordinate system; and determining an external parameter calibration result of the camera based on the position of the calibration reference object under the camera coordinate system and the position of the calibration reference object under the world coordinate system.
At least one calibration reference is placed in the calibration environment, and the ground clearance of the calibration reference is the same as the equipment height of the self-moving equipment. The external reference calibration result is used for converting the position of the calibration reference object in the camera coordinate system into the position in the world coordinate system.
In this embodiment, by setting the calibration reference having the same ground clearance as the device height of the self-mobile device in the calibration environment, it is possible to ensure that the position under the camera coordinate system is converted to the position under the world coordinate system determined based on the device height of the self-mobile device, that is, the position under the world coordinate system of the self-mobile device, and it is possible to improve the accuracy of the subsequent data fusion or data comparison.
The method for calibrating the external parameters of the camera provided by the application is described below based on the external parameter calibration system of the camera shown in fig. 2. The present application describes an example in which the external parameter calibration method of the camera provided in each embodiment is used in the external parameter calibration device 40 in the external parameter calibration system shown in fig. 2.
Fig. 3 is a flowchart of a method for calibrating an external parameter of a camera according to an embodiment of the present application, where the method includes at least the following steps:
step 301, in response to an instruction for calibrating a camera, controlling the camera to acquire an image of a current calibration environment, so as to obtain an environment image of the calibration environment; at least one calibration reference is placed in the calibration environment, and the ground clearance of the calibration reference is the same as the equipment height of the self-moving equipment.
In one example, the external parameter calibration device is provided with a calibration control, and when a calibration operation acting on the calibration control is received, an instruction for calibrating the camera is generated.
Optionally, the calibration control may be an entity button disposed on the parameter calibration device, or the external parameter calibration device has a touch display screen, where the calibration control is a virtual touch button displayed on the touch display screen, and the implementation manner of the calibration control is not limited in this embodiment.
The environmental image captured by the camera includes an image of the calibration reference. Such as: the environmental image acquired by the camera is shown with reference to fig. 4, which includes an image of 4 calibration references 30, each having a 2 x 2 checkerboard on its upper surface.
Step 302, determining a position of a marked reference object in the environment image under a camera coordinate system.
In one example, determining a location of a calibration reference in an environmental image under a camera coordinate system includes: displaying an ambient image in a user interface; and responding to the selection operation of the image position in the environment image, and acquiring the position of the calibration reference object indicated by the selection operation under the camera coordinate system.
When the number of the positions under the camera coordinate system is at least two, determining the corresponding relation between the position of the calibration reference object under the camera coordinate system and the position of the calibration reference object under the world coordinate system; the execution order corresponds to the positions in the world coordinate system one by one.
Such as: referring to fig. 4, the first execution order corresponds to the position (0, 0) under the world coordinate system, the second execution order corresponds to the position (0, 1) under the world coordinate system, the third execution order corresponds to the position (1, 0) under the world coordinate system, and the fourth execution order corresponds to the position (1, 1) under the world coordinate system. At this time, the position 41 in the camera coordinate system indicated by the selection operation performed for the first time is the position in the camera coordinate system corresponding to the position (0, 0) in the world coordinate system; the position 42 in the camera coordinate system indicated by the selection operation performed the second time is the position in the camera coordinate system corresponding to the position (0, 1) in the world coordinate system; the position 43 in the camera coordinate system indicated by the selection operation performed for the third time is the position in the camera coordinate system corresponding to the position (1, 0) in the world coordinate system; the position 44 in the camera coordinate system indicated by the selection operation performed the fourth time is the position in the camera coordinate system corresponding to the position (1, 1) in the world coordinate system.
In practical implementation, the one-to-one correspondence between the execution order and the position in the world coordinate system may be other ways, which are not listed here.
Optionally, the correspondence between the position in the camera coordinate system and the position in the world coordinate system may also be manually adjusted by the user, such as: displaying in the user interface a position in the world coordinate system and a position in the camera coordinate system of the selection operation indication; then, a connection operation between the position in the world coordinate system and the position in the camera coordinate system is received to obtain a corresponding relationship between the position in the camera coordinate system and the position in the world coordinate system, and the method of determining the corresponding relationship between the position in the camera coordinate system and the position in the world coordinate system is not limited in this embodiment.
In another example, determining a location of a calibration reference in a camera coordinate system in an environmental image includes: identifying a designated position of the calibration reference object in the environment image by using an image identification algorithm, and determining the position of the designated position under a camera coordinate system as the position of the calibration reference object under the camera coordinate system; accordingly, in the calibration environment, the position of the calibration reference in the world coordinate system is the position of the specified position in the world coordinate system.
At this time, the designated position is distinguished from other positions of the calibration reference, and the distinction can be recognized by the image recognition algorithm.
Optionally, the image recognition algorithm includes, but is not limited to: the Scale-invariant feature transform (SIFT) algorithm, or the accelerated robust feature (Speeded Up Robust Features, SURF), etc., the present embodiment does not limit the implementation of the image recognition algorithm.
Step 303, determining an external parameter calibration result of the camera based on the position of the calibration reference object under the camera coordinate system and the position of the calibration reference object under the world coordinate system; the external reference calibration result is used for converting the position of the calibration reference object in the camera coordinate system into the position in the world coordinate system.
In one example, the extrinsic calibration results are homography matrices for the camera. Accordingly, the number of positions in the camera coordinate system is 4, and each position of the calibration reference in the camera coordinate system corresponds to one position in the world coordinate system. At this time, determining an external parameter calibration result of the camera based on the position of the calibration reference object under the camera coordinate system and the position of the calibration reference object under the world coordinate system includes: and calculating a homography matrix of the camera by using each position of the calibration reference object under the camera coordinate system and each position of the calibration reference object under the world coordinate system to obtain an external parameter calibration result.
Homography (Homography) transformation is used to describe the positional mapping relationship of objects between a world coordinate system and a pixel coordinate system, and the corresponding transformation matrix is called Homography matrix. The homography matrix may be defined as:
wherein M is an internal reference matrix of the camera, the internal reference matrix is known, and the position under the camera coordinate system can be converted into a pixel coordinate system by using the internal reference matrix, and based on the internal reference matrix, a final homography matrix can be obtained by using the world coordinate system and the pixel coordinate system.
In summary, according to the external parameter calibration method of the camera provided by the embodiment, the camera is controlled to acquire an image of a current calibration environment by responding to the instruction for calibrating the camera, so as to obtain an environment image of the calibration environment; at least one calibration reference object is placed in the calibration environment, and the ground clearance of the calibration reference object is the same as the equipment height of the self-moving equipment; determining the position of a marked reference object in the environment image under a camera coordinate system; determining an external parameter calibration result of the camera based on the position of the calibration reference object under the camera coordinate system and the position of the calibration reference object under the world coordinate system; the external parameter calibration result is used for converting the position of the calibration reference object under the camera coordinate system into the position under the world coordinate system; the problem that the camera coordinates acquired by the camera are converted into a world coordinate system established based on the ground, the world coordinate system of the self-mobile device is established based on a plane where the device height of the self-mobile device is located, and errors exist when the environment data acquired by the camera and the environment data acquired by the device height of the self-mobile device are fused or compared; because the calibration reference object with the same ground clearance as the equipment height of the self-mobile equipment is arranged in the calibration environment, camera calibration is carried out according to the calibration reference object, so that the position under the camera coordinate system can be ensured to be converted into the position under the world coordinate system determined based on the equipment height of the self-mobile equipment, namely, the position under the world coordinate system of the self-mobile equipment can be converted, and the accuracy of subsequent data fusion or data comparison can be improved.
Fig. 5 is a block diagram of an external parameter calibration device for a camera according to an embodiment of the present application. The device is used in an external parameter calibration device 40 in the external parameter calibration system shown in fig. 2, and comprises at least the following modules: an image acquisition module 510, a position determination module 520, and an external reference calibration module 530.
The image acquisition module 510 is configured to control the camera to perform image acquisition on a current calibration environment in response to an instruction for calibrating the camera, so as to obtain an environment image of the calibration environment; at least one calibration reference object is placed in the calibration environment, and the ground clearance of the calibration reference object is the same as the equipment height of the self-moving equipment;
a position determining module 520, configured to determine a position of the calibration reference in the environmental image under a camera coordinate system;
an external parameter calibration module 530, configured to determine an external parameter calibration result of the camera based on a position of the calibration reference object in a camera coordinate system and a position of the calibration reference object in a world coordinate system; the external parameter calibration result is used for converting the position of the calibration reference object under a camera coordinate system into the position under a world coordinate system.
For relevant details reference is made to the method embodiments described above.
It should be noted that: in the external parameter calibration device of the camera provided in the above embodiment, only the division of the above functional modules is used for illustration when the external parameter calibration of the camera is performed, and in practical application, the above functional allocation may be performed by different functional modules according to needs, that is, the internal structure of the external parameter calibration device of the camera is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the external parameter calibration device of the camera provided in the above embodiment and the external parameter calibration method embodiment of the camera belong to the same concept, and detailed implementation processes of the external parameter calibration device and the external parameter calibration method embodiment of the camera are detailed in the method embodiment, and are not described herein again.
Fig. 6 is a block diagram of an external parameter calibration device for a camera according to an embodiment of the present application. The apparatus includes an external reference calibration device 40 in the external reference calibration system shown in fig. 2. The apparatus comprises at least a processor 601 and a memory 602.
Processor 601 may include one or more processing cores, such as: 4 core processors, 8 core processors, etc. The processor 601 may be implemented in at least one hardware form of DSP (Digital Signal Processing ), FPGA (Field-Programmable Gate Array, field programmable gate array), PLA (Programmable Logic Array ). The processor 601 may also include a main processor, which is a processor for processing data in an awake state, also called a CPU (Central Processing Unit ), and a coprocessor; a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 601 may integrate a GPU (Graphics Processing Unit, image processor) for rendering and drawing of content required to be displayed by the display screen. In some embodiments, the processor 601 may also include an AI (Artificial Intelligence ) processor for processing computing operations related to machine learning.
The memory 602 may include one or more computer-readable storage media, which may be non-transitory. The memory 602 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 602 is used to store at least one instruction for execution by processor 601 to implement the method of calibrating external parameters of a camera provided by the method embodiments in the present application.
In some embodiments, the external parameter calibration device may further include: a peripheral interface and at least one peripheral. The processor 601, memory 602, and peripheral interfaces may be connected by buses or signal lines. The individual peripheral devices may be connected to the peripheral device interface via buses, signal lines or circuit boards. Illustratively, peripheral devices include, but are not limited to: radio frequency circuitry, touch display screens, audio circuitry, and power supplies, among others.
Of course, the external reference calibration device may also include fewer or more components, as this embodiment is not limited in this regard.
Optionally, the application further provides a computer readable storage medium, in which a program is stored, and the program is loaded and executed by a processor to implement the external parameter calibration method of the camera according to the above method embodiment.
Optionally, the application further provides a computer product, which includes a computer readable storage medium, where a program is stored, and the program is loaded and executed by a processor to implement the external parameter calibration method of the camera according to the method embodiment.
The technical features of the above-described embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above-described embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples merely represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the invention. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application is to be determined by the claims appended hereto.
Claims (10)
1. A method for calibrating an external parameter of a camera, the method comprising:
responding to an instruction for calibrating the camera, and controlling the camera to acquire an image of a current calibration environment to obtain an environment image of the calibration environment; at least one calibration reference object is placed in the calibration environment, and the ground clearance of the calibration reference object is the same as the equipment height of the self-moving equipment;
determining the position of the calibration reference object in the environment image under a camera coordinate system;
determining an external parameter calibration result of the camera based on the position of the calibration reference object under a camera coordinate system and the position of the calibration reference object under a world coordinate system; the external parameter calibration result is used for converting the position of the calibration reference object under a camera coordinate system into the position under a world coordinate system.
2. The method of claim 1, wherein the determining the location of the calibration reference in the environmental image under a camera coordinate system comprises:
displaying the environmental image in a user interface;
and responding to the selection operation of the image position in the environment image, and acquiring the position of the calibration reference object indicated by the selection operation under a camera coordinate system.
3. The method of claim 2, wherein the number of positions in the camera coordinate system is at least two; the method further comprises the steps of:
determining a corresponding relation between the position of the calibration reference object under a camera coordinate system and the position of the calibration reference object under a world coordinate system according to the execution sequence of the selection operation; the execution order corresponds to the positions in the world coordinate system one by one.
4. The method of claim 1, wherein the determining the location of the calibration reference in the environmental image under a camera coordinate system comprises:
identifying a specified position of a calibration reference object in the environment image by using an image identification algorithm, and determining the position of the specified position under a camera coordinate system as the position of the calibration reference object under the camera coordinate system; accordingly, in the calibration environment, the position of the calibration reference in the world coordinate system is the position of the specified position in the world coordinate system.
5. A method according to any one of claims 1 to 3, wherein the number of positions of the calibration reference under the camera coordinate system is 4, each position of the calibration reference under the camera coordinate system corresponding to a position under the world coordinate system;
the determining the external parameter calibration result of the camera based on the position of the calibration reference object under the camera coordinate system and the position of the calibration reference object under the world coordinate system comprises the following steps:
and calculating a homography matrix of the camera by using each position of the calibration reference object under a camera coordinate system and each position of the calibration reference object under a world coordinate system to obtain the external parameter calibration result.
6. An external parameter calibration device of a camera is characterized in that the device comprises a processor and a memory; the memory stores therein a program that is loaded and executed by the processor to realize the external parameter calibration method of the camera according to any one of claims 1 to 5.
7. A computer readable storage medium, characterized in that the storage medium has stored therein a program which, when executed by a processor, is adapted to carry out a method of calibrating an external parameter of a camera according to any of claims 1 to 5.
8. A camera exogenous calibration system, the system comprising:
at least one calibration reference placed within a calibration environment, the calibration reference having a same elevation above ground as the device of the self-moving device;
the camera to be calibrated comprises the calibration reference object in the acquisition range;
an external parameter calibration device for a camera in communication with said camera to be calibrated, said external parameter calibration device for a camera comprising the external parameter calibration device for a camera of claim 6.
9. The system of claim 8, wherein the specified location of the calibration reference is located at a position under a pre-marked world coordinate system.
10. The system of claim 9, further comprising a laser ground meter; the calibration reference object is a cube;
for each position under the world coordinate system marked in advance, when the calibration reference object is placed at the position under the world coordinate system, the laser ground wire instrument emits two mutually intersected laser lines which are respectively parallel to the x axis and the y axis of the world coordinate system;
when the designated position of the calibration reference is placed at a position in the world coordinate system, the edge of the calibration reference is tangent to the two laser lines, and the designated position overlaps with the position in the world coordinate system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110914889.4A CN113643358B (en) | 2021-08-10 | 2021-08-10 | External parameter calibration method, device, storage medium and system of camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110914889.4A CN113643358B (en) | 2021-08-10 | 2021-08-10 | External parameter calibration method, device, storage medium and system of camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113643358A CN113643358A (en) | 2021-11-12 |
| CN113643358B true CN113643358B (en) | 2023-07-07 |
Family
ID=78420757
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110914889.4A Active CN113643358B (en) | 2021-08-10 | 2021-08-10 | External parameter calibration method, device, storage medium and system of camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113643358B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105678783A (en) * | 2016-01-25 | 2016-06-15 | 西安科技大学 | Data fusion calibration method of catadioptric panorama camera and laser radar |
| CN109102546A (en) * | 2018-07-16 | 2018-12-28 | 珠海市微半导体有限公司 | A kind of scaling method of the robot camera based on more scaling boards |
| CN110322513A (en) * | 2018-03-30 | 2019-10-11 | 杭州海康威视数字技术股份有限公司 | A kind of Camera extrinsic scaling method, device and electronic equipment |
| CN111360810A (en) * | 2018-12-25 | 2020-07-03 | 深圳市优必选科技有限公司 | External parameter calibration method and device for robot sensor, robot and storage medium |
| CN112414444A (en) * | 2019-08-22 | 2021-02-26 | 阿里巴巴集团控股有限公司 | Data calibration method, computer equipment and storage medium |
| CN112509125A (en) * | 2020-12-14 | 2021-03-16 | 广州迦恩科技有限公司 | Three-dimensional reconstruction method based on artificial markers and stereoscopic vision |
| CN112882053A (en) * | 2021-01-21 | 2021-06-01 | 清华大学深圳国际研究生院 | Method for actively calibrating external parameters of laser radar and encoder |
| CN112949478A (en) * | 2021-03-01 | 2021-06-11 | 浙江国自机器人技术股份有限公司 | Target detection method based on holder camera |
| CN113052919A (en) * | 2021-04-23 | 2021-06-29 | 深圳市商汤科技有限公司 | Calibration method and device of visual sensor, electronic equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9386302B2 (en) * | 2014-05-21 | 2016-07-05 | GM Global Technology Operations LLC | Automatic calibration of extrinsic and intrinsic camera parameters for surround-view camera system |
-
2021
- 2021-08-10 CN CN202110914889.4A patent/CN113643358B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105678783A (en) * | 2016-01-25 | 2016-06-15 | 西安科技大学 | Data fusion calibration method of catadioptric panorama camera and laser radar |
| CN110322513A (en) * | 2018-03-30 | 2019-10-11 | 杭州海康威视数字技术股份有限公司 | A kind of Camera extrinsic scaling method, device and electronic equipment |
| CN109102546A (en) * | 2018-07-16 | 2018-12-28 | 珠海市微半导体有限公司 | A kind of scaling method of the robot camera based on more scaling boards |
| CN111360810A (en) * | 2018-12-25 | 2020-07-03 | 深圳市优必选科技有限公司 | External parameter calibration method and device for robot sensor, robot and storage medium |
| CN112414444A (en) * | 2019-08-22 | 2021-02-26 | 阿里巴巴集团控股有限公司 | Data calibration method, computer equipment and storage medium |
| CN112509125A (en) * | 2020-12-14 | 2021-03-16 | 广州迦恩科技有限公司 | Three-dimensional reconstruction method based on artificial markers and stereoscopic vision |
| CN112882053A (en) * | 2021-01-21 | 2021-06-01 | 清华大学深圳国际研究生院 | Method for actively calibrating external parameters of laser radar and encoder |
| CN112949478A (en) * | 2021-03-01 | 2021-06-11 | 浙江国自机器人技术股份有限公司 | Target detection method based on holder camera |
| CN113052919A (en) * | 2021-04-23 | 2021-06-29 | 深圳市商汤科技有限公司 | Calibration method and device of visual sensor, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113643358A (en) | 2021-11-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109918977B (en) | Method, device and equipment for determining idle parking space | |
| EP3039655B1 (en) | System and method for determining the extent of a plane in an augmented reality environment | |
| CN112654886A (en) | External parameter calibration method, device, equipment and storage medium | |
| CN111754578B (en) | Combined calibration method for laser radar and camera, system and electronic equipment thereof | |
| CN113657224B (en) | Method, device and equipment for determining object state in vehicle-road coordination | |
| CN111815707A (en) | Point cloud determining method, point cloud screening device and computer equipment | |
| US20230244227A1 (en) | Data processing method, control apparatus and storage medium | |
| CN110260857A (en) | Calibration method, device and the storage medium of vision map | |
| CN113763478B (en) | Unmanned vehicle camera calibration method, device, equipment, storage medium and system | |
| KR102618285B1 (en) | Method and system for determining camera pose | |
| WO2022217988A1 (en) | Sensor configuration scheme determination method and apparatus, computer device, storage medium, and program | |
| CN114004882A (en) | Two-dimensional map generation method and device, terminal equipment and storage medium | |
| JP5019478B2 (en) | Marker automatic registration method and system | |
| WO2016187752A1 (en) | Method and device for measuring antenna attitude | |
| CN112308768B (en) | Data processing method, device, electronic equipment and storage medium | |
| CN113643358B (en) | External parameter calibration method, device, storage medium and system of camera | |
| CN116386373A (en) | Vehicle positioning method and device, storage medium and electronic equipment | |
| CN116912417A (en) | Texture mapping method, device, equipment and storage medium based on three-dimensional reconstruction of human face | |
| CN116203976A (en) | Indoor inspection method and device for transformer substation, unmanned aerial vehicle and storage medium | |
| CN116642490A (en) | Visual positioning navigation method based on hybrid map, robot and storage medium | |
| CN114792343A (en) | Calibration method of image acquisition equipment, and method and device for acquiring image data | |
| CN113639639A (en) | Data processing method and device for position data and storage medium | |
| CN113446940A (en) | Point cloud scanning method, device and equipment | |
| CN113177988A (en) | Calibration method, device, equipment and storage medium for dome camera and laser | |
| CN113177975A (en) | Depth calculation method and three-dimensional modeling method based on dome camera and laser radar |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |