CN111145249B - Vehicle-mounted-based automatic calibration control method, storage medium and system - Google Patents
Vehicle-mounted-based automatic calibration control method, storage medium and system Download PDFInfo
- Publication number
- CN111145249B CN111145249B CN201910817490.7A CN201910817490A CN111145249B CN 111145249 B CN111145249 B CN 111145249B CN 201910817490 A CN201910817490 A CN 201910817490A CN 111145249 B CN111145249 B CN 111145249B
- Authority
- CN
- China
- Prior art keywords
- calibration
- point
- positioning data
- vehicle
- points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000012545 processing Methods 0.000 claims abstract description 21
- 238000004590 computer program Methods 0.000 claims description 6
- 238000012795 verification Methods 0.000 claims description 4
- 230000000694 effects Effects 0.000 abstract description 3
- 102000027426 receptor tyrosine kinases Human genes 0.000 description 22
- 108091008598 receptor tyrosine kinases Proteins 0.000 description 22
- 239000003550 marker Substances 0.000 description 18
- 239000011159 matrix material Substances 0.000 description 14
- 238000012549 training Methods 0.000 description 10
- 230000009466 transformation Effects 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 238000013519 translation Methods 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000001960 triggered effect Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 210000004556 brain Anatomy 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000002427 irreversible effect Effects 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 229920002430 Fibre-reinforced plastic Polymers 0.000 description 1
- 241000282412 Homo Species 0.000 description 1
- 206010034701 Peroneal nerve palsy Diseases 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000002485 combustion reaction Methods 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000011151 fibre-reinforced plastic Substances 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 239000003292 glue Substances 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 210000004126 nerve fiber Anatomy 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 210000001525 retina Anatomy 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S19/00—Satellite radio beacon positioning systems; Determining position, velocity or attitude using signals transmitted by such systems
- G01S19/38—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system
- G01S19/39—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system the satellite radio beacon positioning system transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO
- G01S19/42—Determining position
- G01S19/43—Determining position using carrier phase measurements, e.g. kinematic positioning; using long or short baseline interferometry
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the field of image processing, and discloses an automatic calibration control method based on a vehicle, which comprises the following steps: acquiring video image data, identifying the number of standard points, and acquiring positioning data of vehicle-mounted positioning points; acquiring positioning data of a first calibration position at a first calibration distance; acquiring positioning data of a second calibration position at a second calibration distance; calculating the heading of the vehicle body according to the positioning data of the first calibration position and the second calibration position; calculating an included angle between the heading of the vehicle body and a connecting line of the first calibration point and the second calibration point; triggering calibration when the included angle is minus 90 degrees less than or equal to a first threshold value. Corresponding storage media and systems are also disclosed. Some technical effects of the invention are: and the automatic control of camera calibration is realized.
Description
Technical Field
The invention relates to the field of image processing, in particular to an automatic calibration control method in the field of image processing.
Background
Vision is an important means for humans to observe and recognize the world, accounting for 70% of the human's information acquired from the external environment. The human being uses eyes to obtain light rays reflected by surrounding objects or emitted by the human being, the light rays form images on retina, the images are transmitted to the brain through nerve fibers, the brain processes and understands visual information, and finally vision is formed. Computer vision simulates the function of human vision, a camera is used for acquiring images of surrounding environment, and a computer is used for processing the images. Computer vision can accomplish the work that human vision is insufficient, for example, the size, the distance of the object to be measured are accurately measured. The computer vision technology can be widely applied to the fields of surveying and mapping, vision detection, automatic driving and the like.
One of the fundamental tasks of computer vision is to calculate geometric information of objects in three-dimensional space starting from image information acquired by a camera, and thereby reconstruct or recognize the objects, and further recognize the real world. Wherein camera calibration is a necessary way to accomplish this task. Coordinate information of the calibration point is obtained by identifying the calibration object. The camera calibration method can be classified into a conventional calibration method and a self-calibration method according to whether a calibration object is required. The traditional camera calibration method is to take a calibration object with a known shape and size as a shooting object of a camera, then perform correlation processing on the shot image, and calculate internal and external parameters of a camera model by using a series of mathematical transformations. The camera self-calibration method does not need a calibration object and only directly performs calibration by means of the relation between corresponding points of a plurality of images. So far, the self-calibration method is flexible, but due to the fact that the unknown parameters involved in calibration are too many, stable results are difficult to obtain. On the contrary, the traditional calibration method is mature, the precision of the calibration result is high, and the method is widely applied.
For the traditional camera calibration technology, the extraction of the coordinates of the characteristic points on the calibration object is an unavoidable step, and the positioning accuracy of the characteristic points has an important influence on the final calibration result. Along with the expansion of the application range of the camera calibration technology, the applied field environment has diversified trends, such as factory environment, outdoor large background environment or coexistence of multiple calibration objects, and the like, so that the problems of insufficient feature point extraction precision or incapability of effectively completing extraction and the like are caused when images are processed. In the traditional camera calibration method, a calibration object is an indispensable component. The standard used as a reference should generally meet the following basic requirements: firstly, in image processing, the image characteristic part of the calibration object should be easy to identify, namely, the reference object should have a clear difference with the background environment; secondly, the feature of the reference should be easily extracted when image processing is performed. Calibration references generally fall into two main categories: three-dimensional calibration object and two-dimensional plane calibration object. The three-dimensional object is typically a cube of a single color. The traditional camera calibration method also has the problem of usability, and the current popular method can be completed by manual interaction. The manual interaction participation is high, so that the repeatability of the traditional calibration method is low, and the calibration needs to be repeated once.
Disclosure of Invention
In order to at least solve the automatic control problem of camera calibration, the invention provides an automatic calibration control method based on vehicle-mounted, which has the following technical scheme:
acquiring image data, identifying the number of standard points, and acquiring positioning data of vehicle-mounted positioning points; acquiring positioning data of a first calibration position at a first calibration distance; acquiring positioning data of a second calibration position at a second calibration distance; calculating the heading of the vehicle body according to the positioning data of the first calibration position and the second calibration position; calculating an included angle between the heading of the vehicle body and a connecting line of the first calibration point and the second calibration point; triggering calibration when the included angle is minus 90 degrees less than or equal to a first threshold value.
Preferably, the positioning data of the vehicle positioning points are acquired after all the positioning points are identified.
Preferably, the positioning data is RTK positioning data.
Preferably, the first calibration distance is equal to or less than the maximum calibration working radius, and the second calibration distance is equal to or more than the minimum calibration working radius.
Preferably, the maximum nominal working radius is 8m and the minimum nominal working radius is 3m.
Preferably, the first threshold is 5 °.
Preferably, the method further comprises the following steps: any one of the calibration points is identified, and the theoretical world coordinates of the calibration points are calculated; and when the absolute value of the error between the theoretical world coordinate and the actual world coordinate of the calibration point is smaller than a second threshold value, storing the calibration parameters, and terminating the calibration.
Preferably, the second threshold is 20cm.
Accordingly, the present invention discloses a readable storage medium having stored thereon a computer program for performing the aforementioned method.
The invention also discloses an automatic calibration control system, which comprises a calibration point identification module, a car body course judgment module, a triggering module and a verification module. The calibration point identification module is used for acquiring and processing image data and identifying calibration points; the vehicle body course judging module is used for acquiring and processing positioning data of the vehicle-mounted positioning points and judging the position relation between the vehicle body course and the positioning points; the triggering module is used for sending a calibration instruction and starting calibration; the calibration module is used for calibrating calibration results.
The method, the storage medium and the device provided by the invention at least provide a solution for automatic control of camera calibration, and can well realize automatic control during camera calibration.
Drawings
For a better understanding of the technical solutions of the present invention, reference is made to the following drawings for assistance in describing the prior art or embodiments. The drawings will illustrate selectively the products or processes involved in the prior art or some embodiments of the present invention. The basic information of these figures is as follows:
FIG. 1 is a flow chart of an automatic calibration control method based on a vehicle in one embodiment.
FIG. 2 is a schematic illustration of the mounting location of a calibration point in one embodiment.
FIG. 3 is a schematic diagram of the installation location of an RTK anchor point in one embodiment.
FIG. 4 is a schematic diagram of coordinate calculation of a coordinate point in one embodiment.
FIG. 5 is a schematic view of the road surface arrangement of a calibration object in one embodiment.
FIG. 6 is a schematic diagram of a default marker placement in one embodiment.
Detailed Description
Further technical means or technical effects to which the present invention relates will be described below, and it is apparent that examples are provided only as some embodiments of the present invention, but not all. All other embodiments, which can be made by those skilled in the art without the exercise of inventive faculty, are intended to be within the scope of the invention, based on the embodiments herein and the explicit or implicit presentation of the drawings.
On the general way, the invention discloses an automatic calibration control method based on vehicle-mounted, which comprises the following steps: acquiring image data, identifying the number of standard points, and acquiring positioning data of vehicle-mounted positioning points; acquiring positioning data of a first calibration position at a first calibration distance; acquiring positioning data of a second calibration position at a second calibration distance; calculating the heading of the vehicle body according to the positioning data of the first calibration position and the second calibration position; calculating an included angle between the heading of the vehicle body and a connecting line of the first calibration point and the second calibration point; triggering calibration when the included angle is minus 90 degrees less than or equal to a first threshold value.
Accordingly, in one embodiment, the present invention provides a readable storage medium having a computer program stored thereon, the computer program performing the foregoing method.
Correspondingly, in one embodiment, the invention provides an automatic calibration control system which comprises a calibration point identification module, a vehicle body course judgment module, a triggering module and a verification module. The calibration point identification module is used for acquiring and processing image data and identifying calibration points; the vehicle body course judging module is used for acquiring and processing positioning data of the vehicle-mounted positioning points and judging the position relation between the vehicle body course and the positioning points; the triggering module is used for sending a calibration instruction and starting calibration; the calibration module is used for calibrating calibration results.
Based on the general idea, it should be understood by a person skilled in the art that "vehicle" in a vehicle according to the present invention refers to a vehicle that is driven or towed by a power plant, typically from an internal combustion engine or an electric motor. The positioning data refers to position information provided by the GNSS. GNSS, satellite navigation systems, include, but are not limited to, GPS in the united states, GLONASS in russia, galileo in the european union, and BDS in china.
Some technical effects of the invention are: the automatic control in the camera calibration process is realized, the participation of manual intervention is reduced, and the repeatability is good.
In some embodiments, as shown in fig. 2 to 4, a camera or a device having a camera function is mounted and fixed on a vehicle. In general, a camera or an apparatus having an image capturing function is mounted and fixed in front of a vehicle, particularly on a front glass of a vehicle, so as to obtain a good working field of view environment. The device with the camera shooting function refers to device equipment capable of shooting and acquiring image data such as videos or pictures, such as acquisition terminal equipment for acquiring map data in the mapping field and terminal equipment for carrying out road condition visual identification in the automatic driving field.
In some embodiments, as shown in fig. 3 and 4, on-board positioning points are set on the vehicle, and the on-board positioning points are used for acquiring positioning data of the vehicle in real time or non-real time. In general, the vehicle-mounted positioning point can be an existing navigation device of the vehicle, and can also be other positioning devices which are additionally selected and arranged at other positions. In one embodiment, the center of the roof is selected as a vehicle-mounted positioning point, and a positioning device is installed.
In some embodiments, as shown in fig. 2 to 5, the calibration points are symmetrically arranged on two sides of the pavement, and the connecting line of any two calibration points on each side is parallel to the centerline of the pavement. This has the advantage of facilitating subsequent data processing.
In some embodiments, a locating device is provided, such as on a calibration point, for receiving locating data of the acquisition and transmission calibration point.
In some embodiments, the positioning data of all positioning points can also be obtained through pre-measurement, and then the positioning data is used for subsequent use.
It will be appreciated that the above embodiment operations may be disposable, i.e. first set, and subsequently without any environmental changes, without resetting; when the calibration is performed again, the operation steps of the above embodiments may be omitted.
In some embodiments, video image data is acquired, the number of anchor points is identified, and positioning data for the vehicle-mounted anchor points is acquired; acquiring positioning data of a first calibration position at a first calibration distance; acquiring positioning data of a second calibration position at a second calibration distance; calculating the heading of the vehicle body according to the positioning data of the first calibration position and the second calibration position; calculating an included angle between the heading of the vehicle body and a connecting line of the first calibration point and the second calibration point; triggering calibration when the included angle is minus 90 degrees less than or equal to a first threshold value.
In some embodiments, a camera or a device with an image capturing function mounted in advance on a vehicle operates to generate image data. The image data may be any one or a combination of video and picture.
In some embodiments, the image data further includes per-frame time stamp information, positioning data information of a vehicle-mounted positioning point at the shooting time.
In some embodiments, the image data is processed, the number of calibration points is identified, and positioning data of the vehicle-mounted positioning points is obtained. The identification herein refers to the use of image processing techniques to resolve the index points in the image data.
In some embodiments, when all the calibration points are identified, the acquisition of the positioning data of the vehicle-mounted positioning point is started, so that the data volume can be reduced.
In some embodiments, the positioning data of the acquired in-vehicle positioning points is RTK positioning data. RTK is a real-time kinematic measurement technique, and is one of relative positioning techniques, and high-precision dynamic relative positioning is realized mainly through a real-time data link between a reference station and a mobile station and a carrier relative positioning rapid calculation technique.
In some embodiments, the vehicle advances, the calibration point enters the working radius of the camera, the image data is obtained through shooting, and the data is processed; and identifying all the calibration points from the image data, and starting to acquire the positioning data of the vehicle-mounted positioning points and the positioning data of the calibration points. And calculating the distance from the vehicle-mounted positioning point to any positioning point according to the positioning data of the vehicle-mounted positioning point and the positioning data of the positioning point. The vehicle continues to advance and the calibration point enters the camera calibration working radius. And identifying all calibration points at the first calibration distance, and acquiring and recording positioning data of the first calibration position at the moment. And identifying all calibration points at the second calibration distance, and acquiring and recording positioning data of the second calibration position at the moment. The first calibration distance is smaller than or equal to the maximum calibration working radius, and the second calibration distance is larger than or equal to the minimum calibration working radius. The camera working radius refers to the maximum working distance that the camera can acquire image data satisfying the image processing work requirements. The calibration working radius refers to the working distance from which the camera can acquire image data satisfying the calibration working requirements.
In some embodiments, the maximum nominal working radius takes on a value of 8m and the minimum nominal working radius takes on a value of 3m.
In some embodiments, the positioning data of the calibration points may be acquired from the outside in advance and manually input into the automatic calibration control system; the input can also be sent by a locating device of the calibration point, and the automatic calibration control system receives the input.
In some embodiments, the body heading is calculated from positioning data of the first calibration location and the second calibration location. The body heading herein refers to characterizing a spatial vector indicative of the direction of travel of the vehicle.
In some embodiments, the RTK positioning data of the O point at the first calibration position is obtained through the vehicle-mounted positioning point, so as to obtain accurate world coordinates (x 1 ，y 1 ，z 1 ) The method comprises the steps of carrying out a first treatment on the surface of the RTK positioning data of the second calibration position O 'point is obtained through the vehicle-mounted positioning point, and accurate O' point world coordinates (x) 2 ，y 2 ，z 2 ). World coordinate system is subtracted to obtain vector
In some embodiments, the angle of the body heading to the line connecting the first calibration point and the second calibration point is calculated.
In some embodiments, a first index point A is obtained 1 And a second calibration point A 2 World coordinates of (2) to obtain a vectorPass vector->Sum vector->And calculating the included angle between the heading of the vehicle body and the connecting line of the first calibration point and the second calibration point. Wherein the first index point A 1 And a second calibration point A 2 Is a symmetrical calibration point respectively positioned at two sides of the center line of the pavement.
In some embodiments, when the first threshold is detected that the included angle is-90 degrees, a calibration working instruction is initiated to the calibration device to trigger calibration.
In some embodiments, a first index point A is obtained 1 And a third calibration point A 3 World coordinates of (2) to obtain a vectorPass vector->Sum vector->And calculating the included angle between the heading of the vehicle body and the connecting line of the first calibration point and the third calibration point. Wherein the first index point A 1 And a third calibration point A 3 Is two marked points respectively positioned on the same side of the road surface, and a straight line A 1 A 3 Parallel to the road centerline. At this time, calibration is triggered by setting the included angle to be less than or equal to a first threshold value.
In some embodiments, the first threshold is set to 5 °.
In some embodiments, calibration parameters are calculated after calibration is triggered, including but not limited to camera intrinsic and pose parameters. Then, according to the calibration parameters, any one calibration point is identified by the calibration equipment, and the theoretical world coordinates of the calibration point are calculated; and when the absolute value of the error between the theoretical world coordinate and the actual world coordinate of the calibration point is smaller than a second threshold value, storing the calibration parameters, and terminating the calibration.
The theoretical world coordinates of the calibration points refer to the actual world coordinates of the calibration points which can be obtained from the outside in advance and manually input into an automatic calibration control system; the automatic calibration control system can also receive and input through the positioning device of the calibration point. The absolute value of the error between the theoretical world coordinate and the actual world coordinate of the calibration point is calculated, and the theoretical world coordinate (x Management device ，y Management device ，z Management device ) An actual coordinate system (x Real world ，y Real world ，z Real world ) Then there is an absolute value of error |x of the corresponding coordinate axis Management device -x Real world |、|y Management device -y Real world |and |z Management device -z Real world | a. The invention relates to a method for producing a fibre-reinforced plastic composite. When, |x Management device -x Real world |、|y Management device -y Real world |and |z Management device -z Real world And when the I is smaller than the second threshold value, storing the calibration parameters, and finishing and terminating the calibration.
In some embodiments, the second threshold is set at 20cm.
In some embodiments, the in-vehicle setpoint RTK is to be 0 The RTK is arranged at the top of the vehicle body, and can acquire world coordinates of positioning points in real time. On both sides of road 5-10 m in front of car, RTKs are respectively used to set 6 points RTKs 1 、RTK 2 、RTK 3 、RTK 4 、RTK 5 、RTK 6 World coordinates A of (2) w1 ，A w2 ，A w3 ，A w4 ，A w5 ，A w6 Respectively correspond to the marked points A 1 ，A 2 ，A 3 ，A 4 ，A 5 ，A 6 。RTK 1 、RTK 3 、RTK 5 Connection of points and RTK 2 、RTK 4 、RTK 6 The lines of the points are respectively parallel to the central line of the road surface. The binocular camera is fixed on the front windshield, and the 6-point position is ensured to be within the visual field of the camera. Starting the vehicle to acquire coordinates of positioning points of the vehicle body, driving forwards, and forming two adjacent RTKs 0 The point O is determined as the car body heading, and the left eye image is grabbed. The O point is used as a coordinate origin, the car body heading O' O is used as a Y axis, the vertical ground direction is used as a Z axis, the vertical car body heading direction is used as an X axis, and a right-hand rectangular coordinate system OXYZ, namely a car body coordinate system is established.
Setting a vectorVector->The included angle of (a) is alpha, vector->Vector->The cross multiplication has:
then: a is that 1 Point-to-vectorIs>The method comprises the following steps:
point C of crossing 1 Perpendicular to the ground plane, the foot drop is B 1 Then |B 1 C 1 I is the Z coordinate minus A of the O point in the world coordinate system 1 The Z coordinate value of a point in world coordinate system, so there is:
A 1 the coordinates in the vehicle body coordinate system with O as the origin of coordinates are:
similarly can be found A 2 ，A 3 ，A 4 ，A 5 ，A 6 Coordinates in a vehicle body coordinate system. Identify A 1 ，A 2 ，A 3 ，A 4 ，A 5 ，A 6 Corresponding image coordinates A in left eye image 1 ′，A 2 ′，A 3 ′，A 4 ′，A 5 ′，A 6 ′，
Next, a transformation matrix M of the pixel coordinates to the vehicle body coordinate system coordinates transformation is calculated.
Transformation matrixMatrix parameter f x ，f y ，c x ，c y Is an in-camera parameter, R is a rotation matrix, and T is a three-dimensional translation vector. m represents parameter values at different positions in the matrix, and subscripts are row numbers and column numbers respectively.
Specifically, the transformation matrix is a=mb; wherein A is a coordinate point of a vehicle body coordinate system, and B is a pixel coordinate point.
Since the index points are all located on the same plane, the z coordinate value is constant. According to the calibration principle, the monocular calibration parameters have an internal parameter f x ，f y ，c x ，c y The external parameters comprise a rotation matrix R and a translation matrix T, and the total number of the external parameters is 10. Where T is the translation vector contains three parameters (translation in x, y, z directions). Four calibration points under the vehicle body coordinate system can list 12 equations, and the internal and external parameters are solved. Since the projective transformation matrix from the three-dimensional coordinate point in the vehicle body coordinate system to the two-dimensional point in the pixel coordinate system is a 3×4 irreversible matrix, and since z is a constant, the redundant two parameters are eliminated, the transformation matrix can be rewritten into a 3×3 reversible matrix. That is, the coordinate point in the pixel coordinate system may be obtained from the coordinate point in the vehicle body coordinate system, or the coordinate point on the road surface where z in the vehicle body coordinate system is constant may be obtained from the coordinate point in the pixel coordinate system.
Wherein, the 3×4 irreversible matrix is:
because the road surface z under the vehicle body coordinate system is constant, eliminating the third row to obtain a transformation matrix M
Then angle is carried outCalculating the degree, and setting the coordinates of the O point and the O' point in the world coordinate system as (x) 1 ，y 1 ，z 1 )，(x 2 ，y 2 ，z 2 ) An included angle between the clockwise direction and the north direction of the earth is alpha, and then:
i.e. angle with y-axis
①x 2 ＞x 1 ，y 2 ＞y 1 ：α＝2π-θ
②x 2 ＜x 1 ，y 2 ＞y 1 ：α＝θ
③x 2 ＞x 1 ，y 2 ＜y 1 ：α＝π+θ
④x 2 ＜x 1 ，y 2 ＜y 1 ：α＝π-θ
Let the coordinates of the p point in the vehicle body coordinate system be (x c ，y c ，z c ) Roof RTK 0 The coordinates are (x) o ，y o ，z o )
The coordinates of the p point in the world coordinate system are:
x＝x c cosα-y c sinα+x o
y＝x c sinα+y c cosα+y o
z＝z c +z o
from this, the theoretical world coordinates of the calibration point can be calculated.
In some embodiments, a three-dimensional volume marker monochrome square is conventionally placed next to the marker point. Conventional placement herein refers to the passage of any one side of a monochromatic square of a three-dimensional volumetric marker past the marker point. Generally, in the conventional technical scheme, a certain right-angle vertex of a single-color square of a three-dimensional calibration object is strictly required to be coincident with a calibration point.
In some embodiments, the midpoint of any one side of the monochrome square of the three-dimensional volume marker coincides with the midpoint of the other side.
In some embodiments, the three-dimensional volume marker monochrome square is parallel to the optical axis centerline of the camera. This has the advantage of facilitating subsequent data processing.
In some embodiments, a rectangular bar is provided as a pre-set marker, where the pre-set marker is placed according to the following rules: the midpoint of any side of the rectangular strip coincides with the calibration point and is externally connected with the calibrated side of the single-color square of the three-dimensional calibration object; the two side edges of the rectangular strip do not exceed the monochromatic small square of the three-dimensional calibration object. The rectangular strip can be made of common monochromatic paper, the length-width ratio is arbitrary, and the length-width size is smaller than the side length of the monochromatic square of the three-dimensional calibration object. In particular, the color of the rectangular bar may be selected according to the external environment identified by the calibration point, and black may be generally selected. The rectangular strip can be fixed beside the calibration point by glue or the like.
It will be appreciated that the setting of the predetermined tag may be one-time, i.e. the first setting, and the rectangular strip may be retained subsequently without resetting; when the camera calibration is performed again, the step of setting the rectangular strips can be omitted.
In some embodiments, the camera is operative to capture video or pictures of the front of the working radius range, and to acquire image data.
In some embodiments, processing the image data to generate an circumscribed rectangle of the calibration object in the image; the lower edge of the circumscribed rectangle extends downwards for a preset pixel coordinate length to generate a region of interest; traversing the region of interest to generate a mark region of a preset marker related to the marker; traversing the marked area to generate a central line, wherein the pixel coordinates of the upper end point of the central line are the pixel coordinates of the marked point.
In some embodiments, the image data is processed by a deep-learning semantic segmentation method. In general, the following steps may be taken: collecting video data containing identification targets; converting the video data into picture data; labeling the picture by using a labeling tool to generate sample data; training by using sample data to generate a network model; and calling the model to identify the target.
In some embodiments, the modeling method of the deep learning neural network model is: firstly, training a pretraining model obtained by VGG16 training, and outputting a trained FCN-32s model; training by taking the FCN-32s model as a pre-training model and using a new sample, and outputting a trained FCN-16s model; and training by taking the FCN-16s model as a pre-training model and using a new sample, and outputting the trained FCN-8s model. And training by taking the FCN-8s model as a pre-training model and a new sample, and outputting the trained FCN-4s model as a target model. Here, FCN is a full convolution of the neural network (Fully Convolutional Networks). VGG is Visual Geometry Group. It should be noted that the required model obtained by training the rest deep neural networks such as *** net can also be adopted.
In some embodiments, processing the image data findContours to obtain all circumscribed quadrilaterals outside the correspondence of the three-dimensional calibration object small square image; and outputting a circumscribed rectangle with the smallest area.
In some embodiments, in the pixel coordinates, the lower edge of the output circumscribed rectangle extends by a preset pixel coordinate length, and the region of interest is generated. Generally, the preset pixel coordinate length value herein mainly takes the pixel error size when the circumscribed rectangle corresponding to the three-dimensional calibration object small square image is obtained as a consideration factor. The pixel error refers to the pixel difference between the theoretical value and the actual value of the three-dimensional calibration object small square image.
In some embodiments, a range of pixel coordinate lengths is preset: the coordinate length of the preset pixels is less than or equal to 10 pixels and less than or equal to 20 pixels.
In some embodiments, the pixel coordinate length is 15 pixels, i.e., the lower edge of the bounding rectangle extends downward 15 unit pixel coordinate lengths.
In some embodiments, the calibration-related predetermined marker is a rectangular bar; the upper edge of the rectangular strip is connected with the lower edge of the calibration object, the midpoint of the upper edge of the rectangular strip coincides with the calibration point, and the side length of the rectangular strip is smaller than the side length of the calibration object, so that the left side and the right side of the image of the rectangular strip are ensured not to exceed the left side and the right side of the image of the calibration object in the image.
In some embodiments, the three-dimensional volume object is parallel to the optical axis of the camera. Traversing the region of interest to generate a marked region of a preset marker related to the marker.
In some embodiments, the method of generating a marker region of a predetermined marker associated with a marker comprises: setting I (x, y) as any pixel point in the region of interest, wherein I (x-delta, y) and I (x+delta, y) are two symmetrical pixel points of I (x, y) along the y axis, and delta is the pixel coordinate length of a preset marker in an image;
and is provided with a plurality of groups,
d 1 ＝I(x，y)-I(x-δ，y)
d 2 ＝I(x，y)-I(x+δ，y)
wherein d 1 ，d 2 The pixel difference value between any pixel point and the corresponding symmetrical pixel point;
D＝d 1 +d 2 -|I(x+δ，y)-I(x-δ，y)|
wherein D represents the sum of pixel difference values of any pixel point and corresponding symmetrical pixel points, and the pixel difference values of the two symmetrical pixel points are subtracted to represent the pixel difference values of any pixel point and the symmetrical pixel points;
let L (x, y) be the pixel value binarization function of the pixel point, when d is satisfied 1 ＞0，d 2 When > 0 and D > L, L (x, y) is 255, and when the above condition is not satisfied, L (x, y) is 0;
that is to say,
wherein, the threshold value l=α×i (x, y) of the binarization function, and α is a threshold coefficient.
In some embodiments, the threshold coefficient α has a value in the range of 0.3.ltoreq.α.ltoreq.0.8.
In another aspect, in some embodiments, a storage medium is provided. The storage medium stores computer program instructions that, when executed by the processor, repeatedly perform at least one of the following steps: acquiring video image data, identifying the number of standard points, and acquiring positioning data of vehicle-mounted positioning points; acquiring positioning data of a first calibration position at a first calibration distance; acquiring positioning data of a second calibration position at a second calibration distance; calculating the heading of the vehicle body according to the positioning data of the first calibration position and the second calibration position; calculating an included angle between the heading of the vehicle body and a connecting line of the first calibration point and the second calibration point; and when the included angle is minus 90 degrees less than or equal to a first threshold value, a calibration instruction is sent, and calibration is triggered.
In some embodiments, the storage medium stores computer program instructions that, when executed by the processor, repeatedly perform at least one of the following steps: any one of the calibration points is identified, and the theoretical world coordinates of the calibration points are calculated; and when the absolute value of the error between the theoretical world coordinate and the actual world coordinate of the calibration point is smaller than a second threshold value, storing the calibration parameters, and terminating the calibration.
In another aspect, in some embodiments, a control system for automatic calibration is provided, including a calibration point identification module, a vehicle body heading determination module, a trigger module, and a verification module. The calibration point identification module is used for acquiring and processing video image data and identifying calibration points; the vehicle body course judging module is used for acquiring and processing positioning data of the vehicle-mounted positioning points and judging the position relation between the vehicle body course and the positioning points; the triggering module is used for sending a calibration instruction and starting calibration; the calibration module is used for calibrating calibration results.
The various embodiments or features mentioned herein may be combined with one another as yet further alternative embodiments, which are not listed one by one and which are formed by a limited number of combinations of features, without departing from the scope of the present disclosure, as would be understood or inferred by those skilled in the art in light of the accompanying drawings and the foregoing, insofar as they are within the knowledge and ability of those skilled in the art.
Finally, it is emphasized that the above-described embodiments, which are typical and preferred embodiments of the present invention, are merely used to describe and explain the technical solutions of the present invention in detail, so that the reader can easily understand the present invention and are not intended to limit the scope or application of the present invention.
It is therefore intended to cover in the appended claims any such departures from the present disclosure as come within known or customary practice in the art to which this invention pertains.
Claims (10)
1. An automatic calibration control method based on vehicle-mounted is characterized by comprising the following steps of: the method comprises the following steps:
acquiring image data, identifying the number of standard points, and acquiring positioning data of vehicle-mounted positioning points; the standard points are provided with positioning information and are symmetrically arranged on two sides of the road surface, and the connecting lines of any two standard points on each side are parallel to the central line of the road surface;
acquiring positioning data of a first calibration position at a first calibration distance; the positioning data of the first calibration position are the positioning data of the first calibration position obtained through the vehicle-mounted positioning point;
acquiring positioning data of a second calibration position at a second calibration distance; the positioning data of the second calibration position are the positioning data of the second calibration position obtained through the vehicle-mounted positioning point;
calculating the heading of the vehicle body according to the positioning data of the first calibration position and the second calibration position;
calculating an included angle between the heading of the vehicle body and a connecting line of the first calibration point and the second calibration point; the vehicle body heading refers to a space vector representing a forward direction of the indicating vehicle; the first calibration point and the second calibration point are symmetrical calibration points respectively positioned at two sides of the central line of the pavement;
triggering calibration when the included angle is minus 90 degrees less than or equal to a first threshold value.
2. The method according to claim 1, characterized in that: the positioning data of the vehicle-mounted positioning points are acquired after all the positioning points are identified.
3. The method according to claim 2, characterized in that: the positioning data is RTK positioning data.
4. The method according to claim 1, characterized in that: the first calibration distance is not more than the maximum calibration working radius, and the second calibration distance is not less than the minimum calibration working radius.
5. The method according to claim 4, wherein: the maximum calibration working radius is 8m, and the minimum calibration working radius is 3m.
6. The method according to claim 1, characterized in that: the first threshold is 5 °.
7. The method according to claim 1, wherein: the method also comprises the following steps:
any one of the calibration points is identified, and the theoretical world coordinates of the calibration points are calculated;
and when the absolute value of the error between the theoretical world coordinate and the actual world coordinate of the calibration point is smaller than a second threshold value, storing the calibration parameters, and terminating the calibration.
8. The method according to claim 7, wherein: the second threshold is 20cm.
9. A readable storage medium, characterized in that it has stored thereon a computer program that performs the method according to any of claims 1 to 8.
10. The automatic calibration control system is characterized in that:
the system comprises a calibration point identification module, a car body course judgment module, a trigger module and a verification module;
the calibration point identification module is used for acquiring and processing video image data and identifying calibration points; the standard points are provided with positioning information and are symmetrically arranged on two sides of the road surface, and the connecting lines of any two standard points on each side are parallel to the central line of the road surface;
the vehicle body course judging module is used for acquiring and processing positioning data of the vehicle-mounted positioning points and judging the position relation between the vehicle body course and the positioning points; acquiring positioning data of a first calibration position at a first calibration distance; the positioning data of the first calibration position are the positioning data of the first calibration position obtained through the vehicle-mounted positioning point; acquiring positioning data of a second calibration position at a second calibration distance; the positioning data of the second calibration position are the positioning data of the second calibration position obtained through the vehicle-mounted positioning point;
calculating the heading of the vehicle body according to the positioning data of the first calibration position and the second calibration position; calculating an included angle between the heading of the vehicle body and a connecting line of the first calibration point and the second calibration point; the vehicle body heading refers to a space vector representing a forward direction of the indicating vehicle; the first calibration point and the second calibration point are symmetrical calibration points respectively positioned at two sides of the central line of the pavement;
the triggering module is used for sending a calibration instruction and starting calibration when the included angle is minus 90 degrees less than or equal to a first threshold value;
the calibration module is used for calibrating calibration results.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910817490.7A CN111145249B (en) | 2019-08-30 | 2019-08-30 | Vehicle-mounted-based automatic calibration control method, storage medium and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910817490.7A CN111145249B (en) | 2019-08-30 | 2019-08-30 | Vehicle-mounted-based automatic calibration control method, storage medium and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111145249A CN111145249A (en) | 2020-05-12 |
| CN111145249B true CN111145249B (en) | 2024-02-27 |
Family
ID=70516783
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910817490.7A Active CN111145249B (en) | 2019-08-30 | 2019-08-30 | Vehicle-mounted-based automatic calibration control method, storage medium and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111145249B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102037735A (en) * | 2008-05-22 | 2011-04-27 | 通用汽车环球科技运作公司 | Self calibration of extrinsic camera parameters for a vehicle camera |
| CN103927754A (en) * | 2014-04-21 | 2014-07-16 | 大连理工大学 | Calibration method of on-board camera |
| CN106558080A (en) * | 2016-11-14 | 2017-04-05 | 天津津航技术物理研究所 | Join on-line proving system and method outside a kind of monocular camera |
| CN108805934A (en) * | 2017-04-28 | 2018-11-13 | 华为技术有限公司 | A kind of method for calibrating external parameters and device of vehicle-mounted vidicon |
| CN108898638A (en) * | 2018-06-27 | 2018-11-27 | 江苏大学 | A kind of on-line automatic scaling method of vehicle-mounted camera |
| CN109188438A (en) * | 2018-09-12 | 2019-01-11 | 百度在线网络技术（北京）有限公司 | Yaw angle determines method, apparatus, equipment and medium |
| CN110148177A (en) * | 2018-02-11 | 2019-08-20 | 百度在线网络技术（北京）有限公司 | For determining the method, apparatus of the attitude angle of camera, calculating equipment, computer readable storage medium and acquisition entity |
-
2019
- 2019-08-30 CN CN201910817490.7A patent/CN111145249B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102037735A (en) * | 2008-05-22 | 2011-04-27 | 通用汽车环球科技运作公司 | Self calibration of extrinsic camera parameters for a vehicle camera |
| CN103927754A (en) * | 2014-04-21 | 2014-07-16 | 大连理工大学 | Calibration method of on-board camera |
| CN106558080A (en) * | 2016-11-14 | 2017-04-05 | 天津津航技术物理研究所 | Join on-line proving system and method outside a kind of monocular camera |
| CN108805934A (en) * | 2017-04-28 | 2018-11-13 | 华为技术有限公司 | A kind of method for calibrating external parameters and device of vehicle-mounted vidicon |
| CN110148177A (en) * | 2018-02-11 | 2019-08-20 | 百度在线网络技术（北京）有限公司 | For determining the method, apparatus of the attitude angle of camera, calculating equipment, computer readable storage medium and acquisition entity |
| CN108898638A (en) * | 2018-06-27 | 2018-11-27 | 江苏大学 | A kind of on-line automatic scaling method of vehicle-mounted camera |
| CN109188438A (en) * | 2018-09-12 | 2019-01-11 | 百度在线网络技术（北京）有限公司 | Yaw angle determines method, apparatus, equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111145249A (en) | 2020-05-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110969663B (en) | Static calibration method for external parameters of camera | |
| CN113657224B (en) | Method, device and equipment for determining object state in vehicle-road coordination | |
| CN110517325B (en) | Coordinate transformation and method and system for positioning objects around vehicle body through coordinate transformation | |
| CN111145260B (en) | Vehicle-mounted-based double-target setting method | |
| CN111145262B (en) | Vehicle-mounted-based monocular calibration method | |
| CN109690622A (en) | Camera registration in multicamera system | |
| CN111462249B (en) | Traffic camera calibration method and device | |
| CN111260539B (en) | Fish eye pattern target identification method and system thereof | |
| CN109741241B (en) | Fisheye image processing method, device, equipment and storage medium | |
| CN109146958B (en) | Traffic sign space position measuring method based on two-dimensional image | |
| CN112232275A (en) | Obstacle detection method, system, equipment and storage medium based on binocular recognition | |
| CN105809706A (en) | Global calibration method of distributed multi-camera system | |
| CN110750153A (en) | Dynamic virtualization device of unmanned vehicle | |
| CN114413958A (en) | Monocular vision distance and speed measurement method of unmanned logistics vehicle | |
| CN112446915A (en) | Picture-establishing method and device based on image group | |
| CN116433865B (en) | Space-ground collaborative acquisition path planning method based on scene reconstructability analysis | |
| CN111191596B (en) | Closed area drawing method, device and storage medium | |
| CN114140533A (en) | Method and device for calibrating external parameters of camera | |
| CN111145249B (en) | Vehicle-mounted-based automatic calibration control method, storage medium and system | |
| CN111145261A (en) | Method for identifying index point and storage medium | |
| CN111243021A (en) | Vehicle-mounted visual positioning method and system based on multiple combined cameras and storage medium | |
| CN113902666B (en) | Vehicle-mounted multiband stereoscopic vision sensing method, device, equipment and medium | |
| CN115100290A (en) | Monocular vision positioning method, monocular vision positioning device, monocular vision positioning equipment and monocular vision positioning storage medium in traffic scene | |
| CN110503683B (en) | Narrow parking space reversing and warehousing guiding method and system based on monocular ranging | |
| CN114170323A (en) | Parameter calibration method and equipment for image shooting device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |