CN112489141A - Production line calibration method and device for single board single-image relay lens of vehicle-mounted camera - Google Patents
Production line calibration method and device for single board single-image relay lens of vehicle-mounted camera Download PDFInfo
- Publication number
- CN112489141A CN112489141A CN202011521340.0A CN202011521340A CN112489141A CN 112489141 A CN112489141 A CN 112489141A CN 202011521340 A CN202011521340 A CN 202011521340A CN 112489141 A CN112489141 A CN 112489141A
- Authority
- CN
- China
- Prior art keywords
- calibration
- vehicle
- mounted camera
- relay lens
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000004519 manufacturing process Methods 0.000 title claims abstract description 39
- 239000007787 solid Substances 0.000 claims abstract description 88
- 230000003287 optical effect Effects 0.000 claims description 24
- 239000011159 matrix material Substances 0.000 claims description 23
- 238000005457 optimization Methods 0.000 claims description 17
- 238000012545 processing Methods 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 11
- 238000000605 extraction Methods 0.000 claims description 10
- 238000004422 calculation algorithm Methods 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 4
- 239000000284 extract Substances 0.000 claims description 2
- 238000003384 imaging method Methods 0.000 description 18
- 239000013598 vector Substances 0.000 description 18
- 238000001514 detection method Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000013519 translation Methods 0.000 description 5
- 238000013461 design Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000004088 simulation Methods 0.000 description 3
- 238000012897 Levenberg–Marquardt algorithm Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000012634 optical imaging Methods 0.000 description 2
- 239000000243 solution Substances 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 239000012482 calibration solution Substances 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G06T2207/20032—Median filtering
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Studio Devices (AREA)
Abstract
The invention discloses a method and a device for calibrating a production line of a single board single-image relay lens of a vehicle-mounted camera, wherein the method comprises the following steps: s1, acquiring a virtual image of a calibration template formed by a relay lens through a vehicle-mounted camera as a calibration template image; s2, extracting the pixel position coordinates of the center of the solid circle in the calibration template image; s3, calculating an initial value of a calibration parameter of the vehicle-mounted camera; and S4, optimizing the initial values of the calibration parameters to obtain accurate calibration parameters of the vehicle-mounted camera. The device comprises a relay lens, a calibration template and a vehicle-mounted camera to be calibrated, wherein the relay lens is arranged between a lens of the vehicle-mounted camera and the calibration template. The invention saves the calibration time of the production line, improves the yield in unit time, and can simulate different focusing distances, thereby realizing accurate and efficient calibration on the production lines of different types of cameras and having higher compatibility.
Description
Technical Field
The invention relates to the field of camera calibration methods, in particular to a production line calibration method and device for a single-board single-image relay lens of a vehicle-mounted camera.
Background
With the popularization of driving assistance and automatic driving techniques, automobiles are increasingly using cameras, for example, ranging, 3D environmental perception, and the like are accomplished through binocular cameras. The calibration of the camera is the key for realizing the 3D application of the vehicle-mounted camera. In camera-based measurement and vision applications, the relationship between the three-dimensional position of an object point in space and its corresponding two-dimensional pixel point position in an image is described mathematically by a geometric projection model. The parameters of the model are generally obtained by photographing a calibration pattern (such as a solid circle array or a black and white checkerboard) with known dimensions, processing the image and calculating, and the process of determining the parameters of the camera projection model is called calibration. These parameters include: internal parameter, which refers to the principal point position c when the camera is imagingx、cyAnd focal length f of lensx、fySuch parameters are only relevant to the camera itself; the external parameters refer to the position of the camera in space, and generally refer to a rotation vector R and a translation vector T of the camera in a certain reference coordinate system; distortion refers to the deviation between the actual corresponding pixel position of an object point in an image and a theoretical projection point calculated based on an imaging model in the shooting process of a camera, and the deviation is generally determined by a radial distortion parameter k1、k2、k3And tangential distortion parameter p1、p2It is mainly caused by the design, manufacturing and assembly errors of the lens and camera module.
The calibration solution commonly used in current cameras has two limitations. Firstly, because the used calibration algorithm requires to shoot images of a plurality of calibration templates from different angles, a 2D plane calibration template is rotated or a calibration three-dimensional template spliced by a plurality of plane calibration templates (fixed angles are formed between the plane calibration templates) is used on the current production line. The device for rotating the calibration template is complex in mechanism, and the rotation lengthens the measurement time, reduces the yield per unit time and increases the cost. The three-dimensional template spliced by the plurality of plane calibration templates is increased in cost, or needs to be accurately fixed according to a certain angle, or needs to be accurately measured before use.
The prior patent CN 209640928U (shenzhen, prompture technologies ltd., filing date 2019.03.31) introduces a calibration device on a camera production line, which includes: the calibration device comprises a support frame, a mechanical arm and a calibration template fixed on the mechanical arm. The device finishes camera calibration by controlling the mechanical arm to move the calibration template to a plurality of positions and analyzing and processing images shot at each calibration position. The position of the calibration template moved by the mechanical arm needs to be adjusted for multiple times to perform multi-image calibration, the time consumption is long, the operation is complex, and the accurate calibration of a camera with a long focal distance cannot be performed.
Patent CN 110490940 a (beijing migwey science and technology limited, filing date 2019.08.15) introduces a camera calibration method and device based on checkerboard single image, which can realize fast calibration of camera parameters. The method mainly comprises the steps of obtaining internal parameters of a camera through an absolute quadratic curve equation by utilizing pixel coordinates of two vanishing points corresponding to two groups of parallel lines acquired on a checkerboard image. The camera with a long focal distance cannot be accurately calibrated.
Patent CN 207123866U (guyton-a-gabby-technologies, llc, filing date 2017.07.26) describes a calibration system based on single-frame image calibration, which can complete camera calibration based on one calibration image. The calibration template used by the patent consists of a square array with fixed intervals and at least eight spherical calibration parts positioned outside four corners of an array image, the structure is complex, and the patent cannot accurately calibrate a camera with a longer focal distance.
Patent CN 106570907B (haixin corporation, filing date 2016.11.22) describes a camera calibration method and apparatus. The method solves the rotation angles of the camera along two axis directions of a world coordinate system respectively through a simple linear relation and then solves other parameters of the camera. The patent does not relate to the calibration of distortion parameters, only the focal length and the external parameters of the camera are calibrated, and the patent cannot accurately calibrate the camera with a longer focal length.
Patent CN 110033491 a (south kyo institute of engineering, filing date 2019.04.15) describes a camera calibration method. The method comprises the steps of constructing a multi-dimensional vector based on camera internal parameters and lens distortion parameters, constructing a new multi-dimensional vector after carrying out distortion removal processing on a calibrated image by using the lens distortion parameters, and circularly iterating until the Euclidean distance between two adjacent multi-dimensional vectors is smaller than a set value to obtain the internal and external parameters of the camera. The method takes the Euclidean distance of a multi-dimensional vector formed by two adjacent internal parameters and external parameters as a cost function, and conducts iterative optimization on numerical values, so that the camera with a longer focusing distance cannot be accurately calibrated.
The acquisition of the coordinate position of the feature point of the calibration pattern (such as the center of a solid circle) in the captured calibration image is a precondition and a basis for performing camera calibration. The imaging definition can directly influence the accuracy of the extraction of the position of the characteristic point and indirectly influence the calibration precision of the camera. Therefore, the camera needs to take a picture in focus at the target distance, so as to obtain the accurate parameter value of the camera based on the clear calibration template image. For the vehicle-mounted camera, the focusing distance of the vehicle-mounted camera is generally several meters, dozens of meters and hundred meters, and the size of the calibration template is in direct proportion to the focusing distance, so that the size of the required calibration equipment is too large, and the cost of the vehicle-mounted camera production line in a clean room is high or even cannot be realized. Therefore, a method and a calibration device capable of completing calibration of the vehicle-mounted camera in a short distance are needed in the production line.
Disclosure of Invention
The invention aims to provide a method and a device for calibrating a production line of a single board with a relay lens of a vehicle-mounted camera, which aim to solve the problem that the prior art can not accurately and efficiently calibrate the vehicle-mounted camera on the production line.
In order to achieve the purpose, the technical scheme adopted by the invention is as follows:
the production line calibration method of the single board single-image relay lens of the vehicle-mounted camera comprises the following steps of constructing a calibration system comprising a calibration template, the relay lens and the vehicle-mounted camera to be calibrated, adjusting the distance between the relay lens and the calibration template to be suitable for the vehicle-mounted cameras with different focal lengths, wherein the calibration template is provided with a solid circle array, and the vehicle-mounted camera shoots the calibration template through the relay lens, and the specific process comprises the following steps:
s1, acquiring a virtual image of a calibration template formed by a relay lens through the acquisition of a vehicle-mounted camera as a calibration template image;
s2, extracting the pixel position coordinates of the center of the solid circle in the calibration template image by using an image processing algorithm;
s3, based on the calibration template image, combining the homography matrix and the center position coordinates of the solid circle, calculating the initial values of the calibration parameters of the vehicle-mounted camera, including the internal and external parameters and the distortion parameters;
and S4, optimizing the initial values of the internal and external parameters and the distortion parameters of the vehicle-mounted camera obtained in the step S3 to obtain accurate calibration parameters of the vehicle-mounted camera.
Firstly, shooting a calibration template image in a form of a virtual image at a long distance, which is formed by simulating a calibration template through a relay lens; then analyzing and processing the single calibration template image corresponding to the simulated calibration distance, and calculating to obtain the corresponding camera calibration parameters.
The calibration pattern on the surface of the calibration template is a solid circle array, the solid circle array consists of C rows multiplied by L columns of solid circles, and C and L are natural numbers; each solid circle has the same size, the same radius and the same horizontal and vertical center distance. The image of the calibration template shot by the relay lens is consistent with the original pattern and also consists of C multiplied by L solid circles; the circle radius and the circle center distance of the simulated remote calibration template image are related to the distance from the relay lens to the calibration template. When a calibration template image is shot, the optical axis of the vehicle-mounted camera and the optical axis of the relay lens are parallel and concentric as much as possible, and the optical axis of the vehicle-mounted camera and the optical axis of the relay lens are perpendicular to the calibration template, so that the calibration template image is focused and pattern deformation caused by a shooting angle does not exist.
In the invention, the process of analyzing and processing the single calibration template image corresponding to the calibration distance comprises the following steps:
1. extracting the pixel position of the center of the solid circle in the shot calibration template image through an image processing algorithm; the center of the solid circle of the calibration template image can be extracted through a binarization and median filtering method to obtain the pixel position coordinates of the center of the solid circle in the calibration template image, or the center of mass of the circle region is obtained as the coordinates of the center of the circle by detecting the solid circle region based on the gray value of the image.
2. Based on the calibration template image, according to the homography matrix of the calibration template image, the initial values of the internal parameters, the external parameters and the distortion parameters of the camera are calculated and obtained by combining the principal points of the calibration template image and the pixel positions of the centers of the solid circles. The position of the central coordinate of the image can be preset as a main point cxAnd cyAccording to the main point and the circle center position of the solid circle, the focal length f is solved based on the homography matrix of a single calibration template imagexAnd fyIs started. The initial value of the spatial position T of the vehicle-mounted camera in the calibration template coordinate system can be estimated according to the position of the camera relative to the calibration template and the type of the vehicle-mounted camera. As the optical axis of the vehicle-mounted camera is approximately vertical to the calibration template, the initial value of the rotation matrix R is set as an identity matrix. Radial and tangential distortion parameters [ k ] of camera lens in general1 k2 k3]And [ p ]1 p2]Is set to zero.
3. Based on the principle that the quadratic sum of the reprojection errors of the circle centers of the solid circles of the calibration template images is the minimum, the Levenberg-Marquardt optimization algorithm is utilized to optimize the internal parameters, the external parameters and the distortion parameters of the camera, so that accurate camera calibration parameters are obtained.
The production line calibration device comprises a relay lens, a calibration template and a vehicle-mounted camera to be calibrated, wherein the relay lens is arranged between a lens of the vehicle-mounted camera and the calibration template, the optical axes of the relay lens and the vehicle-mounted camera are arranged in a superposition mode, and the optical axes of the relay lens and the vehicle-mounted camera are perpendicular to the surface of the calibration template. The distance between the relay lens and the calibration template is adjustable, the pattern of the calibration template is a solid circle array, the solid circle array is a row-column array formed by a plurality of solid circles, the radius of each solid circle is the same, and the center distances of the adjacent solid circles in the horizontal direction and the vertical direction are the same.
The device also comprises an extraction unit, a calibration unit and an optimization unit which are constructed in the computer, wherein the extraction unit extracts the pixel position coordinates of the center of a solid circle in the calibration template image, the calibration unit calculates the initial values of the calibration parameters of the vehicle-mounted camera, including the internal and external parameters and the distortion parameters, and the optimization unit optimizes the internal and external parameters and the distortion parameters of the vehicle-mounted camera to obtain the accurate calibration parameters of the vehicle-mounted camera.
The invention can obtain a frame of calibration template image to finish the calibration of the vehicle-mounted camera. Based on the imaging characteristics of the relay lens, the distance from the relay lens to the calibration template is changed to simulate a long-distance virtual calibration image, so that the device is suitable for calibration of various (generally long) vehicle-mounted cameras with different focal distances. Based on the single-picture single board and the scheme with the relay lens, the method disclosed by the invention has the advantages that on one hand, the calibration time is shortened, the calibration of the vehicle-mounted camera can be quickly completed, on the other hand, the calibration equipment is small in size, occupies less space of a production line of a clean room, and has the advantages of simplicity and convenience in operation, good universality, low cost and the like.
The calibration scheme of the invention has the advantages of short time consumption and simple operation, and the relay lens is introduced between the camera and the calibration template, so that a clear focused calibration image can be obtained even in a long calibration distance, the vehicle-mounted camera with a long focal distance can be accurately calibrated, and the internal and external parameters and the distortion parameters of the vehicle-mounted camera can be calibrated. Meanwhile, the invention can also optimize the internal and external parameters, and seek the optimal solution based on the minimization of the reprojection error during optimization.
Compared with the prior art, the invention has the advantages and effects that:
1. on the production line, each camera is calibrated at one distance, only one plane calibration template is used, and the calibration template does not need to rotate. Compared with the current universal calibration method which needs a plurality of angle images, the calibration time of a production line is saved, the yield in unit time is improved, and the production cost is reduced.
2. According to the invention, the remote distance is simulated based on the relay lens, and the vehicle-mounted camera focuses on the virtual calibration template for calibration, so that accurate calibration parameters of the vehicle-mounted camera during remote focusing are obtained. The relay lens simulates a long distance, so that the calibration of the vehicle-mounted camera in a small space can be realized, the sizes of camera calibration equipment and a production line are reduced, the clean room space is saved, the production cost of the vehicle-mounted camera is reduced, and the product competitiveness is enhanced. And different focusing distances can be simulated by changing the distance between the relay lens and the physical calibration template, so that different types of cameras can be accurately calibrated, and the compatibility is high.
3. The method of the invention is also suitable for calibrating the multi-view camera, and although there are a plurality of cameras, as long as the calibration parameter of each camera is calculated based on the above principle, the calibration parameter can be regarded as the repeated times of the calibration of the single-view camera (although the simulation distance of the relay lens of different cameras may be different), and still belongs to the protection scope of the invention.
Drawings
FIG. 1 is a block diagram of a method flow of an embodiment of the present invention.
Fig. 2 is a schematic structural diagram of a calibration device according to an embodiment of the present invention.
Fig. 3 is a schematic block diagram of a complete calibration system according to an embodiment of the present invention.
FIG. 4 is a schematic diagram of a calibration template pattern according to an embodiment of the invention.
FIG. 5 is a schematic diagram of the simulated long-distance imaging relationship of the relay lens in the embodiment of the invention.
Detailed Description
The invention is further illustrated with reference to the following figures and examples.
As shown in fig. 1, the production line calibration method for a single board single-image relay lens of a vehicle-mounted camera constructs a calibration system including a calibration template, the relay lens and the vehicle-mounted camera to be calibrated, the calibration system is suitable for vehicle-mounted cameras with different focal lengths by adjusting the distance between the relay lens and the calibration template, the pattern of the calibration template is a solid circular array, the vehicle-mounted camera shoots the calibration template through the relay lens, and the specific process is as follows:
and S1, acquiring a virtual image of the calibration template formed by the relay lens through the vehicle-mounted camera as a calibration template image.
S2, extracting the pixel position coordinates of the center of the solid circle in the calibration template image by using an image processing algorithm; specifically, extracting the characteristic points of the center of the solid circle of the calibration template image by a binarization and median filtering method to obtain the pixel position coordinates of the center of the solid circle of the calibration template image; or detecting the solid circle area in the calibration template image through the image gray value, and calculating the mass center of the area to obtain the pixel position coordinate of the center of the solid circle in the calibration template image.
S3, based on the calibration template image, combining the homography matrix and the center position coordinates of the solid circle, calculating the initial values of the calibration parameters of the vehicle-mounted camera including the internal and external parameters and the distortion parameters, wherein:
and calculating to obtain an initial value of the internal parameter of the vehicle-mounted camera according to the homography matrix of the calibration template image and the pixel positions of the main point of the calibration template image and the center of the solid circle. The initial value of the external parameter rotation matrix of the vehicle-mounted camera is an identity matrix. The initial value of the distortion parameter of the vehicle-mounted camera is zero.
And S4, optimizing the initial values of the internal and external parameters and the distortion parameters of the vehicle-mounted camera obtained in the step S3 to obtain accurate calibration parameters of the vehicle-mounted camera, wherein the Levenberg-Marquardt optimization algorithm is used for optimizing the initial values of the internal and external parameters and the distortion parameters of the vehicle-mounted camera based on the principle that the quadratic sum of reprojection errors of the circle center of the solid circle of the calibration template image is minimum.
As shown in fig. 2, the production line calibration device for the single board relay lens comprises a relay lens 1, a calibration template 2 and a vehicle-mounted camera 3 to be calibrated, wherein the relay lens 1 is arranged between a lens of the vehicle-mounted camera 3 and the calibration template 2, the distance between the relay lens 1 and the calibration template 2 is adjustable, and the surface of the calibration template 2 is provided with a solid circular array.
The optical axes of the relay lens 1 and the vehicle-mounted camera 3 are overlapped, and the optical axes of the relay lens 1 and the vehicle-mounted camera 3 are both perpendicular to the calibration template 2. The solid circle array on the surface of the calibration template 2 is a row-column array formed by a plurality of solid circles, the radius of each solid circle is the same, and the distance between centers of adjacent solid circles in the horizontal direction and the vertical direction is the same.
The calibration device for the single board single image of the vehicle-mounted camera comprises a single plane 2D calibration template, a relay lens and a vehicle-mounted camera to be calibrated. The complete calibration system is realized by an optical system (including a graph obtaining unit) and an algorithm software system (including an extraction unit, a calibration unit and an optimization unit) arranged in a computer as shown in fig. 3, wherein:
the image obtaining unit is a calibration system optical system comprising a relay lens 1, a calibration template 2 and a vehicle-mounted camera 3 to be calibrated, and is used for obtaining an image of the simulated remote calibration template obtained by shooting the calibration template through the relay lens.
The extraction unit is used for detecting a solid circle in the image of the calibration template so as to extract the pixel position of the circle center;
the calibration unit is used for calculating or setting initial values of internal parameters, external parameters (rotation vectors and translation vectors) and distortion parameters of the vehicle-mounted camera and preparing for next calibration parameter optimization;
and the optimization unit is used for optimizing the internal parameters, the external parameters and the distortion parameters of the vehicle-mounted camera by using a Levenberg-Marquardt algorithm based on the minimization of the square sum of the reprojection errors of all the central points to obtain an accurate calibration result.
The concrete description is as follows:
1. picture obtaining unit
The calibration template is typically a repeating pattern with fixed spacing, such as a black and white checkerboard calibration template, an equally spaced solid circular array calibration template, etc. (as shown in fig. 4). The calibration scheme adopts a solid circle array as a calibration pattern, the solid circle array is composed of C rows multiplied by L columns of solid circles, and C and L are natural numbers. The solid circles are the same in size, the radiuses of the solid circles are the same, the distances between the centers of the horizontal direction and the vertical direction are the same, and the radiuses and the distances between the centers of the solid circles are determined by the number C of rows and the number L of columns of the solid circle array and the size of the standard template. The calibration pattern shot by the relay lens is consistent with the original pattern and also consists of C multiplied by L solid circles; the radius and the center distance of the simulated long-distance calibration pattern are related to the distance from the relay lens to the calibration template.
The relay lens is positioned between the vehicle-mounted camera to be calibrated and the calibration template, and an enlarged virtual image (i.e. a virtual calibration template 4, as shown in fig. 5) is generated at the same side and a long distance from the physical calibration template 2 at a short distance. The relay lens 1 is used for shooting a short-distance object, namely equivalently obtaining a long-distance virtual image of the calibration template. According to the size of the physical calibration template 2 and the distance between the relay lens 1 and the physical calibration template 2, the size and the simulation distance of the virtual calibration template 4 can be calculated based on an optical Gaussian formula, so that the specified focusing long distance can be accurately calibrated through the relay lens 1.
By the Gaussian formula
The distance from the virtual calibration template 4 to the relay lens 1 can be obtained
And the size of the virtual calibration template 4
Wherein:
f': the focal length of the relay lens 1;
l': the distance (simulated focus distance) from the virtual calibration template 4 to the relay lens 1;
l: the distance from the physical calibration template 2 to the relay lens 1;
y': virtually calibrating the size of the template 4 in the vertical direction;
y: the vertical dimension of the physical calibration template 2;
according to the formulas (1) and (2), the simulated distance between the virtual calibration template 4 and the relay lens 1 is determined by the distance between the physical calibration template 2 and the relay lens 1, so that when vehicle-mounted cameras with different focal lengths are calibrated, the distance between the relay lens and the physical calibration template 2 is adjusted based on a Gauss formula according to the target focusing distance of the vehicle-mounted cameras, and the effect of accurately simulating the target long distance can be achieved.
2. Extraction unit
Under the action of distance increase of the relay lens, the calibration object of the camera at the moment is a remote virtual calibration template, and a frame of calibration image acquired by the camera is subjected to circle center feature point extraction through methods such as binarization, median filtering and the like. Or detecting a solid circle region based on the image gray value, and calculating the center of mass of the circle region as the coordinate of the center of the circle.
3. Calibration unit
Based on the image of the calibration template shot by a single sheet, a homography matrix corresponding to the image can be calculated, and the initial value of the internal parameter of the vehicle-mounted camera is obtained by calculation according to the pixel position of the center of a solid circle in the image of the calibration template. The initial value of the external parameter rotation matrix of the vehicle-mounted camera is a unit matrix, and the initial value of the distortion parameter of the vehicle-mounted camera is zero. The calibration process is described in detail in the examples below.
4. Optimization unit
And generating a corresponding image pixel point in the image when the center of each solid circle is photographed. Based on the pinhole imaging principle and the projection model of image distortion, each circle center can also calculate a theoretical imaging position (including image distortion). The deviation of the actual image point from this theoretical pixel position is called the reprojection error. The parameters of the geometric projection model, namely the calibration parameters of the camera, are to ensure that the square sum of the reprojection errors of the centers of the solid circles on all the calibration templates is minimum, and at the moment, the projection model describes the optical imaging projection process of the camera most accurately
Wherein M is the position of the center of the solid circle on the virtual calibration template, M is the position of the pixel corresponding to the center point in the calibration image for correcting the distortion,
for the reprojection position, K is an internal parameter of the camera, the rotation vector R and translation vector T are external parameters of the camera, and K is [ K ═ K [, K [ ]1，k2，k3]As a parameter of the radial distortion,
is a tangential distortion parameter.
The equation was optimized using Levenberg-Marquardt. After inputting the initial value and carrying out a plurality of iterations, when the error of the iteration is smaller than a preset threshold value, the optimization iteration is finished, and the obtained results K, R, T, K and p are respectively the internal parameter, the external parameter and the distortion parameter corresponding to the calibration distance. The camera calibration parameters obtained through the nonlinear optimization are more accurate.
The following describes a calibration method and apparatus for a single-board single-image vehicle-mounted camera with a relay lens according to the present invention by way of specific embodiments. When the method is adopted for calibration, a 2D plane calibration template needs to be installed on a production line of the vehicle-mounted camera; placing a relay lens at a specific distance of a calibration template to generate a long-distance virtual image of the calibration template, wherein the calibration distance only takes an image of one calibration template, and the calibration template does not need to rotate; and analyzing and processing the single calibration template image shot at the distance, and calculating to obtain the vehicle-mounted camera calibration parameter corresponding to the distance.
And selecting a proper relay lens according to the parameters (focal length, focusing distance and field angle) of the vehicle-mounted camera to be calibrated, including determining the focal length f' of the relay lens. According to the calibration distance l 'to be simulated and the focal length f' of the relay lens, the distance l from the relay lens to the calibration template can be determined based on the formula (1).
The vertical direction size y 'of the virtual calibration template is determined by the field angle and the calibration distance l' of the vehicle-mounted camera to be calibrated. The vertical dimension y of the 2D planar calibration template can be determined based on the foregoing equation (2) according to the simulated calibration distance l' and the distance l from the relay lens to the calibration template.
For the distance calibration, the detailed flow of the calibration method is as follows:
1. acquiring a simulated remote calibration template image obtained by shooting a calibration template through a relay lens; (corresponding to S1)
The calibration template is typically a repeating pattern with fixed spacing, such as a black and white checkerboard calibration template, an equally spaced solid circular array calibration template, and the like. As shown in fig. 4, the present calibration scheme employs a solid circle array consisting of 8 rows by 11 columns of solid circles as the calibration template pattern. The solid circles are the same in size, the same in radius, and the same in horizontal and vertical center distance, and the radius and the center distance of the solid circles are determined by the number of rows 8 and the number of columns 11 of the solid circle array and the size of the standard template. The calibration pattern shot by the relay lens is consistent with the original pattern and also consists of 8 multiplied by 11 solid circles; the radius and the center distance of the simulated long-distance calibration pattern are related to the distance from the relay lens to the calibration template. According to the number and the center distance of the solid circles in the horizontal direction and the vertical direction in the calibration template, the distribution of the solid circle array in a calibration template coordinate system can be determined; based on the simulated distance of the relay lens, a homogeneous coordinate [ X Y Z1 ] of the circle center can be obtained.
In this example, only one calibration template image needs to be shot for the simulated calibration distance, and the optical axis of the vehicle-mounted camera and the optical axis of the relay lens are parallel, concentric and perpendicular to the calibration template during shooting, so that the calibration image pattern is clear and no pattern deformation caused by shooting angles exists. In addition, because the calibration template is arranged on the LED panel lamp, the contrast between the calibration pattern and the white background is strong, the contrast between the edge of the calibration pattern is strong, and the calibration pattern is easy to extract. Based on the above two advantageous factors, the solid circle array calibration template is suitable in the present embodiment.
2. Detecting a solid circle of the calibration template image to extract a central point; (corresponding to S2)
In the field of computer vision such as three-dimensional scene reconstruction, repeated solid circles are often used to construct calibration patterns. The center of the solid circle has the advantages of easy detection, high position precision, reliable matching, real-time processing and the like. The current circle center detection algorithm comprises: circle center detection based on Blob area analysis, circle center detection based on edge extraction, circle center detection based on Hough transform and the like. In this example, after the calibration image is captured, a solid circle region is detected based on the gray value of the image, and a homogeneous coordinate [ x y 1] with the center of mass of the circle region as the center pixel position is obtained. The image processing steps are simple in calculation and strong in real-time performance.
3. Calculating or setting initial values of internal parameters, external parameters (rotation vectors and translation vectors) and distortion parameters of the vehicle-mounted camera; (corresponding to S3)
In computer vision, the interrelationship of a point on an object in space to its projected position on an image plane by an imaging system is generally described by a geometric projection model of a camera (or camera) system. A commonly used projection model is the central projection in optics based on the pinhole imaging principle. In the model, a point on an object passes through the projection center, namely the optical center of a lens, and is projected on an imaging chip along a straight line.
The homogeneous coordinate of the center of the solid circle in the reference coordinate system is [ X Y Z1 ], and the homogeneous coordinate of the pixel obtained by photographing the point in the camera is assumed to be [ X Y1 ]. According to a projection model based on pinhole imaging, the center [ X Y1 ] of a solid circle of a calibration template is projected on an image according to the following relation to obtain a corresponding imaging pixel point [ x y 1] (the Z coordinate is assumed to be 0 in the case of a plane calibration template)
Where σ is a scale factor. The rotation vector R and the translation vector T are external parameters of the camera and describe the spatial position of the camera in a calibration template coordinate system. K is an internal parameter of the camera and is defined as
Wherein f isxAnd fyFocal length of the lens in horizontal and vertical directions, cxAnd cyIs the principal point of the image.
Based on each shot image of the calibration template, the corresponding homography matrix can be calculated
Wherein h isjIs the column vector of the jth column (j ═ 1, 2, 3), hijIs the H matrix element in the ith row and jth column (i, j ═ 1, 2, 3). According to the definition of the homography matrix, the following are defined:
according to the nature of the rotation matrix, r1And r2Is an orthogonal unit vector, having:
wherein:
obtained from the equations (8) and (9), respectively
h11h12·B11+(h31h12+h11h32)·B13+h21h22·B22+(h31h22+h21h32)·B23+h31h32·B33＝0 (11)
In the production process of the camera, the optical lens and the imaging chip are coupledDuring assembling, the main point of the image is required to be away from the central point of the imaging chip and not exceed a certain pixel range. Thus presetting the principal point cxAnd cyIs the image center. (11) In the formula (12), the homography matrix H can be obtained by shooting the image of the calibration template, and the element H thereofij(i, j ═ 1, 2, 3) is known; c. CxAnd cyAs the center point of the image, known, BijIs the camera intrinsic parameter focal length fx、fyAnd principal point cx、cyIs an intermediate quantity, B, occurring during the calculationijHas only two unknowns f thereinxAnd fyThus, the two equations (11), (12) can solve two internal parameters: focal length fxAnd fyIs started. The initial value of the spatial position T of the vehicle-mounted camera in the calibration template coordinate system can be estimated according to the position of the camera relative to the calibration template and the type of the vehicle-mounted camera. Because the optical axis of the vehicle-mounted camera is approximately vertical to the calibration template, the initial value of the rotation matrix R is set as the unit matrix
Because the lens has optical distortion, the actually projected pixel point generally has a small deviation on the image. The image distortion is mainly caused by the following reasons: the processing error of the lens surface causes the defect of the radial curvature; the optical center of each lens cannot be strictly kept collinear, and an eccentricity error is generated; due to tolerances in lens design, production and camera assembly processes, the lens and the imaging chip are not parallel and inclined. The above errors cause distortion of the image in both radial and tangential directions. Radial distortion refers to the fact that the actual image point moves inward or outward on its ideal location and the optical center line, i.e., radially. Tangential distortion refers to the fact that the actual image point is shifted in the direction perpendicular to the sagittal direction, i.e. in the tangential direction.
The theoretical pixel location based on the central projection model [ x y ] described above]Is affected by distortion, and is shifted to its actual projection position
Simulation with the following relationship
Wherein, [ k ]1 k2 k3]As a radial distortion parameter, [ p ]1 p2]As a tangential distortion parameter, r2＝x2+y2. Radial and tangential distortion parameters [ k ] of camera lens in general1 k2 k3]And [ p ]1 p2]Is set to zero.
4. Optimizing internal parameters, external parameters and distortion parameters of the vehicle-mounted camera to obtain an accurate calibration result; (corresponding to S4)
And generating a corresponding image pixel point in the image when the center of each solid circle is photographed. According to the projection model based on the pinhole imaging principle and the distortion model, a theoretical imaging position (including offset generated by distortion) can be calculated for each circle center. The deviation of the actual image point from this theoretical pixel position is called the reprojection error. The parameters of the geometric projection model, namely the calibration parameters of the camera, are such that the square sum of the reprojection errors of the centers of all solid circles on the calibration template is the minimum. The projection model now describes the optical imaging projection process of the camera most accurately at this depth of field, where the imaging projection process is as shown in the foregoing formula (3).
Wherein M is the position of the center of a solid circle in a virtual image simulated by the calibration template through the relay lens, M is the position of the center of the circle in the image corresponding to an actual pixel,
pixel positions calculated for the reprojection positions, i.e. the positions of pixels displaced by image distortion after the centre of a solid circle is projected at the center
And
including distortion offset), K is an internal parameter of the camera, the rotation vector R and the translational vector T are external parameters of the camera, and K is [ K ═ K [1 k2 k3]For radial distortion parameter, p ═ p1 p2]Is a tangential distortion parameter. The equation is optimized by using Levenberg-Marquardt and Levenberg-Marquardt algorithm, after a plurality of iterations, when the error of the iteration is smaller than a preset threshold value, the optimization iteration is finished, and the obtained results K, R, T, K and p are respectively the internal parameter, the external parameter and the distortion parameter corresponding to the calibration distance. The calibration parameters of the vehicle-mounted camera obtained through the nonlinear optimization are more accurate.
The embodiments of the present invention are described only for the preferred embodiments of the present invention, and not for the limitation of the concept and scope of the present invention, and various modifications and improvements made to the technical solution of the present invention by those skilled in the art without departing from the design concept of the present invention shall fall into the protection scope of the present invention, and the technical content of the present invention which is claimed is fully set forth in the claims.
Claims (13)
1. The production line calibration method of the single board single-image relay lens of the vehicle-mounted camera is characterized in that a calibration system comprising a calibration template, the relay lens and the vehicle-mounted camera to be calibrated is established, the distance between the relay lens and the calibration template is adjusted to be suitable for the vehicle-mounted cameras with different focal lengths, the calibration template is provided with a solid circle array, the vehicle-mounted camera shoots the calibration template through the relay lens, and the specific process is as follows:
s1, acquiring a virtual image of a calibration template formed by a relay lens through the acquisition of a vehicle-mounted camera as a calibration template image;
s2, extracting the pixel position coordinates of the center of the solid circle in the calibration template image by using an image processing algorithm;
s3, based on the calibration template image, combining the homography matrix and the center position coordinates of the solid circle, calculating the initial values of the calibration parameters of the vehicle-mounted camera, including the internal and external parameters and the distortion parameters;
and S4, optimizing the initial values of the internal and external parameters and the distortion parameters of the vehicle-mounted camera obtained in the step S3 to obtain accurate calibration parameters of the vehicle-mounted camera.
2. The production line calibration method for the single-plate single-belt relay lens of the vehicle-mounted camera according to claim 1, wherein in step S1, the optical axes of the vehicle-mounted camera and the relay lens are kept parallel and concentric, and the optical axes of the vehicle-mounted camera and the relay lens are kept perpendicular to the calibration template.
3. The production line calibration method for the single-plate single-belt relay lens of the vehicle-mounted camera according to claim 1, wherein in step S2, the feature points of the center of the solid circle of the calibration template image are extracted by binarization and median filtering methods to obtain the pixel position coordinates of the center of the solid circle in the calibration template image.
4. The production line calibration method for the single-plate single-belt relay lens of the vehicle-mounted camera according to claim 1, wherein in step S2, the solid circle region in the calibration template image is detected through the image gray value, and the center of mass of the region is calculated to obtain the pixel position coordinates of the center of the solid circle in the calibration template image.
5. The production line calibration method for the single-plate single-belt relay lens of the vehicle-mounted camera according to claim 1, wherein in step S3, the initial value of the internal parameter of the vehicle-mounted camera is obtained through calculation according to the homography matrix of the calibration template image and by combining the pixel position of the center of the solid circle in the calibration template image.
6. The production line calibration method for the single-plate single-belt relay lens of the vehicle-mounted camera according to claim 1, wherein in step S3, an initial value of the extrinsic parameter rotation matrix of the vehicle-mounted camera is an identity matrix.
7. The production line calibration method for the single-plate single-belt relay lens of the vehicle-mounted camera according to claim 1, wherein in step S3, an initial value of a distortion parameter of the vehicle-mounted camera is zero.
8. The production line calibration method for the single-plate single-belt relay lens of the vehicle-mounted camera according to claim 1, wherein in step S4, the initial values of the inner and outer parameters and the distortion parameter of the vehicle-mounted camera are optimized based on the principle of minimizing the sum of squares of the reprojection errors of the centers of the solid circles of the calibration template images.
9. The production line calibration method for the single-plate single-image relay lens of the vehicle-mounted camera according to claim 1 or 8, wherein in step S4, a Levenberg-Marquardt optimization algorithm is used to optimize initial values of internal and external parameters and distortion parameters of the vehicle-mounted camera.
10. The production line calibration device for the single board single-board relay lens of the vehicle-mounted camera is characterized by comprising the relay lens, a calibration template and the vehicle-mounted camera to be calibrated, wherein the relay lens is arranged between a lens of the vehicle-mounted camera and the calibration template, the distance between the relay lens and the calibration template is adjustable, and the pattern of the calibration template is a solid circular array.
11. The production line calibration device of a single board single-board relay lens of a vehicle-mounted camera according to claim 10, wherein optical axes of the relay lens and the vehicle-mounted camera are overlapped, and the optical axes of the relay lens and the vehicle-mounted camera are perpendicular to the surface of the calibration template.
12. The production line calibration device of a single board with relay lens for a vehicle-mounted camera according to claim 10, wherein the array of solid circles on the surface of the calibration template is a row-column array formed by a plurality of solid circles, each solid circle has the same radius, and the centers of adjacent solid circles are at the same distance in the horizontal and vertical directions.
13. The production line calibration device for the single board image relay lens of the vehicle-mounted camera according to claim 10, further comprising an extraction unit, a calibration unit, and an optimization unit, which are constructed in a computer, wherein the extraction unit extracts the coordinates of the pixel position of the center of the solid circle in the calibration template image, the calibration unit calculates the initial values of the calibration parameters of the vehicle-mounted camera including the internal and external parameters and the distortion parameters, and the optimization unit optimizes the internal and external parameters and the distortion parameters of the vehicle-mounted camera to obtain the accurate calibration parameters of the vehicle-mounted camera.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011521340.0A CN112489141B (en) | 2020-12-21 | 2020-12-21 | Production line calibration method and device for single-board single-image strip relay lens of vehicle-mounted camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011521340.0A CN112489141B (en) | 2020-12-21 | 2020-12-21 | Production line calibration method and device for single-board single-image strip relay lens of vehicle-mounted camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112489141A true CN112489141A (en) | 2021-03-12 |
| CN112489141B CN112489141B (en) | 2024-01-30 |
Family
ID=74915173
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011521340.0A Active CN112489141B (en) | 2020-12-21 | 2020-12-21 | Production line calibration method and device for single-board single-image strip relay lens of vehicle-mounted camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112489141B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113850870A (en) * | 2021-09-13 | 2021-12-28 | 安徽光阵光电科技有限公司 | Calibration method of vehicle-mounted camera |
| CN115170674A (en) * | 2022-07-20 | 2022-10-11 | 禾多科技（北京）有限公司 | Camera principal point calibration method, device, equipment and medium based on single image |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109360245A (en) * | 2018-10-26 | 2019-02-19 | 魔视智能科技(上海)有限公司 | The external parameters calibration method of automatic driving vehicle multicamera system |
| CN109543660A (en) * | 2018-12-20 | 2019-03-29 | 深圳奥比中光科技有限公司 | A kind of focus control and focus adjustment method |
| CN109767476A (en) * | 2019-01-08 | 2019-05-17 | 像工场(深圳)科技有限公司 | A kind of calibration of auto-focusing binocular camera and depth computing method |
| CN110274752A (en) * | 2018-03-14 | 2019-09-24 | 深圳市隆测技术有限公司 | The Multifunctional test card and its test method of relay lens image quality |
| US20200226789A1 (en) * | 2019-01-14 | 2020-07-16 | Beijing Boe Optoelectronics Technology Co., Ltd. | Camera calibration plate, camera calibration method and device, and image acquisition system |
-
2020
- 2020-12-21 CN CN202011521340.0A patent/CN112489141B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110274752A (en) * | 2018-03-14 | 2019-09-24 | 深圳市隆测技术有限公司 | The Multifunctional test card and its test method of relay lens image quality |
| CN109360245A (en) * | 2018-10-26 | 2019-02-19 | 魔视智能科技(上海)有限公司 | The external parameters calibration method of automatic driving vehicle multicamera system |
| CN109543660A (en) * | 2018-12-20 | 2019-03-29 | 深圳奥比中光科技有限公司 | A kind of focus control and focus adjustment method |
| CN109767476A (en) * | 2019-01-08 | 2019-05-17 | 像工场(深圳)科技有限公司 | A kind of calibration of auto-focusing binocular camera and depth computing method |
| US20200226789A1 (en) * | 2019-01-14 | 2020-07-16 | Beijing Boe Optoelectronics Technology Co., Ltd. | Camera calibration plate, camera calibration method and device, and image acquisition system |
Non-Patent Citations (1)
| Title |
|---|
| 杨敏, 沈春林: "基于单平面模板的摄像机定标研究", 数据采集与处理, no. 01 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113850870A (en) * | 2021-09-13 | 2021-12-28 | 安徽光阵光电科技有限公司 | Calibration method of vehicle-mounted camera |
| CN115170674A (en) * | 2022-07-20 | 2022-10-11 | 禾多科技（北京）有限公司 | Camera principal point calibration method, device, equipment and medium based on single image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112489141B (en) | 2024-01-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109767476B (en) | Automatic focusing binocular camera calibration and depth calculation method | |
| CN109859272B (en) | Automatic focusing binocular camera calibration method and device | |
| CN109598762B (en) | High-precision binocular camera calibration method | |
| CN110276808B (en) | Method for measuring unevenness of glass plate by combining single camera with two-dimensional code | |
| CN107255443B (en) | Method and device for calibrating binocular vision sensor in site in complex environment | |
| CN111080705B (en) | Calibration method and device for automatic focusing binocular camera | |
| CN105931222B (en) | The method for realizing high-precision camera calibration with low precision two dimensional surface target | |
| CN103530880B (en) | Based on the camera marking method of projection Gaussian network pattern | |
| CN111047649B (en) | Camera high-precision calibration method based on optimal polarization angle | |
| CN102810205B (en) | The scaling method of a kind of shooting or photographic means | |
| CN103278138B (en) | Method for measuring three-dimensional position and posture of thin component with complex structure | |
| CN109272555B (en) | External parameter obtaining and calibrating method for RGB-D camera | |
| CN113920205B (en) | Calibration method of non-coaxial camera | |
| CN109712232B (en) | Object surface contour three-dimensional imaging method based on light field | |
| CN112465912A (en) | Three-dimensional camera calibration method and device | |
| CN112489141B (en) | Production line calibration method and device for single-board single-image strip relay lens of vehicle-mounted camera | |
| CN110827360B (en) | Photometric stereo measurement system and method for calibrating light source direction thereof | |
| CN113119129A (en) | Monocular distance measurement positioning method based on standard ball | |
| CN112465915A (en) | Vehicle-mounted panoramic system calibration method | |
| Vilaca et al. | Calibration procedure for 3D measurement systems using two cameras and a laser line | |
| CN116740187A (en) | Multi-camera combined calibration method without overlapping view fields | |
| CN113658279B (en) | Camera internal reference and external reference estimation method, device, computer equipment and storage medium | |
| CN114998448A (en) | Method for calibrating multi-constraint binocular fisheye camera and positioning space point | |
| Zhang et al. | Improved camera calibration method and accuracy analysis for binocular vision | |
| He et al. | Calibration method for a central catadioptric-perspective camera system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |