CN114463437A - Camera calibration method, device, equipment and computer readable medium - Google Patents
Camera calibration method, device, equipment and computer readable medium Download PDFInfo
- Publication number
- CN114463437A CN114463437A CN202210037926.2A CN202210037926A CN114463437A CN 114463437 A CN114463437 A CN 114463437A CN 202210037926 A CN202210037926 A CN 202210037926A CN 114463437 A CN114463437 A CN 114463437A
- Authority
- CN
- China
- Prior art keywords
- calibration
- camera
- dot matrix
- coordinates
- parameter
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 80
- 239000011159 matrix material Substances 0.000 claims abstract description 102
- 230000001131 transforming effect Effects 0.000 claims abstract description 8
- 230000002159 abnormal effect Effects 0.000 claims description 25
- 238000004590 computer program Methods 0.000 claims description 21
- 230000008569 process Effects 0.000 claims description 18
- 238000013519 translation Methods 0.000 claims description 13
- 230000006870 function Effects 0.000 claims description 12
- 238000000605 extraction Methods 0.000 claims description 11
- 238000006073 displacement reaction Methods 0.000 claims description 6
- 238000007476 Maximum Likelihood Methods 0.000 claims description 5
- 230000008859 change Effects 0.000 claims description 3
- 230000008030 elimination Effects 0.000 claims description 2
- 238000003379 elimination reaction Methods 0.000 claims description 2
- 238000005457 optimization Methods 0.000 claims description 2
- 238000005316 response function Methods 0.000 claims description 2
- 239000000126 substance Substances 0.000 claims 1
- 238000011160 research Methods 0.000 abstract description 2
- 230000007246 mechanism Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 7
- 238000003384 imaging method Methods 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 241000226585 Antennaria plantaginifolia Species 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Studio Devices (AREA)
Abstract
The invention provides a camera calibration method, a camera calibration device, camera calibration equipment and a computer readable medium. The method belongs to the technical field of camera calibration, and solves the problem that the calibration precision of the camera is reduced due to deviation of dot matrix image coordinates in the prior art. The method comprises the following steps: acquiring a picture shot by a camera on a calibration plate; extracting dot matrix image coordinates in the picture; calibrating a camera by using a Zhangyingyou calibration method to obtain a first calibration parameter; and calculating a homography matrix by using the first calibration parameters, transforming the dot matrix image, acquiring the coordinates of the transformed dot matrix image, completing the calibration of the camera and outputting second calibration parameters. Compared with the prior art, the method has the advantages of simple and convenient realization conditions, high calibration precision, wide applicable scenes and important application value and research significance.
Description
Technical Field
The invention relates to the field of computer vision and camera calibration, in particular to a camera calibration method, a camera calibration device, camera calibration equipment and a computer readable medium.
Background
The calibration of the camera is very important in the field of computer vision measurement, and the calibration precision directly determines the capability of the camera to correctly restore the real world scene. The calibration of the camera generally adopts a Zhangyingyou calibration method, namely a template with an accurate positioning dot matrix is drawn, and then the template and the camera move and are static one by one to obtain template images of at least 3 different directions. And calculating a homography matrix between the image and the template by determining the matching of the image and the dot matrix on the template, and linearly solving the internal parameter, the external parameter and the distortion coefficient of the camera by using the homography matrix. The commonly used template is a two-dimensional calibration plate, and the calibration plate is mostly a checkerboard which is uniformly distributed, as shown in figure 1.
However, the existing dot matrix identification method has low precision, so that the image coordinates of the identified partial two-dimensional points have deviation, and the precision of the camera calibration result is further reduced. Therefore, there is a need for an improved camera calibration method.
Disclosure of Invention
In view of this, embodiments of the present invention provide a camera calibration method, an apparatus, a terminal device, and a computer readable medium, which solve the problem that the calibration accuracy of a camera is reduced due to the deviation of dot matrix image coordinates.
A first aspect of an embodiment of the present invention provides a camera calibration method, including:
s101: acquiring a picture shot by a camera on a calibration plate;
s102: extracting dot matrix image coordinates in the picture;
s103: calibrating a camera by using a Zhangyingyou calibration method to obtain a first calibration parameter;
s104: calculating a first reprojection error of each feature point in the dot matrix image according to the first calibration parameter, and eliminating two-dimensional coordinates of abnormal feature points and three-dimensional coordinates corresponding to the abnormal feature points;
s105: calibrating the camera according to the eliminated dot matrix image coordinates to obtain a second calibration parameter, if the calibration precision is higher than the first calibration parameter, repeating the step S104, otherwise, performing the next step;
s106: and calculating a homography matrix by using the second calibration parameters, transforming the dot matrix image, acquiring the transformed dot matrix image coordinates, completing camera calibration and outputting a result.
A second aspect of the embodiments of the present invention provides a camera calibration apparatus, including:
an acquisition module: the system is used for acquiring pictures shot by the camera on the calibration board;
an extraction module: the system is used for extracting dot matrix image coordinates in the picture;
a calibration module: the camera calibration method comprises the steps of calibrating a camera by using a Zhang Yong calibration method to obtain a first calibration parameter;
the lattice high-precision extraction module: the calibration module is used for calculating a homography matrix by using the first calibration parameters, transforming the dot matrix image, obtaining the coordinates of the transformed dot matrix image, completing the calibration of the camera and outputting second calibration parameters
An abnormal point eliminating module: the first reprojection error of each characteristic point in the dot matrix image is calculated according to the first calibration parameter, and the two-dimensional coordinates of the abnormal characteristic points and the corresponding three-dimensional coordinates of the abnormal characteristic points are eliminated;
a judgment and output module: and the camera calibration is carried out according to the rejected dot matrix image coordinates to obtain a third calibration parameter, if the calibration precision is higher than the second calibration parameter, the abnormal point rejection module process is repeated, and if not, the third calibration parameter is output.
A third aspect of the embodiments of the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the camera calibration method when executing the computer program.
A fourth aspect of the embodiments of the present invention provides a computer-readable medium, which stores a computer program, and when the computer program is processed and executed, the computer program implements the steps of the camera calibration method described above.
Compared with the prior art, the camera calibration method provided by the invention has the following effects:
1. the high-precision extraction step of the dot matrix firstly calculates a homography matrix, and the homography matrix converts a shot image into an image in the front view direction by taking a calibration plate plane as a reference. Then, the image is converted into a front view through the homography matrix, and the lattice is extracted by using a lattice identification algorithm, so that the lattice identification precision can be obviously improved.
2. The abnormal feature point eliminating step firstly calculates the reprojection error of each feature point in the dot matrix, and eliminates the two-dimensional coordinates of the feature points with larger image coordinate deviation and the corresponding three-dimensional coordinates for multiple times according to a certain strategy, so that the average reprojection error of the dot matrix is finally reduced, and the accuracy of the calibration result can be obviously improved.
3. By the method, the internal parameter, the external parameter and the distortion coefficient of the camera can be accurately calculated.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
FIG. 1 is a schematic illustration of a prior art calibration plate;
FIG. 2 is a flowchart of a camera calibration method according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating a calibration result determination process according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating the effect of extracting a lattice according to an embodiment of the present invention;
FIG. 5 is a front view of a calibration board and a dot matrix extraction effect diagram provided in the embodiment of the present invention;
FIG. 6 is a schematic diagram of a dedicated camera calibration device provided in an embodiment of the present invention;
fig. 7 is a block diagram of a camera calibration apparatus according to an embodiment of the present invention;
fig. 8 is a schematic diagram of a camera calibration apparatus provided in an embodiment of the present invention.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.
In image measurement processes and machine vision applications, in order to determine the correlation between the three-dimensional position of a certain point on the surface of an object in space and the corresponding point in the image, a geometric model imaged by a camera needs to be established. The parameters in the geometric model are camera parameters. Under most conditions, the parameters must be obtained through experiments and calculation, and the process of solving the parameters is called camera calibration or camera calibration.
The purpose of camera calibration is as follows: and acquiring an internal reference matrix and an external reference matrix of the camera (and simultaneously acquiring a selection matrix and a translation matrix of each calibration image), wherein the internal reference coefficient and the external reference coefficient can correct the images shot by the camera later, so that the images with relatively small distortion are obtained.
Inputting camera calibration: the image coordinates of all inner corner points on the image are calibrated, and the spatial three-dimensional coordinates of all inner corner points on the plate image are calibrated (generally, the image is assumed to be positioned on a plane with Z being 0).
Output of camera calibration: internal reference, external reference coefficient and distortion coefficient of the camera.
The geometric model parameters determined in the camera calibration can be divided into two types, i.e., camera internal parameters and camera external parameters. The camera internal parameter is used for determining the projection relation of the camera from a three-dimensional space to a two-dimensional image, namely determining the conversion relation between a camera coordinate system and a pixel coordinate system. The camera external parameter is used for determining the relative position relation between the camera coordinate and the world coordinate system, namely determining the conversion relation between the camera coordinate system and the world coordinate system.
The world coordinate system is an absolute coordinate system of the objective three-dimensional world, also referred to as an objective coordinate system. Its purpose is to describe the position of an object in the real world.
The camera coordinate system takes the optical center of the camera as the origin of coordinates, and the X-axis and the Y-axis are respectively parallel to the X-axis and the Y-axis of the image coordinate system. Its purpose is to describe the position of the object from the perspective of the camera.
The image coordinate system is based on the center of the image plane (also called imaging plane) of the camera as the coordinate origin, and the X-axis and the Y-axis are respectively parallel to two vertical sides of the image plane. Which represents the position of a pixel in an image in units of length. The method has the function of describing the projection relation of the target object from the camera coordinate system to the pixel coordinate system in the imaging process, and is convenient for further obtaining the coordinates under the pixel coordinate system.
The pixel coordinate system takes the top left corner vertex of the image plane as the origin, and the X axis and the Y axis are respectively parallel to the X axis and the Y axis of the image coordinate system, and the unit is the pixel number. The pixel coordinate system is introduced to describe the coordinates of the imaged pixel points on the digital image.
The ideal imaging model is a pinhole imaging model, and objects and images will satisfy the relationship of similar triangles. However, due to processing and assembly errors of the camera optical system, the lens cannot satisfy the relation of similar triangle between the object and the image, and the distortion exists between the image actually formed on the camera image plane and the ideal image. The distortion mainly includes radial distortion and tangential distortion. Radial distortion arises because of the barrel or pincushion distortion of the image formed by the camera during actual imaging due to the greater deflection of the rays away from the center of the lens as compared to the center of the lens. The reason for the tangential distortion is that the lens is not perfectly parallel to the image plane. If there is tangential distortion, the resulting image will likely become a trapezoid if a rectangle is projected onto the image plane. Thus, the camera may describe the imaging distortion of the camera using corresponding sagittal distortion parameters and tangential distortion parameters.
When a point in the world coordinate system is to be projected onto the image plane of the camera, the coordinates of the point need to be transformed from the world coordinate system into the camera coordinate system, which can be obtained by rotation and translation. Thus, the camera external parameters may include a rotation matrix and a translation matrix. The rotation matrix is used to describe the angle of rotation about the X-axis, the angle of rotation about the Y-axis, and the angle of rotation about the Z-axis. The translation matrix is used to describe displacement in the X-axis direction, displacement in the Y-axis direction, and displacement in the Z-axis direction.
In order to explain the technical means of the present invention, the following description will be given by way of specific examples.
Referring to fig. 2, fig. 2 is a flowchart of a camera calibration method according to an embodiment of the present invention. As shown in fig. 2, comprises the following steps:
s101: and acquiring the picture of the calibration plate shot by the camera.
In the process of implementing S101 specifically, the dedicated camera calibration device provided in the present invention is used to acquire images, and generally, at least three calibration images need to be acquired. As shown in fig. 6, the present invention provides a special camera calibration apparatus, including: the device comprises a single-shaft translation machine, a bottom plate 2, a rotating mechanism 3, a calibration plate 4, a camera 5, a camera supporting structure 6, a two-shaft translation mechanism 7, a horizontal direction sliding groove 8, a vertical direction sliding groove 9 and a front-back direction sliding groove 10. Wherein the calibration plate pattern is shown in FIG. 1; if a world coordinate system is established on the basis of the calibration board 4, the real coordinates of each calibration block in the world coordinate system are fixed and known.
In addition, the bottom of the rotating mechanism 3 is fixedly arranged at the end part of the bottom plate 2, and the back surface of the calibration plate 4 is connected with the top of the rotating mechanism 3. In this embodiment, the top of the rotating mechanism 3 refers to a rotatable component, that is, the calibration plate 4 can rotate, and the rotation angle is α; the rotating mechanism 3 may be a component capable of automatically rotating, such as a motor, or may be a component that needs to be rotated by an external force, which is not limited in this embodiment.
In this embodiment, the first fixing bracket 6, the second fixing bracket 11, the third fixing bracket 12, and the fourth fixing bracket 13 are separately provided, and are used to fix different components. Specifically, the first fixed bracket 6 is used for supporting the two-axis translation mechanism 7, the end of the second fixed bracket 11 is used for fixing the camera 5, the third fixed bracket 12 is used for fixing the left single-axis translation mechanism 1, and the fourth fixed bracket 13 is used for fixing the right single-axis translation mechanism 1. Meanwhile, the front surface of the calibration plate 4 faces the first fixing bracket 6, so that the camera 5 fixed on the second fixing bracket 11 can capture an image of the front surface of the calibration plate 4 while facing the base plate 2. The range corresponding to the dotted line at the lens of the camera 5 represents the acquisition range of the camera 5.
The image acquisition process of the special camera calibration equipment provided by the embodiment of the invention is as follows: fixing the camera 5 at the end of the second fixing bracket 11 so that the camera 5 can take a calibration image containing the front surface of the calibration plate 4; after a calibration image is acquired, the position and posture of the calibration plate 4 are changed by changing the rotation angle alpha of the rotating mechanism 3, then the three-dimensional coordinates of the space where the camera 5 is located are changed by using the left-side and right-side single-axis translation mechanisms 1 and the two-axis translation mechanism 7, and the camera 5 is used for acquiring a complete calibration image of the calibration plate 4 to complete one-time acquisition. And the acquisition is repeated for several times, so that a plurality of calibration images can be acquired.
S102: and extracting the dot matrix image coordinates in the picture.
In the process of implementing S102 specifically, the invention adopts a lattice extraction algorithm to extract the lattice in the image. First, a window W (x, y) of a fixed size is set, the pixel gray value of the window is I (x, y), the window is moved in the x and y directions by small displacements u and v, and the pixel gray value corresponding to the new position after the movement is I (x + u, y + v), so that the change value [ I (x + u, y + v) -I (x, y) ] of the gray value caused by the current movement of the window can be obtained. Meanwhile, let the gaussian kernel function ω (x, y) be a window function of W (x, y), and represent the weight of each pixel in the window. The window is moved in various directions (u, v), and the variation E (u, v) of the gray-level value generated by the window can be expressed as:
E(u，v)＝∑(x，y)ω(x，y)[I(x+u，y+v)-I(x，y)]2 (1)
to simplify the operation of equation (1), the Taylor equation is used to expand I (x + u, y + v):
I(x+u，y+v)≈I(x，y)+uIx+vIy (2)
wherein, Ix、IyRepresenting the gradient values of the image gray in the x and y directions, respectively. And further:
using eigenvalues lambda of the matrix M1、λ2The corner response function R for each window is calculated. Setting a threshold τcIf R corresponding to a pixel point satisfies the following condition:
R＝min(λ1，λ2)＞τc(5) the point is set as one of the feature points of the lattice, thereby extracting the feature point image coordinates.
In a specific embodiment, after performing the process of S102, the method further includes sorting the image coordinates of all feature points according to the order from left to right and from top to bottom, with the point at the top left corner being the starting point. The extracted dot matrix effect graph is shown in FIG. 4
S103: calibrating a camera by using a Zhangyingyou calibration method to obtain a first calibration parameter;
in the specific implementation process of S103, camera calibration is performed by using a zhangying calibration method, and a first calibration parameter is obtained: namely camera internal parameters, camera external parameters and camera distortion coefficients. Setting the point of three-dimensional world coordinate as X ═ X, Y, Z, 1 by Zhangyingyou calibration method]TThe pixel coordinate of the two-dimensional camera image is m ═ u, v, 1]TTherefore, the homography relationship from the calibration plate plane to the image plane for calibration is:
sm＝K[R，T]X (6)
wherein s is a scale factor, K is an internal parameter of the camera, R is a rotation matrix, and T is a translation vector. Order to
And setting the world coordinate system on a calibration plate plane, wherein the calibration plate plane is a plane with Z being 0. Then it can be obtained
Let K [ r1, r2, t ] be the homography matrix H, i.e.
H＝[h1h2h3]＝λK[r1r2t] (9)
In one particular embodiment of the present invention,
calculating an internal reference matrix K through a homography matrix:
solving an external parameter matrix [ R, T ] through the obtained internal parameter matrix:
And (3) minimizing the gross weight projection error of all the feature points by a maximum likelihood estimation method, thereby optimizing the internal reference matrix and the external reference matrix:
wherein M isijFor the three-dimensional world coordinate, m, of the jth point of the marking plate in the ith drawingijIts image coordinates.
Solving distortion coefficient k by minimizing gross weight projection error of all feature points through maximum likelihood estimation method1，k2：
In the above equations (12) and (13), there is a function
Minimizing the total weight projection error of equations 12 and 13, respectively; due to the fact that
The method belongs to a nonlinear optimization problem and needs to be solved by using a maximum likelihood estimation method. Once the cover is closed
Solved, the parameters in the function can be jointly optimized.
S104: and calculating a homography matrix by using the first calibration parameters, transforming the dot matrix image, acquiring the coordinates of the transformed dot matrix image, completing the calibration of the camera and outputting second calibration parameters.
In the specific implementation process of S104, the first calibration parameters (the current camera internal reference matrix, the external reference matrix and the distortion coefficient) obtained in the process of S103 are used, including
Computing a homography matrix Hm：
Then through the homography matrix HmThe shot image is converted into a front view by taking the plane of the calibration plate as a reference, and because the distribution of neighborhood information near each point in the front view is more similar, a two-dimensional lattice with smaller image coordinate deviation can be obtained by adopting a lattice extraction algorithm, as shown in fig. 5. Finally through HmAnd back projecting the image coordinates of the two-dimensional lattice in the front view back to the original shot image. And calibrating the camera for the transformed image to obtain a second calibration parameter.
S105: and calculating a first reprojection error of each characteristic point in the dot matrix image according to the second calibration parameter, and eliminating two-dimensional coordinates of abnormal characteristic points and three-dimensional coordinates corresponding to the abnormal characteristic points.
In the specific implementation of S105, the second calibration parameter obtained in S104 is used to calculate a first reprojection error of each point (each feature point in the dot matrix image) of the calibration plate, which may specifically adopt formula 13. And sequencing the points according to the error from large to small, trying to remove the points with the largest error, and re-calibrating the camera by using the rest points to obtain a third calibration parameter. The feature points with large errors are referred to as abnormal feature points.
S106: and calibrating the camera according to the eliminated dot matrix image coordinates to obtain a third calibration parameter, if the calibration precision is higher than the second calibration parameter, repeating the step S105, and otherwise, outputting the third calibration parameter.
In the specific implementation process of S106, referring to fig. 3, the method includes:
s109: acquiring a second reprojection error by using all the feature points and the second calibration parameter before abnormal feature point elimination;
s110: acquiring a third reprojection error by using all the characteristic points subjected to abnormal characteristic point rejection and the second calibration parameter;
s111: and comparing the sizes of the first re-projection error, the second re-projection error and the third re-projection error, if the third re-projection error is smaller than the second re-projection error and the second re-projection error is smaller than the first re-projection error, determining that the precision of the second calibration parameter is higher than that of the first calibration parameter, repeating the step S105 until the precision of the second calibration parameter is lower than that of the first calibration parameter, and stopping the abnormal point removing process.
And outputting the final third calibration parameter.
In the above embodiments, the calculation of the reprojection error can be referred to the above equation 13.
In the above embodiment, the reprojection error of each feature point in the dot matrix is first calculated, and the two-dimensional coordinates of the feature point with a large image coordinate deviation and the corresponding three-dimensional coordinates thereof are removed multiple times according to a certain strategy, so that the average reprojection error of the dot matrix is finally reduced. Therefore, the accuracy of the calibration result can be obviously improved.
The invention firstly extracts the dot matrix image coordinates of the calibration plate in the image, calibrates the camera by using Zhangyingyou calibration method and outputs the result. And then, using a high-precision extraction process to improve the accuracy of the image coordinates of the dot matrix, re-calibrating the camera, rejecting two-dimensional points with larger image coordinate deviation and three-dimensional points corresponding to the two-dimensional points by using an abnormal point rejecting step, re-calibrating the camera, and continuing rejecting the camera if the precision of the calibrated result is higher than that of the last time, re-calibrating the camera and the like. And finally, outputting a calibration result with higher precision. Compared with the prior art, the method has the advantages of simple and convenient realization conditions, wide applicable scenes and important application value and research significance.
Referring to fig. 7, fig. 7 is a block diagram of a camera calibration apparatus according to an embodiment of the present invention. As shown in fig. 7, the method includes an obtaining module 101, an extracting module 102, a calibrating module 103, a lattice high-precision extracting module 104, an abnormal point eliminating module 105, and a judging and outputting module 106. The above modules are respectively used for executing the specific methods in S101, S102, S103, S104, S105 and S106 in fig. 2, and the details can be referred to the related introduction of fig. 2, which is only briefly described here:
the acquisition module 101: the system is used for acquiring pictures shot by the camera on the calibration board;
the extraction module 102: the system is used for extracting dot matrix image coordinates in the picture;
the calibration module 103: the camera calibration method comprises the steps of calibrating a camera by using a Zhang Yong calibration method to obtain a first calibration parameter;
the lattice high-precision extraction module 104: the calibration module is used for calculating a homography matrix by using the first calibration parameters, transforming the dot matrix image, obtaining the coordinates of the transformed dot matrix image, completing the calibration of the camera and outputting second calibration parameters
The abnormal point culling module 105: the device is used for calculating a first reprojection error of each characteristic point in the dot matrix image according to the first calibration parameter, and eliminating the two-dimensional coordinates of the characteristic points with larger errors and the corresponding three-dimensional coordinates thereof;
the judgment and output module 106: and the camera calibration is carried out according to the rejected dot matrix image coordinates to obtain a third calibration parameter, if the calibration precision is higher than the second calibration parameter, the abnormal point rejection module process is repeated, and if not, the third calibration parameter is output.
In a camera calibration apparatus provided in fig. 7, a homography matrix is first calculated, which converts a captured image into an image in a front view direction with reference to a calibration plate plane. Then, the image is converted into a front view through the homography matrix, and the dot matrix is extracted by using a dot matrix identification algorithm, so that the dot matrix identification precision can be obviously improved. By calculating the reprojection error of each feature point in the dot matrix and rejecting the two-dimensional coordinates of the feature points with larger image coordinate deviation and the corresponding three-dimensional coordinates for multiple times according to a certain strategy, the average reprojection error of the dot matrix is finally reduced, and the accuracy of the calibration result can be obviously improved. By the method, the internal parameter, the external parameter and the distortion coefficient of the camera can be accurately calculated.
Fig. 8 is a schematic diagram of a camera calibration apparatus according to an embodiment of the present invention. As shown in fig. 8, the calibration apparatus 14 of this embodiment includes: a processor 140, a memory 141, and a computer program 142 stored in said memory 141 and executable on said processor 140. The processor 140 implements the steps in the above-described method embodiments, e.g., S101 to S106 shown in fig. 2, when executing the computer program 142. Alternatively, the processor 140, when executing the computer program 142, implements the functions of each module/unit in the above-mentioned device embodiments, for example, the functions of the modules 101 to 106 shown in fig. 7.
Illustratively, the computer program 142 may be partitioned into one or more modules/units that are stored in the memory 141 and executed by the processor 140 to implement the present invention. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution of the computer program 142 in the camera calibration device 14. For example, the computer program 142 may be divided into modules 101 to 106. A module in a virtual device.
The Processor 140 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 141 may be an internal storage unit of the camera calibration device 14, such as a hard disk or a memory of the camera calibration device 14. The memory 141 may also be an external storage device of the camera calibration device 14, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the camera calibration device 14. Further, the memory 141 may also include both an internal storage unit of the camera calibration device 14 and an external storage device. The memory 141 is used for storing the computer programs and other programs and data required by the camera calibration device 14. The memory 141 may also be used to temporarily store data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/terminal device are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
In the above embodiments, the first calibration parameter, the second calibration parameter, the third calibration parameter, the first reprojection error, the second reprojection error, and the third reprojection error do not represent a certain determined value, but a current result, a current previous result, and a next result in one cycle of determination or calibration, for convenience of description, in the present invention, they are defined as the first, the second, and the third in one cycle, and are not limited to the embodiments of the present invention.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.
Claims (9)
1. A camera calibration method, comprising:
s101: acquiring a picture shot by a camera on a calibration plate;
s102: extracting dot matrix image coordinates in the picture;
s103: calibrating a camera by using a Zhangyingyou calibration method to obtain a first calibration parameter;
s104: and calculating a homography matrix by using the first calibration parameters, transforming the dot matrix image, acquiring the coordinates of the transformed dot matrix image, completing the calibration of the camera and outputting second calibration parameters.
S105: calculating a first reprojection error of each feature point in the dot matrix image according to the second calibration parameter, and eliminating two-dimensional coordinates of abnormal feature points and three-dimensional coordinates corresponding to the abnormal feature points;
s106: and calibrating the camera according to the eliminated dot matrix image coordinates to obtain a third calibration parameter, if the calibration precision is higher than the second calibration parameter, repeating the step S105, and otherwise, outputting the third calibration parameter.
2. The camera calibration method according to claim 1, wherein the extracting coordinates of the dot matrix image in the picture in step S102 includes:
setting a window W (x, y) with a fixed size in the picture, wherein the pixel gray value of the window W (x, y) is I (x, y); setting a Gaussian kernel function omega (x, y) as a window function of W (x, y) and representing the weight of each pixel in the window;
the window is moved in the x direction and the y direction to obtain micro displacement u and micro displacement v, the pixel gray value corresponding to the new position of the window after movement is I (x + u, y + v), and a change value [ I (x + u, y + v) -I (x, y) ] of the gray value and a change value E (u, v) of the gray value are obtained;
wherein E (u, v) ═ Σ(x,y)ω(x,y)[I(x+u,y+v)-I(x,y)]2；
And (5) expanding the I (x + u, y + v) by using a Taylor formula to obtain: i (x + u, y + v) ≈ I (x, y) + uIx+vIyIn which Ix、IyRespectively representing gradient values of image gray in x and y directions;
calculate E (u, v), and matrix M, respectively:
using eigenvalues lambda of the matrix M1、λ2Calculating a characteristic point response function R corresponding to each window; setting a threshold τcIf R corresponding to the pixel point meets the condition: r ═ min (lambda)1,λ2)>τcThen the feature point is set as one of the points of the lattice, and the feature point image coordinates are extracted.
3. The camera calibration method according to claim 2, wherein: after the image coordinates of the feature points are extracted, the image coordinates of all the feature points are sorted from left to right and from top to bottom according to the point at the top left corner as a starting point.
4. The camera calibration method according to claim 1, wherein: in step S103, calibrating the camera by using a zhangying calibration method to obtain a first calibration parameter, including:
calculating a camera internal reference matrix K according to the homography matrix;
calculating a camera external parameter matrix [ R, T ] according to the camera internal parameter matrix K;
optimizing the camera internal parameters and the camera external parameter matrix according to the reprojection error of the minimized point of the maximum likelihood estimation method;
obtaining a distortion coefficient k of the camera according to a reprojection error of the minimum point of the maximum likelihood estimation method1,k2：
In the above formula, MijFor the three-dimensional world coordinate, m, of the jth point of the marking plate in the ith drawingijIts image coordinates; r is a rotation matrix, T is a translation vector,
representing an optimization function.
5. The camera calibration method according to claim 1, wherein: the step S106 is to calibrate the camera according to the eliminated dot matrix image coordinates to obtain a third calibration parameter, if the calibration precision is higher than the second calibration parameter, the step S105 is repeated, otherwise, the third calibration parameter is output, and the method comprises the following steps:
s107: acquiring a second reprojection error by using all the feature points and the second calibration parameter before abnormal feature point elimination;
s108: acquiring a third reprojection error by using all the characteristic points subjected to abnormal characteristic point rejection and the second calibration parameter;
s109: and comparing the sizes of the first re-projection error, the second re-projection error and the third re-projection error, if the third re-projection error is smaller than the second re-projection error and the second re-projection error is smaller than the first re-projection error, determining that the precision of the second calibration parameter is higher than that of the first calibration parameter, and repeating the step S105.
6. The camera calibration method according to claim 1, wherein: in step S106, calculating a homography matrix by using the second calibration parameters, transforming the dot matrix image, obtaining coordinates of the transformed dot matrix image, completing calibration of the camera, and outputting a result, including:
the homography matrix formula is:
wherein the content of the first and second substances,
indicated as second calibration parameter.
7. A camera calibration device is characterized in that: the method comprises the following steps:
an acquisition module: the system is used for acquiring pictures shot by the camera on the calibration board;
an extraction module: the system is used for extracting dot matrix image coordinates in the picture;
a calibration module: the camera calibration method comprises the steps of calibrating a camera by using a Zhang friend calibration method to obtain a first calibration parameter;
the lattice high-precision extraction module: the calibration module is used for calculating a homography matrix by using the first calibration parameters, transforming the dot matrix image, obtaining the coordinates of the transformed dot matrix image, completing the calibration of the camera and outputting second calibration parameters
An abnormal point eliminating module: the first reprojection error of each characteristic point in the dot matrix image is calculated according to the first calibration parameter, and the two-dimensional coordinates of the abnormal characteristic points and the corresponding three-dimensional coordinates of the abnormal characteristic points are eliminated;
a judgment and output module: and the camera calibration is carried out according to the rejected dot matrix image coordinates to obtain a third calibration parameter, if the calibration precision is higher than the second calibration parameter, the abnormal point rejection module process is repeated, and if not, the third calibration parameter is output.
8. Camera calibration device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor realizes the steps of the method according to any of the claims 1-6 when executing the computer program.
9. A computer-readable medium, in which a computer program is stored which, when being processed and executed, carries out the steps of the method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210037926.2A CN114463437A (en) | 2022-01-13 | 2022-01-13 | Camera calibration method, device, equipment and computer readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210037926.2A CN114463437A (en) | 2022-01-13 | 2022-01-13 | Camera calibration method, device, equipment and computer readable medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114463437A true CN114463437A (en) | 2022-05-10 |
Family
ID=81409484
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210037926.2A Pending CN114463437A (en) | 2022-01-13 | 2022-01-13 | Camera calibration method, device, equipment and computer readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114463437A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116773148A (en) * | 2023-06-21 | 2023-09-19 | 德特克(上海)数字技术有限责任公司 | Vision-based laser projection system accuracy verification method |
| CN117593378A (en) * | 2023-11-01 | 2024-02-23 | 誉昊光电科技(珠海)有限公司 | Device and method for calibrating internal parameters of vehicle-mounted camera module |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102800096A (en) * | 2012-07-19 | 2012-11-28 | 北京航空航天大学 | Robustness estimation algorithm of camera parameter |
| WO2020062434A1 (en) * | 2018-09-30 | 2020-04-02 | 初速度（苏州）科技有限公司 | Static calibration method for external parameters of camera |
| CN113838138A (en) * | 2021-08-06 | 2021-12-24 | 杭州灵西机器人智能科技有限公司 | System calibration method, system, device and medium for optimizing feature extraction |
-
2022
- 2022-01-13 CN CN202210037926.2A patent/CN114463437A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102800096A (en) * | 2012-07-19 | 2012-11-28 | 北京航空航天大学 | Robustness estimation algorithm of camera parameter |
| WO2020062434A1 (en) * | 2018-09-30 | 2020-04-02 | 初速度（苏州）科技有限公司 | Static calibration method for external parameters of camera |
| CN113838138A (en) * | 2021-08-06 | 2021-12-24 | 杭州灵西机器人智能科技有限公司 | System calibration method, system, device and medium for optimizing feature extraction |
Non-Patent Citations (2)
| Title |
|---|
| 宋丽梅等: "机器视觉与机器学习", 31 May 2020, 机械工业出版社, pages: 105 - 107 * |
| 马哈迪·雷猜等: "驾驶辅助***计算机视觉技术", 31 July 2020, 机械工业出版社, pages: 134 - 135 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116773148A (en) * | 2023-06-21 | 2023-09-19 | 德特克(上海)数字技术有限责任公司 | Vision-based laser projection system accuracy verification method |
| CN117593378A (en) * | 2023-11-01 | 2024-02-23 | 誉昊光电科技(珠海)有限公司 | Device and method for calibrating internal parameters of vehicle-mounted camera module |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111145238B (en) | Three-dimensional reconstruction method and device for monocular endoscopic image and terminal equipment | |
| CN107633536B (en) | Camera calibration method and system based on two-dimensional plane template | |
| CN108564617B (en) | Three-dimensional reconstruction method and device for multi-view camera, VR camera and panoramic camera | |
| CN114549652A (en) | Camera calibration method, device, equipment and computer readable medium | |
| CN110276808B (en) | Method for measuring unevenness of glass plate by combining single camera with two-dimensional code | |
| CN109416744B (en) | Improved camera calibration system, object and process | |
| CN106651942B (en) | Three-dimensional rotating detection and rotary shaft localization method based on characteristic point | |
| CN110717942B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| US9767383B2 (en) | Method and apparatus for detecting incorrect associations between keypoints of a first image and keypoints of a second image | |
| CN114463437A (en) | Camera calibration method, device, equipment and computer readable medium | |
| CN109345597B (en) | Camera calibration image acquisition method and device based on augmented reality | |
| CN113409391B (en) | Visual positioning method and related device, equipment and storage medium | |
| CN113920205B (en) | Calibration method of non-coaxial camera | |
| CN108182708B (en) | Calibration method and calibration device of binocular camera and terminal equipment | |
| CN112686950B (en) | Pose estimation method, pose estimation device, terminal equipment and computer readable storage medium | |
| CN111028205A (en) | Eye pupil positioning method and device based on binocular ranging | |
| CN111627073B (en) | Calibration method, calibration device and storage medium based on man-machine interaction | |
| CN112489141B (en) | Production line calibration method and device for single-board single-image strip relay lens of vehicle-mounted camera | |
| CN111327892B (en) | Intelligent terminal multi-camera static imaging analysis force testing method and device | |
| CN111462246B (en) | Equipment calibration method of structured light measurement system | |
| CN110458951B (en) | Modeling data acquisition method and related device for power grid pole tower | |
| Fasogbon et al. | Calibration of fisheye camera using entrance pupil | |
| CN114897990A (en) | Camera distortion calibration method and system based on neural network and storage medium | |
| CN113920196A (en) | Visual positioning method and device and computer equipment | |
| CN110874820B (en) | Material simulation deformation data acquisition method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |