CN116652970B - Four-axis mechanical arm 2D hand-eye calibration method and system and storage medium - Google Patents
Four-axis mechanical arm 2D hand-eye calibration method and system and storage medium Download PDFInfo
- Publication number
- CN116652970B CN116652970B CN202310941577.1A CN202310941577A CN116652970B CN 116652970 B CN116652970 B CN 116652970B CN 202310941577 A CN202310941577 A CN 202310941577A CN 116652970 B CN116652970 B CN 116652970B
- Authority
- CN
- China
- Prior art keywords
- mechanical arm
- hand
- pose
- eye
- steps
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 99
- 239000011159 matrix material Substances 0.000 claims abstract description 63
- 238000013519 translation Methods 0.000 claims abstract description 22
- 238000000354 decomposition reaction Methods 0.000 claims abstract description 11
- 238000012545 processing Methods 0.000 claims description 11
- 238000005457 optimization Methods 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 6
- 230000000007 visual effect Effects 0.000 abstract description 11
- 238000009434 installation Methods 0.000 abstract description 4
- 238000004364 calculation method Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 230000009466 transformation Effects 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 5
- 238000006243 chemical reaction Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 1
- 238000012897 Levenberg–Marquardt algorithm Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011426 transformation method Methods 0.000 description 1
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
- B25J9/1692—Calibration of manipulator
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
- B25J19/0095—Means or methods for testing manipulators
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Image Processing (AREA)
- Manipulator (AREA)
Abstract
The application provides a four-axis mechanical arm 2D hand-eye calibration method and system and a storage medium, wherein the method comprises the following steps: the hand-eye matrix relationRespectively decomposed into rotation matrix relationsTranslation component relationThe method comprises the steps of carrying out a first treatment on the surface of the At an angular distanceDefining the distance between the poses of the two rotation matrix representations willAs a scalar, an optimized objective function is establishedThe method comprises the steps of carrying out a first treatment on the surface of the Through minimization ofTo solve for rotational component by value of (2)Acquisition ofThree-direction gestures; decomposition based on SVD methodTo solve for the translational component inValues. Therefore, the hand-eye pose relation with five degrees of freedom is obtained, and the visual grabbing mode of the four-axis SCARA mechanical arm is not limited by the fixed installation angle of the camera and the fixed height of the grabbing plane.
Description
Technical Field
The application relates to a robot hand-eye calibration technology, in particular to a four-axis mechanical arm 2D hand-eye calibration method and system and a storage medium.
Background
At present, visual identification plays a vital role in a scene that a mechanical arm and an object have relative position movement. One of the key factors for the visual grabbing of the mechanical arm is the calibration of the 'hand-eye relation', namely the conversion relation from the camera to the tail end of the mechanical arm.
The mounting modes of the cameras in the hand-eye system can be divided into two cases: the eye is on the hand (eye in hand) and the eye is off the hand (eye on hand). In order to determine the hand-eye relationship, a common method is a "nine-point calibration method", that is, a conversion relationship between a camera and a mechanical arm is calculated by pixel coordinates of nine points on a calibration plate in an image and physical coordinates marked by a mechanical arm base. The nine-point method has the characteristic of dimensional fixation, and is only suitable for the condition that the grabbing plane and the camera plane are relatively fixed, in other words, the pose relation of 6 degrees of freedom in the hand-eye relationThe nine-point method can only calculate three +.>Therefore, the applicable scene is greatly limited.
To calibrate the complete 6 degree of freedom hand-eye relationship, it is generally considered necessary to construct a secondary transformation matrix from rotation and translation, based on the transformation relationship between the base coordinates, camera, end-grip, calibration plateAnd (3) a relational expression, namely acquiring a plurality of groups of data, and calculating a hand-eye matrix by using a linear or nonlinear algorithm. But this method requires that the end pose, which contains at least two non-parallel rotational movements, be solved (Tsai and Lenz 1989,A New Technique for Fully Autonomous and Efficient 3D Robotics Hand/Eye registration).
In the four-axis SCARA robot arm, the rotation axes are all parallel, so that the hand-eye relationship cannot be solved by the method.
For a four-axis SCARA mechanical arm, various hand-eye calibration schemes have been proposed in the prior art, such as "high-precision method for hand-eye calibration of a SCARA mechanical arm fixed camera vision system" (Chinese patent publication No. CN 109159114A), "a hand of a four-axis mechanical armThe eye calibration method and device (Chinese patent publication No. CN 116038701A) and the like all provide a calibration method for the four-axis mechanical arm on the basis of nine-point method, but still only calculate the hand-eye relationshipThe three degrees of freedom relation limits that the camera plane and the grabbing plane in the application scene are parallel, and the grabbing height is relatively fixed, so that the application scene is limited.
Disclosure of Invention
Therefore, the main purpose of the application is to provide a four-axis mechanical arm 2D hand-eye calibration method, a system and a storage medium, so as to realize that the hand-eye pose relation of five degrees of freedom can be calculated at least, and the visual grabbing mode of the four-axis SCARA mechanical arm is not limited by the fixed installation angle of a camera and the height of a grabbing plane.
In order to achieve the above object, according to a first aspect of the present application, there is provided a four-axis mechanical arm 2D hand-eye calibration method, comprising the steps of:
step S100, hand-eye matrix relationRespectively decomposed into rotation matrix relations>Translation component relation->;
Step S200 is performed at an angular distanceDefining the distance between the poses of the two rotation matrix representations, will +.>As a scalar, an optimized objective function is created +.>The method comprises the steps of carrying out a first treatment on the surface of the To pass throughMinimize->To solve for the rotation component>, wherein />For the rotational component of the hand-eye matrix X, comprising +.>Three-direction gestures;
step S300 is based on SVD method decompositionTo solve for the translational component of +.>Values.
In a possible preferred embodiment, the four-axis mechanical arm 2D hand-eye calibration method, wherein the angular distance in step S200
。
wherein ,for the transpose of matrix Q, tr denotes the trace of the matrix,/->;
While whenWhen in use, then->。
In a possibly preferred embodiment, wherein in step S300The solving step of (1) comprises:
step S310 pairSingular value decomposition into->The method comprises the steps of carrying out a first treatment on the surface of the Combining the translation component relation, converting into +.>;
Step S320 orderAfter SVD decomposition, S is a diagonal matrix composed of singular values:thus->To obtain +.>Values.
In a possible preferred embodiment, the four-axis mechanical arm 2D hand-eye calibration method further includes:
step S400 defines error termsThe method comprises the steps of carrying out a first treatment on the surface of the Establishing an optimized objective function->Performing iterative optimization solution to optimize +.>Value of>Is a preset fixed value;
wherein W is an adjustment parameter of the influence caused by the difference of the rotation amount and the data dimension of the translation vector in the balance matrix E.
In a possible preferred embodiment, the four-axis mechanical arm 2D hand-eye calibration method further includes:
step S500, moving the tail end of the mechanical arm tool to the point of point 1, so that the camera can clearly observe the calibration plate, and acquiring the point 1 corresponding to the point and />The method comprises the steps of carrying out a first treatment on the surface of the Moving the end of the mechanical arm tool to the origin of the calibration plate, denoted as ose2, and obtaining +.>;
Step S600 of recording the pose changes of the phase 1 to the phase 2 asThe method comprises the steps of carrying out a first treatment on the surface of the Changing the actual z-axis of the robot tool tip from post 1 to post 2 to +.>Is denoted by>Whereas phase 1 to phase 2, the z-axis in the camera is transformed to +.>Is denoted by>The method comprises the steps of carrying out a first treatment on the surface of the Calculating +.>。
In a possibly preferred embodiment, in step S100, the hand-eye matrix relation is,, wherein />Respectively representing the pose data of the tail ends of two groups of mechanical arm tools +.>And->Respectively representing pose data of two groups of calibration plates under a camera coordinate system>。
In a possibly preferred embodiment, whereinThe step of acquiring each set of data of (a) includes:
step S110, the mechanical arm is moved, so that the camera can clearly capture the picture of the calibration plate, and the pose of the tail end of the mechanical arm tool at the moment is recorded asCollecting corresponding calibration plate images;
step S120, repeatedly moving the mechanical arm, collecting n groups of data, and recording the pose of the tool tail end of the mechanical armThe method comprises the following steps:。
in a possibly preferred embodiment, whereinThe calculating step of each group of data of (a) comprises:
step S130 establishes a projection relationship from the world coordinate system to the pixel coordinate system:
wherein For pixel coordinate system coordinates +.>Is an internal reference matrix of a camera>Representing the pose of the calibration plate to the camera coordinate system, and uniformly matrix +.>Marked as->,/>Representing coordinates in the world coordinate system, +.>Representing the depth of the feature points in a camera coordinate system;
step S140 solves the optimal parameters by minimizing the re-projection error method:
Meanwhile, each calibration plate image acquired in the step S110 corresponds to a pose under a camera coordinate system, and is marked as:
。
in order to achieve the above object, corresponding to the above method, according to a second aspect of the present application, there is also provided a four-axis mechanical arm 2D hand-eye calibration system, comprising:
the storage unit is used for storing a program comprising the steps of the four-axis mechanical arm 2D hand-eye calibration method, so that the control unit and the processing unit can timely adjust and execute the program;
the control unit is used for controlling the mechanical arm to enable the camera to clearly capture the picture of the calibration plate, and recording the pose of the tail end of the mechanical arm at the momentAnd acquiring images of corresponding calibration plates, repeatedly moving the mechanical arm, acquiring n groups of data, and recording the terminal pose as;
The processing unit is used for calculating the pose of the calibration plate under the camera coordinate system according to the camera internal parametersSimultaneously, each acquired calibration plate image corresponds to a pose under a camera coordinate system and is recorded asThe method comprises the steps of carrying out a first treatment on the surface of the The hand-eye matrix relation is->Respectively decomposed into rotation matrix relations>Translation component relation->; wherein />,/>, wherein />Respectively representing the pose data of the tail ends of two groups of mechanical arm tools +.>And->Respectively representing pose data of two groups of calibration plates under a camera coordinate system>The method comprises the steps of carrying out a first treatment on the surface of the At an angular distance +>Defining the distance between the poses of the two rotation matrix representations, will +.>As a scalar, an optimized objective function is created +.>The method comprises the steps of carrying out a first treatment on the surface of the By minimizing +.>To solve for the rotation component>, wherein />For the rotational component of the hand-eye matrix X, comprising +.>Three-direction gestures; when->When in use, thenThe method comprises the steps of carrying out a first treatment on the surface of the Then decompose ∈10 based on SVD method>To solve for the translational component of +.>Values.
In order to achieve the above object, according to a third aspect of the present application, there is further provided a computer readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the four-axis mechanical arm 2D hand-eye calibration method as described in any one of the above.
The four-axis mechanical arm 2D hand-eye calibration method, the system and the storage medium provided by the application can skillfully decompose the hand-eye matrix relation into the rotation matrix relation and the translation component relation so as to respectively calculate and take out the hand-eye matrix relation) (at->) Five degrees of freedom are adopted, so that the calibration scheme of the four-axis SCARA mechanical arm in the prior art is broken through, and only the calculation can be performedThree degrees of freedom relation.
Furthermore, in the application of the four-axis SCARA mechanical arm, due toThe value of (2) is usually set to be a fixed value, so that in theory, the complete 6-degree-of-freedom hand-eye pose relationship can be obtained, but in order to adapt to some special visual application modes, the application also provides a calculation method>The proposal of the scheme ensures that the scheme can completely calculate the hand-eye pose relation with 6 degrees of freedomThe visual grabbing mode of the four-axis SCARA mechanical arm is not affected by the installation angle of the camera and the grabbing plane heightFixed, etc. Thereby expanding the application range of the four-axis SCARA mechanical arm in industrial scenes.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:
FIG. 1 is a schematic diagram of steps of a four-axis mechanical arm 2D hand-eye calibration method of the present application;
FIG. 2 is a schematic diagram of a four-axis SCARA robot in the prior art;
FIG. 3 is a diagram showing the data conversion relationship between the ROBOT base coordinate system ROBOT and the calibration plate coordinate system WORLD in different poses in the four-axis mechanical arm 2D hand-eye calibration method of the present application;
fig. 4 is a schematic structural diagram of a four-axis mechanical arm 2D hand-eye calibration system according to the present application.
Detailed Description
In order that those skilled in the art can better understand the technical solutions of the present application, the following description will clearly and completely describe the specific technical solutions of the present application in conjunction with the embodiments to help those skilled in the art to further understand the present application. It will be apparent that the embodiments described herein are merely some, but not all embodiments of the application. It should be noted that embodiments of the present application and features of embodiments may be combined with each other by those of ordinary skill in the art without departing from the spirit of the present application and without conflicting with each other. All other embodiments, which are derived from the embodiments herein without creative effort for a person skilled in the art, shall fall within the disclosure and the protection scope of the present application.
Furthermore, the terms "first," "second," "S100," "S200," and the like in the description and in the claims and drawings are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented in sequences other than those described herein. Also, the terms "comprising" and "having" and any variations thereof herein are intended to cover a non-exclusive inclusion. Unless specifically stated or limited otherwise, the terms "disposed," "configured," "mounted," "connected," "coupled" and "connected" are to be construed broadly, e.g., as being either permanently connected, removably connected, or integrally connected; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the terms in this case will be understood by those skilled in the art in view of the specific circumstances and in combination with the prior art.
According to the theory of robot kinematics and robot dynamics, the four-axis SCARA mechanical arm is generally designed into a structure shown in fig. 2, wherein an axis 1 is a vertical lifting axis, axes 2, 3 and 4 are axes of rotation around Z, wherein the axis 4 is an axis where a terminal flange is located, and a clamping tool is fixed on the axes. Due to the characteristics of the four-axis SCARA mechanical arm, the space pose which can be represented by the four-axis SCARA mechanical arm has only four degrees of freedom, namely x, y, z and Rz. That is, the gripping attitude of the four-axis SCARA robot arm can only rotate around the Z axis except for the spatial position coordinates (X, Y, Z), and the rotation around the X axis and the Y axis cannot be realized.
Further, the pose of the rigid body in the three-dimensional space can be represented by rotation R and translation t, and the rotation and translation of the rigid body motion are usually represented by a homogeneous transformation matrix H, and in the hand-eye calibration process of the six-axis articulated mechanical arm, if the eye is on hand (eye in hand), the calibration plate is in the position relationship under the calibration of the mechanical arm baseRepresented by the chained homogeneous coordinate transformation as:
if the eye is out of hand (eye on hand), the calibration is fixed at the end of the mechanical arm, so the calibration plate and the end are fixedPose relationshipThe chain homogeneous coordinate transformation in the calibration process is kept unchanged and is as follows:
wherein For the pose of the mechanical arm tool end under the base standard system, +.>Representing the pose of the base coordinates in the robot arm end tool coordinate system, obviously +.>The method comprises the steps of carrying out a first treatment on the surface of the The pose data can be read directly from the robotic arm,representing the pose relation between the camera coordinate system and the tool tail end coordinate system, namely, a hand-eye matrix when the eye in hand needs to be solved; />The pose relation between the camera coordinate system and the base standard system is represented, and the pose relation is also the hand-eye relation when the eye on hand needs to be solved; />The pose of the calibration plate under the camera coordinate system is represented and can be obtained by a calibration plate identification algorithm.
Since there is in the above and />(or-> and />) Two unknown quantities, to be determined isOr->At the same time in view of->Or->Is fixed (because the position relation between the calibration plate and the mechanical arm base mark is fixed in the calibration process or the position relation between the calibration plate and the end tool is fixed), the calibration process needs to collect a plurality of groups of mechanical arm end data and calibration plate identification data, and eliminate->Or->Constructing an equation to solve +.>Or (b)。
For this reason, as shown in fig. 1, the application provides a four-axis mechanical arm 2D hand-eye calibration method, which comprises the following steps:
step S100
Assuming that the ROBOT base standard system is ROBOT, the calibration plate coordinate system is WORLD, and the hand-eye matrix relation can be established according to the data conversion relation of different poses as shown in figure 3
。
wherein ,,/>,/>mechanical arm tail end pose data respectively representing the first group data and the second group data>,/>Pose data of calibration plates respectively for first group data and second group data under camera coordinate system,/>For hand-eye relation to be solved->. A, B, X are homogeneous matrices of rigid body pose transformation:
specifically, according to the different mounting modes of the camera, the mechanical arm visual grabbing can be divided into eyes on the hand and eyes outside the hand. When the eyes are on the hands, the camera can be arranged at the tail end of the mechanical arm, so that the calibration plate and the base calibration position are ensured to be relatively fixed; when eyes are outside hands, the camera can be fixedly installed in the environment, and the calibration plate can be installed at the tail end of the mechanical arm so as to ensure that the positions of the camera and the tail end of the mechanical arm are relatively fixed. The manner of data acquisition is the same for both cases.
Before the subsequent calculation, if the camera internal parameters are not determined, the camera internal parameters can be calibrated first. Internal parameters of cameraIs an inherent attribute of a camera, consisting ofFocal length +.>Origin offset->And a series of distortion coefficients, wherein the internal parameters are fixed after the focal length and the focusing ring of the camera are fixed. According to a camera aperture imaging model:
。
wherein ,is pixel coordinate point +.>Is a spatial coordinate point. The internal reference calibration process usually needs to use specific calibration objects, and the commonly used calibration objects are calibration boards with specific patterns, such as checkerboard calibration boards, round calibration boards, churco, apriltag grid and other various forms.
The calibration plate being a plane in three-dimensional spaceObtaining an imaging plane according to the projection imaging relationship shown in the above formula>The homography matrix between the two planes can be determined by the coordinates of the corresponding points of the two planes>. The specification and dimensions on the calibration plate are known, and the corner points in the image can be calculated by corner point detectionExtracting by the method according to the formula ∈>, wherein />For the coordinates of points in the image, +.>For calibrating the point coordinates on the plate, +.>The homography matrix representing two planes is changed, the pose relation between the camera and the calibration plate is changed, a plurality of calibration plate images are collected, a plurality of groups of homography matrixes can be obtained, and the +_ is solved through least square>Will->The camera parameter K can be obtained by decomposition.
Further, whereinThe data acquisition step of each group of (a) includes:
step S110, the mechanical arm is moved, so that the camera can clearly capture the picture of the calibration plate, and the pose of the tail end of the mechanical arm tool at the moment is recorded asAnd acquiring the corresponding calibration plate image.
The calibration board in this example may be configured as a pattern with a specific specification, including but not limited to a checkerboard calibration board, a round calibration board, a churco, an Apriltag grid, and other forms, and as long as a scheme of obtaining the pose relationship from the calibration object to the camera can be within the scope of the disclosure of the calibration board.
Step S120, repeatedly moving the mechanical arm and collecting n groups of dataRecording the pose of the tool tail end of the mechanical arm>The method comprises the following steps:
i.e. n sets of data, by combining two by two to form one A, e.g。
Further, whereinThe calculating step of each group of data of (a) comprises:
step S130, in case the camera parameters are known, the calculation is exemplified using PnP (Persponsive-n-Point) algorithm. The solving modes of the PnP algorithm are various, and a direct linear transformation method (Direct Linear Transform, DLT), an EPnP method and a minimum reprojection error method are common, and the example takes the minimum reprojection error method as an example to solve the position and the pose of the calibration plate.
For example: according to the principle of pinhole imaging, a projection relation from a world coordinate system to a pixel coordinate system is established:
wherein For pixel coordinate system coordinates +.>Is an internal reference matrix of a camera>Representing the pose of the calibration plate to the camera coordinate system, and uniformly matrix +.>Marked as->,/>Representing coordinates in the world coordinate system, +.>Representing the depth of the feature points in a camera coordinate system;
step S140 solves the optimal parameters by minimizing the re-projection error method:
/>
Meanwhile, each calibration plate image acquired in the step S110 corresponds to a pose under a camera coordinate system, and is marked as:
i.e. n sets of data, generating a B, e.g. B, by combining two by two。
Further, either the eye in hand or the eye on hand may be represented by the formulaThe method comprises the following steps of:
/>
/>
as can be seen from (2) and (3), the rotation matrix R is independent of the translation component and can be calculated independently, and the translation vectorThen in relation to rotation, it is necessary to rely on +.>Is calculated. For the 6-joint mechanical arm, the mechanical arm can freely rotate around x, y and z axes in space, so that the hand-eye matrix can be directly calculated according to (2) and (3)>。
Step S200 calculates a rotation component。
Specifically, the rotation component according to expression (2) can be calculated alone,are all rotation matrices representing the rotational pose of a rigid body in space, wherein +.>Representing the rotational component of the hand-eye matrix X, comprising +.>Three-directional gestures. WhileThe rotation matrix representation on the left and right sides of the representation equation is consistent in terms of pose.
At this time, angular distanceRepresenting the distance between the poses represented by the two rotation matrices, letAngular distance>The definition is as follows:
/>
wherein ,for the transpose of matrix Q, tr denotes the trace of the matrix (sum of matrix eigenvalues). />Is a scalar by minimizing +.>To calculate the rotation component +.>. The optimized objective function is:
/>
then an example uses the Levenberg-Marquardt algorithm for iterative optimization solutionCan be calculated to obtainAs a result of (a), let->When in use, then->。
Step S300 calculation
Specifically, the following equation (3) is obtainedIn the case of (1) calculate->Due to->Rank of 2 (wherein +.>A constant in linear algebra, corresponding to 1 in real numbers, is the identity matrix, thus ∈1)>The solution of (1) is not unique and can not be directly solved, and the application provides a translation component solving scheme based on SVD (singular value decomposition)>Singular value decomposition is performed:
/>
the combination of formulas (3) (9) can be obtained:
/>
the right side of formula (10) is a known amount, let:
/>
after SVD decomposition, S is a diagonal matrix composed of singular values:
/>
by combining (10) (11) (12), it is possible to obtain:
/>
can be obtained according to the formula (13)For n sets of data acquired, each set of data can be evaluated as a set +.>Value, calculate n data sets +.>As a final result, the accuracy and robustness of the result may be increased.
Step S400 optimizationValue of
Since the present example is to solve for rotation and translation in two stepsThe calculation process is affected by the accumulated error of Rx, resulting in reduced accuracy. For this purpose, the present example proposes to use nonlinear optimization to fine tune the calculation result, further improve the precision of the calibration result, and the steps include:
defining an error term E as:
/>
the optimized objective function is:
/>
wherein The role of W is to balance the effect of the difference in the data dimensions of the rotation vector and the translation vector in matrix E (9 for the rotation matrix and 3 for the translation vector).
While all n sets of data are illustrated as being optimized by Levenberg-Marquardt, those skilled in the art will appreciate that all of the non-linear optimization algorithms referred to in this example may be substituted by, for example, gaussian-newton, steepest descent, etc.
In addition becauseCannot be determined, so that the whole optimization process needs to be kept +.>The preset fixed value is adopted.
Further, since in a four-axis SCARA robot application,the value of (2) is usually set to a fixed value, but for some special visual applications +.>The value of (2) must be accurate.
Thus, the present application also exemplifies a confirmation in order to obtain a complete external referenceThe values were as follows:
step S500, moving the tail end of the mechanical arm tool to the point of point 1, so that the camera can clearly observe the calibration plate, and acquiring the point 1 corresponding to the point and />The method comprises the steps of carrying out a first treatment on the surface of the Moving the end of the mechanical arm tool to the origin of the calibration plate, denoted as ose2, and obtaining +.>。
Step S600 of recording the pose changes of the phase 1 to the phase 2 asThe method comprises the steps of carrying out a first treatment on the surface of the Changing the actual z-axis of the robot tool tip from post 1 to post 2 to +.>Is denoted by>The method comprises the steps of carrying out a first treatment on the surface of the Whereas the z-axis in the camera is transformed to +.>Is denoted by>The method comprises the steps of carrying out a first treatment on the surface of the Whereby the +.about.in the hand-eye relationship can be calculated>Thereby determining +.>Values.
In practical implementation, the operation difficulty of overlapping the tail end of the mechanical arm with the origin of the calibration plate is relatively high, and as an alternative method, the tail end of the mechanical arm can be moved to a point right above the origin of the calibration plate or right below the origin of the calibration plate to a known height, and then the known height is compensated in the calculation process.
Thus, through the above example steps, the six degrees of freedom hand-eye pose relationship of the complete four-axis SCARA mechanical arm can be calculated. The four-axis SCARA mechanical arm is free from the influence of factors such as a camera mounting mode, a clamping jaw mounting mode and the like in the visual grabbing application.
On the other hand, corresponding to the four-axis mechanical arm 2D hand-eye calibration method of the above example, as shown in fig. 4, the present application further provides a four-axis mechanical arm 2D hand-eye calibration system, which includes:
the storage unit is used for storing a program comprising the steps of the four-axis mechanical arm 2D hand-eye calibration method, so that the control unit and the processing unit can timely adjust and execute the program;
the control unit is used for controlling the mechanical arm to enable the camera to clearly capture the picture of the calibration plate, and recording the pose of the tail end of the mechanical arm at the momentAnd acquiring images of corresponding calibration plates, repeatedly moving the mechanical arm, acquiring n groups of data, and recording the terminal pose as;
A processing unit for calculating a target according to the camera internal parametersPose of fixed plate under camera coordinate systemSimultaneously, each acquired calibration plate image corresponds to a pose under a camera coordinate system and is recorded as。
The processing unit also uses the hand-eye matrix relationRespectively decomposed into rotation matrix relationsTranslation component relation->; wherein />,/>, wherein />Respectively representing the pose data of the tail ends of two groups of mechanical arm tools +.>And->Respectively representing pose data of two groups of calibration plates under a camera coordinate system>The method comprises the steps of carrying out a first treatment on the surface of the And then>Defining the distance between the poses of the two rotation matrix representations, will +.>As a scalar, an optimized objective function is established:
through minimization ofTo solve for the rotation component>, wherein />Is a rotation component of the hand-eye matrix X, and comprisesThree-directional gestures.
Wherein whenWhen in use, then->。
The processing unit then also decomposes based on SVDTo solve for the translational component of +.>Values.
Further, when it is required to doWhen the value is optimized, the processing unit also passes through an optimized objective function:
the Levenberg-Marquardt was used for all n sets of dataOptimizing, thereby completing the pair ofOptimization of the values.
On the other hand, whenWhen the value of (2) must be determined, the control unit is further configured to move the end of the mechanical arm tool to the point of point 1, so that the camera can clearly observe the calibration board, and obtain +_corresponding to point 1> and />The method comprises the steps of carrying out a first treatment on the surface of the Moving the end of the mechanical arm tool to the origin of the calibration plate, denoted as ose2, and obtaining +.>。
The processing unit is also used for recording the pose changes of the pos 1 to the pos 2 asThe method comprises the steps of carrying out a first treatment on the surface of the Changing the actual z-axis of the robot tool tip from post 1 to post 2 to +.>Is denoted by>Whereas phase 1 to phase 2, the z-axis in the camera is transformed to +.>Is denoted by>To calculate the +.>Thereby determining +.>Values.
On the other hand, the application also provides a computer readable storage medium, corresponding to the method, wherein the computer program is stored thereon, and when the computer program is executed by a processor, the steps of the four-axis mechanical arm 2D hand-eye calibration method are realized.
Finally, the four-axis mechanical arm 2D hand-eye calibration method, the system and the storage medium provided by the application skillfully decompose the hand-eye matrix relational expression into the rotation matrix relational expression and the translation component relational expression so as to respectively calculate and take out the hand-eye matrix relational expression and the translation component relational expression) (at->) Five degrees of freedom are adopted, so that the calibration scheme of the four-axis SCARA mechanical arm in the prior art is broken through, and only the calculation can be performedThree degrees of freedom relation.
Furthermore, in the application of the four-axis SCARA mechanical arm, due toThe value of (2) is usually set to be a fixed value, so that in theory, the complete 6-degree-of-freedom hand-eye pose relationship can be obtained, but in order to adapt to some special visual application modes, the application also provides a calculation method>The proposal of the scheme ensures that the scheme can completely calculate the hand-eye pose relation with 6 degrees of freedomThe visual grabbing mode of the four-axis SCARA mechanical arm is not limited by scenes such as camera installation angles, fixed grabbing plane heights and the like. Thereby expanding the application range of the four-axis SCARA mechanical arm in industrial scenes.
The preferred embodiments of the application disclosed above are intended only to assist in the explanation of the application. The preferred embodiments are not exhaustive or to limit the application to the precise form disclosed. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the application and the practical application, to thereby enable others skilled in the art to best understand and utilize the application. The application is to be limited only by the following claims and their full scope and equivalents, and any modifications, equivalents, improvements, etc., which fall within the spirit and principles of the application are intended to be included within the scope of the application.
It will be appreciated by those skilled in the art that the system, apparatus and various modules/units thereof provided by the application may be implemented entirely by logic programming method steps, in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc., in addition to the system, apparatus and various modules/units thereof provided by the application being embodied in pure computer readable program code. Therefore, the system, the apparatus, and the respective modules/units thereof provided by the present application may be regarded as one hardware component, and the modules/units included therein for implementing various programs may also be regarded as structures within the hardware component; modules/units for implementing various functions may also be considered as either software programs for implementing the methods or structures within hardware components.
Furthermore, all or part of the steps in implementing the methods of the embodiments described above may be implemented by a program, where the program is stored in a storage medium and includes several instructions for causing a single-chip microcomputer, chip or processor (processor) to execute all or part of the steps in the methods of the embodiments of the application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In addition, any combination of various embodiments of the present application may be performed, so long as the concept of the embodiments of the present application is not violated, and the disclosure of the embodiments of the present application should also be considered.
Claims (8)
1. A four-axis mechanical arm 2D hand-eye calibration method comprises the following steps:
step S100, hand-eye matrix relationRespectively decomposed into rotation matrix relations>Translation component relation->; wherein />,/>, wherein />Respectively representing the pose data of the tail ends of two groups of mechanical arm tools +.>And->Respectively representing pose data of two groups of calibration plates under a camera coordinate system>;/>、/>Are all rotation matrixes; />Is a unit matrix;
step S200 is performed at an angular distanceDefining the distance between the poses of the two rotation matrix representations, will +.>As a scalar, an optimized objective function is created +.>The method comprises the steps of carrying out a first treatment on the surface of the By minimizing +.>To solve for the rotation component>, wherein />For the rotational component of the hand-eye matrix X, comprising +.>Three-direction gestures;
step S300 is based on SVD method decompositionTo solve for the translational component of +.>The value step includes:
step S310 pairSingular value decomposition into->The method comprises the steps of carrying out a first treatment on the surface of the Combining the translation component relation to convert into;
Step S320 orderAfter SVD decomposition, S is a diagonal matrix composed of singular values:thus->To obtain +.>Values.
2. The four-axis mechanical arm 2D hand-eye calibration method according to claim 1, wherein the angular distance in step S200
;
wherein ,for the transpose of matrix Q, tr denotes the trace of the matrix,/->;
While whenWhen in use, then->。
3. The four-axis mechanical arm 2D hand-eye calibration method according to claim 1, further comprising the steps of:
step S400 defines error terms, wherein />For the position relation of the calibration plate under the base of the mechanical arm, +.>Pose of the tool end of the mechanical arm under the base standard system, < >>For the pose relation of the camera coordinate system and the base coordinate system, < +.>Representing the pose of the calibration plate under a camera coordinate system; establishing an optimized objective function->Performing iterative optimization solution to optimize +.>Value of>Is a preset fixed value;
;
wherein W is an adjustment parameter of the influence caused by the difference of the rotation amount and the data dimension of the translation vector in the balance matrix E.
4. The four-axis mechanical arm 2D hand-eye calibration method according to claim 1, further comprising the steps of:
step S500, moving the tail end of the mechanical arm tool to the point of point 1, so that the camera can clearly observe the calibration plate, and acquiring the point 1 corresponding to the point and />The method comprises the steps of carrying out a first treatment on the surface of the Moving the tail end of the mechanical arm tool to the origin of the calibration plate, namely, a point 2, and acquiring the point 2;
Step S600 of recording the pose changes of the phase 1 to the phase 2 asThe method comprises the steps of carrying out a first treatment on the surface of the Changing the actual z-axis of the robot tool tip from post 1 to post 2 to +.>Is denoted by>Whereas phase 1 to phase 2, the z-axis in the camera is transformed to +.>Is denoted by>The method comprises the steps of carrying out a first treatment on the surface of the Calculating +.>。
5. The four-axis mechanical arm 2D hand-eye calibration method according to claim 1, whereinThe data acquisition step of each group of (a) includes:
step S110, the mechanical arm is moved, so that the camera can clearly capture the picture of the calibration plate, and the pose of the tail end of the mechanical arm tool at the moment is recorded asCollecting corresponding calibration plate images;
step S120, repeatedly moving the mechanical arm, collecting n groups of data, and recording the pose of the tool tail end of the mechanical armThe method comprises the following steps:。
6. the four-axis mechanical arm 2D hand-eye calibration method according to claim 1, whereinThe calculating step of each group of data of (a) comprises:
step S130 establishes a projection relationship from the world coordinate system to the pixel coordinate system:
;
wherein For pixel coordinate system coordinates +.>Is an internal reference matrix of a camera>Representing the pose of the calibration plate to the camera coordinate system, and uniformly matrix +.>Marked as->,/>Representing coordinates in the world coordinate system,representing the depth of the feature points in a camera coordinate system;
step S140 solves the optimal parameters by minimizing the re-projection error method:
;
Meanwhile, each calibration plate image acquired in the step S110 corresponds to a pose under a camera coordinate system, and is marked as:
。
7. a four-axis robotic arm 2D hand-eye calibration system, comprising:
a storage unit, configured to store a program including the steps of the four-axis mechanical arm 2D hand-eye calibration method according to any one of claims 1 to 6, so that the control unit, the processing unit, and the processing unit can perform timely adjustment;
the control unit is used for controlling the mechanical arm to enable the camera to clearly capture the picture of the calibration plate, and recording the pose of the tail end of the mechanical arm at the momentAnd acquiring images of corresponding calibration plates, repeatedly moving the mechanical arm, acquiring n groups of data, and recording the terminal pose as;
The processing unit is used for calculating the pose of the calibration plate under the camera coordinate system according to the camera internal parametersSimultaneously, each acquired calibration plate image corresponds to a pose under a camera coordinate system and is marked as +.>The method comprises the steps of carrying out a first treatment on the surface of the The hand-eye matrix relation is->Respectively decomposed into rotation matrix relations>Translation component relation; wherein />,/>, wherein />Respectively representing the pose data of the tail ends of two groups of mechanical arm tools +.>And->Respectively representing pose data of two groups of calibration plates under a camera coordinate system>The method comprises the steps of carrying out a first treatment on the surface of the At an angular distance +>Defining the distance between the poses of the two rotation matrix representations, will +.>As a scalar, an optimized objective function is created +.>The method comprises the steps of carrying out a first treatment on the surface of the By minimizing +.>To solve for the rotation component>, wherein />For the rotational component of the hand-eye matrix X, comprising +.>Three-direction gestures; when->When in use, thenThe method comprises the steps of carrying out a first treatment on the surface of the Then decompose ∈10 based on SVD method>To solve for the translational component of +.>Values.
8. A computer readable storage medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the steps of the four-axis robotic arm 2D hand-eye calibration method of any of claims 1 to 6.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310941577.1A CN116652970B (en) | 2023-07-28 | 2023-07-28 | Four-axis mechanical arm 2D hand-eye calibration method and system and storage medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310941577.1A CN116652970B (en) | 2023-07-28 | 2023-07-28 | Four-axis mechanical arm 2D hand-eye calibration method and system and storage medium |
Publications (2)
Publication Number | Publication Date |
---|---|
CN116652970A CN116652970A (en) | 2023-08-29 |
CN116652970B true CN116652970B (en) | 2023-10-31 |
Family
ID=87720964
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202310941577.1A Active CN116652970B (en) | 2023-07-28 | 2023-07-28 | Four-axis mechanical arm 2D hand-eye calibration method and system and storage medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN116652970B (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117182931B (en) * | 2023-11-07 | 2024-01-23 | 上海仙工智能科技有限公司 | Calibration method and system for calibration data of mechanical arm and storage medium |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109483516A (en) * | 2018-10-16 | 2019-03-19 | 浙江大学 | A kind of mechanical arm hand and eye calibrating method based on space length and epipolar-line constraint |
CN111775152A (en) * | 2020-06-29 | 2020-10-16 | 深圳大学 | Method and system for guiding mechanical arm to grab scattered stacked workpieces based on three-dimensional measurement |
CN111823233A (en) * | 2020-06-30 | 2020-10-27 | 浙江德尚韵兴医疗科技有限公司 | Mechanical arm hand-eye calibration system and method based on high-precision three-dimensional optical positioning |
EP4094897A1 (en) * | 2021-05-26 | 2022-11-30 | MVTec Software GmbH | Hand-eye calibration of camera-guided devices |
CN116038701A (en) * | 2022-12-30 | 2023-05-02 | 北京中科原动力科技有限公司 | Hand-eye calibration method and device for four-axis mechanical arm |
CN116352710A (en) * | 2023-03-24 | 2023-06-30 | 电子科技大学 | Robot automatic calibration and three-dimensional measurement method for large aerospace component |
WO2023134237A1 (en) * | 2022-01-14 | 2023-07-20 | 北京京东乾石科技有限公司 | Coordinate system calibration method, apparatus and system for robot, and medium |
-
2023
- 2023-07-28 CN CN202310941577.1A patent/CN116652970B/en active Active
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109483516A (en) * | 2018-10-16 | 2019-03-19 | 浙江大学 | A kind of mechanical arm hand and eye calibrating method based on space length and epipolar-line constraint |
CN111775152A (en) * | 2020-06-29 | 2020-10-16 | 深圳大学 | Method and system for guiding mechanical arm to grab scattered stacked workpieces based on three-dimensional measurement |
CN111823233A (en) * | 2020-06-30 | 2020-10-27 | 浙江德尚韵兴医疗科技有限公司 | Mechanical arm hand-eye calibration system and method based on high-precision three-dimensional optical positioning |
EP4094897A1 (en) * | 2021-05-26 | 2022-11-30 | MVTec Software GmbH | Hand-eye calibration of camera-guided devices |
WO2023134237A1 (en) * | 2022-01-14 | 2023-07-20 | 北京京东乾石科技有限公司 | Coordinate system calibration method, apparatus and system for robot, and medium |
CN116038701A (en) * | 2022-12-30 | 2023-05-02 | 北京中科原动力科技有限公司 | Hand-eye calibration method and device for four-axis mechanical arm |
CN116352710A (en) * | 2023-03-24 | 2023-06-30 | 电子科技大学 | Robot automatic calibration and three-dimensional measurement method for large aerospace component |
Also Published As
Publication number | Publication date |
---|---|
CN116652970A (en) | 2023-08-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110276806B (en) | Online hand-eye calibration and grabbing pose calculation method for four-degree-of-freedom parallel robot stereoscopic vision hand-eye system | |
CN109483516B (en) | Mechanical arm hand-eye calibration method based on space distance and polar line constraint | |
CN109859275B (en) | Monocular vision hand-eye calibration method of rehabilitation mechanical arm based on S-R-S structure | |
TWI672206B (en) | Method and apparatus of non-contact tool center point calibration for a mechanical arm, and a mechanical arm system with said calibration function | |
CN109591011B (en) | Automatic tracking method for unilateral suture laser visual path of composite material three-dimensional structural part | |
JP4021413B2 (en) | Measuring device | |
CN109658460A (en) | A kind of mechanical arm tail end camera hand and eye calibrating method and system | |
CN113379849B (en) | Robot autonomous recognition intelligent grabbing method and system based on depth camera | |
CN110717943A (en) | Method and system for calibrating eyes of on-hand manipulator for two-dimensional plane | |
CN116652970B (en) | Four-axis mechanical arm 2D hand-eye calibration method and system and storage medium | |
CN110136204B (en) | Sound film dome assembly system based on calibration of machine tool position of bilateral telecentric lens camera | |
CN111801198A (en) | Hand-eye calibration method, system and computer storage medium | |
CN111862238B (en) | Full-space monocular light pen type vision measurement method | |
CN110666798A (en) | Robot vision calibration method based on perspective transformation model | |
CN114643578B (en) | Calibration device and method for improving robot vision guiding precision | |
CN115861445B (en) | Hand-eye calibration method based on three-dimensional point cloud of calibration plate | |
CN111612794A (en) | Multi-2D vision-based high-precision three-dimensional pose estimation method and system for parts | |
CN110136068B (en) | Sound membrane dome assembly system based on position calibration between bilateral telecentric lens cameras | |
CN112894209A (en) | Automatic plane correction method for intelligent tube plate welding robot based on cross laser | |
CN111360821A (en) | Picking control method, device and equipment and computer scale storage medium | |
CN112621743B (en) | Robot, hand-eye calibration method for fixing camera at tail end of robot and storage medium | |
CN114055444A (en) | Robot, control method and control device thereof, calibration method and calibration control device thereof, and storage medium | |
CN112958960A (en) | Robot hand-eye calibration device based on optical target | |
CN114029982A (en) | Hand-eye calibration device and calibration method of camera outside robot arm | |
CN113593050A (en) | Binocular vision guided robot intelligent assembly method, system and device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |