CN114939867A - Calibration method and system for mechanical arm external irregular asymmetric tool based on stereoscopic vision - Google Patents
Calibration method and system for mechanical arm external irregular asymmetric tool based on stereoscopic vision Download PDFInfo
- Publication number
- CN114939867A CN114939867A CN202210351439.3A CN202210351439A CN114939867A CN 114939867 A CN114939867 A CN 114939867A CN 202210351439 A CN202210351439 A CN 202210351439A CN 114939867 A CN114939867 A CN 114939867A
- Authority
- CN
- China
- Prior art keywords
- manipulator
- tool
- coordinate system
- target object
- tail end
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 230000001788 irregular Effects 0.000 title claims abstract description 40
- 239000011159 matrix material Substances 0.000 claims description 64
- 230000009466 transformation Effects 0.000 claims description 34
- 238000013519 translation Methods 0.000 claims description 25
- 230000008569 process Effects 0.000 claims description 11
- 230000006870 function Effects 0.000 claims description 10
- 238000006243 chemical reaction Methods 0.000 claims description 9
- 230000008859 change Effects 0.000 claims description 4
- 239000000126 substance Substances 0.000 claims 1
- 238000003860 storage Methods 0.000 description 7
- 238000004519 manufacturing process Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000036544 posture Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000003466 welding Methods 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1664—Programme controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
- B25J9/1697—Vision controlled systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T7/85—Stereo camera calibration
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Manipulator (AREA)
Abstract
The invention discloses a calibration method and a calibration system for an irregular and asymmetrical tool externally connected with a manipulator based on stereoscopic vision. The invention solves the problem that the tool coordinate system of the irregular asymmetric external tool is difficult to calibrate in the application of the mechanical arm.
Description
Technical Field
The invention relates to the technical field of mechanical arm calibration, in particular to a calibration method and a calibration system of an irregular and asymmetrical tool externally connected with a mechanical arm based on stereoscopic vision.
Background
In China, industrial robots are widely applied to the manufacturing industry, not only to the automobile manufacturing industry, but also to the production of space shuttles, military equipment, the development of high-speed rails and the production of ball pens. Compared with the application of a robot with fixed walking position by a fixed program, the robot positioning and guiding device combines machine vision, and can realize the positioning and guiding of the robot by detecting the position and the direction of the reporting element through the machine vision, so that the automation level is further improved. In the robot positioning guide, since a vision (camera) and a tool (a suction cup, a welding gun, a jig, a poking tool, or the like) are generally fixed to a robot end joint by a flange, it is necessary to calibrate a positional relationship between the vision (camera) and the robot, between the robot and the tool, or the like in advance.
The calibration of the robot and the tool coordinate system still has some problems to be solved in the application. The origin of the tool coordinate system is called as a tool Center point TCP (tool Center point), such as a suction cup Center, a fixture virtual Center, etc., a general robot system provides TCP calibration itself, and the calibration is achieved by aligning the Center to a certain point manually and making several special postures, and other high-precision TCP calibration methods are also available and are relatively mature. However, in TCP calibration, since only the position of the tip is aligned, the obtained conversion relationship between the tool coordinate system and the robot end coordinate system is only translation transformation, and there is no rotation transformation, that is, the direction of the tool coordinate system obtained only through TCP calibration is the same as the direction of the robot end. In practical application, the alignment is not only required for TCP, and there are many scenarios that require a robot to grab or align an irregular asymmetric object, such as a "D" hole, and also scenarios in which the orientation (normal direction) of the object is not fixed, and in addition to TCP (center point) alignment, it is also required that a tool and the object are aligned in all directions.
The existing calibration method does not solve the problem that a tool coordinate system of the irregular asymmetric external tool is difficult to calibrate in manipulator application, so that a calibration method of the irregular asymmetric tool externally connected to the manipulator based on stereoscopic vision is urgently needed.
Disclosure of Invention
In view of the above problems, an object of the present invention is to provide a calibration method and system for an external irregular asymmetric tool of a manipulator based on stereoscopic vision, which can accurately match and align the tool with a target, cope with complex scenes, and improve the automation level of industrial applications.
The invention provides a calibration method of an irregular and asymmetrical tool externally connected with a manipulator based on stereoscopic vision, which comprises the following steps:
determining the direction and the position of a coordinate system of a target object, and calibrating a tool center point and a camera respectively;
calculating a relative movement transformation matrix of the tail end of the manipulator, wherein the center point of the tool moves from the shooting position to the center of the target object, and the direction of the tail end of the manipulator is aligned with the direction of a coordinate system of the target object;
calculating a relative movement according to a relative movement transformation matrix, the relative movement comprising: the amount of relative translation and rotation;
the tail end of the mechanical arm performs first relative movement according to the obtained relative translation, and the pose of the tail end of the mechanical arm under the mechanical arm base coordinate system at the moment is recorded as P1;
controlling the mechanical hand to rotate around the tool center point, so that the tool direction is aligned with the target object direction; recording the pose of the tail end of the manipulator under the manipulator base coordinate system as P2;
calculating a compensation matrix according to the poses P1 and P2;
and calculating a relative motion Euler angle by using the compensation matrix, and controlling the manipulator to perform relative motion, namely directly aligning the tool with the target object.
In this scheme, the determining the direction and the position of the target coordinate system, and calibrating the tool center point and the camera respectively specifically include:
determining the direction of a target object coordinate system according to the characteristics of the target object;
determining the direction and position of the target object coordinate system in the camera coordinate system;
calibrating the tool center point in a manipulator tail end coordinate system to obtain a calibration result;
performing hand-eye calibration on the camera and the tail end of the manipulator to obtain a hand-eye calibration result;
in this embodiment, the determining the direction of the coordinate system of the target object according to the characteristics of the target object includes:
the target object coordinate system comprises a normal Vec _ Z, Y direction Vec _ Y, X direction Vec _ X, wherein the direction is a normal direction of the point cloud at the target center point or a normal direction of the extension of the target virtual center point; the Y direction is a direction which is easy to grab on the target and is fixed in position and is vertical to the normal direction; the X direction is perpendicular to the normal direction and the Y direction.
In this scheme, the direction and position of the target coordinate system in the camera coordinate system are represented as:
in the scheme, the tool center point is calibrated in a manipulator tail end coordinate system, and the obtained calibration result is expressed as:
in the scheme, a relative movement transformation matrix of the tail end of the manipulator is calculated, wherein the center point of the tool moves from a shooting position to the center of a target object, and the direction of the tail end of the manipulator is aligned with the direction of a coordinate system of the target object;
the expression is as follows:
wherein
And showing the hand-eye calibration result of the camera and the tail end of the manipulator.
In this scheme, the calculating the relative movement according to the relative movement transformation matrix includes: the relative translation amount and rotation amount are specifically as follows: the relative movement transformation matrix is rigid transformation, the first three rows and three columns are rotation matrixes R, the last column is translation amount t, the position change of the relative movement, namely the translation amount t in the transformation matrix, is calculated by utilizing matlab function rotm2eul to obtain the rotation amount, and the direction rotation amount of the manipulator is expressed by three Euler angles of Rz, Ry and Rx.
In the scheme, the calculation of the compensation matrix according to the poses P1 and P2 comprises the following specific processes:
the (x, y, z) position of the pose P1 is t1, where x, y, z represent the coordinates of the pose P1, and the euler angular pose is converted to a rotation matrix R1 using matlab function eul2 rotm; the position of the pose P2 is t2, and the pose rotation matrix is R2. The compensation matrix for pose P1 to pose P2 is then:
in the scheme, the Euler angle of the relative motion is calculated by using the compensation matrix, the manipulator is controlled to carry out the relative motion based on the terminal coordinate system, and the specific process of directly aligning the tool and the target object is as follows:
determining a fixed conversion relation: camera-manipulator tip
Tool TCP-manipulator tip
Tool direction and robot end rotation compensation
Acquiring the position and the direction of a target object shot by a camera in the camera
Calculating a manipulator tail end relative motion conversion matrix from the shooting position to the target object aligned by the irregular asymmetric tool:
the Euler angle of the movement of the manipulator is calculated by using the existing tool, and the manipulator is controlled to carry out relative movement according to the calculated Euler angle, so that the tool can be directly aligned with the target object.
The invention provides a calibration system of an external irregular and asymmetric manipulator tool based on stereoscopic vision, which comprises a memory and a processor, wherein the memory comprises a calibration program of the external irregular and asymmetric manipulator tool based on stereoscopic vision, and when the processor executes the program, the calibration method of the external irregular and asymmetric manipulator tool based on stereoscopic vision realizes the following steps:
determining the direction and the position of a coordinate system of a target object, and calibrating a tool center point and a camera respectively;
calculating a relative movement transformation matrix of the tail end of the manipulator, wherein the center point of the tool moves from the shooting position to the center of the target object, and the direction of the tail end of the manipulator is aligned with the direction of a coordinate system of the target object;
calculating a relative movement according to a relative movement transformation matrix, the relative movement comprising: relative translation and rotation;
the tail end of the manipulator carries out first relative motion according to the obtained relative translation, and the pose of the tail end of the manipulator at the moment under the manipulator base coordinate system is recorded as P1;
controlling the mechanical hand to rotate around the tool center point, so that the tool direction is aligned with the target object direction; recording the pose of the tail end of the manipulator under the manipulator base coordinate system as P2;
calculating a compensation matrix according to the poses P1 and P2;
and calculating a relative motion Euler angle by using the compensation matrix, and controlling the manipulator to perform relative motion, namely directly aligning the tool with the target object.
The invention discloses a calibration method and a calibration system for an irregular and asymmetrical tool externally connected with a manipulator based on stereoscopic vision.
Drawings
Fig. 1 shows a flow chart of a calibration method of an external irregular asymmetric tool of a manipulator based on stereoscopic vision.
Fig. 2 shows a flow chart of determining the direction and position of the target coordinate system and calibrating the tool center point and the camera respectively in the present application.
Fig. 3 shows a block diagram of a calibration system of the external irregular asymmetric tool of the manipulator based on stereoscopic vision.
Detailed Description
In order that the above objects, features and advantages of the present invention can be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those specifically described herein, and therefore the scope of the present invention is not limited by the specific embodiments disclosed below.
Fig. 1 shows a flow chart of a calibration method of an external irregular asymmetric tool of a manipulator based on stereoscopic vision.
As shown in fig. 1, the application discloses a calibration method of an external irregular asymmetric tool of a manipulator based on stereoscopic vision, which comprises the following steps:
s102, determining the direction and the position of a coordinate system of the target object, and calibrating a tool center point and a camera respectively;
s104, calculating a relative movement transformation matrix of the tail end of the manipulator, wherein the center point of the tool moves from the shooting position to the center of the target object, and the direction of the tail end of the manipulator is aligned with the direction of a coordinate system of the target object;
s106, calculating relative movement according to the relative movement transformation matrix, wherein the relative movement comprises the following steps: the amount of relative translation and rotation;
s108, carrying out first relative motion on the tail end of the manipulator according to the obtained relative translation, and recording the pose of the tail end of the manipulator under a manipulator base coordinate system at the moment as P1;
s110, controlling the mechanical hand to rotate around the central point of the tool, so that the direction of the tool is aligned with the direction of a target object; recording the pose of the tail end of the manipulator under the manipulator base coordinate system at the moment as P2;
s112, calculating a compensation matrix according to the poses P1 and P2;
and S114, calculating a relative motion Euler angle by using the compensation matrix, and controlling the manipulator to perform relative motion, namely directly aligning the tool with the target object.
It should be noted that, before implementation, the method according to the embodiment of the present invention needs to perform tool center point calibration (i.e., TCP calibration) and hand-eye calibration (i.e., hand-eye calibration performed by the camera and the end of the manipulator), and convert the position, normal direction, and Y direction (feature edge easy to grasp) of the target object in the camera coordinate system (i.e., point cloud) into the position, normal direction, and Y direction in the tool coordinate system. The robot is allowed to perform relative movement, since the tool coordinate system direction is the same as the robot end coordinate system direction, so the tool and the target may not be aligned, and the robot is required to perform relative rotation around the TCP to align the target, this relative rotation is referred to as the rotation compensation of the tool coordinate system and the robot end, and since the tool is fixedly mounted at the robot end, this rotation compensation is fixed.
It should be noted that the camera described in the embodiment of the present invention is a 3D camera, the manipulator described in the present invention is not limited to specific types and use scenarios, and the tool is an external tool, and the type of the tool is not limited, and all irregular asymmetric tools suitable for the manipulator are suitable for the present invention.
According to the embodiment of the present invention, the determining the direction and the position of the coordinate system of the target object, and calibrating the tool center point and the camera respectively includes:
s202, determining the direction of a target object coordinate system according to the characteristics of the target object;
s204, determining the direction and the position of the target object coordinate system in the camera coordinate system;
s206, calibrating the tool center point in a manipulator tail end coordinate system to obtain a calibration result;
s208, performing hand-eye calibration on the camera and the tail end of the manipulator to obtain a hand-eye calibration result;
in a specific embodiment, the tool external connection tool at the end of the robot is aligned with the target object in actual use, the direction of the tool coordinate system may be considered to be aligned with the target object in the coordinate system direction, and the coordinate system direction of the target object may be defined according to the characteristics of the target object.
According to the embodiment of the present invention, the determining the target object coordinate system direction according to the target object feature includes: the target object coordinate system comprises a normal Vec _ Z, Y direction Vec _ Y, X direction Vec _ X, wherein the direction is a normal direction of the point cloud at the target center point or a normal direction extending from the target virtual center point; the Y direction is a direction which is easy to grab and fixed in position on the target and is vertical to the normal direction; the X direction is perpendicular to the normal direction and the Y direction.
According to the embodiment of the invention, the direction and position of the target coordinate system in the camera coordinate system are expressed as follows:
it should be noted that the direction and position of the Target (Target) coordinate system in the Camera (Camera) coordinate system (i.e. the point cloud coordinate system) are obtained by analyzing the point cloud, such as: the Target is a hole, the center of the hole is the origin of the Target coordinate system, and the outward direction of the hole is the Z direction of the Target coordinate system.
According to the embodiment of the invention, the tool center point is calibrated in the terminal coordinate system of the manipulator, and the obtained calibration result is expressed as:
note that the tool center point is calibrated in the robot End coordinate system, that is, the TCP calibration result, is made possible by the robot system itself, and the position of the TCP in the robot End (End) coordinate system is obtained (since the tool center point is a point, there is no concept of direction).
According to the embodiment of the invention, a relative movement transformation matrix of the tail end of the manipulator is calculated, wherein the center point of the tool moves from a shooting position to the center of a target object, and the direction of the tail end of the manipulator is aligned with the direction of a coordinate system of the target object;
the expression is as follows:
wherein
And showing the hand-eye calibration result of the camera and the tail end of the manipulator.
It should be noted that, the hand-eye calibration result of the Camera and the End of the manipulator is obtained by a general hand-eye calibration process, and the position and the direction of the Camera (Camera) coordinate system in the manipulator End (End) coordinate system are obtained, because the Camera is installed at the End of the manipulator, the relationship is fixed, and can be obtained by calibration.
According to an embodiment of the invention, the calculating of the relative movement according to the relative movement transformation matrix comprises: the relative translation amount and rotation amount are specifically as follows: the relative movement transformation matrix is rigid transformation, the first three rows and three columns are rotation matrixes R, the last column is translation amount t, the position change of the relative movement, namely the translation amount t in the transformation matrix, is calculated by utilizing matlab function rotm2eul to obtain the rotation amount, and the direction rotation amount of the manipulator is expressed by three Euler angles of Rz, Ry and Rx.
According to the embodiment of the invention, the calculation of the compensation matrix according to the poses P1 and P2 comprises the following specific processes: let the (x, y, z) position of the pose P1 be t1, where x, y, z represent the coordinates of the pose P1, and the euler angular pose is converted into a rotation matrix R1 using matlab function eul2 rotm; the position of the pose P2 is t2, and the pose rotation matrix is R2. The compensation matrix for pose P1 to pose P2 is then:
according to the embodiment of the invention, the euler angle of the relative motion is calculated by using the compensation matrix, the manipulator is controlled to carry out the relative motion, and the specific process of directly aligning the tool and the target object is as follows:
determining a fixed conversion relationship: camera-manipulator tip
Tool TCP-manipulator tip
Tool direction and robot end rotation compensation
Acquiring the position and the direction of a target object shot by a camera in the camera
Calculating a manipulator tail end relative motion conversion matrix from the shooting position to the irregular asymmetric tool aiming at the target object:
the Euler angle of the movement of the manipulator is calculated by using the existing tool, and the manipulator is controlled to carry out relative movement according to the calculated Euler angle, so that the tool can be directly aligned with the target object.
It should be noted that, in the embodiment of the present invention, when controlling the relative motion of the manipulator, the relative motion is performed based on the manipulator end coordinate system, and the existing calculation tool calculates the euler angle of the relative motion, for example, by using matlab function rotm2 eul.
The invention provides a calibration system of an external mechanical arm irregular asymmetric tool based on stereoscopic vision, which comprises a memory and a processor, wherein the memory comprises a calibration program of the external mechanical arm irregular asymmetric tool based on stereoscopic vision, and the calibration method program of the external mechanical arm irregular asymmetric tool based on stereoscopic vision realizes the following steps when being executed by the processor:
s102, determining the direction and the position of a coordinate system of the target object, and calibrating a tool center point and a camera respectively;
s104, calculating a relative movement transformation matrix of the tail end of the manipulator, wherein the center point of the tool moves from the shooting position to the center of the target object, and the direction of the tail end of the manipulator is aligned with the direction of a coordinate system of the target object;
s106, calculating relative movement according to the relative movement transformation matrix, wherein the relative movement comprises the following steps: relative translation and rotation;
s108, carrying out first relative motion on the tail end of the manipulator according to the obtained relative translation, and recording the pose of the tail end of the manipulator under a manipulator base coordinate system at the moment as P1;
s110, controlling the mechanical hand to rotate around the central point of the tool, so that the direction of the tool is aligned with the direction of a target object; recording the pose of the tail end of the manipulator under the manipulator base coordinate system as P2;
s112, calculating a compensation matrix according to the poses P1 and P2;
and S114, calculating a relative movement Euler angle by using the compensation matrix, and controlling the manipulator to perform relative movement based on the terminal coordinate system, so that the tool can be directly aligned with the target object.
It should be noted that, before implementation, the method according to the embodiment of the present invention needs to perform tool center point calibration (i.e., TCP calibration) and hand-eye calibration (i.e., hand-eye calibration performed at the end of the camera and the manipulator), and convert the position, the normal direction, and the Y direction (the feature edge that is easy to grasp) of the target object in the camera coordinate system (i.e., point cloud) into the position, the normal direction, and the Y direction in the tool coordinate system. The robot is allowed to perform relative movement, since the tool coordinate system direction is the same as the robot end coordinate system direction, so the tool and the target may not be aligned, and the robot is required to perform relative rotation around the TCP to align the target, this relative rotation is referred to as the rotation compensation of the tool coordinate system and the robot end, and since the tool is fixedly mounted at the robot end, this rotation compensation is fixed.
It should be noted that the camera described in the embodiment of the present invention is a 3D camera, the manipulator described in the present invention is not limited to specific types and use scenarios, and the tool is an external tool, and the type of the tool is not limited, and all irregular asymmetric tools suitable for the manipulator are suitable for the present invention.
According to the embodiment of the present invention, the determining the direction and the position of the coordinate system of the target object, and calibrating the tool center point and the camera respectively includes:
s202, determining the direction of a target object coordinate system according to the characteristics of the target object;
s204, determining the direction and the position of the target object coordinate system in the camera coordinate system;
s206, calibrating the tool center point in a manipulator tail end coordinate system to obtain a calibration result;
s208, performing hand-eye calibration on the camera and the tail end of the manipulator to obtain a hand-eye calibration result;
in a specific embodiment, the external tool at the end of the manipulator is aligned with the target object in actual application, the direction of the tool coordinate system may be considered to be consistent with the direction of the target object coordinate system in alignment with the target object, and the direction of the target object coordinate system may be defined according to the characteristics of the target object.
According to the embodiment of the present invention, the determining the target object coordinate system direction according to the target object feature includes: the target object coordinate system comprises a normal Vec _ Z, Y direction Vec _ Y, X direction Vec _ X, wherein the direction is a normal direction of the point cloud at the target center point or a normal direction extending from the target virtual center point; the Y direction is a direction which is easy to grab and fixed in position on the target and is vertical to the normal direction; the X direction is perpendicular to both the normal direction and the Y direction.
According to an embodiment of the present invention, the direction and position of the target object coordinate system in the camera coordinate system are represented as:
it should be noted that, the direction and position of the Target (Target) coordinate system in the Camera (Camera) coordinate system (i.e. the point cloud coordinate system) are obtained by analyzing the point cloud, such as: the Target is a hole, the center of the hole is the origin of the Target coordinate system, and the outward direction of the hole is the Z direction of the Target coordinate system.
According to the embodiment of the invention, the tool center point is calibrated in the terminal coordinate system of the manipulator, and the obtained calibration result is expressed as:
note that the tool center point is calibrated in the robot End coordinate system, that is, the TCP calibration result, is made possible by the robot system itself, and the position of the TCP in the robot End (End) coordinate system is obtained (since the tool center point is a point, there is no concept of direction).
According to the embodiment of the invention, a relative movement transformation matrix of the tail end of the manipulator is calculated, wherein the center point of the tool moves from a shooting position to the center of a target object, and the direction of the tail end of the manipulator is aligned with the direction of a coordinate system of the target object;
the expression is as follows:
wherein
And showing the hand-eye calibration result of the camera and the tail end of the manipulator.
It should be noted that, the hand-eye calibration result of the Camera and the End of the manipulator is obtained by a general hand-eye calibration process, and the position and the direction of the Camera (Camera) coordinate system in the manipulator End (End) coordinate system are obtained, because the Camera is installed at the End of the manipulator, the relationship is fixed, and can be obtained by calibration.
According to an embodiment of the invention, the calculating of the relative movement according to the relative movement transformation matrix comprises: the relative translation amount and rotation amount specifically comprises the following steps: the relative movement transformation matrix is rigid transformation, the first three rows and three columns are rotation matrixes R, the last column is translation amount t, the position change of the relative movement, namely the translation amount t in the transformation matrix, is calculated by utilizing matlab function rotm2eul to obtain the rotation amount, and the direction rotation amount of the manipulator is expressed by three Euler angles of Rz, Ry and Rx.
According to the embodiment of the invention, the calculation of the compensation matrix according to the poses P1 and P2 comprises the following specific processes: let the (x, y, z) position of the pose P1 be t1, where x, y, z represent the coordinates of the pose P1, and the euler angular pose is converted into a rotation matrix R1 using matlab function eul2 rotm; the position of the pose P2 is t2, and the pose rotation matrix is R2. The compensation matrix for pose P1 to pose P2 is then:
according to the embodiment of the invention, the compensation matrix is used for calculating the relative motion Euler angle, the manipulator is controlled to carry out relative motion, and the specific process of directly aligning the tool and the target object is as follows:
determining a fixed conversion relationship: camera-manipulator tip
Tool TCP-manipulator tip
Tool direction and robot end rotation compensation
Acquiring the position and the direction of a target object shot by a camera in the camera
Calculating a manipulator tail end relative motion conversion matrix from the shooting position to the irregular asymmetric tool aiming at the target object:
the Euler angle of the movement of the manipulator is calculated by using the existing tool, and the manipulator is controlled to carry out relative movement according to the calculated Euler angle, so that the tool can be directly aligned with the target object.
It should be noted that, in the embodiment of the present invention, when controlling the relative motion of the manipulator, the relative motion is performed based on the manipulator end coordinate system, and the existing calculation tool calculates the euler angle of the relative motion, for example, by using matlab function rotm2 eul.
The invention discloses a calibration method and a calibration system for an irregular and asymmetrical tool externally connected with a manipulator based on stereoscopic vision.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. The above-described device embodiments are merely illustrative, for example, the division of the unit is only a logical functional division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units; can be located in one place or distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, all the functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium includes: a mobile storage device, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Alternatively, the integrated unit of the present invention may be stored in a computer-readable storage medium if it is implemented in the form of a software functional module and sold or used as a separate product. Based on such understanding, the technical solutions of the embodiments of the present invention may be essentially implemented or a part contributing to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a removable storage device, a ROM, a RAM, a magnetic or optical disk, or various other media that can store program code.
Claims (10)
1. A calibration method of an irregular and asymmetrical tool externally connected with a manipulator based on stereoscopic vision is characterized by comprising the following steps:
determining the direction and the position of a coordinate system of a target object, and calibrating a tool center point and a camera respectively;
calculating a relative movement transformation matrix of the tail end of the manipulator, wherein the center point of the tool moves from the shooting position to the center of the target object, and the direction of the tail end of the manipulator is aligned with the direction of a coordinate system of the target object;
calculating a relative movement according to a relative movement transformation matrix, the relative movement comprising: the amount of relative translation and rotation;
the tail end of the manipulator carries out first relative motion according to the obtained relative translation, and the pose of the tail end of the manipulator at the moment under the manipulator base coordinate system is recorded as P1;
controlling the mechanical hand to rotate around the tool center point, so that the tool direction is aligned with the target object direction; recording the pose of the tail end of the manipulator under the manipulator base coordinate system at the moment as P2;
calculating a compensation matrix according to poses P1 and P2;
and calculating a relative motion Euler angle by using the compensation matrix, and controlling the manipulator to perform relative motion, so that the tool can be directly aligned with the target object.
2. The method for calibrating the externally-connected irregular and asymmetrical tool for the manipulator based on the stereoscopic vision as claimed in claim 1, wherein the determining the direction and the position of the coordinate system of the target object and calibrating the center point of the tool and the camera respectively comprises:
determining the direction of a target object coordinate system according to the characteristics of the target object;
determining the direction and the position of the target object coordinate system in the camera coordinate system;
calibrating the tool center point in a manipulator tail end coordinate system to obtain a calibration result;
and carrying out hand-eye calibration on the camera and the tail end of the manipulator to obtain a hand-eye calibration result.
3. The method for calibrating the robot-circumscribed irregular asymmetric tool based on stereoscopic vision according to claim 2, wherein the determining the target object coordinate system direction according to the target object feature comprises: the target object coordinate system comprises a normal Vec _ Z, Y direction Vec _ Y, X direction Vec _ X, wherein the direction is a normal direction of the point cloud at the target center point or a normal direction extending from the target virtual center point; the Y direction is a direction which is easy to grab and fixed in position on the target and is vertical to the normal direction; the X direction is perpendicular to both the normal direction and the Y direction.
4. The calibration method for the external irregular and asymmetrical tool of the manipulator based on the stereoscopic vision as claimed in claim 2, wherein the direction and the position of the target object coordinate system in the camera coordinate system are represented as follows:
wherein the content of the first and second substances,
it is shown that,
it is shown that the process of the present invention,
it is shown that,
and (4) showing.
5. The calibration method for the external irregular and asymmetrical tool of the manipulator based on the stereoscopic vision as claimed in claim 2, wherein the tool center point is calibrated in the coordinate system of the manipulator end, and the calibration result is expressed as:
wherein t is x Denotes, t y Denotes t z And (4) showing.
6. The calibration method of the externally-connected irregular asymmetric tool for the manipulator based on the stereoscopic vision as claimed in claim 1, wherein a relative movement transformation matrix of the manipulator end is calculated, wherein the center point of the tool is moved from a shooting position to the center of the target object, and the direction of the manipulator end is aligned with the direction of the coordinate system of the target object;
the expression is as follows:
wherein
And showing the hand-eye calibration result of the camera and the tail end of the manipulator.
7. The method for calibrating the robot-circumscribed irregular asymmetric tool based on stereoscopic vision according to claim 1, wherein the calculating the relative movement according to the relative movement transformation matrix comprises: the relative translation amount and rotation amount are specifically as follows: the relative movement transformation matrix is rigid transformation, the first three rows and three columns are rotation matrixes R, the last column is translation amount t, the position change of the relative movement, namely the translation amount t in the transformation matrix, is calculated by utilizing matlab function rotm2eul to obtain the rotation amount, and the direction rotation amount of the manipulator is expressed by three Euler angles of Rz, Ry and Rx.
8. The calibration method of the manipulator external irregular asymmetric tool based on the stereoscopic vision as claimed in claim 1, wherein the compensation matrix is calculated according to poses P1 and P2 by the following specific processes: the (x, y, z) position of the pose P1 is t1, where x, y, z represent the coordinates of the pose P1, and the euler angular pose is converted to a rotation matrix R1 using matlab function eul2 rotm; the position of the pose P2 is t2, and the pose rotation matrix is R2. The compensation matrix for pose P1 to pose P2 is then:
9. the method for calibrating the external irregular and asymmetrical tool of the manipulator based on the stereoscopic vision as claimed in claim 8, wherein the compensation matrix is used to calculate the euler angle of the relative motion and control the manipulator to perform the relative motion, so that the specific process of directly aligning the tool with the target object is as follows:
determining a fixed conversion relationship: camera-manipulator tip
Tool TCP-manipulator tip
Tool direction and manipulator end rotation compensation
Acquiring a target shot by a cameraPosition and orientation of an object in a camera
Calculating a manipulator tail end relative motion conversion matrix from the shooting position to the target object aligned by the irregular asymmetric tool:
the Euler angle of the movement of the manipulator is calculated by using the existing tool, and the manipulator is controlled to carry out relative movement according to the calculated Euler angle, so that the tool can be directly aligned with the target object.
10. The calibration system of the external mechanical arm irregular asymmetric tool based on the stereoscopic vision is further characterized by comprising a memory and a processor, wherein the memory comprises a calibration program of the external mechanical arm irregular asymmetric tool based on the stereoscopic vision, and the calibration method program of the external mechanical arm irregular asymmetric tool based on the stereoscopic vision realizes the following steps when being executed by the processor:
determining the direction and the position of a coordinate system of a target object, and calibrating a tool center point and a camera respectively;
calculating a relative movement transformation matrix of the tail end of the manipulator, wherein the center point of the tool moves from the shooting position to the center of the target object, and the direction of the tail end of the manipulator is aligned with the direction of a coordinate system of the target object;
calculating a relative movement according to a relative movement transformation matrix, the relative movement comprising: the amount of relative translation and rotation;
the tail end of the manipulator carries out first relative motion according to the obtained relative translation, and the pose of the tail end of the manipulator at the moment under the manipulator base coordinate system is recorded as P1;
controlling the mechanical hand to rotate around the tool center point, so that the tool direction is aligned with the target object direction; recording the pose of the tail end of the manipulator under the manipulator base coordinate system at the moment as P2;
calculating a compensation matrix according to the poses P1 and P2;
and calculating a relative motion Euler angle by using the compensation matrix, and controlling the manipulator to perform relative motion, namely directly aligning the tool with the target object.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210351439.3A CN114939867A (en) | 2022-04-02 | 2022-04-02 | Calibration method and system for mechanical arm external irregular asymmetric tool based on stereoscopic vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210351439.3A CN114939867A (en) | 2022-04-02 | 2022-04-02 | Calibration method and system for mechanical arm external irregular asymmetric tool based on stereoscopic vision |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114939867A true CN114939867A (en) | 2022-08-26 |
Family
ID=82908152
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210351439.3A Pending CN114939867A (en) | 2022-04-02 | 2022-04-02 | Calibration method and system for mechanical arm external irregular asymmetric tool based on stereoscopic vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114939867A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117103286A (en) * | 2023-10-25 | 2023-11-24 | 杭州汇萃智能科技有限公司 | Manipulator eye calibration method and system and readable storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104165585A (en) * | 2013-05-17 | 2014-11-26 | 上海三菱电梯有限公司 | Non-contact high-precision calibration method of tool coordinate system of single robot |
| CN109974584A (en) * | 2019-04-12 | 2019-07-05 | 山东大学 | The calibration system and scaling method of a kind of auxiliary laser bone-culting operation robot |
| CN110276806A (en) * | 2019-05-27 | 2019-09-24 | 江苏大学 | Online hand-eye calibration and crawl pose calculation method for four-freedom-degree parallel-connection robot stereoscopic vision hand-eye system |
| US20210291310A1 (en) * | 2020-03-23 | 2021-09-23 | Lei & So Co., Ltd. | Direct pose feedback control method and direct pose feedback controlled machine |
| CN114022636A (en) * | 2021-09-18 | 2022-02-08 | 杭州汇萃智能科技有限公司 | Point cloud distortion correction method based on 3D line scanning camera and related equipment |
-
2022
- 2022-04-02 CN CN202210351439.3A patent/CN114939867A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104165585A (en) * | 2013-05-17 | 2014-11-26 | 上海三菱电梯有限公司 | Non-contact high-precision calibration method of tool coordinate system of single robot |
| CN109974584A (en) * | 2019-04-12 | 2019-07-05 | 山东大学 | The calibration system and scaling method of a kind of auxiliary laser bone-culting operation robot |
| CN110276806A (en) * | 2019-05-27 | 2019-09-24 | 江苏大学 | Online hand-eye calibration and crawl pose calculation method for four-freedom-degree parallel-connection robot stereoscopic vision hand-eye system |
| US20210291310A1 (en) * | 2020-03-23 | 2021-09-23 | Lei & So Co., Ltd. | Direct pose feedback control method and direct pose feedback controlled machine |
| CN114022636A (en) * | 2021-09-18 | 2022-02-08 | 杭州汇萃智能科技有限公司 | Point cloud distortion correction method based on 3D line scanning camera and related equipment |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117103286A (en) * | 2023-10-25 | 2023-11-24 | 杭州汇萃智能科技有限公司 | Manipulator eye calibration method and system and readable storage medium |
| CN117103286B (en) * | 2023-10-25 | 2024-03-19 | 杭州汇萃智能科技有限公司 | Manipulator eye calibration method and system and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107738254B (en) | Conversion calibration method and system for mechanical arm coordinate system | |
| CN110640745B (en) | Vision-based robot automatic calibration method, equipment and storage medium | |
| CN111369625B (en) | Positioning method, positioning device and storage medium | |
| CN109794963B (en) | Robot rapid positioning method facing curved surface component | |
| CN112894823B (en) | Robot high-precision assembling method based on visual servo | |
| CN111844130B (en) | Method and device for correcting pose of robot end tool | |
| CN112621743B (en) | Robot, hand-eye calibration method for fixing camera at tail end of robot and storage medium | |
| CN112907683B (en) | Camera calibration method and device for dispensing platform and related equipment | |
| CN111360821A (en) | Picking control method, device and equipment and computer scale storage medium | |
| CN113379849A (en) | Robot autonomous recognition intelligent grabbing method and system based on depth camera | |
| CN113211445A (en) | Robot parameter calibration method, device, equipment and storage medium | |
| US7957834B2 (en) | Method for calculating rotation center point and axis of rotation, method for generating program, method for moving manipulator and positioning device, and robotic system | |
| CN114939867A (en) | Calibration method and system for mechanical arm external irregular asymmetric tool based on stereoscopic vision | |
| CN114260908A (en) | Robot teaching method, device, computer equipment and computer program product | |
| CN114022551A (en) | Method for accurately identifying and estimating pose of fuel filling cover of fuel vehicle | |
| CN115439633A (en) | Calibration method and device and electronic equipment | |
| CN115741720A (en) | Zero calibration system and method for robot based on binocular vision technology and LM algorithm | |
| CN110962127A (en) | Auxiliary calibration device for tail end pose of mechanical arm and calibration method thereof | |
| CN114074331A (en) | Disordered grabbing method based on vision and robot | |
| CN112894814B (en) | Mechanical arm DH parameter identification method based on least square method | |
| CN112621378B (en) | Method and device for calibrating structural parameters of machine tool and machine tool control system | |
| CN109311163B (en) | Method for correcting motion control command of robot and related equipment thereof | |
| CN111975756B (en) | Hand-eye calibration system and method of 3D vision measurement system | |
| CN110533727B (en) | Robot self-positioning method based on single industrial camera | |
| CN111360797A (en) | Robot TCP calibration method based on vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |