CN115488878A - Hand-eye calibration method, system, terminal and medium for robot vision system - Google Patents
Hand-eye calibration method, system, terminal and medium for robot vision system Download PDFInfo
- Publication number
- CN115488878A CN115488878A CN202211041992.3A CN202211041992A CN115488878A CN 115488878 A CN115488878 A CN 115488878A CN 202211041992 A CN202211041992 A CN 202211041992A CN 115488878 A CN115488878 A CN 115488878A
- Authority
- CN
- China
- Prior art keywords
- calibration
- robot
- point cloud
- ball
- camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
- B25J9/1692—Calibration of manipulator
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
- B25J19/0095—Means or methods for testing manipulators
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention provides a hand-eye calibration method and a hand-eye calibration system for a robot vision system, wherein the method comprises the following steps: installing a calibration ball at the end part of the robot, moving the calibration ball into a working space of the three-dimensional point cloud camera, controlling the position of the calibration ball in the working space, and calculating the center coordinates of the calibration ball under a robot coordinate system; acquiring three-dimensional point cloud information of a calibration ball in a working space, and performing point cloud screening to obtain calibration ball point cloud information; extracting the calibration ball according to the point cloud information of the calibration ball to obtain the center coordinates of the calibration ball under a three-dimensional camera coordinate system; and calculating the hand-eye calibration relation between the camera and the robot according to the coordinates of the sphere center of the calibration sphere under the camera coordinate system and the robot coordinate system each time, and completing the hand-eye calibration of the robot vision system. The invention has convenient deployment, low cost and wide application, and has important significance for promoting the development of manufacturing industry, reducing the production cost of enterprises and improving the production efficiency.
Description
Technical Field
The invention relates to a camera-robot system calibration technology in the technical field of identification and grabbing in industrial production, in particular to a hand-eye calibration method, a system, a terminal and a medium of a high-precision robot vision system.
Background
With the development of machine vision technology and robot technology, more and more work occasions are provided for the depth camera and the industrial robot to cooperate with each other in industrial production, the robot needs to sense scene information in real time through the camera, and the three-dimensional camera-robot system is required to be calibrated in advance with high precision. A general camera-robot system changes the position of a camera or a robot due to a change of a work task, so that the three-dimensional camera-robot system needs to be calibrated again, and therefore, a fast and high-precision calibration system is needed.
The current commonly used calibration system is based on a plane calibration plate, a camera is used for identifying the pose of the calibration plate, and the coordinate relation between the camera and the robot is solved by combining the coordinate of the robot. The method requires the two-dimensional calibration plate to be in the visual field range of the camera, and the method is greatly limited due to the large volume of the calibration plate and the limited motion space of the robot. The other common method is based on an auxiliary device, such as a laser tracker or a three-axis tracker, to realize hand-eye calibration, and the method can greatly improve the cost and the operation difficulty of a calibration system. Therefore, the three-dimensional camera-robot calibration system which is rapid, universal and low in cost has important significance.
The prior art is searched to find that:
chinese patent No. CN110842901B, entitled disclosure of invention, discloses a method and apparatus for calibrating robot hand and eye based on a novel three-dimensional calibration block, which enables a three-dimensional vision device to obtain a point cloud containing three key points on the three-dimensional calibration block by adjusting the posture of the robot and the placing posture of the three-dimensional calibration block, and can solve the relation between the camera and the robot hand and eye based on the coordinates of the key points in the camera coordinate system and the robot coordinate system.
The invention discloses a Chinese patent with an authorization publication number of CN112091971B, which discloses a robot hand-eye calibration method, a device, electronic equipment and a system, and comprises the following steps: 1. acquiring three-dimensional point cloud image information shot by a three-dimensional camera, wherein the three-dimensional point cloud image information comprises three-dimensional point cloud information of at least three identification balls which are not on the same straight line; 2. calculating first position data of the centroid of the identification ball in a camera coordinate system according to the three-dimensional point cloud image information; 3. acquiring second position data of the mass center of the identification ball in a robot base coordinate system; 4. and calculating a conversion matrix between the camera coordinate system and the robot base coordinate system according to the first position data and the second position data.
The two patents have the characteristics of low cost, high speed and the like, but the problems that the calibration piece is too complex, information of three key points needs to be extracted simultaneously, the pose of the calibration piece is limited, the applicability is low and the like still exist.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a hand-eye calibration method, a hand-eye calibration system, a hand-eye calibration terminal and a hand-eye calibration medium of a robot vision system.
According to an aspect of the present invention, there is provided a hand-eye calibration method for a robot vision system, comprising:
setting a working space of a three-dimensional point cloud camera;
installing a calibration ball at the end part of the robot, moving the calibration ball into a working space of the three-dimensional point cloud camera, controlling the position of the calibration ball in the working space, calculating the center coordinates of the calibration ball in a robot coordinate system, and sending the center coordinates to a calibration module;
acquiring three-dimensional point cloud information of the calibration ball in a working space, and performing point cloud screening to obtain calibration ball point cloud information;
extracting the calibration ball according to the point cloud information of the calibration ball to obtain the center coordinates of the calibration ball under a three-dimensional camera coordinate system;
repeating the steps, and obtaining the sphere center coordinates of the calibration sphere under a robot coordinate system and a three-dimensional camera coordinate system for multiple times;
and calculating the hand-eye calibration relation between the camera and the robot according to the coordinates of the sphere center of the calibration sphere under the camera coordinate system and the robot coordinate system each time, and completing the hand-eye calibration of the robot vision system.
Optionally, the setting a working space of the three-dimensional point cloud camera includes:
and fixing the three-dimensional point cloud camera at a specific working position, and adjusting working parameters of the three-dimensional point cloud camera according to environmental factors to obtain a working space of the three-dimensional point cloud camera.
Optionally, the installing a calibration ball at the end of the robot, moving the calibration ball into a working space of the three-dimensional point cloud camera, controlling the position of the calibration ball in the working space, and calculating the coordinates of the center of the calibration ball in the robot coordinate system includes:
customizing a calibration ball, wherein the calibration ball comprises a calibration piece and a positioning piece; the positioning piece comprises a metal ball, a connecting rod and a base flange, and is provided with a cylindrical groove matched with the radius of the metal ball;
fixing the metal ball with the end of the robot through the connecting rod and the base flange in sequence; fixing the positioning piece on the working platform; the metal ball is contacted with the bottom surface and the side surface of the cylindrical groove to obtain a determined matching state between the calibration piece and the positioning piece, and the position of the calibration ball in a working space is fixed;
and changing the pose of the robot for many times, ensuring the position of the sphere center of the calibration ball to be unchanged by utilizing the matching state between the calibration piece and the positioning piece, and calculating the coordinate of the sphere center of the calibration ball under the robot coordinate system.
Optionally, the method further comprises: and changing the position of the calibration ball in the working space for multiple times to obtain the center coordinates of the calibration ball at the corresponding position under the robot coordinate system.
Optionally, the performing point cloud screening includes:
removing point clouds outside the working space according to the distance between the three-dimensional point cloud camera and the working platform and the size of the working space, and only keeping the point clouds in the working space;
and screening out irrelevant point cloud information by using a voxel filtering method or a straight-through filtering method to obtain calibration spherical point cloud information.
Optionally, the extracting the calibration ball includes:
removing point clouds with non-centralized distribution by using an outlier removing algorithm;
dividing the point cloud into different parts by using a clustering algorithm, and selecting the point cloud where the calibration sphere is located according to the volume of a point cloud surrounding frame;
performing preliminary fitting on the calibration ball by using a random sampling consistency algorithm;
further screening point clouds according to the fitting result;
and performing a random sampling consistency algorithm again, and extracting fine coordinates of the calibration ball to obtain the spherical center coordinates of the calibration ball in the three-dimensional camera coordinate system.
Optionally, the calculating a hand-eye calibration relationship between the camera and the robot according to the coordinates of the center of sphere of the calibration sphere in the camera coordinate system and the robot coordinate system each time includes:
respectively calculating the mean values of the spherical center coordinates under a camera coordinate system and a robot coordinate system, and taking the difference value of the mean values of the spherical center coordinates under the two coordinate systems as the translation amount between the two coordinate systems;
respectively obtaining a group of spherical center coordinates under a camera coordinate system and a group of spherical center coordinates under a robot coordinate system, and respectively subtracting the mean value of the spherical center coordinates to complete centralization processing to obtain two groups of spherical center coordinates;
and calculating a covariance matrix H of the two groups of spherical center coordinates, and performing singular value decomposition on the covariance matrix H to obtain a pose matrix of the robot coordinate system relative to a camera coordinate system, thereby obtaining a hand-eye calibration relation between the camera and the robot.
According to another aspect of the present invention, there is provided a hand-eye calibration system of a robot vision system, comprising:
the robot module comprises a calibration ball which is arranged at the end part of the robot in advance, the position of the calibration ball in a working space is obtained by moving the calibration ball into the working space of the three-dimensional point cloud camera, and the center coordinates of the calibration ball under a robot coordinate system are calculated;
the point cloud acquisition module is used for acquiring three-dimensional point cloud information of the calibration ball in a working space of the three-dimensional point cloud camera based on the three-dimensional point cloud camera, screening point clouds to obtain calibration ball point cloud information and sending the calibration ball point cloud information to the calibration module;
the calibration module is used for controlling the position of the robot module in the working space of the three-dimensional point cloud camera, extracting a calibration ball according to the point cloud information of the calibration ball and acquiring the spherical center coordinate of the calibration ball under a three-dimensional camera coordinate system; and calculating the hand-eye calibration relation between the camera and the robot according to the coordinates of the sphere center of the calibration sphere under the camera coordinate system and the robot coordinate system each time, and completing the hand-eye calibration of the robot vision system.
According to a third aspect of the present invention, there is provided a terminal comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor when executing the program being operable to perform the method of any of the above.
According to a fourth aspect of the invention, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, is operable to perform the method of any of the above.
Due to the adoption of the technical scheme, compared with the prior art, the invention has the following beneficial effects:
the hand-eye calibration method, the hand-eye calibration system, the hand-eye calibration terminal and the hand-eye calibration medium of the robot vision system are convenient to deploy, low in cost and wide in application, can be rapidly deployed in a system formed by industrial robots of any type and three-dimensional cameras, and have important significance for promoting the manufacturing development, reducing the production cost of enterprises and improving the production efficiency.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:
fig. 1 is a flowchart illustrating a hand-eye calibration method of a robot vision system according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of the components of a hand-eye calibration system of a robot vision system according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a calibration ball according to a preferred embodiment of the present invention;
fig. 4 is a schematic diagram of the positioning of the calibration ball in the calibration link under the robot coordinate system according to a preferred embodiment of the present invention.
In the figure, 1 is a calibration piece, 2 is a positioning piece, 11 is a metal ball, 12 is a connecting rod, 13 is a base flange, and 21 is a cylindrical groove.
Detailed Description
The following examples illustrate the invention in detail: the embodiment is implemented on the premise of the technical scheme of the invention, and gives a detailed implementation mode and a specific operation process. It should be noted that various changes and modifications can be made by those skilled in the art without departing from the spirit of the invention, and these changes and modifications are all within the scope of the invention.
An embodiment of the invention provides a hand-eye calibration method for a robot vision system.
As shown in fig. 1, the hand-eye calibration method for a robot vision system provided in this embodiment may include the following steps:
s100, setting a working space of a three-dimensional point cloud camera;
s200, mounting a calibration ball at the end part of the robot, moving the calibration ball into a working space of a three-dimensional point cloud camera, controlling the position of the calibration ball in the working space, and calculating the spherical center coordinate of the calibration ball under a robot coordinate system;
s300, acquiring three-dimensional point cloud information of the calibration ball in a working space, and performing point cloud screening to obtain calibration ball point cloud information;
s400, extracting the calibration ball according to the point cloud information of the calibration ball, and acquiring the center coordinates of the calibration ball in a three-dimensional camera coordinate system;
repeating the steps, and obtaining the sphere center coordinates of the calibration sphere under the robot coordinate system and the three-dimensional camera coordinate system for multiple times;
and S500, calculating a hand-eye calibration relation between the camera and the robot according to the coordinates of the sphere center of the calibration sphere in the camera coordinate system and the robot coordinate system each time, and completing the hand-eye calibration of the robot vision system.
In a preferred embodiment of S100, the setting a working space of the three-dimensional point cloud camera includes:
and fixing the three-dimensional point cloud camera at a specific working position, and adjusting working parameters of the three-dimensional point cloud camera according to environmental factors to obtain a working space of the three-dimensional point cloud camera.
In a preferred embodiment of S200, installing a calibration ball at the end of the robot, moving the calibration ball into the working space of the three-dimensional point cloud camera, controlling the position of the calibration ball in the working space, and calculating the coordinates of the center of the calibration ball in the robot coordinate system, includes:
customizing a calibration ball, wherein the calibration ball comprises a calibration piece and a positioning piece; the positioning piece is provided with a cylindrical groove matched with the radius of the metal ball;
fixing the metal ball with the end of the robot through the connecting rod and the base flange in sequence; fixing a positioning piece on the working platform; the metal ball is contacted with the bottom surface and the side surface of the cylindrical groove to obtain a determined matching state between the calibration piece and the positioning piece, and the position of the calibration ball in the working space is fixed;
and changing the pose of the robot for many times, ensuring the position of the center of the calibration ball to be unchanged by utilizing the matching state between the calibration piece and the positioning piece, and calculating the coordinates of the center of the calibration ball in the robot coordinate system.
In a preferred embodiment of S200, the position of the calibration ball in the working space is changed multiple times, and the coordinates of the center of the sphere of the calibration ball in the robot coordinate system at the corresponding position are obtained.
In a preferred embodiment of S300, the point cloud screening includes:
removing point clouds outside the working space according to the distance between the three-dimensional point cloud camera and the working platform and the size of the working space, and only keeping the point clouds in the working space;
and screening out irrelevant point cloud information by using a voxel filtering method or a straight-through filtering method to obtain calibration spherical point cloud information.
In a preferred embodiment of S400, extracting the calibration ball includes:
removing point clouds with non-centralized distribution by using an outlier removing algorithm;
dividing the point cloud into different parts by using a clustering algorithm, and selecting the point cloud where the calibration sphere is located according to the volume of a point cloud surrounding frame; in the step, the point cloud surrounding frame is a minimum cube surrounding the specific point cloud, and the point cloud where the calibration ball is located is spherical, so the length, width and height of the surrounding frame are close to the radius of the ball. The surrounding frames of other point clouds such as the tail end of the robot, sundries, noise and the like do not have the characteristics, so that the point clouds of the calibration ball can be identified;
performing preliminary fitting on the calibration ball by using a random sampling consistency algorithm;
further screening point clouds according to the fitting result;
performing random sampling consistency algorithm again, and extracting fine coordinates of the calibration ball to obtain the center coordinates of the calibration ball in the three-dimensional camera coordinate system; in this step, at the time of initial RANSAC (random sample consensus), since the point cloud of the calibration sphere includes part of the point cloud of the connecting rod, in order to ensure that the sphere is identified, the RANSAC used this time adopts a lower standard, and calibration sphere information with poor accuracy is obtained. Before RANSAC for the second time, the point cloud outside the sphere which is preliminarily identified is filtered, only the point cloud of the sphere is basically reserved at the moment, then the information of the sphere is calculated by using high-precision RANSAC, and the coordinate of the sphere center at the moment is more accurate; the fine coordinate and the sphere center coordinate both refer to the coordinate of the sphere center under the camera coordinate system, the fine coordinate is obtained after the RANSAC for the second time, and the representing result is more accurate.
In a preferred embodiment of S500, calculating a hand-eye calibration relationship between the camera and the robot according to coordinates of the sphere center of the sphere in the camera coordinate system and the robot coordinate system each time, including:
respectively calculating the mean value of the sphere center coordinates under a camera coordinate system and a robot coordinate system, and taking the difference value of the mean values of the sphere center coordinates under the two coordinate systems as the translation amount between the two coordinate systems;
respectively obtaining a group of spherical center coordinates under a camera coordinate system and a group of spherical center coordinates under a robot coordinate system, and respectively subtracting the mean value of the spherical center coordinates to complete centralization processing to obtain two groups of spherical center coordinates;
and calculating a covariance matrix H of the two groups of spherical center coordinates, and performing singular value decomposition on the covariance matrix H to obtain a pose matrix of the robot coordinate system relative to the camera coordinate system, thereby obtaining a hand-eye calibration relation between the camera and the robot. In a specific application example, the pose matrix is a 4x4 matrix.
According to the hand-eye calibration method of the robot vision system, provided by the embodiment of the invention, the point cloud information of the calibration piece is obtained, and the point cloud information is subjected to noise reduction, segmentation and the like based on the shooting result; fixing a special spherical calibration ball at the tail end of the robot, wherein the installation error of the calibration piece is less than 0.1mm due to the special calibration ball and the flange; the robot is controlled, the space coordinates of the calibration ball are transformed, the position of the calibration ball is identified, and a hand-eye transformation matrix of the three-dimensional camera-robot is solved by combining multiple identification results.
An embodiment of the invention provides a hand-eye calibration system of a robot vision system.
As shown in fig. 2, the hand-eye calibration system of the robot vision system provided by this embodiment may include the following modules:
the robot module comprises a calibration ball which is arranged at the end part of the robot in advance, the position of the calibration ball in a working space is obtained by moving the calibration ball into the working space of the three-dimensional point cloud camera, the spherical center coordinate of the calibration ball under a robot coordinate system is calculated, and the calibration ball is sent to the calibration module;
the point cloud acquisition module is used for acquiring three-dimensional point cloud information of the calibration sphere in a working space of the three-dimensional point cloud camera based on the three-dimensional point cloud camera, screening point clouds to obtain calibration sphere point cloud information and sending the calibration sphere point cloud information to the calibration module;
the calibration module is used for controlling the position of the robot module in the working space of the three-dimensional point cloud camera, extracting a calibration ball according to the calibration ball point cloud information and acquiring the center coordinates of the calibration ball under the coordinate system of the three-dimensional camera; and calculating the hand-eye calibration relation between the camera and the robot according to the coordinates of the sphere center of the calibration sphere under the camera coordinate system and the robot coordinate system each time, and completing the hand-eye calibration of the robot vision system.
It should be noted that, the steps in the method provided by the present invention can be implemented by using corresponding modules, devices, units, etc. in the system, and those skilled in the art can implement the composition of the system by referring to the technical solution of the method, that is, the embodiment in the method can be understood as a preferred example of constructing the system.
Further, the hand-eye calibration system of the robot vision system provided by the invention comprises the following modules:
the robot module comprises a calibration ball matched with a robot (an industrial robot or a cooperative robot), the calibration ball is fixed at the tail end of the robot, and coordinates of a calibration ball center under a robot coordinate system are determined through a specially-made calibration ball calibration tool (namely a positioning part). The module can change the position of the calibration ball for many times in the calibration stage, and outputs the center coordinates of the calibration ball in the robot coordinate system to the calibration module. In a preferred embodiment, the calibration ball can be customized according to the precision and the optimal working distance of the three-dimensional camera, and comprises a metal ball with a specific radius and a connecting rod with a specific height which are fixed with the tail end of the robot through a base flange; the calibration ball is subjected to surface treatment, so that the problem that the three-dimensional camera cannot shoot due to light reflection of the metal ball is solved. The cylindrical groove of the calibration ball calibration tool can be matched with the metal ball of the calibration ball in a high-precision mode and is used for calibrating the calibration ball under a robot coordinate system.
The point cloud acquisition module is used for acquiring three-dimensional point cloud information of the calibration ball in a working space based on the three-dimensional camera, adjusting working parameters of the three-dimensional camera according to environmental factors such as environmental illumination, shooting noise and the like, and reducing the interference of external factors on the shooting effect; then, performing point cloud screening processing on the three-dimensional point cloud information, screening out irrelevant point cloud information such as a workbench, a material frame, a robot, a flange plate, a calibration ball connecting rod and the like, performing point cloud screening processing on the acquired point cloud after each point cloud acquisition to remove the irrelevant point cloud information, only keeping the point cloud information of the calibration ball, and greatly reducing the volume of a point cloud file;
and the calibration module controls the robot module to move and calls the point cloud acquisition module to acquire the point cloud information of the calibration ball at different positions. And then, extracting the calibration sphere, fitting the sphere based on the point cloud of the spherical surface of the calibration sphere, and obtaining the coordinates of the sphere center under the coordinate system of the three-dimensional camera. After at least three times of sphere center recognition, calculating the hand-eye calibration relation between the camera and the robot according to the coordinates of the sphere center under the camera coordinate system and the robot coordinate system each time, and outputting the calibration error condition.
Further, the point cloud acquisition module comprises:
and the point cloud acquisition module acquires three-dimensional point cloud information of a calibration ball in a working space based on a three-dimensional point cloud camera, the camera can shoot three-dimensional point cloud information of an object within 2 meters, and the three-dimensional point cloud information comprises three-dimensional coordinates of points in a three-dimensional camera coordinate system and gray scale and RGB color information of the points. The three-dimensional camera type is a structured light camera, and an additional light source does not need to be configured for the system.
And the camera support fixes the three-dimensional point cloud camera at a specific working position.
And the point cloud screening algorithm module adopts a point cloud screening algorithm, deletes unnecessary point cloud data (namely irrelevant point cloud information) according to the working distance input in advance and the position parameters and the size parameters of the material frame, and obtains calibration ball cloud information.
Further, the robot module includes:
based on an industrial robot or a cooperative robot, a special calibration ball is adopted and fixed at the tail end of the robot; after the calibration ball is installed, the calibration program carried by the robot is combined with the calibration tool matched with the calibration ball to complete the tool calibration of the calibration ball. The structure of the calibration ball is shown in figure 3. The calibration ball calibrates the positioning of the link in the robot coordinate system, as shown in fig. 4.
Further, the calibration module includes:
a calibration ball extraction algorithm module which extracts the precise position of the calibration ball from the complex point cloud by using a calibration ball extraction algorithm and returns the coordinates of the center of the calibration ball;
and the hand-eye calibration algorithm module calculates the hand-eye relationship between the camera and the robot based on the coordinates of the calibration balls at the at least three positions in the robot coordinate system and the camera coordinate system, and returns a calibrated matrix and an error result.
Further, the point cloud screening algorithm comprises the following steps:
step 1, removing point clouds outside a working space according to the distance between a three-dimensional point cloud camera and a working platform and the size of the working space, and only keeping the point clouds in the working space;
and 2, screening out irrelevant point cloud information by using a voxel filtering method or a straight-through filtering method, reducing the density of the point cloud, and keeping the distribution characteristics of the point cloud to obtain calibrated spherical point cloud information.
Further, the calibration sphere extraction algorithm comprises the following steps:
firstly, removing point clouds which are not distributed in a centralized manner by using an outlier removing algorithm, thereby realizing the effects of retaining the point clouds of the calibration ball and separating the calibration ball from other point clouds;
secondly, dividing the point cloud into different parts by using a clustering algorithm, and selecting the point cloud where the calibration sphere is located according to the volume of a surrounding frame of the point cloud;
thirdly, performing preliminary fitting on the calibration sphere by using a RANSAC (random sample consensus) algorithm;
fourthly, further screening the point cloud according to the fitting result;
and fifthly, performing RANSAC (random sample consensus) algorithm again, extracting fine coordinates of the calibration ball, and returning the coordinates of the center of the ball obtained by calibration as a result.
In the hand-eye calibration system of the robot vision system provided by the above embodiment of the present invention, the robot module is provided with the calibration ball in advance, calculates the tool coordinate system of the calibration ball, moves the calibration ball into the working space of the camera, and transmits the center coordinates of the calibration ball in the camera coordinate system to the calibration module; the point cloud acquisition module is used for acquiring three-dimensional point cloud information in a working space, performing down-sampling processing (point cloud screening processing) on the three-dimensional point cloud information, and transmitting the processed point cloud to the calibration module; the calibration module starts to work after acquiring the spherical center coordinates and the point cloud data of at least three groups of calibration balls at different positions, extracts the calibration balls, acquires the spherical center coordinates of the calibration balls in a camera coordinate system, and solves a calibration matrix of the camera-robot according to a calibration algorithm.
An embodiment of the present invention provides a terminal, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor is configured to execute the method according to any one of the above embodiments of the present invention when executing the computer program.
Optionally, a memory for storing a program; a Memory, which may include a volatile Memory (RAM), such as a Random Access Memory (SRAM), a Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM), and the like; the memory may also comprise a non-volatile memory, such as a flash memory. The memories are used to store computer programs (e.g., applications, functional modules, etc. that implement the above-described methods), computer instructions, etc., which may be stored in partition in the memory or memories. And the computer programs, computer instructions, data, etc. described above may be invoked by a processor.
The computer programs, computer instructions, etc. described above may be stored in partitions in one or more memories. And the computer programs, computer instructions, data, etc. described above may be invoked by a processor.
A processor for executing the computer program stored in the memory to implement the steps of the method according to the above embodiments. Reference may be made in particular to the description relating to the preceding method embodiment.
The processor and the memory may be separate structures or may be an integrated structure integrated together. When the processor and the memory are separate structures, the memory, the processor may be coupled by a bus.
An embodiment of the invention also provides a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, is adapted to carry out the method of any of the above-mentioned embodiments of the invention.
The hand-eye calibration method, the hand-eye calibration system, the hand-eye calibration terminal and the hand-eye calibration medium of the robot vision system provided by the embodiment of the invention are convenient to deploy, low in cost and wide in application, can be rapidly deployed in a system formed by industrial robots of any type and three-dimensional cameras, and have important significance for promoting the development of manufacturing industry, reducing the production cost of enterprises and improving the production efficiency.
The above embodiments of the present invention are not exhaustive and are all known in the art.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention.
Claims (10)
1. A hand-eye calibration method of a robot vision system is characterized by comprising the following steps:
setting a working space of a three-dimensional point cloud camera;
installing a calibration ball at the end part of the robot, moving the calibration ball into a working space of the three-dimensional point cloud camera, controlling the position of the calibration ball in the working space, and calculating the center coordinates of the calibration ball under a robot coordinate system;
acquiring three-dimensional point cloud information of the calibration ball in a working space, and performing point cloud screening to obtain calibration ball point cloud information;
extracting the calibration ball according to the point cloud information of the calibration ball to obtain the center coordinates of the calibration ball under a three-dimensional camera coordinate system;
repeating the steps, and obtaining the sphere center coordinates of the calibration sphere under a robot coordinate system and a three-dimensional camera coordinate system for multiple times;
and calculating the hand-eye calibration relation between the camera and the robot according to the coordinates of the sphere center of the calibration sphere under the camera coordinate system and the robot coordinate system each time, and completing the hand-eye calibration of the robot vision system.
2. The method of claim 1, wherein the setting a working space of the three-dimensional point cloud camera comprises:
and fixing the three-dimensional point cloud camera at a specific working position, and adjusting working parameters of the three-dimensional point cloud camera according to environmental factors to obtain a working space of the three-dimensional point cloud camera.
3. The hand-eye calibration method of the robot vision system according to claim 1, wherein the installing a calibration sphere at the end of the robot and moving the calibration sphere into the working space of the three-dimensional point cloud camera, controlling the position of the calibration sphere in the working space, and calculating the center coordinates of the calibration sphere in the robot coordinate system comprises:
customizing a calibration ball, wherein the calibration ball comprises a calibration piece and a positioning piece; the positioning piece comprises a metal ball, a connecting rod and a base flange, and is provided with a cylindrical groove matched with the radius of the metal ball;
fixing the metal ball with the end of the robot through the connecting rod and the base flange in sequence; fixing the positioning piece on the working platform; the metal ball is contacted with the bottom surface and the side surface of the cylindrical groove to obtain a determined matching state between the calibration piece and the positioning piece, and the position of the calibration ball in a working space is fixed;
and changing the pose of the robot for many times, ensuring the position of the sphere center of the calibration ball to be unchanged by utilizing the matching state between the calibration piece and the positioning piece, and calculating the coordinate of the sphere center of the calibration ball under the robot coordinate system.
4. A hand-eye calibration method for a robot vision system according to claim 3, further comprising:
and changing the position of the calibration ball in the working space for multiple times to obtain the spherical center coordinates of the calibration ball at the corresponding position under the robot coordinate system.
5. The method for calibrating a hand-eye of a robot vision system according to claim 1, wherein the performing point cloud screening comprises:
removing point clouds outside the working space according to the distance between the three-dimensional point cloud camera and the working platform and the size of the working space, and only keeping the point clouds in the working space;
and screening out irrelevant point cloud information by using a voxel filtering method or a straight-through filtering method to obtain calibration spherical point cloud information.
6. The hand-eye calibration method of a robot vision system according to claim 1, wherein said extracting calibration balls comprises:
removing point clouds with non-centralized distribution by using an outlier removing algorithm;
dividing the point cloud into different parts by using a clustering algorithm, and selecting the point cloud where the calibration sphere is located according to the volume of a point cloud surrounding frame;
carrying out preliminary fitting on the calibration ball by using a random sampling consistency algorithm;
further screening point clouds according to the fitting result;
and performing a random sampling consistency algorithm again, and extracting fine coordinates of the calibration ball to obtain the center coordinates of the calibration ball in the three-dimensional camera coordinate system.
7. The hand-eye calibration method for the robot vision system according to claim 1, wherein the calculating the hand-eye calibration relationship between the camera and the robot according to the coordinates of the sphere center of the calibration sphere in the camera coordinate system and the robot coordinate system each time comprises:
respectively calculating the mean values of the spherical center coordinates under a camera coordinate system and a robot coordinate system, and taking the difference value of the mean values of the spherical center coordinates under the two coordinate systems as the translation amount between the two coordinate systems;
respectively obtaining a group of spherical center coordinates under a camera coordinate system and a group of spherical center coordinates under a robot coordinate system, and respectively subtracting the mean value of the spherical center coordinates to complete centralization processing to obtain two groups of spherical center coordinates;
and calculating a covariance matrix H of the two groups of spherical center coordinates, and performing singular value decomposition on the covariance matrix H to obtain a pose matrix of the robot coordinate system relative to a camera coordinate system, thereby obtaining a hand-eye calibration relation between the camera and the robot.
8. A hand-eye calibration system for a robotic vision system, comprising:
the robot module comprises a calibration ball which is arranged at the end part of the robot in advance, the position of the calibration ball in a working space is obtained by moving the calibration ball into the working space of the three-dimensional point cloud camera, the center coordinates of the calibration ball under a robot coordinate system are calculated, and the center coordinates are sent to the calibration module;
the point cloud acquisition module is used for acquiring three-dimensional point cloud information of the calibration ball in a working space of the three-dimensional point cloud camera based on the three-dimensional point cloud camera, screening point clouds to obtain calibration ball point cloud information and sending the calibration ball point cloud information to the calibration module;
the calibration module is used for controlling the position of the robot module in the working space of the three-dimensional point cloud camera, extracting a calibration ball according to the calibration ball point cloud information and acquiring the spherical center coordinates of the calibration ball in a three-dimensional camera coordinate system; and calculating the hand-eye calibration relation between the camera and the robot according to the coordinates of the sphere center of the calibration sphere under the camera coordinate system and the robot coordinate system each time, and completing the hand-eye calibration of the robot vision system.
9. A terminal comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor, when executing the program, is operative to perform the method of any of claims 1-7.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, is adapted to carry out the method of any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211041992.3A CN115488878A (en) | 2022-08-29 | 2022-08-29 | Hand-eye calibration method, system, terminal and medium for robot vision system |
| PCT/CN2022/125016 WO2024045274A1 (en) | 2022-08-29 | 2022-10-13 | Hand-eye calibration method and system for robot vision system, and terminal and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211041992.3A CN115488878A (en) | 2022-08-29 | 2022-08-29 | Hand-eye calibration method, system, terminal and medium for robot vision system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115488878A true CN115488878A (en) | 2022-12-20 |
Family
ID=84466307
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211041992.3A Pending CN115488878A (en) | 2022-08-29 | 2022-08-29 | Hand-eye calibration method, system, terminal and medium for robot vision system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN115488878A (en) |
| WO (1) | WO2024045274A1 (en) |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103925879A (en) * | 2014-04-24 | 2014-07-16 | 中国科学院合肥物质科学研究院 | Indoor robot vision hand-eye relation calibration method based on 3D image sensor |
| CN110355754B (en) * | 2018-12-15 | 2023-09-22 | 深圳铭杰医疗科技有限公司 | Robot hand-eye system, control method, device and storage medium |
| CN110355755B (en) * | 2018-12-15 | 2023-05-16 | 深圳铭杰医疗科技有限公司 | Robot hand-eye system calibration method, device, equipment and storage medium |
| CN110116411B (en) * | 2019-06-06 | 2020-10-30 | 浙江汉振智能技术有限公司 | Robot 3D vision hand-eye calibration method based on spherical target |
| CN112091971B (en) * | 2020-08-21 | 2021-10-12 | 季华实验室 | Robot eye calibration method and device, electronic equipment and system |
| JP2022122648A (en) * | 2021-02-10 | 2022-08-23 | 株式会社キーエンス | Control device |
| CN113362396B (en) * | 2021-06-21 | 2024-03-26 | 上海仙工智能科技有限公司 | Mobile robot 3D hand-eye calibration method and device |
-
2022
- 2022-08-29 CN CN202211041992.3A patent/CN115488878A/en active Pending
- 2022-10-13 WO PCT/CN2022/125016 patent/WO2024045274A1/en unknown
Also Published As
| Publication number | Publication date |
|---|---|
| WO2024045274A1 (en) | 2024-03-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107063228B (en) | Target attitude calculation method based on binocular vision | |
| CN111267095B (en) | Mechanical arm grabbing control method based on binocular vision | |
| CN112066879A (en) | Air floatation motion simulator pose measuring device and method based on computer vision | |
| CN113223091B (en) | Three-dimensional target detection method, three-dimensional target capture device and electronic equipment | |
| DE112012005350T5 (en) | Method for estimating the position of an object | |
| CN111476841B (en) | Point cloud and image-based identification and positioning method and system | |
| CN107588721A (en) | The measuring method and system of a kind of more sizes of part based on binocular vision | |
| CN113269840A (en) | Combined calibration method for camera and multi-laser radar and electronic equipment | |
| CN113379849A (en) | Robot autonomous recognition intelligent grabbing method and system based on depth camera | |
| US20220230348A1 (en) | Method and apparatus for determining a three-dimensional position and pose of a fiducial marker | |
| CN113221953B (en) | Target attitude identification system and method based on example segmentation and binocular depth estimation | |
| CN112907735A (en) | Flexible cable identification and three-dimensional reconstruction method based on point cloud | |
| CN113361365B (en) | Positioning method, positioning device, positioning equipment and storage medium | |
| CN111583342B (en) | Target rapid positioning method and device based on binocular vision | |
| CN114494150A (en) | Design method of monocular vision odometer based on semi-direct method | |
| CN115685160A (en) | Target-based laser radar and camera calibration method, system and electronic equipment | |
| CN113269723A (en) | Unordered grasping system for three-dimensional visual positioning and mechanical arm cooperative work parts | |
| CN110992416A (en) | High-reflection-surface metal part pose measurement method based on binocular vision and CAD model | |
| CN115284279A (en) | Mechanical arm grabbing method and device based on aliasing workpiece and readable medium | |
| JP2730457B2 (en) | Three-dimensional position and posture recognition method based on vision and three-dimensional position and posture recognition device based on vision | |
| JP6939729B2 (en) | Control systems, control methods and programs | |
| JP2778430B2 (en) | Three-dimensional position and posture recognition method based on vision and three-dimensional position and posture recognition device based on vision | |
| CN115488878A (en) | Hand-eye calibration method, system, terminal and medium for robot vision system | |
| CN116038701B (en) | Hand-eye calibration method and device for four-axis mechanical arm | |
| CN114187312A (en) | Target object grabbing method, device, system, storage medium and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |