CN115519547A - Reinforcing steel bar binding hand-eye calibration method and system - Google Patents
Reinforcing steel bar binding hand-eye calibration method and system Download PDFInfo
- Publication number
- CN115519547A CN115519547A CN202211309702.9A CN202211309702A CN115519547A CN 115519547 A CN115519547 A CN 115519547A CN 202211309702 A CN202211309702 A CN 202211309702A CN 115519547 A CN115519547 A CN 115519547A
- Authority
- CN
- China
- Prior art keywords
- point
- calibration plate
- image
- mechanical arm
- camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 229910001294 Reinforcing steel Inorganic materials 0.000 title claims description 14
- 239000011159 matrix material Substances 0.000 claims abstract description 66
- 230000009466 transformation Effects 0.000 claims abstract description 31
- 229910000831 Steel Inorganic materials 0.000 claims abstract description 24
- 239000010959 steel Substances 0.000 claims abstract description 24
- 230000002787 reinforcement Effects 0.000 claims description 78
- 238000013519 translation Methods 0.000 claims description 13
- 238000001514 detection method Methods 0.000 claims description 10
- 238000000354 decomposition reaction Methods 0.000 claims description 8
- 238000004891 communication Methods 0.000 claims description 5
- 238000003384 imaging method Methods 0.000 claims 1
- 230000003014 reinforcing effect Effects 0.000 abstract description 4
- 238000004590 computer program Methods 0.000 description 6
- 230000000007 visual effect Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 210000003128 head Anatomy 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000004308 accommodation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000004438 eyesight Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
- B25J9/1692—Calibration of manipulator
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1669—Programme controls characterised by programming, planning systems for manipulators characterised by special application, e.g. multi-arm co-operation, assembly, grasping
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Automation & Control Theory (AREA)
- Image Processing (AREA)
Abstract
The invention provides a hand-eye calibration method and a hand-eye calibration system for binding steel bars, wherein the method comprises the following steps: placing the calibration plate at an image acquisition point of a camera, and controlling the camera to acquire an image of the calibration plate; identifying each angular point in the calibration plate image so as to obtain a first coordinate set of the angular point of the calibration plate in a camera coordinate system; moving the calibration plate to a binding work site of a steel bar, controlling the mechanical arm to enable a pointer of the mechanical arm to point to each angular point of the calibration plate in sequence, and recording the pose of the mechanical arm when the mechanical arm points to each angular point each time, so as to obtain a second coordinate set of the angular points of the calibration plate under a mechanical arm coordinate system; and calculating a transformation matrix between the first coordinate set and the second coordinate set. The problem of among the prior art the eye in the calibration mode of hand outside, only be applicable to the condition that reinforcing bar image acquisition point and reinforcing bar ligature worker position point distance are close, the scene that is suitable for reinforcing bar ligature is more single, can not satisfy the diversified ligature task's of scene technical problem is solved.
Description
Technical Field
The invention relates to the field of intelligent assembly type buildings, in particular to a reinforcing steel bar binding hand-eye calibration method and system.
Background
The automatic steel bar binding robot collects images through the camera, so that binding nodes (intersection points of two steel bars) of a steel bar cage are identified, then the mechanical arm is controlled to bind the steel bar binding nodes, and the realization of the technology needs to be calibrated by hands and eyes, namely, the coordinate of a camera coordinate system and the transformation relation of the coordinate of a mechanical arm coordinate system are obtained, so that the mechanical arm can accurately reach the positions of the binding nodes.
In the prior art, a calibration mode that eyes are outside hands is often adopted, namely, the moving range of a mechanical arm is in the visual field range of a camera, and then the calibration of the hands and the eyes is realized by utilizing a calibration plate.
It should be noted here that, in the prior art, the calibration mode with eyes outside the hand is only suitable for the situation that the distance between the image acquisition point of the steel bar and the work site of the steel bar binding is short, and the scene suitable for the steel bar binding is single, and the diversified binding task of the scene cannot be satisfied.
The invention is provided in view of the above.
Disclosure of Invention
The invention provides a reinforcing steel bar binding hand-eye calibration method and system, which are used for solving the technical problems that in the prior art, a calibration mode with eyes outside hands is only suitable for the condition that a reinforcing steel bar image acquisition point is close to a reinforcing steel bar binding work site, a scene suitable for binding reinforcing steel bars is single, and the binding task with diversified scenes cannot be met.
According to a first aspect of the invention, a reinforcing steel bar binding hand-eye calibration method is provided, which comprises the following steps: placing the calibration plate at an image acquisition point of a camera, and controlling the camera to acquire an image of the calibration plate; identifying each angular point in the calibration plate image so as to obtain a first coordinate set of the angular point of the calibration plate in a camera coordinate system; moving the calibration plate to a binding work site of a steel bar, controlling the mechanical arm to enable a pointer of the mechanical arm to point to each angular point of the calibration plate in sequence, and recording the pose of the mechanical arm when the mechanical arm points to each angular point each time, so as to obtain a second coordinate set of the angular points of the calibration plate under a mechanical arm coordinate system; and calculating a transformation matrix between the first coordinate set and the second coordinate set.
Further, before placing the calibration plate at an image capture point of the camera, the method further comprises: acquiring a steel bar binding task, wherein the steel bar binding task comprises a set image acquisition point position and a binding work site position; determining the position of the camera according to the position of the image acquisition point, the working distance and the precision of the camera; and determining the position of the mechanical arm according to the position of the binding work site and the working interval of the mechanical arm.
Further, after calculating a transformation matrix between the first coordinate set and the second coordinate set, the method further includes: after the reinforcement cage reaches the image acquisition point, the camera acquires an image of the reinforcement cage, and a first node coordinate set of each node in the reinforcement cage under a camera coordinate system is obtained through image identification of the reinforcement cage; controlling the reinforcement cage to reach the binding work site from the image acquisition point; converting the first node coordinate set based on the transformation matrix to obtain a second node coordinate set of each node in the reinforcement cage under the mechanical arm coordinate; and controlling the mechanical arm to bind each node of the reinforcement cage according to the second node coordinate set.
Furthermore, a preset distance is reserved between the image acquisition point and the binding work site, so that the moving range of the mechanical arm is out of the visual field range of the camera, and when the mechanical arm binds each node of the reinforcement cage according to the second node coordinate set, the next reinforcement cage is controlled to reach the image acquisition point and the camera is controlled to acquire the point cloud image of the next reinforcement cage.
Further, the calibration plate image is a three-dimensional point cloud image, wherein identifying each corner point in the calibration plate image comprises: performing two-dimensional projection on the calibration plate image to obtain a two-dimensional image of the calibration plate; performing corner detection on the two-dimensional image by adopting a checkerboard corner detection method to obtain two-dimensional coordinates of each corner in the calibration plate; and restoring each angular point from the two-dimensional coordinates to the calibration plate three-dimensional point cloud image to obtain the three-dimensional coordinates of each angular point in the calibration plate point cloud image.
Further, calculating a transformation matrix between the first set of coordinates and the second set of coordinates includes: and performing singular value decomposition on the first coordinate set and the second coordinate set to obtain the transformation matrix, wherein the transformation matrix comprises a rotation matrix R and a translation matrix T.
Further, calculating the rotation matrix R includes: calculating to obtain a first central point in the first coordinate set and a second central point in the second coordinate set; moving each coordinate in the first coordinate set according to the first central point to obtain a moved first coordinate set; moving each coordinate in the second coordinate set to the second central point to obtain a moved second coordinate set; calculating a covariance matrix H between the first coordinate set and the second coordinate set;
and carrying out singular value decomposition on the covariance matrix H to obtain the rotation matrix R.
Further, a translation matrix T is calculated, comprising: and obtaining the translation matrix T based on the rotation matrix R, the first central point and the second central point.
According to a second aspect of the invention, a reinforcing steel bar binding hand-eye calibration system is provided, which comprises: a camera for capturing an image at an image capture point; the mechanical arm is used for binding the binding station; calibrating the plate; the slide rail is used for moving the calibration plate from the image acquisition point to the binding work site under the condition of receiving a calibration instruction sent by a controller, wherein when the calibration plate is positioned at the image acquisition point, the camera acquires an image of the calibration plate, and when the calibration plate is positioned at the binding work site, the mechanical arm points to each angular point in the calibration plate in sequence; the controller is in communication relation with the camera and the mechanical arm and used for receiving the image of the calibration board and recording the pose of the mechanical arm when the mechanical arm points to each angular point; the controller is further used for generating a transformation matrix from the coordinate system of the camera to the coordinate system of the mechanical arm according to the image of the calibration plate and the pose of the mechanical arm when the mechanical arm points to each corner point.
Further, the controller is also used for determining the position of the image acquisition point and the position of the binding work site from the received steel bar binding task, the controller is also used for determining the position of the camera according to shooting parameters of the camera and the position of the image acquisition point, and the controller is also used for determining the position of the mechanical arm according to the position of the binding work site and working parameters of the mechanical arm.
The invention provides a hand-eye calibration method and a hand-eye calibration system for binding steel bars, wherein the method comprises the following steps: placing the calibration plate at an image acquisition point of a camera, and controlling the camera to acquire an image of the calibration plate; identifying each angular point in the calibration plate image so as to obtain a first coordinate set of the angular point of the calibration plate in a camera coordinate system; moving the calibration plate to a binding work site of a steel bar, controlling the mechanical arm to enable a pointer of the mechanical arm to point to each angular point of the calibration plate in sequence, and recording the pose of the mechanical arm when the mechanical arm points to each angular point each time, so as to obtain a second coordinate set of the angular points of the calibration plate under a mechanical arm coordinate system; and calculating a transformation matrix between the first coordinate set and the second coordinate set. The problem of among the prior art the eye in the calibration mode of outside hand, only be applicable to the condition that reinforcing bar image acquisition point and reinforcement work station point distance are close is solved, the scene that is suitable for reinforcement is single, can not satisfy the diversified ligature task of scene technical problem.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a flow chart of a reinforcing steel bar binding hand-eye calibration method provided by the invention;
FIG. 2 is a schematic view of a calibration plate provided by the present invention;
FIG. 3 is a schematic diagram of a scene of hand-eye calibration provided by the present invention;
FIG. 4 is a schematic diagram of the robot pointing to each corner point provided by the present invention;
FIG. 5 is a schematic view of a rebar tying scenario provided by the present invention;
fig. 6 is a schematic diagram of the hand-eye calibration system for reinforcing steel bar binding provided by the invention.
Detailed Description
In order to make the aforementioned and other features and advantages of the invention more apparent, the invention is further described below with reference to the accompanying drawings. It is understood that the specific embodiments described herein are for purposes of illustration only and are not intended to be limiting.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to one of ordinary skill in the art that the specific details need not be employed to practice the present invention. In other instances, well-known steps or operations are not described in detail to avoid obscuring the invention.
Example one
The application provides a reinforcing steel bar binding hand-eye calibration method, as shown in fig. 1, the method comprises the following steps:
and S11, placing the calibration plate at an image acquisition point of the camera, and controlling the camera to acquire an image of the calibration plate.
Specifically, in the present solution, a computer device with a data processing function may be used as an execution main body of the method of the present solution, and the computer device may be a controller of the steel bar binding robot. According to the scheme, the calibration plate can be placed at an image acquisition point in the visual field range in front of the camera, and then the camera is controlled to photograph the calibration plate. It should be noted that fig. 2 is a schematic diagram of the calibration board provided in this embodiment, in order to identify the calibration board more easily, the calibration board provided in this embodiment may be in a checkerboard shape, the calibration board is composed of a plurality of black blocks and white blocks, a plurality of corner points are formed between each black block and the white block on the periphery of the black block, and the corner point shown in fig. 2 is one of the plurality of corner points.
And S13, identifying each corner point in the calibration plate image so as to obtain a first coordinate set of the calibration plate corner point under a camera coordinate system.
Specifically, because the calibration plate image is acquired by the camera, the scheme identifies the image acquired by the camera to obtain a set of coordinates of each corner point of the calibration plate, and the set of coordinates of the corner points of the calibration plate under the camera coordinates is all the coordinates of the corner points of the calibration plate under the camera coordinates.
And S15, moving the calibration plate to a binding work site of a steel bar, controlling the mechanical arm to enable the pointer of the mechanical arm to point to each angular point of the calibration plate in sequence, and recording the pose of the mechanical arm each time the mechanical arm points to each angular point, so as to obtain a second coordinate set of the angular points of the calibration plate under a mechanical arm coordinate system. In fig. 3, C is a camera, B is a mechanical arm (which may also be referred to as a manipulator) of a banding robot, and the mechanical arm is provided with a banding head, which can achieve automatic banding of the rebar junction. P1 is an image acquisition point and is positioned in the acquisition range of the camera, and P2 is a binding work site and is positioned in the moving range of the mechanical arm. When the calibration plate is in the coordinate system P1, the camera collects an image of the calibration plate and obtains a first coordinate set of angular points of the calibration plate under the coordinate system of the camera, then the calibration plate is moved from the image collection point P1 to the binding work station point P2, and in combination with the graph of FIG. 4, after a user sees a plurality of angular points in the calibration plate, the controller of the mechanical arm can be manually operated to enable the mechanical arm to point to each angular point in the calibration plate in sequence, then the pointer of the mechanical arm points to each angular point every time, the controller of the mechanical arm records the pose of the mechanical arm when the mechanical arm points to each angular point, and the pose of the mechanical arm can be a physical coordinate value of the angular point of the calibration plate under the coordinate system of the mechanical arm, so that a second coordinate set of the calibration plate under the coordinate system of the mechanical arm can be obtained.
And S17, calculating to obtain a transformation matrix between the first coordinate set and the second coordinate set.
It should be noted here that, the present scheme is different from the calibration mode of "hand is out of eye" in the prior art, and the calibration mode of "hand is out of eye" in the prior art limits that the image acquisition point must be close to the binding work site to satisfy that the moving range of the mechanical arm is always within the visual field range of the camera, so as to perform calibration of "hand is out of eye", that is, the existing calibration mode of "hand is out of eye" can only satisfy the reinforcement cage image acquisition point and the reinforcement cage binding work site closer reinforcement task, and if the reinforcement cage image acquisition point and the binding work site are required to be far away by the reinforcement task, the calibration mode of "hand is out of eye" in the prior art cannot achieve calibration. According to the scheme, the calibration plate is moved, the first coordinate set and the second coordinate set are obtained successively, then the transformation matrix between the first coordinate set and the second coordinate set is obtained, various binding tasks can be met based on the calibration mode of the movable calibration plate, namely no matter what distance is between the reinforcement cage image acquisition point and the reinforcement cage binding station, no matter whether the distance is close or far, the calibration mode of the scheme can meet the working condition without being limited by the distance, therefore, the scheme solves the problem that the calibration mode of eyes outside hands in the prior art is only suitable for the condition that the distance between the reinforcement cage image acquisition point and the reinforcement binding station is close, the applicable reinforcement binding scene is single, and the technical problem that the diversified binding tasks in the scene cannot be met.
Optionally, it should be noted that the camera may be a laser radar, the calibration plate image may be a point cloud image, the point cloud image contains rich information, and the point cloud image is used for calculation, so that the reinforcement bar binding operation is more accurate.
Optionally, in step S11, before the calibration board is placed at the image capturing point of the camera, the method may include:
and S08, acquiring a steel bar binding task, wherein the steel bar binding task comprises the position of a set image acquisition point and the position of the binding work site.
And S09, determining the position of the camera according to the position of the image acquisition point and the shooting parameters of the camera.
Specifically, according to the scheme, the position of the camera can be determined according to the position of the image acquisition point and the shooting parameters of the camera, so that under the determined position of the camera, on one hand, the image acquisition point is in the shooting view of the camera, and on the other hand, the camera can achieve the best shooting effect on the image acquisition point.
And S10, determining the position of the mechanical arm according to the position of the binding work station and the working parameters of the mechanical arm.
Specifically, this scheme can be according to the position of ligature work site, the position of arm is confirmed to the working parameter of arm for, under the position of camera confirmed, on the one hand ligature work site is under the home range of arm, and on the other hand arm can realize realizing best work, ligature effect to ligature work site.
It should be noted that, in this scheme, can confirm the image acquisition point 'S that master worker set for position and the position of ligature worker position point according to the reinforcement task that receives, that is to say, through this scheme, master worker only need set for image acquisition point' S position and the position of ligature worker position point, this scheme then is automatic according to the position of the image acquisition point who sets for and the position of ligature worker position point generates the position of camera and the position of arm then, and then from accomplishing the demarcation task of above-mentioned step S11 to step S17. Through this embodiment, the workman master can be according to the position of the arbitrary image acquisition point of setting for of operating mode of reality and the position of ligature worker position point, this put case then according to the position of the image acquisition point of above-mentioned setting and the hand eye under the operating mode that the workman master set for is realized to the position of ligature worker position point is markd, and the hand eye under the operating mode of arbitrary operating mode is markd can be satisfied to this embodiment promptly.
Optionally, after the step S17 calculates a transformation matrix between the first coordinate set and the second coordinate set, the method provided by the present application further includes:
and S19, controlling the camera to collect the image of the reinforcement cage after the reinforcement cage reaches the image collecting point, and identifying the image of the reinforcement cage to obtain a first node coordinate set of each node in the reinforcement cage under a camera coordinate system.
And S21, controlling the reinforcement cage to reach the binding work site from the image acquisition point.
And S23, converting the first node coordinate set based on the transformation matrix to obtain a second node coordinate set of each node in the reinforcement cage under the mechanical arm coordinate.
And S25, controlling a mechanical arm to bind each node of the reinforcement cage according to the second node coordinate set.
Specifically, after calibration is completed, the conversion relation (namely the transformation matrix) between the camera coordinate system and the mechanical arm coordinate system is obtained, an actual reinforcement binding task can be performed, namely, the reinforcement cage is moved to an image acquisition point P1 through a slide rail, an image of the reinforcement cage is acquired by the camera, and then a coordinate set of a plurality of nodes in the reinforcement cage is obtained through image recognition. After the first node coordinate set is obtained, a second node coordinate set can be obtained through the transformation matrix, and the second node coordinate set is a set of each node in the mechanical arm coordinate system. According to the scheme, after the reinforcement cage is controlled to move from the image acquisition point P1 to the binding work site P2 through the slide rail, the mechanical arm (specifically, the binding head on the mechanical arm) is controlled to bind each node of the reinforcement cage according to the second node coordinate set. It should be noted that the calibration mode in this scheme is not restricted by image acquisition point and reinforcement station distance, therefore this scheme is in accomplishing the reinforcement task of execution reality after maring, and the displacement (namely P1 to P2) that the steel reinforcement cage produced when removing can follow the optional change of operating condition.
Optionally, a preset distance is provided between the image acquisition point and the binding work site, so that a moving range of the mechanical arm is out of a visual field range of the camera, and when the mechanical arm binds each node of the reinforcement cage according to the second node coordinate set, a next reinforcement cage is controlled to reach the image acquisition point and the camera is controlled to acquire a point cloud image of the next reinforcement cage.
It should be noted here that, in this scheme, there may be the interval between image acquisition point and the ligature work site to have the distance of predetermineeing, the camera also can be the same distance of predetermineeing at the interval to the arm, this distance of predetermineeing can make camera and arm have independent working range separately, namely in this embodiment, the home range of arm is not within the field of vision scope of camera, consequently, can have two reinforcement cages to be in image acquisition point and ligature work site respectively at the same moment, combine fig. 5, namely first reinforcement cage is in image acquisition point, the second reinforcement cage is in ligature work site, that is to say at the same moment, this scheme can control the camera and gather the image of first reinforcement cage and carry out node identification, control mechanical arm carries out the ligature to the second reinforcement cage simultaneously. The following analysis is made to the technical effect of this embodiment: in the prior art, after the camera collects an image of a reinforcement cage at a first moment and recognizes a node, the mechanical arm is controlled to bind the reinforcement cage according to a node coordinate after coordinate conversion at a second moment, it should be noted that in the prior art, a moment inevitably exists, the camera or the mechanical arm is in an idle state, for example, at the first moment, the camera collects an image, the mechanical arm is in an idle state, and at the second moment, the mechanical arm is in an idle state, and through the embodiment, because a preset distance exists between the image collection point P1 and the binding station P2, the preset distance can enable the first reinforcement cage to be in the image collection point P1, the second reinforcement cage also has an accommodation space, namely, the second reinforcement cage can be in the binding work site P2, because the preset distance exists between the image collection point P1 and the binding station P2 (the same preset distance also can exist between the camera and the mechanical arm), the camera and the mechanical arm can have respective work ranges at the same moment and do not interfere with each other, therefore, at the camera and the mechanical arm can simultaneously control the node of the first collection of the reinforcement cage, and the binding efficiency of the mechanical arm can be improved. That is to say, this scheme provides a calibration method, provides a brand-new reinforcement mode on this kind of calibration method's basis, can improve the efficiency of reinforcement.
Optionally, the calibration plate image is a three-dimensional point cloud image, wherein step S13 identifies each corner point in the calibration plate image, and includes:
and S131, performing two-dimensional projection on the calibration plate image to obtain a two-dimensional image of the calibration plate.
Step S133, performing corner detection on the two-dimensional image by adopting a growth-based checkerboard corner detection method to obtain two-dimensional coordinates of each corner in the calibration plate. Compared with the common harris corner detection algorithm, the checkerboard corner detection method (Shi-Tomasi corner detection algorithm) is strong in noise resistance and high in accuracy.
Specifically, the scheme can utilize a checkerboard corner detection method (cv:: findchessboardcorrers) in opencv to detect the corner, and further obtain the pixel coordinate value (Xp, yp) of each corner in the two-dimensional image of the calibration board.
And S135, restoring the each angular point from the two-dimensional coordinates to the three-dimensional point cloud image of the calibration plate to obtain the three-dimensional coordinates of each angular point in the calibration plate point cloud image.
Specifically, in the present embodiment, the pixel coordinate values (Xp, yp) of each corner point in step S133 are restored back to the three-dimensional point cloud image by using a back projection point set averaging method, and the three-dimensional physical coordinate values of each corner point are obtained as coordinate values (Xc, yc, zc) of the calibration board corner point in the camera coordinate system.
Optionally, step S17 calculates a transformation matrix between the first coordinate set and the second coordinate set, and includes:
and performing singular value decomposition on the first coordinate set and the second coordinate set to obtain the transformation matrix, wherein the transformation matrix comprises a rotation matrix R and a translation matrix T. The method for solving the transformation matrix by using the singular value decomposition is simple and convenient, is easy to solve, and is suitable for solving the situation of rigid transposed information between two point sets in the process of calibrating the hands and the eyes by knowing the point set under the hand coordinate system and the point set under the corresponding eye coordinate system.
Step S171, specifically, because there is a preset distance between the image capturing point and the reinforcement bar binding point, the transformation matrix includes a translation matrix, and when performing a reinforcement bar binding task, the transformation of coordinates can be realized by the two matrices regardless of the distance of displacement of the reinforcement cage through the rotation matrix R and the translation matrix T.
It should be noted here that since the movement of the calibration plate from the photographing station P1 to the ligating station P2 is a fixed translational movement along the vector P1P2, there are one rotation matrix R and one translation matrix T such that B = R × C + T holds, where B is the set of points of the first set of coordinates and C is the set of points of the second set of coordinates.
Optionally, calculating the rotation matrix R includes:
step S1711, a first center point in the first coordinate set and a second center point in the second coordinate set are obtained through calculation.
Specifically, the first coordinate set may be a point set C, the second coordinate set may be a point set B, and the solution may be calculated by the following two formulas, where a central point μ of the point set C is C And the center point mu of the point set B B ：
Wherein,
for the ith coordinate in point set C,
is the ith coordinate in point set B.
Step S1712, moving each coordinate in the first coordinate set according to the first central point to obtain a moved first coordinate set.
Specifically, according to the scheme, each coordinate in the point set C can be moved to the central point in the point setThe first coordinate set after the movement is C',
step S1713, moving each coordinate in the second coordinate set to the second central point to obtain a moved second coordinate set.
Specifically, the method can move each coordinate in the point set B to the central point in the point set, the first coordinate set after the movement is B',
step S1714, calculating a covariance matrix H between the first coordinate set and the second coordinate set. The calculation formula for H is as follows:
step S1715, performing Singular Value Decomposition (SVD) on the covariance matrix H to obtain the rotation matrix R, where R can be calculated by the following formula:
[U,S,V]＝SVD(H)
R＝VU T
and U, S and V are elements of a matrix after singular value decomposition, wherein U is a left singular value unitary matrix, S is a diagonal matrix, and V is a right singular value unitary matrix.
Optionally, calculating the translation matrix T includes:
step S1716, obtain the translation matrix T based on the rotation matrix R, the first central point, and the second central point.
Specifically, the translation matrix T can be calculated by the following formula:
T＝-R*μ C +μ B
example two
The application also provides a reinforcing steel bar binding hand-eye calibration system, the function of each component in the system is the same as that of the first embodiment, as shown in fig. 6, the system comprises:
a camera 60 for capturing an image at an image capturing point;
the mechanical arm 62 is used for binding the binding station;
a calibration plate 64;
a slide rail 66, configured to move the calibration plate from the image acquisition point to the binding station under the condition of receiving a calibration instruction sent by a controller, where the camera acquires an image of the calibration plate when the calibration plate is located at the image acquisition point, and the mechanical arm sequentially points to each corner point in the calibration plate when the calibration plate is located at the binding station;
the controller 68 is in communication relation with the camera and the mechanical arm, and is used for receiving the image of the calibration board and recording the pose of the mechanical arm when the mechanical arm points to each corner point;
the controller 68 is further configured to generate a transformation matrix from the coordinate system of the camera to the coordinate system of the robotic arm according to the image of the calibration plate and the pose of the robotic arm when pointing to each corner point.
The scheme is different from a calibration mode of 'hands outside eyes' in the prior art, the calibration mode of 'hands outside eyes' in the prior art limits that an image acquisition point must be close to a binding work site to meet the requirement that the moving range of a mechanical arm is always within the visual field range of a camera, so that the calibration of 'hands outside eyes' is carried out, namely, the existing calibration mode of 'hands outside eyes' only can meet the reinforcement cage image acquisition point and the reinforcement cage binding work site which are closer, and if the reinforcement cage image acquisition point and the binding work site are required to be far, the existing calibration mode of 'hands outside eyes' cannot realize calibration. According to the scheme, the calibration plate is moved, the first coordinate set and the second coordinate set are obtained successively, then the transformation matrix between the first coordinate set and the second coordinate set is obtained, various binding tasks can be met based on the calibration mode of the movable calibration plate, namely no matter what the distance between the reinforcement cage image acquisition point and the reinforcement cage binding station is, no matter whether the distance is close or far, the calibration mode of the scheme can meet the working condition without being limited by the distance, therefore, the scheme solves the problem that the calibration mode of eyes outside hands in the prior art is only suitable for the condition that the distance between the reinforcement cage image acquisition point and the reinforcement binding station point is close, the scene suitable for reinforcement is single, and the technical problem that the scene diversified binding tasks can not be met is solved.
Optionally, the controller is further configured to determine the position of the image acquisition point and the position of the binding work site from the received steel bar binding task, determine the position of the camera according to shooting parameters of the camera and the position of the image acquisition point, and determine the position of the mechanical arm according to the position of the binding work site and working parameters of the mechanical arm.
It will be understood that the specific features, operations and details described herein above with respect to the method of the present invention may be similarly applied to the apparatus and system of the present invention, or vice versa. In addition, each step of the method of the present invention described above may be performed by a respective component or unit of the device or system of the present invention.
It should be understood that the various modules/units of the apparatus of the present invention may be implemented in whole or in part by software, hardware, firmware, or a combination thereof. The modules/units may be embedded in the processor of the computer device in the form of hardware or firmware or independent from the processor, or may be stored in the memory of the computer device in the form of software for being called by the processor to execute the operations of the modules/units. Each of the modules/units may be implemented as a separate component or module, or two or more modules/units may be implemented as a single component or module.
In one embodiment, a computer device is provided that includes a memory and a processor, the memory having stored thereon computer instructions executable by the processor that, when executed by the processor, instruct the processor to perform the steps of the method of an embodiment of the invention. The computer device may broadly be a server, a terminal, or any other electronic device having the necessary computing and/or processing capabilities. In one embodiment, the computer device may include a processor, memory, a network interface, a communication interface, etc., connected by a system bus. The processor of the computer device may be used to provide the necessary computing, processing and/or control capabilities. The memory of the computer device may include non-volatile storage media and internal memory. An operating system, a computer program, and the like may be stored in or on the non-volatile storage medium. The internal memory may provide an environment for the operating system and the computer programs in the non-volatile storage medium to run. The network interface and the communication interface of the computer device may be used to connect and communicate with an external device through a network. Which when executed by a processor performs the steps of the method of the invention.
The invention may be implemented as a computer readable storage medium having stored thereon a computer program which, when executed by a processor, causes the steps of the method of an embodiment of the invention to be performed. In one embodiment, the computer program is distributed across a plurality of computer devices or processors coupled by a network such that the computer program is stored, accessed, and executed by one or more computer devices or processors in a distributed fashion. A single method step/operation, or two or more method steps/operations, may be performed by a single computer device or processor or by two or more computer devices or processors. One or more method steps/operations may be performed by one or more computer devices or processors, and one or more other methods
The steps/operations may be performed by one or more other computer devices or processors. One or more computer devices or processors may perform a single method step/operation, or perform two or more method steps/operations.
It will be appreciated by those of ordinary skill in the art that the method steps of the present invention may be directed to associated hardware, such as a computer device or processor, for performing the steps of the present invention by a computer program, which may be stored in a non-transitory computer readable storage medium, which when executed causes the steps of the present invention to be performed. Any reference herein to memory, storage, databases, or other media may include non-volatile and/or volatile memory, as appropriate. Examples of non-volatile memory include read-only memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), flash memory, magnetic tape, floppy disk, magneto-optical data storage device, hard disk, solid state disk, and the like. Examples of volatile memory include Random Access Memory (RAM), external cache memory, and the like.
The respective technical features described above may be arbitrarily combined. Although not all possible combinations of features are described, any combination of features should be considered to be covered by the present specification as long as there is no contradiction between such combinations.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and these modifications or substitutions do not depart from the spirit of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A reinforcing steel bar binding hand-eye calibration method is characterized by comprising the following steps:
placing the calibration plate at an image acquisition point of a camera, and controlling the camera to acquire an image of the calibration plate;
identifying each angular point in the calibration plate image so as to obtain a first coordinate set of the angular point of the calibration plate in a camera coordinate system;
moving the calibration plate to a binding work site of a steel bar, controlling the mechanical arm to enable a pointer of the mechanical arm to point to each angular point of the calibration plate in sequence, and recording the pose of the mechanical arm when the mechanical arm points to each angular point each time, so as to obtain a second coordinate set of the angular points of the calibration plate under a mechanical arm coordinate system;
and calculating a transformation matrix between the first coordinate set and the second coordinate set.
2. The method of claim 1, wherein prior to placing the calibration plate at the image capture point of the camera, the method further comprises:
acquiring a steel bar binding task, wherein the steel bar binding task comprises a set image acquisition point position and a binding work site position;
determining the position of the camera according to the position of the image acquisition point, the working distance and the precision of the camera;
and determining the position of the mechanical arm according to the position of the binding work site and the working interval of the mechanical arm.
3. The method of claim 1, wherein after calculating the transformation matrix between the first set of coordinates and the second set of coordinates, the method further comprises:
after the reinforcement cage reaches the image acquisition point, the camera acquires an image of the reinforcement cage, and a first node coordinate set of each node in the reinforcement cage under a camera coordinate system is obtained through image identification of the reinforcement cage;
controlling the reinforcement cage to reach the binding work site from the image acquisition point;
converting the first node coordinate set based on the transformation matrix to obtain a second node coordinate set of each node in the reinforcement cage under the mechanical arm coordinate;
and controlling the mechanical arm to bind each node of the reinforcement cage according to the second node coordinate set.
4. The method of claim 3, wherein the image acquisition point is a predetermined distance from the ligating work site such that a range of motion of the robotic arm is outside a field of view of the camera, wherein a next reinforcement cage is controlled to reach the image acquisition point and a camera is controlled to acquire a point cloud image of the next reinforcement cage while the robotic arm ligates each node of the reinforcement cage according to the second set of node coordinates.
5. The method of claim 1, wherein the calibration plate image is a three-dimensional point cloud image, and wherein identifying each corner point in the calibration plate image comprises:
performing two-dimensional projection on the calibration plate image to obtain a two-dimensional image of the calibration plate;
performing corner detection on the two-dimensional image by adopting a checkerboard corner detection method to obtain two-dimensional coordinates of each corner in the calibration plate;
and restoring each angular point from the two-dimensional coordinates to the calibration plate three-dimensional point cloud image to obtain the three-dimensional coordinates of each angular point in the calibration plate point cloud image.
6. The method of claim 1, wherein computing a transformation matrix between the first set of coordinates and the second set of coordinates comprises:
and performing singular value decomposition on the first coordinate set and the second coordinate set to obtain the transformation matrix, wherein the transformation matrix comprises a rotation matrix and a translation matrix.
7. The method of claim 6, wherein computing the rotation matrix R comprises:
calculating to obtain a first central point in the first coordinate set and a second central point in the second coordinate set;
moving each coordinate in the first coordinate set according to the first central point to obtain a moved first coordinate set;
moving each coordinate in the second coordinate set to the second central point to obtain a moved second coordinate set;
calculating a covariance matrix between the first coordinate set and the second coordinate set;
and carrying out singular value decomposition on the covariance matrix to obtain the rotation matrix.
8. The method of claim 7, wherein computing a translation matrix comprises:
obtaining the translation matrix based on the rotation matrix, the first center point, and the second center point.
9. The utility model provides a reinforcement hand eye calibration system which characterized in that includes:
a camera for capturing an image at an image capture point;
the mechanical arm is used for binding the binding station;
calibrating the plate;
the slide rail is used for moving the calibration plate from the image acquisition point to the binding work site under the condition of receiving a calibration instruction sent by a controller, wherein when the calibration plate is positioned at the image acquisition point, the camera acquires an image of the calibration plate, and when the calibration plate is positioned at the binding work site, the mechanical arm points to each angular point in the calibration plate in sequence;
the controller is in communication relation with the camera and the mechanical arm and is used for receiving the image of the calibration plate and recording the pose of the mechanical arm when the mechanical arm points to each angular point;
the controller is further used for generating a transformation matrix from the coordinate system of the camera to the coordinate system of the mechanical arm according to the image of the calibration plate and the pose of the mechanical arm when the mechanical arm points to each corner point.
10. The system of claim 9, wherein the controller is further configured to determine the location of the image acquisition site and the location of the ligating station from the received rebar tying task, wherein the controller is further configured to determine the location of the camera based on camera imaging parameters and the location of the image acquisition site, and wherein the controller is further configured to determine the location of the robotic arm based on the location of the ligating station and the robotic arm operating parameters.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211309702.9A CN115519547A (en) | 2022-10-25 | 2022-10-25 | Reinforcing steel bar binding hand-eye calibration method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211309702.9A CN115519547A (en) | 2022-10-25 | 2022-10-25 | Reinforcing steel bar binding hand-eye calibration method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115519547A true CN115519547A (en) | 2022-12-27 |
Family
ID=84704145
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211309702.9A Pending CN115519547A (en) | 2022-10-25 | 2022-10-25 | Reinforcing steel bar binding hand-eye calibration method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115519547A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117381798A (en) * | 2023-12-11 | 2024-01-12 | 法奥意威(苏州)机器人***有限公司 | Hand-eye calibration method and device |
-
2022
- 2022-10-25 CN CN202211309702.9A patent/CN115519547A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117381798A (en) * | 2023-12-11 | 2024-01-12 | 法奥意威(苏州)机器人***有限公司 | Hand-eye calibration method and device |
| CN117381798B (en) * | 2023-12-11 | 2024-04-12 | 法奥意威(苏州)机器人***有限公司 | Hand-eye calibration method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108986161B (en) | Three-dimensional space coordinate estimation method, device, terminal and storage medium | |
| US8861785B2 (en) | Information processing device, information processing method and program | |
| CN108765498B (en) | Monocular vision tracking, device and storage medium | |
| CN110163885B (en) | Target tracking method and device | |
| KR20170017786A (en) | System and method for tying together machine vision coordinate spaces in a guided assembly environment | |
| CN111025283B (en) | Method and device for linking radar and dome camera | |
| CN110111388B (en) | Three-dimensional object pose parameter estimation method and visual equipment | |
| CN110910459A (en) | Camera device calibration method and device and calibration equipment | |
| CN111590593B (en) | Calibration method, device and system of mechanical arm and storage medium | |
| KR102111655B1 (en) | Automatic calibration method and apparatus for robot vision system | |
| CN109949232B (en) | Image and RTK combined measurement method, system, electronic equipment and medium | |
| CN112330752A (en) | Multi-camera combined calibration method and device, terminal equipment and readable storage medium | |
| CN113329179B (en) | Shooting alignment method, device, equipment and storage medium | |
| CN115519547A (en) | Reinforcing steel bar binding hand-eye calibration method and system | |
| CN111429521A (en) | External parameter calibration method, device, medium and electronic equipment for camera and laser radar | |
| CN108627178B (en) | Robot eye calibration method and system | |
| CN114640833A (en) | Projection picture adjusting method and device, electronic equipment and storage medium | |
| CN115439633A (en) | Calibration method and device and electronic equipment | |
| JP4227037B2 (en) | Imaging system and calibration method | |
| JP2023503750A (en) | ROBOT POSITIONING METHOD AND DEVICE, DEVICE, STORAGE MEDIUM | |
| CN115393299A (en) | Monocular vision-based assembly line workpiece distance measuring method and device | |
| KR102220173B1 (en) | Automatic calibration method and apparatus for robot vision system | |
| CN113686240A (en) | Positioning method and device based on electric power tower, computer equipment and storage medium | |
| CN112847350A (en) | Hand-eye calibration method, system, computer equipment and storage medium | |
| CN110796596A (en) | Image splicing method, imaging device and panoramic imaging system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |