CN110883770A - Robot hybrid vision servo control method based on position and image - Google Patents
Robot hybrid vision servo control method based on position and image Download PDFInfo
- Publication number
- CN110883770A CN110883770A CN201910387248.0A CN201910387248A CN110883770A CN 110883770 A CN110883770 A CN 110883770A CN 201910387248 A CN201910387248 A CN 201910387248A CN 110883770 A CN110883770 A CN 110883770A
- Authority
- CN
- China
- Prior art keywords
- image
- camera
- robot
- vision servo
- control
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 30
- 239000011159 matrix material Substances 0.000 claims description 11
- 238000003384 imaging method Methods 0.000 claims description 3
- 239000000126 substance Substances 0.000 claims description 2
- 230000000007 visual effect Effects 0.000 description 19
- 230000007547 defect Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000007613 environmental effect Effects 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000005507 spraying Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000003466 welding Methods 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
- B25J9/1607—Calculation of inertia, jacobian matrixes and inverses
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
- B25J9/1697—Vision controlled systems
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Automation & Control Theory (AREA)
- Manipulator (AREA)
Abstract
The invention discloses a robot mixed vision servo method based on positions and images, and aims to improve the performance of robot vision servo. The position relation between the camera and the target object is established through image errors, the posture relation between the camera and the target object is established through the position errors, and the robot moves towards the expected posture through the hybrid vision servo controller. Compared with a single vision servo method, the method has good track precision and controllable track, and can be widely applied to robot control based on vision.
Description
Technical Field
The invention relates to the field of robot vision servo control, in particular to a robot hybrid vision servo control method based on positions and images.
Background
The application of the robot in the industry is continuously increased, and the robot is widely applied to the fields of spraying, welding, assembling, stacking and the like. With the upgrading of the industry, the requirements on the autonomy and the intelligence of the robot are higher and higher. At present, the traditional robot works according to a track specified in advance, and the environmental adaptability is poor, so that people are forced to improve the perception capability of the robot by adding a sensor, and the robot has certain environmental adaptability. Among other things, vision sensors provide an efficient solution. The robot vision servo utilizes the vision perception capability of the camera to establish the mapping relation between the space coordinate system and the image coordinate system of the robot, and then approaches the target object by controlling the speed of the robot.
Currently, common visual control methods are mainly classified into image-based visual servoing methods and position-based visual servoing methods. The error of the visual servo based on the image is defined on an image plane, and the movement speed of the robot is controlled by constructing a Jacobian matrix of the image speed and the camera speed, so that the visual servo control is realized. The method has the advantages of strong robustness and the defect that the track of the robot cannot be controlled. The position-based visual servo utilizes the image information of the feature points to construct the relative position relationship between the camera and the target object, and designs the visual servo according to the difference between the current value and the expected value of the position parameter so as to control the robot to complete the work task. The method has the advantages of controllable track and poor robustness to image errors.
After the prior related technology is subjected to document retrieval, Chinese patent numbers: CN107901041A, name: a robot vision servo control method based on image mixing moments is disclosed. This patent uses the mixing moment characteristic information values of the images as control errors. The method has certain robustness on image noise, but the selection of the image mixing moment is complex, the calculated amount is large, and the applicability has certain limitation. Chinese patent No.: CN106041927A, name: a hybrid vision servo system and method incorporating eye-to-hand and eys-in-hand structures. The patent improves the positioning accuracy of the target recognition by combining two using modes of the cameras eye-to-hand and eys-in-hand, but the method has certain requirements on the installation environment, and in addition, the cost is additionally increased due to the use of the two cameras.
Therefore, a new visual servoing method is proposed to solve the above-mentioned problems, aiming at the disadvantages of the image-based and location-based visual servoing methods.
Disclosure of Invention
The invention aims to solve the technical problem of overcoming the defects of the existing robot vision servo method. For this reason, a robot hybrid vision servo control method is proposed. The method can obviously improve the technical defects in the prior art and improve the performance of visual servo.
In order to achieve the purpose, the invention adopts the following technical scheme:
a method for hybrid visual servo control based on image and position, comprising:
and S1, establishing a camera coordinate system and a target object coordinate system, and generating coordinate values u of the feature points on the target object through a camera installed on the robot.
S2, a desired value Sd of the control signal is set, a current value S of the control signal is calculated from the coordinate value U of the feature point, and a control error Δ S is constructed as a control input.
And S3, constructing an image Jacobian matrix Ls by using the camera parameters and the imaging principle.
S4, calculating the speed Vc of the camera according to the Jacobian matrix Ls of the image of S3, and controlling the speed Vc to achieve the expected target pose.
Further, the feature points of the image are generally selected from four points of the target object plane to form a vector u ═ u1，u2，u3，u4]T。
Further, the desired value Sd of the control signal is [ Pd, θ d ═ Pd]TWherein: pd is the expected value of the image coordinates, and θ d is the expected value of the rotation angle of the object with respect to the camera.
Further, the robot hybrid vision servo control method obtains the speed of the camera through designed vision servo according to the Jacobian matrix Ls constructed in the step S3 and the delta S in the step S2
Wherein the content of the first and second substances,
is the inverse of the Jacobian matrix Ls and λ is the control gain.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flow chart of the method of the present invention;
fig. 2 is a schematic diagram of the robot visual servo system.
FIG. 3 is a schematic diagram of hybrid vision servo control of a robot;
FIG. 4 is a camera speed curve;
FIG. 5 is a camera spatial trajectory;
FIG. 6 is an image coordinate trace;
Detailed Description
The method of the present invention is further described in detail below with reference to specific examples, wherein the process is shown in fig. 1, and the specific implementation comprises the following steps:
(1) selecting image characteristic points of a target object for visual servo control;
the robot vision servo system is composed as shown in fig. 2, taking the example of guiding the robot to reach a specified target by vision servo, four points are selected as characteristic points on a target object coordinate system, and the four points are positioned on a circular plane with the diameter of 0.1 m;
(2) constructing a visual servo error;
the expected value of the image coordinates is set to Pd [ -0.050.050.2 [ ]]TThe desired value of the rotation angle of the object with respect to the camera is [30 ° -30 ° ], θ d]T. The camera collects the current coordinate value U of the feature point in real time, calculates the current value S of the control signal through a depth estimation algorithm, and constructs a control error delta S which is Sd-S as control input.
(3) Calculating an image Jacobian matrix Ls;
the change of the target object caused by the movement of the camera in the image can be obtained by using the imaging principle of the camera, and the Jacobian matrix of the image represents the relation between the moving speed of the camera and the coordinate change speed of the target in the image, namely Vi-LsVc,
where Vi is the rate of change of the image feature point, x, y are image coordinate values, and Z is the depth of the image.
(4) Calculating a camera speed Vc;
the camera speed of the visual servo can be obtained by controlling the error signal Delta S to show exponential speed decay
By controlling the speed VcThe expected target pose is achieved.
Fig. 3 is a schematic diagram of hybrid visual servo control of a robot, fig. 4 is a camera speed curve, it can be seen that a speed track of a camera is smooth, fig. 5 is a camera space track which is close to a straight line and has good controllability, and fig. 6 is an image coordinate track which smoothly reaches a target position.
The invention has the beneficial effects that: the invention combines the advantages of image-based visual servo and position-based visual servo, has the advantages of strong robustness and controllable track compared with the existing visual servo control system, and can be widely applied to the field of robots.
The above description is only for the specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (4)
1. A method for hybrid vision servo control of an image and position based robot, comprising:
and S1, establishing a camera coordinate system and a target object coordinate system, and generating coordinate values u of the feature points on the target object through a camera installed on the robot.
S2, a desired value Sd of the control signal is set, a current value S of the control signal is calculated from the coordinate value U of the feature point, and a control error Δ S is constructed as a control input.
And S3, constructing an image Jacobian matrix Ls by using the camera parameters and the imaging principle.
S4, calculating the speed Vc of the camera according to the Jacobian matrix Ls of the image of S3, and controlling the speed Vc to achieve the expected target pose.
2. The method according to claim 1, wherein in step S1, the feature points of the image are generally selected from four points of a target object plane to form a vector u ═ u1，u2，u3，u4]T。
3. The method according to claim 1, wherein in step S2, a desired value Sd of the control signal is [ Pd, θ d ]]TWherein: pd is the expected value of the image coordinates, and θ d is the expected value of the rotation angle of the object with respect to the camera.
4. The method as claimed in claim 1, wherein the hybrid vision servo control method of the robot obtains the speed of the camera through the designed vision servo according to the Jacobian matrix Ls constructed in the step S3 in combination with the Δ S in the step S2
Wherein the content of the first and second substances,
is the inverse of the Jacobian matrix Ls and λ is the control gain.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910387248.0A CN110883770A (en) | 2019-05-08 | 2019-05-08 | Robot hybrid vision servo control method based on position and image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910387248.0A CN110883770A (en) | 2019-05-08 | 2019-05-08 | Robot hybrid vision servo control method based on position and image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110883770A true CN110883770A (en) | 2020-03-17 |
Family
ID=69745790
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910387248.0A Pending CN110883770A (en) | 2019-05-08 | 2019-05-08 | Robot hybrid vision servo control method based on position and image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110883770A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113146623A (en) * | 2021-03-23 | 2021-07-23 | 华中科技大学 | Robot position control method and device based on cloud computing |
| CN116079697A (en) * | 2022-12-23 | 2023-05-09 | 北京纳通医用机器人科技有限公司 | Monocular vision servo method, device, equipment and medium based on image |
| CN116725730A (en) * | 2023-08-11 | 2023-09-12 | 北京市农林科学院智能装备技术研究中心 | Pig vaccine injection method, system and storage medium based on visual guidance |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2012130977A (en) * | 2010-12-20 | 2012-07-12 | Toshiba Corp | Robot control apparatus |
| WO2013183190A1 (en) * | 2012-06-04 | 2013-12-12 | 株式会社安川電機 | Robot system, robot control device, and robot system control method |
| CN106041927A (en) * | 2016-06-22 | 2016-10-26 | 西安交通大学 | Hybrid vision servo system and method combining eye-to-hand and eye-in-hand structures |
| CN107901041A (en) * | 2017-12-15 | 2018-04-13 | 中南大学 | A kind of robot vision servo control method based on image blend square |
| CN108621167A (en) * | 2018-07-23 | 2018-10-09 | 中南大学 | A kind of visual servo decoupling control method based on profile side and the interior feature that takes all of |
| CN109015633A (en) * | 2018-07-12 | 2018-12-18 | 深圳市寒武纪智能科技有限公司 | A kind of visual servo tracking, apparatus and system |
-
2019
- 2019-05-08 CN CN201910387248.0A patent/CN110883770A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2012130977A (en) * | 2010-12-20 | 2012-07-12 | Toshiba Corp | Robot control apparatus |
| WO2013183190A1 (en) * | 2012-06-04 | 2013-12-12 | 株式会社安川電機 | Robot system, robot control device, and robot system control method |
| CN106041927A (en) * | 2016-06-22 | 2016-10-26 | 西安交通大学 | Hybrid vision servo system and method combining eye-to-hand and eye-in-hand structures |
| CN107901041A (en) * | 2017-12-15 | 2018-04-13 | 中南大学 | A kind of robot vision servo control method based on image blend square |
| CN109015633A (en) * | 2018-07-12 | 2018-12-18 | 深圳市寒武纪智能科技有限公司 | A kind of visual servo tracking, apparatus and system |
| CN108621167A (en) * | 2018-07-23 | 2018-10-09 | 中南大学 | A kind of visual servo decoupling control method based on profile side and the interior feature that takes all of |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113146623A (en) * | 2021-03-23 | 2021-07-23 | 华中科技大学 | Robot position control method and device based on cloud computing |
| CN116079697A (en) * | 2022-12-23 | 2023-05-09 | 北京纳通医用机器人科技有限公司 | Monocular vision servo method, device, equipment and medium based on image |
| CN116079697B (en) * | 2022-12-23 | 2024-05-28 | 北京纳通医用机器人科技有限公司 | Monocular vision servo method, device, equipment and medium based on image |
| CN116725730A (en) * | 2023-08-11 | 2023-09-12 | 北京市农林科学院智能装备技术研究中心 | Pig vaccine injection method, system and storage medium based on visual guidance |
| CN116725730B (en) * | 2023-08-11 | 2023-12-05 | 北京市农林科学院智能装备技术研究中心 | Pig vaccine injection method, system and storage medium based on visual guidance |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111360827B (en) | Visual servo switching control method and system | |
| CN109159151B (en) | Mechanical arm space trajectory tracking dynamic compensation method and system | |
| CN110883770A (en) | Robot hybrid vision servo control method based on position and image | |
| CN107627303B (en) | PD-SMC control method of visual servo system based on eye-on-hand structure | |
| CN111203880B (en) | Image visual servo control system and method based on data driving | |
| CN112894812A (en) | Visual servo trajectory tracking control method and system for mechanical arm | |
| CN109358507A (en) | A kind of visual servo adaptive tracking control method of time-varying performance boundary constraint | |
| CN111618861A (en) | Double-follow-up intelligent arm control method based on four-axis structure | |
| CN110928311B (en) | Indoor mobile robot navigation method based on linear features under panoramic camera | |
| CN109582026B (en) | Autonomous underwater vehicle path tracking control method based on self-tuning sight line and drift angle compensation | |
| CN108927807B (en) | Robot vision control method based on point characteristics | |
| CN109693235B (en) | Human eye vision-imitating tracking device and control method thereof | |
| CN114442054A (en) | Sensor and chassis combined calibration system and method for mobile robot | |
| CN109773795A (en) | A kind of five axis door shape manipulator control system of intelligence and method | |
| CN113051767A (en) | AGV sliding mode control method based on visual servo | |
| CN111251303A (en) | Robot motion control method for periodic attitude adjustment | |
| López-Nicolás et al. | Vision-based exponential stabilization of mobile robots | |
| Chen et al. | Application of visual servoing to an X-ray based welding inspection robot | |
| Wang et al. | Visual regulation of a nonholonomic wheeled mobile robot with two points using Lyapunov functions | |
| CN111738047A (en) | Self-position estimation method | |
| CN110614636A (en) | Measurement time lag compensation method for visual servo robot | |
| Yin et al. | Monitoring-based visual servoing of wheeled mobile robots | |
| CN112415516B (en) | Method and device for sensing obstacle area in front of vehicle | |
| Wang et al. | A new way to detect the position and orientation of the wheeled mobile robot on the image plane | |
| Hwang et al. | Robust 2D map building with motion-free ICP algorithm for mobile robot navigation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| DD01 | Delivery of document by public notice |
Addressee: Ding Yadong Document name: Notification of Publication of the Application for Invention |
|
| DD01 | Delivery of document by public notice | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |