CN108942929B - Mechanical arm positioning and grabbing method and device based on binocular stereo vision - Google Patents
Mechanical arm positioning and grabbing method and device based on binocular stereo vision Download PDFInfo
- Publication number
- CN108942929B CN108942929B CN201810752380.2A CN201810752380A CN108942929B CN 108942929 B CN108942929 B CN 108942929B CN 201810752380 A CN201810752380 A CN 201810752380A CN 108942929 B CN108942929 B CN 108942929B
- Authority
- CN
- China
- Prior art keywords
- mechanical arm
- coordinate system
- instrument
- target
- target power
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
- B25J9/1697—Vision controlled systems
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J11/00—Manipulators not otherwise provided for
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T7/85—Stereo camera calibration
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Manipulator (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The application relates to a method and a device for positioning and grabbing mechanical arms based on binocular stereo vision. The method comprises the steps of determining a conversion relation between a world coordinate system and a mechanical arm coordinate system; classifying and identifying the target power instrument through a Scale Invariant Feature Transform (SIFT) feature matching rule; calculating a first target position of the target power instrument in a world coordinate system, and determining a second target position of the first target position in a mechanical arm coordinate system according to a conversion relation between the world coordinate system and the mechanical arm coordinate system; and grabbing the target power meter according to the class characteristics of the target power meter and the second target position. The invention can improve the precision of instrument grabbing; the instrument to be detected is placed on the rotary platform, so that the detection efficiency of the whole detection system can be improved.
Description
Technical Field
The application relates to the field of automatic control, in particular to a method and a device for positioning and grabbing mechanical arms based on binocular stereoscopic vision.
Background
At present, pointer instruments are still commonly used in the power industry, and in order to ensure the safe operation of a power system, the instruments need to be periodically verified by scientific research institutions such as power research institutes and metering institutes to ensure the accuracy of the instruments, so that the demand for automatic instrument verification systems is increasing day by day. However, the existing automatic instrument verification systems need manual upper and lower tables, complete automatic verification is not achieved, and the working efficiency of the instrument verification system is severely limited.
Disclosure of Invention
Therefore, in order to solve the technical problems, a method and a device for positioning and grabbing the mechanical arm based on binocular stereoscopic vision are needed, wherein the method and the device can be used for fully automatically calibrating the instrument.
The invention provides a mechanical arm positioning and grabbing method based on binocular stereo vision, which comprises the following steps:
determining a conversion relation between a world coordinate system and a mechanical arm coordinate system;
classifying and identifying the target power instrument through a Scale-invariant feature transform (SIFT) feature matching rule;
calculating a first target position of the target power instrument in a world coordinate system, and determining a second target position of the first target position in a mechanical arm coordinate system according to a conversion relation between the world coordinate system and the mechanical arm coordinate system;
and grabbing the target power meter according to the class characteristics of the target power meter and the second target position.
Optionally, the determining a conversion relationship between the world coordinate system and the robot arm coordinate system includes:
shooting a first reference object through a binocular camera, and determining a conversion relation between a world coordinate system and a camera coordinate system according to the position of the first reference object in the world coordinate system and the camera coordinate system;
determining a second reference object through a binocular camera and the mechanical arm, and determining a conversion relation between the camera coordinate system and the mechanical arm coordinate system according to the positions of the second reference object in the camera coordinate system and the mechanical arm coordinate system;
and determining the conversion relation between the world coordinate system and the mechanical arm coordinate system according to the conversion relation between the world coordinate system and the camera coordinate system and the conversion relation between the camera coordinate system and the mechanical arm coordinate system.
Optionally, before classifying and identifying the target power meter according to the SIFT feature matching rule, the method further includes:
the model of each instrument in the instrument inspection waiting area is identified through the SIFT feature matching rule, so that the identified instrument is transported to an instrument verification area corresponding to the model by a transport belt.
Optionally, the method further includes:
the method comprises the steps that world coordinate information of each instrument in an instrument waiting area is obtained through a binocular camera, so that the mechanical arm grabs each instrument according to the world coordinate information of each instrument in the instrument waiting area and places the instruments on a transmission belt of a verification assembly line.
Optionally, a clamping jaw is further disposed on the mechanical arm, and the grabbing the target power meter according to the class characteristics of the target power meter and the second target position includes:
moving the clamping jaw to a preset position of the target electric power meter according to the class characteristics of the target electric power meter and the second target position;
and controlling the clamping jaws to clamp the target electric power meter.
The invention also provides a mechanical arm positioning and grabbing device based on binocular stereoscopic vision, which comprises a body, and a mechanical arm and a binocular camera which are arranged on the body, wherein the mechanical arm is provided with a clamping jaw, and the body is also provided with a controller used for controlling the mechanical arm, the clamping jaw and the binocular camera;
the controller is used for determining a conversion relation between a world coordinate system and a mechanical arm coordinate system;
the controller is also used for classifying and identifying the target power instrument through a Scale Invariant Feature Transform (SIFT) feature matching rule;
the controller is further used for calculating a first target position of the target power instrument in a world coordinate system and determining a second target position of the first target position in a mechanical arm coordinate system according to a conversion relation between the world coordinate system and the mechanical arm coordinate system;
the controller is further used for controlling the clamping jaws on the mechanical arm to grab the target electric power meter according to the class characteristics of the target electric power meter and the second target position.
Optionally, the controller is specifically configured to:
controlling the binocular camera to shoot a first reference object, and determining a conversion relation between a world coordinate system and a camera coordinate system according to the positions of the first reference object in the world coordinate system and the camera coordinate system;
controlling the binocular camera and the mechanical arm to determine a second reference object, and determining a conversion relation between the camera coordinate system and the mechanical arm coordinate system according to the positions of the second reference object in the camera coordinate system and the mechanical arm coordinate system;
and determining the conversion relation between the world coordinate system and the mechanical arm coordinate system according to the conversion relation between the world coordinate system and the camera coordinate system and the conversion relation between the camera coordinate system and the mechanical arm coordinate system.
Optionally, the controller is further configured to:
the model of each instrument in the instrument inspection waiting area is identified through the SIFT feature matching rule, so that the identified instrument is transported to an instrument verification area corresponding to the model by a transport belt.
Optionally, the controller is further configured to:
and controlling the binocular camera to obtain the world coordinate information of each instrument in the instrument waiting area, so that the mechanical arm grabs each instrument according to the world coordinate information of each instrument in the instrument waiting area and places the instrument on a transmission belt of a verification assembly line.
Optionally, the controller is specifically configured to:
controlling the mechanical arm to move the clamping jaw to a preset position of the target electric power meter according to the class characteristics of the target electric power meter and the second target position;
and controlling the clamping jaws to clamp the target electric power meter.
In the method for positioning and grabbing the mechanical arm based on the binocular stereo vision, a binocular stereo vision system with a hand-eye structure is adopted in a positioning link in a production line process on the instrument to be detected, and the instrument to be detected is accurately positioned through a conversion relation between world coordinates and mechanical arm coordinates; the model of the instrument to be detected can be identified by using the image acquired by the binocular stereoscopic vision system and using a digital image processing technology, so that the subsequent detection platform can conveniently detect the instrument, the size of the instrument can be obtained according to the identified model of the instrument, and the instrument capturing precision can be improved; the instrument to be detected is placed on the rotary platform, so that the detection efficiency of the whole detection system can be improved.
Drawings
FIG. 1 is a schematic diagram of a mechanical arm positioning and gripping device based on binocular stereo vision;
FIG. 2 is a schematic diagram of the mechanical arm positioning and grabbing method based on binocular stereo vision;
FIG. 3 is a diagram of one embodiment of a method for positioning and grabbing a mechanical arm based on binocular stereo vision;
FIG. 4 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The method provided by the application can be applied to the application environment as shown in figure 1, and comprises a controller, a mechanical arm, a binocular stereoscopic vision system, an electric clamping jaw and a detected instrument waiting and detecting rotary platform. The binocular stereoscopic vision system and the electric clamping jaw are fixed on a flange plate of the mechanical arm, and the instrument to be detected is placed on a detection rotary platform; the controller is connected with the mechanical arm and the binocular stereoscopic vision system through Ethernet cables; the controller is connected with the electric clamping jaw and the to-be-detected instrument rotary platform through the PCI interface, so that the controller can control all the lower devices. The controller is connected with the binocular stereoscopic vision system and the mechanical arm through Ethernet, images collected by the camera are transmitted to the controller through the Ethernet cable, and the controller transmits calculated data to the mechanical arm through the Ethernet cable; the controller is connected with the metering standard source through a serial port to realize the control of the standard source. The controller is connected with the electric clamping jaw and the rotary platform of the instrument area to be detected through the PCI interface, and the electric clamping jaw and the rotary platform of the instrument area to be detected are controlled.
The controller can be a computer, a single chip microcomputer and the like, the controller controls the binocular camera 106 to classify and identify the electric power instrument to be detected and acquire position information of the electric power instrument, and the controller 102 further controls the mechanical arm and the clamping jaw to grab the electric power instrument. A binocular stereoscopic vision system in the verification system adopts an MV-EM200M type black-and-white camera for maintaining images, an IRB 2600-20/1.65 type mechanical arm of ABB is selected as the mechanical arm, an LEHF32K2-64-6P3D type electric clamping jaw of SMC is selected as the electric clamping jaw, and a loose MSME082G1T type servo motor is selected as a rotary platform motor.
The binocular stereoscopic vision system adopts an MV-EM200M type black-and-white camera for maintaining images, the camera has a high-speed image processing function and 200 ten thousand pixels, and the quality and the efficiency of image acquisition are effectively guaranteed; the mechanical arm is an IRB 2600-20/1.65 type mechanical arm of ABB, and the position repetition precision can reach 0.04 mm; an LEHF32K2-64-6P3D type electric clamping jaw of an SMC is selected as the electric clamping jaw, the stroke is 64mm, and the maximum clamping force is 120N; a loose MSME082G1T type servo motor is selected as a rotary platform motor, the rated output of the servo motor is 750W, and the servo motor can load the rotary platform and the instrument to be detected.
In one embodiment, as shown in fig. 2, there is provided a method for positioning and grabbing a mechanical arm based on binocular stereo vision, comprising the following steps:
202. and determining a conversion relation between the world coordinate system and the mechanical arm coordinate system.
The world coordinate system is a coordinate system in a real three-dimensional world, and the mechanical arm coordinate system is a coordinate system referred to when the mechanical arm moves.
Alternatively, the conversion relationship between the world coordinate system and the robot arm coordinate system may be specifically converted in the following manner:
shooting a first reference object through a binocular camera, and determining a conversion relation between a world coordinate system and a camera coordinate system according to the position of the first reference object in the world coordinate system and the camera coordinate system;
determining a second reference object through a binocular camera and the mechanical arm, and determining a conversion relation between the camera coordinate system and the mechanical arm coordinate system according to the positions of the second reference object in the camera coordinate system and the mechanical arm coordinate system;
and determining the conversion relation between the world coordinate system and the mechanical arm coordinate system according to the conversion relation between the world coordinate system and the camera coordinate system and the conversion relation between the camera coordinate system and the mechanical arm coordinate system.
Therefore, the conversion process actually adopts the process of converting the coordinates twice to establish the conversion relation between the world coordinates and the coordinates of the mechanical arm, so that the controller can more accurately control the mechanical arm to move and control the clamping jaw to grab the power instrument.
204. And classifying and identifying the target power instrument through a Scale Invariant Feature Transform (SIFT) feature matching rule.
Optionally, before classifying and identifying the target power meter according to the SIFT feature matching rule, the method further includes:
the model of each instrument in the instrument inspection waiting area is identified through the SIFT feature matching rule, so that the identified instrument is transported to an instrument verification area corresponding to the model by a transport belt.
Optionally, the method further includes:
the method comprises the steps that world coordinate information of each instrument in an instrument waiting area is obtained through a binocular camera, so that the mechanical arm grabs each instrument according to the world coordinate information of each instrument in the instrument waiting area and places the instruments on a transmission belt of a verification assembly line.
Optionally, a clamping jaw is further disposed on the mechanical arm, and the grabbing the target power meter according to the class characteristics of the target power meter and the second target position includes:
moving the clamping jaw to a preset position of the target electric power meter according to the class characteristics of the target electric power meter and the second target position;
and controlling the clamping jaws to clamp the target electric power meter.
206. Calculating a first target position of the target power meter in a world coordinate system, and determining a second target position of the first target position in a mechanical arm coordinate system according to a conversion relation between the world coordinate system and the mechanical arm coordinate system.
208. And grabbing the target power meter according to the class characteristics of the target power meter and the second target position.
In the embodiment, in the positioning link in the assembly line process of the instrument to be detected, a binocular stereoscopic vision system with a hand-eye structure is adopted, and the instrument to be detected is accurately positioned through the conversion relation between world coordinates and mechanical arm coordinates; the model of the instrument to be detected can be identified by using the image acquired by the binocular stereoscopic vision system and using a digital image processing technology, so that the subsequent detection platform can conveniently detect the instrument, the size of the instrument can be obtained according to the identified model of the instrument, and the instrument capturing precision can be improved; the instrument to be detected is placed on the rotary platform, so that the detection efficiency of the whole detection system can be improved.
The above method is explained below as an example of an actual operation.
Referring to fig. 3, fig. 3 is a diagram of an embodiment of a method for positioning and grabbing a mechanical arm based on binocular stereo vision, as shown in fig. 3, including:
a hardware platform is built according to the attached figure 1, a binocular stereoscopic vision system and an electric clamping jaw are fixed on a flange plate of a mechanical arm, and a controller is connected with a camera through a network cable; the controller is respectively connected with the electric clamping jaw and the rotating platform through a PCI interface.
And placing the instrument to be detected on a platform rotating with the instrument area to be detected, selecting an initial detection range, and performing zero calibration work on the instrument to be detected.
And opening the binocular stereoscopic vision system to enable the camera to collect clear images.
And controlling the mechanical arm to move to the starting point, and opening the rotating platform to start moving.
The controller starts to detect a moving target, namely the instrument to be detected, and when the target moves to the center of the visual field of the left camera, the rotating platform is closed to stop moving.
And controlling a binocular camera to acquire instrument images by the computer to perform stereo matching, and calculating the three-dimensional information of the instrument.
And the controller controls the mechanical arm to move right above the instrument and close to the instrument, and the model of the instrument is identified by characteristic matching.
And (6) obtaining the size of the instrument according to the identified instrument model, and controlling the mechanical arm and the electric clamping jaw to grab the instrument by combining the three-dimensional information of the instrument obtained in the step (6).
And the controller controls the mechanical arm to move above a transmission belt of the power instrument verification production line and place the instrument, and then controls the mechanical arm to return to the initial point.
Before a stop command and an emergency stop command of the main control system are not received, the step of controlling the mechanical arm to move to the starting point is repeated until the controller controls the mechanical arm to move to the position above a transmission belt of the verification assembly line of the electric instrument and places the instrument, and then the step of controlling the mechanical arm to return to the starting point is repeated.
The invention automatically upgrades the existing pointer instrument calibration system, uses a binocular stereo vision system to position the instrument, uses a digital image processing technology to identify the model of the instrument, and uses a mechanical arm and an electric clamping jaw to grab and place the instrument to be detected. Therefore, compared with the prior art, the invention has the advantage of higher automation level.
The above description is made on the mechanical arm positioning and gripping method based on binocular stereo vision, and the following description is made on the mechanical arm positioning and gripping device based on binocular stereo vision.
The device comprises a body, and a mechanical arm and a binocular camera which are arranged on the body, wherein a clamping jaw is arranged on the mechanical arm, and a controller for controlling the mechanical arm, the clamping jaw and the binocular camera is also arranged on the body;
the controller is used for determining a conversion relation between a world coordinate system and a mechanical arm coordinate system;
the controller is also used for classifying and identifying the target power instrument through a Scale Invariant Feature Transform (SIFT) feature matching rule;
the controller is further used for calculating a first target position of the target power instrument in a world coordinate system and determining a second target position of the first target position in a mechanical arm coordinate system according to a conversion relation between the world coordinate system and the mechanical arm coordinate system;
the controller is further used for controlling the clamping jaws on the mechanical arm to grab the target electric power meter according to the class characteristics of the target electric power meter and the second target position.
Optionally, the controller is specifically configured to:
controlling the binocular camera to shoot a first reference object, and determining a conversion relation between a world coordinate system and a camera coordinate system according to the positions of the first reference object in the world coordinate system and the camera coordinate system;
controlling the binocular camera and the mechanical arm to determine a second reference object, and determining a conversion relation between the camera coordinate system and the mechanical arm coordinate system according to the positions of the second reference object in the camera coordinate system and the mechanical arm coordinate system;
and determining the conversion relation between the world coordinate system and the mechanical arm coordinate system according to the conversion relation between the world coordinate system and the camera coordinate system and the conversion relation between the camera coordinate system and the mechanical arm coordinate system.
Optionally, the controller is further configured to:
the model of each instrument in the instrument inspection waiting area is identified through the SIFT feature matching rule, so that the identified instrument is transported to an instrument verification area corresponding to the model by a transport belt.
Optionally, the controller is further configured to:
and controlling the binocular camera to obtain the world coordinate information of each instrument in the instrument waiting area, so that the mechanical arm grabs each instrument according to the world coordinate information of each instrument in the instrument waiting area and places the instrument on a transmission belt of a verification assembly line.
Optionally, the controller is specifically configured to:
controlling the mechanical arm to move the clamping jaw to a preset position of the target electric power meter according to the class characteristics of the target electric power meter and the second target position;
and controlling the clamping jaws to clamp the target electric power meter.
In the embodiment, in the positioning link in the assembly line process of the instrument to be detected, a binocular stereoscopic vision system with a hand-eye structure is adopted, and the instrument to be detected is accurately positioned through the conversion relation between world coordinates and mechanical arm coordinates; the model of the instrument to be detected can be identified by using the image acquired by the binocular stereoscopic vision system and using a digital image processing technology, so that the subsequent detection platform can conveniently detect the instrument, the size of the instrument can be obtained according to the identified model of the instrument, and the instrument capturing precision can be improved; the instrument to be detected is placed on the rotary platform, so that the detection efficiency of the whole detection system can be improved.
In one embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 4. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for user information, service information, etc. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to realize the mechanical arm positioning and grabbing method based on binocular stereo vision.
Those skilled in the art will appreciate that the architecture shown in fig. 4 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory storing a computer program and a processor implementing the steps of the above-described method embodiments when the processor executes the computer program.
In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the respective method embodiment as described above.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A mechanical arm positioning and grabbing method based on binocular stereo vision comprises the following steps:
shooting a first reference object through a binocular camera, and determining a conversion relation between a world coordinate system and a camera coordinate system according to the position of the first reference object in the world coordinate system and the camera coordinate system; determining a second reference object through a binocular camera and the mechanical arm, and determining a conversion relation between the camera coordinate system and the mechanical arm coordinate system according to the positions of the second reference object in the camera coordinate system and the mechanical arm coordinate system; determining a conversion relation between the world coordinate system and the mechanical arm coordinate system according to the conversion relation between the world coordinate system and the camera coordinate system and the conversion relation between the camera coordinate system and the mechanical arm coordinate system;
the binocular camera is used for obtaining world coordinate information of each instrument in an instrument waiting area, so that the mechanical arm can grab each instrument according to the world coordinate information of each instrument in the instrument waiting area and place the instrument on a transmission belt of a verification assembly line;
identifying and classifying the models of the instruments in the instrument inspection waiting area through Scale Invariant Feature Transform (SIFT) feature matching rules to obtain class features of the target power instrument so that the identified instruments are transported to instrument inspection areas corresponding to the models by a transport belt;
calculating a first target position of the target power meter in the world coordinate system, and determining a second target position of the first target position in the mechanical arm coordinate system according to a conversion relation between the world coordinate system and the mechanical arm coordinate system;
and grabbing the target power meter according to the class characteristics of the target power meter and the second target position.
2. The binocular stereo vision based mechanical arm positioning and grabbing method according to claim 1, wherein a clamping jaw is further arranged on the mechanical arm, and grabbing the target power meter according to the class characteristics of the target power meter and the second target position comprises:
moving the clamping jaw to a preset position of the target electric power meter according to the class characteristics of the target electric power meter and the second target position;
and controlling the clamping jaws to clamp the target electric power meter.
3. The binocular stereo vision based mechanical arm positioning and grabbing method according to claim 2, wherein the controlling of the clamping jaw clamping to grab the target power meter comprises:
controlling the clamping jaw to clamp the target electric power instrument according to the size and the three-dimensional information of the target electric power instrument; the size of the target power meter is determined by the category characteristic; and the three-dimensional information of the target power meter is determined after the image of the target power meter is subjected to stereo matching.
4. The method of claim 3, wherein before controlling the clamping of the clamping jaws to grab the target power meter according to the size and three-dimensional information of the target power meter, further comprising:
opening a binocular stereoscopic vision system to enable the binocular camera to acquire clear images;
controlling the mechanical arm to move to a starting point, and opening the rotating platform to start moving;
starting moving target detection, and closing the rotary platform to stop moving when the target moves to the center of the visual field of the left camera;
and controlling the binocular camera to acquire the image of the target power instrument for stereo matching, and calculating the three-dimensional information of the target power instrument.
5. The method according to claim 4, wherein after the controlling the binocular camera to acquire the image of the target power meter for stereo matching and calculating the three-dimensional information of the target power meter, the method further comprises:
controlling the mechanical arm to move right above the target electric power instrument, and performing feature matching to identify the model of the target electric power instrument;
determining the size of the target power instrument according to the model of the target power instrument, and controlling the mechanical arm and the clamping jaw to grab the target power instrument according to the size of the target power instrument and the three-dimensional information of the target power instrument;
and controlling the mechanical arm to move above the conveying belt and place the target power meter, and controlling the mechanical arm to return to the starting point.
6. The mechanical arm positioning and grabbing device based on binocular stereoscopic vision is characterized by comprising a body, a mechanical arm and a binocular camera, wherein the mechanical arm and the binocular camera are arranged on the body;
the controller is used for controlling the binocular camera to shoot a first reference object, and determining a conversion relation between a world coordinate system and a camera coordinate system according to the positions of the first reference object in the world coordinate system and the camera coordinate system; controlling the binocular camera and the mechanical arm to determine a second reference object, and determining a conversion relation between the camera coordinate system and the mechanical arm coordinate system according to the positions of the second reference object in the camera coordinate system and the mechanical arm coordinate system; determining a conversion relation between the world coordinate system and the mechanical arm coordinate system according to the conversion relation between the world coordinate system and the camera coordinate system and the conversion relation between the camera coordinate system and the mechanical arm coordinate system;
the controller is also used for controlling the binocular camera to obtain world coordinate information of each instrument in an instrument waiting area, so that the mechanical arm grabs each instrument according to the world coordinate information of each instrument in the instrument waiting area and places the instrument on a transmission belt of a verification assembly line;
identifying and classifying the models of the instruments in the instrument inspection waiting area through Scale Invariant Feature Transform (SIFT) feature matching rules to obtain class features of the target power instrument so that the identified instruments are transported to instrument inspection areas corresponding to the models by a transport belt;
the controller is further used for calculating a first target position of the target power instrument in a world coordinate system and determining a second target position of the first target position in a mechanical arm coordinate system according to a conversion relation between the world coordinate system and the mechanical arm coordinate system;
the controller is further used for controlling the clamping jaws on the mechanical arm to grab the target electric power meter according to the class characteristics of the target electric power meter and the second target position.
7. The binocular stereo vision based mechanical arm positioning and grabbing device of claim 6, wherein the controller is specifically configured to:
controlling the mechanical arm to move the clamping jaw to a preset position of the target electric power meter according to the class characteristics of the target electric power meter and the second target position;
and controlling the clamping jaws to clamp the target electric power meter.
8. The binocular stereo vision based mechanical arm positioning and grabbing device of claim 7, wherein the controller is specifically configured to:
controlling the clamping jaw to clamp the target electric power instrument according to the size and the three-dimensional information of the target electric power instrument; the size of the target power meter is determined by the category characteristic; and the three-dimensional information of the target power meter is determined after the image of the target power meter is subjected to stereo matching.
9. The binocular stereo-vision based mechanical arm positioning and grabbing device of claim 8, wherein the controller is further configured to: opening a binocular stereoscopic vision system to enable the binocular camera to acquire clear images; controlling the mechanical arm to move to a starting point, and opening the rotating platform to start moving; starting moving target detection, and closing the rotary platform to stop moving when the target moves to the center of the visual field of the left camera; and controlling the binocular camera to acquire the image of the target power instrument for stereo matching, and calculating the three-dimensional information of the target power instrument.
10. The binocular stereo vision based mechanical arm positioning and grabbing device of claim 9, wherein the controller is further configured to control the mechanical arm to move right above the target power meter, perform feature matching, and identify the model of the target power meter; determining the size of the target power instrument according to the model of the target power instrument, and controlling the mechanical arm and the clamping jaw to grab the target power instrument according to the size of the target power instrument and the three-dimensional information of the target power instrument; and controlling the mechanical arm to move above the conveying belt and place the target power meter, and controlling the mechanical arm to return to the starting point.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810752380.2A CN108942929B (en) | 2018-07-10 | 2018-07-10 | Mechanical arm positioning and grabbing method and device based on binocular stereo vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810752380.2A CN108942929B (en) | 2018-07-10 | 2018-07-10 | Mechanical arm positioning and grabbing method and device based on binocular stereo vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108942929A CN108942929A (en) | 2018-12-07 |
| CN108942929B true CN108942929B (en) | 2020-11-27 |
Family
ID=64482787
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810752380.2A Active CN108942929B (en) | 2018-07-10 | 2018-07-10 | Mechanical arm positioning and grabbing method and device based on binocular stereo vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108942929B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109465809B (en) * | 2018-12-17 | 2024-07-19 | 中北大学 | Intelligent garbage classification robot based on binocular stereoscopic vision positioning identification |
| CN109483554B (en) * | 2019-01-22 | 2020-05-12 | 清华大学 | Robot dynamic grabbing method and system based on global and local visual semantics |
| CN111037558A (en) * | 2019-12-23 | 2020-04-21 | 北京航天计量测试技术研究所 | Automatic identification and grabbing method and system |
| CN111784782A (en) * | 2020-06-29 | 2020-10-16 | 伯肯森自动化技术（上海）有限公司 | POS machine personification test equipment |
| CN111751136A (en) * | 2020-06-29 | 2020-10-09 | 伯肯森自动化技术（上海）有限公司 | POS machine test system based on binocular vision subassembly |
| CN112285625A (en) * | 2020-09-21 | 2021-01-29 | 广东电网有限责任公司广州供电局 | Universal automatic electric power instrument verification system and method |
| CN112327017B (en) * | 2020-11-06 | 2022-02-11 | 广东电网有限责任公司电力科学研究院 | Switching device and system of distribution automation equipment module |
| CN112648934B (en) * | 2020-12-07 | 2022-07-01 | 新拓三维技术(深圳)有限公司 | Automatic elbow geometric form detection method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH1063317A (en) * | 1996-08-13 | 1998-03-06 | Fanuc Ltd | Method for combining coordinate system in robot and visual sensor system |
| CN103991078A (en) * | 2013-02-14 | 2014-08-20 | 佳能株式会社 | Robot system and method for controlling the same |
| CN105082161A (en) * | 2015-09-09 | 2015-11-25 | 新疆医科大学第一附属医院 | Robot vision servo control device of binocular three-dimensional video camera and application method of robot vision servo control device |
| CN105217324A (en) * | 2015-10-20 | 2016-01-06 | 上海影火智能科技有限公司 | A kind of novel de-stacking method and system |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4387476B2 (en) * | 1998-07-13 | 2009-12-16 | 株式会社明電舎 | Bin picking position data calibration device |
| KR101217231B1 (en) * | 2011-02-14 | 2012-12-31 | 금오공과대학교 산학협력단 | Method and system of object recognition |
| CN102902271A (en) * | 2012-10-23 | 2013-01-30 | 上海大学 | Binocular vision-based robot target identifying and gripping system and method |
| CN103271784B (en) * | 2013-06-06 | 2015-06-10 | 山东科技大学 | Man-machine interactive manipulator control system and method based on binocular vision |
| CN105809182B (en) * | 2014-12-31 | 2019-12-13 | 中国科学院深圳先进技术研究院 | Image classification method and device |
| CN106737665B (en) * | 2016-11-30 | 2019-07-19 | 天津大学 | Based on binocular vision and the matched mechanical arm control system of SIFT feature and implementation method |
| CN107256568B (en) * | 2017-05-08 | 2020-10-27 | 西安交通大学 | High-precision mechanical arm hand-eye camera calibration method and calibration system |
| CN107767423B (en) * | 2017-10-10 | 2019-12-06 | 大连理工大学 | mechanical arm target positioning and grabbing method based on binocular vision |
-
2018
- 2018-07-10 CN CN201810752380.2A patent/CN108942929B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH1063317A (en) * | 1996-08-13 | 1998-03-06 | Fanuc Ltd | Method for combining coordinate system in robot and visual sensor system |
| CN103991078A (en) * | 2013-02-14 | 2014-08-20 | 佳能株式会社 | Robot system and method for controlling the same |
| CN105082161A (en) * | 2015-09-09 | 2015-11-25 | 新疆医科大学第一附属医院 | Robot vision servo control device of binocular three-dimensional video camera and application method of robot vision servo control device |
| CN105217324A (en) * | 2015-10-20 | 2016-01-06 | 上海影火智能科技有限公司 | A kind of novel de-stacking method and system |
Non-Patent Citations (1)
| Title |
|---|
| 基于双目视觉的工件定位与抓取研究;朱代先;《计算机测量与控制》;20110125;第19卷(第1期);第92-94页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108942929A (en) | 2018-12-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108942929B (en) | Mechanical arm positioning and grabbing method and device based on binocular stereo vision | |
| CN107871328B (en) | Machine vision system and calibration method implemented by machine vision system | |
| CN108890652B (en) | Transformer substation inspection robot and transformer substation equipment inspection method | |
| CN112950667B (en) | Video labeling method, device, equipment and computer readable storage medium | |
| CN110672007A (en) | Workpiece surface quality detection method and system based on machine vision | |
| CN111844019A (en) | Method and device for determining grabbing position of machine, electronic device and storage medium | |
| CN114029946A (en) | Method, device and equipment for guiding robot to position and grab based on 3D grating | |
| KR20220089463A (en) | Vision analysis apparatus for picking robot | |
| WO2019123988A1 (en) | Calibration data generating device, calibration data generating method, calibration system, and control program | |
| CN108156359A (en) | Intelligent industrial camera | |
| CN115641517A (en) | Machine vision defect identification method and system, edge side device and storage medium | |
| US11969903B1 (en) | Control method, electronic device and storage medium | |
| CN114608521A (en) | Monocular distance measuring method and device, electronic equipment and storage medium | |
| CN110181504B (en) | Method and device for controlling mechanical arm to move and control equipment | |
| CN109313708B (en) | Image matching method and vision system | |
| KR101412513B1 (en) | Method and system for controlling robot arm using frame grabber board | |
| CN111753588B (en) | Power equipment online perception and focusing algorithm based on deep learning | |
| CN114266822B (en) | Workpiece quality inspection method and device based on binocular robot, robot and medium | |
| CN113635299B (en) | Mechanical arm correction method, terminal device and storage medium | |
| CN115393278A (en) | Method and system for inspecting state of pressing plate, robot and storage medium | |
| CN115219492A (en) | Appearance image acquisition method and device for three-dimensional object | |
| CN113592864A (en) | Transformer monitoring method, device, system and medium based on convolutional neural network | |
| CN114723815A (en) | Multi-view fused image semantic target detection method and system | |
| Kheng et al. | Stereo vision with 3D coordinates for robot arm application guide | |
| CN116319501B (en) | Network system for obtaining equipment operation parameters |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200918 Address after: 510620 Tianhe District, Guangzhou, Tianhe South Road, No. two, No. 2, No. Applicant after: Guangzhou Power Supply Bureau of Guangdong Power Grid Co.,Ltd. Address before: 510620 Tianhe District, Guangzhou, Tianhe South Road, No. two, No. 2, No. Applicant before: GUANGZHOU POWER SUPPLY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |