CN108247635B

CN108247635B - Method for grabbing object by depth vision robot

Info

Publication number: CN108247635B
Application number: CN201810034599.9A
Authority: CN
Inventors: 陈国华; 邢健; 王俊义; 张爱军; 于洪杰; 王永生
Original assignee: Beijing University of Chemical Technology
Current assignee: Beijing University of Chemical Technology
Priority date: 2018-01-15
Filing date: 2018-01-15
Publication date: 2021-03-26
Anticipated expiration: 2038-01-15
Also published as: CN108247635A

Abstract

The invention discloses a method for grabbing an object by a depth vision robot. The system mainly uses a hand-eye system experimental platform constructed by a Mitsubishi mechanical arm carrying a Realsense depth camera. The robot target grabbing method based on the depth vision mainly comprises the following steps: (1) acquiring object point cloud data and a depth image; (2) removing point cloud data of a plane where the object is located; (3) dividing the object by using the point cloud data after the plane is removed by using Euclidean clustering, LCCP (LCCP) and CPC (CPC) methods; (4) selecting an interested area according to the segmentation result; (5) calculating a gradient map of the depth image corresponding to the region of interest; (6) selecting an optimal grabbing point on the depth map corresponding to the obtained contour of the gradient image; (7) and calculating the motion trail of the robot according to the inverse kinematics of the robot and controlling the robot to grab through a serial port instruction.

Description

Method for grabbing object by depth vision robot

Technical Field

The invention relates to the technical field of intelligent robots, in particular to a method for grabbing articles by a robot.

Background

Due to the development of artificial intelligence technology, the automation requirements of the robot are gradually becoming higher, which requires the robot to perform autonomous operations such as grasping and transferring an object according to human instructions.

At present, most robots acquire external data through cameras, position and capture objects through image processing and other modes, and the RGB cameras are mostly used.

When the inventor of the present invention realizes the technical scheme of the present invention, the inventor finds that at least the following problems exist in the prior art:

the existing method for grabbing and searching the identification points through the two-dimensional camera is slow in time and has great dependence on ambient light. The method using the depth camera requires training of a large number of data sets on one hand, and cannot effectively solve the problem of object occlusion on the other hand.

In a real environment, an object with an unknown structure and a situation that the object is shielded are common, so that the grabbing of the unknown object is an important subject.

Disclosure of Invention

In view of the above-mentioned defects of the prior art, the problem to be solved by the present invention is to provide a robot grasping method which can solve the occlusion problem and can stably grasp an object of unknown structure.

In order to achieve the above object, the technical solution provided by this patent is: which comprises the following steps:

(1) acquiring object point cloud data and a depth image;

(2) removing point cloud data of a plane where the object is located;

(3) performing object segmentation by using the point cloud data after the plane is removed;

(4) selecting an interested area according to the segmentation result;

(5) calculating a gradient map of the depth image corresponding to the region of interest by using an edge detection operator;

(6) selecting an optimal grabbing point on the depth map corresponding to the obtained contour of the gradient image;

(7) and calculating the motion trail of the robot according to the inverse kinematics of the robot and controlling the robot to grab through a serial port instruction.

Further, the specific steps of point cloud segmentation and upsampling in the step (3) are as follows:

1) performing clustering analysis on the point cloud after the plane is removed by using Euclidean clustering, and preliminarily dividing the point cloud of the object into a plurality of parts;

2) partitioning each part of point cloud after clustering by using an LCCP algorithm to prevent different objects from clustering into the same class caused by object shielding, thereby partitioning the point cloud into a plurality of independent point clouds;

3) after the LCCP divides the point cloud, the MLS fitting algorithm is used for up-sampling the object point cloud and enabling the object point cloud to be uniformly distributed;

4) and further, the point cloud of the object is segmented by using a CPC method, and the point cloud of a single object is segmented into different parts based on geometric features, so that the time for searching for effective capturing points is reduced as much as possible, and the point cloud information of the whole object does not need to be traversed.

Further, in the step (4), gradient processing of the depth map is performed in the depth map corresponding to the region of interest, and the specific steps are as follows:

1) respectively extracting point cloud centers of the objects segmented by the LCCP;

2) selecting the point cloud where the point cloud center (marked as M) closest to the Euclidean distance of the camera is located as the point cloud for further processing;

3) respectively calculating the centers of the sub-point clouds divided by the CPC method, and sequencing the sub-point clouds from near to far according to the distance from M;

4) and sequentially taking the sorted sub-point clouds as interested parts to calculate gradient images in the corresponding depth maps.

Further, in the step (6), an optimal capture point is selected on the depth map corresponding to the obtained profile of the gradient image according to a certain method, and the specific steps are as follows:

1) selecting any two points on the gradient map outline, and calculating the depth information and the normal magnitude of each point;

2) taking two points as reference points respectively, and taking a series of points which are basically the same as a normal vector of the reference points and have Euclidean distances with the reference points on spatial positions not exceeding a threshold (the threshold is 3 used herein, and the value can be adjusted to be small if the size of an object is small) as contact lines;

3) the reliability (denoted as P) of the two reference points as the grasping points is determined based on the following three conditions_grasp)：

a. The difference of the depth values between the point closest to the camera and the two reference points in the area defined by the two contact lines does not exceed the length of the fingers of the manipulator;

b. the vertical distance between line segments formed by fitting the two contact lines does not exceed the maximum opening size of the manipulator;

c. and calculating the reliability of the grabbing point according to the following formula after the two requirements are met:

number representing contact line, N_jRepresenting the total number of points, m, on the j-th contact line_jiRepresents the normal vector value, L, of the ith point on the jth contact line₁、L₂Each representing the length of two contact lines, L representing the width v of the gripper clamping plate₁₂Representing the vector connecting the midpoints of two lines of contact, m₁Indicating the normal vector value of the first reference point, P_graspSet to 0.8 in this context, if the final result is greater than 0.8, it is assumed that these two points make it possible to grasp the point of contact, and if the accuracy is to be improved, the value can be made appropriately large.

Drawings

FIG. 1 is a flow chart of a depth vision based object capture method in accordance with an embodiment of the present invention;

FIG. 2 is a graphical illustration of various parameters in a catch point confidence approach;

FIG. 3 is a graph showing the effect of the experimental process in each step;

Detailed Description

Table 1 shows the results of the grab experiment using the present invention;

wherein

And

is a clamping plate of the hand grip,

and

is a contact line, G_graspThe depth of the finger grip, W is the maximum opening size of the manipulator, and n₁、n₂Respectively representing the mean vector of the points on two contact lines, L₁、L₂Respectively representing the length of the short and the length of the long of the two contact lines, v₁₂Representing the vector connecting the midpoints of the two contact lines.

TABLE 1

Object	Success rate	Mean time
			Dish with a cover	9/10	2.324
Adhesive tape	10/10	2.012
			Cup with elastic band	10/10	2.435
Spoon	10/10	2.044
			Basin	10/10	2.145
Total of	98％	2.192

Claims

1. A method for robot grabbing based on depth vision is characterized by comprising the following steps:

(1) acquiring object point cloud data and a depth image;

(2) removing point cloud data of a plane where the object is located;

(4) selecting an interested area according to the segmentation result;

(7) calculating the motion track of the robot according to inverse kinematics of the robot and controlling the robot to grab through a serial port instruction;

the step (6) comprises the following specific steps:

2) taking two points as datum points, and recording a series of points which are basically the same as a normal vector of the datum point and have Euclidean distances with the datum point on a space position not exceeding a threshold value as contact lines; the threshold value is 3;

3) the reliability of the two reference points as the capture points is judged according to the following three conditions and is marked as P_grasp)：

j represents the serial number of the contact line, N_jRepresenting the total number of points, m, on the j-th contact line_jiRepresents the normal vector value, L, of the ith point on the jth contact line₁、L₂Respectively representing the lengths of the two contact lines, and L representing the width of the gripper clamping plate; v. of₁₂Representing the vector connecting the midpoints of two lines of contact, m₁Indicating the normal vector value of the first reference point, P_graspSet to 0.8 in this context, these two points can be used as contact points for grasping if the final result is greater than 0.8.