CN116439018A - Seven-degree-of-freedom fruit picking robot and picking method thereof - Google Patents
Seven-degree-of-freedom fruit picking robot and picking method thereof Download PDFInfo
- Publication number
- CN116439018A CN116439018A CN202310493761.4A CN202310493761A CN116439018A CN 116439018 A CN116439018 A CN 116439018A CN 202310493761 A CN202310493761 A CN 202310493761A CN 116439018 A CN116439018 A CN 116439018A
- Authority
- CN
- China
- Prior art keywords
- degree
- picking
- points
- mechanical arm
- fruit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 235000013399 edible fruits Nutrition 0.000 title claims abstract description 116
- 238000000034 method Methods 0.000 title claims abstract description 59
- 239000012636 effector Substances 0.000 claims abstract description 43
- 230000016776 visual perception Effects 0.000 claims abstract description 9
- 238000004422 calculation algorithm Methods 0.000 claims description 29
- 239000011159 matrix material Substances 0.000 claims description 24
- 238000013519 translation Methods 0.000 claims description 23
- 230000007246 mechanism Effects 0.000 claims description 22
- 230000009466 transformation Effects 0.000 claims description 21
- 238000006243 chemical reaction Methods 0.000 claims description 15
- 230000006870 function Effects 0.000 claims description 12
- 238000002372 labelling Methods 0.000 claims description 12
- 238000001514 detection method Methods 0.000 claims description 7
- 239000002390 adhesive tape Substances 0.000 claims description 6
- 238000007619 statistical method Methods 0.000 claims description 5
- 238000009826 distribution Methods 0.000 claims description 4
- 238000009434 installation Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000004891 communication Methods 0.000 claims description 3
- 238000003384 imaging method Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 claims description 3
- 238000005096 rolling process Methods 0.000 claims 2
- 230000008569 process Effects 0.000 abstract description 5
- 239000000463 material Substances 0.000 abstract description 2
- 238000012935 Averaging Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 230000004888 barrier function Effects 0.000 description 3
- 238000010008 shearing Methods 0.000 description 3
- 238000005520 cutting process Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 229910000838 Al alloy Inorganic materials 0.000 description 1
- 244000141359 Malus pumila Species 0.000 description 1
- 241000508269 Psidium Species 0.000 description 1
- 241000220324 Pyrus Species 0.000 description 1
- 230000032683 aging Effects 0.000 description 1
- XAGFODPZIPBFFR-UHFFFAOYSA-N aluminium Chemical compound [Al] XAGFODPZIPBFFR-UHFFFAOYSA-N 0.000 description 1
- 229910052782 aluminium Inorganic materials 0.000 description 1
- 235000021016 apples Nutrition 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000003796 beauty Effects 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 235000021022 fresh fruits Nutrition 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 230000002427 irreversible effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 235000021017 pears Nutrition 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 238000013138 pruning Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 238000010998 test method Methods 0.000 description 1
- 229940088594 vitamin Drugs 0.000 description 1
- 229930003231 vitamin Natural products 0.000 description 1
- 235000013343 vitamin Nutrition 0.000 description 1
- 239000011782 vitamin Substances 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01D—HARVESTING; MOWING
- A01D46/00—Picking of fruits, vegetables, hops, or the like; Devices for shaking trees or shrubs
- A01D46/30—Robotic devices for individually picking crops
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P60/00—Technologies relating to agriculture, livestock or agroalimentary industries
- Y02P60/12—Technologies relating to agriculture, livestock or agroalimentary industries using renewable energies, e.g. solar water pumping
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Environmental Sciences (AREA)
- Manipulator (AREA)
Abstract
The invention provides a seven-degree-of-freedom fruit picking robot which comprises a crawler-type chassis (10), a seven-degree-of-freedom mechanical arm assembly (20), an end effector (30), a visual perception system and an obstacle avoidance path planning system, wherein the crawler-type chassis is provided with a plurality of robot arms; the seven-degree-of-freedom mechanical arm assembly (20) consists of a one-degree-of-freedom mechanical arm (21) and a six-degree-of-freedom mechanical arm (22); the visual perception system comprises a depth camera (40) and a positioning module. The picking robot can automatically locate and identify the space position coordinates of arbor fruits, meanwhile, the picking path planning for avoiding the obstacles such as branches or leaves is completed, automatic and efficient picking of arbor fruits is achieved, manpower and material resources are effectively saved, the problems of wrong picking, missing picking and the like caused by the influence of the obstacles in the picking process are avoided, and further good quality of picked fruits is ensured, and damage to the fruits or fruit trees is avoided.
Description
Technical Field
The invention relates to the technical field of fruit picking, in particular to a seven-degree-of-freedom fruit picking robot and a picking method thereof.
Background
Arbor fruits such as apples, pears, guava, oranges and the like can supplement various vitamins required by human bodies, and also have certain effects of preventing diseases, slowing down aging and maintaining beauty. The demand of arbor fruits is increased, and fresh fruits are mainly used, so that picking, transportation and the like are required to be completed in a short time after the arbor fruits are ripe, the freshness of the arbor fruits is ensured, and the daily demands of people are met. However, the existing agriculture is deficient in labor force, and the picking efficiency of manual picking is low in a single day, so that the picking cost is high, and the production cost of fruits is further increased; meanwhile, arbor fruits have certain picking time limit requirements, and the time limit exceeding can lead to poor selling phases, taste and the like of the fruits, thereby leading to the stagnation of products.
At present, mechanical picking is mostly carried out by a pulling device to assist in manual picking; in the pulling process, the picking head can squeeze fruits, damage the fruits, fruit tree branches and the like, and further influence the transportation and selling of the fruits, influence the secondary results of the fruit trees and the like. In addition, arbor fruits show irregular distribution on the branches of fruit trees, the surrounding of the fruits is often accompanied with barrier shielding, at the present stage, arbor fruits in China are basically planted in three dimensions, and the conditions of pruning are hardly caused due to wide planting area and large planting amount, so that arbor fruits are frequently discovered by shielding conditions of branches, leaves and the like, the existing picking device cannot effectively avoid the barrier such as the branches, the leaves and the like, the problems of wrong picking, missed picking, wrong picking and the like are extremely easy to cause, the success rate of fruit picking is reduced, irreversible damage is easily caused to the fruits and fruit tree branches, and the problems of fruit tree yield reduction, fruit stagnation and the like are also easy to cause.
Disclosure of Invention
Aiming at the problems existing in the prior art, the invention aims to provide the seven-degree-of-freedom fruit picking robot which can automatically position and identify arbor fruits, finish planning of picking paths, automatically avoid obstacles such as branches or leaves, finish automatic and mechanical picking of arbor fruits, and has the advantages of simple operation and high picking efficiency, thereby effectively saving the labor cost of picking and ensuring the yield of fruit trees and the quality of fruits.
The invention further aims to provide a picking method of the seven-degree-of-freedom fruit picking robot, so that intelligent picking of arbor fruits is completed.
The aim of the invention is achieved by the following technical scheme:
a seven degrees of freedom fruit picking robot which is characterized in that: the robot comprises a crawler chassis, a seven-degree-of-freedom mechanical arm assembly, an end effector, a visual perception system and an obstacle avoidance path planning system; the seven-degree-of-freedom mechanical arm assembly consists of a degree-of-freedom mechanical arm and a six-degree-of-freedom mechanical arm, wherein the one-degree-of-freedom mechanical arm is arranged on the crawler chassis, and the six-degree-of-freedom mechanical arm is arranged on the one-degree-of-freedom mechanical arm; the end effector is arranged at one end part of the six-degree-of-freedom mechanical arm far away from the one-degree-of-freedom mechanical arm; the visual perception system comprises a depth camera and a positioning module, wherein the depth camera is arranged on the end effector and moves along with the end effector; the obstacle avoidance path planning system and the positioning module are integrated in a central control device, and the central control device is installed on the crawler chassis and is electrically connected with the crawler chassis, the seven-degree-of-freedom mechanical arm assembly, the end effector and the depth camera respectively.
Further optimized, the crawler-type chassis is provided with a fruit basket for storing picked fruits.
The end effector comprises a base, a crank slide block mechanism and a four-bar mechanism, wherein the crank slide block mechanism is arranged on the base and comprises a rotating disc, a connecting rod and a slide block, the rotating disc is rotationally arranged on the end face of the base, the corresponding end face of the base is slidingly provided with the slide block, one end of the connecting rod is rotationally connected with the outer ring of the rotating disc, and the other end of the connecting rod is rotationally connected with the top surface of the slide block, so that the translation of the slide block is controlled through the rotation of the rotating disc; the four-bar mechanism is two groups of symmetrical arrangement, including first connecting rod, "L" shape connecting rod, "7" font connecting rod and second connecting rod, the slider is kept away from connecting rod one end and is set up the installation piece corresponding two sets of four-bar mechanism, the first connecting rod of two sets of four-bar mechanism rotates with the installation piece respectively and is connected, the one end of "L" shape connecting rod rotates with corresponding first connecting rod respectively and is connected, the other end rotates with corresponding "7" font connecting rod one end respectively and is connected, and the turning and the base of "L" shape connecting rod rotate and are connected, two second connecting rods set up between two "L" shape connecting rods and second connecting rod one end rotate with the base terminal surface respectively and be connected, the other end rotates with corresponding "7" font connecting rod turning respectively and is connected.
And further optimizing, the bottom surface of the base is fixedly provided with a steering engine, and the steering engine output shaft and the rotating disc are coaxial and fixedly sleeved on the outer wall of the steering engine output shaft.
And further optimizing, the base is provided with a sliding guide rail corresponding to the sliding block, and the sliding block is clamped on the sliding guide rail and is in sliding connection.
And further optimizing, clamping rubber strips are respectively arranged on the opposite side surfaces of the 7-shaped connecting rods, and blades are arranged on the end surfaces of the 7-shaped connecting rods, which are positioned on the upper sides of the clamping rubber strips.
Further optimized, the depth camera employs a RealSense D435i camera.
A picking method of a seven-degree-of-freedom fruit picking robot adopts the picking robot and is characterized in that: the method comprises a picking point and obstacle positioning algorithm, a hand-eye calibration algorithm and a path planning algorithm;
the picking point and obstacle positioning algorithm specifically comprises the following steps:
firstly, acquiring an RGB image and a depth image by using a depth camera, and dividing the RGB image by applying a semantic dividing network;
then, labeling each pixel in the image, namely labeling the background of 0, labeling the connected domains with 1,2, …, N-1 and N respectively, and labeling the pixels of the same connected domain with the same label; if the objects are overlapped, a plurality of connected domains of different objects are designed as one connected domain, so that fruit connected domains are extracted; converting each fruit connected domain into a three-dimensional point cloud by combining the depth image, converting the obstacle binarization region into the three-dimensional point cloud, and removing noise in the three-dimensional point cloud by using a statistical analysis method; then, performing sphere fitting on the fruit connected domain in the three-dimensional point cloud by using a least square method to obtain a fruit center point position and a radius value; finally, taking the position with a radius value vertically upwards from the center point of the fruit as a fruit picking point;
then, voxel processing is carried out on the obtained three-dimensional point cloud by using a voxel gridding point cloud simplification method, so that the spatial distribution of a large number of cubes approaching obstacles and fruits is realized;
the hand-eye calibration algorithm is used for solving the pose conversion relation between the camera and the end effector so as to obtain the coordinate conversion relation between the end effector and the picking target;
the path planning algorithm specifically comprises the following steps: firstly, randomly selecting 2-3 points between an initial point and a target point, judging the selected points, and judging whether the selected points are on an obstacle or not, if so, replacing the corresponding random points; determining a random point Q r Will followThe machine point, the initial point and the target point form a plurality of search trees and simultaneously follow a random point Q r The direction is expanded by unit step length to find a new node Q n Repeating the steps with new nodes, wherein collision with obstacles, step sizes and the like are considered in the period until a plurality of search trees are connected with each other at the same time (the presence of the interconnection of the plurality of search trees indicates that a path is found and planning is completed);
the path planning algorithm expands the tree with low cost based on the RRT double-tree opposite alternating expansion exploration mode, has directivity to a certain extent, avoids too random sampling, and has higher success rate and obviously improved searching speed and searching efficiency when searching sampling in the face of a narrow channel.
And further optimizing, wherein the method for using the least square method to perform sphere fitting on the fruit connected domain in the three-dimensional point cloud comprises the following specific steps of:
firstly, establishing a sphere equation:
then, assume that the three-dimensional point cloud coordinates of the ith fruit are (x i ,y i ,z i ) The method is carried into a sphere equation, and can be obtained:
and (3) making:
then:
the center coordinates (x) can be obtained by calculation of the points 0 ,y 0 ,z 0 ) Radius of spherer 0 ；
Then, presetting a reasonable range value H, and calculating the coordinates (x 0 ,y 0 ,z 0 ) Distance h of (2) 0 If |h 0 -r 0 The number of the points in the whole fitting sphere is recorded in sequence when the number of the points in the fitting sphere is smaller than H;
and repeating sphere fitting for a plurality of times, selecting the model parameter corresponding to the sphere with the largest number of points in the model as the best fitting parameter, and outputting the spherical center coordinates (x, y, z) and the radius r of the best fitting parameter as the three-dimensional fitting result of the fruit.
And further optimizing, wherein the reasonable range value H is 0-0.5, and the reasonable range value H is obtained through a large amount of early experimental data.
Further optimizing, the voxel gridding point cloud simplification method specifically comprises the following steps of:
first, a minimum three-dimensional voxel cuboid is created from the point cloud, the volume of which is V:
V＝L x ·L y ·L z ；
wherein: l (L) x Representing the maximum range of the X-axis direction of the point cloud; l (L) y Representing the maximum range of the Y-axis direction of the point cloud; l (L) z Representing the maximum range of the Z-axis direction of the point cloud;
then, calculating the side length L of the small cube grid to be divided, and decomposing the minimum three-dimensional voxel cuboid into the small cubes according to the size of LA plurality of small cube grids; after grid division is finished, the point cloud data are put into the corresponding small grids, and the small grids which do not contain data points are deleted; in each small grid, the data point nearest to the center of the small grid is kept, representing all points in the present small grid, and the remaining points are deleted.
Further optimizing, the hand-eye calibration algorithm specifically comprises the following steps:
firstly, converting according to pixel values, depth values and parameters of a depth camera of all angular points of a checkerboard of a calibration plate through a conversion formula between a pixel coordinate system and a world coordinate system, so as to obtain space coordinates of all the angular points on the checkerboard of the calibration plate, converting the space coordinates into a coordinate array of corresponding points, and recording the coordinate array of imaging points, corresponding to all the angular points in a photographed picture, under an image coordinate system as a plurality of groups of control points; the conversion formula is:
wherein: x, Y, Z the coordinates in the world coordinate system; f represents a focal length; r represents a 3x3 orthogonal rotation matrix; t represents a three-dimensional translation vector; u (u) 0 、v 0 Representing coordinates of an origin of an image coordinate system in a pixel coordinate system; dx, dy denote the physical dimensions of each pixel in the x-direction, y-direction of the image plane; u, v denote pixel coordinates; z is Z C A vector representing the Z-axis in camera coordinates;
then, the coordinates of a plurality of control points in the three-dimensional scene and the perspective projection coordinates of the control points in the image are utilized to obtain the absolute pose relation between a camera coordinate system and a world coordinate system representing the three-dimensional scene structure, wherein the absolute pose relation comprises an absolute translation vector T and a rotation matrix R, so that the rotation and translation amounts from the space coordinates of all the angular points on the checkerboard of the plurality of groups of calibration plates to the transformation matrix T of the camera coordinate system are obtained;
then determining a DH parameter table of the mechanical arm;
and then, reading out the translation position through network port communication, and calculating a unit vector a on the three axes of x, y and z:
due to two adjacent connecting rods T i And T is i-1 The transformation relation of (2) is:
wherein: a, a i Representing the included angle between the distance between the common normal lines and two axes perpendicular to the plane; d, d i Representing the relative positions of the two connecting rods; θ i Indicating the relative position d of the two links i An included angle with a common vertical line of the two connecting rods; c represents a cos () function in the trigonometric function; s represents a sin () function in the trigonometric function;
the seven-degree-of-freedom mechanical arm assembly is moved for multiple times, and rotation and translation amounts of multiple groups of end effectors relative to the base coordinates are obtained; the method comprises the steps of recording a plurality of sets of rotation and translation parameters of an end effector relative to basic coordinates as a plurality of sets of gesture matrixes B, regarding a transformation matrix T from the space coordinates of all angular points on a plurality of sets of calibration plate checkerboards to a camera coordinate system as an external parameter matrix A, and obtainingGroup A, B, the rotation and translation transformation from the camera coordinate system to the seven-degree-of-freedom mechanical arm component end coordinate system is calculated by using ax=xb, thereby obtaining a transformation matrix T from the camera coordinate system to the seven-degree-of-freedom mechanical arm component end coordinate system e ；
Finally, moving the seven-degree-of-freedom mechanical arm assembly, photographing the picking target by using a depth camera, and obtaining the space coordinates of the picking target by using a conversion formula between a pixel coordinate system and a world coordinate system; reuse of transformation matrix T e And calibrating a transformation matrix T from the space coordinates of all the angular points on the checkerboard of the plate to a camera coordinate system to obtain the pose of the picking target relative to the end effector coordinate system.
Further optimizing, in order to optimize the path with less unnecessary turns, in the path planning algorithm, a tree set T is searched ree Any one of the nodes in (a) takes two points Q i And Q is equal to j Wherein i, j e [1,2,3, …, n]The method comprises the steps of carrying out a first treatment on the surface of the For Q i And Q is equal to j Collision detection is performed on the paths between the paths, and if no collision exists, Q is deleted i And Q is equal to j All nodes in between.
Further preferably, the collision detection is: adopting a mode of projecting onto three coordinate axes of a space coordinate system, and performing collision detection between the cube and the cylinder, wherein if the projection length of a line segment connecting the cube and the cylinder after projection is larger than the sum of the distances from the respective center points to the respective longest boundaries after projection, and the three axes are simultaneously satisfied, the cube and the cylinder are regarded as not being collided;
the method comprises the following steps:
first, find four vertexes of the cube and obtain the center point P of the cube as projection by averaging its coordinates lm The method comprises the steps of carrying out a first treatment on the surface of the Finding out the center coordinates of the upper and lower bottom surfaces of the cylinder, and obtaining the center point P of the cylinder when the cylinder is not projected by averaging the center coordinates ym The method comprises the steps of carrying out a first treatment on the surface of the Then, calculate and obtain the center point P lm And a center point P ym Connection line between d Its projection on the x-axis is l D At the same time, the center points P are respectively obtained lm And a center point P ym Projection P on x-axis lM And P yM The method comprises the steps of carrying out a first treatment on the surface of the Thereafter, calculate P lM And P yM Distance r to the boundary of the respective object (i.e. cube, cylinder) after projection a And r b ：
If |l D |＞r a +r b And also satisfies the obtained |l in the y-axis and the z-axis D |＞r a +r b The cube and the cylinder have no collision; otherwise, there is a collision of the cube with the cylinder.
The invention has the following technical effects:
the cooperation of this application crawler-type chassis, seven degrees of freedom arm components and end effector has realized effectively reducing the artificial labour cost of picking the in-process to arbor class fruit's automation and has picked, improves picking efficiency.
Meanwhile, the method and the device divide by matching the RGB image with the depth image and utilizing the semantic division network, and then obtain the interested areas of the fruits and the obstacles through matching the labels with the connected domains, so that the robustness and the instantaneity are high; converting each fruit connected domain into three-dimensional point cloud by combining the data of the depth image, removing noise in the three-dimensional point cloud by using a statistical analysis method, performing sphere fitting on the three-dimensional point cloud by using a least square method, and obtaining the position and the radius value of the center point of the fruit, thereby obtaining picking points of the fruit, having accurate positioning, high precision and small positioning error, and being capable of effectively realizing accurate picking of arbor fruits; through a path planning algorithm, the planning of a picking path is completed, so that obstacle avoidance in the picking process is effectively completed, damage to fruit trees in the picking process is avoided, picking effectiveness is ensured, the problems of wrong picking, missing picking and the like caused by the influence of obstacles in the picking process are avoided, and damage to fruits or fruit trees, influence on the quality of the fruits or the resultof the fruit trees are avoided. In addition, this application is through the cooperation of base, slider-crank mechanism and four pole mechanism, realizes the shearing to arbor class fruit, and it is good to pick the continuity, can guarantee simultaneously to pick the back to the collection of fruit, pick the timeliness strong, can effectively guarantee to pick efficiency, use manpower sparingly material resources and pick the cost.
Drawings
Fig. 1 is a schematic structural view of a fruit picking robot according to an embodiment of the present application.
Fig. 2 is a schematic structural view of an end effector of the fruit picking robot in an embodiment of the present application.
Fig. 3 is a semantic segmentation effect diagram of fruits and obstacles in the embodiment of the present application.
Fig. 4 is a three-dimensional reconstruction effect diagram of an obstacle in an embodiment of the present application.
10, a crawler-type chassis; 11. fruit basket; 20. a seven degree of freedom robotic arm assembly; 21. a degree of freedom mechanical arm; 22. a six degree of freedom mechanical arm; 30. an end effector; 31. a base; 310. steering engine; 321. a rotating disc; 322. a connecting rod; 323. a slide block; 3230. a mounting block; 331. a first link; 332. an L-shaped connecting rod; 333. a 7-shaped connecting rod; 3331. a blade; 3332. clamping the adhesive tape; 334. a second link; 40. a depth camera; 50. and the central control device.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments.
Example 1:
as shown in fig. 1-2: a seven degrees of freedom fruit picking robot which is characterized in that: the robot comprises a crawler-type chassis 10, a seven-degree-of-freedom mechanical arm assembly 20, an end effector 30, a visual perception system and an obstacle avoidance path planning system; the crawler chassis 10 adopts a crawler chassis of a Songling robot (Shenzhen limited company), and the top of the crawler chassis is provided with a set of supporting devices which are composed of an aluminum alloy flat plate, an 8080 aluminum profile and the like. The seven-degree-of-freedom mechanical arm assembly 20 consists of a degree-of-freedom mechanical arm 21 and a six-degree-of-freedom mechanical arm 22, wherein the one-degree-of-freedom mechanical arm 21 is arranged on the crawler chassis 10, and the six-degree-of-freedom mechanical arm 22 is arranged on the one-degree-of-freedom mechanical arm 21; the mechanical arm 21 with one degree of freedom adopts a 24V stepping motor as a prime motor, and adopts a ball screw mechanism to realize linear motion; the six-degree-of-freedom robot arm 22 adopts Rui Er Man RML63 series robots, and has a dead weight of 10.2kg and a working radius of 900 mm.
The end effector 30 is mounted at an end of the six-degree-of-freedom mechanical arm 22 remote from the one-degree-of-freedom mechanical arm 21; the device comprises a base 31, a crank-slider mechanism and a four-bar mechanism, wherein the crank-slider mechanism is arranged on the base 31 and comprises a rotating disc 321, a connecting rod 322 and a slider 323, the rotating disc 321 is rotatably arranged on the end face of the base 31, the slider 323 is arranged on the corresponding end face of the base 31 in a sliding manner (as shown in fig. 2, namely the upper end face of the base 31), one end of the connecting rod 322 is rotatably connected with the outer ring of the rotating disc 321, and the other end of the connecting rod is rotatably connected with the top face of the slider 323, so that the translation of the slider 323 is controlled through the rotation of the rotating disc 321; the bottom surface of the base 31 is fixedly provided with a steering engine 310, an output shaft of the steering engine 310 is coaxial with a rotating disc 321, and the rotating disc 321 is fixedly sleeved on the outer wall of the output shaft of the steering engine 310; the base 31 is provided with a sliding guide rail corresponding to the sliding block 323, and the sliding block 323 is clamped on the sliding guide rail and is in sliding connection, so that a hard limit is formed on the sliding direction of the sliding block 323. The four-bar mechanism is two groups symmetrically arranged, including a first link 331, an L-shaped link 332, a 7-shaped link 333 and a second link 334, one end of a slider 323 far away from the link 322 is provided with a mounting block 3230 corresponding to the two groups of four-bar mechanisms, the first links 331 of the two groups of four-bar mechanisms are respectively and rotatably connected with the mounting block 3230, one ends of the L-shaped links 332 are respectively and rotatably connected with the corresponding first links 331, the other ends of the L-shaped links 332 are respectively and rotatably connected with one ends of the corresponding 7-shaped links 333, the corners of the L-shaped links 332 are rotatably connected with the base 31 through first rotating shafts arranged on the base 31 (i.e. as shown in fig. 2, the first rotating shafts are rotatably arranged on the end surface of the base 31, the corners of the L-shaped links 332 are fixedly sleeved on the corresponding first rotating shaft outer walls), and the two second links 334 are arranged between the two L-shaped links 332 and one ends of the second links 334 are respectively rotatably connected with the end surface of the base 31 through second rotating shafts arranged on the base 31 (i.e. as shown in fig. 2, the second rotating shafts are rotatably sleeved on the end surfaces of the second links 334 are fixedly sleeved on the second outer walls), and are respectively connected with the corresponding 7-shaped links 333). The opposite sides of the 7-shaped connecting rod 333 are respectively provided with a clamping adhesive tape 3332, and the end surface of the 7-shaped connecting rod 333 positioned on the upper side of the clamping adhesive tape 3332 is provided with a blade 3331 (shown in fig. 2).
The visual perception system comprises a depth camera 40 and a positioning module, wherein the depth camera 40 adopts a RealSense D435i camera which is arranged on the end effector 30 (in particular to the end face of the base 31) and moves along with the end effector 30; the obstacle avoidance path planning system and the positioning module are integrated in a central control device 50, and the central control device 50 is mounted on the crawler chassis 10 and is electrically connected with the crawler chassis 10, the seven-degree-of-freedom mechanical arm assembly 20, the end effector 30 and the depth camera 40 respectively.
The crawler-type chassis 10 is provided with a fruit basket 11 for storing picked fruits.
Example 2:
a picking method of a seven-degree-of-freedom fruit picking robot, which adopts the fruit picking robot as shown in embodiment 1, is characterized in that: the method comprises a picking point and obstacle positioning algorithm, a hand-eye calibration algorithm and a path planning algorithm; the picking point and obstacle positioning algorithm and the hand-eye calibration algorithm are operated by the positioning module, and the path planning algorithm is operated by the obstacle avoidance path planning system.
The picking point and obstacle positioning algorithm specifically comprises the following steps:
firstly, acquiring RGB images and depth images with 640 x 480 pixels of resolution by using a depth camera, and dividing the RGB images by using a semantic division network deep LabV3+;
then, labeling each pixel in the image, namely labeling the background of 0, labeling the connected domains with 1,2, …, N-1 and N respectively, and labeling the pixels of the same connected domain with the same label; if the objects are overlapped, a plurality of connected domains of different objects are designed as one connected domain, so that fruit connected domains are extracted;
and then combining the depth image, converting each fruit connected domain into a three-dimensional point cloud, and simultaneously converting the barrier binarization region into the three-dimensional point cloud, wherein the method specifically comprises the following steps of:
in (x) i ,y i ,z i ) Representing three-dimensional point cloud coordinates of the pixel point i; (u) i ,v i ) Representing the image coordinates of a pixel point i in a fruit area of arbor fruits; (u) 0 ,v 0 ) Representing the optical center position of the depth camera, and obtaining the optical center position through earlier-stage image calibration; (f) x ,f y ) Representing the focal length of the depth camera, and obtaining the focal length through earlier-stage image calibration; i d Representing the obtained depth image.
Removing noise in the three-dimensional point cloud by using a statistical analysis method, wherein the method specifically comprises the following steps: after the three-dimensional point cloud is obtained, calculating the average distance from each point in the three-dimensional point cloud to all adjacent pointsThen calculate the distance from each point to all its nearest points, difference its average distance +.>Then squaring, averaging to obtain variance, using variance root to obtain standard deviation, and using Laida test method in statistical analysis method: i.e. if suspicious data x p Arithmetic mean value with experimental data ∈ ->The absolute value of the deviation is more than three times of standard deviation, and the deviation is considered as an outlier and is deleted; if not (i.e., as opposed to the above), then it remains.
Then, performing sphere fitting on the fruit connected domain in the three-dimensional point cloud by using a least square method to obtain a fruit center point position and a radius value; the method comprises the following specific steps:
firstly, establishing a sphere equation:
then, assume that the three-dimensional point cloud coordinates of the ith fruit are (x i ,y i ,z i ) The method is carried into a sphere equation, and can be obtained:
and (3) making:
then:
(due to (M) T ·M) -1 ＝M -1 ·(M T ) -1 ,(M T ) -1 ·M T ＝1)；
The center coordinates (x) can be obtained by calculation of the points 0 ,y 0 ,z 0 ) Radius r of sphere 0 ；
Then presetting a reasonable range value H, wherein H is 0-0.5, and calculating coordinates (x) from each point to the sphere center in the three-dimensional point cloud through a large amount of early experimental data 0 ,y 0 ,z 0 ) Distance h of (2) 0 If |h 0 -r 0 I < H, then this pointSequentially recording the number of points in the whole fitting sphere in the fitting sphere;
and repeating sphere fitting for a plurality of times, selecting the model parameter corresponding to the sphere with the largest number of points in the model as the best fitting parameter, and outputting the spherical center coordinates (x, y, z) and the radius r of the best fitting parameter as the three-dimensional fitting result of the fruit.
Finally, taking the position with a radius value vertically upwards from the center point of the fruit as a fruit picking point;
then, voxel processing is carried out on the obtained three-dimensional point cloud by using a voxel gridding point cloud simplification method, so that the spatial distribution of a large number of cubes approaching obstacles and fruits is realized; the method comprises the following steps:
first, a minimum three-dimensional voxel cuboid is created from the point cloud, the volume of which is V:
V＝L x ·L y ·L z ；
wherein: l (L) x Representing the maximum range of the X-axis direction of the point cloud; l (L) y Representing the maximum range of the Y-axis direction of the point cloud; l (L) z Representing the maximum range of the Z-axis direction of the point cloud;
then, calculating the side length L of the small cube grid to be divided, and decomposing the minimum three-dimensional voxel cuboid into the small cubes according to the size of LA plurality of small cube grids; after grid division is finished, the point cloud data are put into the corresponding small grids, and the small grids which do not contain data points are deleted; in each small grid, the data point nearest to the center of the small grid is kept, representing all points in the present small grid, and the remaining points are deleted.
The hand-eye calibration algorithm is used for solving the pose conversion relation between the camera and the end effector, so that the coordinate conversion relation between the end effector and the picking target is obtained; the method comprises the following specific steps:
firstly, converting according to pixel values, depth values and parameters of a depth camera of all angular points of a checkerboard of a calibration plate through a conversion formula between a pixel coordinate system and a world coordinate system, so as to obtain space coordinates of all the angular points on the checkerboard of the calibration plate, converting the space coordinates into a coordinate array of corresponding points, and recording the coordinate array of imaging points, corresponding to all the angular points in a photographed picture, under an image coordinate system as a plurality of groups of control points; the conversion formula is:
wherein: x, Y, Z the coordinates in the world coordinate system; f represents a focal length; r represents a 3x3 orthogonal rotation matrix; t represents a three-dimensional translation vector; u (u) 0 、v 0 Representing coordinates of an origin of an image coordinate system in a pixel coordinate system; dx, dy denote the physical dimensions of each pixel in the x-direction, y-direction of the image plane; u, v denote pixel coordinates; z is Z C A vector representing the Z-axis in camera coordinates;
then, the coordinates of a plurality of control points in the three-dimensional scene and the perspective projection coordinates of the control points in the image are utilized to obtain the absolute pose relation between a camera coordinate system and a world coordinate system representing the three-dimensional scene structure, wherein the absolute pose relation comprises an absolute translation vector T and a rotation matrix R, so that the rotation and translation amounts from the space coordinates of all the angular points on the checkerboard of the plurality of groups of calibration plates to the transformation matrix T of the camera coordinate system are obtained;
the robot DH parameter table was then determined as shown in Table 1 below:
in the table: a, a i (mm) represents the distance between common normal lines; a, a i (°) represents the distance a between common normal lines i (mm) and perpendicular to a i (mm) the angle between the two axes in the plane; d, d i (mm) represents the relative position between the two links; θ i Two connecting rods (degree)Is set by the relative position di d of (d) i (mm) and the included angle between the two connecting rod common vertical lines.
And then, reading out the translation position through network port communication, and calculating a unit vector a on the three axes of x, y and z:
simultaneously, DH parameters of each mechanical arm at the moment can be read out, and the two adjacent connecting rods T i And T is i-1 The transformation relation of (2) is:
wherein: a, a i Representing the included angle between the distance between the common normal lines and two axes perpendicular to the plane; d, d i Representing the relative positions of the two connecting rods; θ i Indicating the relative position d of the two links i An included angle with a common vertical line of the two connecting rods; c represents a cos () function in the trigonometric function; s represents a sin () function in the trigonometric function;
the seven-degree-of-freedom mechanical arm assembly is moved for multiple times, and rotation and translation amounts of multiple groups of end effectors relative to the base coordinates are obtained; the method comprises the steps of recording a plurality of sets of rotation and translation parameters of an end effector relative to basic coordinates as a plurality of sets of gesture matrixes B, regarding a transformation matrix T from the space coordinates of all angular points on a plurality of sets of calibration plate checkerboards to a camera coordinate system as an external parameter matrix A, and obtainingGroup A, B, the rotation and translation transformation from the camera coordinate system to the seven-degree-of-freedom mechanical arm component end coordinate system is calculated by using ax=xb, thereby obtaining a transformation matrix T from the camera coordinate system to the seven-degree-of-freedom mechanical arm component end coordinate system e ；
Finally, moving the seven-degree-of-freedom mechanical arm assembly, photographing the picking target by using the depth camera, and obtaining the picking by using a conversion formula between a pixel coordinate system and a world coordinate systemSpace coordinates of the target; reuse of transformation matrix T e And calibrating a transformation matrix T from the space coordinates of all the angular points on the checkerboard of the plate to a camera coordinate system to obtain the pose of the picking target relative to the end effector coordinate system. Finally, the pose of the picking target relative to the end effector coordinate system can be converted into the pose relative to the manipulator base coordinate system, so that the manipulator can better approach the picking target.
The path planning algorithm specifically comprises the following steps: firstly, randomly selecting 2-3 points between an initial point and a target point, judging the selected points, and judging whether the selected points are on an obstacle or not, if so, replacing the corresponding random points; determining a random point Q r Then forming a plurality of search trees by the random points, the initial points and the target points, and simultaneously along the random point Q r The direction is expanded by a unit step length to find a new node Qn, and the steps are repeated by the new node, wherein collision with an obstacle, step length and the like are considered in the period until a plurality of search trees are connected with each other at the same time (the plurality of search trees are connected with each other at the same time to indicate that a path is found and planning is completed);
in order to optimize the path with less unnecessary turns, in the path planning algorithm, a set T of search trees is searched ree Any one of the nodes in (a) takes two points Q i And Q is equal to j Wherein i, j e [1,2,3, …, n]The method comprises the steps of carrying out a first treatment on the surface of the For Q i And Q is equal to j Collision detection is performed on the paths between the paths, and if no collision exists, Q is deleted i And Q is equal to j All nodes in between.
Wherein, collision detection is: adopting a mode of projecting onto three coordinate axes of a space coordinate system, and performing collision detection between the cube and the cylinder, wherein if the projection length of a line segment connecting the cube and the cylinder after projection is larger than the sum of the distances from the respective center points to the respective longest boundaries after projection, and the three axes are simultaneously satisfied, the cube and the cylinder are regarded as not being collided;
the method comprises the following steps:
first, find four vertexes of the cube and obtain the center point P of the cube as projection by averaging its coordinates lm The method comprises the steps of carrying out a first treatment on the surface of the Finding the cylinder againCenter point P of the cylinder when not projected is obtained by averaging the center coordinates of the upper and lower bottom surfaces of the body ym The method comprises the steps of carrying out a first treatment on the surface of the Then, calculate and obtain the center point P lm And a center point P ym Connection line between d Its projection on the x-axis is l D At the same time, the center points P are respectively obtained lm And a center point P ym Projection P on x-axis lM And P yM The method comprises the steps of carrying out a first treatment on the surface of the Thereafter, calculate P lM And P yM Distance r to the boundary of the respective object (i.e. cube, cylinder) after projection a And r b ：
If |l D |＞r a +r b And also satisfies the obtained |l in the y-axis and the z-axis D |＞r a +r b The cube and the cylinder have no collision; otherwise, there is a collision of the cube with the cylinder.
The specific method for picking fruits by the picking robot comprises the following steps:
firstly, the whole arbor fruit picking robot is driven by the crawler-type chassis 10 to move forwards to a certain picking position, and meanwhile, the central control device 50 initializes the position of the end effector 30, namely, initializes the three-dimensional space coordinates of the end effector 30; the seven-degree-of-freedom mechanical arm assembly 20 performs zero setting operation, and data of arbor fruit relative to the end effector 30 (initialization position) in the horizontal, vertical and front-back directions are obtained according to a singlechip algorithm; thereafter, the central control device 50 obtains picking points of the fruits through the visual perception system and obtains picking paths of the end effector 30 through the obstacle avoidance path planning system
The end effector 30 reaches a certain picking position through the horizontal and vertical adjustment of the one-degree-of-freedom mechanical arm 21 and the six-degree-of-freedom mechanical arm 22, namely, the fruit stems are positioned in the shearing space under the two blades 3331 of the end effector 30; at this time, the central control device 50 controls the operation of the steering engine 310, the steering engine 310 pulls the slider 323 to move to a side close to the steering engine 310 (i.e. to move from a far point to a near point) through the rotating disc 321 and the connecting rod 322, the slider 323 pulls the first connecting rod 331 to move through the mounting block 3230, so that the L-shaped connecting rod 332 rotates for the rotating pair in the mechanism, the L-shaped connecting rod 332 pushes the 7-shaped connecting rod 333 to move, and the 7-shaped connecting rod 333 receives the displacement restriction of the second connecting rod 334, so that the two 7-shaped connecting rods 333 move to a direction close to each other, and the shearing of the fruit stems is realized. During cutting, the clamping adhesive tape 3332 at the lower side of the blade 3331 is in a clamping state to clamp the fruit stalks of the fruits, so that the fruits are prevented from falling off after cutting. Then, the seven-degree-of-freedom mechanical arm assembly 20 is started to enable the end effector 30 to be located above the fruit basket 11 (the final position is determined, so that control can be achieved through a limit sensor or through distance control, and the conventional technology in the field is not excessively discussed in the specific embodiment of the application), the steering engine 310 is continuously started to operate, the sliding block 323 moves to the side far away from the steering engine 310 (namely, moves from a near point to a far point), the 7-shaped connecting rod 333 is opened, and fruits fall into the fruit basket 11 and are collected; the fruit collection work is completed in such a cycle.
Claims (9)
1. A seven degrees of freedom fruit picking robot which is characterized in that: the robot comprises a crawler chassis, a seven-degree-of-freedom mechanical arm assembly, an end effector, a visual perception system and an obstacle avoidance path planning system; the seven-degree-of-freedom mechanical arm assembly consists of a degree-of-freedom mechanical arm and a six-degree-of-freedom mechanical arm, wherein the one-degree-of-freedom mechanical arm is arranged on the crawler chassis, and the six-degree-of-freedom mechanical arm is arranged on the one-degree-of-freedom mechanical arm; the end effector is arranged at one end part of the six-degree-of-freedom mechanical arm far away from the one-degree-of-freedom mechanical arm; the visual perception system comprises a depth camera and a positioning module, wherein the depth camera is arranged on the end effector; the obstacle avoidance path planning system and the positioning module are integrated in a central control device, and the central control device is installed on the crawler chassis and is electrically connected with the crawler chassis, the seven-degree-of-freedom mechanical arm assembly, the end effector and the depth camera respectively.
2. A seven degree of freedom fruit picking robot according to claim 1, wherein: the crawler-type chassis is provided with a fruit basket for storing picked fruits.
3. A seven degree of freedom fruit picking robot according to claim 1 or 2, characterized in that: the end effector comprises a base, a crank-slider mechanism and a four-bar mechanism, wherein the crank-slider mechanism is arranged on the base and comprises a rotating disc, a connecting rod and a slider, the rotating disc is rotationally arranged on the end face of the base, the slider is slidingly arranged on the corresponding end face of the base, one end of the connecting rod is rotationally connected with the outer ring of the rotating disc, and the other end of the connecting rod is rotationally connected with the top surface of the slider; the four-bar mechanism is two groups of symmetrical arrangement, including first connecting rod, "L" shape connecting rod, "7" font connecting rod and second connecting rod, the slider is kept away from connecting rod one end and is set up the installation piece corresponding two sets of four-bar mechanism, the first connecting rod of two sets of four-bar mechanism rotates with the installation piece respectively and is connected, the one end of "L" shape connecting rod rotates with corresponding first connecting rod respectively and is connected, the other end rotates with corresponding "7" font connecting rod one end respectively and is connected, and the turning and the base of "L" shape connecting rod rotate and are connected, two second connecting rods set up between two "L" shape connecting rods and second connecting rod one end rotate with the base terminal surface respectively and be connected, the other end rotates with corresponding "7" font connecting rod turning respectively and is connected.
4. A seven degree of freedom fruit picking robot according to any one of claims 1 to 3, characterized in that: the bottom surface of base is fixed to be set up the steering wheel, steering wheel output shaft and rolling disc coaxial line and the fixed cover of rolling disc are at steering wheel output shaft outer wall.
5. A seven degree of freedom fruit picking robot according to claim 4, wherein: the opposite side surfaces of the 7-shaped connecting rod are respectively provided with a clamping adhesive tape, and the end surface of the 7-shaped connecting rod, which is positioned on the upper side of the clamping adhesive tape, is provided with a blade.
6. The method of picking by a seven degree of freedom fruit picking robot of claim 5 wherein: the method comprises a picking point and obstacle positioning algorithm, a hand-eye calibration algorithm and a path planning algorithm;
the picking point and obstacle positioning algorithm specifically comprises the following steps:
firstly, acquiring an RGB image and a depth image by using a depth camera, and dividing the RGB image by applying a semantic dividing network;
then, labeling each pixel in the image, namely labeling the background of 0, labeling the connected domains with 1,2, …, N-1 and N respectively, and labeling the pixels of the same connected domain with the same label; if the objects are overlapped, a plurality of connected domains of different objects are designed as one connected domain, so that fruit connected domains are extracted; converting each fruit connected domain into a three-dimensional point cloud by combining the depth image, converting the obstacle binarization region into the three-dimensional point cloud, and removing noise in the three-dimensional point cloud by using a statistical analysis method; then, performing sphere fitting on the fruit connected domain in the three-dimensional point cloud by using a least square method to obtain a fruit center point position and a radius value; finally, taking the position with a radius value vertically upwards from the center point of the fruit as a fruit picking point;
then, voxel processing is carried out on the obtained three-dimensional point cloud by using a voxel gridding point cloud simplification method, so that the spatial distribution of a large number of cubes approaching obstacles and fruits is realized;
the hand-eye calibration algorithm is used for solving the pose conversion relation between the camera and the end effector so as to obtain the coordinate conversion relation between the end effector and the picking target;
the path planning algorithm specifically comprises the following steps: firstly, randomly selecting 2-3 points between an initial point and a target point, judging the selected points, and judging whether the selected points are on an obstacle or not, if so, replacing the corresponding random points; determining a random point Q r Then forming a plurality of search trees by the random points, the initial points and the target points, and simultaneously along the random point Q r The direction is expanded by unit step length to find a new node Q n The steps are repeated by new nodes, and collision with obstacles, step sizes and the like are considered in the period until a plurality of search trees are connected at the same time.
7. The method of picking by a seven degree of freedom fruit picking robot of claim 6 wherein: the method for obtaining the position and radius value of the fruit center point comprises the specific steps of:
firstly, establishing a sphere equation:
then, assume that the three-dimensional point cloud coordinates of the ith fruit are (x i ,yi,z i ) The method is carried into a sphere equation, and can be obtained:
and (3) making:
then:
the center coordinates (x) can be obtained by calculation of the points 0 ,y 0 ,z 0 ) Radius r of sphere 0 ；
Then, presetting a reasonable range value H, and calculating the coordinates (x 0 ,y 0 ,z 0 ) Distance h of (2) 0 If |h 0 -r 0 The number of the points in the whole fitting sphere is recorded in sequence when the number of the points in the fitting sphere is smaller than H;
and repeating sphere fitting for a plurality of times, selecting the model parameter corresponding to the sphere with the largest number of points in the model as the best fitting parameter, and outputting the spherical center coordinates (x, y, z) and the radius r of the best fitting parameter as the three-dimensional fitting result of the fruit.
8. A method of picking by a seven degree of freedom fruit picking robot of claim 7 wherein: the hand-eye calibration algorithm specifically comprises the following steps:
firstly, converting according to pixel values, depth values and parameters of a depth camera of all angular points of a checkerboard of a calibration plate through a conversion formula between a pixel coordinate system and a world coordinate system, so as to obtain space coordinates of all the angular points on the checkerboard of the calibration plate, converting the space coordinates into a coordinate array of corresponding points, and recording the coordinate array of imaging points, corresponding to all the angular points in a photographed picture, under an image coordinate system as a plurality of groups of control points; the conversion formula is:
wherein: x, Y, Z the coordinates in the world coordinate system; f represents a focal length; r represents a 3x3 orthogonal rotation matrix; t represents a three-dimensional translation vector; u (u) 0 、v 0 Representing coordinates of an origin of an image coordinate system in a pixel coordinate system; dx, dy denote the physical dimensions of each pixel in the x-direction, y-direction of the image plane; u, v denote pixel coordinates; z is Z C A vector representing the Z-axis in camera coordinates;
then, the coordinates of a plurality of control points in the three-dimensional scene and the perspective projection coordinates of the control points in the image are utilized to obtain the absolute pose relation between a camera coordinate system and a world coordinate system representing the three-dimensional scene structure, wherein the absolute pose relation comprises an absolute translation vector T and a rotation matrix R, so that the rotation and translation amounts from the space coordinates of all the angular points on the checkerboard of the plurality of groups of calibration plates to the transformation matrix T of the camera coordinate system are obtained;
then determining a DH parameter table of the mechanical arm;
and then, reading out the translation position through network port communication, and calculating a unit vector a on the three axes of x, y and z:
due to two adjacent connecting rods T i And T is i-1 The transformation relation of (2) is:
in which a is i Representing the included angle between the distance between the common normal lines and two axes perpendicular to the plane; d, d i Representing the relative positions of the two connecting rods; θ i Indicating the relative position d of the two links i An included angle with a common vertical line of the two connecting rods; c represents a cos () function in the trigonometric function; s represents a sin () function in the trigonometric function;
the seven-degree-of-freedom mechanical arm assembly is moved for multiple times, and rotation and translation amounts of multiple groups of end effectors relative to the base coordinates are obtained; the method comprises the steps of recording a plurality of sets of rotation and translation parameters of an end effector relative to basic coordinates as a plurality of sets of gesture matrixes B, regarding a transformation matrix T from the space coordinates of all angular points on a plurality of sets of calibration plate checkerboards to a camera coordinate system as an external parameter matrix A, and obtainingGroup A, B, the rotation and translation transformation from the camera coordinate system to the seven-degree-of-freedom mechanical arm component end coordinate system is calculated by using ax=xb, thereby obtaining a transformation matrix T from the camera coordinate system to the seven-degree-of-freedom mechanical arm component end coordinate system e ；
Finally, moving the seven-degree-of-freedom mechanical arm assembly, photographing the picking target by using a depth camera, and obtaining the space coordinates of the picking target by using a conversion formula between a pixel coordinate system and a world coordinate system; reuse of transformation matrix T e And calibrating a transformation matrix T from the space coordinates of all the angular points on the checkerboard of the plate to a camera coordinate system to obtain the pose of the picking target relative to the end effector coordinate system.
9. A method of picking by a seven degree of freedom fruit picking robot of claim 7 wherein: in order to optimize the path with less unnecessary turns, the path planning algorithm is used for searching the tree setT ree Any one of the nodes in (a) takes two points Q i And Q is equal to j Wherein i, j e [1,2,3, …, n]The method comprises the steps of carrying out a first treatment on the surface of the For Q i And Q is equal to j Collision detection is performed on the paths between the paths, and if no collision exists, Q is deleted i And Q is equal to j All nodes in between.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310493761.4A CN116439018B (en) | 2023-05-05 | 2023-05-05 | Seven-degree-of-freedom fruit picking robot and picking method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310493761.4A CN116439018B (en) | 2023-05-05 | 2023-05-05 | Seven-degree-of-freedom fruit picking robot and picking method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116439018A true CN116439018A (en) | 2023-07-18 |
| CN116439018B CN116439018B (en) | 2024-01-02 |
Family
ID=87128590
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310493761.4A Active CN116439018B (en) | 2023-05-05 | 2023-05-05 | Seven-degree-of-freedom fruit picking robot and picking method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116439018B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20220081226A1 (en) * | 2020-09-14 | 2022-03-17 | Yamaha Hatsudoki Kabushiki Kaisha | Movable harvesting apparatus and harvesting unit |
| CN117765085A (en) * | 2024-02-22 | 2024-03-26 | 华南农业大学 | tea bud leaf picking far and near alternate positioning method |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN205305450U (en) * | 2016-01-13 | 2016-06-15 | 浙江大学 | A end effector of robot for tomato bud picking |
| CN208305031U (en) * | 2018-05-02 | 2019-01-01 | 浙江工贸职业技术学院 | Fruit picking machine hand |
| CN109839110A (en) * | 2019-01-09 | 2019-06-04 | 浙江大学 | A kind of multiple target point path planning method based on quick random search tree |
| US20190261565A1 (en) * | 2016-11-08 | 2019-08-29 | Dogtooth Technologies Limited | Robotic fruit picking system |
| CN209755248U (en) * | 2018-11-28 | 2019-12-10 | 东莞理工学院 | End effector of picking manipulator |
| CN112132894A (en) * | 2020-09-08 | 2020-12-25 | 大连理工大学 | Mechanical arm real-time tracking method based on binocular vision guidance |
| CN113733088A (en) * | 2021-09-07 | 2021-12-03 | 河南大学 | Mechanical arm kinematics self-calibration method based on binocular vision |
| CN114067206A (en) * | 2021-11-16 | 2022-02-18 | 哈尔滨理工大学 | Spherical fruit identification and positioning method based on depth image |
| CN114260895A (en) * | 2021-12-22 | 2022-04-01 | 江苏大学 | Method and system for determining grabbing obstacle avoidance direction of mechanical arm of picking machine |
| WO2022212830A2 (en) * | 2021-04-02 | 2022-10-06 | Appharvest Technology, Inc. | Robotic fruit harvesting system |
| CN115250745A (en) * | 2022-08-05 | 2022-11-01 | 华南农业大学 | Full-automatic fruit picking robot and picking method based on vision technology |
| CN115958590A (en) * | 2021-12-24 | 2023-04-14 | 合肥哈工图南智控机器人有限公司 | RRT-based mechanical arm deep frame obstacle avoidance motion planning method and device |
-
2023
- 2023-05-05 CN CN202310493761.4A patent/CN116439018B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN205305450U (en) * | 2016-01-13 | 2016-06-15 | 浙江大学 | A end effector of robot for tomato bud picking |
| US20190261565A1 (en) * | 2016-11-08 | 2019-08-29 | Dogtooth Technologies Limited | Robotic fruit picking system |
| CN208305031U (en) * | 2018-05-02 | 2019-01-01 | 浙江工贸职业技术学院 | Fruit picking machine hand |
| CN209755248U (en) * | 2018-11-28 | 2019-12-10 | 东莞理工学院 | End effector of picking manipulator |
| CN109839110A (en) * | 2019-01-09 | 2019-06-04 | 浙江大学 | A kind of multiple target point path planning method based on quick random search tree |
| CN112132894A (en) * | 2020-09-08 | 2020-12-25 | 大连理工大学 | Mechanical arm real-time tracking method based on binocular vision guidance |
| WO2022212830A2 (en) * | 2021-04-02 | 2022-10-06 | Appharvest Technology, Inc. | Robotic fruit harvesting system |
| CN113733088A (en) * | 2021-09-07 | 2021-12-03 | 河南大学 | Mechanical arm kinematics self-calibration method based on binocular vision |
| CN114067206A (en) * | 2021-11-16 | 2022-02-18 | 哈尔滨理工大学 | Spherical fruit identification and positioning method based on depth image |
| CN114260895A (en) * | 2021-12-22 | 2022-04-01 | 江苏大学 | Method and system for determining grabbing obstacle avoidance direction of mechanical arm of picking machine |
| CN115958590A (en) * | 2021-12-24 | 2023-04-14 | 合肥哈工图南智控机器人有限公司 | RRT-based mechanical arm deep frame obstacle avoidance motion planning method and device |
| CN115250745A (en) * | 2022-08-05 | 2022-11-01 | 华南农业大学 | Full-automatic fruit picking robot and picking method based on vision technology |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20220081226A1 (en) * | 2020-09-14 | 2022-03-17 | Yamaha Hatsudoki Kabushiki Kaisha | Movable harvesting apparatus and harvesting unit |
| CN117765085A (en) * | 2024-02-22 | 2024-03-26 | 华南农业大学 | tea bud leaf picking far and near alternate positioning method |
| CN117765085B (en) * | 2024-02-22 | 2024-06-07 | 华南农业大学 | Tea bud leaf picking far and near alternate positioning method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116439018B (en) | 2024-01-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116439018B (en) | Seven-degree-of-freedom fruit picking robot and picking method thereof | |
| Li et al. | Detection of fruit-bearing branches and localization of litchi clusters for vision-based harvesting robots | |
| Ling et al. | Dual-arm cooperation and implementing for robotic harvesting tomato using binocular vision | |
| Lili et al. | Development of a tomato harvesting robot used in greenhouse | |
| Si et al. | Location of apples in trees using stereoscopic vision | |
| CN110728715B (en) | Intelligent inspection robot camera angle self-adaptive adjustment method | |
| Wang et al. | Localisation of litchi in an unstructured environment using binocular stereo vision | |
| Davidson et al. | Proof-of-concept of a robotic apple harvester | |
| CN101581575B (en) | Three-dimensional rebuilding method based on laser and camera data fusion | |
| Nielsen et al. | Vision-based 3D peach tree reconstruction for automated blossom thinning | |
| Yu et al. | A lab-customized autonomous humanoid apple harvesting robot | |
| CN114067309B (en) | Sweet pepper identification and picking sequence determination method based on multi-view three-dimensional reconstruction | |
| CN112529948A (en) | Mature pomegranate positioning method based on Mask R-CNN and 3-dimensional sphere fitting | |
| CN114260895A (en) | Method and system for determining grabbing obstacle avoidance direction of mechanical arm of picking machine | |
| Hu et al. | Simplified 4-DOF manipulator for rapid robotic apple harvesting | |
| CN109684941A (en) | One kind picking region partitioning method based on MATLAB image procossing litchi fruits | |
| CN114830915B (en) | Litchi vision picking robot based on laser radar navigation and implementation method thereof | |
| CN113812262A (en) | Tea-oil camellia fruit picking robot based on machine vision | |
| Paturkar et al. | Overview of image-based 3D vision systems for agricultural applications | |
| Li et al. | Multi-vision-based Localization and Pose Estimation of Occluded Apple Fruits for Harvesting Robots | |
| Peng et al. | Semantic mapping of orchards | |
| Mokrane et al. | UAV coverage path planning for supporting autonomous fruit counting systems | |
| Hu et al. | Design and experiment of a new citrus harvesting robot | |
| CN112372633B (en) | Orchard tree form type autonomous reasoning and scene understanding method of robot | |
| CN115810188A (en) | Method and system for identifying three-dimensional pose of fruit on tree based on single two-dimensional image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |