CN114571467A - Mechanical arm control method and system - Google Patents
Mechanical arm control method and system Download PDFInfo
- Publication number
- CN114571467A CN114571467A CN202210359518.9A CN202210359518A CN114571467A CN 114571467 A CN114571467 A CN 114571467A CN 202210359518 A CN202210359518 A CN 202210359518A CN 114571467 A CN114571467 A CN 114571467A
- Authority
- CN
- China
- Prior art keywords
- stacking
- grabbed
- grabbing
- target object
- area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Manipulator (AREA)
Abstract
The invention discloses a mechanical arm control method and a mechanical arm control system. The control method comprises the following steps: dividing the two-dimensional image of the grabbing operation area to obtain the category and the outline information of each object to be grabbed; dividing the point cloud of the grabbing operation area according to the outline information to obtain the point cloud of each object to be grabbed; discretizing point clouds of each object to be grabbed, fitting the obtained point clouds of each sub-point, and acquiring a large plane of the object to be grabbed based on a plurality of sub-planes obtained by fitting; determining an optimal grabbing large plane in all the obtained large planes, and further generating a grabbing control instruction; and generating a stacking control instruction according to the point cloud of the stacking operation area, the stacking position information of the target object and the grabbing position information of the target object. The control system comprises functional modules for correspondingly realizing the steps. According to the automatic stacking system, the problem that the existing mechanical arm automatic stacking system cannot grab and stack boxes which are randomly and disorderly placed and have different specifications can be solved.
Description
Technical Field
The invention belongs to the field of mechanical arm control, and particularly relates to a mechanical arm control method and a mechanical arm control system.
Background
The existing automatic mechanical arm stacking system mainly comprises a stacking mechanical arm and a control system, and the working mode is as follows: under the control of the control system, the stacking mechanical arm sequentially grabs the boxes of the same specification conveyed by the packing platform in sequence, and stacks the grabbed boxes to a preset placing area according to a preset stacking rule. However, in actual operation, a large number of boxes with different specifications to be transferred and stacked are often placed in the temporary placement area disorderly, and the existing automatic mechanical arm stacking system cannot cope with the situation due to the working mode of the automatic mechanical arm stacking system, that is, the boxes with different specifications which are placed in the temporary area disorderly cannot be grabbed and stacked.
Disclosure of Invention
The invention aims to solve the problem that the existing automatic mechanical arm stacking system cannot grab and stack boxes which are randomly and disorderly arranged and have different specifications.
In order to achieve the purpose, the invention provides a mechanical arm control method and a mechanical arm control system.
According to a first aspect of the present invention, there is provided a robot arm control method including the steps of:
segmenting a two-dimensional image of a pre-acquired grabbing operation area to obtain the category and contour information of each object to be grabbed;
dividing the pre-acquired point cloud of the grabbing operation area according to the contour information of the object to be grabbed to obtain the point cloud of each object to be grabbed;
discretizing the point cloud of each object to be grabbed to obtain a plurality of sub-point clouds,
fitting each sub-point cloud to obtain a plurality of sub-planes of the object to be grabbed,
performing similarity splicing operation on the plurality of sub-planes to obtain a large plane of the object to be grabbed;
determining an optimal grabbing large plane in the obtained large planes of all the objects to be grabbed, and generating a grabbing control instruction based on the optimal grabbing large plane so as to control the mechanical arm to grab a corresponding target object;
generating a stacking control instruction according to the point cloud of the pre-acquired stacking operation area, the pre-determined target object stacking position information and the pre-acquired target object grabbing position information so as to control the mechanical arm to stack the target object, wherein a determination factor of the target object stacking position information comprises the category of the target object;
after the current target object corresponding to the current optimal grabbing large plane is grabbed and stacked, the next optimal grabbing large plane is continuously determined again in the large planes of the rest objects to be grabbed, and a grabbing control instruction is generated based on the next optimal grabbing large plane so as to control the mechanical arm to grab the corresponding next target object; since the point cloud of the stacking work area changes after the current target object is stacked, the point cloud of the stacking work area needs to be acquired again before a next target object is stacked.
Optionally, the two-dimensional image of the grabbing operation area, the point cloud of the grabbing operation area, and the point cloud of the stacking operation area are obtained from a machine vision camera.
Optionally, the pre-acquired two-dimensional image of the grabbing operation area is segmented to obtain the category and the profile information of each object to be grabbed, which are realized based on a deep learning network;
the deep learning network includes:
the residual error network is used for extracting the characteristics of the two-dimensional image of the grabbing operation area to obtain a characteristic diagram of each object to be grabbed;
the characteristic pyramid network is used for generating a plurality of characteristic graphs with different resolutions of the object to be grabbed based on the characteristic graph of each object to be grabbed;
the region selection network is used for extracting regions of interest of a plurality of feature maps with different resolutions of each object to be grabbed to obtain a plurality of regions of interest;
and the convolution layer is used for performing convolution on a plurality of interested areas of each object to be grabbed to obtain the category and the outline information of the object to be grabbed.
Optionally, discretizing the point cloud of the object to be grabbed to obtain a plurality of sub-point clouds specifically comprises:
and carrying out equidistant segmentation on the point cloud of the object to be grabbed on an X axis, a Y axis and a Z axis.
Optionally, the fitting is performed on each sub-point cloud, and the obtaining of the plurality of sub-planes of the object to be grabbed is realized based on a least square method.
Optionally, the determining an optimal large gripping plane among the acquired large planes of all the objects to be gripped includes:
screening out all grippable large planes from the large planes of all the objects to be gripped, wherein the grippable large planes simultaneously meet the following conditions:
the mechanical arm does not collide with other objects when grabbing the object corresponding to the large plane,
when the mechanical arm grabs the large plane, each inclination angle of the mechanical arm is smaller than the corresponding inclination angle limit value;
and taking the largest area of all the screened graspable large planes as the optimal graspable large plane.
Optionally, the generating of the grabbing control instruction based on the optimal grabbing large plane includes:
determining the center of the optimal grabbing large plane;
determining a normal vector of the optimal grabbing large plane according to the center of the optimal grabbing large plane;
and generating the grabbing control instruction based on the normal vector of the optimal grabbing large plane and the current position information of the mechanical arm.
Optionally, the generating a stacking control instruction according to the pre-acquired point cloud of the stacking operation area, the pre-determined stacking position information of the target object, and the pre-acquired grabbing position information of the target object includes:
acquiring a plurality of mechanical arm movement paths according to the target object stacking position information and the target object grabbing position information;
determining an optimal mechanical arm motion path in the obtained multiple mechanical arm motion paths according to the point cloud of the stacking operation area;
and generating the stacking control instruction according to the optimal mechanical arm motion path.
Optionally, the method for acquiring the target object stacking position information includes:
dividing the stacking operation area into a corresponding number of stacking areas according to the number of the categories of the objects to be grabbed;
determining a stacking area of the target object according to the category of the target object;
determining whether the remaining space of the stacking area is sufficient to accommodate the target object,
determining, based on a predetermined stacking rule, stacking position information of the target object in the stacking area in response to a determination result that the remaining space of the stacking area is sufficient to accommodate the target object,
responding to a judgment result that the residual space of the stacking area is not enough to accommodate the target object, judging whether an empty stacking area exists or not, if so, determining the stacking position information of the target object in the empty stacking area according to the stacking rule, if not, selecting a standby stacking area from other stacking areas, and determining the stacking position information of the target object in the standby stacking area according to the stacking rule,
the spare stacking area is the stacking area with the largest residual space and the stacking object in the spare stacking area and the target object have a matching relation,
the matching relation is that under the combination of a preset number relation, the target object and the stacking object in the standby stacking area can form a rectangular body,
the stacking rule is that objects are placed in the stacking area from far to near and from high to low.
According to a second aspect of the present invention, there is provided a robot arm control system including the following functional modules:
the image segmentation module is used for segmenting a pre-acquired two-dimensional image of the grabbing operation area to obtain the category and contour information of each object to be grabbed;
the point cloud dividing module is used for dividing the point cloud of the pre-acquired grabbing operation area according to the outline information of the object to be grabbed to obtain the point cloud of each object to be grabbed;
a large plane acquisition module used for discretizing the point cloud of each object to be grabbed to obtain a plurality of sub point clouds,
fitting each sub-point cloud to obtain a plurality of sub-planes of the object to be grabbed,
performing similarity splicing operation on the plurality of sub-planes to obtain a large plane of the object to be grabbed;
the grabbing control instruction generating module is used for determining an optimal grabbing large plane in the obtained large planes of all the objects to be grabbed and generating grabbing control instructions based on the optimal grabbing large plane so as to control the mechanical arm to grab corresponding target objects;
and the stacking control instruction generating module is used for generating a stacking control instruction according to the pre-acquired point cloud of the stacking operation area, the pre-determined target object stacking position information and the pre-acquired target object grabbing position information so as to control the mechanical arm to stack the target object, wherein the determination factor of the target object stacking position information comprises the category of the target object.
The invention has the beneficial effects that:
the mechanical arm control method comprises the steps of segmenting a two-dimensional image of a pre-acquired grabbing operation area to obtain the category and contour information of each object to be grabbed; dividing the pre-acquired point clouds in the grabbing operation area according to the outline information of the object to be grabbed to obtain the point clouds of each object to be grabbed; for each object to be grabbed, discretizing point clouds of the object to be grabbed to obtain a plurality of sub-point clouds, fitting each sub-point cloud to obtain a plurality of sub-planes of the object to be grabbed, and performing similarity splicing operation on the plurality of sub-planes to obtain a large plane of the object to be grabbed; and determining an optimal grabbing large plane in the obtained large planes of all the objects to be grabbed, and generating a grabbing control instruction based on the optimal grabbing large plane so as to control the mechanical arm to grab the corresponding target object. According to the above contents, when the mechanical arm control method is adopted, the corresponding mechanical arm can select the best box body to be grabbed from the box bodies with different specifications which are randomly and disorderly placed in the temporary area for grabbing, and grab the rest box bodies to be grabbed in sequence according to the same mode until all the box bodies are grabbed.
According to the mechanical arm control method, a stacking control instruction is generated according to the pre-acquired point cloud of the stacking operation area, the pre-determined stacking position information of the target object and the pre-acquired grabbing position information of the target object, so that the mechanical arm is controlled to stack the target object. According to the above content, when the mechanical arm control method is adopted, the corresponding mechanical arm can stack the grabbed optimal box bodies to be grabbed to the stacking area, and stack the rest box bodies to the stacking area in sequence according to the same mode.
Therefore, when the mechanical arm control method is applied, the corresponding mechanical arm automatic stacking system can grab and stack boxes which are randomly and disorderly placed and have different specifications.
The robot arm control system and the robot arm control method of the present invention belong to a general inventive concept, and thus have the same beneficial effects as the robot arm control method, and are not described herein again.
Additional features and advantages of the invention will be set forth in the detailed description which follows.
Drawings
The above and other objects, features and advantages of the present invention will become more apparent by describing in more detail exemplary embodiments thereof with reference to the attached drawings, in which like reference numerals generally represent like parts throughout.
FIG. 1 illustrates a flow chart for implementing a robot arm control method according to an embodiment of the present invention;
FIG. 2 shows a schematic structural diagram of a deep learning network according to an embodiment of the invention;
FIG. 3 is a flow chart illustrating an implementation of a method for generating a placement control instruction according to an embodiment of the present invention;
FIG. 4 illustrates a functional block diagram of a robotic arm control system according to an embodiment of the present invention.
Detailed Description
Preferred embodiments of the present invention will be described in more detail below. While the following describes preferred embodiments of the present invention, it should be understood that the present invention may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
Example (b): fig. 1 shows a flowchart of an implementation of a robot arm control method according to an embodiment of the present invention.
Referring to fig. 1, a robot arm control method according to an embodiment of the present invention includes the steps of:
s100, segmenting a pre-acquired two-dimensional image of a grabbing operation area to obtain the category and contour information of each object to be grabbed;
s200, dividing the pre-acquired point clouds in the grabbing operation area according to the outline information of the object to be grabbed to obtain the point clouds of each object to be grabbed;
step S300, for each object to be grabbed, discretizing the point cloud of the object to be grabbed to obtain a plurality of sub-point clouds,
fitting each sub-point cloud to obtain a plurality of sub-planes of the object to be grabbed,
performing similarity splicing operation on the plurality of sub-planes to obtain a large plane of the object to be grabbed;
s400, determining an optimal grabbing large plane in the obtained large planes of all the objects to be grabbed, and generating a grabbing control instruction based on the optimal grabbing large plane so as to control a mechanical arm to grab a corresponding target object;
step S500, generating a stacking control instruction according to the pre-acquired point cloud of the stacking operation area, the pre-determined target object stacking position information and the pre-acquired target object grabbing position information so as to control the mechanical arm to stack the target object, wherein a determination factor of the target object stacking position information comprises the category of the target object.
Further, in the embodiment of the present invention, the two-dimensional image of the grabbing operation area, the point cloud of the grabbing operation area, and the point cloud of the stacking operation area are all obtained from a machine vision camera.
Specifically, in the embodiment of the invention, the machine vision camera adopts a DLP binocular stereo camera with high frame rate and high resolution, the DLP binocular stereo camera has extremely strong identification degree for the outlines of small pieces and stacked pieces, and the problems of reflection, light absorption, wrinkles, deformation and other types of package imaging can be effectively solved by combining a multiple exposure technology.
Still further, fig. 2 shows a schematic structural diagram of the deep learning network according to the embodiment of the present invention. Referring to fig. 2, in the embodiment of the present invention, the pre-acquired two-dimensional image of the grabbing operation area is segmented in step S100, and the category and the profile information of each object to be grabbed are obtained based on a deep learning network;
wherein the deep learning network comprises:
the residual error network is used for extracting the characteristics of the two-dimensional image of the grabbing operation area to obtain a characteristic diagram of each object to be grabbed;
the characteristic pyramid network is used for generating a plurality of characteristic graphs with different resolutions of the object to be grabbed based on the characteristic graph of each object to be grabbed;
the region selection network is used for extracting regions of interest of a plurality of feature maps with different resolutions of each object to be grabbed to obtain a plurality of regions of interest;
and the convolution layer is used for performing convolution on a plurality of interested areas of each object to be grabbed to obtain the category and the outline information of the object to be grabbed.
Specifically, in the embodiment of the present invention, the pre-acquired two-dimensional image of the grabbing work area is segmented in step S100, and the obtained category and contour information of each object to be grabbed is realized based on a deep learning segmentation algorithm, where the deep learning segmentation algorithm is based on a mask _ rcnn algorithm, a feature extraction part adopts a resnet50 structure, a fpn network is used to generate feature maps with different resolutions, a rpn structure is used to generate a region frame suggestion on the feature maps with different resolutions, the extracted roi is convolved, and finally the category, the smallest regular rectangle, and the boundary contour of each parcel are obtained. The deep learning segmentation algorithm is divided into a training part and an inference part, wrapping pictures under different scenes are collected at the early stage, the pictures are marked, then a mask _ rcnn algorithm is operated for training, and after the training is finished, the algorithm is deployed on an algorithm server
Specifically, in the embodiment of the present invention, the dividing of the pre-acquired point clouds in the grabbing work area according to the contour information of the object to be grabbed in step S200 to obtain an exemplary scene of the point clouds of each object to be grabbed is:
if 3 objects to be grabbed exist in the grabbing operation area, dividing the point cloud of the grabbing operation area into point cloud of the 3 objects to be grabbed and background point cloud.
Further, in S300 of the embodiment of the present invention, discretizing the point cloud of the object to be grabbed to obtain a plurality of sub-point clouds specifically includes:
and carrying out equidistant segmentation on the point cloud of the object to be grabbed on an X axis, a Y axis and a Z axis.
Further, in S300 of the embodiment of the present invention, the fitting is performed on each sub-point cloud to obtain a plurality of sub-planes of the object to be grabbed based on a least square method.
Specifically, in the embodiment of the invention, point cloud of an object to be grabbed is cut in a magic-like manner to obtain a plurality of rectangular spaces, each rectangular space is formed by point cloud data, a least square method is adopted to fit a plane AX B for the point cloud data of each rectangular space, and a least square method is adopted to solve coefficients of a plane equation to obtain a plurality of sub-planes of the objects to be grabbed.
Still further, in S400 according to the embodiment of the present invention, the determining an optimal large gripping plane among the acquired large planes of all the objects to be gripped includes the following steps:
screening out all grippable large planes from the large planes of all the objects to be gripped, wherein the grippable large planes simultaneously meet the following conditions:
the mechanical arm does not collide with other objects when grabbing the object corresponding to the large plane,
when the mechanical arm grabs the large plane, each inclination angle of the mechanical arm is smaller than the corresponding inclination angle limit value;
and taking the largest area of all the screened graspable large planes as the optimal graspable large plane.
Still further, in S400 in the embodiment of the present invention, the generating a fetch control instruction based on the optimal fetch large plane includes the following steps:
determining the center of the optimal grabbing large plane;
determining a normal vector of the optimal grabbing large plane according to the center of the optimal grabbing large plane;
and generating the grabbing control instruction based on the normal vector of the optimal grabbing large plane and the current position information of the mechanical arm.
Specifically, in the embodiment of the present invention, the normal vector of the optimal grabbing large plane is used to obtain the execution angle of each joint of the robot arm.
Still further, fig. 3 shows a flowchart of an implementation of the method for generating a code control instruction according to the embodiment of the present invention. Referring to fig. 3, in the embodiment of the present invention, the generating of the stacking control instruction according to the pre-acquired point cloud of the stacking work area, the pre-determined stacking position information of the target object, and the pre-acquired grabbing position information of the target object in step S500 includes the following steps:
step S510, acquiring a plurality of mechanical arm movement paths according to the target object stacking position information and the target object grabbing position information;
s520, determining an optimal mechanical arm motion path from the multiple mechanical arm motion paths according to the point clouds in the stacking operation area;
and S530, generating the stacking control instruction according to the optimal mechanical arm motion path.
Still further, in step S500 of the embodiment of the present invention, the method for acquiring the target object stacking position information includes the following steps:
dividing the stacking operation area into a corresponding number of stacking areas according to the number of the categories of the objects to be grabbed;
determining a stacking area of the target object according to the category of the target object;
determining whether the remaining space of the stacking area is sufficient to accommodate the target object,
determining, based on a predetermined stacking rule, stacking position information of the target object in the stacking area in response to a determination result that the remaining space of the stacking area is sufficient to accommodate the target object,
responding to a judgment result that the residual space of the stacking area is not enough to accommodate the target object, judging whether an empty stacking area exists or not, if so, determining the stacking position information of the target object in the empty stacking area according to the stacking rule, if not, selecting a standby stacking area from other stacking areas, and determining the stacking position information of the target object in the standby stacking area according to the stacking rule,
the spare stacking area is the stacking area with the largest residual space and the stacking object in the spare stacking area and the target object have a matching relationship,
the matching relation is that under the combination of a preset number relation, the target object and the stacking object in the standby stacking area can form a rectangular body,
the stacking rule is that objects are placed in the stacking area from far to near and from high to low.
Specifically, in the embodiment of the present invention, if the stacking operation area is small, the stacking operation area cannot be partitioned, that is, the stacking operation area is not partitioned into the corresponding number of stacking areas according to the number of categories of the objects to be grasped, and based on the principle that after stacking the target objects, the stacked objects are in a regular shape as much as possible on the whole, stacking position information of the target objects is determined, for example, the plurality of objects are stacked on one plane, which is convenient for subsequent packages to continue stacking; for packages with multiple length, width and height (the height of the box is reduced to one height), the packages are stacked on a plane.
Correspondingly, on the basis of the mechanical arm control method provided by the embodiment of the invention, the embodiment of the invention also provides a mechanical arm control system. FIG. 4 illustrates a functional block diagram of a robotic arm control system in accordance with an embodiment of the present invention. Referring to fig. 4, the robot arm control system of the embodiment of the present invention includes the following functional modules:
the image segmentation module is used for segmenting a pre-acquired two-dimensional image of the grabbing operation area to obtain the category and contour information of each object to be grabbed;
the point cloud dividing module is used for dividing the point cloud of the pre-acquired grabbing operation area according to the outline information of the object to be grabbed to obtain the point cloud of each object to be grabbed;
a large plane acquisition module used for discretizing the point cloud of each object to be grabbed to obtain a plurality of sub point clouds,
fitting each sub-point cloud to obtain a plurality of sub-planes of the object to be grabbed,
performing similarity splicing operation on the plurality of sub-planes to obtain a large plane of the object to be grabbed;
the grabbing control instruction generating module is used for determining an optimal grabbing large plane in the obtained large planes of all the objects to be grabbed and generating grabbing control instructions based on the optimal grabbing large plane so as to control the mechanical arm to grab the corresponding target object;
and the stacking control instruction generating module is used for generating a stacking control instruction according to the pre-acquired point cloud of the stacking operation area, the pre-determined target object stacking position information and the pre-acquired target object grabbing position information so as to control the mechanical arm to stack the target object, wherein the determination factor of the target object stacking position information comprises the category of the target object.
Having described embodiments of the present invention, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments.
Claims (10)
1. A robot arm control method is characterized by comprising:
segmenting a two-dimensional image of a pre-acquired grabbing operation area to obtain the category and contour information of each object to be grabbed;
dividing the pre-acquired point clouds in the grabbing operation area according to the outline information of the object to be grabbed to obtain the point clouds of each object to be grabbed;
discretizing the point cloud of each object to be grabbed to obtain a plurality of sub-point clouds,
fitting each sub-point cloud to obtain a plurality of sub-planes of the object to be grabbed,
performing similarity splicing operation on the plurality of sub-planes to obtain a large plane of the object to be grabbed;
determining an optimal grabbing large plane in the obtained large planes of all the objects to be grabbed, and generating a grabbing control instruction based on the optimal grabbing large plane so as to control the mechanical arm to grab a corresponding target object;
generating a stacking control instruction according to the pre-acquired point cloud of the stacking operation area, the pre-determined stacking position information of the target object and the pre-acquired grabbing position information of the target object so as to control the mechanical arm to stack the target object, wherein a determination factor of the stacking position information of the target object comprises the category of the target object.
2. The robot arm control method according to claim 1, wherein the grasping work area two-dimensional image, the grasping work area point cloud, and the piling work area point cloud are acquired from a machine vision camera.
3. The mechanical arm control method according to claim 1, wherein the segmentation is performed on the pre-acquired two-dimensional image of the grabbing operation area to obtain the category and contour information of each object to be grabbed based on a deep learning network;
the deep learning network includes:
the residual error network is used for extracting the characteristics of the two-dimensional image of the grabbing operation area to obtain a characteristic diagram of each object to be grabbed;
the characteristic pyramid network is used for generating a plurality of characteristic graphs with different resolutions of the object to be grabbed based on the characteristic graph of each object to be grabbed;
the region selection network is used for extracting regions of interest of a plurality of feature maps with different resolutions of each object to be grabbed to obtain a plurality of regions of interest;
and the convolution layer is used for performing convolution on a plurality of interested areas of each object to be grabbed to obtain the category and the outline information of the object to be grabbed.
4. The method for controlling a robot arm according to claim 1, wherein discretizing the point cloud of the object to be grabbed to obtain a plurality of sub-point clouds specifically comprises:
and carrying out equidistant segmentation on the point cloud of the object to be grabbed on an X axis, a Y axis and a Z axis.
5. The robot arm control method of claim 1, wherein the fitting of each sub-point cloud to obtain the plurality of sub-planes of the object to be grabbed is performed based on a least squares method.
6. The robot arm control method according to claim 1, wherein the determining an optimum gripping large plane among the acquired large planes of all the objects to be gripped includes:
screening out all grippable large planes from the large planes of all the objects to be gripped, wherein the grippable large planes simultaneously meet the following conditions:
the mechanical arm does not collide with other objects when grabbing the object corresponding to the large plane,
when the mechanical arm grabs the large plane, each inclination angle of the mechanical arm is smaller than the corresponding inclination angle limit value;
and taking the largest area of all the screened graspable large planes as the optimal graspable large plane.
7. The robot arm control method according to claim 1, wherein the generating of the gripping control instruction based on the optimal gripping large plane comprises:
determining the center of the optimal grabbing large plane;
determining a normal vector of the optimal grabbing large plane according to the center of the optimal grabbing large plane;
and generating the grabbing control instruction based on the normal vector of the optimal grabbing large plane and the current position information of the mechanical arm.
8. The robot arm control method according to any one of claims 1 to 7, wherein the generating a stacking control command according to the pre-acquired point cloud of the stacking work area, the pre-determined target object stacking position information, and the pre-acquired target object grabbing position information includes:
acquiring a plurality of mechanical arm movement paths according to the target object stacking position information and the target object grabbing position information;
determining an optimal mechanical arm motion path in the obtained multiple mechanical arm motion paths according to the point clouds in the stacking operation area;
and generating the stacking control instruction according to the optimal mechanical arm motion path.
9. The robot arm control method according to claim 8, wherein the target object stacking position information acquiring method comprises:
dividing the stacking operation area into a corresponding number of stacking areas according to the number of the categories of the objects to be grabbed;
determining a stacking area of the target object according to the category of the target object;
determining whether the remaining space of the stacking area is sufficient to accommodate the target object,
determining, based on a predetermined stacking rule, stacking position information of the target object in the stacking area in response to a determination result that the remaining space of the stacking area is sufficient to accommodate the target object,
responding to a judgment result that the residual space of the stacking area is not enough to accommodate the target object, judging whether an empty stacking area exists or not, if so, determining the stacking position information of the target object in the empty stacking area according to the stacking rule, if not, selecting a standby stacking area from other stacking areas, and determining the stacking position information of the target object in the standby stacking area according to the stacking rule,
the spare stacking area is the stacking area with the largest residual space and the stacking object in the spare stacking area and the target object have a matching relationship,
the matching relation is that under the combination of a preset number relation, the target object and the stacking object in the standby stacking area can form a rectangular body,
the stacking rule is that objects are placed in the stacking area from far to near and from high to low.
10. A robot arm control system, comprising:
the image segmentation module is used for segmenting a pre-acquired two-dimensional image of the grabbing operation area to obtain the category and contour information of each object to be grabbed;
the point cloud dividing module is used for dividing the point cloud of the pre-acquired grabbing operation area according to the outline information of the object to be grabbed to obtain the point cloud of each object to be grabbed;
a large plane acquisition module used for discretizing the point cloud of each object to be grabbed to obtain a plurality of sub point clouds,
fitting each sub-point cloud to obtain a plurality of sub-planes of the object to be grabbed,
performing similarity splicing operation on the plurality of sub-planes to obtain a large plane of the object to be grabbed;
the grabbing control instruction generating module is used for determining an optimal grabbing large plane in the obtained large planes of all the objects to be grabbed and generating grabbing control instructions based on the optimal grabbing large plane so as to control the mechanical arm to grab corresponding target objects;
and the stacking control instruction generating module is used for generating a stacking control instruction according to the pre-acquired point cloud of the stacking operation area, the pre-determined target object stacking position information and the pre-acquired target object grabbing position information so as to control the mechanical arm to stack the target object, wherein the determination factor of the target object stacking position information comprises the category of the target object.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210359518.9A CN114571467B (en) | 2022-04-07 | 2022-04-07 | Mechanical arm control method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210359518.9A CN114571467B (en) | 2022-04-07 | 2022-04-07 | Mechanical arm control method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114571467A true CN114571467A (en) | 2022-06-03 |
| CN114571467B CN114571467B (en) | 2022-09-13 |
Family
ID=81778658
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210359518.9A Active CN114571467B (en) | 2022-04-07 | 2022-04-07 | Mechanical arm control method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114571467B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116307573A (en) * | 2023-03-09 | 2023-06-23 | 中天建设集团有限公司 | Green energy-saving treatment method, equipment and medium for building site materials |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109176521A (en) * | 2018-09-19 | 2019-01-11 | 北京因时机器人科技有限公司 | A kind of mechanical arm and its crawl control method and system |
| CN110948492A (en) * | 2019-12-23 | 2020-04-03 | 浙江大学 | Three-dimensional grabbing platform and grabbing method based on deep learning |
| US20200206932A1 (en) * | 2017-09-21 | 2020-07-02 | Beijing Jingdong Shangke Information Technology Co., Ltd. | Palletizing control device, system and method and storage medium |
| CN112634181A (en) * | 2019-09-24 | 2021-04-09 | 北京百度网讯科技有限公司 | Method and apparatus for detecting ground point cloud points |
| CN113808201A (en) * | 2021-08-06 | 2021-12-17 | 亿嘉和科技股份有限公司 | Target object detection method and guided grabbing method |
-
2022
- 2022-04-07 CN CN202210359518.9A patent/CN114571467B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200206932A1 (en) * | 2017-09-21 | 2020-07-02 | Beijing Jingdong Shangke Information Technology Co., Ltd. | Palletizing control device, system and method and storage medium |
| CN109176521A (en) * | 2018-09-19 | 2019-01-11 | 北京因时机器人科技有限公司 | A kind of mechanical arm and its crawl control method and system |
| CN112634181A (en) * | 2019-09-24 | 2021-04-09 | 北京百度网讯科技有限公司 | Method and apparatus for detecting ground point cloud points |
| CN110948492A (en) * | 2019-12-23 | 2020-04-03 | 浙江大学 | Three-dimensional grabbing platform and grabbing method based on deep learning |
| CN113808201A (en) * | 2021-08-06 | 2021-12-17 | 亿嘉和科技股份有限公司 | Target object detection method and guided grabbing method |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116307573A (en) * | 2023-03-09 | 2023-06-23 | 中天建设集团有限公司 | Green energy-saving treatment method, equipment and medium for building site materials |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114571467B (en) | 2022-09-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP2019509559A (en) | Box location, separation, and picking using a sensor-guided robot | |
| JP7433609B2 (en) | Method and computational system for object identification | |
| JP5295828B2 (en) | Object gripping system and interference detection method in the system | |
| CN112070838B (en) | Object identification and positioning method and device based on two-dimensional-three-dimensional fusion characteristics | |
| CN110395515B (en) | Cargo identification and grabbing method and equipment and storage medium | |
| CN112802105A (en) | Object grabbing method and device | |
| CN112233181A (en) | 6D pose recognition method and device and computer storage medium | |
| US12017368B2 (en) | Mix-size depalletizing | |
| CN113284178B (en) | Object stacking method, device, computing equipment and computer storage medium | |
| CN109657518B (en) | Container laser scanning identification method and device, electronic equipment and readable medium | |
| US20220379475A1 (en) | Transparent object bin picking | |
| CN113610833A (en) | Material grabbing method and device, electronic equipment and storage medium | |
| CN115330819A (en) | Soft package segmentation positioning method, industrial personal computer and robot grabbing system | |
| JP5544464B2 (en) | 3D position / posture recognition apparatus and method for an object | |
| WO2023092519A1 (en) | Grabbing control method and apparatus, and electronic device and storage medium | |
| CN114571467B (en) | Mechanical arm control method and system | |
| CN116228854B (en) | Automatic parcel sorting method based on deep learning | |
| JPH08293029A (en) | Recognition device for state of piled box-shaped objects | |
| JP2023081310A (en) | Algorithm for mix-size depalletizing | |
| US11875528B2 (en) | Object bin picking with rotation compensation | |
| CN114653629A (en) | Sorting method based on visual identification, intelligent sorting system and readable storage medium | |
| CN111191083B (en) | Method and computing system for object identification | |
| CN113935368A (en) | Recognition, positioning and grabbing method for scattered stacking state and plane object capable of being grabbed | |
| CN112837370A (en) | Object stacking judgment method and device based on 3D bounding box and computing equipment | |
| US20240257505A1 (en) | Method for neural network adaptation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |