CN105856232A - Grabbing method and grabbing system for object - Google Patents

Grabbing method and grabbing system for object Download PDF

Info

Publication number
CN105856232A
CN105856232A CN201610370841.0A CN201610370841A CN105856232A CN 105856232 A CN105856232 A CN 105856232A CN 201610370841 A CN201610370841 A CN 201610370841A CN 105856232 A CN105856232 A CN 105856232A
Authority
CN
China
Prior art keywords
distance
point
right corner
left comer
corner point
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201610370841.0A
Other languages
Chinese (zh)
Inventor
曾旭心
刘志超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen city Serse Biological Technology Co. Ltd.
Original Assignee
Pioneer Intelligent Machinery (shenzhen) Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Pioneer Intelligent Machinery (shenzhen) Co Ltd filed Critical Pioneer Intelligent Machinery (shenzhen) Co Ltd
Priority to CN201610370841.0A priority Critical patent/CN105856232A/en
Publication of CN105856232A publication Critical patent/CN105856232A/en
Pending legal-status Critical Current

Links

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00Programme-controlled manipulators
    • B25J9/16Programme controls
    • B25J9/1679Programme controls characterised by the tasks executed
    • B25J9/1684Tracking a line or surface by means of sensors

Landscapes

  • Engineering & Computer Science (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Manipulator (AREA)

Abstract

The invention discloses a grabbing method and a grabbing system for an object. The grabbing method for the object comprises the steps of collecting the image information of the object and recognizing the left corner point and the right corner point of the object according to the image information; calculating the distance from the left corner point and the right corner point of the object to obstacles around the object; judging whether the distance is safe; if yes, controlling a manipulator to move to the surface, close to the left corner point or the right corner point, of the object and grab the object; and if no, controlling the manipulator to move to the surface, away from the left corner point or the right corner point, of the object and grab the object. According to the grabbing method for the object, by judging whether the distance from the left corner point and the right corner point of the object to the obstacles is safe and controlling the manipulator to grab the object correspondingly according to the judging results, the risk that the manipulator may touch the obstacles in the process of grabbing the object is lowered.

Description

The grasping means of object and grasping system
Technical field
The present invention relates to robot field, more specifically, relate to grasping means and the grasping system of a kind of object.
Background technology
In the related, the grasping means of object is mainly by calculating specified point, then basis on grasping body face The position of the specified point calculated, control mechanical hand captures toward the position of this specified point.
But, said method does not consider the safe distance of object and periphery barrier, result in during capturing It is likely to occur the dangerous situation of mechanical hand collision obstacle.
Summary of the invention
It is contemplated that at least solve one of technical problem present in prior art.To this end, the present invention needs to provide one Plant grasping means and the grasping system of object.
A kind of grasping means of object, including step:
Gather the image information of this object, and according to the left comer point of this image information this object of identification and right corner point;
Calculate this left comer point and this right corner point respectively with the distance of the barrier of this object periphery;
Judge whether this distance meets safe distance;
If so, wherein the body surface crawl at place should toward in this left comer point and this right corner point to control mechanical hand Object;
If it is not, control this mechanical hand toward away from the body surface crawl at wherein place in this left comer point and this right corner point This object.
Whether the grasping means of above-mentioned object is met by the distance of the left comer point of judgment object and right corner point with barrier Safe distance, and control the corresponding crawl of mechanical hand work according to judged result, decrease during the crawl of object, in fact it could happen that The risk of the dangerous situation of barrier encountered by mechanical hand.
In some embodiments, this left comer point of described calculating and this right corner point respectively with the barrier of this object periphery The step of distance, including: calculate this left comer point the first distance relative to this barrier on this left comer point side, and calculate this right side Angle point is relative to the second distance of this barrier on this right corner point side;
The described step judging whether this distance meets safe distance, including:
Judge whether this first distance and this second distance meet this safe distance;
If this first distance meets this safe distance and this this safe distance of second distance not met, control this mechanical hand past In this left comer point and this right corner point, wherein the body surface at place captures the step of this object, including:
Control this mechanical hand and capture this object toward the body surface near this left comer point place.
In some embodiments, this left comer point of described calculating and this right corner point respectively with the barrier of this object periphery The step of distance, including: calculate this left comer point the first distance relative to this barrier on this left comer point side, and calculate this right side Angle point is relative to the second distance of this barrier on this right corner point side;
The described step judging whether this distance meets safe distance, including:
Judge whether this first distance and this second distance meet this safe distance;
If this first distance and this second distance are satisfied by this safe distance and this first distance more than this second distance, control Make this mechanical hand toward in this left comer point and this right corner point wherein the body surface at place capture the step of this object, wrap Include:
Control this mechanical hand and capture this object toward the body surface near this left comer point place.
In some embodiments, this left comer point of described calculating and this right corner point respectively with the barrier of this object periphery The step of distance, including: calculate this left comer point the first distance relative to this barrier on this left comer point side, and calculate this right side Angle point is relative to the second distance of this barrier on this right corner point side;
The described step judging whether this distance meets safe distance, including:
Judge whether this first distance and this second distance meet this safe distance;
If this first distance this safe distance of not met and this second distance meet this safe distance, control this mechanical hand past In this left comer point and this right corner point, wherein the body surface at place captures the step of this object, including:
Control this mechanical hand and capture this object toward the body surface near this right corner point place.
In some embodiments, this left comer point of described calculating and this right corner point respectively with the barrier of this object periphery The step of distance, including: calculate this left comer point the first distance relative to this barrier on this left comer point side, and calculate this right side Angle point is relative to the second distance of this barrier on this right corner point side;
The described step judging whether this distance meets safe distance, including:
Judge whether this first distance and this second distance meet this safe distance;
If this first distance and this second distance are satisfied by this safe distance and this second distance more than this first distance, control Make this mechanical hand toward in this left comer point and this right corner point wherein the body surface at place capture the step of this object, wrap Include:
Control this mechanical hand and capture this object toward the body surface near this right corner point place.
In some embodiments, this left comer point of described calculating and this right corner point respectively with the barrier of this object periphery The step of distance, including: calculate this left comer point the first distance relative to this barrier on this left comer point side, and calculate this right side Angle point is relative to the second distance of this barrier on this right corner point side;
The described step judging whether this distance meets safe distance, including:
Judge whether this first distance and this second distance meet this safe distance;
If this first distance and this second distance this safe distance of equal not met, control this mechanical hand toward away from this left comer point With in this right corner point wherein the body surface at place capture the step of this object, including:
Control the body surface crawl toward the place, centre position between this left comer point and this right corner point of this mechanical hand This object.
In some embodiments, this mechanical hand of described control is toward wherein institute in this left comer point and this right corner point Body surface capture the step of this object, including:
Control this mechanical hand and capture three orthogonal objects at wherein place in this left comer point and this right corner point Surface.
In some embodiments, this mechanical hand of described control is toward away from wherein institute in this left comer point and this right corner point Body surface capture the step of this object, including:
Control this mechanical hand and capture two orthogonal objects at wherein place in this left comer point and this right corner point Surface.
The grasping system of a kind of object, including image collecting device, processing means and mechanical hand, this processing means connects should Image collecting device and this mechanical hand;This image collecting device is for gathering the image information of object;This processing means is used for root According to left comer point and the right corner point of this image information this object of identification, calculate this left comer point and this right corner point respectively with this object periphery The distance of barrier, and judge whether this distance meets safe distance;If so, to be used for controlling this mechanical hand past for this processing means In this left comer point and this right corner point, wherein the body surface at place captures this object;If it is not, this processing means is used for Control this mechanical hand toward away from this left comer point and this right corner point wherein the body surface at place capture this object.
Whether the grasping system of above-mentioned object is met by the distance of the left comer point of judgment object and right corner point with barrier Safe distance, and control the corresponding crawl of mechanical hand work according to judged result, decrease during the crawl of object, in fact it could happen that The risk of the dangerous situation of barrier encountered by mechanical hand.
In some embodiments, this processing means is used for:
Calculate this left comer point the first distance relative to this barrier on this left comer point side, and it is relative to calculate this right corner point Second distance in this barrier on this right corner point side;
Judge whether this first distance and this second distance meet this safe distance;
If this first distance meets this safe distance and this this safe distance of second distance not met, control this mechanical hand past Body surface near this left comer point place captures this object.
In some embodiments, this processing means is used for:
Calculate this left comer point the first distance relative to this barrier on this left comer point side, and it is relative to calculate this right corner point Second distance in this barrier on this right corner point side;
Judge whether this first distance and this second distance meet this safe distance;
If this first distance and this second distance are satisfied by this safe distance and this first distance more than this second distance, control Make this mechanical hand and capture this object toward the body surface near this left comer point place.
In some embodiments, this processing means is used for:
Calculate this left comer point the first distance relative to this barrier on this left comer point side, and it is relative to calculate this right corner point Second distance in this barrier on this right corner point side;
Judge whether this first distance and this second distance meet this safe distance;
If this first distance this safe distance of not met and this second distance meet this safe distance, control this mechanical hand past Body surface near this right corner point place captures this object.
In some embodiments, this processing means is used for:
Calculate this left comer point the first distance relative to this barrier on this left comer point side, and it is relative to calculate this right corner point Second distance in this barrier on this right corner point side;
Judge whether this first distance and this second distance meet this safe distance;
If this first distance and this second distance are satisfied by this safe distance and this second distance more than this first distance, control Make this mechanical hand and capture this object toward the body surface near this right corner point place.
In some embodiments, this processing means is used for:
Calculate this left comer point the first distance relative to this barrier on this left comer point side, and it is relative to calculate this right corner point Second distance in this barrier on this right corner point side;
Judge whether this first distance and this second distance meet this safe distance;
If this first distance and this second distance this safe distance of equal not met, control this mechanical hand toward being positioned at this left comer point And the body surface at the place, centre position between this right corner point captures this object.
In some embodiments, if this distance meets this safe distance, this processing means is used for controlling this gripper of manipulator Take three orthogonal body surfaces at wherein place in this left comer point and this right corner point.
In some embodiments, if this this safe distance of distance not met, this processing means is used for controlling this mechanical hand Capture two orthogonal body surfaces at wherein place in this left comer point and this right corner point.
The additional aspect of the present invention and advantage will part be given in the following description, and part will become from the following description Obtain substantially, or recognized by the practice of the present invention.
Accompanying drawing explanation
Above-mentioned and/or the additional aspect of the present invention and advantage are from combining the accompanying drawings below description to embodiment and will become Obtain substantially with easy to understand, wherein:
Fig. 1 is the schematic flow sheet of the grasping means of the object of embodiment of the present invention.
Fig. 2 be the object of embodiment of the present invention grasping means in object put schematic diagram.
Fig. 3 be the object of embodiment of the present invention grasping means in the profile schematic diagram of object.
Fig. 4 be the object of embodiment of the present invention grasping means in the point cloud chart of object as schematic diagram.
Fig. 5 is another schematic flow sheet of the grasping means of the object of embodiment of the present invention.
Fig. 6 is the another schematic flow sheet of the grasping means of the object of embodiment of the present invention.
Fig. 7 is another schematic flow sheet of the grasping means of the object of embodiment of the present invention.
Fig. 8 is another schematic flow sheet again of the grasping means of the object of embodiment of the present invention.
Fig. 9 is another again schematic flow sheet of the grasping means of the object of embodiment of the present invention.
Figure 10 is the module diagram of the grasping system of the object of embodiment of the present invention.
Detailed description of the invention
Embodiments of the present invention are described below in detail, and the example of this embodiment is shown in the drawings, the most extremely Same or similar label represents same or similar element or has the element of same or like function eventually.Below with reference to The embodiment that accompanying drawing describes is exemplary, is only used for explaining the present invention, and is not considered as limiting the invention.
In describing the invention, it is to be understood that term " " center ", " longitudinally ", " laterally ", " length ", " width ", " thickness ", " on ", D score, "front", "rear", "left", "right", " vertically ", " level ", " top ", " end ", " interior ", " outward ", " up time Pin ", the orientation of the instruction such as " counterclockwise " or position relationship be based on orientation shown in the drawings or position relationship, be for only for ease of Describe the present invention and simplification description rather than instruction or the device of hint indication or element must have specific orientation, Yi Te Fixed azimuth configuration and operation, be therefore not considered as limiting the invention.Additionally, term " first ", " second " are only used for Purpose is described, and it is not intended that indicate or imply relative importance or the implicit quantity indicating indicated technical characteristic. Thus, define " first ", the feature of " second " can express or implicitly include one or more this feature.At this In the description of invention, " multiple " are meant that two or more, unless otherwise expressly limited specifically.
In describing the invention, it should be noted that unless otherwise clearly defined and limited, term " is installed ", " phase Even ", " connection " should be interpreted broadly, for example, it may be fixing connection, it is also possible to be to removably connect, or be integrally connected;Can Be mechanically connected, it is also possible to be electrical connection or can mutually communication;Can be to be joined directly together, it is also possible to by between intermediary Connect connected, can be connection or the interaction relationship of two elements of two element internals.Ordinary skill for this area For personnel, above-mentioned term concrete meaning in the present invention can be understood as the case may be.
In the present invention, unless otherwise clearly defined and limited, fisrt feature second feature it " on " or D score Can include that the first and second features directly contact, it is also possible to include that the first and second features are not directly contact but by it Other characterisation contact between.And, fisrt feature second feature " on ", " top " and " above " include that first is special Levy directly over second feature and oblique upper, or be merely representative of fisrt feature level height higher than second feature.Fisrt feature exists Second feature " under ", " lower section " and " below " include that fisrt feature directly over second feature and oblique upper, or is merely representative of Fisrt feature level height is less than second feature.
Following disclosure provides many different embodiments or example for realizing the different structure of the present invention.In order to Simplifying disclosure of the invention, hereinafter parts and setting to specific examples are described.Certainly, they are the most merely illustrative, and And be not intended to limit the present invention.Additionally, the present invention can in different examples repeat reference numerals and/or reference letter, This repetition is for purposes of simplicity and clarity, between itself not indicating discussed various embodiment and/or arranging Relation.Additionally, the various specific technique that the invention provides and the example of material, but those of ordinary skill in the art are permissible Recognize the application of other techniques and/or the use of other materials.
Please join Fig. 1, the grasping means of a kind of object of embodiment of the present invention, including step:
S11, gathers the image information of object, and according to the left comer point of image information identification object and right corner point;
S12, calculate left comer point and right corner point respectively with the distance of the barrier of object periphery;
S13, it is judged that whether distance meets safe distance;
If so, S14, controls the mechanical hand body surface crawl thing toward wherein place in left comer point and right corner point Body;
If it is not, S15, control mechanical hand toward away from the body surface crawl thing at wherein place in left comer point and right corner point Body.
Therefore, by the distance of the left comer point of judgment object and right corner point and barrier whether the grasping means of above-mentioned object Meet safe distance, and control the corresponding crawl of mechanical hand work according to judged result, decrease during the crawl of object, may Occur that the risk of the dangerous situation of barrier encountered by mechanical hand.
Specifically, the grasping means of the object of the present invention is applicable to capture cuboid etc. such as and has the object of marginal point.
In some embodiments, collect the image information of object after treatment, the depth image letter of available object Breath and some cloud information.Deep image information can be used for identifying left comer point and the right corner point of object.Point cloud information can be used for calculating a left side The distance of angle point and barrier and calculate the distance of right corner point and barrier, such as, can obtain left comer from the some cloud information of object Point, right corner point and the three-dimensional coordinate of barrier, and then left comer point and the distance of barrier and right corner point and barrier can be calculated Distance.So can improve the precision of the grasping means of object.
More specifically, the image of the background not having object can first be gathered, gather image when having object the most again, then pass through Image processing method obtains the left comer point of object and the position of right corner point.Such as, please join Fig. 2, object 200 can be placed in cupboard In 300, first gather the image of empty cupboard 300 (not having the cupboard 300 during object 200), then gather in cupboard 300 and be placed on object The image of 200, then obtains the left comer point of object 200 and the position of right corner point.Fig. 4 is object obtained in example of the present invention The point cloud chart picture of 200, the point cloud chart picture of object 200 comprises the some cloud information of object 200.
In some instances, the inner surface of cupboard 300 can be considered as barrier, calculate left comer point and the cupboard of object 200 The distance on left inside surface, and calculate the right corner point of object 200 and the distance of right inner surface.
In some instances, available three-dimensional visual sensor gathers the image information of object 200.In some actual application In, three-dimensional visual sensor can use the Kinect device of available from Microsoft Corporation.
In some embodiments, collect the image information of object 200 after treatment, the some cloud of available object 200 Information, and the some cloud information of available object 200 identifies left comer point and the right corner point of object 200, and calculate left comer point and barrier Hinder the distance of thing, and the distance of right corner point and barrier.
It addition, left comer point meets safety with one the distance in the distance of barrier and the distance of right corner point and barrier Being considered as during distance meeting safe distance, two distances are all unsatisfactory for during safe distance being considered as being unsatisfactory for safe distance.Work as distance During more than or equal to safe distance, i.e. meet safe distance, otherwise, then it is unsatisfactory for safe distance.
In one example, safe distance can be set to 25 centimetres.
In embodiments of the present invention, available following methods identification left comer point and right corner point, and calculate left comer point and barrier Hinder the distance of thing, and calculate the distance of right corner point and barrier.
First, it may be determined that the profile central point of object 200:
After the image information gathering object 200, profile and the wheel of object 200 can be obtained by edge detection algorithm Wide spatial moment.Fig. 3 is the profile diagram of object 200 in example of the present invention, owing to shooting direction becomes a fixed inclination with object 200, Therefore the profile of object 200 is solid line, dotted line is the right corner point of object, is shown as dotted line in profile diagram.
The formula of spatial moment is:
m j i = Σ x , y ( a r r a y ( x , y ) · x j · y i ) .
Above-mentioned formula is profile space square formula in opencv, and wherein x, y, i, j are that the coordinate of region point is (in digital picture In pixel coordinate), in formula, j is the square on corresponding x latitude, and i is the square on corresponding y-dimension, and exponent number represents corresponding portion The index divided, xjAnd yiRepresentative image pixel ad-hoc location.
Xc, Yc is made to represent the coordinate of regional barycenter, then:
Xc=M10/M00;
Yc=M01/M00;
Wherein, for the image of 01 binaryzation, M00 is the area (A is number a little) of profile.M10 and M01 represents right The integration of point.
Object overall weight distribution ratio owing to detecting is more uniform, so the center of gravity found and the equation of the ecentre are away from not quite, because of This can be simply considered that the center of gravity of image is exactly the central point of object 200 needing to find.
Then, the left and right angle point of central point the found object of judgement as a reference point 200 of object 200 is utilized.So energy The accuracy of left and right angle point is found in enough raisings, and method is simple.
Determine left and right angle point:
Usually, in the illustrated example shown in fig. 2, because gather image photographic head be placed in capture area right side and with The attitude shooting object 200 overlooked, thus this allow for obtaining relatively easily before target object 200, end face and right side Face.
And due to the relation at photographic head visual angle, the left and right corner of target object 200 can be obtained by simple geometrical relationship Point.
Left comer point:
By traveling through all data of object 200 profile, X, Y (the pixel coordinate of two dimensional image) is found to be minimum Point, can be simply considered that here it is the left comer point of target object 200;
Right corner point:
Determined the scope at target object 200 place by the profile information of object 200, in the range of find distance shooting Recently and point similar to left comer point in Y-direction, point may be considered right corner point to head.
Target object 200 and photographic head distance:
Target object 200 can be directly obtained by the some cloud information reading object 200 with the distance of photographic head.
In some actual application, Kinect device includes the photographic head of above-mentioned collection image information.
In some embodiments, Fig. 5 please be join, described calculating left comer point and right corner point respectively with the barrier of object periphery The step of distance, including S21, calculate the left comer point the first distance relative to the barrier on left comer point side, and calculate right corner Point is relative to the second distance of the barrier on right corner point side.
Whether described judging distance meets the step of safe distance, including:
S22, it is judged that whether the first distance and second distance meet safe distance;
If the first distance meets safe distance and second distance not met safe distance, control mechanical hand toward near left comer point With in right corner point wherein place body surface capture object step, including:
S23, controls mechanical hand and captures object toward the body surface near left comer point place.
Therefore, mechanical hand captures object toward the body surface near left comer point place, decreases the crawl process at object In, in fact it could happen that the risk of the dangerous situation of barrier encountered by mechanical hand.
In some embodiments, Fig. 6 please be join, described calculating left comer point and right corner point respectively with the barrier of object periphery The step of distance, including S31, calculate the left comer point the first distance relative to the barrier on left comer point side, and calculate right corner Point is relative to the second distance of the barrier on right corner point side;
Whether described judging distance meets the step of safe distance, including:
S32, it is judged that whether the first distance and second distance meet safe distance;
If the first distance and second distance are satisfied by safe distance and the first distance more than second distance, control mechanical hand past In left comer point and right corner point, wherein the body surface at place captures the step of object, including:
S33, controls mechanical hand and captures object toward the body surface near left comer point place.
Specifically, it is satisfied by safe distance and the first distance situation more than second distance in the first distance and second distance Under, the surface crawl object at control mechanical hand angle point place to the left, for right corner point, more can reduce in grasping body mistake Journey occurring, the risk of the dangerous situation of barrier encountered by mechanical hand.
In some embodiments, Fig. 7 please be join, described calculating left comer point and right corner point respectively with the barrier of object periphery The step of distance, including S41, calculate the left comer point the first distance relative to the barrier on left comer point side, and calculate right corner Point is relative to the second distance of the barrier on right corner point side.
Whether described judging distance meets the step of safe distance, including:
S42, it is judged that whether the first distance and second distance meet safe distance;
If the first distance not met safe distance and second distance meet safe distance, control mechanical hand toward near left comer point With in right corner point wherein place body surface capture object step, including:
S43, controls mechanical hand and captures object toward the body surface near right corner point place.
Therefore, mechanical hand captures object toward the body surface near right corner point place, decreases the crawl process at object In, in fact it could happen that the risk of the dangerous situation of barrier encountered by mechanical hand.
In some embodiments, Fig. 8 please be join, described calculating left comer point and right corner point respectively with the barrier of object periphery The step of distance, including S51, calculate the left comer point the first distance relative to the barrier on left comer point side, and calculate right corner Point is relative to the second distance of the barrier on right corner point side.
Whether described judging distance meets the step of safe distance, including:
S52, it is judged that whether the first distance and second distance meet safe distance;
If the first distance and second distance are satisfied by safe distance and second distance is more than the first distance, control mechanical hand past In left comer point and right corner point, wherein the body surface at place captures the step of object, including:
S53, controls mechanical hand and captures object toward the body surface near right corner point place.
Specifically, it is satisfied by safe distance and the second distance situation more than the first distance in the first distance and second distance Under, the surface crawl object at control mechanical hand angle point place to the right, for left comer point, more can reduce in grasping body mistake Journey occurring, the risk of the dangerous situation of barrier encountered by mechanical hand.
In some embodiments, Fig. 9 please be join, described calculating left comer point and right corner point respectively with the barrier of object periphery The step of distance, including S61, calculate the left comer point the first distance relative to the barrier on left comer point side, and calculate right corner Point is relative to the second distance of the barrier on right corner point side;
Whether described judging distance meets the step of safe distance, including:
S62, it is judged that whether the first distance and second distance meet safe distance;
If the first distance and second distance equal not met safe distance, control mechanical hand toward away from left comer point and right corner point Wherein the body surface at place captures the step of object, including:
S63, controls the mechanical hand body surface crawl thing toward the place, centre position between left comer point and right corner point Body.
Therefore, in the case of two distances are all unsatisfactory for safe distance, mechanical hand can be from the object at place, centre position Surface captures object, decreases during the crawl of object, in fact it could happen that the wind of the dangerous situation of barrier encountered by mechanical hand Danger.
In some embodiments, described control mechanical hand is toward the thing at wherein place in left comer point and right corner point Surface captures the step of object, including:
Control mechanical hand and capture three orthogonal body surfaces at wherein place in left comer point and right corner point.
Specifically, such as, Fig. 2 please be join, when the object needing the object 200 captured to be cuboid, left comer point place Three orthogonal body surfaces can be respectively left surface, above and end face, three orthogonal things at right corner point place Surface can be respectively right flank, above and end face.By capturing three orthogonal body surfaces at angle point place, can make Mechanical hand captures during object more firm.
Can be provided with grasping mechanism on the finger of mechanical hand, such as sucking disc mechanism, each grasping mechanism can capture the one of correspondence Individual body surface.
In some embodiments, described control mechanical hand is toward away from the thing at wherein place in left comer point and right corner point Surface captures the step of object, including:
Control mechanical hand and capture two orthogonal body surfaces at wherein place in left comer point and right corner point.
Specifically, such as, Fig. 2 please be join, when the object needing the object 200 captured to be cuboid, left comer point place Before two orthogonal body surfaces can be respectively and end face, two orthogonal body surfaces at right corner point place can Before Fen Bieweiing and end face.In the present example, two orthogonal body surfaces at left comer point place and right corner point institute Two orthogonal body surfaces be identical body surface.
By capturing two orthogonal body surfaces at angle point place, mechanical hand can be made to capture during object more firm. Can be provided with grasping mechanism on the finger of mechanical hand, such as sucking disc mechanism, each grasping mechanism can capture an object table of correspondence Face.
It is pointed out that when the quantity of the grasping mechanism of mechanical hand is more than the quantity of body surface to be crawled, Two or more grasping mechanism can be made to capture same body surface.Such as, in utilization, there are three grasping mechanisms When mechanical hand captures two orthogonal body surfaces, an available grasping mechanism captures the front of object, and utilizes two Individual grasping mechanism captures the end face of object.
In embodiments of the present invention, when controlling mechanical hand and capturing object, can be by photographic head and robot coordinate Conversion, mechanical hand goes to go one by one to capture object corresponding on shelf with fixed pose, and such as mechanical hand is with relative to object front 45 degree of angular direction capture objects.So solve when mechanical hand captures object and easily run into asking of shelf or other barrier Topic.Mechanical hand goes to capture object with fixed pose, can improve the success rate of crawl.
Please join Figure 10, the grasping system 100 of a kind of object of embodiment of the present invention, including image collecting device 102, place Reason device 104 and mechanical hand 106, processing means 104 connects image collecting device 102 and mechanical hand 106.
Image collecting device 102 is for gathering the image information of object 200.Processing means 104 is for according to image information Identify the left comer point of object 200 and right corner point, calculate left comer point and right corner point respectively with the barrier of object 200 periphery away from From, and whether judging distance meet safe distance.
If so, processing means 104 is for controlling mechanical hand 06 toward wherein place in close left comer point and right corner point Body surface captures object 200.If it is not, processing means 104 be used for controlling mechanical hand 106 toward away from left comer point and right corner point its In some places body surface capture object 200.
Therefore, the grasping system 100 of above-mentioned object by the left comer point of judgment object 200 and right corner point and barrier away from From whether meeting safe distance, and capture accordingly according to judged result control mechanical hand 106, decrease grabbing at object 200 During taking, in fact it could happen that the risk of the dangerous situation of barrier encountered by mechanical hand 106.
It should be noted that the above-mentioned explanation for the embodiment of the grasping means of object is also applied for this enforcement The grasping system 100 of the object of mode, for avoiding redundancy, is no longer developed in details at this.
In some embodiments, processing means 104 is used for:
Calculate the left comer point the first distance relative to the barrier on left comer point side, and calculate right corner point relative to right corner point The second distance of the barrier on side;
Judge whether the first distance and second distance meet safe distance;
If the first distance meets safe distance and second distance not met safe distance, control mechanical hand 106 toward near left The body surface at angle point place captures object 200.
It should be noted that the above-mentioned explanation for the embodiment of the grasping means of object is also applied for this enforcement The grasping system 100 of the object of mode, for avoiding redundancy, is no longer developed in details at this.
In some embodiments, processing means 104 is used for:
Calculate the left comer point the first distance relative to the barrier on left comer point side, and calculate right corner point relative to right corner point The second distance of the barrier on side;
Judge whether the first distance and second distance meet safe distance;
If the first distance and second distance are satisfied by safe distance and the first distance more than second distance, control mechanical hand 106 capture object 200 toward the body surface near left comer point place.
It should be noted that the above-mentioned explanation for the embodiment of the grasping means of object is also applied for this enforcement The grasping system 100 of the object of mode, for avoiding redundancy, is no longer developed in details at this.
In some embodiments, processing means 104 is used for:
Calculate the left comer point the first distance relative to the barrier on left comer point side, and calculate right corner point relative to right corner point The second distance of the barrier on side;
Judge whether the first distance and second distance meet safe distance;
If the first distance not met safe distance and second distance meet safe distance, control mechanical hand 106 toward near right The body surface at angle point place captures object 200.
It should be noted that the above-mentioned explanation for the embodiment of the grasping means of object is also applied for this enforcement The grasping system 100 of the object of mode, for avoiding redundancy, is no longer developed in details at this.
In some embodiments, processing means 104 is used for:
Calculate the left comer point the first distance relative to the barrier on left comer point side, and calculate right corner point relative to right corner point The second distance of the barrier on side;
Judge whether the first distance and second distance meet safe distance;
If the first distance and second distance are satisfied by safe distance and second distance more than the first distance, control mechanical hand 106 capture object 200 toward the body surface near right corner point place.
It should be noted that the above-mentioned explanation for the embodiment of the grasping means of object is also applied for this enforcement The grasping system 100 of the object of mode, for avoiding redundancy, is no longer developed in details at this.
In some embodiments, processing means 104 is used for:
Calculate the left comer point the first distance relative to the barrier on left comer point side, and calculate right corner point relative to right corner point The second distance of the barrier on side;
Judge whether the first distance and second distance meet safe distance;
If the first distance and second distance equal not met safe distance, control mechanical hand 106 toward being positioned at left comer point and right corner The body surface at the place, centre position between point captures object 200.
It should be noted that the above-mentioned explanation for the embodiment of the grasping means of object is also applied for this enforcement The grasping system 100 of the object of mode, for avoiding redundancy, is no longer developed in details at this.
In some embodiments, if distance meets safe distance, processing means 104 is used for controlling mechanical hand 106 and captures Three orthogonal body surfaces at wherein place in left comer point and right corner point.
It should be noted that the above-mentioned explanation for the embodiment of the grasping means of object is also applied for this enforcement The grasping system 100 of the object of mode, for avoiding redundancy, is no longer developed in details at this.
In some embodiments, if distance not met safe distance, processing means 104 is used for controlling mechanical hand 106 and grabs Take two orthogonal body surfaces at wherein place in left comer point and right corner point.
It should be noted that the above-mentioned explanation for the embodiment of the grasping means of object is also applied for this enforcement The grasping system 100 of the object of mode, for avoiding redundancy, is no longer developed in details at this.
In the description of this specification, reference term " embodiment ", " some embodiment ", " schematically implement Mode ", " example ", the description of " concrete example " or " some examples " etc. mean to combine described embodiment or example describes Specific features, structure, material or feature are contained at least one embodiment or the example of the present invention.In this specification In, the schematic representation of above-mentioned term is not necessarily referring to identical embodiment or example.And, the concrete spy of description Levy, structure, material or feature can combine in any one or more embodiments or example in an appropriate manner.
Additionally, term " first ", " second " are only used for describing purpose, and it is not intended that instruction or hint relative importance Or the implicit quantity indicating indicated technical characteristic.Thus, define " first ", the feature of " second " can express or Implicitly include at least one described feature.In describing the invention, " multiple " are meant that at least two, such as two, Three etc., unless otherwise expressly limited specifically.
In flow chart or at this, any process described otherwise above or method description are construed as, and expression includes One or more is for realizing the module of code, fragment or the portion of the executable instruction of the step of specific logical function or process Point, and the scope of the preferred embodiment of the present invention includes other realization, wherein can not by shown or discuss suitable Sequence, including according to involved function by basic mode simultaneously or in the opposite order, performs function, and this should be by the present invention Embodiment person of ordinary skill in the field understood.
Represent in flow charts or the logic described otherwise above at this and/or step, for example, it is possible to be considered as to use In the sequencing list of the executable instruction realizing logic function, may be embodied in any computer-readable medium, for Instruction execution system, device or equipment (system such as computer based system, including processor or other can hold from instruction Row system, device or equipment instruction fetch also perform the system instructed) use, or combine these instruction execution systems, device or set Standby and use.For the purpose of this specification, " computer-readable medium " can be any can to comprise, store, communicate, propagate or pass Defeated program is for instruction execution system, device or equipment or combines these instruction execution systems, device or equipment and the dress that uses Put.The more specifically example (non-exhaustive list) of computer-readable medium includes following: have the electricity of one or more wiring Connecting portion (mobile terminal), portable computer diskette box (magnetic device), random access memory (RAM), read only memory (ROM), erasable read only memory (EPROM or flash memory), the fiber device edited, and portable optic disk is read-only deposits Reservoir (CDROM).It addition, computer-readable medium can even is that and can print the paper of described program thereon or other are suitable Medium, because then can carry out editing, interpreting or if desired with it such as by paper or other media are carried out optical scanning His suitable method is processed to electronically obtain described program, is then stored in computer storage.
Should be appreciated that each several part of the present invention can realize by hardware, software, firmware or combinations thereof.Above-mentioned In embodiment, the software that multiple steps or method in memory and can be performed by suitable instruction execution system with storage Or firmware realizes.Such as, if realized with hardware, with the most the same, available well known in the art under Any one or their combination in row technology realize: have the logic gates for data signal realizes logic function Discrete logic, there is the special IC of suitable combination logic gate circuit, programmable gate array (PGA), on-the-spot Programmable gate array (FPGA) etc..
Those skilled in the art are appreciated that realize that above-mentioned embodiment method carries all or part of Step can be by program and completes to instruct relevant hardware, and described program can be stored in a kind of computer-readable storage In medium, described program upon execution, including one or a combination set of the step of method embodiment.
Additionally, each functional unit in each embodiment of the present invention can be integrated in a processing module, it is possible to Being that unit is individually physically present, it is also possible to two or more unit are integrated in a module.Above-mentioned integrated Module both can realize to use the form of hardware, it would however also be possible to employ the form of software function module realizes.Described integrated module If realized using the form of software function module and as independent production marketing or when using, it is also possible to be stored in a calculating In machine read/write memory medium.Storage medium mentioned above can be read only memory, disk or CD etc..
Although above it has been shown and described that embodiments of the present invention, it is to be understood that above-mentioned embodiment is Exemplary, it is impossible to being interpreted as limitation of the present invention, those of ordinary skill in the art within the scope of the invention can be right Above-mentioned embodiment is changed, revises, replaces and modification.

Claims (16)

1. the grasping means of an object, it is characterised in that include step:
Gather the image information of this object, and according to the left comer point of this image information this object of identification and right corner point;
Calculate this left comer point and this right corner point respectively with the distance of the barrier of this object periphery;
Judge whether this distance meets safe distance;
If so, control mechanical hand toward in this left comer point and this right corner point wherein the body surface at place capture this thing Body;
If it is not, control this mechanical hand toward away from this left comer point and this right corner point wherein the body surface at place capture this thing Body.
2. the grasping means of object as claimed in claim 1, it is characterised in that this left comer point of described calculating and this right corner point minute Not and the step of distance of barrier of this object periphery, including: calculate this left comer point this barrier relative to this left comer point side Hinder the first distance of thing, and calculate this right corner point second distance relative to this barrier on this right corner point side;
The described step judging whether this distance meets safe distance, including:
Judge whether this first distance and this second distance meet this safe distance;
If this first distance meets this safe distance and this this safe distance of second distance not met, control this mechanical hand toward close In this left comer point and this right corner point, wherein the body surface at place captures the step of this object, including:
Control this mechanical hand and capture this object toward the body surface near this left comer point place.
3. the grasping means of object as claimed in claim 1, it is characterised in that this left comer point of described calculating and this right corner point minute Not and the step of distance of barrier of this object periphery, including: calculate this left comer point this barrier relative to this left comer point side Hinder the first distance of thing, and calculate this right corner point second distance relative to this barrier on this right corner point side;
The described step judging whether this distance meets safe distance, including:
Judge whether this first distance and this second distance meet this safe distance;
It is somebody's turn to do more than this second distance, control if this first distance and this second distance are satisfied by this safe distance and this first distance Mechanical hand toward in this left comer point and this right corner point wherein the body surface at place capture the step of this object, including:
Control this mechanical hand and capture this object toward the body surface near this left comer point place.
4. the grasping means of object as claimed in claim 1, it is characterised in that this left comer point of described calculating and this right corner point minute Not and the step of distance of barrier of this object periphery, including: calculate this left comer point this barrier relative to this left comer point side Hinder the first distance of thing, and calculate this right corner point second distance relative to this barrier on this right corner point side;
The described step judging whether this distance meets safe distance, including:
Judge whether this first distance and this second distance meet this safe distance;
If this first distance this safe distance of not met and this second distance meet this safe distance, control this mechanical hand toward close In this left comer point and this right corner point, wherein the body surface at place captures the step of this object, including:
Control this mechanical hand and capture this object toward the body surface near this right corner point place.
5. the grasping means of object as claimed in claim 1, it is characterised in that this left comer point of described calculating and this right corner point minute Not and the step of distance of barrier of this object periphery, including: calculate this left comer point this barrier relative to this left comer point side Hinder the first distance of thing, and calculate this right corner point second distance relative to this barrier on this right corner point side;
The described step judging whether this distance meets safe distance, including:
Judge whether this first distance and this second distance meet this safe distance;
It is somebody's turn to do more than this first distance, control if this first distance and this second distance are satisfied by this safe distance and this second distance Mechanical hand toward in this left comer point and this right corner point wherein the body surface at place capture the step of this object, including:
Control this mechanical hand and capture this object toward the body surface near this right corner point place.
6. the grasping means of object as claimed in claim 1, it is characterised in that this left comer point of described calculating and this right corner point minute Not and the step of distance of barrier of this object periphery, including: calculate this left comer point this barrier relative to this left comer point side Hinder the first distance of thing, and calculate this right corner point second distance relative to this barrier on this right corner point side;
The described step judging whether this distance meets safe distance, including:
Judge whether this first distance and this second distance meet this safe distance;
If this first distance and this second distance this safe distance of equal not met, control this mechanical hand toward away from this left comer point and should In right corner point, wherein the body surface at place captures the step of this object, including:
Control this mechanical hand and capture this thing toward the body surface at the place, centre position between this left comer point and this right corner point Body.
7. the grasping means of object as claimed in claim 1, it is characterised in that this mechanical hand of described control is toward near this left comer In point and this right corner point, wherein the body surface at place captures the step of this object, including:
Control this mechanical hand and capture three orthogonal body surfaces at wherein place in this left comer point and this right corner point.
8. the grasping means of object as claimed in claim 1, it is characterised in that this mechanical hand of described control is toward away from this left comer In point and this right corner point, wherein the body surface at place captures the step of this object, including:
Control this mechanical hand and capture two orthogonal body surfaces at wherein place in this left comer point and this right corner point.
9. the grasping system of an object, it is characterised in that include image collecting device, processing means and mechanical hand, this process Device connects this image collecting device and this mechanical hand;
This image collecting device is for gathering the image information of object;
This processing means, for the left comer point according to this image information this object of identification and right corner point, calculates this left comer point and this right side Angle point respectively with the distance of the barrier of this object periphery, and judge whether this distance meets safe distance;
If so, this processing means is for controlling this mechanical hand thing toward wherein place in this left comer point and this right corner point Surface captures this object;
If it is not, this processing means is for controlling this mechanical hand toward the thing at wherein place in away from this left comer point and this right corner point Surface captures this object.
10. the grasping system of object as claimed in claim 9, it is characterised in that this processing means is used for:
Calculate this left comer point the first distance relative to this barrier on this left comer point side, and calculate this right corner point relative to this The second distance of this barrier on right corner point side;
Judge whether this first distance and this second distance meet this safe distance;
If this first distance meets this safe distance and this this safe distance of second distance not met, control this mechanical hand toward close The body surface at this left comer point place captures this object.
The grasping system of 11. objects as claimed in claim 9, it is characterised in that this processing means is used for:
Calculate this left comer point the first distance relative to this barrier on this left comer point side, and calculate this right corner point relative to this The second distance of this barrier on right corner point side;
Judge whether this first distance and this second distance meet this safe distance;
It is somebody's turn to do more than this second distance, control if this first distance and this second distance are satisfied by this safe distance and this first distance Mechanical hand captures this object toward the body surface near this left comer point place.
The grasping system of 12. objects as claimed in claim 9, it is characterised in that this processing means is used for:
Calculate this left comer point the first distance relative to this barrier on this left comer point side, and calculate this right corner point relative to this The second distance of this barrier on right corner point side;
Judge whether this first distance and this second distance meet this safe distance;
If this first distance this safe distance of not met and this second distance meet this safe distance, control this mechanical hand toward close The body surface at this right corner point place captures this object.
The grasping system of 13. objects as claimed in claim 9, it is characterised in that this processing means is used for:
Calculate this left comer point the first distance relative to this barrier on this left comer point side, and calculate this right corner point relative to this The second distance of this barrier on right corner point side;
Judge whether this first distance and this second distance meet this safe distance;
It is somebody's turn to do more than this first distance, control if this first distance and this second distance are satisfied by this safe distance and this second distance Mechanical hand captures this object toward the body surface near this right corner point place.
The grasping system of 14. objects as claimed in claim 9, it is characterised in that this processing means is used for:
Calculate this left comer point the first distance relative to this barrier on this left comer point side, and calculate this right corner point relative to this The second distance of this barrier on right corner point side;
Judge whether this first distance and this second distance meet this safe distance;
If this first distance and this second distance this safe distance of equal not met, control this mechanical hand toward being positioned at this left comer point and being somebody's turn to do The body surface at the place, centre position between right corner point captures this object.
The grasping system of 15. objects as claimed in claim 9, it is characterised in that if this distance meets this safe distance, at this Reason device captures three orthogonal things at wherein place in this left comer point and this right corner point for controlling this mechanical hand Surface.
The grasping system of 16. objects as claimed in claim 9, it is characterised in that if this this safe distance of distance not met, should Processing means for control during this mechanical hand captures this left comer point and this right corner point wherein two of place orthogonal Body surface.
CN201610370841.0A 2016-05-30 2016-05-30 Grabbing method and grabbing system for object Pending CN105856232A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610370841.0A CN105856232A (en) 2016-05-30 2016-05-30 Grabbing method and grabbing system for object

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610370841.0A CN105856232A (en) 2016-05-30 2016-05-30 Grabbing method and grabbing system for object

Publications (1)

Publication Number Publication Date
CN105856232A true CN105856232A (en) 2016-08-17

Family

ID=56642831

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610370841.0A Pending CN105856232A (en) 2016-05-30 2016-05-30 Grabbing method and grabbing system for object

Country Status (1)

Country Link
CN (1) CN105856232A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018076776A1 (en) * 2016-10-25 2018-05-03 深圳光启合众科技有限公司 Robot, robotic arm and control method and device thereof
WO2018161305A1 (en) * 2017-03-09 2018-09-13 深圳蓝胖子机器人有限公司 Grasp quality detection method, and method and system employing same
CN117656048A (en) * 2022-09-08 2024-03-08 无锡芯享信息科技有限公司 AGV grabbing point calibration equipment and calibration method thereof

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1256860A2 (en) * 2001-05-09 2002-11-13 Fanuc Ltd Device for avoiding collisions
EP1442848A2 (en) * 2003-01-30 2004-08-04 Fanuc Ltd Robot hand for taking out objects with means for changing the orientation of the hand
JP2007144546A (en) * 2005-11-28 2007-06-14 Kondo Seisakusho:Kk Gripping device for visual inspection
CN102773858A (en) * 2012-07-17 2012-11-14 北京航空航天大学 Obstacle avoidance method of robot palletizer
CN103170973A (en) * 2013-03-28 2013-06-26 上海理工大学 Man-machine cooperation device and method based on Kinect video camera
TW201529259A (en) * 2014-01-20 2015-08-01 Prec Machinery Res & Dev Ct End effector control method
CN105479461A (en) * 2016-01-26 2016-04-13 先驱智能机械(深圳)有限公司 Control method, control device and manipulator system

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1256860A2 (en) * 2001-05-09 2002-11-13 Fanuc Ltd Device for avoiding collisions
EP1442848A2 (en) * 2003-01-30 2004-08-04 Fanuc Ltd Robot hand for taking out objects with means for changing the orientation of the hand
JP2007144546A (en) * 2005-11-28 2007-06-14 Kondo Seisakusho:Kk Gripping device for visual inspection
CN102773858A (en) * 2012-07-17 2012-11-14 北京航空航天大学 Obstacle avoidance method of robot palletizer
CN103170973A (en) * 2013-03-28 2013-06-26 上海理工大学 Man-machine cooperation device and method based on Kinect video camera
TW201529259A (en) * 2014-01-20 2015-08-01 Prec Machinery Res & Dev Ct End effector control method
CN105479461A (en) * 2016-01-26 2016-04-13 先驱智能机械(深圳)有限公司 Control method, control device and manipulator system

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018076776A1 (en) * 2016-10-25 2018-05-03 深圳光启合众科技有限公司 Robot, robotic arm and control method and device thereof
WO2018161305A1 (en) * 2017-03-09 2018-09-13 深圳蓝胖子机器人有限公司 Grasp quality detection method, and method and system employing same
CN117656048A (en) * 2022-09-08 2024-03-08 无锡芯享信息科技有限公司 AGV grabbing point calibration equipment and calibration method thereof
CN117656048B (en) * 2022-09-08 2024-05-24 无锡芯享信息科技有限公司 AGV grabbing point calibration equipment and calibration method thereof

Similar Documents

Publication Publication Date Title
US9630320B1 (en) Detection and reconstruction of an environment to facilitate robotic interaction with the environment
JP6949107B2 (en) Systems and methods for training robots to drive autonomously on the route
CN106003043A (en) Obstacle avoidance method and obstacle avoidance system of mechanical arm
US9802317B1 (en) Methods and systems for remote perception assistance to facilitate robotic object manipulation
US9205562B1 (en) Integration of depth points into a height map
CN106104203B (en) A kind of distance detection method of mobile object, device and aircraft
US20230286750A1 (en) Method and device for storing goods, robot, warehousing system and storage medium
EP3104194B1 (en) Robot positioning system
EP3149862B1 (en) Mobile device, robot cleaner, and method for controlling the same
US9327406B1 (en) Object segmentation based on detected object-specific visual cues
KR102095817B1 (en) Mobile robot, charging apparatus for the mobile robot, and mobile robot system
JP3805302B2 (en) Work take-out device
CN205043784U (en) Multifunctional machine ware people that can independently remove
US7646917B2 (en) Method and apparatus for detecting corner
CN109917788B (en) Control method and device for robot to walk along wall
Janabi-Sharifi et al. Automatic selection of image features for visual servoing
CN113853279A (en) Robot configuration with three-dimensional lidar
CN104842358A (en) Autonomous mobile multifunctional robot
CN105247431A (en) Autonomous mobile body
WO2008036354A1 (en) System and method of determining object pose
JP6747665B2 (en) robot
CN105856232A (en) Grabbing method and grabbing system for object
WO2017008742A1 (en) Method and device for determining indoor approachable area
JP6328796B2 (en) Manipulator control method, system, and manipulator
CN110744559A (en) Laser automatic navigation library inventory robot

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C41 Transfer of patent application or patent right or utility model
TA01 Transfer of patent application right

Effective date of registration: 20170109

Address after: Futian District Huaqiang road Shenzhen City, Guangdong province 518028 No. 4002 Changsheng building room 1205

Applicant after: Shenzhen city Serse Biological Technology Co. Ltd.

Address before: Nanshan District Keyuan Road Shenzhen city Guangdong province 518000 No. 15 Kexing Science Park building B1 room 1700

Applicant before: Pioneer intelligent machinery (Shenzhen) Co., Ltd.

AD01 Patent right deemed abandoned

Effective date of abandoning: 20180928

AD01 Patent right deemed abandoned