CN109540022A - Close a position robot path planning and decision-making technique based on TOF depth camera - Google Patents
Close a position robot path planning and decision-making technique based on TOF depth camera Download PDFInfo
- Publication number
- CN109540022A CN109540022A CN201910004780.XA CN201910004780A CN109540022A CN 109540022 A CN109540022 A CN 109540022A CN 201910004780 A CN201910004780 A CN 201910004780A CN 109540022 A CN109540022 A CN 109540022A
- Authority
- CN
- China
- Prior art keywords
- point
- robot
- salient point
- moved
- depth camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/22—Measuring arrangements characterised by the use of optical techniques for measuring depth
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
- B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
- B65G69/00—Auxiliary measures taken, or devices used, in connection with loading or unloading
- B65G69/04—Spreading out the materials conveyed over the whole surface to be loaded; Trimming heaps of loose materials
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/20—Instruments for performing navigational calculations
- G01C21/206—Instruments for performing navigational calculations specially adapted for indoor navigation
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Mechanical Engineering (AREA)
- Manipulator (AREA)
- Numerical Control (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
The present invention relates to silo robot navigation's technical fields, a kind of close a position robot path planning and decision-making technique based on TOF depth camera is provided, the three-dimensional image information of grain heap regional area is obtained first, find all convex closures in regional area, and the highest salient point of height is found out as target point, corresponding convex closure is to smooth convex closure；Then 9 grids are formed centered on starting point, the cost for being moved to each grid central point around from starting point is calculated, selects the smallest point of cost as the point to be moved to, and with the point for new starting point, the secondary point being moved under searching, until forming local optimum path；Target point is moved to then along local optimum path treats smooth convex closure and carry out operation of closing a position, then aforesaid operations are carried out to next regional area, finish until entire silo is flattened.The present invention can have higher accuracy and adaptability in the complex environment of discontinuous landform, improve independence of the robot in operation of closing a position of closing a position.
Description
Technical field
The present invention relates to silo robot navigation's technical fields, more particularly to a kind of closing a position based on TOF depth camera
Robot path planning and decision-making technique.
Background technique
Grain reserves are related to national security and social stability, and the innovation and development of grain storage technology are concerned.Make
The last one processing links in conventional stored condition stage will be entered for conveying grain into storehouse, operation of closing a position is vital, because flat
Storehouse operation determines grain face flatness, and grain face flatness is directly related to ventilation, stifling and grain temperature observing and controlling during foodstuff preservation
Etc. grain storage technologies implementation result, and then influence grain in the safety of storage phase.
Currently, the primary operational mode for operation of closing a position is manual type, and by manually carry out operation cause to close a position speed it is slow,
The problems such as large labor intensity, grain are rotten, is not able to satisfy the needs for operation of closing a position, or even can bring seriously to the life security of people
It threatens.To which operation of closing a position needs robot manipulation, namely in the course of work of closing a position, and will be higher by plane by robot
Grain shifts the place lower than grain plane onto.Robot having had extensively in terms of patrolling storehouse, inspection storehouse, report in silo at present
Application, but be also not implemented closing a position in operation using the robot for capableing of utonomous working, the existing robot that closes a position all needs
Remote control is manually removed, time-consuming and laborious in this way and effect of closing a position is not satisfactory；In the silo environment of environment complexity, close a position
Robot is difficult accurately to reach target position.And the robot that closes a position accurately reaches the key of target position and is its path planning and determines
Plan method.The existing accuracy and adaptability of robot path planning and decision-making technique in discontinuous terrain environment of closing a position is equal
It is poor, so that independence of the robot in operation of closing a position of closing a position is not high enough.
Summary of the invention
In view of the problems of the existing technology, the present invention provides a kind of robot path of closing a position based on TOF depth camera
Planning and decision-making technique, can have higher accuracy and adaptability, raising is closed a position in the complex environment of discontinuous landform
Independence of the robot in operation of closing a position.
The technical solution of the present invention is as follows:
A kind of close a position robot path planning and decision-making technique based on TOF depth camera, which is characterized in that including following
Step:
Step 1: the TOF depth camera being installed to and is closed a position in robot, the robot that will close a position is put into silo, is opened
TOF depth camera and the robot that closes a position, establish three-dimensional cartesian coordinate system X-Y-Z；
Step 2: the grain heap in silo being identified by TOF depth camera, obtains the three-dimensional of the regional area of grain heap
The three-dimensional image information is sent to the robot that closes a position by image information, TOF depth camera；The regional area is TOF depth
The field range of camera；The three-dimensional image information includes the D coordinates value (x of each point P in regional areaP,yP,hP),
In, xP、yP、hPRespectively coordinate of the point P in X, Y, Z axis, hPIt is also the height of point P；
Step 3: the robot that closes a position handles the three-dimensional image information received, finds all convex in regional area
Packet and trench, the vertex of all convex closures constitute salient point collection and are combined into B={ B1,B2,B3,...,BM, wherein M is the partial zones of grain heap
The number of convex closure in domain；
Step 4: it is H={ h that the height value for extracting each point in salient point set, which constitutes height value set,1,h2,h3,...,
hM, find out the highest salient point B of height in salient point setmAs target point, salient point BmCorresponding convex closure is to smooth convex closure；
Step 5: using the point A where the robot that closes a position as starting point, planning from starting point A to salient point BmLocal optimum path,
Specific step is as follows:
Step 5.1: three-dimensional image information being further analyzed, the barrier region in regional area is identified and removes
Salient point BmExcept other range coverages, define classification function f (P), when point P be salient point BmWhen, f (P)=1；When point P is located at barrier
When hindering in object area, f (P)=- 1；Salient point B is removed when point X is located atmExcept other range coverages in when, f (P)=0；
Step 5.2: centered on starting point A, forming a square area, the square area is bisected into 9 long
Degree is the grid of n, so that the grid centered on starting point A is intermediate grid, to have center around intermediate grid be point Ai8 sides
Lattice；Wherein, i=1,2,3 ..., 8；
Step 5.3: if f (Ai) ≠ -1 then calculates from starting point A and is moved to point AiCost be F (i)=G (i)+H (i)；Its
In, G (i) is to be moved to point A from starting point AiCost, H (i) is from point AiIt is moved to point BmCost, G (i) starting point A and point
AiBetween manhatton distance measure, H (i) point AiWith point BmBetween manhatton distance measure, namelyIf f (Ai)=- 1, enables F (i)=+ ∞；
Step 5.4: the selection the smallest point A of costjAs this secondary point being moved to, F (j)=min { F (1), F
(2) ..., (8) F }；
Step 5.5: if point AjIt is not salient point Bm, then with point AjFor new starting point, repeat the above steps 5.2- step 5.4
Method determines the point to be moved to next time, until reaching salient point Bm；If point AjFor salient point Bm, then local optimum path is formed,
Enter step 5.6；
Step 5.6: the robot that closes a position is moved to salient point B according to local optimum pathm, treat smooth convex closure and carry out the behaviour that closes a position
Make；
Step 6: the method for the 2- step 5 that repeats the above steps carries out the rule in local optimum path to next regional area
Draw and operation of closing a position, finished until entire silo is flattened.
The step 4 includes the following steps:
Step 4.1: initialization hmax=h1, m=1, i=2；
Step 4.2: comparing hmaxWith hiIf: hmax>hi, then 4.3 are entered step；If hmax<hi, then h is enabledmax=hi, m=i,
Subsequently into step 4.3；
Step 4.3: if i=M, entering step 4.4；If i < M enables i=i+1, return step 4.2；
Step 4.4: salient point BmThe highest salient point of height as in salient point set.
The invention has the benefit that
The present invention uses TOF depth camera as sensor, and the input by the depth information of object as algorithm is realized
The accurate positionin of localized region highest convex closure and the optimal path that can select an arrival target convex closure, improve
The efficiency of work of closing a position being capable of answering in discontinuous landform compared to existing close a position robot path planning and decision-making technique
There is higher accuracy and adaptability, so that the robot that closes a position entirely autonomous can be run in operation of closing a position in heterocycle border.
Detailed description of the invention
Fig. 1 is the flow chart of close a position robot path planning and the decision-making technique of the invention based on TOF depth camera.
Specific embodiment
Below in conjunction with the drawings and specific embodiments, the invention will be further described.
The object of the present invention is to provide a kind of close a position robot path planning and decision-making technique based on TOF depth camera,
Can have higher accuracy and adaptability in the complex environment of discontinuous landform, raising closes a position robot in operation of closing a position
In independence.
As shown in Figure 1, being the stream of close a position robot path planning and the decision-making technique of the invention based on TOF depth camera
Cheng Tu.Close a position robot path planning and decision-making technique based on TOF depth camera of the invention, which is characterized in that including under
State step:
Step 1: the TOF depth camera being installed to and is closed a position in robot, the robot that will close a position is put into silo, is opened
TOF depth camera and the robot that closes a position, establish three-dimensional cartesian coordinate system X-Y-Z；
Step 2: the grain heap in silo being identified by TOF depth camera, obtains the three-dimensional of the regional area of grain heap
The three-dimensional image information is sent to the robot that closes a position by image information, TOF depth camera；The regional area is TOF depth
The field range of camera；The three-dimensional image information includes the D coordinates value (x of each point P in regional areaP,yP,hP),
In, xP、yP、hPRespectively coordinate of the point P in X, Y, Z axis, hPIt is also the height of point P；
Step 3: the robot that closes a position handles the three-dimensional image information received, finds all convex in regional area
Packet and trench, the vertex of all convex closures constitute salient point collection and are combined into B={ B1,B2,B3,…,BM, wherein M is the partial zones of grain heap
The number of convex closure in domain；
Step 4: it is H={ h that the height value for extracting each point in salient point set, which constitutes height value set,1,h2,h3,…,hM,
Find out the highest salient point B of height in salient point setmAs target point, salient point BmCorresponding convex closure is to smooth convex closure；
Step 5: using the point A where the robot that closes a position as starting point, planning from starting point A to salient point BmLocal optimum path,
Specific step is as follows:
Step 5.1: three-dimensional image information being further analyzed, the barrier region in regional area is identified and removes
Salient point BmExcept other range coverages, define classification function f (P), when point P be salient point BmWhen, f (P)=1；When point P is located at barrier
When hindering in object area, f (P)=- 1；Salient point B is removed when point X is located atmExcept other range coverages in when, f (P)=0；
Step 5.2: centered on starting point A, forming a square area, the square area is bisected into 9 long
Degree is the grid of n, so that the grid centered on starting point A is intermediate grid, to have center around intermediate grid be point Ai8 sides
Lattice；Wherein, i=1,2,3 ..., 8；
Step 5.3: if f (Ai) ≠ -1 then calculates from starting point A and is moved to point AiCost be F (i)=G (i)+H (i)；Its
In, G (i) is to be moved to point A from starting point AiCost, H (i) is from point AiIt is moved to point BmCost, G (i) starting point A and point
AiBetween manhatton distance measure, H (i) point AiWith point BmBetween manhatton distance measure, namelyIf f (Ai)=- 1, enables F (i)=+ ∞；
Step 5.4: the selection the smallest point A of costjAs this secondary point being moved to, F (j)=min { F (1), F
(2) ..., (8) F }；
Step 5.5: if point AjIt is not salient point Bm, then with point AjFor new starting point, repeat the above steps 5.2- step 5.4
Method determines the point to be moved to next time, until reaching salient point Bm；If point AjFor salient point Bm, then local optimum path is formed,
Enter step 5.6；
Step 5.6: the robot that closes a position is moved to salient point B according to local optimum pathm, treat smooth convex closure and carry out the behaviour that closes a position
Make；
Step 6: the method for the 2- step 5 that repeats the above steps carries out the rule in local optimum path to next regional area
Draw and operation of closing a position, finished until entire silo is flattened.
In the present embodiment, the step 4 includes the following steps:
Step 4.1: initialization hmax=h1, m=1, i=2；
Step 4.2: comparing hmaxWith hiIf: hmax>hi, then 4.3 are entered step；If hmax<hi, then h is enabledmax=hi, m=i,
Subsequently into step 4.3；
Step 4.3: if i=M, entering step 4.4；If i < M enables i=i+1, return step 4.2；
Step 4.4: salient point BmThe highest salient point of height as in salient point set.
Obviously, above-described embodiment is only a part of the embodiments of the present invention, instead of all the embodiments.Above-mentioned implementation
Example for explaining only the invention, is not intended to limit the scope of the present invention..Based on the above embodiment, those skilled in the art
Member's every other embodiment obtained namely all in spirit herein and original without making creative work
Made all modifications, equivalent replacement and improvement etc., are all fallen within the protection domain of application claims within reason.
Claims (2)
1. a kind of close a position robot path planning and decision-making technique based on TOF depth camera, which is characterized in that including following steps
It is rapid:
Step 1: the TOF depth camera being installed to and is closed a position in robot, the robot that will close a position is put into silo, opens TOF
Depth camera and the robot that closes a position, establish three-dimensional cartesian coordinate system X-Y-Z；
Step 2: the grain heap in silo being identified by TOF depth camera, obtains the 3-D image of the regional area of grain heap
The three-dimensional image information is sent to the robot that closes a position by information, TOF depth camera；The regional area is TOF depth camera
Field range；The three-dimensional image information includes the D coordinates value (x of each point P in regional areaP,yP,hP), wherein
xP、yP、hPRespectively coordinate of the point P in X, Y, Z axis, hPIt is also the height of point P；
Step 3: the robot that closes a position handles the three-dimensional image information received, find all convex closures in regional area and
Trench, the vertex of all convex closures constitute salient point collection and are combined into B={ B1,B2,B3,...,BM, wherein M is in the regional area of grain heap
The number of convex closure；
Step 4: it is H={ h that the height value for extracting each point in salient point set, which constitutes height value set,1,h2,h3,...,hM, it finds out
The highest salient point B of height in salient point setmAs target point, salient point BmCorresponding convex closure is to smooth convex closure；
Step 5: using the point A where the robot that closes a position as starting point, planning from starting point A to salient point BmLocal optimum path, it is specific to walk
It is rapid as follows:
Step 5.1: three-dimensional image information being further analyzed, the barrier region in regional area is identified and removes salient point
BmExcept other range coverages, define classification function f (P), when point P be salient point BmWhen, f (P)=1；When point P is located at barrier
When in region, f (P)=- 1；Salient point B is removed when point X is located atmExcept other range coverages in when, f (P)=0；
Step 5.2: centered on starting point A, forming a square area, it is n that the square area, which is bisected into 9 length,
Grid, so that the grid centered on starting point A is intermediate grid, to have center around intermediate grid be point Ai8 grids；Its
In, i=1,2,3 ..., 8；
Step 5.3: if f (Ai) ≠ -1 then calculates from starting point A and is moved to point AiCost be F (i)=G (i)+H (i)；Wherein, G
It (i) is to be moved to point A from starting point AiCost, H (i) is from point AiIt is moved to point BmCost, G (i) starting point A and point AiIt
Between manhatton distance measure, H (i) point AiWith point BmBetween manhatton distance measure, namelyIf f (Ai)=- 1, enables F (i)=+ ∞；
Step 5.4: the selection the smallest point A of costjAs this secondary point being moved to, F (j)=min { F (1), F (2) ..., F
(8)}；
Step 5.5: if point AjIt is not salient point Bm, then with point AjFor new starting point, the method for the 5.2- step 5.4 that repeats the above steps,
The point to be moved to next time is determined, until reaching salient point Bm；If point AjFor salient point Bm, then local optimum path is formed, into step
Rapid 5.6；
Step 5.6: the robot that closes a position is moved to salient point B according to local optimum pathm, treat smooth convex closure and carry out operation of closing a position；
Step 6: the method for the 2- step 5 that repeats the above steps, to next regional area carry out local optimum path planning with
And operation of closing a position, it is finished until entire silo is flattened.
2. close a position robot path planning and the decision-making technique according to claim 1 based on TOF depth camera, feature
It is, the step 4 includes the following steps:
Step 4.1: initialization hmax=h1, m=1, i=2；
Step 4.2: comparing hmaxWith hiIf: hmax>hi, then 4.3 are entered step；If hmax<hi, then h is enabledmax=hi, m=i, then into
Enter step 4.3；
Step 4.3: if i=M, entering step 4.4；If i < M enables i=i+1, return step 4.2；
Step 4.4: salient point BmThe highest salient point of height as in salient point set.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910004780.XA CN109540022B (en) | 2019-01-03 | 2019-01-03 | Method for planning and deciding path of flat-cabin robot based on TOF depth camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910004780.XA CN109540022B (en) | 2019-01-03 | 2019-01-03 | Method for planning and deciding path of flat-cabin robot based on TOF depth camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109540022A true CN109540022A (en) | 2019-03-29 |
| CN109540022B CN109540022B (en) | 2020-07-07 |
Family
ID=65834026
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910004780.XA Active CN109540022B (en) | 2019-01-03 | 2019-01-03 | Method for planning and deciding path of flat-cabin robot based on TOF depth camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109540022B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112947453A (en) * | 2021-02-24 | 2021-06-11 | 湖北文理学院 | Method for planning and adjusting running path of concrete leveling machine in real time |
| CN113860000A (en) * | 2021-10-21 | 2021-12-31 | 四川阿泰因机器人智能装备有限公司 | Intelligent variable-speed balanced grain throwing method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106054900A (en) * | 2016-08-08 | 2016-10-26 | 电子科技大学 | Temporary robot obstacle avoidance method based on depth camera |
| CN106289232A (en) * | 2016-07-24 | 2017-01-04 | 广东大仓机器人科技有限公司 | A kind of Obstacle Avoidance based on depth transducer |
| CN106855411A (en) * | 2017-01-10 | 2017-06-16 | 深圳市极思维智能科技有限公司 | A kind of robot and its method that map is built with depth camera and obstacle avoidance system |
| CN107092252A (en) * | 2017-04-11 | 2017-08-25 | 杭州光珀智能科技有限公司 | A kind of robot automatic obstacle avoidance method and its device based on machine vision |
| US20180165931A1 (en) * | 2016-12-14 | 2018-06-14 | Nanjing Avatarmind Robot Technology Co., Ltd. | Robot security inspection method based on environment map and robot thereof |
-
2019
- 2019-01-03 CN CN201910004780.XA patent/CN109540022B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106289232A (en) * | 2016-07-24 | 2017-01-04 | 广东大仓机器人科技有限公司 | A kind of Obstacle Avoidance based on depth transducer |
| CN106054900A (en) * | 2016-08-08 | 2016-10-26 | 电子科技大学 | Temporary robot obstacle avoidance method based on depth camera |
| US20180165931A1 (en) * | 2016-12-14 | 2018-06-14 | Nanjing Avatarmind Robot Technology Co., Ltd. | Robot security inspection method based on environment map and robot thereof |
| CN106855411A (en) * | 2017-01-10 | 2017-06-16 | 深圳市极思维智能科技有限公司 | A kind of robot and its method that map is built with depth camera and obstacle avoidance system |
| CN107092252A (en) * | 2017-04-11 | 2017-08-25 | 杭州光珀智能科技有限公司 | A kind of robot automatic obstacle avoidance method and its device based on machine vision |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112947453A (en) * | 2021-02-24 | 2021-06-11 | 湖北文理学院 | Method for planning and adjusting running path of concrete leveling machine in real time |
| CN112947453B (en) * | 2021-02-24 | 2024-04-30 | 湖北文理学院 | Method for planning and real-time adjusting running path of concrete warehouse leveling machine |
| CN113860000A (en) * | 2021-10-21 | 2021-12-31 | 四川阿泰因机器人智能装备有限公司 | Intelligent variable-speed balanced grain throwing method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109540022B (en) | 2020-07-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102357397B1 (en) | Method and system for planning path of user following robot with restrictions on direction of travel | |
| CN112327885B (en) | Unmanned ship self-adaptive global-local mixed path planning method | |
| Kim et al. | Perception-driven navigation: Active visual SLAM for robotic area coverage | |
| CN106548173B (en) | A kind of improvement no-manned plane three-dimensional information acquisition method based on classification matching strategy | |
| CN107990903A (en) | A kind of indoor AGV paths planning methods based on improvement A* algorithms | |
| Choi | Hybrid map-based SLAM using a Velodyne laser scanner | |
| CN110531770A (en) | One kind being based on improved RRT paths planning method and system | |
| CN109540022A (en) | Close a position robot path planning and decision-making technique based on TOF depth camera | |
| CN108571979A (en) | The method for covering triangle and spanning tree realization robot path planning | |
| CN109085569B (en) | Multi-radar track association method based on region division | |
| CN106500704A (en) | A kind of robot path planning method based on improved adaptive GA-IAGA | |
| CN103697895A (en) | Method for determining optimal path of flight vehicle based on self-adaptive A star algorithm | |
| CN109960269A (en) | A kind of simplification environmental modeling method for serving unmanned plane independent navigation | |
| CN107478231A (en) | Unmanned plane Route Planning Algorithm based on polygon obstacle detection | |
| Geng et al. | UAV surveillance mission planning with gimbaled sensors | |
| CN106708049B (en) | A kind of paths planning method of the lower movable body of multistation relay navigation | |
| Liu et al. | A prior information heuristic based robot exploration method in indoor environment | |
| CN110378417A (en) | A method of obtaining construction boundary | |
| Korb et al. | Exploring unstructured environment with frontier trees | |
| Qiu et al. | Improved F‐RRT∗ Algorithm for Flight‐Path Optimization in Hazardous Weather | |
| CN110146110A (en) | A kind of error hiding determination method of indoor environment robot line feature ICNN data correlation | |
| Li et al. | An enhanced direction calibration based on reinforcement learning for indoor localization system | |
| Savkin et al. | A method for decentralized self-deployment of a mobile sensor network with given regular geometric patterns | |
| Wang et al. | SRM: An efficient framework for autonomous robotic exploration in indoor environments | |
| CN107437093A (en) | A kind of initial cluster center optimum option method based on the distribution of RFID data strength time |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |