JP7055127B2 - 移動ロボット及びその制御方法 - Google Patents
移動ロボット及びその制御方法 Download PDFInfo
- Publication number
- JP7055127B2 JP7055127B2 JP2019510776A JP2019510776A JP7055127B2 JP 7055127 B2 JP7055127 B2 JP 7055127B2 JP 2019510776 A JP2019510776 A JP 2019510776A JP 2019510776 A JP2019510776 A JP 2019510776A JP 7055127 B2 JP7055127 B2 JP 7055127B2
- Authority
- JP
- Japan
- Prior art keywords
- obstacle
- mobile robot
- image
- time point
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims description 88
- 230000033001 locomotion Effects 0.000 claims description 50
- 238000001514 detection method Methods 0.000 claims description 44
- 238000010801 machine learning Methods 0.000 claims description 42
- 238000003860 storage Methods 0.000 claims description 34
- 230000008859 change Effects 0.000 claims description 19
- 239000000284 extract Substances 0.000 claims description 14
- 238000004891 communication Methods 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 11
- 230000001133 acceleration Effects 0.000 claims description 5
- 238000013528 artificial neural network Methods 0.000 description 49
- 238000013527 convolutional neural network Methods 0.000 description 34
- 238000004140 cleaning Methods 0.000 description 33
- 238000013135 deep learning Methods 0.000 description 28
- 230000008569 process Effects 0.000 description 27
- 238000010586 diagram Methods 0.000 description 13
- 210000004209 hair Anatomy 0.000 description 11
- 239000000428 dust Substances 0.000 description 10
- 238000000605 extraction Methods 0.000 description 9
- 230000003287 optical effect Effects 0.000 description 8
- 238000012549 training Methods 0.000 description 7
- 238000013473 artificial intelligence Methods 0.000 description 6
- 238000009826 distribution Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 230000002093 peripheral effect Effects 0.000 description 6
- 230000000306 recurrent effect Effects 0.000 description 5
- 230000004913 activation Effects 0.000 description 4
- 238000005070 sampling Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000001914 filtration Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000013500 data storage Methods 0.000 description 2
- 239000003550 marker Substances 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 241000282412 Homo Species 0.000 description 1
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- 238000010521 absorption reaction Methods 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 230000003925 brain function Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 239000011248 coating agent Substances 0.000 description 1
- 238000000576 coating method Methods 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 231100001261 hazardous Toxicity 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000003032 molecular docking Methods 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 230000001151 other effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000002187 spin decoupling employing ultra-broadband-inversion sequences generated via simulated annealing Methods 0.000 description 1
- 238000000547 structure data Methods 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
-
- A—HUMAN NECESSITIES
- A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
- A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
- A47L9/00—Details or accessories of suction cleaners, e.g. mechanical means for controlling the suction or for effecting pulsating action; Storing devices specially adapted to suction cleaners or parts thereof; Carrying-vehicles specially adapted for suction cleaners
- A47L9/28—Installation of the electric equipment, e.g. adaptation or attachment to the suction cleaner; Controlling suction cleaners by electric means
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J11/00—Manipulators not otherwise provided for
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J11/00—Manipulators not otherwise provided for
- B25J11/008—Manipulators for service tasks
- B25J11/0085—Cleaning
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
- B25J19/02—Sensing devices
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
- B25J19/02—Sensing devices
- B25J19/021—Optical sensing devices
- B25J19/023—Optical sensing devices including video camera means
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J5/00—Manipulators mounted on wheels or on carriages
- B25J5/007—Manipulators mounted on wheels or on carriages mounted on wheels
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1664—Programme controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
- B25J9/1666—Avoiding collision or forbidden zones
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
- B25J9/1697—Vision controlled systems
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S15/00—Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems
- G01S15/88—Sonar systems specially adapted for specific applications
- G01S15/93—Sonar systems specially adapted for specific applications for anti-collision purposes
- G01S15/931—Sonar systems specially adapted for specific applications for anti-collision purposes of land vehicles
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
- G05D1/0221—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving a learning process
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0234—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using optical markers or beacons
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0255—Control of position or course in two dimensions specially adapted to land vehicles using acoustic signals, e.g. ultra-sonic singals
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0268—Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means
- G05D1/0272—Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means comprising means for registering the travel distance, e.g. revolutions of wheels
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
- G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
- G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
- G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Mechanical Engineering (AREA)
- Robotics (AREA)
- Multimedia (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Automation & Control Theory (AREA)
- Aviation & Aerospace Engineering (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Electromagnetism (AREA)
- Data Mining & Analysis (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Acoustics & Sound (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Biodiversity & Conservation Biology (AREA)
- Databases & Information Systems (AREA)
- Mathematical Physics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Description
保存フォーマット:(Op, Rp)、
Claims (20)
- 移動ロボットであって
本体を移動させる走行部と、
前記本体の周辺を連続的に撮影して複数の画像を取得する画像取得部と、
前記画像取得部が取得する連続した複数の画像を保存する保存部と、
前記移動中に障害物を感知する1つ以上のセンサを含むセンサ部と、
前記センサ部が障害物を感知すると、前記本体の移動方向と移動速度に基づいて、前記連続した複数の画像のうち前記センサ部の障害物感知時点以前の特定時点画像を選択し、前記選択された特定時点画像に含まれる障害物の属性を認識する障害物認識モジュールを含む制御部と、を備えてなり、
前記センサ部が、障害物感知センサと、動作感知センサと、加速度センサと、を備えてなり、
前記障害物感知センサは、前記移動ロボットの移動中に障害物を感知して障害物情報を出力するものであり、
前記動作感知センサは、前記移動ロボットの動作を感知して動作情報を出力するものであり、
前記加速度センサは、前記移動ロボットの速度変化を感知して速度変化による衝撃量を検出しそれに対応する電圧値を出力するものであり、
前記制御部は、前記動作感知センサから出力された動作情報に基づいて、画像情報を用いた絶対位置に対応した相対位置として、前記移動ロボットの位置変化を算出するものである、移動ロボット。 - 前記制御部は、
前記認識された障害物の属性に基づいて、前記走行部の駆動を制御する走行制御モジュールを更に備えてなることを特徴とする、請求項1に記載の移動ロボット。 - 前記制御部は、
前記移動方向が直進走行であるか、所定基準値未満の回転走行である場合、前記連続した複数の画像のうち前記センサ部の障害物感知時点以前の特定時点画像を選択することを特徴とする、請求項1に記載の移動ロボット。 - 前記制御部は、
前記移動速度が遅いほど前記センサ部の障害物感知時点を基準に更に過去時点の画像を選択することを特徴とする、請求項1に記載の移動ロボット。 - 前記障害物認識モジュールは、
マシンラーニング(machine learning)により既に学習されたデータに基づいて前記選択された特定時点画像に含まれる障害物の属性を認識することを特徴とする、請求項1に記載の移動ロボット。 - 前記選択された特定時点画像を所定サーバに送信し、前記所定サーバからマシンラーニングに関するデータを受信する通信部を更に備えることを特徴とする、請求項1に記載の移動ロボット。
- 前記所定サーバから受信したマシンラーニングに関するデータに基づいて前記障害物認識モジュールをアップデート(update)することを特徴とする、請求項6に記載の移動ロボット。
- 前記制御部は、
前記センサ部が感知する障害物の方向に対応して前記選択された特定時点画像の一部領域を抽出する画像処理モジュールを更に備え、該抽出された特定時点画像の一部領域に含まれる障害物の属性を認識することを特徴とする、請求項1に記載の移動ロボット。 - 前記画像処理モジュールは、
前記障害物が前記本体の前面の右方向で感知される場合、前記選択された特定時点画像の右側下段領域を抽出し、
前記障害物が前記本体の前面の左方向で感知される場合、前記選択された特定時点画像の左側下段領域を抽出し、
前記障害物が前記本体の前面方向で感知される場合、前記選択された特定時点画像の中央(center)下段領域を抽出することを特徴とする、請求項8に記載の移動ロボット。 - 前記制御部は、
前記感知された障害物の位置情報と移動ロボットの位置情報を前記保存部に保存し、前記感知された障害物の位置を中心に所定のサイズを有する領域をマップ(map)に障害物地域として登録し、前記障害物地域で前記画像取得部により取得される画像に対して順次に障害物の属性を認識し、前記順次に認識された複数の認識結果に基づいて前記障害物の最終属性を判別することを特徴とする、請求項1に記載の移動ロボット。 - 移動ロボットの制御方法であって、
画像取得部により、移動中に本体周辺を連続的に撮影して複数の画像を取得するステップと、
前記画像取得部が取得する連続した複数の画像を保存するステップと、
センサ部により、障害物を感知するステップと、
前記センサ部が障害物を感知すると、前記本体の移動方向と移動速度に基づいて、前記連続した複数の画像のうち前記センサ部の障害物感知時点以前の特定時点画像を選択するステップと、
前記選択された特定時点画像に含まれる障害物の属性を認識するステップと、
制御部により、前記認識された障害物の属性に基づいて、走行部の駆動を制御するステップと、を含んでなり、
前記センサ部が、障害物感知センサと、動作感知センサと、加速度センサと、を備えてなり、
前記障害物感知センサは、前記移動ロボットの移動中に障害物を感知して障害物情報を出力し、
前記動作感知センサは、前記移動ロボットの動作を感知して動作情報を出力し、
前記加速度センサは、前記移動ロボットの速度変化を感知して速度変化による衝撃量を検出しそれに対応する電圧値を出力し、
前記制御部は、前記動作感知センサから出力された動作情報に基づいて、画像情報を用いた絶対位置に対応した相対位置として、前記移動ロボットの位置変化を算出する、移動ロボットの制御方法。 - 前記特定時点画像を選択するステップは、
前記移動方向が直進走行であるか、所定基準値未満の回転走行である場合、前記連続した複数の画像のうち前記センサ部の障害物感知時点以前の特定時点画像を選択することを特徴とする、請求項11に記載の移動ロボットの制御方法。 - 前記特定時点画像を選択するステップは、
前記移動の速度が遅いほど前記センサ部の障害物感知時点を基準に更に過去時点の画像を選択することを特徴とする、請求項11に記載の移動ロボットの制御方法。 - 前記障害物の属性を認識するステップは、
マシンラーニング(machine learning)により既に学習されたデータに基づいて前記選択された特定時点画像に含まれる障害物の属性を認識することを特徴とする、請求項11に記載の移動ロボットの制御方法。 - 前記選択された特定時点画像を所定サーバに送信するステップと、
前記所定サーバからマシンラーニングに関するデータを受信するステップと、を更に含むことを特徴とする、請求項11に記載の移動ロボットの制御方法。 - 前記所定サーバから受信したマシンラーニングに関するデータに基づいて前記障害物認識モジュールをアップデート(update)するステップを更に含むことを特徴とする、請求項15に記載の移動ロボットの制御方法。
- 前記センサ部が感知する障害物の方向に対応して前記選択された特定時点画像の一部領域を抽出し、該抽出された特定時点画像の一部領域に含まれる障害物の属性を認識するステップを更に含むことを特徴とする、請求項11に記載の移動ロボットの制御方法。
- 前記選択された特定時点画像の一部領域を抽出するステップは、
前記障害物が前記本体の前面の右方向で感知される場合、前記画像取得部が取得する画像の右側下段領域を抽出し、
前記障害物が前記本体の前面の左方向で感知される場合、前記画像取得部が取得する画像の左側下段領域を抽出し、
前記障害物が前記本体の前面方向で感知される場合、前記画像取得部が取得する画像の中央(center)下段領域を抽出することを特徴とする、請求項17に記載の移動ロボットの制御方法。 - 前記駆動を制御するステップは、
前記感知された障害物が拘束可能性のある障害物である場合、回避動作を行うように制御することを特徴とする、請求項11に記載の移動ロボットの制御方法。 - 前記センサ部が障害物を感知すると、前記感知された障害物の位置情報と移動ロボットの位置情報を保存し、前記感知された障害物の位置を中心に所定のサイズを有する領域をマップ(map)に障害物地域として登録するステップと、
前記障害物地域を移動しながら前記画像取得部により取得される画像に対して順次に障害物の属性を認識するステップと、
前記順次に認識された複数の認識結果に基づいて前記障害物の最終属性を判別するステップと、を更に含むことを特徴とする、請求項11に記載の移動ロボットの制御方法。
Applications Claiming Priority (7)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR1020160108385A KR20180023302A (ko) | 2016-08-25 | 2016-08-25 | 이동 로봇 및 그 제어방법 |
KR10-2016-0108386 | 2016-08-25 | ||
KR1020160108384A KR102548936B1 (ko) | 2016-08-25 | 2016-08-25 | 인공지능 이동 로봇 및 그 제어방법 |
KR10-2016-0108385 | 2016-08-25 | ||
KR10-2016-0108384 | 2016-08-25 | ||
KR1020160108386A KR20180023303A (ko) | 2016-08-25 | 2016-08-25 | 이동 로봇 및 그 제어방법 |
PCT/KR2017/009260 WO2018038553A1 (ko) | 2016-08-25 | 2017-08-24 | 이동 로봇 및 그 제어방법 |
Publications (3)
Publication Number | Publication Date |
---|---|
JP2019528535A JP2019528535A (ja) | 2019-10-10 |
JP2019528535A5 JP2019528535A5 (ja) | 2020-10-22 |
JP7055127B2 true JP7055127B2 (ja) | 2022-04-15 |
Family
ID=61245133
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
JP2019510881A Active JP6861797B2 (ja) | 2016-08-25 | 2017-08-24 | 移動ロボット及びその制御方法 |
JP2019510776A Active JP7055127B2 (ja) | 2016-08-25 | 2017-08-24 | 移動ロボット及びその制御方法 |
JP2019510871A Active JP6785950B2 (ja) | 2016-08-25 | 2017-08-24 | 移動ロボット及びその制御方法 |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
JP2019510881A Active JP6861797B2 (ja) | 2016-08-25 | 2017-08-24 | 移動ロボット及びその制御方法 |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
JP2019510871A Active JP6785950B2 (ja) | 2016-08-25 | 2017-08-24 | 移動ロボット及びその制御方法 |
Country Status (6)
Country | Link |
---|---|
US (3) | US11199852B2 (ja) |
EP (3) | EP3505310B1 (ja) |
JP (3) | JP6861797B2 (ja) |
CN (3) | CN109890575B (ja) |
AU (3) | AU2017316090B2 (ja) |
WO (3) | WO2018038552A1 (ja) |
Families Citing this family (50)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11199852B2 (en) * | 2016-08-25 | 2021-12-14 | Lg Electronics Inc. | Mobile robot and control method for controlling the same |
USD906390S1 (en) * | 2017-12-14 | 2020-12-29 | The Hi-Tech Robotic Systemz Ltd | Mobile robot |
USD896858S1 (en) * | 2017-12-14 | 2020-09-22 | The Hi-Tech Robotic Systemz Ltd | Mobile robot |
USD907084S1 (en) * | 2017-12-14 | 2021-01-05 | The Hi-Tech Robotic Systemz Ltd | Mobile robot |
USD879851S1 (en) * | 2017-12-29 | 2020-03-31 | Beijing Geekplus Technology Co., Ltd. | Robot |
USD879852S1 (en) * | 2018-03-15 | 2020-03-31 | Beijing Geekplus Technology Co., Ltd. | Mobile robot |
JP7194914B2 (ja) * | 2018-03-29 | 2022-12-23 | パナソニックIpマネジメント株式会社 | 自律移動掃除機、自律移動掃除機による掃除方法、及び自律移動掃除機用プログラム |
KR102519064B1 (ko) * | 2018-04-25 | 2023-04-06 | 삼성전자주식회사 | 사용자에게 서비스를 제공하는 이동형 로봇 장치 및 방법 |
US20190332114A1 (en) * | 2018-04-26 | 2019-10-31 | Maidbot, Inc. | Robot Contextualization of Map Regions |
SG11202010477WA (en) * | 2018-04-26 | 2020-11-27 | Maidbot Inc | Automated robot alert system |
KR102100474B1 (ko) * | 2018-04-30 | 2020-04-13 | 엘지전자 주식회사 | 인공지능 청소기 및 그 제어방법 |
WO2019212239A1 (en) * | 2018-05-04 | 2019-11-07 | Lg Electronics Inc. | A plurality of robot cleaner and a controlling method for the same |
CN110470296A (zh) * | 2018-05-11 | 2019-11-19 | 珠海格力电器股份有限公司 | 一种定位方法、定位机器人及计算机存储介质 |
EP3797377A1 (en) * | 2018-05-22 | 2021-03-31 | Starship Technologies OÜ | Method and system for analyzing robot surroundings |
US10885395B2 (en) * | 2018-06-17 | 2021-01-05 | Pensa Systems | Method for scaling fine-grained object recognition of consumer packaged goods |
CN108968811A (zh) * | 2018-06-20 | 2018-12-11 | 四川斐讯信息技术有限公司 | 一种扫地机器人的物体识别方法及*** |
USD911406S1 (en) * | 2018-08-17 | 2021-02-23 | Grey Orange Pte. Ltd | Robot for moving articles within a facility |
KR20200036678A (ko) * | 2018-09-20 | 2020-04-07 | 삼성전자주식회사 | 청소 로봇 및 그의 태스크 수행 방법 |
CN109452914A (zh) * | 2018-11-01 | 2019-03-12 | 北京石头世纪科技有限公司 | 智能清洁设备,清洁模式选择方法,计算机存储介质 |
US11338438B2 (en) * | 2019-01-25 | 2022-05-24 | Bear Robotics, Inc. | Method, system and non-transitory computer-readable recording medium for determining a movement path of a robot |
RU2709523C1 (ru) * | 2019-02-19 | 2019-12-18 | Общество с ограниченной ответственностью "ПРОМОБОТ" | Система определения препятствий движению робота |
CN109760060B (zh) * | 2019-03-02 | 2021-06-08 | 安徽理工大学 | 一种多自由度机器人智能避障方法及其*** |
KR20200115696A (ko) * | 2019-03-07 | 2020-10-08 | 삼성전자주식회사 | 전자 장치 및 그 제어 방법 |
TWI683197B (zh) * | 2019-03-19 | 2020-01-21 | 東元電機股份有限公司 | 移動平台圖資校正系統 |
DE102019109596A1 (de) * | 2019-04-11 | 2020-10-15 | Vorwerk & Co. Interholding Gmbh | System aus einem manuell geführten Bodenbearbeitungsgerät, einem ausschließlich automatisch betriebenen Bodenbearbeitungsgerät und einer Recheneinrichtung |
CN110315553B (zh) * | 2019-06-23 | 2023-10-27 | 大国重器自动化设备(山东)股份有限公司 | 一种餐厅用机器人防碰撞***及方法 |
CN112149458A (zh) * | 2019-06-27 | 2020-12-29 | 商汤集团有限公司 | 障碍物检测方法、智能驾驶控制方法、装置、介质及设备 |
CN110315537A (zh) * | 2019-06-27 | 2019-10-11 | 深圳乐动机器人有限公司 | 一种控制机器人运动的方法、装置及机器人 |
US11650597B2 (en) * | 2019-07-09 | 2023-05-16 | Samsung Electronics Co., Ltd. | Electronic apparatus for identifying object through warped image and control method thereof |
EP3949380A4 (en) * | 2019-07-09 | 2022-05-25 | Samsung Electronics Co., Ltd. | ELECTRONIC DEVICE AND ASSOCIATED CONTROL METHOD |
CN112155487A (zh) * | 2019-08-21 | 2021-01-01 | 追创科技(苏州)有限公司 | 扫地机器人、扫地机器人的控制方法及存储介质 |
US11426885B1 (en) * | 2019-08-27 | 2022-08-30 | X Development Llc | Robot docking station identification surface |
KR20210028426A (ko) * | 2019-09-04 | 2021-03-12 | 엘지전자 주식회사 | 로봇 청소기 및 그 제어방법 |
CN110543436B (zh) * | 2019-09-06 | 2021-09-24 | 北京云迹科技有限公司 | 一种机器人的数据获取方法及装置 |
KR20210039232A (ko) * | 2019-10-01 | 2021-04-09 | 엘지전자 주식회사 | 로봇 청소기 및 청소 경로를 결정하기 위한 방법 |
KR20210057582A (ko) * | 2019-11-12 | 2021-05-21 | 삼성전자주식회사 | 잘못 흡입된 객체를 식별하는 로봇 청소기 및 그 제어 방법 |
CN110861089B (zh) * | 2019-11-29 | 2020-11-06 | 北京理工大学 | 一种多机器人***任务均衡分配协同工作控制方法 |
CN111538329B (zh) * | 2020-04-09 | 2023-02-28 | 北京石头创新科技有限公司 | 一种图像查看方法、终端及清洁机 |
CN111715559A (zh) * | 2020-06-22 | 2020-09-29 | 柴诚芃 | 一种基于机器视觉的垃圾分拣*** |
CN112077840B (zh) * | 2020-08-08 | 2022-02-15 | 浙江科聪控制技术有限公司 | 一种防爆巡检机器人的避障方法及应用于该方法的机器人 |
CN112380942A (zh) * | 2020-11-06 | 2021-02-19 | 北京石头世纪科技股份有限公司 | 一种识别障碍物的方法、装置、介质和电子设备 |
KR102361338B1 (ko) * | 2020-11-27 | 2022-02-15 | (주)엘이디소프트 | 자율주행이 가능한 실내 uv 살균장치 |
USD967883S1 (en) * | 2021-01-06 | 2022-10-25 | Grey Orange International Inc. | Robot for handling goods in a facility |
KR20220111526A (ko) * | 2021-02-02 | 2022-08-09 | 자이메드 주식회사 | 실시간 생체 이미지 인식 방법 및 장치 |
CN112971616B (zh) * | 2021-02-07 | 2022-12-30 | 美智纵横科技有限责任公司 | 一种充电座规避方法、装置、扫地机器人及存储介质 |
WO2023003158A1 (ko) * | 2021-07-20 | 2023-01-26 | 삼성전자주식회사 | 로봇 및 그 제어 방법 |
CN113721603B (zh) * | 2021-07-29 | 2023-08-08 | 云鲸智能(深圳)有限公司 | 基站探索方法、装置、机器人及可读存储介质 |
CN113827152B (zh) * | 2021-08-30 | 2023-02-17 | 北京盈迪曼德科技有限公司 | 区域状态确定方法、装置及机器人 |
CN114670244B (zh) * | 2022-03-29 | 2023-10-20 | 中国铁建重工集团股份有限公司 | 一种结构制造方法及装置 |
CN116038716B (zh) * | 2023-03-14 | 2023-07-18 | 煤炭科学研究总院有限公司 | 机器人的控制方法和机器人的控制模型的训练方法 |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20150336274A1 (en) | 2014-05-20 | 2015-11-26 | International Business Machines Corporation | Information Technology Asset Type Identification Using a Mobile Vision-Enabled Robot |
US20160167226A1 (en) | 2014-12-16 | 2016-06-16 | Irobot Corporation | Systems and Methods for Capturing Images and Annotating the Captured Images with Information |
JP2016134145A (ja) | 2015-01-22 | 2016-07-25 | 株式会社日本総合研究所 | 歩行障害箇所判定システム及びその自走型走行装置 |
Family Cites Families (47)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5314037A (en) * | 1993-01-22 | 1994-05-24 | Shaw David C H | Automobile collision avoidance system |
JPH1132253A (ja) * | 1997-07-08 | 1999-02-02 | Hitachi Ltd | 画像処理装置 |
JP3529049B2 (ja) * | 2002-03-06 | 2004-05-24 | ソニー株式会社 | 学習装置及び学習方法並びにロボット装置 |
US7813835B2 (en) * | 2002-03-15 | 2010-10-12 | Sony Corporation | Robot behavior control system, behavior control method, and robot device |
WO2004018159A1 (ja) * | 2002-08-26 | 2004-03-04 | Sony Corporation | 環境同定装置及び環境同定方法、並びにロボット装置 |
US7689321B2 (en) * | 2004-02-13 | 2010-03-30 | Evolution Robotics, Inc. | Robust sensor fusion for mapping and localization in a simultaneous localization and mapping (SLAM) system |
KR100671897B1 (ko) * | 2005-04-04 | 2007-01-24 | 주식회사 대우일렉트로닉스 | 스위치형 센서가 구비된 로봇 청소기 |
KR100669892B1 (ko) | 2005-05-11 | 2007-01-19 | 엘지전자 주식회사 | 장애물 회피 기능을 갖는 이동로봇과 그 방법 |
JP2007011857A (ja) * | 2005-07-01 | 2007-01-18 | Toyota Motor Corp | 自律移動型ロボット及び自律移動型ロボットの制御方法 |
KR100811886B1 (ko) * | 2006-09-28 | 2008-03-10 | 한국전자통신연구원 | 장애물 회피 진행이 가능한 자율이동로봇 및 그 방법 |
JP2008172441A (ja) * | 2007-01-10 | 2008-07-24 | Omron Corp | 検出装置および方法、並びに、プログラム |
KR100877072B1 (ko) * | 2007-06-28 | 2009-01-07 | 삼성전자주식회사 | 이동 로봇을 위한 맵 생성 및 청소를 동시에 수행하는 방법및 장치 |
KR100902929B1 (ko) * | 2007-07-18 | 2009-06-15 | 엘지전자 주식회사 | 이동 로봇 및 그 제어방법 |
CN101271525B (zh) * | 2008-04-10 | 2011-05-04 | 复旦大学 | 一种快速的图像序列特征显著图获取方法 |
KR20090112984A (ko) * | 2008-04-25 | 2009-10-29 | 포항공과대학교 산학협력단 | 장애물 감지 센서를 구비한 이동로봇 |
KR100997656B1 (ko) * | 2008-06-18 | 2010-12-01 | 한양대학교 산학협력단 | 로봇의 장애물 검출방법 및 그 장치 |
KR101495333B1 (ko) * | 2008-07-02 | 2015-02-25 | 삼성전자 주식회사 | 장애물 검출 장치 및 방법 |
JP5106356B2 (ja) * | 2008-11-17 | 2012-12-26 | セコム株式会社 | 画像監視装置 |
KR101524020B1 (ko) * | 2009-03-06 | 2015-05-29 | 엘지전자 주식회사 | 로봇 청소기의 점진적 지도 작성 및 위치 보정 방법 |
TWI388956B (zh) * | 2009-05-20 | 2013-03-11 | Univ Nat Taiwan Science Tech | 行動機器人與其目標物處理路徑的規劃方法 |
US9014848B2 (en) * | 2010-05-20 | 2015-04-21 | Irobot Corporation | Mobile robot system |
KR101271315B1 (ko) * | 2011-03-28 | 2013-06-04 | 고려대학교 산학협력단 | 이동 로봇의 장애물 회피 시스템 |
US8447863B1 (en) * | 2011-05-06 | 2013-05-21 | Google Inc. | Systems and methods for object recognition |
CN103608504B (zh) * | 2011-06-15 | 2014-12-24 | 东丽株式会社 | 复合纤维 |
US9582000B2 (en) * | 2011-09-07 | 2017-02-28 | Lg Electronics Inc. | Robot cleaner, and system and method for remotely controlling the same |
US8311973B1 (en) * | 2011-09-24 | 2012-11-13 | Zadeh Lotfi A | Methods and systems for applications for Z-numbers |
KR101303161B1 (ko) * | 2011-10-18 | 2013-09-09 | 엘지전자 주식회사 | 이동 로봇 및 이의 제어 방법 |
US8396254B1 (en) * | 2012-02-09 | 2013-03-12 | Google Inc. | Methods and systems for estimating a location of a robot |
KR101949277B1 (ko) * | 2012-06-18 | 2019-04-25 | 엘지전자 주식회사 | 이동 로봇 |
KR20140031742A (ko) * | 2012-09-05 | 2014-03-13 | 삼성전기주식회사 | 이미지 특징 추출 장치 및 이미지 특징 추출 방법, 그를 이용한 영상 처리 시스템 |
KR101490170B1 (ko) * | 2013-03-05 | 2015-02-05 | 엘지전자 주식회사 | 로봇 청소기 |
KR101450569B1 (ko) * | 2013-03-05 | 2014-10-14 | 엘지전자 주식회사 | 로봇 청소기 |
KR101395888B1 (ko) * | 2013-03-21 | 2014-05-27 | 엘지전자 주식회사 | 로봇 청소기 및 그 동작방법 |
WO2015006224A1 (en) * | 2013-07-08 | 2015-01-15 | Vangogh Imaging, Inc. | Real-time 3d computer vision processing engine for object recognition, reconstruction, and analysis |
KR102152641B1 (ko) * | 2013-10-31 | 2020-09-08 | 엘지전자 주식회사 | 이동 로봇 |
US9504367B2 (en) * | 2013-11-20 | 2016-11-29 | Samsung Electronics Co., Ltd. | Cleaning robot and method for controlling the same |
KR102016551B1 (ko) * | 2014-01-24 | 2019-09-02 | 한화디펜스 주식회사 | 위치 추정 장치 및 방법 |
KR102158695B1 (ko) * | 2014-02-12 | 2020-10-23 | 엘지전자 주식회사 | 로봇 청소기 및 이의 제어방법 |
US9713982B2 (en) * | 2014-05-22 | 2017-07-25 | Brain Corporation | Apparatus and methods for robotic operation using video imagery |
KR20150142475A (ko) | 2014-06-12 | 2015-12-22 | 연세대학교 산학협력단 | 장애물 식별 장치 및 방법 |
JP2016028311A (ja) | 2014-07-10 | 2016-02-25 | 株式会社リコー | ロボット、プログラム、及び記憶媒体 |
US10289910B1 (en) * | 2014-07-10 | 2019-05-14 | Hrl Laboratories, Llc | System and method for performing real-time video object recognition utilizing convolutional neural networks |
KR101610502B1 (ko) * | 2014-09-02 | 2016-04-07 | 현대자동차주식회사 | 자율주행차량의 주행환경 인식장치 및 방법 |
CN104268882A (zh) * | 2014-09-29 | 2015-01-07 | 深圳市热活力科技有限公司 | 基于双线阵摄像头的高速运动物体检测与测速方法及*** |
CN104552341B (zh) * | 2014-12-29 | 2016-05-04 | 国家电网公司 | 移动工业机器人单点多视角挂表位姿误差检测方法 |
US9987752B2 (en) * | 2016-06-10 | 2018-06-05 | Brain Corporation | Systems and methods for automatic detection of spills |
US11199852B2 (en) * | 2016-08-25 | 2021-12-14 | Lg Electronics Inc. | Mobile robot and control method for controlling the same |
-
2017
- 2017-08-24 US US16/327,454 patent/US11199852B2/en active Active
- 2017-08-24 JP JP2019510881A patent/JP6861797B2/ja active Active
- 2017-08-24 CN CN201780066252.6A patent/CN109890575B/zh active Active
- 2017-08-24 JP JP2019510776A patent/JP7055127B2/ja active Active
- 2017-08-24 EP EP17843971.7A patent/EP3505310B1/en active Active
- 2017-08-24 AU AU2017316090A patent/AU2017316090B2/en active Active
- 2017-08-24 WO PCT/KR2017/009258 patent/WO2018038552A1/ko unknown
- 2017-08-24 AU AU2017316091A patent/AU2017316091B2/en active Active
- 2017-08-24 JP JP2019510871A patent/JP6785950B2/ja active Active
- 2017-08-24 WO PCT/KR2017/009257 patent/WO2018038551A1/ko unknown
- 2017-08-24 CN CN201780066283.1A patent/CN109890576B/zh active Active
- 2017-08-24 EP EP17843972.5A patent/EP3505311B1/en active Active
- 2017-08-24 AU AU2017316089A patent/AU2017316089B2/en active Active
- 2017-08-24 US US16/327,449 patent/US11150666B2/en active Active
- 2017-08-24 CN CN201780066246.0A patent/CN109890574B/zh active Active
- 2017-08-24 WO PCT/KR2017/009260 patent/WO2018038553A1/ko unknown
- 2017-08-24 US US16/327,452 patent/US20190179333A1/en not_active Abandoned
- 2017-08-24 EP EP17843973.3A patent/EP3505312B1/en active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20150336274A1 (en) | 2014-05-20 | 2015-11-26 | International Business Machines Corporation | Information Technology Asset Type Identification Using a Mobile Vision-Enabled Robot |
US20160167226A1 (en) | 2014-12-16 | 2016-06-16 | Irobot Corporation | Systems and Methods for Capturing Images and Annotating the Captured Images with Information |
JP2016134145A (ja) | 2015-01-22 | 2016-07-25 | 株式会社日本総合研究所 | 歩行障害箇所判定システム及びその自走型走行装置 |
Also Published As
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP7055127B2 (ja) | 移動ロボット及びその制御方法 | |
JP2019528535A5 (ja) | ||
KR102662949B1 (ko) | 인공지능 이동 로봇 및 그 제어방법 | |
KR102314539B1 (ko) | 인공지능 이동 로봇의 제어 방법 | |
US10852729B2 (en) | Moving robot and control method thereof | |
KR102548936B1 (ko) | 인공지능 이동 로봇 및 그 제어방법 | |
KR20180023302A (ko) | 이동 로봇 및 그 제어방법 | |
KR20180023303A (ko) | 이동 로봇 및 그 제어방법 | |
KR102500525B1 (ko) | 이동 로봇 | |
KR102688528B1 (ko) | 이동 로봇 및 그 제어방법 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
A621 | Written request for application examination |
Free format text: JAPANESE INTERMEDIATE CODE: A621 Effective date: 20190418 |
|
A977 | Report on retrieval |
Free format text: JAPANESE INTERMEDIATE CODE: A971007 Effective date: 20200330 |
|
A131 | Notification of reasons for refusal |
Free format text: JAPANESE INTERMEDIATE CODE: A131 Effective date: 20200407 |
|
A601 | Written request for extension of time |
Free format text: JAPANESE INTERMEDIATE CODE: A601 Effective date: 20200707 |
|
A524 | Written submission of copy of amendment under article 19 pct |
Free format text: JAPANESE INTERMEDIATE CODE: A524 Effective date: 20200907 |
|
A02 | Decision of refusal |
Free format text: JAPANESE INTERMEDIATE CODE: A02 Effective date: 20201124 |
|
A521 | Request for written amendment filed |
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20210323 |
|
C60 | Trial request (containing other claim documents, opposition documents) |
Free format text: JAPANESE INTERMEDIATE CODE: C60 Effective date: 20210323 |
|
A911 | Transfer to examiner for re-examination before appeal (zenchi) |
Free format text: JAPANESE INTERMEDIATE CODE: A911 Effective date: 20210331 |
|
C21 | Notice of transfer of a case for reconsideration by examiners before appeal proceedings |
Free format text: JAPANESE INTERMEDIATE CODE: C21 Effective date: 20210406 |
|
A912 | Re-examination (zenchi) completed and case transferred to appeal board |
Free format text: JAPANESE INTERMEDIATE CODE: A912 Effective date: 20210416 |
|
C211 | Notice of termination of reconsideration by examiners before appeal proceedings |
Free format text: JAPANESE INTERMEDIATE CODE: C211 Effective date: 20210420 |
|
C22 | Notice of designation (change) of administrative judge |
Free format text: JAPANESE INTERMEDIATE CODE: C22 Effective date: 20210824 |
|
C13 | Notice of reasons for refusal |
Free format text: JAPANESE INTERMEDIATE CODE: C13 Effective date: 20211012 |
|
A521 | Request for written amendment filed |
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20220107 |
|
C23 | Notice of termination of proceedings |
Free format text: JAPANESE INTERMEDIATE CODE: C23 Effective date: 20220201 |
|
C03 | Trial/appeal decision taken |
Free format text: JAPANESE INTERMEDIATE CODE: C03 Effective date: 20220308 |
|
C30A | Notification sent |
Free format text: JAPANESE INTERMEDIATE CODE: C3012 Effective date: 20220308 |
|
A61 | First payment of annual fees (during grant procedure) |
Free format text: JAPANESE INTERMEDIATE CODE: A61 Effective date: 20220405 |
|
R150 | Certificate of patent or registration of utility model |
Ref document number: 7055127 Country of ref document: JP Free format text: JAPANESE INTERMEDIATE CODE: R150 |