WO2020213294A1 - Self-propelled vacuum cleaner, system, and object recognition method - Google Patents

Self-propelled vacuum cleaner, system, and object recognition method Download PDF

Info

Publication number
WO2020213294A1
WO2020213294A1 PCT/JP2020/010044 JP2020010044W WO2020213294A1 WO 2020213294 A1 WO2020213294 A1 WO 2020213294A1 JP 2020010044 W JP2020010044 W JP 2020010044W WO 2020213294 A1 WO2020213294 A1 WO 2020213294A1
Authority
WO
WIPO (PCT)
Prior art keywords
vacuum cleaner
sensor
learning model
distance
object recognition
Prior art date
Application number
PCT/JP2020/010044
Other languages
French (fr)
Japanese (ja)
Inventor
渡邉　優
Original Assignee
パナソニックＩｐマネジメント株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by パナソニックＩｐマネジメント株式会社 filed Critical パナソニックＩｐマネジメント株式会社
Publication of WO2020213294A1 publication Critical patent/WO2020213294A1/en

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L9/00—Details or accessories of suction cleaners, e.g. mechanical means for controlling the suction or for effecting pulsating action; Storing devices specially adapted to suction cleaners or parts thereof; Carrying-vehicles specially adapted for suction cleaners
    • A47L9/28—Installation of the electric equipment, e.g. adaptation or attachment to the suction cleaner; Controlling suction cleaners by electric means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00—Controls for manipulators
    • B25J13/08—Controls for manipulators by means of sensing devices, e.g. viewing or touching devices
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J5/00—Manipulators mounted on wheels or on carriages
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions

Definitions

  • the present invention relates to an autonomous driving vacuum cleaner, a system, and an object recognition method.
  • Patent Document 1 In recent years, many technologies related to autonomous driving vacuum cleaners have been published in published patent gazettes (see, for example, Patent Document 1).
  • the autonomous traveling vacuum cleaner described in Patent Document 1 detects an object based on an image captured by an image pickup device. Furthermore, the distance to the object is calculated, and the alertness of the detected object is set. Then, the movement of the autonomous driving type vacuum cleaner is controlled based on the set alertness and the distance to the object.
  • the autonomous traveling type vacuum cleaner of Patent Document 1 includes an imaging unit that captures an object existing in the space and a sensor that calculates the distance to the object. Then, an obstacle is determined by the image pickup unit and the sensor, and the driving device of the autonomous driving type vacuum cleaner is controlled by using the determined result.
  • an ultrasonic sensor or a laser sensor is used.
  • it is difficult to calculate an accurate distance for an object having a low height such as a rug or a cable arranged in a home because the irradiation surface necessary for distance measurement cannot be secured.
  • the motor of the autonomous driving type vacuum cleaner may be locked and the cleaning action may be interrupted.
  • the autonomous traveling vacuum cleaner of the present invention has a first sensor for detecting the presence or absence of an object, a second sensor for detecting the position of an object, and a control unit.
  • the control unit recognizes the object based on the detection result of the first sensor and the detection result of the second sensor.
  • the system of the present invention includes a learning model generator that generates a learning model from learning image data using an object recognition algorithm, and an autonomous traveling type vacuum cleaner that receives a learning model from the learning model generator. Then, the autonomous traveling type vacuum cleaner has a first object recognition function unit and a first object recognition function unit that recognize an object using the learning model received from the learning model generator and the image captured by the imaging unit. It has a second object recognition function unit that recognizes an object based on the distance to the object recognized in.
  • an object is extracted from the first step of extracting features from learning image data, learning, and generating a learning model, and the generated learning model and the image captured by the imaging unit. It has a second step of recognizing an object and a third step of recognizing an object based on the distance to the object recognized in the second step.
  • an autonomous traveling vacuum cleaner that can identify an object such as a rug or a cable whose cleaning action is likely to be interrupted from the recognized object and avoid the interruption of the cleaning action due to being caught in the brush as much as possible.
  • FIG. 1 is a bottom view of the autonomous traveling type vacuum cleaner according to the embodiment of the present invention.
  • FIG. 2 is a cross-sectional view of the autonomous traveling type vacuum cleaner according to the embodiment.
  • FIG. 3 is a block diagram of a system including an autonomous traveling vacuum cleaner according to the embodiment.
  • FIG. 4 is a diagram showing an example of the recognition result of the object recognition algorithm in the same embodiment.
  • FIG. 1 is a bottom view of the autonomous traveling type vacuum cleaner according to the embodiment of the present invention.
  • the autonomous traveling type vacuum cleaner of the present embodiment has a body 1, a suction port 2 arranged on the body 1 for sucking dust, and two drive wheels arranged on the left and right sides. It includes an imaging unit including a 3, a side brush 4, a camera 5 which is a first sensor, and a distance measuring sensor 6 which is a second sensor. Further, the body 1 includes a control unit 100 arranged inside, a suction motor (not shown), and the like.
  • the camera 5 side shown in FIG. 1 will be described as the front, the opposite side as the rear, the side brush 4 side as the left, and the opposite side as the right.
  • the suction port 2 is arranged on the bottom surface on the front side of the center of the body 1 in the left-right direction.
  • a main brush (not shown) for scooping up dust adhering to a carpet or the like is arranged in the suction port 2.
  • the suction port 2 sucks dust and the like scooped up by the main brush by the suction air generated by the suction motor (not shown).
  • the drive wheel 3 on the right side and the drive wheel 3 on the left side are provided with a motor (drive unit 12 described later) that drives the drive wheel 3 connected to each other.
  • the right drive wheel 3 and the left drive wheel 3 are driven simultaneously or separately via their respective motors.
  • the body 1 can be moved in any direction, such as front-back, left-right, or turning.
  • the side brush 4 is provided on the front left side in FIG. 1 and on the back surface of the body 1.
  • the side brush 4 rotates toward the suction port 2, that is, clockwise in FIG.
  • dust and the like existing in front of the body 1 can be collected toward the suction port 2.
  • the side brush 4 is provided on the back surface of the body 1 on the left front side in FIG. 1, but the present invention is not limited to this.
  • the side brush 4 may be provided on the back surface of the body 1 on the right front side in FIG. Further, the side brush 4 may be provided only on the front right side in FIG.
  • the distance measuring sensor 6 is composed of, for example, an infrared sensor, and is arranged on the left and right in front of the body 1 to form a second sensor.
  • the distance measuring sensor 6 detects the distance to an object such as an obstacle.
  • the distance measuring sensor 6 is an optical sensor including a light emitting element and a light receiving element.
  • the ranging sensor 6 may be configured by using, for example, a laser sensor.
  • the control unit 100 is arranged inside the body 1 and is composed of semiconductor elements such as a CPU (Central Processing Unit), for example.
  • the control unit 100 includes the case where the semiconductor element itself or the entire circuit board on which the semiconductor element is mounted is referred to.
  • the autonomous driving type vacuum cleaner of the present embodiment is configured.
  • the camera 5 which is the first sensor of the autonomous driving type vacuum cleaner and the distance measuring sensor 6 which is the second sensor will be described with reference to FIG.
  • FIG. 2 is a side view of the autonomous driving type vacuum cleaner according to the present embodiment. In FIG. 2, a part of the related configurations is shown by a broken line.
  • the distance measuring sensors 6 are mounted on the left and right (see FIG. 1) near the front surface of the body 1. At this time, the distance measuring sensor 6 is arranged so that the sensor shaft SA faces the floor surface 8 at a distance D in front of the front surface of the body 1.
  • the control unit 100 monitors the distance measurement value obtained from the distance measurement sensor 6. As a result, it is possible to detect a change in height of the body 1 up to the floor surface 8 in the forward direction due to, for example, a cable. At the same time, the distance D from the point where the height change occurs to the autonomous driving type vacuum cleaner can be known.
  • the camera 5 is arranged on the body 1 on the upper side of the distance measuring sensor 6, for example.
  • the example in which the camera 5 is arranged above the distance measuring sensor 6 will be described, but the present invention is not limited to this.
  • the camera 5 may be arranged on the lower side of the distance measuring sensor 6 or at least one of the left and right sides.
  • FIG. 3 is a block diagram of a system including an autonomous driving type vacuum cleaner according to the present embodiment.
  • FIG. 4 is a diagram showing an example of the recognition result of the object recognition algorithm in the system of the present embodiment.
  • the system is composed of an autonomous driving type vacuum cleaner (area surrounded by a single point chain line) having a built-in control unit 100, a PC (Personal Computer) 200 constituting a learning model generator, and the like.
  • an autonomous driving type vacuum cleaner area surrounded by a single point chain line
  • a PC Personal Computer
  • the control unit 100 mainly has a first block 101 that constitutes a first object recognition function unit, a second block 102 that constitutes a second object recognition function unit, and the like.
  • the first block 101 constitutes a block for recognizing an object from an image captured by an imaging unit such as a camera 5 by using an object recognition algorithm.
  • the second block 102 constitutes a block for performing various processes from the recognition result of the first block 101 to more accurately recognizing the object. The details of the second block 102 will be described later.
  • the PC200 is a so-called personal computer. As will be described later, the PC 200 stores a plurality of learning image data in a memory or the like in advance. Then, the PC 200 uses an object recognition algorithm to extract the features of the object from the stored image data for learning, performs learning, and then generates a learning model. That is, the PC 200 constitutes the learning model generation device described above.
  • control unit 100 and the PC 200 of the autonomous driving type vacuum cleaner are configured to be connectable via, for example, a USB (Universal Serial Bus) cable or the like.
  • USB Universal Serial Bus
  • the learning model generated by the PC 200 is transmitted to the control unit 100 of the autonomous driving vacuum cleaner, and can be received by the autonomous driving vacuum cleaner.
  • the control unit 100 includes a storage unit 103 composed of, for example, a non-volatile memory (not shown) such as a flash memory.
  • the storage unit 103 stores the learning model received from the PC 200. As the capacity of the learning model increases, the required memory capacity also increases. Therefore, as the storage unit 103, a large-capacity storage medium such as a hard disk may be used instead of a semiconductor memory such as a non-volatile memory.
  • each block of the PC200 shown in FIG. 3 may be provided in, for example, a server. That is, the server and the control unit 100 may be connected to each other via a network. In this case, the server configures the learning model generator, and the server sends the learning model generated by the server to the control unit 100.
  • the autonomous traveling type vacuum cleaner of the present embodiment recognizes an object (object 10) based on the object recognition algorithm executed in the first block 101 and the detection result of the distance measuring sensor 6.
  • the system of this embodiment uses an object recognition algorithm on the PC200 side to generate a learning model of an object in which an autonomous driving vacuum cleaner is involved and the cleaning operation is likely to be interrupted. Specifically, first, a large amount of learning image data such as a rug or a cable, which is an object whose cleaning operation is easily interrupted, is taken in in advance and stored in a memory (not shown). From the captured image data for learning, the feature extraction of the object is performed using the object recognition algorithm and learning is performed. Then, the PC 200 generates a learning model. The above process corresponds to the first step of generating the learning model.
  • the generated learning model is transmitted from the PC 200 side to the control unit 100 of the autonomous driving vacuum cleaner, and stored in the memory of the autonomous driving vacuum cleaner.
  • transmission to the control unit 100 is executed by connecting the PC 200 and the control unit 100 with, for example, a USB cable or the like.
  • the autonomous driving vacuum cleaner of the present embodiment captures the front of the autonomous driving vacuum cleaner with the camera 5 in the form of a moving image or a still image, for example, during the cleaning operation. Then, the camera 5 transmits the captured image data to the first block 101 of the control unit 100.
  • the first block of the control unit 100 extracts the features of the object from the transmitted image data by using the object recognition algorithm.
  • control unit 100 extracts and learns the features of each object 10 on the PC200 side, and stores the learned learning model in advance in a storage unit 103 such as a non-volatile memory. ing.
  • the object whose features have been extracted in the first block is compared with the object 10 of the learning model stored in the storage unit 103, and it is determined whether or not the object is the object 10. ..
  • the detection frame 11 including whether or not 10 exists is output and displayed in the image data 9. If the object is not a learned object, the detection frame is not displayed. That is, when the object 10 learned in advance exists in the object of the image data 9 transmitted from the camera 5, the detection frame 11 is displayed to notify the user.
  • the length from the lower end of the detection frame 11 (corresponding to the lowermost end of the detection frame 11) displayed in the image data 9 to the lower end of the image data 9 is calculated from the number of pixels of the image data and the like. As a result, the distance X to the object 10 is calculated.
  • the process of acquiring the distance X from the image data 9 is usually a process performed inside the control unit 100. Therefore, it is not always necessary to output the image data 9, display it, and configure it so that the user can confirm it.
  • control unit 100 recognizes the object 10 in the first block 101 constituting the first object recognition function unit.
  • the above process corresponds to the second step of recognizing the object 10 from the image.
  • control unit 100 measures the distance of the object 10 with the distance measuring sensor 6. Then, in the second block 102, the object 10 is recognized based on the distance measured and the distance to the object 10 recognized in the second step.
  • the above process corresponds to the third step of recognizing the object 10 from the distance from the object 10.
  • the second block 102 of the control unit 100 first determines whether or not the distance X from the autonomous driving type vacuum cleaner to the object 10 is x or more. At this time, if the distance X is determined to be less than x, the control unit 100 controls the drive unit 12. Specifically, the control unit 100 controls the drive unit 12 so as to decelerate the autonomous traveling type vacuum cleaner.
  • the second block 102 starts to acquire the sensor value S corresponding to the output value detected by the distance measuring sensor 6.
  • the second block 102 constantly acquires the average of the sensor values S during deceleration traveling of the autonomous driving type vacuum cleaner, and stores the average value as Save in a memory (not shown).
  • the memory may also be used as the storage unit 103.
  • the sensor axis SA of the distance measuring sensor 6 faces the floor surface 8 in the forward direction. Therefore, when the height of the traveling surface of the autonomous traveling vacuum cleaner changes due to the rug laid on the floor surface 8, the direction of the sensor axis SA changes, so that the sensor value S of the distance measuring sensor 6 is detected. Change happens. This change in the sensor value S causes a malfunction of the autonomous driving type vacuum cleaner.
  • the second block 102 sets a predetermined threshold value T in advance assuming a change (fluctuation) in the sensor value S.
  • the control unit 100 of the autonomous traveling type vacuum cleaner determines that the object 10 whose cleaning operation is likely to be interrupted is located at the position of the distance D in the forward direction. As a result, it is possible to prevent the object 10 from being caught in the object 10 and prevent the cleaning operation of the autonomous traveling vacuum cleaner from being interrupted.
  • the sensor value S detected by the distance measuring sensor 6 fluctuates due to, for example, dust.
  • the fluctuating sensor value S may cause the second block 102 to erroneously detect that there is, for example, a carpet in front of it. Therefore, the threshold value T is set, and if it is equal to or less than the threshold value T, it is determined that the object 10 is not, for example, a carpet, and false detection due to fluctuation is avoided. Thereby, the detection accuracy of the object 10 can be improved.
  • the autonomous driving type vacuum cleaner can appropriately determine that there is an object 10 at a position of a distance D in the forward direction in which the cleaning operation is likely to be interrupted, even if the deceleration operation is not performed. As a result, the efficiency of the cleaning operation of the autonomous traveling type vacuum cleaner can be improved.
  • the present invention is limited to this. Absent. For example, it is possible to always acquire the sensor value S and detect the distance at which the height of the traveling surface changes while the autonomous traveling vacuum cleaner is traveling without using the object recognition algorithm. However, in this case, it is not possible to distinguish between an object such as unevenness or dust on the traveling surface and the object 10 in which the cleaning operation of the autonomous traveling vacuum cleaner is likely to be interrupted, and there is a possibility that the object 10 cannot be accurately determined. Therefore, the object 10 is identified from the object by using the object recognition algorithm in the first block 101, and the object 10 is assigned to the identified object 10 based on the sensor value S acquired by the distance measuring sensor 6. A recognizable configuration is more desirable.
  • the distance measuring sensor 6 detects a change in the height of the traveling surface, and further determines the presence or absence of the object 10. As a result, erroneous recognition can be prevented and the object 10 can be detected more accurately.
  • the autonomous traveling type vacuum cleaner of the present embodiment has a low height such as a rug, a cable, etc., which was difficult to detect by the conventional technology, and the autonomous traveling type vacuum cleaner is likely to be involved and the cleaning operation is easily interrupted. The object can be detected accurately. This makes it possible to provide an autonomous traveling type vacuum cleaner in which the cleaning action is not interrupted.
  • the autonomous driving type vacuum cleaner of the present embodiment has a first sensor for detecting the presence or absence of an object, a second sensor for detecting the position of the object, and a control unit.
  • the control unit is configured to recognize an object based on the detection result of the first sensor and the detection result of the second sensor.
  • the first sensor of the autonomous traveling type vacuum cleaner of the present embodiment is composed of an imaging unit that captures a moving image or a still image, and the control unit recognizes an object from the image captured by the imaging unit.
  • a configuration with an algorithm is desirable.
  • the autonomous traveling type vacuum cleaner of the present embodiment further has a drive unit for driving two drive wheels arranged on the body, and the second sensor measures the distance to the object. It consists of sensors. Then, when the control unit recognizes the object by the object recognition algorithm, it drives the drive unit to move the body forward while calculating the average value of the output values from the distance measuring sensor, and the average value and the distance measuring sensor are used. When the difference from the latest output value exceeds the threshold value, it is desirable to calculate the presence / absence of an object and the distance to the object.
  • the system of the present embodiment is an autonomous traveling type that receives a learning model generator that generates a learning model from the learning image data using an object recognition algorithm and a learning model transmitted from the learning model generator.
  • a vacuum cleaner has a first object recognition function unit that recognizes an object and a first object recognition function unit that recognizes an object by using the learning model received from the learning model generator and the image captured by the imaging unit. It is desirable to have a configuration having a second object recognition function unit that recognizes the object based on the distance from the recognized object.
  • the first step of extracting features from the learning image data, learning, and generating a learning model, and the generated learning model and the image captured by the imaging unit It is desirable to have a configuration having a second step of recognizing an object from data and a third step of recognizing an object based on the distance between the object recognized in the second step.
  • the present invention can be widely applied to home-use autonomous driving vacuum cleaners, commercial-use autonomous driving vacuum cleaners, and the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Robotics (AREA)
  • Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Human Computer Interaction (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
  • Electric Vacuum Cleaner (AREA)
  • Manipulator (AREA)

Abstract

This self-propelled vacuum cleaner is provided with: a camera (5) for capturing an image of a space; an object recognition algorithm for recognizing an object from the captured image; and a distance measurement sensor that is directed at the floor in the advancing direction of the self-propelled vacuum cleaner and is for calculating distances. When the object recognition algorithm recognizes an object, the self-propelled vacuum cleaner advances while calculating the average value of the sensor values from the distance measurement sensor, and when the difference between the average value and the latest sensor value exceeds a threshold, the self-propelled vacuum cleaner calculates the presence or absence of an object and the distance to the object. As a result of this configuration, it is possible to provide a self-propelled vacuum cleaner that is capable of avoiding interruption of cleaning due to entanglement of a cable or the like as much as possible.

Description

自律走行型掃除機、システム、および、物体認識方法Autonomous vacuum cleaners, systems, and object recognition methods
　本発明は、自律走行型掃除機、システム、および、物体認識方法に関する。 The present invention relates to an autonomous driving vacuum cleaner, a system, and an object recognition method.
　近年、自律走行型掃除機に関する技術が、公開特許公報（例えば、特許文献１参照）に、数多く公開されている。特許文献１に記載の自律走行型掃除機は、撮像装置で撮像した画像に基づいて、物体を検出する。さらに、物体までの距離を算出し、検出された物体の警戒度を設定する。そして、設定した警戒度と物体までの距離に基づいて、自律走行型掃除機の移動を制御する。 In recent years, many technologies related to autonomous driving vacuum cleaners have been published in published patent gazettes (see, for example, Patent Document 1). The autonomous traveling vacuum cleaner described in Patent Document 1 detects an object based on an image captured by an image pickup device. Furthermore, the distance to the object is calculated, and the alertness of the detected object is set. Then, the movement of the autonomous driving type vacuum cleaner is controlled based on the set alertness and the distance to the object.
　つまり、特許文献１の自律走行型掃除機は、空間内に存在する物体を撮影する撮像部と、物体までの距離を算出するセンサを備える。そして、撮像部とセンサにより、障害物を判定して、判定した結果を用いて、自律走行型掃除機の駆動装置を制御する。 That is, the autonomous traveling type vacuum cleaner of Patent Document 1 includes an imaging unit that captures an object existing in the space and a sensor that calculates the distance to the object. Then, an obstacle is determined by the image pickup unit and the sensor, and the driving device of the autonomous driving type vacuum cleaner is controlled by using the determined result.
　しかしながら、従来の自律走行型掃除機の場合、画像から物体を認識する際に、例えば、実際に敷物が存在するのか、窓から差し込む光が床面に反射して敷物であるかのように見えているのか、を判別することが難しい場合がある。 However, in the case of a conventional autonomous driving vacuum cleaner, when recognizing an object from an image, for example, it looks as if the rug actually exists or the light coming in from the window is reflected on the floor surface and is a rug. It may be difficult to determine if it is.
　また、物体までの距離を算出する場合、例えば超音波センサやレーザセンサなどが用いられる。しかしながら、上記センサだけでは、物体が、何であるのかの判別が難しい。さらに、家庭内に配置される敷物、ケーブルなどのように高さの低い対象物は、測距のために必要な照射面が確保できないため、正確な距離を算出することが難しい。 Also, when calculating the distance to an object, for example, an ultrasonic sensor or a laser sensor is used. However, it is difficult to determine what the object is with the above sensor alone. Further, it is difficult to calculate an accurate distance for an object having a low height such as a rug or a cable arranged in a home because the irradiation surface necessary for distance measurement cannot be secured.
　そのため、従来の自律走行型掃除機は、誤認識により、ブラシなどが、例えばケーブルなどを巻き込む虞がある。これにより、自律走行型掃除機のモータがロックされ、清掃行動が中断される場合がある。 Therefore, in the conventional autonomous driving type vacuum cleaner, there is a risk that the brush etc. may get caught in the cable etc. due to misrecognition. As a result, the motor of the autonomous driving type vacuum cleaner may be locked and the cleaning action may be interrupted.
特開２００８－２００７７０号公報JP-A-2008-200770
　本発明の自律走行型掃除機は、物体の有無を検知する第１のセンサと、物体の位置を検知する第２のセンサと、制御ユニットを有する。制御ユニットは、第１のセンサの検知結果と、第２のセンサの検知結果とに基づいて、物体を認識する。 The autonomous traveling vacuum cleaner of the present invention has a first sensor for detecting the presence or absence of an object, a second sensor for detecting the position of an object, and a control unit. The control unit recognizes the object based on the detection result of the first sensor and the detection result of the second sensor.
　また、本発明のシステムは、学習用画像データから物体認識アルゴリズムを用いて学習モデルを生成する学習モデル生成装置と、学習モデル生成装置からの学習モデルを受信する自律走行型掃除機を有する。そして、自律走行型掃除機は、学習モデル生成装置から受信した学習モデルと、撮像部で撮像した画像とを用いて物体を認識する第１の物体認識機能部と、第１の物体認識機能部で認識した物体との距離に基づいて物体を認識する第２の物体認識機能部を有する。 Further, the system of the present invention includes a learning model generator that generates a learning model from learning image data using an object recognition algorithm, and an autonomous traveling type vacuum cleaner that receives a learning model from the learning model generator. Then, the autonomous traveling type vacuum cleaner has a first object recognition function unit and a first object recognition function unit that recognize an object using the learning model received from the learning model generator and the image captured by the imaging unit. It has a second object recognition function unit that recognizes an object based on the distance to the object recognized in.
　また、本発明の物体認識方法は、学習用画像データから特徴を抽出し、学習し、学習モデルを生成する第１のステップと、生成された学習モデルと撮像部で撮像された画像とから物体を認識する第２のステップと、第２のステップで認識した物体との距離に基づいて、物体を認識する第３のステップと、を有する。 Further, in the object recognition method of the present invention, an object is extracted from the first step of extracting features from learning image data, learning, and generating a learning model, and the generated learning model and the image captured by the imaging unit. It has a second step of recognizing an object and a third step of recognizing an object based on the distance to the object recognized in the second step.
　上記構成によれば、認識した物体から、清掃行動が中断されやすい、敷物やケーブルなどの対象物を識別して、ブラシへの巻き込みによる清掃行動の中断をできるだけ回避できる自律走行型掃除機、それを含むシステムおよび物体認識方法を提供できる。 According to the above configuration, an autonomous traveling vacuum cleaner that can identify an object such as a rug or a cable whose cleaning action is likely to be interrupted from the recognized object and avoid the interruption of the cleaning action due to being caught in the brush as much as possible. Can provide systems and object recognition methods including.
図１は、本発明の実施の形態における自律走行型掃除機の底面図である。FIG. 1 is a bottom view of the autonomous traveling type vacuum cleaner according to the embodiment of the present invention. 図２は、同実施の形態における自律走行型掃除機の断面図である。FIG. 2 is a cross-sectional view of the autonomous traveling type vacuum cleaner according to the embodiment. 図３は、同実施の形態における自律走行型掃除機を含むシステムのブロック図である。FIG. 3 is a block diagram of a system including an autonomous traveling vacuum cleaner according to the embodiment. 図４は、同実施の形態における物体認識アルゴリズムの認識結果の一例を示す図である。FIG. 4 is a diagram showing an example of the recognition result of the object recognition algorithm in the same embodiment.
　以下、本発明の実施の形態について、図面を参照しながら説明する。なお、この実施の形態によって本発明が限定されるものではない。 Hereinafter, embodiments of the present invention will be described with reference to the drawings. The present invention is not limited to this embodiment.
　（実施の形態）
　以下、本発明の実施の形態の自律走行型掃除機の概略構成について、図１を参照しながら、説明する。 (Embodiment)
Hereinafter, the schematic configuration of the autonomous traveling vacuum cleaner according to the embodiment of the present invention will be described with reference to FIG.
　図１は、本発明の実施の形態における自律走行型掃除機の底面図である。 FIG. 1 is a bottom view of the autonomous traveling type vacuum cleaner according to the embodiment of the present invention.
　図１に示すように、本実施の形態の自律走行型掃除機は、ボディ１と、ボディ１に配設される、ゴミを吸い込むための吸込口２と、左右に配置される２つの駆動輪３と、サイドブラシ４と、第１のセンサであるカメラ５などからなる撮像部と、第２のセンサである測距センサ６などを備える。さらに、ボディ１は、内部に配置される制御ユニット１００と、吸引モータ（図示せず）などを備える。 As shown in FIG. 1, the autonomous traveling type vacuum cleaner of the present embodiment has a body 1, a suction port 2 arranged on the body 1 for sucking dust, and two drive wheels arranged on the left and right sides. It includes an imaging unit including a 3, a side brush 4, a camera 5 which is a first sensor, and a distance measuring sensor 6 which is a second sensor. Further, the body 1 includes a control unit 100 arranged inside, a suction motor (not shown), and the like.
　なお、以降では、図１中に示すカメラ５側を前方、その反対側を後方、サイドブラシ４側を左方、その反対側を右方として、説明する。 Hereinafter, the camera 5 side shown in FIG. 1 will be described as the front, the opposite side as the rear, the side brush 4 side as the left, and the opposite side as the right.
　吸込口２は、ボディ１の左右方向の中心よりも前方側の底面に配設される。吸込口２には、絨毯などに付着したゴミをかき上げるためのメインブラシ（図示せず）が配設される。吸込口２は、吸引モータ（図示せず）で発生される吸引風により、メインブラシでかき上げられたゴミなどを吸い込む。 The suction port 2 is arranged on the bottom surface on the front side of the center of the body 1 in the left-right direction. A main brush (not shown) for scooping up dust adhering to a carpet or the like is arranged in the suction port 2. The suction port 2 sucks dust and the like scooped up by the main brush by the suction air generated by the suction motor (not shown).
　右側の駆動輪３と左側の駆動輪３は、それぞれに接続される、駆動輪３を駆動するモータ（後述する駆動部１２）を備える。右側の駆動輪３と左側の駆動輪３は、それぞれのモータを介して、それぞれ同時に、あるいは別々に駆動される。これにより、前後左右、あるいは旋回など、ボディ１を、任意の方向に移動させることができる。 The drive wheel 3 on the right side and the drive wheel 3 on the left side are provided with a motor (drive unit 12 described later) that drives the drive wheel 3 connected to each other. The right drive wheel 3 and the left drive wheel 3 are driven simultaneously or separately via their respective motors. As a result, the body 1 can be moved in any direction, such as front-back, left-right, or turning.
　サイドブラシ４は、図１中の左側前方で、ボディ１の裏面に設けられる。サイドブラシ４は、吸込口２に向かって、すなわち、図１において、時計回りに回転する。これにより、ボディ１前方に存在するゴミなどを吸込口２に向けて集めることができる。なお、本実施の形態では、サイドブラシ４を、図１中の左側前方のボディ１の裏面に設ける例で説明したが、これに限られない。例えば、図１中の右側前方のボディ１の裏面にも、サイドブラシ４を設けてもよい。また、図１中の右側前方のみに、サイドブラシ４を設けてもよい。 The side brush 4 is provided on the front left side in FIG. 1 and on the back surface of the body 1. The side brush 4 rotates toward the suction port 2, that is, clockwise in FIG. As a result, dust and the like existing in front of the body 1 can be collected toward the suction port 2. In the present embodiment, the side brush 4 is provided on the back surface of the body 1 on the left front side in FIG. 1, but the present invention is not limited to this. For example, the side brush 4 may be provided on the back surface of the body 1 on the right front side in FIG. Further, the side brush 4 may be provided only on the front right side in FIG.
　撮像部であるカメラ５は、ボディ１の前方に配置され、ボディ１の前方を撮影する。具体的には、カメラ５は、ボディ１の前方の動画または静止画を撮像する第１のセンサを構成する。これにより、本実施の形態の自律走行型掃除機は、カメラ５が撮像した情報に基づいて、制御ユニット１００で、ボディ１の前方の物体の有無を判定することが可能となる。 The camera 5, which is an imaging unit, is arranged in front of the body 1 and photographs the front of the body 1. Specifically, the camera 5 constitutes a first sensor that captures a moving image or a still image in front of the body 1. As a result, in the autonomous driving type vacuum cleaner of the present embodiment, the control unit 100 can determine the presence or absence of an object in front of the body 1 based on the information captured by the camera 5.
　測距センサ６は、例えば赤外線センサなどで構成され、ボディ１の前方の左右に配置され、第２のセンサを構成する。測距センサ６は、障害物などの物体までの距離を検知する。本実施の形態では、測距センサ６は、発光素子と受光素子からなる光学式のセンサが用いられる。なお、測距センサ６は、上記赤外線センサ以外に、例えばレーザセンサなどを用いて構成してもよい。 The distance measuring sensor 6 is composed of, for example, an infrared sensor, and is arranged on the left and right in front of the body 1 to form a second sensor. The distance measuring sensor 6 detects the distance to an object such as an obstacle. In the present embodiment, the distance measuring sensor 6 is an optical sensor including a light emitting element and a light receiving element. In addition to the infrared sensor, the ranging sensor 6 may be configured by using, for example, a laser sensor.
　制御ユニット１００は、ボディ１内部に配置され、例えばＣＰＵ（Ｃｅｎｔｒａｌ　Ｐｒｏｃｅｓｓｉｎｇ　Ｕｎｉｔ）のような半導体素子で構成される。なお、制御ユニット１００は、上記半導体素子そのもの、あるいは、上記半導体素子を搭載した回路基板全体を指す場合も含む。 The control unit 100 is arranged inside the body 1 and is composed of semiconductor elements such as a CPU (Central Processing Unit), for example. The control unit 100 includes the case where the semiconductor element itself or the entire circuit board on which the semiconductor element is mounted is referred to.
　以上のように、本実施の形態の自律走行型掃除機は構成される。 As described above, the autonomous driving type vacuum cleaner of the present embodiment is configured.
　以下、自律走行型掃除機の第１のセンサであるカメラ５および第２のセンサである測距センサ６について、図２を参照しながら、説明する。 Hereinafter, the camera 5 which is the first sensor of the autonomous driving type vacuum cleaner and the distance measuring sensor 6 which is the second sensor will be described with reference to FIG.
　図２は、本実施の形態における自律走行型掃除機の側面図である。なお、図２では、関連のある構成の一部を、破線で示している。 FIG. 2 is a side view of the autonomous driving type vacuum cleaner according to the present embodiment. In FIG. 2, a part of the related configurations is shown by a broken line.
　図２に示すように、測距センサ６は、ボディ１の前面近傍の左右（図１参照）に搭載される。このとき、測距センサ６は、センサ軸ＳＡが、ボディ１前面から前方の距離Ｄの床面８に向くように配置される。 As shown in FIG. 2, the distance measuring sensors 6 are mounted on the left and right (see FIG. 1) near the front surface of the body 1. At this time, the distance measuring sensor 6 is arranged so that the sensor shaft SA faces the floor surface 8 at a distance D in front of the front surface of the body 1.
　制御ユニット１００は、測距センサ６から得られる測距値を監視する。これにより、例えばケーブルなどによる、ボディ１の前進方向の床面８までの高さの変化を検知できる。同時に、高さの変化が起こる地点から自律走行型掃除機までの距離Ｄが分かる。 The control unit 100 monitors the distance measurement value obtained from the distance measurement sensor 6. As a result, it is possible to detect a change in height of the body 1 up to the floor surface 8 in the forward direction due to, for example, a cable. At the same time, the distance D from the point where the height change occurs to the autonomous driving type vacuum cleaner can be known.
　カメラ５は、例えば測距センサ６の上方側のボディ１に配置される。なお、本実施の形態では、カメラ５を測距センサ６の上方側に配置した例で説明するが、これに限られない。例えば、カメラ５を、測距センサ６の下方側、あるいは、左右側の少なくとも一方に配置してもよい。 The camera 5 is arranged on the body 1 on the upper side of the distance measuring sensor 6, for example. In the present embodiment, the example in which the camera 5 is arranged above the distance measuring sensor 6 will be described, but the present invention is not limited to this. For example, the camera 5 may be arranged on the lower side of the distance measuring sensor 6 or at least one of the left and right sides.
　つぎに、自律走行型掃除機を含むシステムについて、図３および図４を参照しながら、説明する。 Next, the system including the autonomous driving type vacuum cleaner will be described with reference to FIGS. 3 and 4.
　図３は、本実施の形態にかかる自律走行型掃除機を含むシステムのブロック図である。図４は、本実施の形態のシステムにおける物体認識アルゴリズムの認識結果の一例を示す図である。 FIG. 3 is a block diagram of a system including an autonomous driving type vacuum cleaner according to the present embodiment. FIG. 4 is a diagram showing an example of the recognition result of the object recognition algorithm in the system of the present embodiment.
　図３に示すように、システムは、制御ユニット１００を内蔵する自律走行型掃除機（一点鎖線で囲まれる領域）と、学習モデル生成装置を構成するＰＣ（Ｐｅｒｓｏｎａｌ　Ｃｏｍｐｕｔｅｒ）２００などで構成される。 As shown in FIG. 3, the system is composed of an autonomous driving type vacuum cleaner (area surrounded by a single point chain line) having a built-in control unit 100, a PC (Personal Computer) 200 constituting a learning model generator, and the like.
　具体的には、制御ユニット１００は、主に、第１の物体認識機能部を構成する第１ブロック１０１と、第２の物体認識機能部を構成する第２ブロック１０２などを有する。第１ブロック１０１は、物体認識アルゴリズムを用いて、カメラ５などの撮像部で撮像された画像から、物体を認識するためのブロックを構成する。第２ブロック１０２は、第１ブロック１０１の認識結果から、さらに正確に物体を認識するまでの各種処理を行うためのブロックを構成する。なお、第２ブロック１０２の詳細は、後述する。 Specifically, the control unit 100 mainly has a first block 101 that constitutes a first object recognition function unit, a second block 102 that constitutes a second object recognition function unit, and the like. The first block 101 constitutes a block for recognizing an object from an image captured by an imaging unit such as a camera 5 by using an object recognition algorithm. The second block 102 constitutes a block for performing various processes from the recognition result of the first block 101 to more accurately recognizing the object. The details of the second block 102 will be described later.
　ＰＣ２００は、いわゆるパーソナルコンピュータである。ＰＣ２００は、後述するように、予め、複数の学習用画像データを、メモリなどに格納している。そして、ＰＣ２００は、格納された学習用画像データから物体認識アルゴリズムを用いて、物体の特徴抽出を行い、学習を行った後に、学習モデルを生成する。つまり、ＰＣ２００は、上述した学習モデル生成装置を構成する。 The PC200 is a so-called personal computer. As will be described later, the PC 200 stores a plurality of learning image data in a memory or the like in advance. Then, the PC 200 uses an object recognition algorithm to extract the features of the object from the stored image data for learning, performs learning, and then generates a learning model. That is, the PC 200 constitutes the learning model generation device described above.
　また、自律走行型掃除機の制御ユニット１００とＰＣ２００は、例えばＵＳＢ（Ｕｎｉｖｅｒｓａｌ　Ｓｅｒｉａｌ　Ｂｕｓ）ケーブルなどを介して、接続可能に構成される。これにより、ＰＣ２００で生成された学習モデルが、自律走行型掃除機の制御ユニット１００に送信され、自律走行型掃除機での受信が可能となる。 Further, the control unit 100 and the PC 200 of the autonomous driving type vacuum cleaner are configured to be connectable via, for example, a USB (Universal Serial Bus) cable or the like. As a result, the learning model generated by the PC 200 is transmitted to the control unit 100 of the autonomous driving vacuum cleaner, and can be received by the autonomous driving vacuum cleaner.
　制御ユニット１００は、例えばフラッシュメモリなどの不揮発性メモリ（図示せず）などで構成される記憶部１０３を備える。記憶部１０３は、ＰＣ２００から受信した学習モデルを記憶する。なお、学習モデルの容量が多くなる程、必要なメモリの容量も増大する。そのため、記憶部１０３として、不揮発性メモリなどの半導体メモリではなく、例えばハードディスクのような大容量の記憶媒体を用いてもよい。 The control unit 100 includes a storage unit 103 composed of, for example, a non-volatile memory (not shown) such as a flash memory. The storage unit 103 stores the learning model received from the PC 200. As the capacity of the learning model increases, the required memory capacity also increases. Therefore, as the storage unit 103, a large-capacity storage medium such as a hard disk may be used instead of a semiconductor memory such as a non-volatile memory.
　また、図３に示すＰＣ２００の各ブロックを、例えばサーバーなどに設ける構成としてもよい。つまり、ネットワークを介して、サーバーと制御ユニット１００とを接続する構成としてもよい。この場合、サーバーが学習モデル生成装置を構成し、サーバーから制御ユニット１００に、サーバーで生成された学習モデルが送信されることになる。 Further, each block of the PC200 shown in FIG. 3 may be provided in, for example, a server. That is, the server and the control unit 100 may be connected to each other via a network. In this case, the server configures the learning model generator, and the server sends the learning model generated by the server to the control unit 100.
　さらに、本実施の形態の自律走行型掃除機は、第１ブロック１０１で実行される物体認識アルゴリズムと、測距センサ６の検知結果とに基づいて、物体（対象物１０）の認識を行う。 Further, the autonomous traveling type vacuum cleaner of the present embodiment recognizes an object (object 10) based on the object recognition algorithm executed in the first block 101 and the detection result of the distance measuring sensor 6.
　以下、本実施の形態の自律走行型掃除機を含むシステムの物体認識方法について、説明する。 Hereinafter, the object recognition method of the system including the autonomous driving type vacuum cleaner of the present embodiment will be described.
　本実施の形態のシステムは、ＰＣ２００側の物体認識アルゴリズムにより、自律走行型掃除機が巻き込み、清掃動作が中断されやすい物体の学習モデルを生成する。具体的には、まず、予め、清掃動作が中断されやすい物体である敷物、ケーブルなどの多数の学習用画像データを取り込んで、メモリ（図示せず）に記憶する。取り込んだ学習用画像データから、物体認識アルゴリズムを用いて、物体の特徴抽出を行い、学習する。そして、ＰＣ２００は、学習モデルを生成する。なお、上記処理は、学習モデルを生成する第１のステップに相当する。 The system of this embodiment uses an object recognition algorithm on the PC200 side to generate a learning model of an object in which an autonomous driving vacuum cleaner is involved and the cleaning operation is likely to be interrupted. Specifically, first, a large amount of learning image data such as a rug or a cable, which is an object whose cleaning operation is easily interrupted, is taken in in advance and stored in a memory (not shown). From the captured image data for learning, the feature extraction of the object is performed using the object recognition algorithm and learning is performed. Then, the PC 200 generates a learning model. The above process corresponds to the first step of generating the learning model.
　つぎに、ＰＣ２００側から、自律走行型掃除機の制御ユニット１００に、生成した学習モデルを送信し、自律走行型掃除機のメモリに記憶させる。制御ユニット１００への送信は、上述したように、例えばＵＳＢケーブルなどでＰＣ２００と制御ユニット１００を接続することにより実行される。 Next, the generated learning model is transmitted from the PC 200 side to the control unit 100 of the autonomous driving vacuum cleaner, and stored in the memory of the autonomous driving vacuum cleaner. As described above, transmission to the control unit 100 is executed by connecting the PC 200 and the control unit 100 with, for example, a USB cable or the like.
　さらに、本実施の形態の自律走行型掃除機は、清掃動作中に、カメラ５で自律走行型掃除機の前方を、例えば動画または静止画の形態で、撮影する。そして、カメラ５は、撮影した画像データを、制御ユニット１００の第１ブロック１０１に送信する。 Further, the autonomous driving vacuum cleaner of the present embodiment captures the front of the autonomous driving vacuum cleaner with the camera 5 in the form of a moving image or a still image, for example, during the cleaning operation. Then, the camera 5 transmits the captured image data to the first block 101 of the control unit 100.
　そして、制御ユニット１００の第１ブロックは、送信された画像データから、物体認識アルゴリズムを用いて、物体の特徴抽出を行う。 Then, the first block of the control unit 100 extracts the features of the object from the transmitted image data by using the object recognition algorithm.
　このとき、制御ユニット１００は、上述したように、ＰＣ２００側で、それぞれの対象物１０の特徴を抽出して学習し、学習した学習モデルを、予め、不揮発性メモリなどの記憶部１０３に記憶している。 At this time, as described above, the control unit 100 extracts and learns the features of each object 10 on the PC200 side, and stores the learned learning model in advance in a storage unit 103 such as a non-volatile memory. ing.
　そこで、図３に示すように、第１ブロックで特徴抽出した物体と、記憶部１０３に記憶されている学習モデルの対象物１０とを比較して、物体が対象物１０か否かを判断する。 Therefore, as shown in FIG. 3, the object whose features have been extracted in the first block is compared with the object 10 of the learning model stored in the storage unit 103, and it is determined whether or not the object is the object 10. ..
　そして、図４に示すように、カメラ５から送信された画像データ９内の物体から、予め学習した種類の対象物１０（例えば敷物）の有無と、画像データ９内の、どの位置に対象物１０が存在するか、を含む検知枠１１を、画像データ９内に出力して表示する。なお、物体が、学習した対象物で無い場合、検知枠を表示しない。つまり、カメラ５から送信された画像データ９の物体の中に予め学習した対象物１０が存在すると、検知枠１１を表示して、ユーザに知らせる。 Then, as shown in FIG. 4, from the object in the image data 9 transmitted from the camera 5, the presence / absence of the object 10 (for example, a rug) of the type learned in advance and the position of the object in the image data 9 The detection frame 11 including whether or not 10 exists is output and displayed in the image data 9. If the object is not a learned object, the detection frame is not displayed. That is, when the object 10 learned in advance exists in the object of the image data 9 transmitted from the camera 5, the detection frame 11 is displayed to notify the user.
　さらに、画像データ９内に表示された検知枠１１の下端（検知枠１１の最下端に相当）から、画像データ９の下端までの長さを、画像データの画素数などから算出する。これにより、対象物１０までの距離Ｘが算出される。 Further, the length from the lower end of the detection frame 11 (corresponding to the lowermost end of the detection frame 11) displayed in the image data 9 to the lower end of the image data 9 is calculated from the number of pixels of the image data and the like. As a result, the distance X to the object 10 is calculated.
　なお、画像データ９から距離Ｘを取得する処理は、通常、制御ユニット１００の内部で行われる処理である。そのため、必ずしも、画像データ９を出力して、表示し、使用者が確認するように構成する必要はない。 The process of acquiring the distance X from the image data 9 is usually a process performed inside the control unit 100. Therefore, it is not always necessary to output the image data 9, display it, and configure it so that the user can confirm it.
　上記により、制御ユニット１００は、第１の物体認識機能部を構成する第１ブロック１０１で、対象物１０が認識される。なお、上記処理は、画像から対象物１０を認識する第２のステップに相当する。 According to the above, the control unit 100 recognizes the object 10 in the first block 101 constituting the first object recognition function unit. The above process corresponds to the second step of recognizing the object 10 from the image.
　つぎに、制御ユニット１００は、測距センサ６で、対象物１０の距離を測距する。そして、第２ブロック１０２で、測距した、第２のステップで認識した対象物１０との距離に基づいて、対象物１０を認識する。なお、上記処理は、対象物１０との距離から対象物１０を認識する第３のステップに相当する。 Next, the control unit 100 measures the distance of the object 10 with the distance measuring sensor 6. Then, in the second block 102, the object 10 is recognized based on the distance measured and the distance to the object 10 recognized in the second step. The above process corresponds to the third step of recognizing the object 10 from the distance from the object 10.
　詳しくは、制御ユニット１００の第２ブロック１０２は、まず、自律走行型掃除機から対象物１０までの距離Ｘが、ｘ以上か否かを判定する。このとき、距離Ｘが、ｘ未満と判定された場合、制御ユニット１００は、駆動部１２を制御する。具体的には、制御ユニット１００は、自律走行型掃除機を減速させるように、駆動部１２を制御する。 Specifically, the second block 102 of the control unit 100 first determines whether or not the distance X from the autonomous driving type vacuum cleaner to the object 10 is x or more. At this time, if the distance X is determined to be less than x, the control unit 100 controls the drive unit 12. Specifically, the control unit 100 controls the drive unit 12 so as to decelerate the autonomous traveling type vacuum cleaner.
　つぎに、第２ブロック１０２は、測距センサ６で検知した出力値に対応するセンサ値Ｓの取得を開始する。 Next, the second block 102 starts to acquire the sensor value S corresponding to the output value detected by the distance measuring sensor 6.
　つぎに、第２ブロック１０２は、自律走行型掃除機の減速走行中のセンサ値Ｓの平均を常に取得し、その平均の値をＳａｖｅとして、メモリ（図示せず）に記憶する。なお、メモリは、記憶部１０３と兼用してもよい。このとき、上述したように、測距センサ６のセンサ軸ＳＡは、前進方向の床面８を向いている。そのため、床面８に敷かれている敷物などにより、自律走行型掃除機の走行面の高さが変化すると、センサ軸ＳＡの方向が変化するため、検知する測距センサ６のセンサ値Ｓに変化が起こる。このセンサ値Ｓの変化は、自律走行型掃除機の誤動作の要因となる。 Next, the second block 102 constantly acquires the average of the sensor values S during deceleration traveling of the autonomous driving type vacuum cleaner, and stores the average value as Save in a memory (not shown). The memory may also be used as the storage unit 103. At this time, as described above, the sensor axis SA of the distance measuring sensor 6 faces the floor surface 8 in the forward direction. Therefore, when the height of the traveling surface of the autonomous traveling vacuum cleaner changes due to the rug laid on the floor surface 8, the direction of the sensor axis SA changes, so that the sensor value S of the distance measuring sensor 6 is detected. Change happens. This change in the sensor value S causes a malfunction of the autonomous driving type vacuum cleaner.
　そこで、第２ブロック１０２は、予め、センサ値Ｓの変化（変動）を想定して、所定の閾値Ｔを設定している。 Therefore, the second block 102 sets a predetermined threshold value T in advance assuming a change (fluctuation) in the sensor value S.
　そして、第２ブロック１０２は、減速中に検知した１つ前までのセンサ値Ｓの平均の値Ｓａｖｅと、最新のセンサ値Ｓの差分（Ｓａｖｅ―Ｓ）が、所定の閾値Ｔを超えるか否かを判断する。このとき、Ｓａｖｅ―Ｓ＞Ｔとなると、自律走行型掃除機の制御ユニット１００は、前進方向の距離Ｄの位置に、清掃動作が中断されやすい対象物１０があると判断する。これにより、対象物１０の巻き込みを未然に回避して、自律走行型掃除機の清掃動作の中断を防止できる。 Then, in the second block 102, whether or not the difference (Save-S) between the average value Save of the previous sensor values S detected during deceleration and the latest sensor value S exceeds a predetermined threshold value T. To judge. At this time, when Save-S> T, the control unit 100 of the autonomous traveling type vacuum cleaner determines that the object 10 whose cleaning operation is likely to be interrupted is located at the position of the distance D in the forward direction. As a result, it is possible to prevent the object 10 from being caught in the object 10 and prevent the cleaning operation of the autonomous traveling vacuum cleaner from being interrupted.
　なお、自律走行型掃除機の清掃動作中において、例えばゴミなどにより測距センサ６で検知するセンサ値Ｓが変動する。その際、変動するセンサ値Ｓにより、第２ブロック１０２が、前方に、例えば絨毯などがあると誤検知する虞がある。そこで、閾値Ｔを設定して、閾値Ｔ以下であれば、例えば絨毯などの対象物１０ではないと判断して、変動による誤検知を回避している。これにより、対象物１０の検知精度を向上させることができる。 During the cleaning operation of the autonomous driving type vacuum cleaner, the sensor value S detected by the distance measuring sensor 6 fluctuates due to, for example, dust. At that time, the fluctuating sensor value S may cause the second block 102 to erroneously detect that there is, for example, a carpet in front of it. Therefore, the threshold value T is set, and if it is equal to or less than the threshold value T, it is determined that the object 10 is not, for example, a carpet, and false detection due to fluctuation is avoided. Thereby, the detection accuracy of the object 10 can be improved.
　また、上記実施の形態では、より確実に対象物１０の有無を判断するために、物体認識アルゴリズムで対象物１０を認識した後、駆動輪３を制御して減速動作を行う例で説明したが、これに限られない。上記閾値Ｔの設定により、減速動作を行わなくとも、自律走行型掃除機は、前進方向の距離Ｄの位置に清掃動作が中断されやすい対象物１０があると、適切に判断できる。これにより、自律走行型掃除機の清掃動作の効率化が図れる。 Further, in the above embodiment, in order to more reliably determine the presence / absence of the object 10, an example of performing a deceleration operation by controlling the drive wheels 3 after recognizing the object 10 by the object recognition algorithm has been described. , Not limited to this. By setting the threshold value T, the autonomous driving type vacuum cleaner can appropriately determine that there is an object 10 at a position of a distance D in the forward direction in which the cleaning operation is likely to be interrupted, even if the deceleration operation is not performed. As a result, the efficiency of the cleaning operation of the autonomous traveling type vacuum cleaner can be improved.
　また、上記実施の形態では、第１ブロック１０１で物体認識アルゴリズムを用いて物体を認識し、第２ブロック１０２で測距センサ６のセンサ値Ｓを取得する例で説明したが、これに限られない。例えば、物体認識アルゴリズムを用いず、自律走行型掃除機の走行中、常に、センサ値Ｓを取得して、走行面の高さが変化する距離を検知することは、可能である。しかしながら、この場合、走行面の凹凸や塵埃などの物体と、自律走行型掃除機の清掃動作が中断されやすい対象物１０とを区別できず、対象物１０を正確に判断できない虞がある。そのため、第１ブロック１０１で物体認識アルゴリズムを用いて、物体から対象物１０を識別し、識別した対象物１０に対して、測距センサ６で取得したセンサ値Ｓに基づいて、対象物１０を認識する構成が、より望ましい。 Further, in the above embodiment, an example has been described in which the first block 101 recognizes an object using the object recognition algorithm and the second block 102 acquires the sensor value S of the distance measuring sensor 6, but the present invention is limited to this. Absent. For example, it is possible to always acquire the sensor value S and detect the distance at which the height of the traveling surface changes while the autonomous traveling vacuum cleaner is traveling without using the object recognition algorithm. However, in this case, it is not possible to distinguish between an object such as unevenness or dust on the traveling surface and the object 10 in which the cleaning operation of the autonomous traveling vacuum cleaner is likely to be interrupted, and there is a possibility that the object 10 cannot be accurately determined. Therefore, the object 10 is identified from the object by using the object recognition algorithm in the first block 101, and the object 10 is assigned to the identified object 10 based on the sensor value S acquired by the distance measuring sensor 6. A recognizable configuration is more desirable.
　また、測距センサ６を使用せずに、図４に示す画像データ９から距離Ｘを取得して、対象物１０の位置を算出することも可能である。しかしながら、図４に示す検知枠１１は、物体認識技術の性質上、必ずしも対象物１０の下端と検知枠１１の下端とが一致するわけではない。さらに、実際に対象物が存在するのか、例えば窓から差し込む光が床面に反射して、敷物であるかのように見えているだけなのか、の判断が困難で、対象物を誤認識する可能性もある。そこで、本実施の形態では、測距センサ６で走行面の高さの変化を検知して、さらに対象物１０の有無を判断する。これにより、誤認識を防止して、より正確に対象物１０を検知できる。 It is also possible to acquire the distance X from the image data 9 shown in FIG. 4 and calculate the position of the object 10 without using the distance measuring sensor 6. However, in the detection frame 11 shown in FIG. 4, the lower end of the object 10 and the lower end of the detection frame 11 do not always coincide with each other due to the nature of the object recognition technology. Furthermore, it is difficult to determine whether the object actually exists, for example, whether the light coming in through the window is reflected on the floor and looks like a rug, and the object is misrecognized. There is a possibility. Therefore, in the present embodiment, the distance measuring sensor 6 detects a change in the height of the traveling surface, and further determines the presence or absence of the object 10. As a result, erroneous recognition can be prevented and the object 10 can be detected more accurately.
　つまり、本実施の形態の自律走行型掃除機は、従来の技術では検知することが難しかった敷物、ケーブルなどのような高さが低く、自律走行型掃除機が巻き込み、清掃動作が中断されやすい対象物を、正確に検知できる。これにより、清掃行動が中断されない自律走行型掃除機を提供できる。 That is, the autonomous traveling type vacuum cleaner of the present embodiment has a low height such as a rug, a cable, etc., which was difficult to detect by the conventional technology, and the autonomous traveling type vacuum cleaner is likely to be involved and the cleaning operation is easily interrupted. The object can be detected accurately. This makes it possible to provide an autonomous traveling type vacuum cleaner in which the cleaning action is not interrupted.
　以上で説明したように、本実施の形態の自律走行型掃除機は、物体の有無を検知する第１のセンサと、物体の位置を検知する第２のセンサと、制御ユニットを有する。制御ユニットは、第１のセンサの検知結果と、第２のセンサの検知結果とに基づいて、物体を認識するように構成される。 As described above, the autonomous driving type vacuum cleaner of the present embodiment has a first sensor for detecting the presence or absence of an object, a second sensor for detecting the position of the object, and a control unit. The control unit is configured to recognize an object based on the detection result of the first sensor and the detection result of the second sensor.
　この構成によれば、高さが低く、検知が難しい、敷物、ケーブルなどのような、清掃動作が中断されやすい物体を、正確に検知できる。これにより、高さの低い物体の自律走行型掃除機への巻き込みを未然に回避して、清掃行動の中断をできるだけ回避することができる。 According to this configuration, it is possible to accurately detect objects such as rugs, cables, etc., which are low in height and difficult to detect, and whose cleaning operation is likely to be interrupted. As a result, it is possible to prevent an object having a low height from being caught in the autonomous driving type vacuum cleaner and to avoid interruption of the cleaning action as much as possible.
　また、本実施の形態の自律走行型掃除機の第１のセンサは、動画または静止画を撮像する撮像部で構成され、制御ユニットは、撮像部により撮像した画像から物体の認識を行う物体認識アルゴリズムを備える構成が望ましい。 Further, the first sensor of the autonomous traveling type vacuum cleaner of the present embodiment is composed of an imaging unit that captures a moving image or a still image, and the control unit recognizes an object from the image captured by the imaging unit. A configuration with an algorithm is desirable.
　また、本実施の形態の自律走行型掃除機は、ボディに配設される２つの駆動輪を駆動する駆動ユニットを、さらに有し、第２のセンサは、物体までの距離を測定する測距センサで構成される。そして、制御ユニットは、物体認識アルゴリズムにより物体を認識すると、測距センサからの出力値の平均値を算出しながら、駆動ユニットを駆動してボディを前進させ、平均値と、測距センサからの最新の出力値との差分が閾値を上回ると、物体の有無と、物体までの距離を算出する構成が望ましい。 Further, the autonomous traveling type vacuum cleaner of the present embodiment further has a drive unit for driving two drive wheels arranged on the body, and the second sensor measures the distance to the object. It consists of sensors. Then, when the control unit recognizes the object by the object recognition algorithm, it drives the drive unit to move the body forward while calculating the average value of the output values from the distance measuring sensor, and the average value and the distance measuring sensor are used. When the difference from the latest output value exceeds the threshold value, it is desirable to calculate the presence / absence of an object and the distance to the object.
　また、本実施の形態のシステムは、学習用画像データから、物体認識アルゴリズムを用いて、学習モデルを生成する学習モデル生成装置と、学習モデル生成装置から送信される学習モデルを受信する自律走行型掃除機を有する。自律走行型掃除機は、学習モデル生成装置から受信した学習モデルと、撮像部で撮像した画像とを用いて、物体を認識する第１の物体認識機能部と、第１の物体認識機能部で認識した物体との距離に基づいて、物体を認識する第２の物体認識機能部を有する構成が望ましい。 Further, the system of the present embodiment is an autonomous traveling type that receives a learning model generator that generates a learning model from the learning image data using an object recognition algorithm and a learning model transmitted from the learning model generator. Have a vacuum cleaner. The autonomous traveling type vacuum cleaner has a first object recognition function unit that recognizes an object and a first object recognition function unit that recognizes an object by using the learning model received from the learning model generator and the image captured by the imaging unit. It is desirable to have a configuration having a second object recognition function unit that recognizes the object based on the distance from the recognized object.
　また、本実施の形態の物体認識方法は、学習用画像データから特徴を抽出して、学習し、学習モデルを生成する第１のステップと、生成された学習モデルと撮像部で撮像された画像データから物体を認識する第２のステップと、第２のステップで認識した物体との距離に基づいて、物体を認識する第３のステップを有する構成が望ましい。 Further, in the object recognition method of the present embodiment, the first step of extracting features from the learning image data, learning, and generating a learning model, and the generated learning model and the image captured by the imaging unit. It is desirable to have a configuration having a second step of recognizing an object from data and a third step of recognizing an object based on the distance between the object recognized in the second step.
　本発明は、家庭用の自律走行型掃除機や業務用の自律走行型掃除機などに広く適用できる。 The present invention can be widely applied to home-use autonomous driving vacuum cleaners, commercial-use autonomous driving vacuum cleaners, and the like.
　１　　ボディ
　２　　吸込口
　３　　駆動輪
　４　　サイドブラシ
　５　　カメラ（撮像部）（第１のセンサ）
　６　　測距センサ（第２のセンサ）
　８　　床面
　９　　画像データ
　１０　　対象物
　１１　　検知枠
　１２　　駆動部
　１００　　制御ユニット
　１０１　　第１ブロック（第１の物体認識機能部）
　１０２　　第２ブロック（第２の物体認識機能部）
　１０３　　記憶部
　２００　　ＰＣ（学習モデル生成装置） 1 Body 2 Suction port 3 Drive wheel 4 Side brush 5 Camera (imaging unit) (1st sensor)
6 Distance measurement sensor (second sensor)
8 Floor surface 9 Image data 10 Object 11 Detection frame 12 Drive unit 100 Control unit 101 1st block (1st object recognition function unit)
102 Second block (second object recognition function unit)
103 Storage unit 200 PC (learning model generator)

Claims (5)

  1. 物体の有無を検知する第１のセンサと、
    前記物体の位置を検知する第２のセンサと、
    制御ユニットと、を有し、
    前記制御ユニットは、前記第１のセンサの検知結果と前記第２のセンサの検知結果に基づいて、前記物体を認識する、
    自律走行型掃除機。     The first sensor that detects the presence or absence of an object,
    A second sensor that detects the position of the object,
    With a control unit,
    The control unit recognizes the object based on the detection result of the first sensor and the detection result of the second sensor.
    Autonomous vacuum cleaner.
  2. 前記第１のセンサは、動画または静止画を撮像する撮像部で構成され、
    前記制御ユニットは、前記撮像部により撮像した画像から前記物体の認識を行う物体認識アルゴリズムを備える、
    請求項１に記載の自律走行型掃除機。     The first sensor is composed of an imaging unit that captures a moving image or a still image.
    The control unit includes an object recognition algorithm that recognizes the object from an image captured by the image pickup unit.
    The autonomous driving type vacuum cleaner according to claim 1.
  3. ボディに配設される２つの駆動輪を駆動する駆動ユニットを、さらに有し、
    前記第２のセンサは、前記物体までの距離を測定する測距センサで構成され、
    前記制御ユニットは、前記物体認識アルゴリズムにより物体を認識すると、前記測距センサからの出力値の平均値を算出しながら、前記駆動ユニットを駆動して前記ボディを前進させ、前記平均値と前記測距センサからの最新の出力値との差分が閾値を上回ると、物体の有無と、前記物体までの距離を算出する、
    請求項２に記載の自律走行型掃除機。     It also has a drive unit that drives two drive wheels arranged on the body.
    The second sensor is composed of a distance measuring sensor that measures the distance to the object.
    When the control unit recognizes an object by the object recognition algorithm, the control unit drives the drive unit to advance the body while calculating the average value of the output values from the distance measuring sensor, and the average value and the measurement are performed. When the difference from the latest output value from the distance sensor exceeds the threshold value, the presence / absence of an object and the distance to the object are calculated.
    The autonomous driving type vacuum cleaner according to claim 2.
  4. 学習用画像データから、物体認識アルゴリズムを用いて、学習モデルを生成する学習モデル生成装置と、
    前記学習モデル生成装置から送信される前記学習モデルを受信する自律走行型掃除機と、を有するシステムであって、
    前記自律走行型掃除機は、前記学習モデル生成装置から受信した前記学習モデルと、撮像部で撮像した画像とを用いて、物体を認識する第１の物体認識機能部と、
    前記第１の物体認識機能部で認識した前記物体との距離に基づいて、前記物体を認識する第２の物体認識機能部と、を有する、
    システム。     A learning model generator that generates a learning model from image data for training using an object recognition algorithm,
    A system including an autonomous traveling vacuum cleaner that receives the learning model transmitted from the learning model generator.
    The autonomous traveling type vacuum cleaner has a first object recognition function unit that recognizes an object by using the learning model received from the learning model generation device and an image captured by the imaging unit.
    It has a second object recognition function unit that recognizes the object based on the distance to the object recognized by the first object recognition function unit.
    system.
  5. 学習用画像データから特徴を抽出して、学習し、学習モデルを生成する第１のステップと、
    生成された前記学習モデルと撮像部で撮像された画像データから物体を認識する第２のステップと、
    前記第２のステップで認識した前記物体との距離に基づいて、前記物体を認識するステップと、を有する、
    物体認識方法。     The first step of extracting features from training image data, training them, and generating a learning model,
    The second step of recognizing an object from the generated learning model and the image data captured by the imaging unit, and
    It has a step of recognizing the object based on the distance to the object recognized in the second step.
    Object recognition method.
PCT/JP2020/010044 2019-04-18 2020-03-09 Self-propelled vacuum cleaner, system, and object recognition method WO2020213294A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019-078993 2019-04-18
JP2019078993A JP7261958B2 (en) 2019-04-18 2019-04-18 Autonomous vacuum cleaner

Publications (1)

Publication Number Publication Date
WO2020213294A1 true WO2020213294A1 (en) 2020-10-22

Family

ID=72837447

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/010044 WO2020213294A1 (en) 2019-04-18 2020-03-09 Self-propelled vacuum cleaner, system, and object recognition method

Country Status (2)

Country Link
JP (1) JP7261958B2 (en)
WO (1) WO2020213294A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003256041A (en) * 2002-02-28 2003-09-10 Matsushita Electric Ind Co Ltd Self-contained travel device, its monitoring method and remote control method
KR20180101081A (en) * 2017-03-03 2018-09-12 엘지전자 주식회사 Moving Robot and controlling method
US20180353042A1 (en) * 2017-06-08 2018-12-13 Samsung Electronics Co., Ltd. Cleaning robot and controlling method thereof

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3626724B2 (en) * 2001-12-14 2005-03-09 株式会社日立製作所 Self-propelled vacuum cleaner
JP6705636B2 (en) * 2015-10-14 2020-06-03 東芝ライフスタイル株式会社 Vacuum cleaner

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003256041A (en) * 2002-02-28 2003-09-10 Matsushita Electric Ind Co Ltd Self-contained travel device, its monitoring method and remote control method
KR20180101081A (en) * 2017-03-03 2018-09-12 엘지전자 주식회사 Moving Robot and controlling method
US20180353042A1 (en) * 2017-06-08 2018-12-13 Samsung Electronics Co., Ltd. Cleaning robot and controlling method thereof

Also Published As

Publication number Publication date
JP7261958B2 (en) 2023-04-21
JP2020174899A (en) 2020-10-29

Similar Documents

Publication Publication Date Title
TWI689387B (en) Mobile robot
KR102035018B1 (en) Apparatus for controlling cleaning function and robotic cleaner with the apparatus
EP3590665B1 (en) Mobile robot and control method therefor
US10350762B2 (en) Autonomously moving body, movement controlling method, and recording medium storing movement controlling program
EP3230814B1 (en) Using laser sensor for floor type detection
EP3549726B1 (en) Robotic vacuum cleaner, cleaning function control apparatus equipped in robotic vacuum cleaner, and multi-channel lidar-based obstacle detection apparatus equipped in robotic vacuum cleaner
KR102147207B1 (en) Moving Robot and controlling method
KR102201144B1 (en) Mobile robot and its control method
KR20190035376A (en) A robot cleaner and control method thereof
EP4177639A1 (en) Robotic vacuum cleaner with multichannel lidar-based obstacle detection and slip detection
KR102320678B1 (en) Moving Robot and controlling method
TWI748534B (en) Robot cleaner and method for controlling the same
WO2016096046A1 (en) Measuring brush roll current for determining type of surface
KR100820740B1 (en) Robot cleaner and control method of the same of
KR20210007360A (en) Moving Robot and controlling method
WO2020213294A1 (en) Self-propelled vacuum cleaner, system, and object recognition method
KR101371038B1 (en) Mobile robot and method for tracking target of the same
KR102300790B1 (en) Moving Robot and controlling method
KR102428214B1 (en) Moving Robot and controlling method
KR102135369B1 (en) Distance detecting apparatus and method thereof
WO2020059292A1 (en) Autonomous traveling cleaner
KR102318756B1 (en) Robot cleaner and method for controlling the same
KR20180082101A (en) Robot cleaner
KR102500525B1 (en) Moving robot
KR102364623B1 (en) Moving Robot and controlling method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20792160

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20792160

Country of ref document: EP

Kind code of ref document: A1