CN114111703A - Falling detection system and robot - Google Patents
Falling detection system and robot Download PDFInfo
- Publication number
- CN114111703A CN114111703A CN202111405460.9A CN202111405460A CN114111703A CN 114111703 A CN114111703 A CN 114111703A CN 202111405460 A CN202111405460 A CN 202111405460A CN 114111703 A CN114111703 A CN 114111703A
- Authority
- CN
- China
- Prior art keywords
- module
- falling
- image
- laser ranging
- detection system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 19
- 230000004927 fusion Effects 0.000 claims abstract description 19
- 238000007405 data analysis Methods 0.000 claims abstract description 13
- 238000000034 method Methods 0.000 claims abstract description 8
- 238000004458 analytical method Methods 0.000 claims description 12
- 238000005259 measurement Methods 0.000 claims description 11
- 238000013135 deep learning Methods 0.000 claims description 7
- 230000005540 biological transmission Effects 0.000 claims description 3
- 230000009191 jumping Effects 0.000 claims description 3
- 238000004891 communication Methods 0.000 abstract description 2
- 230000000007 visual effect Effects 0.000 abstract description 2
- 230000007547 defect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000010191 image analysis Methods 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010408 sweeping Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C3/00—Measuring distances in line of sight; Optical rangefinders
Landscapes
- Physics & Mathematics (AREA)
- Electromagnetism (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Manipulator (AREA)
Abstract
The invention provides a falling detection system and a robot, relating to the technical field of robots, wherein the method comprises the following steps: a drop sensor module: the device comprises a laser ranging module and a monocular camera module; the laser ranging module measures the distance from the reflecting medium to the laser ranging module, and the monocular camera module shoots an image of a ranging area of the laser module; the sensor data analysis and fusion module: the initial judgment of the falling condition is carried out by receiving the data of the laser ranging module, and the reliability of the falling judgment is ensured by further analyzing the image data if the falling condition is considered. The invention can enable the falling condition to be more visual, facilitates communication between engineers and clients, and improves the reliability of falling judgment.
Description
Technical Field
The invention relates to the technical field of robots, in particular to a falling detection system and a robot.
Background
In order to avoid falling, the robots are provided with falling sensors, and when the falling sensors of the sweeping robot are suspended, the robot must stop moving immediately or decelerate at a higher deceleration.
The existing mainstream falling sensor mainly has two types of depth cameras and laser ranging modules, but the two types of sensors have the following defects respectively:
1. the depth camera is high in cost and low in utilization rate when used as a drop sensor, and large in consumption of computing resources;
2. although the cost of the laser ranging module is low, the amount of falling information is small, and only ranging information is provided, so that an engineer can not conveniently confirm the falling environment;
3. the laser ranging module has a small detection range and is easily interfered by ground reflection.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a falling detection system and a robot.
According to the falling detection system and the robot provided by the invention, the scheme is as follows:
in a first aspect, the present invention provides a fall detection system, the system comprising: a drop sensor module: the device comprises a laser ranging module and a monocular camera module; the laser ranging module measures the distance from the reflecting medium to the laser ranging module, and the monocular camera module shoots an image of a ranging area of the laser module;
the sensor data analysis and fusion module: the initial judgment of the falling condition is carried out by receiving the data of the laser ranging module, and the reliability of the falling judgment is ensured by further analyzing the image data if the falling condition is considered.
Preferably, the number of the laser ranging modules is two, each laser ranging module is provided with a sensor, and the two laser ranging modules respectively output ranging values of the two sensors dist1[ t ] and dist2[ t ] at time t.
Preferably, the monocular camera module captures an image of a ranging area of the laser module, and the image output at time t is image [ t ].
Preferably, the fall sensor module transmits the ranging values dist1[ t ], dist2[ t ] and the image [ t ] to the sensor data analysis and fusion module in real time.
Preferably, the sensor data analysis and fusion module has the following working flow:
step S1: dist1 t, dist2 t and image t for receiving the transmission of the falling module;
step S2: judging whether the distance measurement values of dist1[ t ] and dist2[ t ] are larger than the sum of the ground height value H and the falling threshold value thresh;
step S3: analyzing the image data image [ t ], calling a deep learning module in the sensor data fusion analysis module, and identifying whether relevant conditions including depressions and steps exist in the image [ t ] by adopting a deep learning method;
step S4: combining the falling distance measurement position, the distance measurement value dist1[ t ], dist2[ t ] and the image [ t ], and outputting a result: there is a risk of falling and the results are passed to an after-market engineer for analysis.
Preferably, the step S2 specifically includes: judging whether the distance measurement values of dist1[ t ] and dist2[ t ] are larger than the sum of the ground height value H and the falling threshold value thresh;
if the value is larger than the drop value, jumping to step S3; otherwise, outputting the result: and no falling risk exists.
Preferably, in step S3, it is identified whether there is a depression or a step in the image [ t ];
if yes, outputting a result: no falling risk exists; otherwise, go to step S4 for manual analysis.
In a second aspect, the invention also provides a robot comprising the system.
Compared with the prior art, the invention has the following beneficial effects:
1. the invention enriches the falling information, makes the falling condition more intuitive and facilitates the communication between engineers and clients;
2. the invention improves the reliability of the drop judgment in a low-cost and low-resource occupation mode.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:
FIG. 1 is a three-view diagram of a drop module;
FIG. 2 is a schematic view of a drop module installation method;
FIG. 3 is a schematic diagram of the detection range of the drop module;
fig. 4 is a combined view of fall range finding.
Detailed Description
The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that it would be obvious to those skilled in the art that various changes and modifications can be made without departing from the spirit of the invention. All falling within the scope of the present invention.
The embodiment of the invention provides a falling detection system, which comprises: the system comprises a drop sensor module, a sensor data analysis and fusion module and a sensor data analysis module.
Referring to fig. 1 and 2, a three-view and an installation schematic view of the drop sensor module are shown, and referring to fig. 3, a detection range schematic view is shown. Specifically, the drop sensor module: the device comprises a laser ranging module and a monocular camera module; the laser ranging module measures the distance from the reflecting medium to the laser ranging module, and the monocular camera module shoots an image of a ranging area of the laser module;
the sensor data analysis and fusion module: the initial judgment of the falling condition is carried out by receiving the data of the laser ranging module, and the reliability of the falling judgment is ensured by further analyzing the image data if the falling condition is considered.
The sensor data fusion analysis module is responsible for receiving laser ranging values and images transmitted by the falling sensor module, and the fusion laser ranging module and the images are used for comprehensively analyzing and judging the falling condition.
Specifically, in the drop sensor module, there are two laser ranging modules, each of which is provided with a sensor, and the two laser ranging modules respectively output ranging values dist1[ t ] and dist2[ t ] of the two sensors at time t. And the monocular camera module is used for shooting an image of the ranging area of the laser module, and the image output at the moment t is image [ t ].
The falling sensor module transmits the ranging values dist1[ t ], dist2[ t ] and the image [ t ] to the sensor data analysis and fusion module in real time.
In the sensor data analysis and fusion module, the specific working flow is as follows:
step S1: dist1 t, dist2 t and image t for receiving the transmission of the falling module;
step S2: judging whether the distance measurement values of dist1[ t ] and dist2[ t ] are larger than the sum of the ground height value H and the falling threshold value thresh; if the value is larger than the drop value, jumping to step S3; otherwise, outputting the result: and no falling risk exists.
Step S3: analyzing image data image [ t ], wherein a deep learning module is arranged in the sensor data fusion analysis module, and the deep learning module is called to identify whether depressions, steps and other conditions exist in the image [ t ] by adopting a deep learning method; if yes, outputting a result: no falling risk exists; otherwise, go to step S4 for manual analysis.
Step S4: the fall range position, range values dist1[ t ], dist2[ t ] and image [ t ] are combined, and an example is shown in fig. 4. And outputting a result: there is a risk of falling and the results are passed to an after-market engineer for analysis.
The sensor data analysis and fusion module comprehensively analyzes the distance measurement value and the image data of the falling laser module, and the falling condition is judged more reliably. Because the traditional method only depends on the ranging value of the laser module, the falling condition is not intuitive (only the ranging value is used), and the interference source cannot be determined when the interference is caused by ambient light; the traditional depth camera mode also has the problems of higher cost and inaccurate range finding value and need to be calibrated.
The working principle is as follows: judging whether the distance value is larger than a falling distance value or not according to the distance value (assuming that the distance value exceeds 1m, the falling risk exists, and if the distance value is larger than 1m, the falling is considered), and if the distance value is larger than 1m, turning to image analysis; and during image analysis, whether the image has grooves, steps and the like is identified to further judge the falling.
The implementation principle is as follows:
1) the drop sensor module needs to be assembled as shown in fig. 1, and is provided with two laser ranging modules and a monocular camera.
2) The drop sensor module needs to be installed onto the robot obliquely downward as shown in fig. 2 for ranging with the laser module obliquely downward and photographing the situation of the ground in front of the robot with the monocular camera.
3) The falling sensor module transmits the distance measurement values of the two laser modules and the image of the monocular camera to the sensor data fusion analysis module.
4) The sensor data fusion analysis module comprehensively analyzes the ranging value of the laser module and the image of the monocular camera to judge whether the laser module falls off, and the specific judgment steps are S1-S4 in the scheme.
The embodiment of the invention provides a falling detection system and a robot, wherein the system enriches falling information, so that the falling condition is more visual, and an engineer and a client can communicate conveniently; meanwhile, the falling judgment reliability is improved in a low-cost and low-resource occupation mode.
Those skilled in the art will appreciate that, in addition to implementing the system and its various devices, modules, units provided by the present invention as pure computer readable program code, the system and its various devices, modules, units provided by the present invention can be fully implemented by logically programming method steps in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Therefore, the system and various devices, modules and units thereof provided by the invention can be regarded as a hardware component, and the devices, modules and units included in the system for realizing various functions can also be regarded as structures in the hardware component; means, modules, units for performing the various functions may also be regarded as structures within both software modules and hardware components for performing the method.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention. The embodiments and features of the embodiments of the present application may be combined with each other arbitrarily without conflict.
Claims (8)
1. A fall detection system, comprising:
a drop sensor module: the device comprises a laser ranging module and a monocular camera module; the laser ranging module measures the distance from the reflecting medium to the laser ranging module, and the monocular camera module shoots an image of a ranging area of the laser module;
the sensor data analysis and fusion module: the initial judgment of the falling condition is carried out by receiving the data of the laser ranging module, and the reliability of the falling judgment is ensured by further analyzing the image data if the falling condition is considered.
2. The fall detection system according to claim 1, wherein there are two laser ranging modules, each laser ranging module has a sensor disposed therein, and the two laser ranging modules respectively output ranging values dist1[ t ] and dist2[ t ] of the two sensors at time t.
3. The fall detection system according to claim 2, wherein the monocular camera module takes an image of a laser module ranging area, the image output at time t being image [ t ].
4. The fall detection system according to claim 3, wherein the fall sensor module transmits the range values dist1[ t ], dist2[ t ], and the image [ t ] to the sensor data analysis fusion module in real time.
5. The fall detection system according to claim 4, wherein the sensor data analysis fusion module workflow is as follows:
step S1: dist1 t, dist2 t and image t for receiving the transmission of the falling module;
step S2: judging whether the distance measurement values of dist1[ t ] and dist2[ t ] are larger than the sum of the ground height value H and the falling threshold value thresh;
step S3: analyzing the image data image [ t ], calling a deep learning module in the sensor data fusion analysis module, and identifying whether relevant conditions including depressions and steps exist in the image [ t ] by adopting a deep learning method;
step S4: combining the falling distance measurement position, the distance measurement value dist1[ t ], dist2[ t ] and the image [ t ], and outputting a result: there is a risk of falling and the results are passed to an after-market engineer for analysis.
6. The fall detection system according to claim 5, wherein the step S2 specifically includes: judging whether the distance measurement values of dist1[ t ] and dist2[ t ] are larger than the sum of the ground height value H and the falling threshold value thresh;
if the value is larger than the drop value, jumping to step S3; otherwise, outputting the result: and no falling risk exists.
7. The fall detection system according to claim 5, wherein the step S3 is performed to identify whether there is a recess or step in the image [ t ];
if yes, outputting a result: no falling risk exists; otherwise, go to step S4 for manual analysis.
8. A robot, characterized in that the robot comprises a system according to any of claims 1-7.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111405460.9A CN114111703A (en) | 2021-11-24 | 2021-11-24 | Falling detection system and robot |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111405460.9A CN114111703A (en) | 2021-11-24 | 2021-11-24 | Falling detection system and robot |
Publications (1)
Publication Number | Publication Date |
---|---|
CN114111703A true CN114111703A (en) | 2022-03-01 |
Family
ID=80372171
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202111405460.9A Pending CN114111703A (en) | 2021-11-24 | 2021-11-24 | Falling detection system and robot |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114111703A (en) |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2005242409A (en) * | 2004-02-24 | 2005-09-08 | Matsushita Electric Works Ltd | Autonomous mobile robot system |
CN109186463A (en) * | 2018-09-04 | 2019-01-11 | 浙江梧斯源通信科技股份有限公司 | Anti-fall method applied to mobile robot |
CN110216661A (en) * | 2019-04-29 | 2019-09-10 | 北京云迹科技有限公司 | Fall the method and device of region recognition |
KR20200029651A (en) * | 2018-09-04 | 2020-03-19 | 엘지전자 주식회사 | Robot cleaner and method for controlling the same |
CN112120598A (en) * | 2020-09-02 | 2020-12-25 | 湖南格兰博智能科技有限责任公司 | Walking method and system of anti-falling robot and anti-falling robot |
CN113524265A (en) * | 2021-08-03 | 2021-10-22 | 汤恩智能科技(常熟)有限公司 | Robot anti-falling method, robot and readable storage medium |
-
2021
- 2021-11-24 CN CN202111405460.9A patent/CN114111703A/en active Pending
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2005242409A (en) * | 2004-02-24 | 2005-09-08 | Matsushita Electric Works Ltd | Autonomous mobile robot system |
CN109186463A (en) * | 2018-09-04 | 2019-01-11 | 浙江梧斯源通信科技股份有限公司 | Anti-fall method applied to mobile robot |
KR20200029651A (en) * | 2018-09-04 | 2020-03-19 | 엘지전자 주식회사 | Robot cleaner and method for controlling the same |
CN110216661A (en) * | 2019-04-29 | 2019-09-10 | 北京云迹科技有限公司 | Fall the method and device of region recognition |
CN112120598A (en) * | 2020-09-02 | 2020-12-25 | 湖南格兰博智能科技有限责任公司 | Walking method and system of anti-falling robot and anti-falling robot |
CN113524265A (en) * | 2021-08-03 | 2021-10-22 | 汤恩智能科技(常熟)有限公司 | Robot anti-falling method, robot and readable storage medium |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109598066B (en) | Effect evaluation method, apparatus, device and storage medium for prediction module | |
KR102094341B1 (en) | System for analyzing pot hole data of road pavement using AI and for the same | |
KR101846537B1 (en) | Monitoring system for automatically selecting cctv, monitoring managing server for automatically selecting cctv and managing method thereof | |
CN100579174C (en) | Motion detection method and device | |
KR102168288B1 (en) | System and method for tracking multiple object using multi-LiDAR | |
CN108021891B (en) | Vehicle environment identification method and system based on combination of deep learning and traditional algorithm | |
CN112767644B (en) | Method and device for early warning fire in highway tunnel based on video identification | |
US10438492B2 (en) | Method for evaluating a hazardous situation which is sensed by at least one sensor of a vehicle, method for controlling reproduction of a hazard warning and method for reproducing a hazard warning | |
US11577756B2 (en) | Detecting out-of-model scenarios for an autonomous vehicle | |
KR102403169B1 (en) | Method for providing guide through image analysis, and computer program recorded on record-medium for executing method therefor | |
CN111178119A (en) | Intersection state detection method and device, electronic equipment and vehicle | |
KR102389998B1 (en) | De-identification processing method and a computer program recorded on a recording medium to execute the same | |
Martin et al. | Identification of performance limitations of sensing technologies for automated driving | |
JP2019154027A (en) | Method and device for setting parameter for video monitoring system, and video monitoring system | |
US20130258063A1 (en) | Computing device and method for automatically inspecting quality of products on an automatic production line | |
CN110426714B (en) | Obstacle identification method | |
CN110390813B (en) | Big data processing system based on vehicle type identification | |
CN115830562B (en) | Lane information determination method, computer device and medium | |
CN114111703A (en) | Falling detection system and robot | |
CN112528711B (en) | Method and device for processing information | |
WO2022042853A1 (en) | Critical scenario identification for verification and validation of vehicles | |
CN115891868A (en) | Fault detection method, device, electronic apparatus, and medium for autonomous vehicle | |
CN111986473A (en) | Big data processing method based on vehicle type identification | |
CN112560258A (en) | Test method, device, equipment and storage medium | |
KR102403174B1 (en) | Method for data purification according to importance, and computer program recorded on record-medium for executing method therefor |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |