CN109828267A - The Intelligent Mobile Robot detection of obstacles and distance measuring method of Case-based Reasoning segmentation and depth camera - Google Patents
The Intelligent Mobile Robot detection of obstacles and distance measuring method of Case-based Reasoning segmentation and depth camera Download PDFInfo
- Publication number
- CN109828267A CN109828267A CN201910137652.2A CN201910137652A CN109828267A CN 109828267 A CN109828267 A CN 109828267A CN 201910137652 A CN201910137652 A CN 201910137652A CN 109828267 A CN109828267 A CN 109828267A
- Authority
- CN
- China
- Prior art keywords
- barrier
- obstacles
- intelligent mobile
- mobile robot
- depth camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses the Intelligent Mobile Robot detection of obstacles and distance measuring method of a kind of Case-based Reasoning segmentation and depth camera, obtain the video file of depth camera acquisition；Video file is converted into RGB image and depth image frame by frame；Trained Mask R-CNN network detects the barrier in image and obtains the two-value exposure mask and profile of barrier to example segmentation is carried out in RGB image；By Predistribution Algorithm, depth image, the range information of acquisition robot to barrier are matched.The present invention cooperates without multiple hardwares sensing equipment, it only need to be according to the collected RGB of depth camera and depth image, example dividing function based on Mask R-CNN network can realize detection and ranging of the Intelligent Mobile Robot to the barrier on road, provide a simple and easy detection of obstacles and distance measuring method for the interim avoidance of Intelligent Mobile Robot.
Description
Technical field
The present invention relates to Intelligent Mobile Robot field more particularly to a kind of segmentation of Case-based Reasoning and depth cameras
Intelligent Mobile Robot detection of obstacles and distance measuring method.
Background technique
Intelligent Mobile Robot must make a response in real time to the ambient enviroment of transformation when executing patrol task,
It needs to detect and avoid automatically to carry out the barrier on road, avoids the danger for colliding and falling, continue after avoidance
It is travelled according to the global path of planning.Avoidance process can be divided into roughly detection of obstacles and ranging, temporary path planning
Two parts；Wherein, detection of obstacles and ranging be exactly by the way that the information that various kinds of sensors acquires is handled and is analyzed, from
And obtain the information such as barrier classification, distance, size；Temporary path planning is exactly to utilize currently detected obstacle information simultaneously
Comprehensive other information constitutes local map, is planned again the path that present feasible is sailed.
Existing Intelligent Mobile Robot detection of obstacles and distance measuring method, mainly there is following defect: (1) robot needs
It carries two kinds or more of sensing equipment and is just able to achieve detection of obstacles and survey if depth camera is in conjunction with ultrasonic radar
Away from system structure is complicated, and reliability is not high；(2) the obstacle detection method algorithm based on binocular vision is complicated, and to complicated light
Not high according to the adaptability of condition, range error is larger；(3) target classification can only be carried out based on sorter networks such as VGG, ResNet,
The rectangular bounding box of barrier can only be provided based on the target detections such as Faster R-CNN, YoLo network, it all can not be disposable complete
At detection of obstacles identification and two kinds of tasks of ranging.
Summary of the invention
Goal of the invention: in view of the above problems, the present invention proposes the substation of a kind of Case-based Reasoning segmentation and depth camera
Crusing robot detection of obstacles and distance measuring method, required hardware sensing equipment is less, implementation is simple, stability is good, at low cost,
It can satisfy the range accuracy requirement of the interim avoidance of Intelligent Mobile Robot.
Technical solution: to achieve the purpose of the present invention, the technical scheme adopted by the invention is that: a kind of Case-based Reasoning segmentation
With the Intelligent Mobile Robot detection of obstacles and distance measuring method of depth camera, comprising steps of
(1) video file of depth camera acquisition is obtained；
(2) video file of acquisition is converted into RGB image and depth image frame by frame；
(3) using trained Mask R-CNN network to progress example segmentation in RGB image；
(4) example of road and barrier is divided according in RGB image, detects and identifies the barrier on road；
(5) example of road and barrier is divided according in RGB image, obtains the two-value exposure mask of barrier；
(6) depth image is matched by Predistribution Algorithm according to two-value exposure mask, the distance for obtaining robot to barrier is believed
Breath.
Further, the Mask R-CNN network is formed by the training of a large amount of substation field image pattern.
Further, the Predistribution Algorithm includes:
(6.1) according to the two-value exposure mask of barrier, the profile information of barrier is obtained；
(6.2) traverse contour pixel, find it is most upper, most under, most left, most right four object pixels and obtain its pixel seat
Mark；
(6.3) obtained in depth image according to the coordinate of object pixel object pixel and its 8 adjacent pixels away from
From information；
(6.4) spatial pattern and process excluding outlier is used, average value is calculated with the range data after rejecting, as target
The distance value of pixel.
Further, the distance value l according to the object pixel of highest and lowest two of barrier profile1、l2And depth camera
Head arrives the height h on groundr, be calculated robot to barrier actual range d:
The height h of barrier is calculated:
θ=θ1-θ2
The utility model has the advantages that the present invention is not necessarily to rely on laser or ultrasonic distance-measuring sensor, only need a depth camera can
To realize ranging of the robot to barrier, hardware cost and system complexity greatly reduces.
The present invention is based on the example partitioning algorithms in computer vision technique to identify to barrier, and to road and
Barrier is split, and robot is calculated to the distance of barrier by the method for traversing contour pixel, can also calculate
The height of barrier is obtained, algorithm is simple and easy；Using Grubbs (Grubbs) method excluding outlier, the standard of algorithm is improved
True property, this method, which can be used as, even substitutes existing transformer substation robot to detection of obstacles and identification, the improvement of ranging process.
Detailed description of the invention
Fig. 1 is the flow diagram of Intelligent Mobile Robot detection of obstacles and distance measuring method of the present invention；
Fig. 2 is the Predistribution Algorithm flow diagram for obtaining object pixel range information；
Fig. 3 is the matrix schematic diagram of object pixel and adjacent pixel in depth image；
Fig. 4, which is robot, calculates schematic diagram to obstacle distance and obstacle height.
Specific embodiment
Further description of the technical solution of the present invention with reference to the accompanying drawings and examples.
As shown in Figure 1, the Intelligent Mobile Robot obstacle of Case-based Reasoning segmentation and depth camera of the present invention
Analyte detection and distance measuring method, comprising steps of
(1) video file of depth camera acquisition is obtained；
As one embodiment, using the RealSense depth camera D435 of Intel Company in the present invention, most
It is good to use the avoidance requirement that can satisfy Intelligent Mobile Robot completely apart from being 0.3-8 meters.
(2) video file of acquisition is converted into RGB image and depth image frame by frame；
(3) using trained Mask R-CNN network to progress example segmentation in RGB image；
Mask R-CNN network is formed by the training of a large amount of substation field image pattern, for realizing detection of obstacles
And the example segmentation of identification and road and barrier.The effect of Mask R-CNN network is identified not in Pixel-level scene
Same target obtains the difference of objects in images by the processing of Mask R-CNN network to any one RGB image of input
Classification and profile information.
(4) example of road and barrier is divided according in RGB image, detects and identifies the barrier on road；
(5) example of road and barrier is divided according in RGB image, obtains the two-value exposure mask of barrier；
(6) depth image is matched by Predistribution Algorithm according to two-value exposure mask, the distance for obtaining robot to barrier is believed
Breath.
As shown in Fig. 2, Predistribution Algorithm includes:
(6.1) according to the two-value exposure mask of barrier, the profile information of barrier is obtained；
(6.2) traverse contour pixel, find it is most upper, most under, most left, most right four object pixels and obtain its pixel seat
Mark；
(6.3) as shown in figure 3, obtaining object pixel and its adjacent 8 in depth image according to the coordinate of object pixel
The range information of a pixel；
Depth information according to object pixel and its 8 adjacent pixels makes statistical inference, it is possible to prevente effectively from because strong
The reasons such as light direct beam and caused by single pixel point depth information problem of dtmf distortion DTMF.
(6.4) Grubbs (Grubbs) method excluding outlier is used, calculates average value with the range data after rejecting,
Distance value as object pixel.
Using Grubbs (Grubbs) method excluding outlier, average value then is calculated with the range data after rejecting,
It can effectively improve the accuracy of algorithm.
As shown in figure 4, (b point) two object pixels under according to obtained most upper (a point) of barrier profile and most
Distance value l1、l2And known depth camera is to the height h on groundr, robot can be calculated by following formula
To the actual range d of barrier:
The height h of barrier can be calculated by following formula:
θ=θ1-θ2
Wherein, angle theta1、θ2, θ it is as shown in Figure 4.
Claims (5)
1. a kind of the Intelligent Mobile Robot detection of obstacles and distance measuring method of Case-based Reasoning segmentation and depth camera, special
Sign is, comprising steps of
(1) video file of depth camera acquisition is obtained；
(2) video file of acquisition is converted into RGB image and depth image frame by frame；
(3) example segmentation is carried out to RGB image using trained Mask R-CNN network；
(4) example of road and barrier is divided according in RGB image, detects and identifies the barrier on road；
(5) example of road and barrier is divided according in RGB image, obtains the two-value exposure mask of barrier；
(6) depth image, the range information of acquisition robot to barrier are matched by Predistribution Algorithm according to two-value exposure mask.
2. the Intelligent Mobile Robot detection of obstacles of Case-based Reasoning segmentation and depth camera according to claim 1
And distance measuring method, which is characterized in that the Mask R-CNN network is formed by the training of a large amount of substation field image pattern.
3. the Intelligent Mobile Robot detection of obstacles of Case-based Reasoning segmentation and depth camera according to claim 1
And distance measuring method, which is characterized in that the Predistribution Algorithm includes:
(6.1) according to the two-value exposure mask of barrier, the profile information of barrier is obtained；
(6.2) traverse contour pixel, find it is most upper, most under, most left, most right four object pixels and obtain its pixel coordinate；
(6.3) the distance letter of object pixel and its 8 adjacent pixels is obtained in depth image according to the coordinate of object pixel
Breath；
(6.4) spatial pattern and process excluding outlier is used, average value is calculated with the range data after rejecting, as object pixel
Distance value.
4. the Intelligent Mobile Robot detection of obstacles of Case-based Reasoning segmentation and depth camera according to claim 3
And distance measuring method, which is characterized in that the distance value l of the object pixel of highest and lowest two according to barrier profile1、l2And it is deep
Height h of the degree camera to groundr, be calculated robot to barrier actual range d:
5. the Intelligent Mobile Robot detection of obstacles of Case-based Reasoning segmentation and depth camera according to claim 3
And distance measuring method, which is characterized in that the height h of barrier is calculated:
θ=θ1-θ2
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910137652.2A CN109828267A (en) | 2019-02-25 | 2019-02-25 | The Intelligent Mobile Robot detection of obstacles and distance measuring method of Case-based Reasoning segmentation and depth camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910137652.2A CN109828267A (en) | 2019-02-25 | 2019-02-25 | The Intelligent Mobile Robot detection of obstacles and distance measuring method of Case-based Reasoning segmentation and depth camera |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109828267A true CN109828267A (en) | 2019-05-31 |
Family
ID=66864365
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910137652.2A Pending CN109828267A (en) | 2019-02-25 | 2019-02-25 | The Intelligent Mobile Robot detection of obstacles and distance measuring method of Case-based Reasoning segmentation and depth camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109828267A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110298330A (en) * | 2019-07-05 | 2019-10-01 | 东北大学 | A kind of detection of transmission line polling robot monocular and localization method |
| CN110307848A (en) * | 2019-07-04 | 2019-10-08 | 南京大学 | A kind of Mobile Robotics Navigation method |
| CN110826512A (en) * | 2019-11-12 | 2020-02-21 | 深圳创维数字技术有限公司 | Ground obstacle detection method, ground obstacle detection device, and computer-readable storage medium |
| CN110913098A (en) * | 2019-10-28 | 2020-03-24 | 香港理工大学深圳研究院 | High-definition depth information acquisition system, system preparation method and system ranging method |
| CN111123915A (en) * | 2019-12-05 | 2020-05-08 | 国电南瑞科技股份有限公司 | Inspection robot obstacle crossing method and system, storage medium and computing equipment |
| CN111552300A (en) * | 2020-06-09 | 2020-08-18 | 南开大学 | Crop picking system based on instance segmentation and path planning |
| CN111652889A (en) * | 2020-06-04 | 2020-09-11 | 深圳市瓴鹰智能科技有限公司 | Edge calculation processing method, device and equipment based on intelligent detection equipment |
| CN113119099A (en) * | 2019-12-30 | 2021-07-16 | 深圳富泰宏精密工业有限公司 | Computer device and method for controlling mechanical arm to clamp and place object |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106940185A (en) * | 2016-01-05 | 2017-07-11 | 北京雷动云合智能技术有限公司 | A kind of localization for Mobile Robot and air navigation aid based on depth camera |
| CN107225570A (en) * | 2017-04-20 | 2017-10-03 | 深圳前海勇艺达机器人有限公司 | The barrier-avoiding method and device of intelligent robot |
| WO2018066352A1 (en) * | 2016-10-06 | 2018-04-12 | 株式会社アドバンスド・データ・コントロールズ | Image generation system, program and method, and simulation system, program and method |
| CN108388880A (en) * | 2018-03-15 | 2018-08-10 | 广东工业大学 | A kind of method and device that monitoring driver drives using mobile phone |
| CN108805906A (en) * | 2018-05-25 | 2018-11-13 | 哈尔滨工业大学 | A kind of moving obstacle detection and localization method based on depth map |
-
2019
- 2019-02-25 CN CN201910137652.2A patent/CN109828267A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106940185A (en) * | 2016-01-05 | 2017-07-11 | 北京雷动云合智能技术有限公司 | A kind of localization for Mobile Robot and air navigation aid based on depth camera |
| WO2018066352A1 (en) * | 2016-10-06 | 2018-04-12 | 株式会社アドバンスド・データ・コントロールズ | Image generation system, program and method, and simulation system, program and method |
| CN107225570A (en) * | 2017-04-20 | 2017-10-03 | 深圳前海勇艺达机器人有限公司 | The barrier-avoiding method and device of intelligent robot |
| CN108388880A (en) * | 2018-03-15 | 2018-08-10 | 广东工业大学 | A kind of method and device that monitoring driver drives using mobile phone |
| CN108805906A (en) * | 2018-05-25 | 2018-11-13 | 哈尔滨工业大学 | A kind of moving obstacle detection and localization method based on depth map |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110307848A (en) * | 2019-07-04 | 2019-10-08 | 南京大学 | A kind of Mobile Robotics Navigation method |
| CN110298330A (en) * | 2019-07-05 | 2019-10-01 | 东北大学 | A kind of detection of transmission line polling robot monocular and localization method |
| CN110913098A (en) * | 2019-10-28 | 2020-03-24 | 香港理工大学深圳研究院 | High-definition depth information acquisition system, system preparation method and system ranging method |
| CN110826512A (en) * | 2019-11-12 | 2020-02-21 | 深圳创维数字技术有限公司 | Ground obstacle detection method, ground obstacle detection device, and computer-readable storage medium |
| WO2021093418A1 (en) * | 2019-11-12 | 2021-05-20 | 深圳创维数字技术有限公司 | Ground obstacle detection method and device, and computer-readable storage medium |
| CN110826512B (en) * | 2019-11-12 | 2022-03-08 | 深圳创维数字技术有限公司 | Ground obstacle detection method, ground obstacle detection device, and computer-readable storage medium |
| CN111123915A (en) * | 2019-12-05 | 2020-05-08 | 国电南瑞科技股份有限公司 | Inspection robot obstacle crossing method and system, storage medium and computing equipment |
| CN113119099A (en) * | 2019-12-30 | 2021-07-16 | 深圳富泰宏精密工业有限公司 | Computer device and method for controlling mechanical arm to clamp and place object |
| CN111652889A (en) * | 2020-06-04 | 2020-09-11 | 深圳市瓴鹰智能科技有限公司 | Edge calculation processing method, device and equipment based on intelligent detection equipment |
| CN111552300A (en) * | 2020-06-09 | 2020-08-18 | 南开大学 | Crop picking system based on instance segmentation and path planning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109828267A (en) | The Intelligent Mobile Robot detection of obstacles and distance measuring method of Case-based Reasoning segmentation and depth camera | |
| CN104282020B (en) | A kind of vehicle speed detection method based on target trajectory | |
| CN103065323B (en) | Subsection space aligning method based on homography transformational matrix | |
| CN111462200A (en) | Cross-video pedestrian positioning and tracking method, system and equipment | |
| US9286678B2 (en) | Camera calibration using feature identification | |
| CN109472831A (en) | Obstacle recognition range-measurement system and method towards road roller work progress | |
| CN111241988B (en) | Method for detecting and identifying moving target in large scene by combining positioning information | |
| CN108320510A (en) | One kind being based on unmanned plane video traffic information statistical method and system | |
| CN110379168B (en) | Traffic vehicle information acquisition method based on Mask R-CNN | |
| Wu et al. | An algorithm for automatic vehicle speed detection using video camera | |
| CN110197173B (en) | Road edge detection method based on binocular vision | |
| Momeni-k et al. | Height estimation from a single camera view | |
| CN110147748B (en) | Mobile robot obstacle identification method based on road edge detection | |
| CN112798811A (en) | Speed measurement method, device and equipment | |
| CN116310679A (en) | Multi-sensor fusion target detection method, system, medium, equipment and terminal | |
| CN108416798A (en) | A kind of vehicle distances method of estimation based on light stream | |
| CN112232139A (en) | Obstacle avoidance method based on combination of Yolo v4 and Tof algorithm | |
| CN110703760A (en) | Newly-increased suspicious object detection method for security inspection robot | |
| Beauvisage et al. | Multi-spectral visual odometry for unmanned air vehicles | |
| CN117078717A (en) | Road vehicle track extraction method based on unmanned plane monocular camera | |
| CN114905512A (en) | Panoramic tracking and obstacle avoidance method and system for intelligent inspection robot | |
| CN114066847B (en) | Method for detecting lifting state of container truck based on 2D laser and image data fusion | |
| Hsu et al. | Detecting drivable space in traffic scene understanding | |
| Rasmussen et al. | Trail following with omnidirectional vision | |
| Du et al. | A real-time curb detection method for vehicle by using a 3D-LiDAR sensor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190531 |