CN110488805A - A kind of unmanned vehicle obstacle avoidance system and method based on 3D stereoscopic vision - Google Patents
A kind of unmanned vehicle obstacle avoidance system and method based on 3D stereoscopic vision Download PDFInfo
- Publication number
- CN110488805A CN110488805A CN201810485140.0A CN201810485140A CN110488805A CN 110488805 A CN110488805 A CN 110488805A CN 201810485140 A CN201810485140 A CN 201810485140A CN 110488805 A CN110488805 A CN 110488805A
- Authority
- CN
- China
- Prior art keywords
- camera
- information
- vehicle
- unmanned vehicle
- processing unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 18
- 238000013135 deep learning Methods 0.000 claims abstract description 5
- 230000004888 barrier function Effects 0.000 claims description 10
- 230000000007 visual effect Effects 0.000 claims description 3
- 238000005265 energy consumption Methods 0.000 claims 1
- 230000007547 defect Effects 0.000 abstract description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000004033 diameter control Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000002604 ultrasonography Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
- G05D1/0214—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory in accordance with safety or protection criteria, e.g. avoiding hazardous areas
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
- G05D1/0251—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Electromagnetism (AREA)
- Traffic Control Systems (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
Barrier-avoiding method is the basis in unmanned.The present invention provides the defects to involve great expense for the problem that the laser radar generally used, and camera to be unable to get obstacle length, provides a kind of method for replacing laser radar avoidance by depth camera.Binocular vision unit is used to obtain the image under two camera perspectives of left and right, calculates disparity map and obtains depth map, and calculates the point cloud under camera coordinates system by depth map；Image processing unit by point cloud and deep learning obtain can traffic areas extract obstacle information, resolve avoidance angle and distance and combine light stream planning path；Control unit obtains control information from image processing unit and other sensors, controls state of motion of vehicle.
Description
Technical field
The present invention relates to technical field of computer vision, especially application of the 3D stereoscopic vision in avoidance scheme.
Background technique
With the development of science and technology unmanned drive with auxiliary has begun the visual field for slowly coming into people.Avoidance is pair
The various movements for hiding obstacle made by the extraneous direction of motion hindered to object, and continue forward movement, this mistake
Journey is exactly avoidance.Avoidance scheme is then the basis in unmanned.
The main sensors that robot obstacle-avoiding uses at present are laser radar or camera, then are aided with other sensors such as
Ultrasonic wave etc. is assisted.Laser radar avoidance mainly obtains a cloud by scanning ambient enviroment to carry out avoidance.But laser thunder
Constantly rotation is needed to be easily damaged to obtain the point cloud information of surrounding up to cost height, and when running.
Another method is to carry out avoidance using camera.At present generally in the processing using binocular camera of application
Method is the dot matrix for depth camera simulation being become laser radar.Laser radar of this method relative to mainstream, it is at low cost,
It is hardly damaged.But the problem is that can only obtain the obstacle information in a plane, can not learn barrier length and
Size.It is not easy to avoidance.
In view of the foregoing, the designer relies on the technical experience and professional knowledge abundant of many years related fields, no
Disconnected experiment and improvement, propose barrier-avoiding method of one of the present invention based on binocular camera.
Summary of the invention
Present invention solves the technical problem that being the defect to involve great expense for the laser radar generally used；
Another problem that the present invention solves is that camera is unable to get obstacle length.
It is kept away by depth camera combination deep learning instead of laser radar in view of the above-mentioned problems, the present invention provides one kind
The method of barrier, comprising steps of
1. obtaining depth map by binocular camera first；
2. resolving the point cloud coordinate under camera coordinates system by depth map；
3. according to deep learning obtain can traffic areas be fitted ground, extract the point cloud information of barrier；
4. determining the left and right information of barrier, sliding window is used on top view, finds the position of P Passable.It calculates
Avoidance angle and distance plan local path in conjunction with barrier light stream to obtain mulitpath planning information.Send information
To control module；
5. control module unit obtains the path of image processing unit planning, an optimal road is selected according to other sensors
Diameter controls vehicle movement.
The present invention has following beneficial effect compared with prior art:
1, the present invention compares laser radar avoidance scheme, and at low cost, precision is higher, is hardly damaged.
2, blind area is small, can calculate the obstacle information of more than one plane, even if barrier is lower than sensor plane,
It can also carry out avoidance.
3, it calculates fastly, operational efficiency is high, can achieve 50 frames or more in embedded system.
Detailed description of the invention
Present invention will be further explained below with reference to the attached drawings and examples.The accompanying drawings in the following description is only the present invention
Some embodiments for those of ordinary skill in the art without creative efforts, can also basis
The content of the embodiment of the present invention and these attached drawings obtain other attached drawings.
Fig. 1 is the structural diagram of the present invention: wherein 1 is binocular camera, and 2 be image processing unit, and 3 is single for control
Member；
Fig. 2 is algorithm flow chart of the invention；
Specific embodiment
In order to keep the purposes, technical schemes and advantages of the embodiment of the present invention clearer, with reference to the accompanying drawing to specific reality
Example is applied to be described in detail.
1, connection modules form obstacle avoidance system as shown in Figure 1.Binocular camera is connected to image processing unit, image
Processing unit passes through serial ports again and is connected with control unit, and control unit directly controls the motion state of vehicle；
2, Fig. 2 show the operational process of this method.It is demarcated firstly the need of to camera, obtains the parameter of camera
Information.
3, system calculates in a cloud each point in camera from the three-dimensional point cloud information calculated in the visual field in binocular camera
Coordinate under coordinate system.
4, by deep learning obtain under current scene can traffic areas, by can the point in traffic areas extract and correspond to
Three-dimensional point cloud；
5, RANSAC algorithm fit Plane can be being used in the three-dimensional point cloud of traffic areas, is putting to be located in cloud and is somebody's turn to do
Point more than plane is determined as barrier；
6, disturbance in judgement object point is on the left side of current vehicle position or right side.Corresponding side on top view, to take the photograph
The x-axis center of camera is that starting point using the mode of sliding window finds out transitable position, calculate multiple groups avoidance angle and
Distance.
7, in the color image that camera obtains, by obstacle position information obtained in 5, its light stream letter is calculated
Breath.
8, by multiple groups avoidance angle and distance obtained in 6, a plurality of avoidance path is planned in conjunction with Optic flow information, is handed down to
Control unit.
7, control module unit obtains the path of image processing unit planning, from bottom ultrasound unit and millimetre-wave radar
The path that unit carrys out planning is assessed, and an optimal path is selected from assessment result, controls vehicle movement.
Claims (4)
1. a kind of unmanned vehicle obstacle avoidance system and method based on 3D stereoscopic vision, which is characterized in that include binocular vision unit, figure
As processing unit, control unit for vehicle.Binocular vision unit is used to obtain the image under two camera perspectives of left and right, calculates disparity map
Depth map is obtained, and calculates the point cloud under camera coordinates system by depth map；Image processing unit passes through data reduction obstacle
Object information resolves avoidance angle and distance；Control unit obtains control information from image processing unit and other sensors, control
State of motion of vehicle.
2. a kind of unmanned vehicle obstacle avoidance system and method based on 3D stereoscopic vision according to claim 1, which is characterized in that
Binocular camera is mounted on front of the car, in order to expand visual angle, reduce blind area, the position of camera is lowerd, is made on ground
Barrier is as much as possible to be entered in camera coverage.
3. a kind of unmanned vehicle obstacle avoidance system and method based on 3D stereoscopic vision according to claim 1, which is characterized in that
Image processing unit integrates CPU using advanced Pascal framework GPU and 2 and A57 of Denver, and development board is high-efficient, property
Can be strong, low in energy consumption, it is suitable for unmanned vehicle system；There is outstanding operational efficiency with bottom vehicle motion control unit matching.
4. a kind of unmanned vehicle obstacle avoidance system and method based on 3D stereoscopic vision, method includes the following steps:
Step 1. calculates the point cloud information in unmanned vehicle operation front using binocular camera
Step 2. by point cloud information and deep learning obtain can traffic areas information, obtain the barrier three in front of unmanned vehicle
Tie up information
Step 3. resolves barrier three-dimensional information, and influence of the disturbance in judgement object to vehicle current motion state passes through dynamic window
Method be handed down to control unit for vehicle in conjunction with Optic flow information planning path again in real time
Step 4. control unit for vehicle is moved according to elementary area information and other sensor informations control vehicle.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810485140.0A CN110488805A (en) | 2018-05-15 | 2018-05-15 | A kind of unmanned vehicle obstacle avoidance system and method based on 3D stereoscopic vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810485140.0A CN110488805A (en) | 2018-05-15 | 2018-05-15 | A kind of unmanned vehicle obstacle avoidance system and method based on 3D stereoscopic vision |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110488805A true CN110488805A (en) | 2019-11-22 |
Family
ID=68544988
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810485140.0A Pending CN110488805A (en) | 2018-05-15 | 2018-05-15 | A kind of unmanned vehicle obstacle avoidance system and method based on 3D stereoscopic vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110488805A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111624604A (en) * | 2020-04-24 | 2020-09-04 | 山东昆仑度智能科技有限公司 | Obstacle avoidance method for unmanned vehicle |
| CN112068574A (en) * | 2020-10-19 | 2020-12-11 | 中国科学技术大学 | Control method and system for unmanned vehicle in dynamic complex environment |
| CN112660145A (en) * | 2020-12-24 | 2021-04-16 | 李敏 | Control system and control method of unmanned vehicle |
| WO2021134325A1 (en) * | 2019-12-30 | 2021-07-08 | 深圳元戎启行科技有限公司 | Obstacle detection method and apparatus based on driverless technology and computer device |
| CN113989347A (en) * | 2021-11-09 | 2022-01-28 | 北京智芯原动科技有限公司 | Binocular parallax calculation method and device |
| TWI757999B (en) * | 2020-12-04 | 2022-03-11 | 國立陽明交通大學 | Real-time obstacle avoidance system, real-time obstacle avoidance method and unmanned vehicle with real-time obstacle avoidance function |
| CN115100622A (en) * | 2021-12-29 | 2022-09-23 | 中国矿业大学 | Method for detecting travelable area and automatically avoiding obstacle of unmanned transportation equipment in deep limited space |
| CN115540896A (en) * | 2022-12-06 | 2022-12-30 | 广汽埃安新能源汽车股份有限公司 | Path planning method, path planning device, electronic equipment and computer readable medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103955920A (en) * | 2014-04-14 | 2014-07-30 | 桂林电子科技大学 | Binocular vision obstacle detection method based on three-dimensional point cloud segmentation |
| CN106325277A (en) * | 2016-09-28 | 2017-01-11 | 关健生 | Distributed framework based autonomous navigation and inspection robot control method |
| CN106681353A (en) * | 2016-11-29 | 2017-05-17 | 南京航空航天大学 | Unmanned aerial vehicle (UAV) obstacle avoidance method and system based on binocular vision and optical flow fusion |
| CN107133559A (en) * | 2017-03-14 | 2017-09-05 | 湖北工业大学 | Mobile object detection method based on 360 degree of panoramas |
| CN107491070A (en) * | 2017-08-31 | 2017-12-19 | 成都通甲优博科技有限责任公司 | A kind of method for planning path for mobile robot and device |
| CN107563313A (en) * | 2017-08-18 | 2018-01-09 | 北京航空航天大学 | Multiple target pedestrian detection and tracking based on deep learning |
| CN107613262A (en) * | 2017-09-30 | 2018-01-19 | 驭势科技（北京）有限公司 | A kind of Vision information processing System and method for |
-
2018
- 2018-05-15 CN CN201810485140.0A patent/CN110488805A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103955920A (en) * | 2014-04-14 | 2014-07-30 | 桂林电子科技大学 | Binocular vision obstacle detection method based on three-dimensional point cloud segmentation |
| CN106325277A (en) * | 2016-09-28 | 2017-01-11 | 关健生 | Distributed framework based autonomous navigation and inspection robot control method |
| CN106681353A (en) * | 2016-11-29 | 2017-05-17 | 南京航空航天大学 | Unmanned aerial vehicle (UAV) obstacle avoidance method and system based on binocular vision and optical flow fusion |
| CN107133559A (en) * | 2017-03-14 | 2017-09-05 | 湖北工业大学 | Mobile object detection method based on 360 degree of panoramas |
| CN107563313A (en) * | 2017-08-18 | 2018-01-09 | 北京航空航天大学 | Multiple target pedestrian detection and tracking based on deep learning |
| CN107491070A (en) * | 2017-08-31 | 2017-12-19 | 成都通甲优博科技有限责任公司 | A kind of method for planning path for mobile robot and device |
| CN107613262A (en) * | 2017-09-30 | 2018-01-19 | 驭势科技（北京）有限公司 | A kind of Vision information processing System and method for |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021134325A1 (en) * | 2019-12-30 | 2021-07-08 | 深圳元戎启行科技有限公司 | Obstacle detection method and apparatus based on driverless technology and computer device |
| CN111624604A (en) * | 2020-04-24 | 2020-09-04 | 山东昆仑度智能科技有限公司 | Obstacle avoidance method for unmanned vehicle |
| CN112068574A (en) * | 2020-10-19 | 2020-12-11 | 中国科学技术大学 | Control method and system for unmanned vehicle in dynamic complex environment |
| TWI757999B (en) * | 2020-12-04 | 2022-03-11 | 國立陽明交通大學 | Real-time obstacle avoidance system, real-time obstacle avoidance method and unmanned vehicle with real-time obstacle avoidance function |
| CN112660145A (en) * | 2020-12-24 | 2021-04-16 | 李敏 | Control system and control method of unmanned vehicle |
| CN113989347A (en) * | 2021-11-09 | 2022-01-28 | 北京智芯原动科技有限公司 | Binocular parallax calculation method and device |
| CN113989347B (en) * | 2021-11-09 | 2024-05-03 | 北京智芯原动科技有限公司 | Binocular parallax calculation method and device |
| CN115100622A (en) * | 2021-12-29 | 2022-09-23 | 中国矿业大学 | Method for detecting travelable area and automatically avoiding obstacle of unmanned transportation equipment in deep limited space |
| CN115100622B (en) * | 2021-12-29 | 2023-09-22 | 中国矿业大学 | Method for detecting driving area of unmanned transportation equipment in deep limited space and automatically avoiding obstacle |
| CN115540896A (en) * | 2022-12-06 | 2022-12-30 | 广汽埃安新能源汽车股份有限公司 | Path planning method, path planning device, electronic equipment and computer readable medium |
| CN115540896B (en) * | 2022-12-06 | 2023-03-07 | 广汽埃安新能源汽车股份有限公司 | Path planning method and device, electronic equipment and computer readable medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110488805A (en) | A kind of unmanned vehicle obstacle avoidance system and method based on 3D stereoscopic vision | |
| Pfeiffer et al. | Efficient representation of traffic scenes by means of dynamic stixels | |
| JP7210165B2 (en) | Method, device and display device for displaying virtual route | |
| EP3607489B1 (en) | Direct vehicle detection as 3d bounding boxes using neural network image processing | |
| CN108196535B (en) | Automatic driving system based on reinforcement learning and multi-sensor fusion | |
| US11107230B2 (en) | Systems and methods for depth estimation using monocular images | |
| CN106681353B (en) | The unmanned plane barrier-avoiding method and system merged based on binocular vision with light stream | |
| EP2209091B1 (en) | System and method for object motion detection based on multiple 3D warping and vehicle equipped with such system | |
| US10129521B2 (en) | Depth sensing method and system for autonomous vehicles | |
| CN100494900C (en) | Environment sensing one-eye visual navigating method adapted to self-aid moving vehicle | |
| Sobh et al. | End-to-end multi-modal sensors fusion system for urban automated driving | |
| Bounini et al. | Autonomous vehicle and real time road lanes detection and tracking | |
| US8630806B1 (en) | Image processing for vehicle control | |
| KR101913168B1 (en) | Egomotion estimation system and method | |
| US11348263B2 (en) | Training method for detecting vanishing point and method and apparatus for detecting vanishing point | |
| JP5023186B2 (en) | Object motion detection system based on combination of 3D warping technique and proper object motion (POM) detection | |
| CN111986506A (en) | Mechanical parking space parking method based on multi-vision system | |
| CN113085896B (en) | Auxiliary automatic driving system and method for modern rail cleaning vehicle | |
| EP3859390A1 (en) | Method and system for rendering a representation of an evinronment of a vehicle | |
| Barua et al. | A self-driving car implementation using computer vision for detection and navigation | |
| CN113031597A (en) | Autonomous obstacle avoidance method based on deep learning and stereoscopic vision | |
| CN103743394A (en) | Light-stream-based obstacle avoiding method of mobile robot | |
| Capito et al. | Optical flow based visual potential field for autonomous driving | |
| Newman et al. | Self-driving cars: A platform for learning and research | |
| Jun et al. | Autonomous driving system design for formula student driverless racecar |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20191122 |
|
| WD01 | Invention patent application deemed withdrawn after publication |