CN110472508A - Lane line distance measuring method based on deep learning and binocular vision - Google Patents

Lane line distance measuring method based on deep learning and binocular vision Download PDF

Info

Publication number
CN110472508A
CN110472508A CN201910636651.2A CN201910636651A CN110472508A CN 110472508 A CN110472508 A CN 110472508A CN 201910636651 A CN201910636651 A CN 201910636651A CN 110472508 A CN110472508 A CN 110472508A
Authority
CN
China
Prior art keywords
lane line
lane
distance
binocular vision
binocular
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910636651.2A
Other languages
Chinese (zh)
Other versions
CN110472508B (en
Inventor
杨嘉琛
王晨光
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianjin University
Original Assignee
Tianjin University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianjin University filed Critical Tianjin University
Priority to CN201910636651.2A priority Critical patent/CN110472508B/en
Publication of CN110472508A publication Critical patent/CN110472508A/en
Application granted granted Critical
Publication of CN110472508B publication Critical patent/CN110472508B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/56Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/588Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02TCLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00Road transport of goods or passengers
    • Y02T10/10Internal combustion engine [ICE] based vehicles
    • Y02T10/40Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Traffic Control Systems (AREA)

Abstract

The present invention relates to a kind of lane line distance measuring method based on deep learning and binocular vision: including image capture module, the lane detection module based on convolutional neural networks, based on the range finder module of binocular vision.Lane detection module based on convolutional neural networks is for accurately identifying the lane line positioned at vehicle two sides, use vgg16 network frame and full convolutional network FCN, three layers after vgg16 network full articulamentum are become into warp lamination, warp lamination up-samples the characteristic pattern of convolutional layer, to generate a prediction to each pixel, lane line forecasting problem is transformed into image pixel-class to solve, obtains the binary map and instance graph of prediction lane line by convolutional neural networks;Range finder module based on binocular vision, the lane line position that will have been obtained using Binocular Stereo Matching Algorithm SGBM algorithm are extracted the two sides lane line nearest point of camera of adjusting the distance and carry out depth calculation and realize real time distance.

Description

Lane line distance measuring method based on deep learning and binocular vision
Technical field
The invention belongs to computer vision fields, and in particular to a kind of based on the lane detection of deep learning and ranging side Method.
Background technique
Deep learning is a branch of machine learning, and the mind of analytic learning is carried out its object is to establish, simulate human brain Through network, by simulating the mechanism unscrambling data of human brain, essence be by by the combination of extracted low-level feature information from And high-rise attributive character is formed, to find that the distributed nature of data indicates, deep learning is widely used in text, sound at present In terms of the identification field of image, especially image recognition, the method for being based primarily upon physical features is identified compared to traditional images, Have many advantages, such as that accuracy rate height, strong robustness can be realized Weigh sensor image.
Current many vehicles all have the function of the driving of auxiliary driver, for example lane keeps function, lane line early warning function Energy.The function vehicle can be made to be maintained at lane between appropriate location, this function is for having potential deviation or driving automatically Trajectory planning and decision in sailing are crucial.Traditional lane detection method is mentioned dependent on height definitionization, manual feature It takes and heuristic, is typically required post-processing technology, and this often makes computationally intensive, and it is changeable to be unfavorable for road scene Under application extension.And the ranging of lane line need to rely on lane detection as a result, carrying out ranging using laser radar Cost is larger, and the restrictive condition for carrying out ranging using monocular vision is more, so using binocular vision come this close to lane line The target of distance carries out ranging accuracy rate high cost but also low not only, in general, develops a kind of based on deep learning and binocular vision Lane detection, the distance measuring method of feel are necessary, and have very big potentiality and value in intelligent driving field.
Summary of the invention
The purpose of the present invention is to provide a kind of high cost of accuracy rate and low lane line distance measuring methods.Technical solution is such as Under:
A kind of lane line distance measuring method based on deep learning and binocular vision: including image capture module, it is based on convolution The lane detection module of neural network, the range finder module based on binocular vision.Wherein,
Image capture module is corresponding binocular camera, for acquiring the realtime graphic in vehicle travel process;
Lane detection module based on convolutional neural networks is used for accurately identifying the lane line positioned at vehicle two sides Three layers after vgg16 network full articulamentum are become warp lamination, warp lamination pair by vgg16 network frame and full convolutional network FCN The characteristic pattern of convolutional layer is up-sampled, to generate a prediction to each pixel, lane line forecasting problem is converted It is solved to image pixel-class, obtains the binary map and instance graph of prediction lane line by convolutional neural networks, then after passing through Reason and cluster show the lane line that neural network forecast goes out on the image;
Range finder module based on binocular vision, the lane line position that will have been obtained using Binocular Stereo Matching Algorithm SGBM algorithm It sets, extracts the two sides lane line nearest point of camera of adjusting the distance and carry out depth calculation and realize real time distance;
The depth and distance of all the points in view can be obtained, it can lane line is obtained by the result of lane detection On distance apart from camera closest approach;
Step 6: application can have been obtained by physical computing according to actual parameters such as vehicle width, camera installation sites The distance of value.
It is of the present invention that tradition can solve based on deep learning and the lane detection distance measuring method of binocular ranging The problems such as robustness of method for detecting lane lines is poor, real-time is poor, while the range finder module based on binocular vision is added, it can To realize real-time and inexpensive lane detection and ranging, designed by the structure of high-precision neural network, in addition more scenes, The continuous training study of a wide range of sample, it can be achieved that for lane detection precision up to 90% or more, and binocular ranging for The measurement accuracy of short distance is very high, so that high-precision lane detection and ranging may be implemented again, for current intelligent driving Field should be very helpful.
Detailed description of the invention
Fig. 1 convolutional neural networks vgg16+FCN structure
Lane detection and ranging effect picture of the Fig. 2 based on deep learning and binocular vision
Specific embodiment
The present invention includes: image capture module, the lane detection module based on convolutional neural networks, based on binocular vision Range finder module.Wherein image capture module is corresponding binocular camera, real-time in vehicle travel process for acquiring Image, optional left mesh or right mesh image are as display;Lane detection module based on convolutional neural networks is for accurately identifying Lane line positioned at vehicle two sides, using vgg16 network frame and full convolutional network FCN, by three layers after vgg16 network full connection Layer becomes warp lamination, and warp lamination up-samples the characteristic pattern of convolutional layer, so as to generate to each pixel One prediction, is transformed into image pixel-class for lane line forecasting problem to solve, our available prediction lanes by network The binary map and instance graph of line, then shown the lane line that neural network forecast goes out on the image by the work such as post-processing and cluster; The lane line for being used to have obtained using traditional Binocular Stereo Matching Algorithm SGBM algorithm based on the range finder module of binocular vision Position extracts the two sides lane line nearest point of camera of adjusting the distance and carries out depth calculation and realize real time distance.
Step 1: being demarcated to the binocular camera used, the internal reference of acquisition vision collecting module and outer ginseng.
Step 2: making the training set sample met with actual conditions in conjunction with actual conditions.Thus data disclosed in field Collection Tusimple data set and Culane data set filter out nearly 15000 picture, and the size for being no headstock is 1280 × 720 Picture, the network of vgg16 and FCN are trained including daytime, more scene samples such as night, vehicle is crowded, vehicle two is sparse, is used Large data sets are to obtain better robustness, reduce situations such as over-fitting.
Step 3: using the realtime graphic of image capture module acquisition vehicle driving, left and right view is intended to be stored up frame by frame It deposits to carry out next step detection and ranging, as far as possible without the vehicle head part of vehicle when Image Acquisition, to guarantee the essence of detection and ranging Exactness.
Step 4: optional left view or right view, predict each frame by trained lane detection module Obtain the lane detection result of vehicle two sides.
Step 5: the binocular solid using binocular distance measurement module matches SGBM algorithm, all the points in view can be obtained Depth and distance, it can the distance on lane line apart from camera closest approach is obtained by the result of lane detection.
Step 6: application can have been obtained by physical computing according to actual parameters such as vehicle width, camera installation sites The distance of value.

Claims (1)

1. a kind of lane line distance measuring method based on deep learning and binocular vision: including image capture module, based on convolution mind Lane detection module through network, the range finder module based on binocular vision.Wherein,
Image capture module is corresponding binocular camera, for acquiring the realtime graphic in vehicle travel process;
Lane detection module based on convolutional neural networks is used for accurately identifying the lane line positioned at vehicle two sides Three layers after vgg16 network full articulamentum are become warp lamination, warp lamination pair by vgg16 network frame and full convolutional network FCN The characteristic pattern of convolutional layer is up-sampled, to generate a prediction to each pixel, lane line forecasting problem is converted It is solved to image pixel-class, obtains the binary map and instance graph of prediction lane line by convolutional neural networks, then after passing through Reason and cluster show the lane line that neural network forecast goes out on the image;
Range finder module based on binocular vision, the lane line position that will have been obtained using Binocular Stereo Matching Algorithm SGBM algorithm, The two sides lane line nearest point of camera of adjusting the distance is extracted to carry out depth calculation and realize real time distance;
The depth and distance of all the points in view can be obtained, it can by the result of lane detection obtain on lane line away from With a distance from camera closest approach;
Step 6: application value can have been obtained by physical computing according to actual parameters such as vehicle width, camera installation sites Distance.
CN201910636651.2A 2019-07-15 2019-07-15 Lane line distance measurement method based on deep learning and binocular vision Active CN110472508B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910636651.2A CN110472508B (en) 2019-07-15 2019-07-15 Lane line distance measurement method based on deep learning and binocular vision

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910636651.2A CN110472508B (en) 2019-07-15 2019-07-15 Lane line distance measurement method based on deep learning and binocular vision

Publications (2)

Publication Number Publication Date
CN110472508A true CN110472508A (en) 2019-11-19
CN110472508B CN110472508B (en) 2023-04-28

Family

ID=68508674

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910636651.2A Active CN110472508B (en) 2019-07-15 2019-07-15 Lane line distance measurement method based on deep learning and binocular vision

Country Status (1)

Country Link
CN (1) CN110472508B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111309032A (en) * 2020-04-08 2020-06-19 江苏盛海智能科技有限公司 Autonomous obstacle avoidance method and control end of unmanned vehicle
CN112613392A (en) * 2020-12-18 2021-04-06 北京新能源汽车技术创新中心有限公司 Lane line detection method, device and system based on semantic segmentation and storage medium
CN115019278A (en) * 2022-07-13 2022-09-06 北京百度网讯科技有限公司 Lane line fitting method and device, electronic equipment and medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018058356A1 (en) * 2016-09-28 2018-04-05 驭势科技(北京)有限公司 Method and system for vehicle anti-collision pre-warning based on binocular stereo vision
CN109084724A (en) * 2018-07-06 2018-12-25 西安理工大学 A kind of deep learning barrier distance measuring method based on binocular vision
CN109635744A (en) * 2018-12-13 2019-04-16 合肥工业大学 A kind of method for detecting lane lines based on depth segmentation network

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018058356A1 (en) * 2016-09-28 2018-04-05 驭势科技(北京)有限公司 Method and system for vehicle anti-collision pre-warning based on binocular stereo vision
CN109084724A (en) * 2018-07-06 2018-12-25 西安理工大学 A kind of deep learning barrier distance measuring method based on binocular vision
CN109635744A (en) * 2018-12-13 2019-04-16 合肥工业大学 A kind of method for detecting lane lines based on depth segmentation network

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
张新钰;高洪波;赵建辉;周沫;: "基于深度学习的自动驾驶技术综述" *
林付春;张荣芬;刘宇红;: "基于深度学习的智能辅助驾驶***设计" *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111309032A (en) * 2020-04-08 2020-06-19 江苏盛海智能科技有限公司 Autonomous obstacle avoidance method and control end of unmanned vehicle
CN112613392A (en) * 2020-12-18 2021-04-06 北京新能源汽车技术创新中心有限公司 Lane line detection method, device and system based on semantic segmentation and storage medium
CN115019278A (en) * 2022-07-13 2022-09-06 北京百度网讯科技有限公司 Lane line fitting method and device, electronic equipment and medium

Also Published As

Publication number Publication date
CN110472508B (en) 2023-04-28

Similar Documents

Publication Publication Date Title
CN109597087B (en) Point cloud data-based 3D target detection method
CN106919915B (en) Map road marking and road quality acquisition device and method based on ADAS system
CN105892471B (en) Automatic driving method and apparatus
CN105260699B (en) A kind of processing method and processing device of lane line data
CN107576960B (en) Target detection method and system for visual radar space-time information fusion
CN105512623B (en) Based on multisensor travelling in fog day vision enhancement and visibility early warning system and method
CN112700470B (en) Target detection and track extraction method based on traffic video stream
CN110738121A (en) front vehicle detection method and detection system
CN105260712B (en) A kind of vehicle front pedestrian detection method and system
Wu et al. Applying a functional neurofuzzy network to real-time lane detection and front-vehicle distance measurement
CN109460709A (en) The method of RTG dysopia analyte detection based on the fusion of RGB and D information
CN108645375B (en) Rapid vehicle distance measurement optimization method for vehicle-mounted binocular system
DE102011078615B4 (en) OBJECT DETECTION DEVICE AND OBJECT DETECTION PROGRAM
CN111027430B (en) Traffic scene complexity calculation method for intelligent evaluation of unmanned vehicles
CN111209780A (en) Lane line attribute detection method and device, electronic device and readable storage medium
CN110472508A (en) Lane line distance measuring method based on deep learning and binocular vision
CN106446785A (en) Passable road detection method based on binocular vision
CN113592905B (en) Vehicle driving track prediction method based on monocular camera
CN109726699A (en) Electric bicycle based on artificial intelligence occupies car lane recognition methods
CN110516633A (en) A kind of method for detecting lane lines and system based on deep learning
Tourani et al. Motion-based vehicle speed measurement for intelligent transportation systems
CN113822221A (en) Target detection method based on antagonistic neural network and multi-sensor fusion
Gao et al. A real-time lane detection and tracking algorithm
CN111046723B (en) Lane line detection method based on deep learning
CN114419603A (en) Automatic driving vehicle control method and system and automatic driving vehicle

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant