CN109543601A - A kind of unmanned vehicle object detection method based on multi-modal deep learning - Google Patents

A kind of unmanned vehicle object detection method based on multi-modal deep learning Download PDF

Info

Publication number
CN109543601A
CN109543601A CN201811388553.3A CN201811388553A CN109543601A CN 109543601 A CN109543601 A CN 109543601A CN 201811388553 A CN201811388553 A CN 201811388553A CN 109543601 A CN109543601 A CN 109543601A
Authority
CN
China
Prior art keywords
laser radar
image
unmanned vehicle
deep learning
object detection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201811388553.3A
Other languages
Chinese (zh)
Inventor
程洪
林子彧
许成凤
赵洋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Electronic Science and Technology of China
Original Assignee
University of Electronic Science and Technology of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Electronic Science and Technology of China filed Critical University of Electronic Science and Technology of China
Priority to CN201811388553.3A priority Critical patent/CN109543601A/en
Publication of CN109543601A publication Critical patent/CN109543601A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G06V20/584—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10032—Satellite or aerial image; Remote sensing
    • G06T2207/10044—Radar image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Traffic Control Systems (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a kind of unmanned vehicle object detection methods based on multi-modal deep learning, comprising the following steps: (1) data acquire；(2) data are synchronous；(3) laser radar point cloud data is handled；(4) feature extraction；(5) tagsort returns；(6) it projects；(7) redundancy candidate frame is removed；(8) it corrects；(9) three dimensional detection frame is exported.The advantage of present invention combination multisensor overcomes the shortcomings of that image modalities to lane scene ability to express, cope with the scene perception under the complex scene that may be faced in unmanned vehicle traveling.

Description

A kind of unmanned vehicle object detection method based on multi-modal deep learning
Technical field
The present invention relates to the technical fields of pilotless target detection, are based on multi-modal depth more particularly to one kind The unmanned vehicle object detection method of habit.
Background technique
Since automobile is born, the characteristics of convenience and rapidity, is that everybody is obvious to all, but along with vapour Vehicle safety problem is also always the emphasis of car manufactures and consumer's concern.Avoiding road barricade in time is road vehicle peace Full important prerequisite, so how to accomplish to allow vehicle can " identification " present road situation either dynamic object or static object Body just becomes one ring of key of intelligent vehicle scene understanding.
The prior art is usually to use image modalities, but only be difficult to do intelligent vehicle road scene using image modalities One accurately reproduces and analyzes out, and camera has larger parameter variation when illumination variation is big, to make effect by shadow It rings, and ability to express is poor under the scenes weather such as rainy day, snowy day.It therefore, can be by increasing millimetre-wave radar, laser thunder Up to the modal information of equal multisensors, the disadvantage of camera information deficiency is made up, so that intelligent vehicle can prejudge under various circumstances The position of obstacle.
The patent of Publication No. CN107169421A describes a kind of car steering field based on depth convolutional neural networks Scape object detection method, but the patent there is no solve camera for light sensitive, bad weather when effect it is very poor lack Point can not provide when driving reliable scene perception information in unmanned vehicle, be only the framework for changing CNN network, improve Detection effect when single image mode is as input.
The patent of Publication No. CN102253391A describes a kind of pedestrian target track side based on multilasered optical radar Method detects pedestrian using the sensor network of multiple line laser radars composition, but the patent is provided only for pedestrian's Detection, is unable to satisfy unmanned vehicle when driving for the demand of scene perception, and since the point cloud of laser radar is very sparse, far Not as good as image data, so that it is extremely difficult in space search target.
Summary of the invention
In order to solve the above technical problems, the present invention provides a kind of advantage of combination multisensor, overcome image modalities to vehicle The deficiency of road scene ability to express copes with the base of the scene perception under the complex scene that may be faced in unmanned vehicle traveling In the unmanned vehicle object detection method of multi-modal deep learning.
A kind of unmanned vehicle object detection method based on multi-modal deep learning, comprising the following steps:
(1) data acquire: utilizing the image data and laser thunder of in-vehicle camera and mobile lidar acquisition monocular camera The point cloud data reached；
(2) it handles laser radar point cloud data: looking down visual angle by look down that visual angle projection algorithm obtains laser radar Figure；
(3) feature extraction: the deep layer for extracting laser radar top view and image data respectively using convolutional neural networks is special Sign；
(4) tagsort returns, and respectively obtains the candidate frame of laser radar and the multiple targets of image data；
(5) it projects: laser radar being looked down into visual angle figure and the respective candidate frame of image data projects to the feature of same specification Figure；
(6) it removes redundancy candidate frame: using laser radar candidate frame as standard, according to candidate frame central threshold range, abandoning Redundancy anchor in image data does not occur in laser radar such as but the anchor of candidate frame occurs in image data, then deletes；
(7) the nomination correction of image aspects avoids differing biggish laser radar target nomination with practical；
(8) three dimensional detection frame is exported: the laser radar point cloud data array and reservation of rule-based input convolutional network Depth information is returned out together by loss function to be detected the coordinate of vehicle and saves.
Wherein, the loss function be intersect entropy function and image section loss function multinomial and:
Wherein, Nl、NcRespectively represent the number of training of laser radar and image；ynFor the mark value of n-th of sample, it is Vehicle is represented when 1, represents other objects when being 0,It is convolutional neural networks for the predicted value of sample；W is net Parameter in network, multinomial first half are exactly the loss function definition about laser radar, and aft section is damaged about image Considering for function is lost, image impairment function loses L by confidence levelconfL is lost with positioninglocIt forms, x represents prediction block in formula Whether matched with true frame, c represents the matching degree of prediction block Yu true frame, and α is balance factor, when cross validation Value is that 1, l represents prediction block, and g represents true frame, also represent the finally target detection classification level of confidence that returns out and Whether the accurate positioning of each coordinate points of target detection, multiple risks function constraint is exactly to fit close to label data True value improves the performance level of network entirety whereby.
The invention also includes data synchronizing steps, by the image of in-vehicle camera and mobile lidar acquisition monocular camera The point cloud data of data and laser radar synchronizes processing.
The coordinate quantity of the detected vehicle is 8.
The tagsort returns, the sub-step including following non-sequential execution:
A. the feature extracted from laser radar point cloud data is sent into RPN network, mentioned feature is classified and returned Return, obtain the frame coordinate value of multiple targets under 3D coordinate, by converting the candidate frame for obtaining 2D and looking down under visual angle；
B. the extracted feature of institute it will be sent into RPN network from image data, mentioned feature is classified and returned, Obtain the candidate frame of multiple targets on the plane of delineation.
Described looks down visual angle projection algorithm, including following sub-step:
(1) rectangular area for needing to project is incorporated by origin of this vehicle；
(2) Multidimensional numerical dimensionality reduction；
(3) laser radar point is mapped to location of pixels；
(4) translational coordination origin；
(5) normalized coordinates value is to 0~255；
(6) spectrum mapping is done to grayscale image based on the depth information of preservation；
Each laser radar point cloud data can be attached to a reflectivity and receive natural conditions to characterize the point The strength conditions of lower object reflection are the arrays that N row 4 arranges, first three is classified as the three-dimensional coordinate information of laser radar, and the Four are classified as the Reflection intensity information of corresponding points.
The present invention will be schemed using image combination laser radar and its multi-modal lane scene detection method of depth information Laser radar point cloud data is projected directly on image as pixel coordinate system and radar fix system translate rotation, overcomes biography That unites is only insufficient to lane scene ability to express with image modalities, has real-time good and obstacle target detection is accurate Advantage.
Detailed description of the invention
Fig. 1 is flow chart of the invention.
Specific embodiment
With reference to embodiment, the embodiment of the present invention is furthur described in detail.Following embodiment is used for Illustrate the present invention, but is not intended to limit the scope of the invention.
A kind of unmanned vehicle object detection method based on multi-modal deep learning, comprising the following steps:
S1, the image data of monocular camera and the point cloud of laser radar are acquired using in-vehicle camera and mobile lidar Data, and synchronize；
S2, basis look down visual angle projection algorithm process generation radar and look down perspective data, incorporate into first by origin of this vehicle The rectangular area for needing to project, is then mapped to location of pixels for laser radar point.Each point cloud data point can be attached to one Reflectivity receives the strength conditions that object reflects under natural conditions to characterize the point, is the array that N row 4 arranges.First three columns The three-dimensional coordinate information of laser radar is saved, and the 4th column then save the Reflection intensity information of corresponding points.Standardize reflected intensity It is worth 0-255, then grayscale image does spectrum mapping；
S3, laser radar top view further feature is extracted using convolutional neural networks, is operated by convolution operation, pondization Feature extraction is carried out to the laser radar radar top view of input；
Laser radar top view further feature after S4, extraction is sent into RPN network, and mentioned feature is classified and returned Return, obtain laser radar 2D looks down perspective data candidate frame.
S5, image further feature is extracted using convolutional neural networks, the figure to input is operated by convolution operation, pondization As carrying out feature extraction；
Image further feature after S6, extraction is sent into RPN network, and mentioned feature is classified and returned, image is obtained Data candidate frame, as shown in Fig. 1 right part of flg；
S7, the characteristic pattern that laser radar and the respective candidate frame of image are projected to same specification, by each coordinate first divided by The picture of input obtains the frame coordinate on characteristic pattern with the ratio of the size of characteristic pattern, is obtained using the down-sampling pond that is averaged Output, due to the scale that the image and radar of input look down visual angle figure be all it is unified, using average pond；
S8, removal redundancy candidate frame, the present invention is using laser radar candidate frame as standard, according to candidate frame central threshold model It encloses, abandons the redundancy anchor in image.
S9, three dimensional detection frame is obtained, is classified by softmax function, the loss carried by tensorflow Each coordinate points of function -- mean square error (MSE) Lai Huigui three dimensional detection frame.
Wherein, softmax function is the real vector of a N-dimensional to be mapped to another N-dimensional real vector, normalization Whole elements between 0 to 1 to get the probability for belonging to each classification to each detection block,
Tensorflow is that a kind of deep learning frame of open source is similar with caffe, pytorch etc..
It is described that look down visual angle projection algorithm process as follows:
(1) rectangular area for needing to project is incorporated by origin of this vehicle；
Ff=np, logical_and ((x_lidar>0), (x_lidar<q))
Ss=np, logical_and ((y_lidar>-t), (y_lidar<t))
(2) Multidimensional numerical dimensionality reduction；
Indices=np.argwhere (np.logical_and (ff, ss)) .flatten ()
(3) laser radar point is mapped to location of pixels；
For y_points=-t ... .t do
For x_points=-0 ... .q do
X_img=(- y_pointaes) .astype (np.int32)
Y_img=(- x_pointaes) .astype (np.int32)
end for
(4) translational coordination origin；
X_img-=-t
Y_img-=0
(5) normalized coordinates value is to 0~255；
(6) spectrum mapping is done to grayscale image based on the depth information of preservation.
Wherein, np.logical_and () function is logic and function in (1), and x_lidar is the seat of x in radar points cloud Mark, it is to set true which, which meets 0 < x_lidar < q value for ff corresponding position, is unsatisfactory for being set to false, y_lidar is same Reason；(2) np.argwhere () function is index function in, finds the index that ff and ss same position is true, that is, finds out Meet 0 < x_lidar < q in radar points cloud, the index of-t < y_lidar < t point, the index found out is a Multidimensional numerical, is needed One-dimensional array is translated into flatten () function；(3) it converts point cloud data to for two-dimensional image data in, X_img and y_img is the coordinate in top view, and res is the actual distance between two pixels, then passes through astype () Function is converted into integer value；(4) in that radar top view coordinate system and image coordinate system is unified.
A possibility that RPN prediction anchor in the present invention the is background or prospect i.e. process of classification, and extract anchor namely recurrence No matter position of the anchor on characteristic pattern out, take the anchor of what ratio, last testing result it is accurate whether largely by The candidate frame that RPN network provides, which is nominated, to be determined.The mistake nomination candidate frame even provided is more, and it is complicated not only to increase the algorithm time Degree, also has a significant impact to accuracy.Anchor described in the text is the regional frame on characteristic pattern in fact.
The softmax function that the present invention uses is the real vector of a N-dimensional to be mapped to another N-dimensional real vector, Whole elements are normalized between 0 to 1, more classification tasks are finally determined according to its range.
The advantage of present invention combination multisensor provides a kind of complexity that copes with and may face in unmanned vehicle traveling Scene perception under scene has been evaded and has only detected bring lane object scene false retrieval, missing inspection situation with image modalities, so that The result of target detection is relatively reliable.
And the present invention overcomes the shortcomings of that image modalities to lane scene ability to express, propose a kind of image combination laser Radar utilizes the multi-modal vehicle object detection method of deep learning, by the fusion of multi-modal information, when realizing vehicle driving pair In the accurate perception of scene around.
Loss function used in the present invention has quantified the gap between the predicted value of artificial neural network and true value, And train the purpose of neural network to be that and reduce this gap, therefore, minimizes loss function and namely train neural network Target.
Loss function of the present invention be intersect entropy function and image section loss function multinomial and form:
Wherein, Nl、NcRespectively represent the number of training of laser radar and image；ynFor the mark value of n-th of sample, it is Vehicle is represented when 1, represents other objects when being 0,It is convolutional neural networks for the predicted value of sample；W is net Parameter in network, multinomial first half are exactly the loss function definition about laser radar, and aft section is damaged about image Considering for function is lost, image impairment function loses L by confidence levelconfL is lost with positioninglocIt forms, x represents prediction block in formula Whether matched with true frame, c represents the matching degree of prediction block Yu true frame, and α is balance factor, when cross validation Value is that 1, l represents prediction block, and g represents true frame, also represent the finally target detection classification level of confidence that returns out and Whether the accurate positioning of each coordinate points of target detection, multiple risks function constraint is exactly to fit close to label data True value improves the performance level of network entirety whereby.
The above is only a preferred embodiment of the present invention, it is noted that for the ordinary skill people of the art For member, without departing from the technical principles of the invention, several improvements and modifications can also be made, these improve and become Type also should be regarded as protection scope of the present invention.

Claims (7)

1. a kind of unmanned vehicle object detection method based on multi-modal deep learning, which comprises the following steps:
(1) data acquire: utilizing the image data and laser radar of in-vehicle camera and mobile lidar acquisition monocular camera Point cloud data；
(2) it handles laser radar point cloud data: looking down visual angle figure by look down that visual angle projection algorithm obtains laser radar；
(3) feature extraction: the further feature of laser radar top view and image data is extracted respectively using convolutional neural networks；
(4) tagsort returns, and respectively obtains the candidate frame of laser radar and the multiple targets of image data；
(5) it projects: laser radar being looked down into visual angle figure and the respective candidate frame of image data projects to the characteristic pattern of same specification；
(6) it removes redundancy candidate frame: using laser radar candidate frame as standard, according to candidate frame central threshold range, abandoning image Redundancy anchor in data does not occur in laser radar such as but the anchor of candidate frame occurs in image data, then deletes；
(7) the nomination correction of image aspects avoids differing biggish laser radar target nomination with practical；
(8) three dimensional detection frame is exported: the laser radar point cloud data array of rule-based input convolutional network and the depth of reservation Information is returned out together by loss function to be detected the coordinate of vehicle and saves.
2. a kind of unmanned vehicle object detection method based on multi-modal deep learning according to claim 1, feature exist In, the loss function be intersect entropy function and image section loss function multinomial and:
Wherein, Nl、NcRespectively represent the number of training of laser radar and image, ynFor the mark value of n-th of sample,For volume Product neural network is for the predicted value of sample, and W is the parameter in network, and multinomial first half is exactly the damage about laser radar Function definition is lost, aft section is considering about image impairment function, and image impairment function loses L by confidence levelconfWith it is fixed Bit-loss LlocComposition, x represent whether prediction block matches with true frame, and c represents the matching degree of prediction block Yu true frame, and α is flat Weigh the factor, and l represents prediction block, and g represents true frame.
3. a kind of unmanned vehicle object detection method based on multi-modal deep learning according to claim 1, feature exist In, further include data synchronizing step, by in-vehicle camera and mobile lidar acquisition monocular camera image data and laser thunder The point cloud data reached synchronizes processing.
4. a kind of unmanned vehicle object detection method based on multi-modal deep learning according to claim 1, feature exist In the coordinate quantity of the detected vehicle is 8.
5. a kind of unmanned vehicle object detection method based on multi-modal deep learning according to claim 1, feature exist In the tagsort returns, the sub-step including following non-sequential execution:
A. the feature extracted from laser radar point cloud data is sent into RPN network, mentioned feature is classified and returned, is obtained To the frame coordinate value of targets multiple under 3D coordinate, by converting the candidate frame for obtaining 2D and looking down under visual angle；
B. the extracted feature of institute it will be sent into RPN network from image data, mentioned feature is classified and returned, figure is obtained As the candidate frame of targets multiple in plane.
6. a kind of unmanned vehicle object detection method based on multi-modal deep learning according to claim 1, feature exist In described looks down visual angle projection algorithm, including following sub-step:
(1) rectangular area for needing to project is incorporated by origin of this vehicle；
(2) Multidimensional numerical dimensionality reduction；
(3) laser radar point is mapped to location of pixels；
(4) translational coordination origin；
(5) normalized coordinates value is to 0~255；
(6) spectrum mapping is done to grayscale image based on the depth information of preservation.
7. a kind of unmanned vehicle object detection method based on multi-modal deep learning according to claim 1, feature exist In each laser radar point cloud data can be attached to a reflectivity and receive object under natural conditions to characterize the point The strength conditions of reflection, are the arrays that N row 4 arranges, first three is classified as the three-dimensional coordinate information of laser radar, and the 4th is classified as pair The Reflection intensity information that should be put.
CN201811388553.3A 2018-11-21 2018-11-21 A kind of unmanned vehicle object detection method based on multi-modal deep learning Pending CN109543601A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811388553.3A CN109543601A (en) 2018-11-21 2018-11-21 A kind of unmanned vehicle object detection method based on multi-modal deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811388553.3A CN109543601A (en) 2018-11-21 2018-11-21 A kind of unmanned vehicle object detection method based on multi-modal deep learning

Publications (1)

Publication Number Publication Date
CN109543601A true CN109543601A (en) 2019-03-29

Family

ID=65848703

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811388553.3A Pending CN109543601A (en) 2018-11-21 2018-11-21 A kind of unmanned vehicle object detection method based on multi-modal deep learning

Country Status (1)

Country Link
CN (1) CN109543601A (en)

Cited By (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110222626A (en) * 2019-06-03 2019-09-10 宁波智能装备研究院有限公司 A kind of unmanned scene point cloud target mask method based on deep learning algorithm
CN110232315A (en) * 2019-04-29 2019-09-13 华为技术有限公司 Object detection method and device
CN110441789A (en) * 2019-06-27 2019-11-12 南京莱斯信息技术股份有限公司 A kind of non-homogeneous thread laser radar Target Recognition Algorithms
CN110531759A (en) * 2019-08-02 2019-12-03 深圳大学 Path generating method, device, computer equipment and storage medium are explored by robot
CN110706288A (en) * 2019-10-10 2020-01-17 上海眼控科技股份有限公司 Target detection method, device, equipment and readable storage medium
CN110743818A (en) * 2019-11-29 2020-02-04 苏州嘉诺环境工程有限公司 Garbage sorting system and garbage sorting method based on vision and deep learning
CN110781927A (en) * 2019-10-11 2020-02-11 苏州大学 Target detection and classification method based on deep learning under cooperation of vehicle and road
CN110992731A (en) * 2019-12-12 2020-04-10 苏州智加科技有限公司 Laser radar-based 3D vehicle detection method and device and storage medium
CN111209825A (en) * 2019-12-31 2020-05-29 武汉中海庭数据技术有限公司 Method and device for dynamic target 3D detection
CN111242015A (en) * 2020-01-10 2020-06-05 同济大学 Method for predicting driving danger scene based on motion contour semantic graph
CN111274988A (en) * 2020-02-10 2020-06-12 安徽大学 Multispectral-based vehicle weight identification method and device
CN111693050A (en) * 2020-05-25 2020-09-22 电子科技大学 Indoor medium and large robot navigation method based on building information model
CN111695403A (en) * 2020-04-19 2020-09-22 东风汽车股份有限公司 2D and 3D image synchronous detection method based on depth perception convolutional neural network
WO2020199834A1 (en) * 2019-04-03 2020-10-08 腾讯科技（深圳）有限公司 Object detection method and apparatus, and network device and storage medium
CN111860493A (en) * 2020-06-12 2020-10-30 北京图森智途科技有限公司 Target detection method and device based on point cloud data
CN112560717A (en) * 2020-12-21 2021-03-26 青岛科技大学 Deep learning-based lane line detection method
WO2021081808A1 (en) * 2019-10-30 2021-05-06 深圳市大疆创新科技有限公司 Artificial neural network-based object detection system and method
CN113239749A (en) * 2021-04-27 2021-08-10 四川大学 Cross-domain point cloud semantic segmentation method based on multi-modal joint learning
CN113516685A (en) * 2021-07-09 2021-10-19 东软睿驰汽车技术（沈阳）有限公司 Target tracking method, device, equipment and storage medium
CN113780257A (en) * 2021-11-12 2021-12-10 紫东信息科技(苏州)有限公司 Multi-mode fusion weak supervision vehicle target detection method and system
CN114648579A (en) * 2022-02-15 2022-06-21 浙江零跑科技股份有限公司 Multi-branch input laser radar target detection method
CN115019270A (en) * 2022-05-31 2022-09-06 电子科技大学 Automatic driving night target detection method based on sparse point cloud prior information
CN115457496A (en) * 2022-09-09 2022-12-09 北京百度网讯科技有限公司 Automatic driving retaining wall detection method and device and vehicle
CN115527074A (en) * 2022-11-29 2022-12-27 深圳依时货拉拉科技有限公司 Vehicle detection frame generation method and device and computer equipment
CN117475397A (en) * 2023-12-26 2024-01-30 安徽蔚来智驾科技有限公司 Target annotation data acquisition method, medium and device based on multi-mode sensor

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2637126A2 (en) * 2012-03-07 2013-09-11 Ricoh Company, Ltd. Method and apparatus for detecting vehicle
CN103793722A (en) * 2014-03-07 2014-05-14 电子科技大学 Rapid vehicle detection method and device under low resolution
CN105701479A (en) * 2016-02-26 2016-06-22 重庆邮电大学 Intelligent vehicle multi-laser radar fusion recognition method based on target features
WO2016113904A1 (en) * 2015-01-16 2016-07-21 株式会社日立製作所 Three-dimensional-information-calculating device, method for calculating three-dimensional information, and autonomous mobile device
CN107451602A (en) * 2017-07-06 2017-12-08 浙江工业大学 A kind of fruits and vegetables detection method based on deep learning
WO2018056212A1 (en) * 2016-09-22 2018-03-29 株式会社デンソー Object detection device and object detection method
WO2018170472A1 (en) * 2017-03-17 2018-09-20 Honda Motor Co., Ltd. Joint 3d object detection and orientation estimation via multimodal fusion
WO2018204128A1 (en) * 2017-05-01 2018-11-08 Mentor Graphics Development (Deutschland) Gmbh Embedded automotive perception with machine learning classification of sensor data

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2637126A2 (en) * 2012-03-07 2013-09-11 Ricoh Company, Ltd. Method and apparatus for detecting vehicle
CN103793722A (en) * 2014-03-07 2014-05-14 电子科技大学 Rapid vehicle detection method and device under low resolution
WO2016113904A1 (en) * 2015-01-16 2016-07-21 株式会社日立製作所 Three-dimensional-information-calculating device, method for calculating three-dimensional information, and autonomous mobile device
CN105701479A (en) * 2016-02-26 2016-06-22 重庆邮电大学 Intelligent vehicle multi-laser radar fusion recognition method based on target features
WO2018056212A1 (en) * 2016-09-22 2018-03-29 株式会社デンソー Object detection device and object detection method
WO2018170472A1 (en) * 2017-03-17 2018-09-20 Honda Motor Co., Ltd. Joint 3d object detection and orientation estimation via multimodal fusion
WO2018204128A1 (en) * 2017-05-01 2018-11-08 Mentor Graphics Development (Deutschland) Gmbh Embedded automotive perception with machine learning classification of sensor data
CN107451602A (en) * 2017-07-06 2017-12-08 浙江工业大学 A kind of fruits and vegetables detection method based on deep learning

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
林子彧: "多模态深度学习目标检测及其应用", 《中国优秀硕士学位论文全文数据库 (信息科技辑)》 *

Cited By (36)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020199834A1 (en) * 2019-04-03 2020-10-08 腾讯科技（深圳）有限公司 Object detection method and apparatus, and network device and storage medium
CN110232315A (en) * 2019-04-29 2019-09-13 华为技术有限公司 Object detection method and device
CN110222626B (en) * 2019-06-03 2021-05-28 宁波智能装备研究院有限公司 Unmanned scene point cloud target labeling method based on deep learning algorithm
CN110222626A (en) * 2019-06-03 2019-09-10 宁波智能装备研究院有限公司 A kind of unmanned scene point cloud target mask method based on deep learning algorithm
CN110441789A (en) * 2019-06-27 2019-11-12 南京莱斯信息技术股份有限公司 A kind of non-homogeneous thread laser radar Target Recognition Algorithms
CN110531759A (en) * 2019-08-02 2019-12-03 深圳大学 Path generating method, device, computer equipment and storage medium are explored by robot
CN110706288A (en) * 2019-10-10 2020-01-17 上海眼控科技股份有限公司 Target detection method, device, equipment and readable storage medium
CN110781927A (en) * 2019-10-11 2020-02-11 苏州大学 Target detection and classification method based on deep learning under cooperation of vehicle and road
CN110781927B (en) * 2019-10-11 2023-05-23 苏州大学 Target detection and classification method based on deep learning under vehicle-road cooperation
WO2021081808A1 (en) * 2019-10-30 2021-05-06 深圳市大疆创新科技有限公司 Artificial neural network-based object detection system and method
CN110743818A (en) * 2019-11-29 2020-02-04 苏州嘉诺环境工程有限公司 Garbage sorting system and garbage sorting method based on vision and deep learning
CN110992731A (en) * 2019-12-12 2020-04-10 苏州智加科技有限公司 Laser radar-based 3D vehicle detection method and device and storage medium
CN111209825A (en) * 2019-12-31 2020-05-29 武汉中海庭数据技术有限公司 Method and device for dynamic target 3D detection
CN111209825B (en) * 2019-12-31 2022-07-01 武汉中海庭数据技术有限公司 Method and device for dynamic target 3D detection
CN111242015B (en) * 2020-01-10 2023-05-02 同济大学 Method for predicting driving dangerous scene based on motion profile semantic graph
CN111242015A (en) * 2020-01-10 2020-06-05 同济大学 Method for predicting driving danger scene based on motion contour semantic graph
CN111274988A (en) * 2020-02-10 2020-06-12 安徽大学 Multispectral-based vehicle weight identification method and device
CN111274988B (en) * 2020-02-10 2023-03-24 安徽大学 Multispectral-based vehicle weight identification method and device
CN111695403B (en) * 2020-04-19 2024-03-22 东风汽车股份有限公司 Depth perception convolutional neural network-based 2D and 3D image synchronous detection method
CN111695403A (en) * 2020-04-19 2020-09-22 东风汽车股份有限公司 2D and 3D image synchronous detection method based on depth perception convolutional neural network
CN111693050A (en) * 2020-05-25 2020-09-22 电子科技大学 Indoor medium and large robot navigation method based on building information model
CN111860493B (en) * 2020-06-12 2024-02-09 北京图森智途科技有限公司 Target detection method and device based on point cloud data
CN111860493A (en) * 2020-06-12 2020-10-30 北京图森智途科技有限公司 Target detection method and device based on point cloud data
CN112560717B (en) * 2020-12-21 2023-04-21 青岛科技大学 Lane line detection method based on deep learning
CN112560717A (en) * 2020-12-21 2021-03-26 青岛科技大学 Deep learning-based lane line detection method
CN113239749A (en) * 2021-04-27 2021-08-10 四川大学 Cross-domain point cloud semantic segmentation method based on multi-modal joint learning
CN113516685A (en) * 2021-07-09 2021-10-19 东软睿驰汽车技术（沈阳）有限公司 Target tracking method, device, equipment and storage medium
CN113780257A (en) * 2021-11-12 2021-12-10 紫东信息科技(苏州)有限公司 Multi-mode fusion weak supervision vehicle target detection method and system
CN114648579A (en) * 2022-02-15 2022-06-21 浙江零跑科技股份有限公司 Multi-branch input laser radar target detection method
CN115019270A (en) * 2022-05-31 2022-09-06 电子科技大学 Automatic driving night target detection method based on sparse point cloud prior information
CN115019270B (en) * 2022-05-31 2024-04-19 电子科技大学 Automatic driving night target detection method based on sparse point cloud priori information
CN115457496A (en) * 2022-09-09 2022-12-09 北京百度网讯科技有限公司 Automatic driving retaining wall detection method and device and vehicle
CN115457496B (en) * 2022-09-09 2023-12-08 北京百度网讯科技有限公司 Automatic driving retaining wall detection method and device and vehicle
CN115527074A (en) * 2022-11-29 2022-12-27 深圳依时货拉拉科技有限公司 Vehicle detection frame generation method and device and computer equipment
CN117475397A (en) * 2023-12-26 2024-01-30 安徽蔚来智驾科技有限公司 Target annotation data acquisition method, medium and device based on multi-mode sensor
CN117475397B (en) * 2023-12-26 2024-03-22 安徽蔚来智驾科技有限公司 Target annotation data acquisition method, medium and device based on multi-mode sensor

Similar Documents

Publication Publication Date Title
CN109543601A (en) A kind of unmanned vehicle object detection method based on multi-modal deep learning
Xia et al. Geometric primitives in LiDAR point clouds: A review
CN110415342B (en) Three-dimensional point cloud reconstruction device and method based on multi-fusion sensor
Maddern et al. 1 year, 1000 km: The oxford robotcar dataset
Geiger et al. Vision meets robotics: The kitti dataset
CN113111887B (en) Semantic segmentation method and system based on information fusion of camera and laser radar
Zhou et al. Self‐supervised learning to visually detect terrain surfaces for autonomous robots operating in forested terrain
CN110009765A (en) A kind of automatic driving vehicle contextual data system and scene format method for transformation
CN114254696A (en) Visible light, infrared and radar fusion target detection method based on deep learning
Zhao et al. Road network extraction from airborne LiDAR data using scene context
CN113989797A (en) Three-dimensional dynamic target detection method and device based on voxel point cloud fusion
EP4377913A1 (en) Training method for training a change detection system, training set generating method therefor, and change detection system
US20220410381A1 (en) Systems and methods for picking objects using 3-d geometry and segmentation
CN117058646B (en) Complex road target detection method based on multi-mode fusion aerial view
CN114639115A (en) 3D pedestrian detection method based on fusion of human body key points and laser radar
CN114298151A (en) 3D target detection method based on point cloud data and image data fusion
CN116612468A (en) Three-dimensional target detection method based on multi-mode fusion and depth attention mechanism
CN116664856A (en) Three-dimensional target detection method, system and storage medium based on point cloud-image multi-cross mixing
Jindal et al. Bollard segmentation and position estimation from lidar point cloud for autonomous mooring
CN114494248A (en) Three-dimensional target detection system and method based on point cloud and images under different visual angles
Zhang et al. Infrastructure 3D Target detection based on multi-mode fusion for intelligent and connected vehicles
CN117372991A (en) Automatic driving method and system based on multi-view multi-mode fusion
CN116817891A (en) Real-time multi-mode sensing high-precision map construction method
Lertniphonphan et al. 2d to 3d label propagation for object detection in point cloud
CN116740514A (en) Space-time error tolerant multi-agent cooperative sensing method and device and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20190329

RJ01 Rejection of invention patent application after publication