CN108875902A - Neural network training method and device, vehicle detection estimation method and device, storage medium - Google Patents

Neural network training method and device, vehicle detection estimation method and device, storage medium Download PDF

Info

Publication number
CN108875902A
CN108875902A CN201711262814.2A CN201711262814A CN108875902A CN 108875902 A CN108875902 A CN 108875902A CN 201711262814 A CN201711262814 A CN 201711262814A CN 108875902 A CN108875902 A CN 108875902A
Authority
CN
China
Prior art keywords
vehicle
training
network
vehicle location
detection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201711262814.2A
Other languages
Chinese (zh)
Inventor
王曼晨
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Megvii Technology Co Ltd
Beijing Maigewei Technology Co Ltd
Original Assignee
Beijing Megvii Technology Co Ltd
Beijing Maigewei Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Megvii Technology Co Ltd, Beijing Maigewei Technology Co Ltd filed Critical Beijing Megvii Technology Co Ltd
Priority to CN201711262814.2A priority Critical patent/CN108875902A/en
Publication of CN108875902A publication Critical patent/CN108875902A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a kind of neural network training method and devices, vehicle detection estimation method and device, storage medium, include for vehicle location detection and Attitude estimation, the neural network training method：Training data is obtained, the training data includes the picture for being labelled with vehicle location coordinate and three-dimensional vehicle encirclement frame vertex projection coordinate；Following steps are repeated until meeting setting condition：Using the training data training vehicle location detection branches network, the training exported using the training data and the vehicle location detection branches network estimates branching networks with the vehicle detection frame training vehicle attitude.End-to-end training may be implemented in the neural network training method, and marks precision without higher data, and the accuracy rate of network can be improved.The vehicle neural metwork training device, vehicle detection estimation method and device, storage medium have the advantages that similar.

Description

Neural network training method and device, vehicle detection estimation method and device, storage Medium
Technical field
The present invention relates to the training method and device of technical field of image detection more particularly to a kind of neural network, it is used for The method and device and storage medium of vehicle location detection and Attitude estimation.
Background technique
In the past few years, with the development of science and technology, by intelligence system realize automobile automatic Pilot academia with Industrial circle all becomes more and more very powerful and exceedingly arrogant.And in automatic Pilot, one of most basic function be exactly detect vehicle, pedestrian, Barrier etc., this has been converted to object detection problem.Object detection problem in image is always computer vision field Hot topic, the arrival in deep learning epoch brings very big promotion to detection accuracy.Deep learning is examined in 2D object Some very important achievements are obtained in survey, such as RCNN (Regions with Convolutional Neural Network Features has the region of convolutional neural networks feature), the work such as FastRCNN, FasterRCNN.In automatic Pilot, to the greatest extent Pipe object detection provides important information for content in scene, but two-dimensional detection frame is for the description of three-dimensional real-world scene It is still insufficient.For the automobile of an automatic Pilot, it is allowed for according to the information reason extracted from picture scene It solves current traffic condition and predicts possible road conditions of next second.In order to which speed, the direction etc. that obtain surrounding vehicles are believed Breath needs to carry out Accurate Prediction to the vehicle attitude of different moments.In addition, in order to better understand road traffic condition, how Vehicle effectively around description is extremely important.
Summary of the invention
A series of concept of reduced forms is introduced in Summary, this will be in specific embodiment part into one Step is described in detail.Summary of the invention is not meant to attempt to limit the pass of technical solution claimed Key feature and essential features do not mean that the protection scope for attempting to determine technical solution claimed more.
To solve the above-mentioned problems, the present invention provides a kind of training method of neural network, the neural network includes Vehicle location detection branches network and vehicle attitude estimate branching networks, and the training method includes：
Training data is obtained, the training data includes being labelled with vehicle location coordinate and three-dimensional vehicle encirclement frame vertex throwing The picture of shadow coordinate；
Following steps are repeated until meeting setting condition：
The vehicle location detection branches network is trained using the training data,
The training vehicle detection frame training exported using the training data and the vehicle location detection branches network The vehicle attitude estimates branching networks.
In one embodiment of the invention, the neural network further include deep-neural-network, region suggest network and Area-of-interest pond layer, the training vehicle location detection branches network include：
The training data is handled by the deep-neural-network to obtain trained characteristic pattern；
Suggest that network handles with characteristic pattern to obtain trained candidate frame the training by the region；
The trained characteristic pattern and training candidate frame are inputted into the area-of-interest Chi Huacengzuochi Hua Chu Reason, to obtain the training with the corresponding training first eigenvector of candidate frame；
Training first eigenvector is inputted into the vehicle location detection branches network, is used with obtaining the training Vehicle detection frame.
In one embodiment of the invention, the training vehicle attitude estimation branching networks include：
Training vehicle detection frame is inputted into the area-of-interest Chi Huacengzuochiization processing, to obtain the instruction Practice with the corresponding training second feature vector of vehicle detection frame；
Training second feature vector is inputted into the vehicle attitude and estimates branching networks, is estimated with obtaining vehicle attitude Count information.
According to another aspect of the present invention, a kind of training device of neural network, the neural network packet are additionally provided Vehicle location detection branches network and vehicle attitude estimation branching networks are included, the training device includes：
Data capture unit, for obtaining training data, the training data includes being labelled with vehicle location coordinate and vehicle The three-dimensional picture for surrounding frame vertex projection coordinate；
First training unit, for utilizing the training data training vehicle location detection branches network；
Second training unit, the training for being exported using the training data and the vehicle location detection branches network Branching networks are estimated with the vehicle detection frame training vehicle attitude.
In one embodiment of the invention, the neural network further include deep-neural-network, region suggest network and Area-of-interest pond layer, first training unit execute following steps：
The training data is handled by the deep-neural-network to obtain trained characteristic pattern；
Suggest that network handles with characteristic pattern to obtain trained candidate frame the training by the region；
The trained characteristic pattern and training candidate frame are inputted into the area-of-interest Chi Huacengzuochi Hua Chu Reason, to obtain the training with the corresponding training first eigenvector of candidate frame；
Training first eigenvector is inputted into the position detection branching networks, to obtain the training vehicle Detection block.
In one embodiment of the invention, second training unit executes following steps：
Training vehicle detection frame is inputted into the area-of-interest Chi Huacengzuochiization processing, to obtain the instruction Practice with the corresponding training second feature vector of vehicle detection frame；
Training second feature vector is inputted into the vehicle attitude and estimates branching networks, is estimated with obtaining vehicle attitude Count information.
The training method and training device of neural network according to the present invention, one side training data, which relies only on, is labelled with vehicle Position coordinates and three-dimensional vehicle surround the picture of frame vertex projection coordinate, do not need additional finer mark, to training number It is low according to the requirement of mark, reduce trained difficulty；On the other hand, while vehicle location detection branches network and vehicle attitude being trained Estimate branching networks, the two-dimensional detection frame of vehicle and the Attitude estimation information of vehicle can be exported simultaneously based on training data, no Need manually pretreatment and subsequent processing.
Further, the training method and device of neural network according to the present invention estimates branch in training vehicle attitude The training exported when network by vehicle location detection branches network assists pre- measuring car with the corresponding feature vector of vehicle detection frame Posture information improves the accuracy rate of vehicle attitude estimation；And the addition of vehicle attitude estimation task also improves vehicle The accuracy of position detection, two kinds of tasks assist mutually, promote mutually.
According to another aspect of the present invention, a kind of nerve net for vehicle location detection and Attitude estimation is additionally provided The method of network, including：
Obtain image to be detected；
Using in advance training for vehicle location detection and Attitude estimation neural network to described image to be detected into Row processing, to obtain vehicle detection frame and vehicle attitude estimated information；
Wherein, the neural network for vehicle location detection and Attitude estimation of the training in advance includes vehicle location inspection It surveys branching networks and vehicle attitude estimates branching networks,
The vehicle location detection branches network is used to export vehicle detection frame based on testing image；
The vehicle attitude estimation branching networks are used to export vehicle based on the testing image and the vehicle detection frame Attitude estimation information.
In one embodiment of the invention, the nerve for vehicle location detection and Attitude estimation of the training in advance Network further includes that network and area-of-interest pond layer are suggested in deep-neural-network, region, the method also includes：
The testing image is handled to obtain characteristic pattern by the deep-neural-network；
Suggest that network handles to obtain candidate frame the characteristic pattern by the region；
The candidate frame and the characteristic pattern are inputted into the area-of-interest Chi Huacengzuochiization processing, it is described to obtain The corresponding first eigenvector of candidate frame.
In one embodiment of the invention, further include：
The first eigenvector, which is based on, by the position detection branching networks exports the vehicle detection frame and confidence Degree；
The vehicle detection frame is inputted into the area-of-interest Chi Huacengzuochiization processing, to obtain the vehicle detection The corresponding second feature vector of frame；
Estimate that branching networks are based on the second feature vector and export the vehicle attitude estimation by the vehicle attitude Information.
In one embodiment of the invention, the vehicle attitude estimated information includes that three-dimensional vehicle surrounds the projection of frame vertex Coordinate or three-dimensional vehicle surround the position of the relatively described vehicle detection frame center point coordinate of frame vertex projection coordinate.
In one embodiment of the invention, further include：User setting is based on according to the vehicle detection frame and confidence level Or selection obtains the vehicle location testing result.
In one embodiment of the invention, further include：According to the vehicle location testing result and the vehicle attitude Estimated information obtains the vehicle attitude estimated result.
According to another aspect of the present invention, a kind of nerve net for vehicle location detection and Attitude estimation is additionally provided The device of network, including：
Image collection module, for obtaining image to be detected；
Vehicle location detection and Attitude estimation module, for handling described image to be detected, to obtain vehicle inspection Survey frame and vehicle attitude estimated information；
Wherein, the vehicle location detection and Attitude estimation module include vehicle location detection branches network and vehicle attitude Estimate branching networks,
The vehicle location detection branches network is used to export vehicle detection frame based on testing image；
The vehicle attitude estimation branching networks are used to export vehicle based on the testing image and the vehicle detection frame Attitude estimation information.
In one embodiment of the invention, the vehicle location detection and Attitude estimation module further include：
Deep-neural-network, for being handled the testing image to obtain characteristic pattern；
Network is suggested in region, for being handled the characteristic pattern to obtain candidate frame；
Area-of-interest pond layer, for doing pondization processing to the candidate frame and the characteristic pattern, to obtain the time Select the corresponding first eigenvector of frame.
In one embodiment of the invention, the position detection branching networks are based on the first eigenvector and export institute State vehicle detection frame and confidence level；
Area-of-interest pond layer is also used to do the vehicle detection frame and the characteristic pattern pondization processing, with To the corresponding second feature vector of the vehicle detection frame；
The vehicle attitude estimation branching networks are based on the second feature vector and export the vehicle attitude estimated information.
In one embodiment of the invention, the vehicle attitude estimated information includes that three-dimensional vehicle surrounds the projection of frame vertex Coordinate or three-dimensional vehicle surround the position of the relatively described vehicle detection frame center point coordinate of frame vertex projection coordinate.
In one embodiment of the invention, the vehicle location detection and Attitude estimation module further include：Vehicle location Testing result acquisition submodule, for being based on user setting or the selection acquisition vehicle according to the vehicle detection frame and confidence level Position detection result.
In one embodiment of the invention, the vehicle location detection and Attitude estimation module further include：Vehicle attitude Estimated result acquisition submodule, for according to the vehicle location testing result and vehicle attitude estimated information acquisition Vehicle attitude estimated result.
Method and apparatus according to the present invention for vehicle location detection and Attitude estimation, can be to the to be detected of input Image carries out vehicle location detection and vehicle attitude estimation simultaneously, and instructs due to using by above-mentioned training method or device The experienced neural network for vehicle location detection and Attitude estimation, has higher accuracy.
Further, according to the present invention to carry out vehicle for the method and apparatus of vehicle location detection and Attitude estimation When Attitude estimation, vehicle attitude estimation problem is converted to three-dimensional 8 three-dimensional tops for surrounding frame (cube) in order to predict vehicle Projection (the i.e. 8 two-dimensional coordinates) problem of point on picture, and what is preferably predicted in actual task is that this 8 two dimensions are sat The relative position for marking the vehicle detection frame central point detected relative to vehicle location, further improves the accurate of prediction Rate.
According to another aspect of the present invention, a kind of training device of neural network, the training device packet are additionally provided Memory and processor are included, the computer program run by the processor, the computer journey are stored on the memory Sequence executes the training method of neural network as described above when being run by the processor.
According to another aspect of the present invention, a kind of storage medium is additionally provided, calculating is stored on the storage medium Machine program, the computer program execute the training method of neural network as described above at runtime.
According to another aspect of the present invention, a kind of nerve net for vehicle location detection and Attitude estimation is additionally provided The device of network, described device include memory and processor, and the calculating run by the processor is stored on the memory Machine program, the computer program are executed as described above when being run by the processor for vehicle location detection and posture The method of estimation.
According to another aspect of the present invention, a kind of storage medium is additionally provided, calculating is stored on the storage medium Machine program, the computer program execute the method as described above for vehicle location detection and Attitude estimation at runtime.
Detailed description of the invention
The following drawings of the embodiment of the present invention is incorporated herein as part of the present invention for the purpose of understanding the present invention.It is shown in attached drawing Embodiments of the present invention and descriptions thereof are used to explain the principles of the present invention.In the accompanying drawings,
Fig. 1 is for realizing the training method and device of neural network according to an embodiment of the present invention and for vehicle position Set the schematic block diagram of the exemplary electronic device of the method and device of detection and Attitude estimation；
Fig. 2 is the schematic diagram according to initial training in the training method of the neural network of the embodiment of the present invention；
Fig. 3 is according to vehicle location detection branches network training in the training method of the neural network of the embodiment of the present invention Schematic diagram；
Fig. 4 is that branching networks training is estimated according to vehicle attitude in the training method of the neural network of the embodiment of the present invention Schematic diagram；
Fig. 5 is the schematic diagram according to repetitive exercise in the training method of the neural network of the embodiment of the present invention；
Fig. 6 is the schematic block diagram according to the training device of the neural network of the embodiment of the present invention；
Fig. 7 is the schematic diagram for vehicle location detection and the method for Attitude estimation according to the embodiment of the present invention；
Fig. 8 is the method detailed maps for vehicle location detection and Attitude estimation according to the embodiment of the present invention.
Fig. 9 is the structural schematic diagram for vehicle location detection and the device of Attitude estimation according to the embodiment of the present invention；
Figure 10 is the detailed construction schematic diagram of vehicle location detection and Attitude estimation module in Fig. 9.
Specific embodiment
In the following description, a large amount of concrete details are given so as to provide a more thorough understanding of the present invention.So And it will be apparent to one skilled in the art that the embodiment of the present invention may not need one or more of these details And it is carried out.In other examples, in order to avoid obscuring with the embodiment of the present invention, for more well known in the art Technical characteristic is not described.
It should be understood that the present invention can be implemented in different forms, and should not be construed as being limited to propose here Embodiment.On the contrary, provide these embodiments will make it is open thoroughly and completely, and will fully convey the scope of the invention to Those skilled in the art.In the accompanying drawings, for clarity, the size and relative size of component, element etc. may be exaggerated.From beginning Identical element is indicated to whole same reference numerals.
In order to enable the object, technical solutions and advantages of the present invention become apparent, root is described in detail below with reference to accompanying drawings According to example embodiments of the present invention.Obviously, described embodiment is only a part of the embodiments of the present invention, rather than this hair Bright whole embodiments, it should be appreciated that the present invention is not limited by example embodiment described herein.Based on described in the present invention The embodiment of the present invention, those skilled in the art's obtained all other embodiment in the case where not making the creative labor It should all fall under the scope of the present invention.
Firstly, Fig. 1 is training method and device for realizing neural network according to an embodiment of the present invention and is used for The schematic block diagram 100 of the exemplary electronic device of the method and device of vehicle location detection and Attitude estimation.As shown in Figure 1, electric Sub- equipment 100 includes one or more processors 102, one or more storage devices 104, input/output device 106, communication Interface 108 and one or more image collecting devices 110, the company that these components pass through bus system 112 and/or other forms The interconnection of connection mechanism (not shown).It should be noted that the component and structure of electronic equipment 100 shown in FIG. 1 are only exemplary, rather than Restrictive, as needed, the electronic equipment also can have other assemblies and structure, can not also include part above-mentioned Component, such as can not include image collecting device.
The processor 102 typicallys represent the place for being capable of handling data or explanation and execute instruction of any type or form Manage unit.In general, processor can be central processing unit (CPU) or there is data-handling capacity and/or instruction to hold The processing unit of the other forms of row ability, and can control other components in the electronic equipment 100 to execute expectation Function.In addition, processor is also possible to neural network processor, graphics processor (GPU), field programmable gate array (FPGA) or digital signal processor (DSP) or their one or more combinations.In a particular embodiment, processor 102 can receive the instruction from software application or module.These instruction can cause processor 102 completion be described herein and/ Or the function of the one or more example embodiments shown.
The storage device 104 may include one or more computer program products, and the computer program product can To include various forms of computer readable storage mediums, such as volatile memory and/or nonvolatile memory.It is described easy The property lost memory for example may include random access memory (RAM) and/or cache memory (cache) etc..It is described non- Volatile memory for example may include read-only memory (ROM), hard disk, flash memory etc..In the computer readable storage medium On can store one or more computer program instructions, processor 102 can run described program instruction, to realize hereafter institute The client functionality (realized by processor) in the embodiment of the present invention stated and/or other desired functions.In the meter Can also store various application programs and various data in calculation machine readable storage medium storing program for executing, for example, the application program use and/or The various data etc. generated.
The input/output device 106 can be user and be used to input instruction and be output to the outside the device of various information, Such as input unit may include one or more of button, keyboard, mouse, microphone and touch screen etc..Output device can To include one or more of display, loudspeaker, projector, network interface card etc..
Communication interface 108 widely indicate any type or form can promote exemplary electronic device 100 and one or The equipment or adapter of communication between multiple optional equipments.For example, communication interface 108 can promote electronic equipment 100 with before The communication at end or accessory electronic device and back-end server or cloud.The example of communication interface 108 is including but not limited to wired Network interface (such as network interface card), radio network interface (such as wireless network interface card), modem and it is any its His suitable interface.In one embodiment, communication interface 108 is taken by the direct-connected offer of the network with such as internet to long-range Device/remote front-end equipment of being engaged in is direct-connected.In a particular embodiment, communication interface 108 by with dedicated network, such as video monitoring The direct-connected offer of the networks such as network, Skynet system network is direct-connected to remote server/remote front-end equipment.Communication interface 108 It can also provide indirectly this by any other connection properly connected.
Described image acquisition device 110 can shoot the desired image of user (such as photo, video etc.), and will be clapped The image taken the photograph is stored in the storage device 104 for the use of other components.Image collecting device 110 can use various conjunctions Suitable imaging sensor or cam device.
Illustratively, for realizing the training method and device of neural network according to an embodiment of the present invention and for vehicle The electronic equipment of position detection and the method and device of Attitude estimation can integrate in a device of vehicle, can also divide Cloth is arranged in the different device of vehicle.
Illustratively, it can be deployed in for realizing the training method and device of neural network according to an embodiment of the present invention Cloud or server end, the method and device for vehicle location detection and Attitude estimation can dispose in the car.
Fig. 2 is the schematic diagram according to initial training in the training method of the neural network of the embodiment of the present invention；According to Fig. 3 The schematic diagram of repetitive exercise in the training method of the neural network of the embodiment of the present invention；Fig. 4 is the mind according to the embodiment of the present invention The schematic diagram of vehicle location detection branches network training in training method through network；Fig. 5 is the mind according to the embodiment of the present invention The schematic diagram of vehicle attitude estimation branching networks training in training method through network.Below in conjunction with Fig. 2~Fig. 5 to according to this The training method of inventive embodiments is described.
The embodiment of the present invention discloses a kind of training method for neural network, and the neural network is examined for vehicle location It surveys and Attitude estimation, the neural network includes vehicle location detection branches network and vehicle attitude estimation branching networks, such as Fig. 2 The shown training method includes：
Firstly, in step s 201, obtaining training data, the training data includes being labelled with vehicle location coordinate and vehicle The three-dimensional picture for surrounding frame vertex projection coordinate.
In the present embodiment, prediction vehicle location coordinate is converted by vehicle location test problems, vehicle attitude is estimated Problem is converted into three-dimensional vehicle and surrounds frame vertex projection coordinate, therefore only needs to be labelled with vehicle location seat when obtaining training data Mark and three-dimensional vehicle surround the picture of frame vertex projection coordinate, are not necessarily to finer mark, reduce data acquisition difficulty.
It should be noted that the three-dimensional three-dimensional box for surrounding encirclement vehicle in the picture that frame refers to, is, for example, one vertical Cube, correspondingly, it is, for example, projection of eight vertex of the cube on picture that three-dimensional vehicle, which surrounds frame vertex projection coordinate, Point coordinate.It surrounds frame that is, converting the three-dimensional of prediction vehicle for vehicle attitude estimation problem in the application (namely one vertical Cube) subpoint of corresponding eight vertex on picture, it is easier in this way and vehicle location is detected and trained together, training effect Also more preferable.
Then, in step S202, the initial training of vehicle location detection branches network is carried out.
After obtaining training data in step s 201, then using the training data (be labelled with vehicle location coordinate and The picture of three-dimensional vehicle encirclement frame vertex projection coordinate) the training vehicle location detection branches network, namely carry out vehicle position Set the initial training of detection branches network.
The detailed initial training process of vehicle location detection branches network is as shown in figure 3, include：
Step 301, training data is obtained, the training data includes the picture and vehicle three for being labelled with vehicle location coordinate Dimension surrounds frame vertex projection coordinate.
Step 302, trained characteristic pattern is obtained by deep-neural-network.Vehicle location coordinate and vehicle will be labelled with The picture that three-dimensional surrounds frame vertex projection coordinate inputs deep-neural-network, just does by convolution or pondization by whole figure of input Piece transforms into trained characteristic pattern, that is, extracts the feature of picture.As an example, deep-neural-network can be convolution Layer, ReLu layers, combination one or more of in pond layer etc., can according to need and suitably constructed.
Step S303, by the region suggest network (i.e. RPN network) to the training with characteristic pattern handled with Obtain trained candidate frame.
As an example, region suggests that network (i.e. RPN network) includes sliding window, two parallel convolutional layers/complete Articulamentum and other required network layers.Illustratively, it is trained candidate to suggest that network (i.e. RPN network) obtains for region The process of frame is：The training characteristic pattern that deep-neural-network is obtained and inputted, with 3*3 (i.e. 3scale*3*aspect Ratio sliding window slip scan), by being mapped as after the feature vector of low-dimensional using ReLU, each sliding window position corresponds to k Anchors (candidate region)；Then, the convolutional layer that low-dimensional feature vector is inputted to two parallel connections, is respectively used to return Whether region proposals generates bounding-box and to being that prospect or background are given a mark.In the present embodiment, RPN network Concrete operations are equivalent to the candidate frame in 3 scale and 3 ratio of each point generation of trained characteristic pattern, then to these Candidate frame is adjusted, and output candidate frame belongs to the probability and coordinate modification value of prospect or background.
Step S304, by the trained candidate frame and the characteristic pattern input region of interest Chi Huacengzuochi Hua Chu Reason, to obtain the training with the corresponding training first eigenvector of candidate frame.
In step S303, region suggests the more than one candidate frame of network (i.e. RPN network) output (in the present embodiment In, candidate frame is rectangle frame), pond (Pooling) is carried out to multiple semi-cylindrical hills (ROI) in this step, ROI The process of Pooling is exactly that box rectangle frame of different sizes one by one is all mapped to the rectangle frame that size is WxH, output Batch vector, wherein the value of batch is equal to the number of roi, and the size of vector (vector) is channelxWxH, Channel indicates channel, and H and W are a layer hyper parameters.
The process of ROI Pooling is that first the coordinate in roi is mapped on trained characteristic pattern, and mapping ruler is, for example, Each coordinate divided by the ratio of the size of input picture and trained characteristic pattern, the box obtained on trained characteristic pattern is sat After mark, exported using pond；Since the picture size of input is different, so the spp pooling that we use here, Spp pooling needs to calculate the corresponding two pixels reflection of the result after pooling society during pooling and arrives Shared range, then carries out taking maximum or be averaged in that range on feature map.
That is, for each candidate frame, the pond area-of-interest (RoI) layer extracted from characteristic pattern the feature of regular length to Amount.The pond RoI layer is using maximum pond or average pond method by the Feature Conversion in any effective area-of-interest at tool There is the small characteristics map of the fixed space range of H × W (for example, 7 × 7), wherein H and W is a layer hyper parameter, independently of any specific RoI layer.Herein, each RoI is a rectangle frame, is indicated with a four-tuple (r, c, h, w), wherein (r, c) is fixed The justice top left co-ordinate of rectangle frame, (h, w) define the height and width of rectangle frame.
Illustratively, in the present embodiment, using maximum pond method, RoI maximum pondization is exactly by the RoI window of h × w It is divided into the grid of H × W, the size of each child window is approximately h/H × w/W, then the value of each child window of maximum pondization to phase The output grid cell answered.
Step S305 inputs the training at the vehicle location detection branches network with first eigenvector Reason, to obtain the training vehicle detection frame.
After obtaining the training first eigenvector, it is inputted vehicle detection position branch network, is examined in vehicle Location is set in branching networks, obtains training vehicle detection frame by the Return Law.More specifically, vehicle location detection branches network Output rectangle frame belongs to the probability of each type of vehicle, and suggests the time of network (i.e. RPN network) output relative to region The correction value of frame is selected, then judges that each detection block belongs to the score of vehicle and the object detection frame (candidate relative to RPN output Frame) correction value, then exported using the highest detection block of score as vehicle detection frame.
It should be noted that vehicle detection frame is the rectangle frame for the encirclement vehicle that detected on picture, by a left side for rectangle Totally 4 parameters indicate (the i.e. similar above-mentioned upper left corner (r to upper angular vertex with length and width；C) and its height and width (h；w)).
Then, in step S203, the initial training of vehicle attitude estimation branching networks is carried out.
After obtaining training data in step s 201, then (it is labelled with three-dimensional vehicle using the training data and surrounds frame Vertex projection coordinate and three-dimensional vehicle surround the picture of frame vertex projection coordinate) and vehicle location detection branches network output Training estimate branching networks with the vehicle detection frame training vehicle attitude.
Vehicle attitude estimates the detailed initial training process of branching networks as shown in figure 4, including：
Step 401, training data is obtained, the training data includes being labelled with vehicle location coordinate and three-dimensional vehicle encirclement The picture of frame vertex projection coordinate.
Step 402, trained characteristic pattern is obtained by deep-neural-network.Vehicle location coordinate and vehicle will be labelled with The picture that three-dimensional surrounds frame vertex projection coordinate inputs deep-neural-network, by the way that the whole picture of input is transformed into training With characteristic pattern, that is, extract the feature of picture.As an example, deep-neural-network can be convolutional layer, ReLu layers, pond layer One or more of combinations, can according to need and suitably constructed in.
Step 403, use vehicle detection frame described trained special the training by area-of-interest pond layer Sign figure does pondization processing, to obtain the training corresponding training second feature vector of vehicle detection frame.
The training that vehicle location detection branches network exports is trained with vehicle detection frame and deep-neural-network It is input to area-of-interest pond layer with characteristic pattern, handles to obtain training vehicle detection frame pair by area-of-interest pond layer The training answered second feature vector.
Since the size of the vehicle detection frame of vehicle location detection branches network output is different (because there is cart to have Trolley, headlight for vehicle cause the vehicle size on picture also different), so feature vector dimension is also different.Feel emerging Interesting pool area layer is the feature vector that the feature vector of these different dimensions is converted into same dimension by some modes, Then the training of subsequent vehicle attitude prediction is carried out.Pond process with it is aforementioned similar, i.e., training with vehicle detection frame will be mapped to instruction On experienced characteristic pattern, to extract the feature vector of regular length.
Step 404, the training vehicle attitude estimation branching networks are inputted with the training of second feature vector to carry out Processing, to obtain vehicle attitude estimated information.
After obtaining the training second feature vector, vehicle attitude estimation branching networks are inputted, in vehicle appearance State estimates branching networks, obtains vehicle attitude estimated information by the Return Law.As previously mentioned, in the present embodiment, by vehicle appearance State forecasting problem converts projection (i.e. 8 of three-dimensional 8 three-dimensional vertices for surrounding frame (cube) on picture in order to predict vehicle A two-dimensional coordinate) problem, therefore projection coordinate of vehicle attitude estimated information i.e. 8 three-dimensional vertices on picture here.
Further in the present embodiment, in order to improve the true rate of prediction, what we predicted is this 8 two-dimensional coordinates (xi, yi) Relative to the relative position ((xi-xc)/w, (yi-yc)/h) of the vehicle detection frame central point (xc, yx) obtained before, w here It is the width and height of vehicle detection frame with h, the accuracy rate predicted in this way is relatively high.
It should be appreciated that the associated description of above-mentioned steps S202 and S203, only provide the processing of each network layer of training process Process, after obtaining training vehicle detection frame and vehicle attitude estimated information, the training further includes and the mark on training data The process that quasi- data are compared, and network parameter etc. is adjusted by the methods of backpropagation according to comparison result and was trained Journey, these processes use the common method of neural metwork training, and details are not described herein.
Then, in step S204, it is iterated training.
That is, step S202 and step S203 are repeated, until setting condition until meeting.The setting condition is, for example, The loss function of the loss function of the vehicle location detection branches network and vehicle attitude estimation branching networks is full respectively The threshold value that foot is respectively set.
Repetitive exercise process is as shown in figure 5, include：
Firstly, executing in step 501, using the training data of acquisition, the iteration of vehicle location detection branches network is carried out Training, for example, carry out n-th training, N be more than or equal to natural number.
Then, step 502 is executed, using the training data of acquisition, carries out the iteration instruction of vehicle attitude estimation branching networks Practice, such as n-th training.And it when carrying out the repetitive exercise of vehicle attitude estimation branching networks, is detected using vehicle location The training of branching networks output estimates branching networks with vehicle detection frame supplemental training vehicle attitude, to improve the standard of Attitude estimation True rate.As to how being estimated using the training vehicle detection frame supplemental training vehicle attitude that vehicle location detection branches network exports Branching networks are counted, then may refer to Fig. 4 and associated description.
After completing n-th training, then step S503 is then executed, the training data obtained or the instruction newly obtained are utilized Practice data, continues the repetitive exercise of vehicle location detection branches network, that is, carry out the N+1 times training.
Then, step S504 is executed, using the training data obtained or the training data newly obtained, carries out vehicle attitude Estimate the repetitive exercise of branching networks, such as the N+1 times training.And in the iteration instruction for carrying out vehicle attitude estimation branching networks When practicing, branched network is estimated using the training vehicle detection frame supplemental training vehicle attitude of vehicle location detection branches network output Network, to improve the accuracy rate of Attitude estimation.As to how being examined using the training that vehicle location detection branches network exports with vehicle It surveys frame supplemental training vehicle attitude and estimates branching networks, then may refer to Fig. 4 and associated description.
When executed the N+1 times it is trained after, then continue to execute the N+2 times repetitive exercise, training process is similar, herein no longer It repeats, and so on repetitive exercise, it is trained for vehicle location detection and vehicle to obtain until setting condition until meeting Attitude estimation puts in network.
When completing after estimating branching networks repetitive exercise with vehicle attitude of vehicle location branching networks, then follow the steps S205 detects picture using the neural network that training is completed, and is obtained by the output of vehicle location detection branches network Vehicle detection frame and confidence level estimate that the output of branching networks obtains vehicle attitude estimated information by vehicle attitude.
It illustratively, can be in setting with memory and processor according to the training method of the neural network of the present embodiment It is realized in standby, device or system.
According to the training method of the neural network of the present embodiment, one side training data, which relies only on, is labelled with vehicle location seat Mark and three-dimensional vehicle surround the picture of frame vertex projection coordinate, do not need additional finer mark, to training data mark It is required that it is low, reduce trained difficulty；On the other hand, while training vehicle location detection branches network and vehicle attitude estimate branch Network can export the two-dimensional detection frame of vehicle and the Attitude estimation information of vehicle based on training data simultaneously, not need artificial Pretreatment and subsequent processing.
Further, the training method of neural network according to the present invention, in training vehicle attitude estimation branching networks The training corresponding feature vector auxiliary prediction vehicle attitude of vehicle detection frame exported by vehicle location detection branches network Information improves the accuracy rate of vehicle attitude estimation；And the addition of vehicle attitude estimation task also improves vehicle location inspection The accuracy of survey, two kinds of tasks assist mutually, promote mutually.
Fig. 6 is the schematic block diagram according to the training device of the neural network of the embodiment of the present invention.Below with reference to Fig. 6 Training device according to an embodiment of the present invention for vehicle location detection and the neural network of Attitude estimation is retouched in detail It states.
As shown in fig. 6, the present embodiment discloses a kind of training device of neural network, the neural network includes deep layer nerve Network, region suggest that network, area-of-interest pond layer, vehicle location detection branches network and vehicle attitude estimate branched network Network, the training device 600 include data capture unit 601, the first training unit 602 and the second training unit 603.
Data capture unit 601 for obtaining training data, the training data include be labelled with vehicle location coordinate and The picture of three-dimensional vehicle encirclement frame vertex projection coordinate.Illustratively, the electronic equipment of data capture unit 601 as shown in Figure 1 In image collecting device come realize or electronic equipment as shown in Figure 1 in 102 Running storage device 104 of processor in deposit The program instruction of storage realizes, and can execute according to an embodiment of the present invention for vehicle location detection and Attitude estimation Step S201, S301 and S401 in the training method of neural network.
First training unit 602 is used to utilize the picture training for the being labelled with vehicle location coordinate vehicle location detection point Branch network.First training unit 602 can be deposited in 102 Running storage device 104 of processor in electronic equipment as shown in Figure 1 The program instruction of storage realizes, and can execute according to an embodiment of the present invention for vehicle location detection and Attitude estimation Step S202, S302~S305, S501 and S503 in the training method of neural network.
Second training unit 603 is used to utilize the picture and the vehicle for being labelled with three-dimensional vehicle encirclement frame vertex projection coordinate The training of the position detection branching networks output vehicle detection frame training vehicle attitude estimation branching networks.Second training The program instruction that unit 603 can store in 102 Running storage device 104 of processor in electronic equipment as shown in Figure 1 comes It realizes, and the training according to an embodiment of the present invention for vehicle location detection and the neural network of Attitude estimation can be executed Step S203, S402~S404, S502 and S504 in method.
It illustratively, can be in setting with memory and processor according to the training device of the neural network of the present embodiment It is realized in standby, device or system.
According to the training device of the neural network of the present embodiment, one side training data, which relies only on, is labelled with vehicle location seat Mark and three-dimensional vehicle surround the picture of frame vertex projection coordinate, do not need additional finer mark, to training data mark It is required that it is low, reduce trained difficulty；On the other hand, while training vehicle location detection branches network and vehicle attitude estimate branch Network can export the two-dimensional detection frame of vehicle and the Attitude estimation information of vehicle based on training data simultaneously, not need artificial Pretreatment and subsequent processing.
Further, the training device of neural network according to the present invention, in training vehicle attitude estimation branching networks The training corresponding feature vector auxiliary prediction vehicle attitude of vehicle detection frame exported by vehicle location detection branches network Information improves the accuracy rate of vehicle attitude estimation；And the addition of vehicle attitude estimation task also improves vehicle location inspection The accuracy of survey, two kinds of tasks assist mutually, promote mutually.
Fig. 7 is the schematic diagram for vehicle location detection and the method for Attitude estimation according to the embodiment of the present invention；Fig. 8 is Method detailed maps according to an embodiment of the present invention for vehicle location detection and Attitude estimation.
The present embodiment discloses a kind of method for vehicle location detection and Attitude estimation, as shown in fig. 7, this method packet It includes：
Step S701 obtains image to be detected.
Illustratively, by the camera or image collecting device that configure on vehicle, image to be detected is obtained, it is described to be checked Altimetric image is, for example, the image of vehicle front.
Step S702, using training in advance for the neural network of vehicle location detection and Attitude estimation to described to be checked Altimetric image is handled, to obtain vehicle detection frame and vehicle attitude estimated information.
Wherein, the neural network for vehicle location detection and Attitude estimation of the training in advance includes deep layer nerve net Network, region suggest that network, area-of-interest pond layer, vehicle location detection branches network and vehicle attitude estimate branching networks.
The detailed process for vehicle location detection and the method for Attitude estimation of the present embodiment, as shown in figure 8, including：
Step S801 obtains image to be detected.
Step S802 is handled to obtain characteristic pattern the testing image by deep-neural-network.
Step S803 suggests that network handles to obtain candidate frame the characteristic pattern by region.
Step S804 handles the candidate frame and the characteristic pattern input region of interest Chi Huacengzuochiization, with To the corresponding first eigenvector of the candidate frame.
Step S805, by position detection branching networks be based on the first eigenvector export the vehicle detection frame and Confidence level.
Step S806 handles the vehicle detection frame and the characteristic pattern input region of interest Chi Huacengzuochiization, To obtain the corresponding second feature vector of the vehicle detection frame.
Step S807 estimates that branching networks are based on the second feature vector and export vehicle attitude by the vehicle attitude Estimated information.The vehicle attitude estimated information includes that three-dimensional vehicle surrounds frame vertex projection coordinate or three-dimensional vehicle encirclement frame top The position of the point relatively described vehicle detection frame center point coordinate of projection coordinate.
Step S808 obtains vehicle location testing result and vehicle attitude estimated result.
Specifically, it according to the vehicle detection frame and confidence level of the output of vehicle location detection branches network, based on setting or uses Family selects (such as given threshold, non-maximum suppression etc.) to obtain vehicle location testing result, and estimates to believe according to vehicle attitude Breath and vehicle detection frame obtain vehicle attitude estimated result.
It should be noted that given threshold indicate for example when confidence level be less than setting value vehicle detection frame be then omitted, Non- maximum suppression indicates confidence level it is not that the vehicle detection frame of maximum value curbs.
Method according to the present invention for vehicle location detection and Attitude estimation, can be same to image to be detected of input The detection of Shi Jinhang vehicle location and vehicle attitude cathode, and due to using the use by above-mentioned training method or device training In the neural network of vehicle location detection and Attitude estimation, there is higher accuracy.
Further, according to the present invention to estimate for the method for vehicle location detection and Attitude estimation in progress vehicle attitude Timing converts vehicle attitude estimation problem to predict that three-dimensional 8 three-dimensional vertices for surrounding frame (cube) of vehicle are being schemed Projection (i.e. 8 two-dimensional coordinates) problem of on piece, and what is preferably predicted in actual task is that this 8 two-dimensional coordinates are opposite In the relative position for the vehicle detection frame central point that vehicle location detects, the accuracy rate of prediction is further improved.
Fig. 9 is the structural schematic diagram for vehicle location detection and the device of Attitude estimation according to the embodiment of the present invention； Figure 10 is the detailed construction schematic diagram of vehicle location detection and Attitude estimation module in Fig. 9.Below with reference to Fig. 9 and Figure 10 to basis The device for vehicle location detection and Attitude estimation of the embodiment of the present invention is described in detail.
As shown in figure 9, the device 900 according to an embodiment of the present invention for vehicle location detection and Attitude estimation includes figure As obtaining module 901 and vehicle location detection and Attitude estimation module 902.
Image collection module 901 is for obtaining image to be detected.Image collection module 901 can electronics as shown in Figure 1 Image collecting device in equipment is realized, and can be executed according to an embodiment of the present invention for vehicle location detection and appearance Step S701 and S801 in the method for state estimation.
Vehicle location detection and Attitude estimation module 902 are for handling described image to be detected, to obtain vehicle Position detection result and vehicle attitude estimated result.Vehicle location detection and Attitude estimation module 902 are detected including vehicle location Branching networks and vehicle attitude estimate that branching networks, the vehicle location detection branches network are used to export vehicle based on testing image Detection block and confidence level；The vehicle attitude estimation branching networks are used to be based on the testing image and the vehicle detection frame Export vehicle attitude estimated information.Vehicle location detection and Attitude estimation module 902 can be in electronic equipments as shown in Figure 1 The program instruction that stores in 102 Running storage device 104 of processor is realized, and can be executed according to an embodiment of the present invention For the step S702 and S802~S808 in the method for vehicle location detection and Attitude estimation.
Further, as shown in Figure 10, in the present embodiment, vehicle location detection and Attitude estimation module 902 include deep Layer neural network 1 000, RPN network 1001, area-of-interest pond layer 1002, vehicle location detection branches network 1003, vehicle Position detection result acquisition submodule 1004, vehicle attitude estimation branching networks 1005 and vehicle attitude estimated result obtain submodule Block 1006.
Deep-neural-network 1000 is used to obtain the testing image that module 901 obtains to described image and be handled to obtain Characteristic pattern.Deep-neural-network 1000 can be in 102 Running storage device 104 of processor in electronic equipment as shown in Figure 1 The program instruction of storage is realized, and can be executed according to an embodiment of the present invention for vehicle location detection and Attitude estimation Method step S802.
Suggest at the characteristic pattern that network (RPN network) 1001 is used to export deep-neural-network 1000 in region Reason is to obtain candidate frame.Suggest that network (RPN network) 1001 can processor 102 in electronic equipment as shown in Figure 1 in region The program instruction stored in Running storage device 104 is realized, and can be executed according to an embodiment of the present invention for vehicle The step S803 of position detection and the method for Attitude estimation.
Area-of-interest pond layer 1002 is used to do the candidate frame and the characteristic pattern pondization processing, described to obtain The corresponding first eigenvector of candidate frame, and pondization processing is done to the vehicle detection frame and the characteristic pattern, to obtain State the corresponding second feature vector of vehicle detection frame.Area-of-interest pond layer 1002 can be in electronic equipment as shown in Figure 1 102 Running storage device 104 of processor in the program instruction that stores realize, and can execute according to embodiments of the present invention For vehicle location detection and Attitude estimation method step S804 and S806.
Vehicle location detection branches network 1003 be used to export based on the first eigenvector vehicle detection frame and Confidence level.Vehicle location detection branches network 1003 can the operation storage dress of processor 102 in electronic equipment as shown in Figure 1 The program instruction that stores in 104 is set to realize, and can execute it is according to an embodiment of the present invention for vehicle location detection and The step S805 of the method for Attitude estimation.
Vehicle location testing result acquisition submodule 1004 is used to be based on user according to the vehicle detection frame and confidence level Setting or selection obtain the vehicle location testing result.Vehicle location testing result acquisition submodule 1004 can be by Fig. 1 institute The program instruction that stores in 102 Running storage device 104 of processor in the electronic equipment shown is realized, and can execute root According to the step S808 for vehicle location detection and the method for Attitude estimation of the embodiment of the present invention.
Vehicle attitude estimates that branching networks 1005 are used to export vehicle attitude estimated information based on the second feature vector. Vehicle attitude estimates that branching networks 1005 can be in 102 Running storage device 104 of processor in electronic equipment as shown in Figure 1 The program instruction of storage is realized, and can be executed according to an embodiment of the present invention for vehicle location detection and Attitude estimation Method step S807.
Vehicle attitude estimated result acquisition submodule 1006 is used for according to the vehicle location testing result and the vehicle Vehicle attitude estimated result described in Attitude estimation information acquisition.Vehicle attitude estimated result acquisition submodule 1006 can be by Fig. 1 Shown in the program instruction that stores in 102 Running storage device 104 of processor in electronic equipment realize, and can execute Step S808 according to an embodiment of the present invention for vehicle location detection and the method for Attitude estimation.
Device according to the present invention for vehicle location detection and Attitude estimation, can be same to image to be detected of input The detection of Shi Jinhang vehicle location and vehicle attitude cathode, and due to using the use by above-mentioned training method or device training In the neural network of vehicle location detection and Attitude estimation, there is higher accuracy.
Further, according to the present invention to estimate for the device of vehicle location detection and Attitude estimation in progress vehicle attitude Timing converts vehicle attitude estimation problem to predict that three-dimensional 8 three-dimensional vertices for surrounding frame (cube) of vehicle are being schemed Projection (i.e. 8 two-dimensional coordinates) problem of on piece, and what is preferably predicted in actual task is that this 8 two-dimensional coordinates are opposite In the relative position for the vehicle detection frame central point that vehicle location detects, the accuracy rate of prediction is further improved.
In addition, according to an embodiment of the invention, additionally providing a kind of nerve for vehicle location detection and Attitude estimation The training device of network, the training device include memory and processor, are stored on the memory by the processor The computer program of operation, the computer program are executed when being run by the processor and are examined as mentioned for vehicle location The corresponding steps with the training method of the neural network of Attitude estimation are surveyed, and for realizing training according to an embodiment of the present invention Data capture unit, the first training unit and the second training unit in device.
In one embodiment, the computer program executes following steps when being run by the processor：Obtain instruction Practice data, the training data includes the picture for being labelled with vehicle location coordinate and three-dimensional vehicle encirclement frame vertex projection coordinate； Following steps are repeated until meeting setting condition；Utilize the training data training vehicle location detection branches Network；The training exported using the training data and the vehicle location detection branches network is described in vehicle detection frame training Vehicle attitude estimates branching networks.
In addition, according to embodiments of the present invention, additionally providing a kind of storage medium, storing calculating on said storage Machine program, when the computer program is run by computer or processor for execute the embodiment of the present invention for vehicle position The corresponding steps of the training method of the neural network of detection and Attitude estimation are set, and for realizing according to an embodiment of the present invention For vehicle location detection and Attitude estimation neural network training device in data capture unit, the first training unit and Second training unit.The storage medium for example may include the storage card of smart phone, the storage unit of tablet computer, individual The hard disk of computer, read-only memory (ROM), Erasable Programmable Read Only Memory EPROM (EPROM), portable compact disc is read-only deposits Any combination of reservoir (CD-ROM), USB storage or above-mentioned storage medium.The computer readable storage medium can be with It is any combination of one or more computer readable storage mediums.
In one embodiment, the computer program executes following steps when being run by computer：Obtain training number According to the training data includes the picture for being labelled with vehicle location coordinate and three-dimensional vehicle encirclement frame vertex projection coordinate；It repeats Following training is executed until meeting setting condition；Utilize the training data training vehicle location detection branches net Network；The training vehicle detection frame training vehicle exported using the training data and the vehicle location detection branches network Attitude estimation branching networks.
Further, according to an embodiment of the invention, additionally providing a kind of for vehicle location detection and Attitude estimation Device, described device include memory and processor, and the computer journey run by the processor is stored on the memory Sequence, the computer program execute as mentioned when by processor operation for vehicle location detection and Attitude estimation The corresponding steps of method, and for realizing the device according to an embodiment of the present invention for vehicle location detection and Attitude estimation In image collection module and vehicle location detection and Attitude estimation module.
In one embodiment, the computer program executes following steps when being run by the processor：Obtain to Detection image；Using in advance training for vehicle location detection and Attitude estimation neural network to described image to be detected into Row processing, to obtain vehicle detection frame and vehicle attitude estimated information.
In addition, according to embodiments of the present invention, additionally providing a kind of storage medium, storing calculating on said storage Machine program, when the computer program is run by computer or processor for execute the embodiment of the present invention for vehicle position The corresponding steps of the method for detection and Attitude estimation are set, and are examined for realizing according to an embodiment of the present invention for vehicle location It surveys the image collection module in the device with Attitude estimation and vehicle location detects and Attitude estimation module.The storage medium It such as may include the storage card of smart phone, the storage unit of tablet computer, the hard disk of personal computer, read-only memory (ROM), Erasable Programmable Read Only Memory EPROM (EPROM), portable compact disc read-only memory (CD-ROM), USB storage, Or any combination of above-mentioned storage medium.The computer readable storage medium can be one or more computer-readable deposit Any combination of storage media, such as a computer readable storage medium include the calculating that judgement whether is normally opened for human eye The readable program code of machine, another computer readable storage medium include whether to be in high beam irradiating state for driver The computer-readable program code of judgement.
In one embodiment, the computer program executes following steps when being run by computer：It obtains to be detected Image；Using in advance training for vehicle location detection and Attitude estimation neural network to described image to be detected at Reason, to obtain vehicle detection frame and vehicle attitude estimated information.
Although describing example embodiment by reference to attached drawing here, it should be understood that above example embodiment are only exemplary , and be not intended to limit the scope of the invention to this.Those of ordinary skill in the art can carry out various changes wherein And modification, it is made without departing from the scope of the present invention and spiritual.All such changes and modifications are intended to be included in appended claims Within required the scope of the present invention.
Those of ordinary skill in the art may be aware that list described in conjunction with the examples disclosed in the embodiments of the present disclosure Member and algorithm steps can be realized with the combination of electronic hardware or computer software and electronic hardware.These functions are actually It is implemented in hardware or software, the specific application and design constraint depending on technical solution.Professional technician Each specific application can be used different methods to achieve the described function, but this realization is it is not considered that exceed The scope of the present invention.
In several embodiments provided herein, it should be understood that disclosed device and method can pass through it Its mode is realized.For example, apparatus embodiments described above are merely indicative, for example, the division of the unit, only Only a kind of logical function partition, there may be another division manner in actual implementation, such as multiple units or components can be tied Another equipment is closed or is desirably integrated into, or some features can be ignored or not executed.
In the instructions provided here, numerous specific details are set forth.It is to be appreciated, however, that implementation of the invention Example can be practiced without these specific details.In some instances, well known method, structure is not been shown in detail And technology, so as not to obscure the understanding of this specification.
Similarly, it should be understood that in order to simplify the present invention and help to understand one or more of the various inventive aspects, To in the description of exemplary embodiment of the present invention, each feature of the invention be grouped together into sometimes single embodiment, figure, Or in descriptions thereof.However, the method for the invention should not be construed to reflect following intention：It is i.e. claimed The present invention claims features more more than feature expressly recited in each claim.More precisely, such as corresponding power As sharp claim reflects, inventive point is that the spy of all features less than some disclosed single embodiment can be used Sign is to solve corresponding technical problem.Therefore, it then follows thus claims of specific embodiment are expressly incorporated in this specific Embodiment, wherein each, the claims themselves are regarded as separate embodiments of the invention.
It will be understood to those skilled in the art that any combination pair can be used other than mutually exclusive between feature All features disclosed in this specification (including adjoint claim, abstract and attached drawing) and so disclosed any method Or all process or units of equipment are combined.Unless expressly stated otherwise, this specification (is wanted including adjoint right Ask, make a summary and attached drawing) disclosed in each feature can be replaced with an alternative feature that provides the same, equivalent, or similar purpose.
In addition, it will be appreciated by those of skill in the art that although some embodiments described herein include other embodiments In included certain features rather than other feature, but the combination of the feature of different embodiments mean it is of the invention Within the scope of and form different embodiments.For example, in detail in the claims, embodiment claimed it is one of any Can in any combination mode come using.
Various component embodiments of the invention can be implemented in hardware, or to run on one or more processors Software module realize, or be implemented in a combination thereof.It will be understood by those of skill in the art that can be used in practice Microprocessor or digital signal processor (DSP) realize some moulds in article analytical equipment according to an embodiment of the present invention The some or all functions of block.The present invention is also implemented as a part or complete for executing method as described herein The program of device (for example, computer program and computer program product) in portion.It is such to realize that program of the invention can store On a computer-readable medium, it or may be in the form of one or more signals.Such signal can be from internet Downloading obtains on website, is perhaps provided on the carrier signal or is provided in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and ability Field technique personnel can be designed alternative embodiment without departing from the scope of the appended claims.In the claims, Any reference symbol between parentheses should not be configured to limitations on claims.Word "comprising" does not exclude the presence of not Element or step listed in the claims.Word "a" or "an" located in front of the element does not exclude the presence of multiple such Element.The present invention can be by means of including the hardware of several different elements and being come by means of properly programmed computer real It is existing.In the unit claims listing several devices, several in these devices can be through the same hardware branch To embody.The use of word first, second, and third does not indicate any sequence.These words can be explained and be run after fame Claim.
The above description is merely a specific embodiment or to the explanation of specific embodiment, protection of the invention Range is not limited thereto, and anyone skilled in the art in the technical scope disclosed by the present invention, can be easily Expect change or replacement, should be covered by the protection scope of the present invention.Protection scope of the present invention should be with claim Subject to protection scope.

Claims (15)

1. a kind of training method of neural network, which is characterized in that the neural network includes vehicle location detection branches network Branching networks are estimated with vehicle attitude, and the training method includes：
Training data is obtained, the training data includes being labelled with vehicle location coordinate and three-dimensional vehicle encirclement frame vertex projection seat Target picture；
Following steps are repeated until meeting setting condition：
The vehicle location detection branches network is trained using the training data,
The training exported using the training data and the vehicle location detection branches network is described in vehicle detection frame training Vehicle attitude estimates branching networks.
2. training method according to claim 1, which is characterized in that the neural network further include deep-neural-network, Network is suggested in region and area-of-interest pond layer, the training vehicle location detection branches network include：
The training data is handled by the deep-neural-network to obtain trained characteristic pattern；
Suggest that network handles with characteristic pattern to obtain trained candidate frame the training by the region；
The trained characteristic pattern and training candidate frame are inputted the area-of-interest Chi Huacengzuochiization and handled, with The training is obtained with the corresponding training first eigenvector of candidate frame；
Training first eigenvector is inputted into the vehicle location detection branches network, to obtain the training vehicle Detection block.
3. training method according to claim 2, which is characterized in that the training vehicle attitude estimates branching networks Including：
Training vehicle detection frame is inputted into the area-of-interest Chi Huacengzuochiization processing, is used with obtaining the training The corresponding training second feature vector of vehicle detection frame；
Training second feature vector is inputted into the vehicle attitude and estimates branching networks, to obtain vehicle attitude estimation letter Breath.
4. a kind of training device of neural network, the neural network includes that vehicle location detection branches network and vehicle attitude are estimated Count branching networks, which is characterized in that the training device includes：
Data capture unit, for obtaining training data, the training data includes being labelled with vehicle location coordinate and vehicle three Dimension surrounds the picture of frame vertex projection coordinate；
First training unit, for utilizing the training data training vehicle location detection branches network；
Second training unit, for the training vehicle using the training data and vehicle location detection branches network output The detection block training vehicle attitude estimates branching networks.
5. a kind of training device of neural network, which is characterized in that the training device includes memory and processor, described to deposit The computer program run by the processor is stored on reservoir, the computer program is held when being run by the processor The training method of neural network of the row as described in any one of claim 1-3.
6. a kind of storage medium, which is characterized in that be stored with computer program on the storage medium, the computer program exists The training method of the neural network as described in any one of claim 1-3 is executed when operation.
7. a kind of method for vehicle location detection and Attitude estimation, which is characterized in that including：
Obtain image to be detected；
Using in advance training for vehicle location detection and Attitude estimation neural network to described image to be detected at Reason, to obtain vehicle detection frame and vehicle attitude estimated information；
Wherein, the neural network for vehicle location detection and Attitude estimation of the training in advance includes vehicle location detection point Branch network and vehicle attitude estimate branching networks,
The vehicle location detection branches network is used to export vehicle detection frame based on testing image；
The vehicle attitude estimation branching networks are used to export vehicle attitude based on the testing image and the vehicle detection frame Estimated information.
8. the method according to the description of claim 7 is characterized in that the training in advance is used for vehicle location detection and posture The neural network of estimation further includes that deep-neural-network, region suggestion network and area-of-interest pond layer, the method are also wrapped It includes：
The testing image is handled to obtain characteristic pattern by the deep-neural-network；
Suggest that network handles to obtain candidate frame the characteristic pattern by the region；
The candidate frame and the characteristic pattern are inputted into the area-of-interest Chi Huacengzuochiization processing, to obtain the candidate The corresponding first eigenvector of frame.
9. according to the method described in claim 8, it is characterized in that, further including：
The first eigenvector, which is based on, by the position detection branching networks exports the vehicle detection frame and confidence level；
The vehicle detection frame is inputted into the area-of-interest Chi Huacengzuochiization processing, to obtain the vehicle detection frame pair The second feature vector answered；
Estimate that branching networks are based on the second feature vector and export the vehicle attitude estimated information by the vehicle attitude.
10. according to method described in claim 7-9 any one, which is characterized in that the vehicle attitude estimated information includes Three-dimensional vehicle surrounds frame vertex projection coordinate or three-dimensional vehicle surrounds frame vertex projection coordinate vehicle detection frame center relatively The position of point coordinate.
11. according to the method described in claim 9, it is characterized in that, further including：According to the vehicle detection frame and confidence level base Vehicle location testing result is obtained in user setting or selection.
12. according to the method for claim 11, which is characterized in that further include：According to the vehicle location testing result and The vehicle attitude estimated information obtains vehicle attitude estimated result.
13. a kind of device for vehicle location detection and Attitude estimation, which is characterized in that including：
Image collection module, for obtaining image to be detected；
Vehicle location detection and Attitude estimation module, for handling described image to be detected, to obtain vehicle detection frame With vehicle attitude estimated information；
Wherein, the vehicle location detection and Attitude estimation module include vehicle location detection branches network and vehicle attitude estimation Branching networks,
The vehicle location detection branches network is used to export vehicle detection frame based on testing image；
The vehicle attitude estimation branching networks are used to export vehicle attitude based on the testing image and the vehicle detection frame Estimated information.
14. a kind of device for vehicle location detection and Attitude estimation, which is characterized in that described device includes memory and place Device is managed, is stored with the computer program run by the processor on the memory, the computer program is by the place Manage the method for vehicle location detection and Attitude estimation executed as described in any one of claim 7-12 when device operation.
15. a kind of storage medium, which is characterized in that be stored with computer program, the computer program on the storage medium The method for vehicle location detection and Attitude estimation as described in any one of claim 7-12 is executed at runtime.
CN201711262814.2A 2017-12-04 2017-12-04 Neural network training method and device, vehicle detection estimation method and device, storage medium Pending CN108875902A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711262814.2A CN108875902A (en) 2017-12-04 2017-12-04 Neural network training method and device, vehicle detection estimation method and device, storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711262814.2A CN108875902A (en) 2017-12-04 2017-12-04 Neural network training method and device, vehicle detection estimation method and device, storage medium

Publications (1)

Publication Number Publication Date
CN108875902A true CN108875902A (en) 2018-11-23

Family

ID=64325672

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711262814.2A Pending CN108875902A (en) 2017-12-04 2017-12-04 Neural network training method and device, vehicle detection estimation method and device, storage medium

Country Status (1)

Country Link
CN (1) CN108875902A (en)

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109919245A (en) * 2019-03-18 2019-06-21 北京市商汤科技开发有限公司 Deep learning model training method and device, training equipment and storage medium
CN110696835A (en) * 2019-10-11 2020-01-17 深圳职业技术学院 Automatic early warning method and automatic early warning system for dangerous driving behaviors of vehicle
CN110939351A (en) * 2019-10-28 2020-03-31 优创嘉(大连)科技有限公司 Visual intelligent control method and visual intelligent control door
CN111274927A (en) * 2020-01-17 2020-06-12 北京三快在线科技有限公司 Training data generation method and device, electronic equipment and storage medium
CN111274926A (en) * 2020-01-17 2020-06-12 深圳佑驾创新科技有限公司 Image data screening method and device, computer equipment and storage medium
CN111383325A (en) * 2018-12-29 2020-07-07 顺丰科技有限公司 Carriage three-dimensional image generation method and device
CN111709415A (en) * 2020-04-29 2020-09-25 北京迈格威科技有限公司 Target detection method, target detection device, computer equipment and storage medium
CN111895931A (en) * 2020-07-17 2020-11-06 嘉兴泊令科技有限公司 Coal mine operation area calibration method based on computer vision
CN112949470A (en) * 2021-02-26 2021-06-11 上海商汤智能科技有限公司 Method, device and equipment for identifying lane-changing steering lamp of vehicle and storage medium
CN113435318A (en) * 2021-06-25 2021-09-24 上海商汤临港智能科技有限公司 Neural network training, image detection and driving control method and device
CN113574535A (en) * 2019-03-13 2021-10-29 标致雪铁龙汽车股份有限公司 Training neural networks to assist driving vehicles by determining hard-to-observe bounds
CN113591936A (en) * 2021-07-09 2021-11-02 厦门市美亚柏科信息股份有限公司 Vehicle attitude estimation method, terminal device and storage medium
WO2021218124A1 (en) * 2020-04-29 2021-11-04 北京百度网讯科技有限公司 Method and device for detecting vehicle
CN117930224A (en) * 2024-03-19 2024-04-26 山东科技大学 Vehicle ranging method based on monocular vision depth estimation

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5548512A (en) * 1994-10-04 1996-08-20 The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration Autonomous navigation apparatus with neural network for a mobile vehicle
CN105654067A (en) * 2016-02-02 2016-06-08 北京格灵深瞳信息技术有限公司 Vehicle detection method and device
CN106371104A (en) * 2016-08-16 2017-02-01 长春理工大学 Vehicle targets recognizing method and anti-collision device using multi-line point cloud data machine learning
CN107025642A (en) * 2016-01-27 2017-08-08 百度在线网络技术（北京）有限公司 Vehicle's contour detection method and device based on cloud data
CN107169468A (en) * 2017-05-31 2017-09-15 北京京东尚科信息技术有限公司 Method for controlling a vehicle and device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5548512A (en) * 1994-10-04 1996-08-20 The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration Autonomous navigation apparatus with neural network for a mobile vehicle
CN107025642A (en) * 2016-01-27 2017-08-08 百度在线网络技术（北京）有限公司 Vehicle's contour detection method and device based on cloud data
CN105654067A (en) * 2016-02-02 2016-06-08 北京格灵深瞳信息技术有限公司 Vehicle detection method and device
CN106371104A (en) * 2016-08-16 2017-02-01 长春理工大学 Vehicle targets recognizing method and anti-collision device using multi-line point cloud data machine learning
CN107169468A (en) * 2017-05-31 2017-09-15 北京京东尚科信息技术有限公司 Method for controlling a vehicle and device

Cited By (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111383325A (en) * 2018-12-29 2020-07-07 顺丰科技有限公司 Carriage three-dimensional image generation method and device
CN111383325B (en) * 2018-12-29 2023-06-30 深圳市丰驰顺行信息技术有限公司 Carriage three-dimensional image generation method and device
CN113574535A (en) * 2019-03-13 2021-10-29 标致雪铁龙汽车股份有限公司 Training neural networks to assist driving vehicles by determining hard-to-observe bounds
CN109919245A (en) * 2019-03-18 2019-06-21 北京市商汤科技开发有限公司 Deep learning model training method and device, training equipment and storage medium
CN110696835A (en) * 2019-10-11 2020-01-17 深圳职业技术学院 Automatic early warning method and automatic early warning system for dangerous driving behaviors of vehicle
CN110939351A (en) * 2019-10-28 2020-03-31 优创嘉(大连)科技有限公司 Visual intelligent control method and visual intelligent control door
CN111274927A (en) * 2020-01-17 2020-06-12 北京三快在线科技有限公司 Training data generation method and device, electronic equipment and storage medium
CN111274926A (en) * 2020-01-17 2020-06-12 深圳佑驾创新科技有限公司 Image data screening method and device, computer equipment and storage medium
CN111274926B (en) * 2020-01-17 2023-09-22 武汉佑驾创新科技有限公司 Image data screening method, device, computer equipment and storage medium
WO2021218124A1 (en) * 2020-04-29 2021-11-04 北京百度网讯科技有限公司 Method and device for detecting vehicle
CN111709415A (en) * 2020-04-29 2020-09-25 北京迈格威科技有限公司 Target detection method, target detection device, computer equipment and storage medium
CN111709415B (en) * 2020-04-29 2023-10-27 北京迈格威科技有限公司 Target detection method, device, computer equipment and storage medium
CN111895931B (en) * 2020-07-17 2021-11-26 嘉兴泊令科技有限公司 Coal mine operation area calibration method based on computer vision
CN111895931A (en) * 2020-07-17 2020-11-06 嘉兴泊令科技有限公司 Coal mine operation area calibration method based on computer vision
CN112949470A (en) * 2021-02-26 2021-06-11 上海商汤智能科技有限公司 Method, device and equipment for identifying lane-changing steering lamp of vehicle and storage medium
CN113435318A (en) * 2021-06-25 2021-09-24 上海商汤临港智能科技有限公司 Neural network training, image detection and driving control method and device
CN113591936A (en) * 2021-07-09 2021-11-02 厦门市美亚柏科信息股份有限公司 Vehicle attitude estimation method, terminal device and storage medium
CN113591936B (en) * 2021-07-09 2022-09-09 厦门市美亚柏科信息股份有限公司 Vehicle attitude estimation method, terminal device and storage medium
CN117930224A (en) * 2024-03-19 2024-04-26 山东科技大学 Vehicle ranging method based on monocular vision depth estimation

Similar Documents

Publication Publication Date Title
CN108875902A (en) Neural network training method and device, vehicle detection estimation method and device, storage medium
US11762475B2 (en) AR scenario-based gesture interaction method, storage medium, and communication terminal
Pizzoli et al. REMODE: Probabilistic, monocular dense reconstruction in real time
CN105164726B (en) Camera Attitude estimation for 3D reconstruct
CN104869387B (en) Method for acquiring binocular image maximum parallax based on optical flow method
CN109176512A (en) A kind of method, robot and the control device of motion sensing control robot
CN103703758B (en) mobile augmented reality system
CN110084304B (en) Target detection method based on synthetic data set
CN109003253A (en) Neural network point cloud generates system
CN108401461A (en) Three-dimensional mapping method, device and system, cloud platform, electronic equipment and computer program product
CN106797458B (en) The virtual change of real object
CN109084746A (en) Monocular mode for the autonomous platform guidance system with aiding sensors
CN111028358B (en) Indoor environment augmented reality display method and device and terminal equipment
CN108230437A (en) Scene reconstruction method and device, electronic equipment, program and medium
KR20150121179A (en) Real time stereo matching
JP2015015021A (en) Method and system for generating object three-dimensional expression
CN109214366A (en) Localized target recognition methods, apparatus and system again
CN113052109A (en) 3D target detection system and 3D target detection method thereof
CN110148217A (en) A kind of real-time three-dimensional method for reconstructing, device and equipment
CN109472828A (en) A kind of localization method, device, electronic equipment and computer readable storage medium
CN110503686A (en) Object pose estimation method and electronic equipment based on deep learning
CN109002761A (en) A kind of pedestrian's weight identification monitoring system based on depth convolutional neural networks
CN107045631A (en) Facial feature points detection method, device and equipment
CN104346816A (en) Depth determining method and device and electronic equipment
CN103310188A (en) Method and apparatus for pose recognition

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20181123