CN110398957A - Automatic Pilot behavior prediction method, apparatus, computer equipment and storage medium - Google Patents
Automatic Pilot behavior prediction method, apparatus, computer equipment and storage medium Download PDFInfo
- Publication number
- CN110398957A CN110398957A CN201910527673.5A CN201910527673A CN110398957A CN 110398957 A CN110398957 A CN 110398957A CN 201910527673 A CN201910527673 A CN 201910527673A CN 110398957 A CN110398957 A CN 110398957A
- Authority
- CN
- China
- Prior art keywords
- convolution
- vector
- obtains
- excitation
- characteristic feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000003860 storage Methods 0.000 title claims abstract description 20
- 230000006399 behavior Effects 0.000 claims abstract description 51
- 230000000306 recurrent effect Effects 0.000 claims abstract description 45
- 239000000203 mixture Substances 0.000 claims abstract description 41
- 238000003062 neural network model Methods 0.000 claims abstract description 41
- 230000006835 compression Effects 0.000 claims abstract description 36
- 238000007906 compression Methods 0.000 claims abstract description 36
- 230000033001 locomotion Effects 0.000 claims abstract description 33
- 230000005284 excitation Effects 0.000 claims description 92
- 239000011159 matrix material Substances 0.000 claims description 54
- 230000006870 function Effects 0.000 claims description 27
- 238000004590 computer program Methods 0.000 claims description 14
- 230000015654 memory Effects 0.000 claims description 13
- 238000012512 characterization method Methods 0.000 claims description 7
- 230000005540 biological transmission Effects 0.000 claims description 5
- 235000013399 edible fruits Nutrition 0.000 claims 1
- 238000013528 artificial neural network Methods 0.000 abstract description 16
- 230000013016 learning Effects 0.000 abstract description 9
- 230000008447 perception Effects 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 17
- 239000000047 product Substances 0.000 description 9
- 230000008569 process Effects 0.000 description 8
- 238000012545 processing Methods 0.000 description 7
- 230000009471 action Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 210000005036 nerve Anatomy 0.000 description 5
- 210000004218 nerve net Anatomy 0.000 description 5
- 238000012549 training Methods 0.000 description 5
- 230000001133 acceleration Effects 0.000 description 4
- 230000003542 behavioural effect Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 238000009826 distribution Methods 0.000 description 4
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 239000006227 byproduct Substances 0.000 description 2
- 239000000470 constituent Substances 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 238000005520 cutting process Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000007787 long-term memory Effects 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 210000004205 output neuron Anatomy 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000000151 deposition Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/0088—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
- G05D1/0221—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving a learning process
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Remote Sensing (AREA)
- Aviation & Aerospace Engineering (AREA)
- Software Systems (AREA)
- Automation & Control Theory (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Radar, Positioning & Navigation (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Medical Informatics (AREA)
- Game Theory and Decision Science (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses automatic Pilot behavior prediction method, apparatus, computer equipment and storage mediums.This method comprises: receiving 2D picture frame of the automatic Pilot end currently in video sequence collected, using the 2D picture frame as the input of variation self-encoding encoder, obtains compression corresponding with the 2D picture frame and be abstracted characteristic feature vector；Abstract characteristic feature vector is compressed as mixture density network-Recognition with Recurrent Neural Network model input trained in advance using described, obtains predicted vector；The abstract characteristic feature vector of compression and the predicted vector are input to controller, generation obtains movement vector；And the movement vector is sent to automatic Pilot end.The method achieve view-based access control model perception to realize to following prediction by mixing different neural network learnings, increase the accuracy of decision.
Description
Technical field
The present invention relates to intelligent Decision Technology field more particularly to a kind of automatic Pilot behavior prediction method, apparatus, calculate
Machine equipment and storage medium.
Background technique
Unmanned Systems are one and integrate the comprehensive of the functions such as environment sensing, programmed decision-making, multi-grade auxiliary driving
Collaboration system, it, which is concentrated, has used the technologies such as computer, modern sensing, information fusion, communication, artificial intelligence and automatic control, is
Typical new and high technology synthesis.And the key technology of automatic Pilot can successively be divided into environment sensing, behaviour decision making, path rule
It draws and motion control four is most of.
Currently, the machine learning system being often used in Unmanned Systems is all based on supervised learning foundation, but this is needed
Largely there is label training sample, and also lacks the ability of common sense and independent prediction.In automatic Pilot, extraneous complex environment
It is usually detached from trained sample, thus the ability for making model lose decision.
Summary of the invention
The embodiment of the invention provides a kind of automatic Pilot behavior prediction method, apparatus, computer equipment and storage medium,
It aims to solve the problem that the machine learning system being often used in Unmanned Systems in the prior art is all based on supervised learning foundation, needs
Largely there is label training sample, and extraneous complex environment is usually detached from trained sample in automatic Pilot, to make model
Lose decision ability and independent prediction ability the problem of.
In a first aspect, the embodiment of the invention provides a kind of automatic Pilot behavior prediction methods comprising:
2D picture frame of the automatic Pilot end currently in video sequence collected is received, using the 2D picture frame as variation
The input of self-encoding encoder obtains compress corresponding with the 2D picture frame and is abstracted characteristic feature vector；
Abstract characteristic feature vector is compressed as mixture density network-Recognition with Recurrent Neural Network model trained in advance using described
Input, obtain predicted vector；Wherein, Recognition with Recurrent Neural Network model in the mixture density network-Recognition with Recurrent Neural Network model
Output be to compress the corresponding probability density function of abstract characteristic feature vector with described；
Compress abstract characteristic feature vector and the predicted vector is input to controller for described, generation obtain acting to
Amount；Wherein, the controller is linear model；And
The movement vector is sent to automatic Pilot end.
Second aspect, the embodiment of the invention provides a kind of automatic Pilot behavior prediction devices comprising:
Image receiving unit, for receiving 2D picture frame of the automatic Pilot end currently in video sequence collected, by institute
State input of the 2D picture frame as variation self-encoding encoder, obtain it is corresponding with the 2D picture frame compress abstract characteristic feature to
Amount；
Predicted vector acquiring unit, for compressing abstract characteristic feature vector as hybrid density trained in advance for described
The input of network-Recognition with Recurrent Neural Network model, obtains predicted vector；Wherein, the mixture density network-Recognition with Recurrent Neural Network mould
The output of Recognition with Recurrent Neural Network model is probability density function corresponding with the abstract characteristic feature vector of the compression in type；
Acquiring unit is acted, for the abstract characteristic feature vector of compression and the predicted vector to be input to control
Device, generation obtain movement vector；Wherein, the controller is linear model；And
Vector transmission unit, for the movement vector to be sent to automatic Pilot end.
The third aspect, the embodiment of the present invention provide a kind of computer equipment again comprising memory, processor and storage
On the memory and the computer program that can run on the processor, the processor execute the computer program
Automatic Pilot behavior prediction method described in the above-mentioned first aspect of Shi Shixian.
Fourth aspect, the embodiment of the invention also provides a kind of computer readable storage mediums, wherein the computer can
It reads storage medium and is stored with computer program, it is above-mentioned that the computer program when being executed by a processor executes the processor
Automatic Pilot behavior prediction method described in first aspect.
The embodiment of the invention provides a kind of automatic Pilot behavior prediction method, apparatus, computer equipment and storage mediums.
This method includes the 2D picture frame for receiving automatic Pilot end currently in video sequence collected, using the 2D picture frame as change
The input for dividing self-encoding encoder obtains compress corresponding with the 2D picture frame and is abstracted characteristic feature vector；The compression is abstract
Characteristic feature vector obtains predicted vector as mixture density network-Recognition with Recurrent Neural Network model input trained in advance；It will
The abstract characteristic feature vector of compression and the predicted vector are input to controller, and generation obtains movement vector；And it will
The movement vector is sent to automatic Pilot end.The method achieve view-based access control model perception, by mixing different neural networks
Study is realized to following prediction, increases the accuracy of decision.
Detailed description of the invention
Technical solution in order to illustrate the embodiments of the present invention more clearly, below will be to needed in embodiment description
Attached drawing is briefly described, it should be apparent that, drawings in the following description are some embodiments of the invention, general for this field
For logical technical staff, without creative efforts, it is also possible to obtain other drawings based on these drawings.
Fig. 1 is the flow diagram of automatic Pilot behavior prediction method provided in an embodiment of the present invention；
Fig. 2 is the sub-process schematic diagram of automatic Pilot behavior prediction method provided in an embodiment of the present invention；
Fig. 3 is self-editing for picture element matrix is input to variation in automatic Pilot behavior prediction method provided in an embodiment of the present invention
Code device carries out the structural schematic diagram of neural network used by repeatedly excitation convolution sum excitation deconvolution；
Fig. 4 is another sub-process schematic diagram of automatic Pilot behavior prediction method provided in an embodiment of the present invention；
Fig. 5 is the schematic diagram of data flow in automatic Pilot behavior prediction method provided in an embodiment of the present invention；
Fig. 6 is mixture density network-circulation nerve net in automatic Pilot behavior prediction method provided in an embodiment of the present invention
The schematic diagram of network model；
Fig. 7 is the schematic block diagram of automatic Pilot behavior prediction device provided in an embodiment of the present invention；
Fig. 8 is the subelement schematic block diagram of automatic Pilot behavior prediction device provided in an embodiment of the present invention；
Fig. 9 is another subelement schematic block diagram of automatic Pilot behavior prediction device provided in an embodiment of the present invention；
Figure 10 is the schematic block diagram of computer equipment provided in an embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are some of the embodiments of the present invention, instead of all the embodiments.Based on this hair
Embodiment in bright, every other implementation obtained by those of ordinary skill in the art without making creative efforts
Example, shall fall within the protection scope of the present invention.
It should be appreciated that ought use in this specification and in the appended claims, term " includes " and "comprising" instruction
Described feature, entirety, step, operation, the presence of element and/or component, but one or more of the other feature, whole is not precluded
Body, step, operation, the presence or addition of element, component and/or its set.
It is also understood that mesh of the term used in this description of the invention merely for the sake of description specific embodiment
And be not intended to limit the present invention.As description of the invention and it is used in the attached claims, unless on
Other situations are hereafter clearly indicated, otherwise " one " of singular, "one" and "the" are intended to include plural form.
It will be further appreciated that the term "and/or" used in description of the invention and the appended claims is
Refer to any combination and all possible combinations of one or more of associated item listed, and including these combinations.
Referring to Fig. 1, Fig. 1 is the flow diagram of automatic Pilot behavior prediction method provided in an embodiment of the present invention, it should
Automatic Pilot behavior prediction method is applied to can be in unpiloted intelligent automobile, and this method can be unpiloted by being installed on
Application software in intelligent automobile is executed.
As shown in Fig. 2, the method comprising the steps of S110~S140.
S110,2D picture frame of the automatic Pilot end currently in video sequence collected is received, the 2D picture frame is made
For the input of variation self-encoding encoder, obtains compress corresponding with the 2D picture frame and be abstracted characteristic feature vector.
In the present embodiment, if be set to can the camera of unpiloted intelligent automobile (i.e. automatic Pilot end) acquire
Video randomly selects one or more after can cutting to video, obtains 2D picture frame, the 2D picture frame is input to variation
Self-encoding encoder (variation self-encoding encoder is abbreviated as VAE) after being handled by variation self-encoding encoder, can be obtained and scheme with the 2D
Abstract characteristic feature vector is compressed as frame is corresponding.Wherein, the coding/decoding process in variation self-encoding encoder be a convolution/
For the process namely variation self-encoding encoder of deconvolution neural network as vision processing module, task is that study has each been observed
The abstract compression of input frame characterizes, then on each time frame compact model finding (picture frame).
By VAE model, it will be observed that input picture concentration be Gaussian distributed 32 dimension latent variable (z), this
Mean smaller environment attribute, accelerates learning process.The effect of this step is in driving procedure, for example by the environment of surrounding
The straightness of road, upcoming bend and you be concentrated relative to the position of road, to determine next behavior.
In one embodiment, as shown in Fig. 2, step S110 includes:
S111, picture element matrix corresponding with the 2D picture frame is obtained, the picture element matrix is input to variation from coding
Device carries out repeatedly excitation convolution, obtains coding result；
S112, the coding result is connected by the dense layer of variation self-encoding encoder entirely, obtains classification results；
S113, repeatedly excitation deconvolution is carried out to the classification results, obtains compression corresponding with the 2D picture frame and takes out
As characteristic feature vector；Wherein, the picture element matrix is input to the excitation volume that variation self-encoding encoder carries out repeatedly excitation convolution
Product number is identical as the repeatedly excitation excitation number of deconvolution of deconvolution is carried out to the classification results.
In the present embodiment, when obtain the corresponding picture element matrix of the 2D picture frame (the usually image of 64*64*3,
Indicate 3 channel images of 64*64), picture element matrix need to be input to variation self-encoding encoder and carry out repeatedly motivating convolution sum excitation anti-
Convolution, to obtain compressing abstract characteristic feature vector.
As shown in figure 3, it is that picture element matrix is input to variation self-encoding encoder to carry out repeatedly excitation convolution sum excitation warp
The structural schematic diagram of neural network used by product.After 3 excitation convolution sums, 3 excitation deconvolution, it can be realized institute
It states 2D picture frame and carries out abstract compression characterization, so that obtaining compress corresponding with the 2D picture frame is abstracted characteristic feature vector.
Wherein,
In one embodiment, it is carried out as shown in figure 3, the picture element matrix is input to variation self-encoding encoder in step S111
Repeatedly excitation convolution, obtains coding result, comprising:
Obtain the picture element matrix of 64*64*3 corresponding with the 2D picture frame；
Excitation convolution for the first time is carried out by the picture element matrix that the first convolution of 32*4 checks 64*64*3, obtains 31*31*
32 the first convolution results；
Second of excitation convolution is carried out by the first convolution results that the second convolution of 64*4 checks 31*31*32, is obtained
The second convolution results of 14*14*64；
Third time excitation convolution is carried out to the second convolution results of 14*14*64 by the third convolution kernel of 128*4, is obtained
The third convolution results of 6*6*128 are using as coding result.
In the present embodiment, the picture element matrix of 64*64*3 corresponding to the 2D picture frame carries out 3 excitation convolution realizations
After coding, the important feature in picture element matrix is obtained, but also produces the pixel of many blank simultaneously.In order to subsequent to volume
Code result is restored, and can be restored using the excitation deconvolution with excitation convolution same number to coding result, not only
Restore the quality for the image for being exaggerated coding result, and ensuring to a certain extent.
In one embodiment, it as shown in figure 3, carrying out repeatedly excitation deconvolution to the classification results in step S113, obtains
Abstract characteristic feature vector is compressed to corresponding with the 2D picture frame, comprising:
Obtain the convolution results of 5*5*128 corresponding with classification results；
Swash for the first time by the convolution results that the Volume Four product of 64*5 checks the corresponding 5*5*128 of the classification results
Deconvolution is encouraged, the first deconvolution result of 13*13*64 is obtained；
Second of excitation deconvolution is carried out to the first deconvolution result of 13*13*64 by the 5th convolution kernel of 32*6, is obtained
To the second deconvolution result of 30*30*32；
Third time excitation deconvolution is carried out to the second deconvolution result of 30*30*32 by the 6th convolution kernel of 3*6, is obtained
To 64*64*3 third deconvolution as a result, to compress abstract characteristic feature vector as corresponding with the 2D picture frame.
In the present embodiment, the third convolution results of 6*6*128 as coding result be input to dense layer (namely convolution mind
Through the full articulamentum in network) after connected entirely, classification results corresponding with the 2D picture frame can be obtained.In order to complete
After constituent class, the classification results are reduced into picture element matrix, can be used at this time sharp with excitation convolution same number
It encourages deconvolution to restore coding result, realizes the reconstruction to image.
S120, abstract characteristic feature vector is compressed as mixture density network-circulation nerve net trained in advance using described
The input of network model, obtains predicted vector；Wherein, nerve net is recycled in the mixture density network-Recognition with Recurrent Neural Network model
The output of network model is probability density function corresponding with the abstract characteristic feature vector of the compression.
In the present embodiment, when realize compress each time frame observation (i.e. obtain it is corresponding with the 2D picture frame
Compress abstract characteristic feature vector), also to compress other informations of all variations occurred with the time, when specific implementation can adopt
Future is predicted with mixture density network-Recognition with Recurrent Neural Network (i.e. MDN-RNN), and MDN-RNN model can serve as variation from coding
The prediction model of the expected following z vector generated of device.Due to many complex environments in nature be it is random, RNN is to export one
A probability density function p (z) rather than a deterministic forecast z.
In one embodiment, as shown in Figure 4-Figure 6, step S120 includes:
S121, abstract characteristic feature vector is compressed as mixture density network-circulation nerve net trained in advance using described
The input of Recognition with Recurrent Neural Network model in network model obtains probability density letter corresponding with the abstract characteristic feature vector of the compression
Number；
S122, using the probability density function and control parameter as mixture density network trained in advance-circulation nerve
The input of mixture density network model, is calculated predicted vector in network model.
In the present embodiment, when mixture density network-Recognition with Recurrent Neural Network model trained in advance, rate distribution P need to be modeled
(z(t+1)|at,zt,ht), wherein atFor the action (i.e. movement vector) taken in t moment, and htIt is Recognition with Recurrent Neural Network model
Hiding state in t moment, τ are for the probabilistic parameter of Controlling model.Mixture density network-Recognition with Recurrent Neural Network model
Specifically being exactly one has the LSTM (i.e. shot and long term memory network) of 256 hidden units similar with VAE, circulation nerve
Network model attempts the potential understanding of vehicle's current condition in capturing ambient, but is to the potential understanding of vehicle's current condition this time
Will by before z (compress abstract characteristic feature vector) and behavior based on, predict which type of next z may be, more
The newly hidden state of oneself.
S130, the abstract characteristic feature vector of compression and the predicted vector are input to controller, generation obtains
Act vector；Wherein, the controller is linear model.
In the present embodiment, controller is then the task for action selection.Briefly, controller is exactly one intensive
The neural network of connection, the input of this network are cascade z (sneak condition-length obtained from VAE is 32) and h (RNN
Hidden state-length be 256).These three output neurons correspond to three behaviors, and are scaled to suitable range.Then
This behavior is sent in environment, this can return to the observation of a update, then start subsequent cycle.
In one embodiment, step S130 includes:
Obtain the linear model a in controllert=Wc[ztht]+bc；Wherein, atTo act vector, ztTo compress abstract table
Levy feature vector, htFor predicted vector, WcFor weight matrix, bcFor bias vector；
It is obtained according to the linear model in controller and compresses abstract characteristic feature vector and the predicted vector pair with described
The movement vector answered.
In the present embodiment, if given current state zt, can produce zt+1Probability distribution, then from zt+1Middle sampling is simultaneously
Observed value as real world.In each time step (timestep, it is understood that be time frame), can all it be fed
One observation (the environment color image namely 2D picture frame of the road and vehicle that are received by visual sensor), it is also necessary to
Return to a series of behavioral parameters next taken --- direction (- 1 to 1), acceleration (0 to 1) and the brake namely turned to
Then this behavior is transmitted in environment by vehicle (0 to 1), return to next observation, then starts to recycle next time, thus in the past
Real-time learning is carried out in the sequence time and space, predicts the behavior of next frame, has better adaptability for environment.
S140, the movement vector is sent to automatic Pilot end.
In the present embodiment, after obtaining current action vector, movement vector is sent to automatic Pilot end, to control
It makes unmanned.Include at least that the following are behavioral parameters: the direction (- 1 to 1) that namely turns to, acceleration (0 in movement vector
To 1) and brake (0 to 1).
The method achieve view-based access control model perception, by the different neural network learning of mixing, realize to following prediction,
Increase the accuracy of decision.
The embodiment of the present invention also provides a kind of automatic Pilot behavior prediction device, which is used for
Execute any embodiment of aforementioned automatic Pilot behavior prediction method.Specifically, referring to Fig. 7, Fig. 7 is that the embodiment of the present invention mentions
The schematic block diagram of the automatic Pilot behavior prediction device of confession.The automatic Pilot behavior prediction device 100 can be configured at can nothing
In the intelligent automobile that people drives.
As shown in fig. 7, automatic Pilot behavior prediction device 100 includes image receiving unit 110, predicted vector acquiring unit
120, acquiring unit 130 and vector transmission unit 140 are acted.
Image receiving unit 110 will for receiving 2D picture frame of the automatic Pilot end currently in video sequence collected
Input of the 2D picture frame as variation self-encoding encoder, obtain it is corresponding with the 2D picture frame compress abstract characteristic feature to
Amount.
In the present embodiment, if be set to can the camera of unpiloted intelligent automobile (i.e. automatic Pilot end) acquire
Video randomly selects one or more after can cutting to video, obtains 2D picture frame, the 2D picture frame is input to variation
Self-encoding encoder (variation self-encoding encoder is abbreviated as VAE) after being handled by variation self-encoding encoder, can be obtained and scheme with the 2D
Abstract characteristic feature vector is compressed as frame is corresponding.Wherein, the coding/decoding process in variation self-encoding encoder be a convolution/
For the process namely variation self-encoding encoder of deconvolution neural network as vision processing module, task is that study has each been observed
The abstract compression of input frame characterizes, then on each time frame compact model finding (picture frame).
By VAE model, it will be observed that input picture concentration be Gaussian distributed 32 dimension latent variable (z), this
Mean smaller environment attribute, accelerates learning process.The effect of this step is in driving procedure, for example by the environment of surrounding
The straightness of road, upcoming bend and you be concentrated relative to the position of road, to determine next behavior.
In one embodiment, as shown in figure 8, image receiving unit 110 includes:
The picture element matrix is input to by coding unit 111 for obtaining picture element matrix corresponding with the 2D picture frame
Variation self-encoding encoder carries out repeatedly excitation convolution, obtains coding result；
Full connection unit 112 is obtained for being connected entirely by the dense layer of variation self-encoding encoder to the coding result
To classification results；
Decoding unit 113 obtains and the 2D picture frame pair for carrying out repeatedly excitation deconvolution to the classification results
The compression answered is abstracted characteristic feature vector；Wherein, the picture element matrix is input to variation self-encoding encoder and carries out repeatedly excitation volume
Long-pending excitation convolution number is identical as the repeatedly excitation excitation number of deconvolution of deconvolution is carried out to the classification results.
In the present embodiment, when obtain the corresponding picture element matrix of the 2D picture frame (the usually image of 64*64*3,
Indicate 3 channel images of 64*64), picture element matrix need to be input to variation self-encoding encoder and carry out repeatedly motivating convolution sum excitation anti-
Convolution, to obtain compressing abstract characteristic feature vector.
As shown in figure 3, it is that picture element matrix is input to variation self-encoding encoder to carry out repeatedly excitation convolution sum excitation warp
The structural schematic diagram of neural network used by product.After 3 excitation convolution sums, 3 excitation deconvolution, it can be realized institute
It states 2D picture frame and carries out abstract compression characterization, so that obtaining compress corresponding with the 2D picture frame is abstracted characteristic feature vector.
In one embodiment, coding unit 111 includes:
Picture element matrix acquiring unit, for obtaining the picture element matrix of 64*64*3 corresponding with the 2D picture frame；
First excitation convolution unit, the picture element matrix for checking 64*64*3 for the first convolution by 32*4 carry out first
Secondary excitation convolution obtains the first convolution results of 31*31*32；
Second excitation convolution unit, the first convolution results for checking 31*31*32 for the second convolution by 64*4 carry out
Second of excitation convolution, obtains the second convolution results of 14*14*64；
Third motivates convolution unit, for by the third convolution kernel of 128*4 to the second convolution results of 14*14*64 into
Row third time excitation convolution obtains the third convolution results of 6*6*128 using as coding result.
In the present embodiment, the picture element matrix of 64*64*3 corresponding to the 2D picture frame carries out 3 excitation convolution realizations
After coding, the important feature in picture element matrix is obtained, but also produces the pixel of many blank simultaneously.In order to subsequent to volume
Code result is restored, and can be restored using the excitation deconvolution with excitation convolution same number to coding result, not only
Restore the quality for the image for being exaggerated coding result, and ensuring to a certain extent.
In one embodiment, decoding unit 113 includes:
Convolution results acquiring unit, for obtaining the convolution results of 5*5*128 corresponding with classification results；
First excitation warp product unit checks the corresponding 5*5* of the classification results for the Volume Four product by 64*5
128 convolution results carry out excitation deconvolution for the first time, obtain the first deconvolution result of 13*13*64；
Second excitation warp product unit, for the 5th convolution kernel by 32*6 to the first deconvolution result of 13*13*64
Second of excitation deconvolution is carried out, the second deconvolution result of 30*30*32 is obtained；
Third motivates warp product unit, for the 6th convolution kernel by 3*6 to the second deconvolution result of 30*30*32
Third time excitation deconvolution is carried out, obtains the third deconvolution of 64*64*3 as a result, using as pressure corresponding with the 2D picture frame
Contract abstract characteristic feature vector.
In the present embodiment, the third convolution results of 6*6*128 as coding result be input to dense layer (namely convolution mind
Through the full articulamentum in network) after connected entirely, classification results corresponding with the 2D picture frame can be obtained.In order to complete
After constituent class, the classification results are reduced into picture element matrix, can be used at this time sharp with excitation convolution same number
It encourages deconvolution to restore coding result, realizes the reconstruction to image.
Predicted vector acquiring unit 120, for compressing abstract characteristic feature vector as mixing trained in advance for described
The input of density network-Recognition with Recurrent Neural Network model, obtains predicted vector；Wherein, the mixture density network-circulation nerve net
The output of Recognition with Recurrent Neural Network model is probability density function corresponding with the abstract characteristic feature vector of the compression in network model.
In the present embodiment, when realize compress each time frame observation (i.e. obtain it is corresponding with the 2D picture frame
Compress abstract characteristic feature vector), also to compress other informations of all variations occurred with the time, when specific implementation can adopt
Future is predicted with mixture density network-Recognition with Recurrent Neural Network (i.e. MDN-RNN), and MDN-RNN model can serve as variation from coding
The prediction model of the expected following z vector generated of device.Due to many complex environments in nature be it is random, RNN is to export one
A probability density function p (z) rather than a deterministic forecast z.
In one embodiment, as shown in figure 9, predicted vector acquiring unit 120 includes:
First nerves network processing unit 121, for compressing abstract characteristic feature vector as training in advance for described
The input of Recognition with Recurrent Neural Network model in mixture density network-Recognition with Recurrent Neural Network model obtains and the abstract characterization of the compression
The corresponding probability density function of feature vector；
Nervus opticus network processing unit 122, for using the probability density function and control parameter as preparatory training
Mixture density network-Recognition with Recurrent Neural Network model in mixture density network model input, predicted vector is calculated.
In the present embodiment, when mixture density network-Recognition with Recurrent Neural Network model trained in advance, rate distribution P need to be modeled
(z(t+1)|at,zt,ht), wherein atFor the action (i.e. movement vector) taken in t moment, and htIt is Recognition with Recurrent Neural Network model
Hiding state in t moment, τ are for the probabilistic parameter of Controlling model.Mixture density network-Recognition with Recurrent Neural Network model
Specifically being exactly one has the LSTM (i.e. shot and long term memory network) of 256 hidden units similar with VAE, circulation nerve
Network model attempts the potential understanding of vehicle's current condition in capturing ambient, but is to the potential understanding of vehicle's current condition this time
Will by before z (compress abstract characteristic feature vector) and behavior based on, predict which type of next z may be, more
The newly hidden state of oneself.
Acquiring unit 130 is acted, for the abstract characteristic feature vector of compression and the predicted vector to be input to
Controller, generation obtain movement vector；Wherein, the controller is linear model.
In the present embodiment, controller is then the task for action selection.Briefly, controller is exactly one intensive
The neural network of connection, the input of this network are cascade z (sneak condition-length obtained from VAE is 32) and h (RNN
Hidden state-length be 256).These three output neurons correspond to three behaviors, and are scaled to suitable range.Then
This behavior is sent in environment, this can return to the observation of a update, then start subsequent cycle.
In one embodiment, movement acquiring unit 130 includes:
Linear model acquiring unit, for obtaining the linear model a in controllert=Wc[ztht]+bc；Wherein, atIt is
Make vector, ztTo compress abstract characteristic feature vector, htFor predicted vector, WcFor weight matrix, bcFor bias vector；
Vector acquiring unit is acted, compresses abstract characteristic feature with described for obtaining according to the linear model in controller
Vector and the corresponding movement vector of the predicted vector.
In the present embodiment, if given current state zt, can produce zt+1Probability distribution, then from zt+1Middle sampling is simultaneously
Observed value as real world.In each time step (timestep, it is understood that be time frame), can all it be fed
One observation (the environment color image namely 2D picture frame of the road and vehicle that are received by visual sensor), it is also necessary to
Return to a series of behavioral parameters next taken --- direction (- 1 to 1), acceleration (0 to 1) and the brake namely turned to
Then this behavior is transmitted in environment by vehicle (0 to 1), return to next observation, then starts to recycle next time, thus in the past
Real-time learning is carried out in the sequence time and space, predicts the behavior of next frame, has better adaptability for environment.
Vector transmission unit 140, for the movement vector to be sent to automatic Pilot end.
In the present embodiment, after obtaining current action vector, movement vector is sent to automatic Pilot end, to control
It makes unmanned.Include at least that the following are behavioral parameters: the direction (- 1 to 1) that namely turns to, acceleration (0 in movement vector
To 1) and brake (0 to 1).
The arrangement achieves view-based access control model perception, by the different neural network learning of mixing, realize to following prediction,
Increase the accuracy of decision.
Above-mentioned automatic Pilot behavior prediction device can be implemented as the form of computer program, which can be
It is run in computer equipment as shown in Figure 10.
Referring to Fig. 10, Figure 10 is the schematic block diagram of computer equipment provided in an embodiment of the present invention.The computer is set
Standby 500 be can unpiloted intelligent automobile vehicle intelligent terminal.
Refering to fig. 10, which includes processor 502, memory and the net connected by system bus 501
Network interface 505, wherein memory may include non-volatile memory medium 503 and built-in storage 504.
The non-volatile memory medium 503 can storage program area 5031 and computer program 5032.The computer program
5032 are performed, and processor 502 may make to execute automatic Pilot behavior prediction method.
The processor 502 supports the operation of entire computer equipment 500 for providing calculating and control ability.
The built-in storage 504 provides environment for the operation of the computer program 5032 in non-volatile memory medium 503, should
When computer program 5032 is executed by processor 502, processor 502 may make to execute automatic Pilot behavior prediction method.
The network interface 505 is for carrying out network communication, such as the transmission of offer data information.Those skilled in the art can
To understand, structure shown in Figure 10, only the block diagram of part-structure relevant to the present invention program, is not constituted to this hair
The restriction for the computer equipment 500 that bright scheme is applied thereon, specific computer equipment 500 may include than as shown in the figure
More or fewer components perhaps combine certain components or with different component layouts.
Wherein, the processor 502 is for running computer program 5032 stored in memory, to realize following function
Can: 2D picture frame of the automatic Pilot end currently in video sequence collected is received, the 2D picture frame is self-editing as variation
The input of code device obtains compress corresponding with the 2D picture frame and is abstracted characteristic feature vector；The abstract characterization of the compression is special
Vector is levied as mixture density network-Recognition with Recurrent Neural Network model input trained in advance, obtains predicted vector；Wherein, institute
It is special with the abstract characterization of the compression for stating the output of Recognition with Recurrent Neural Network model in mixture density network-Recognition with Recurrent Neural Network model
Levy the corresponding probability density function of vector；The abstract characteristic feature vector of compression and the predicted vector are input to control
Device, generation obtain movement vector；Wherein, the controller is linear model；And the movement vector is sent to and is driven automatically
Sail end.
In one embodiment, processor 502 is executing described lead to using the 2D picture frame as the defeated of variation self-encoding encoder
Enter, when obtaining the step for compressing abstract characteristic feature vector corresponding with the 2D picture frame, performs the following operations: acquisition and institute
The corresponding picture element matrix of 2D picture frame is stated, the picture element matrix is input to variation self-encoding encoder and carries out repeatedly excitation convolution, is obtained
To coding result；The coding result is connected entirely by the dense layer of variation self-encoding encoder, obtains classification results；To institute
It states classification results and carries out repeatedly excitation deconvolution, obtain compress corresponding with the 2D picture frame and be abstracted characteristic feature vector；Its
In, the picture element matrix is input to variation self-encoding encoder and carries out the repeatedly excitation convolution number of excitation convolution and to the classification
As a result the number for carrying out the excitation deconvolution of repeatedly excitation deconvolution is identical.
In one embodiment, processor 502 is executing acquisition picture element matrix corresponding with the 2D picture frame, by institute
It states picture element matrix and is input to variation self-encoding encoder progress repeatedly excitation convolution, when obtaining the step of coding result, execute following grasp
Make: obtaining the picture element matrix of 64*64*3 corresponding with the 2D picture frame；Check 64*64*3's by the first convolution of 32*4
Picture element matrix carries out excitation convolution for the first time, obtains the first convolution results of 31*31*32；It is checked by the second convolution of 64*4
The first convolution results of 31*31*32 carry out second of excitation convolution, obtain the second convolution results of 14*14*64；Pass through 128*4
Third convolution kernel third time excitation convolution is carried out to the second convolution results of 14*14*64, obtain the third convolution of 6*6*128
As a result using as coding result.
In one embodiment, processor 502 is described to classification results progress repeatedly excitation deconvolution in execution, obtains
It when the step for compressing abstract characteristic feature vector corresponding with the 2D picture frame, performs the following operations: acquisition and classification results
The convolution results of corresponding 5*5*128；The convolution of the corresponding 5*5*128 of the classification results is checked by the Volume Four product of 64*5
As a result excitation deconvolution for the first time is carried out, the first deconvolution result of 13*13*64 is obtained；Pass through the 5th convolution kernel pair of 32*6
The first deconvolution result of 13*13*64 carries out second of excitation deconvolution, obtains the second deconvolution result of 30*30*32；It is logical
The 6th convolution kernel for crossing 3*6 carries out third time excitation deconvolution to the second deconvolution result of 30*30*32, obtains 64*64*3's
Third deconvolution is as a result, to compress abstract characteristic feature vector as corresponding with the 2D picture frame.
In one embodiment, processor 502 is described using the abstract characteristic feature vector of the compression as preparatory instruction in execution
The input of experienced mixture density network-Recognition with Recurrent Neural Network model performs the following operations when obtaining the step of predicted vector: will
It is described to compress abstract characteristic feature vector as circulation nerve in mixture density network-Recognition with Recurrent Neural Network model trained in advance
The input of network model obtains probability density function corresponding with the abstract characteristic feature vector of the compression；The probability is close
Function and control parameter are spent as mixture density network model in mixture density network-Recognition with Recurrent Neural Network model trained in advance
Input, predicted vector is calculated.
In one embodiment, processor 502 is described by the abstract characteristic feature vector of compression and the prediction in execution
Vector is input to controller, when generating the step for obtaining acting vector, performs the following operations: obtaining the linear mould in controller
Type at=Wc[ztht]+bc；Wherein, atTo act vector, ztTo compress abstract characteristic feature vector, htFor predicted vector, WcFor power
Weight matrix, bcFor bias vector；It is obtained according to the linear model in controller and compresses abstract characteristic feature vector and institute with described
State the corresponding movement vector of predicted vector.
It will be understood by those skilled in the art that the embodiment of computer equipment shown in Figure 10 is not constituted to computer
The restriction of equipment specific composition, in other embodiments, computer equipment may include components more more or fewer than diagram, or
Person combines certain components or different component layouts.For example, in some embodiments, computer equipment can only include depositing
Reservoir and processor, in such embodiments, the structure and function of memory and processor are consistent with embodiment illustrated in fig. 10,
Details are not described herein.
It should be appreciated that in embodiments of the present invention, processor 502 can be central processing unit (Central
Processing Unit, CPU), which can also be other general processors, digital signal processor (Digital
Signal Processor, DSP), specific integrated circuit (Application Specific Integrated Circuit,
ASIC), ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic
Device, discrete gate or transistor logic, discrete hardware components etc..Wherein, general processor can be microprocessor or
Person's processor is also possible to any conventional processor etc..
Computer readable storage medium is provided in another embodiment of the invention.The computer readable storage medium can be with
For non-volatile computer readable storage medium.The computer-readable recording medium storage has computer program, wherein calculating
Machine program performs the steps of the 2D image for receiving automatic Pilot end currently in video sequence collected when being executed by processor
Frame obtains the abstract characterization of compression corresponding with the 2D picture frame using the 2D picture frame as the input of variation self-encoding encoder
Feature vector；Abstract characteristic feature vector is compressed as mixture density network-Recognition with Recurrent Neural Network mould trained in advance using described
The input of type, obtains predicted vector；Wherein, Recognition with Recurrent Neural Network mould in the mixture density network-Recognition with Recurrent Neural Network model
The output of type is probability density function corresponding with the abstract characteristic feature vector of the compression；Abstract characteristic feature is compressed by described
Vector and the predicted vector are input to controller, and generation obtains movement vector；Wherein, the controller is linear model；
And the movement vector is sent to automatic Pilot end.
In one embodiment, described logical using the 2D picture frame as the input of variation self-encoding encoder, it obtains and the 2D
Picture frame is corresponding to compress abstract characteristic feature vector, comprising: picture element matrix corresponding with the 2D picture frame is obtained, it will be described
Picture element matrix is input to variation self-encoding encoder and carries out repeatedly excitation convolution, obtains coding result；Pass through the thick of variation self-encoding encoder
Close layer connects the coding result entirely, obtains classification results；Repeatedly excitation deconvolution is carried out to the classification results, is obtained
Abstract characteristic feature vector is compressed to corresponding with the 2D picture frame；Wherein, that the picture element matrix is input to variation is self-editing
The excitation that code device carries out the multiple excitation convolution number for motivating convolution and carries out repeatedly excitation deconvolution to the classification results is anti-
The number of convolution is identical.
In one embodiment, described to obtain picture element matrix corresponding with the 2D picture frame, the picture element matrix is inputted
Repeatedly excitation convolution is carried out to variation self-encoding encoder, obtains coding result, comprising: obtain 64* corresponding with the 2D picture frame
The picture element matrix of 64*3；Excitation convolution for the first time is carried out by the picture element matrix that the first convolution of 32*4 checks 64*64*3, is obtained
The first convolution results of 31*31*32；It is carried out second by the first convolution results that the second convolution of 64*4 checks 31*31*32
Convolution is motivated, the second convolution results of 14*14*64 are obtained；By the third convolution kernel of 128*4 to the second convolution of 14*14*64
As a result third time excitation convolution is carried out, obtains the third convolution results of 6*6*128 using as coding result.
In one embodiment, described that repeatedly excitation deconvolution is carried out to the classification results, it obtains and the 2D picture frame
It is corresponding to compress abstract characteristic feature vector, comprising: to obtain the convolution results of 5*5*128 corresponding with classification results；Pass through 64*
The convolution results that 5 Volume Four product checks the corresponding 5*5*128 of the classification results carry out excitation deconvolution for the first time, obtain
The first deconvolution result of 13*13*64；The is carried out by the first deconvolution result of the 5th convolution kernel of 32*6 to 13*13*64
Secondary excitation deconvolution obtains the second deconvolution result of 30*30*32；By the 6th convolution kernel of 3*6 to the of 30*30*32
Two deconvolution results carry out third time excitation deconvolution, obtain the third deconvolution of 64*64*3 as a result, to scheme as with the 2D
Abstract characteristic feature vector is compressed as frame is corresponding.
In one embodiment, described to compress abstract characteristic feature vector as hybrid density net trained in advance for described
The input of network-Recognition with Recurrent Neural Network model, obtains predicted vector, comprising: compresses abstract characteristic feature vector as pre- for described
The input of Recognition with Recurrent Neural Network model, obtains and the compression in the mixture density network-Recognition with Recurrent Neural Network model first trained
The abstract corresponding probability density function of characteristic feature vector；Using the probability density function and control parameter as training in advance
The input of mixture density network model, is calculated predicted vector in mixture density network-Recognition with Recurrent Neural Network model.
In one embodiment, described that the abstract characteristic feature vector of compression and the predicted vector are input to control
Device, generation obtain movement vector, comprising: obtain the linear model a in controllert=Wc[ztht]+bc；Wherein, atFor movement to
Amount, ztTo compress abstract characteristic feature vector, htFor predicted vector, WcFor weight matrix, bcFor bias vector；According to controller
In linear model obtain and be abstracted characteristic feature vector and the corresponding movement vector of the predicted vector with the compression.
It is apparent to those skilled in the art that for convenience of description and succinctly, foregoing description is set
The specific work process of standby, device and unit, can refer to corresponding processes in the foregoing method embodiment, and details are not described herein.
Those of ordinary skill in the art may be aware that unit described in conjunction with the examples disclosed in the embodiments of the present disclosure and algorithm
Step can be realized with electronic hardware, computer software, or a combination of the two, in order to clearly demonstrate hardware and software
Interchangeability generally describes each exemplary composition and step according to function in the above description.These functions are studied carefully
Unexpectedly the specific application and design constraint depending on technical solution are implemented in hardware or software.Professional technician
Each specific application can be used different methods to achieve the described function, but this realization is it is not considered that exceed
The scope of the present invention.
In several embodiments provided by the present invention, it should be understood that disclosed unit and method, it can be with
It realizes by another way.For example, the apparatus embodiments described above are merely exemplary, for example, the unit
It divides, only logical function partition, there may be another division manner in actual implementation, can also will be with the same function
Unit set is at a unit, such as multiple units or components can be combined or can be integrated into another system or some
Feature can be ignored, or not execute.In addition, shown or discussed mutual coupling, direct-coupling or communication connection can
Be through some interfaces, the indirect coupling or communication connection of device or unit, be also possible to electricity, mechanical or other shapes
Formula connection.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
In network unit.Some or all of unit therein can be selected to realize the embodiment of the present invention according to the actual needs
Purpose.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit
It is that each unit physically exists alone, is also possible to two or more units and is integrated in one unit.It is above-mentioned integrated
Unit both can take the form of hardware realization, can also realize in the form of software functional units.
If the integrated unit is realized in the form of SFU software functional unit and sells or use as independent product
When, it can store in one storage medium.Based on this understanding, technical solution of the present invention is substantially in other words to existing
The all or part of part or the technical solution that technology contributes can be embodied in the form of software products, should
Computer software product is stored in a storage medium, including some instructions are used so that a computer equipment (can be
Personal computer, server or network equipment etc.) execute all or part of step of each embodiment the method for the present invention
Suddenly.And storage medium above-mentioned include: USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), magnetic disk or
The various media that can store program code such as person's CD.
The above description is merely a specific embodiment, but scope of protection of the present invention is not limited thereto, any
Those familiar with the art in the technical scope disclosed by the present invention, can readily occur in various equivalent modifications or replace
It changes, these modifications or substitutions should be covered by the protection scope of the present invention.Therefore, protection scope of the present invention should be with right
It is required that protection scope subject to.
Claims (10)
1. a kind of automatic Pilot behavior prediction method characterized by comprising
2D picture frame of the automatic Pilot end currently in video sequence collected is received, the 2D picture frame is self-editing as variation
The input of code device obtains compress corresponding with the 2D picture frame and is abstracted characteristic feature vector；
The abstract characteristic feature vector of the compression is defeated as mixture density network-Recognition with Recurrent Neural Network model trained in advance
Enter, obtains predicted vector；Wherein, in the mixture density network-Recognition with Recurrent Neural Network model Recognition with Recurrent Neural Network model it is defeated
It is out probability density function corresponding with the abstract characteristic feature vector of the compression；
The abstract characteristic feature vector of compression and the predicted vector are input to controller, generation obtains movement vector；
Wherein, the controller is linear model；And
The movement vector is sent to automatic Pilot end.
2. automatic Pilot behavior prediction method according to claim 1, which is characterized in that described logical by the 2D picture frame
As the input of variation self-encoding encoder, obtains compress corresponding with the 2D picture frame and is abstracted characteristic feature vector, comprising:
Picture element matrix corresponding with the 2D picture frame is obtained, the picture element matrix is input to variation self-encoding encoder and is carried out repeatedly
Convolution is motivated, coding result is obtained；
The coding result is connected entirely by the dense layer of variation self-encoding encoder, obtains classification results；
Repeatedly excitation deconvolution is carried out to the classification results, compress corresponding with the 2D picture frame is obtained and is abstracted characteristic feature
Vector；Wherein, by the picture element matrix be input to variation self-encoding encoder carry out repeatedly excitation convolution excitation convolution number with it is right
The number that the classification results carry out the excitation deconvolution of repeatedly excitation deconvolution is identical.
3. automatic Pilot behavior prediction method according to claim 2, which is characterized in that the acquisition and the 2D image
The picture element matrix is input to variation self-encoding encoder and carries out repeatedly excitation convolution, obtains coding knot by the corresponding picture element matrix of frame
Fruit, comprising:
Obtain the picture element matrix of 64*64*3 corresponding with the 2D picture frame；
Excitation convolution for the first time is carried out by the picture element matrix that the first convolution of 32*4 checks 64*64*3, obtains 31*31*32's
First convolution results；
Second of excitation convolution is carried out by the first convolution results that the second convolution of 64*4 checks 31*31*32, obtains 14*14*
64 the second convolution results；
Third time excitation convolution is carried out to the second convolution results of 14*14*64 by the third convolution kernel of 128*4, obtains 6*6*
128 third convolution results are using as coding result.
4. automatic Pilot behavior prediction method according to claim 2, which is characterized in that it is described to the classification results into
Row repeatedly excitation deconvolution obtains compress corresponding with the 2D picture frame and is abstracted characteristic feature vector, comprising:
Obtain the convolution results of 5*5*128 corresponding with classification results；
Motivate for the first time by the convolution results that the Volume Four product of 64*5 checks the corresponding 5*5*128 of the classification results anti-
Convolution obtains the first deconvolution result of 13*13*64；
Second of excitation deconvolution is carried out to the first deconvolution result of 13*13*64 by the 5th convolution kernel of 32*6, is obtained
The second deconvolution result of 30*30*32；
Third time excitation deconvolution is carried out to the second deconvolution result of 30*30*32 by the 6th convolution kernel of 3*6, obtains 64*
The third deconvolution of 64*3 is as a result, to compress abstract characteristic feature vector as corresponding with the 2D picture frame.
5. automatic Pilot behavior prediction method according to claim 1, which is characterized in that described to compress abstract table for described
Feature vector is levied as mixture density network-Recognition with Recurrent Neural Network model input trained in advance, obtains predicted vector, is wrapped
It includes:
Abstract characteristic feature vector is compressed as following in the mixture density network-Recognition with Recurrent Neural Network model trained in advance using described
The input of ring neural network model obtains probability density function corresponding with the abstract characteristic feature vector of the compression；
Using the probability density function and control parameter as in mixture density network-Recognition with Recurrent Neural Network model trained in advance
The input of mixture density network model, is calculated predicted vector.
6. automatic Pilot behavior prediction method according to claim 1, which is characterized in that described to compress abstract table for described
Sign feature vector and the predicted vector are input to controller, and generation obtains movement vector, comprising:
Obtain the linear model a in controllert=Wc[zt ht]+bc；Wherein, atTo act vector, ztIt is special to compress abstract characterization
Levy vector, htFor predicted vector, WcFor weight matrix, bcFor bias vector；
It is obtained according to the linear model in controller corresponding with the abstract characteristic feature vector of compression and the predicted vector
Act vector.
7. a kind of automatic Pilot behavior prediction device characterized by comprising
Image receiving unit, for receiving 2D picture frame of the automatic Pilot end currently in video sequence collected, by the 2D
Input of the picture frame as variation self-encoding encoder obtains compress corresponding with the 2D picture frame and is abstracted characteristic feature vector；
Predicted vector acquiring unit, for compressing abstract characteristic feature vector as hybrid density net trained in advance for described
The input of network-Recognition with Recurrent Neural Network model, obtains predicted vector；Wherein, the mixture density network-Recognition with Recurrent Neural Network model
The output of middle Recognition with Recurrent Neural Network model is probability density function corresponding with the abstract characteristic feature vector of the compression；
Acquiring unit is acted, for the abstract characteristic feature vector of compression and the predicted vector to be input to controller,
Generation obtains movement vector；Wherein, the controller is linear model；And
Vector transmission unit, for the movement vector to be sent to automatic Pilot end.
8. automatic Pilot behavior prediction device according to claim 7, which is characterized in that described image receiving unit, packet
It includes:
The picture element matrix is input to variation certainly for obtaining picture element matrix corresponding with the 2D picture frame by coding unit
Encoder carries out repeatedly excitation convolution, obtains coding result；
Full connection unit is classified for being connected entirely by the dense layer of variation self-encoding encoder to the coding result
As a result；
Decoding unit obtains pressure corresponding with the 2D picture frame for carrying out repeatedly excitation deconvolution to the classification results
Contract abstract characteristic feature vector；Wherein, the picture element matrix is input to variation self-encoding encoder and carries out repeatedly swashing for excitation convolution
It is identical as the repeatedly excitation excitation number of deconvolution of deconvolution is carried out to the classification results to encourage convolution number.
9. a kind of computer equipment, including memory, processor and it is stored on the memory and can be on the processor
The computer program of operation, which is characterized in that the processor realizes such as claim 1 to 6 when executing the computer program
Any one of described in automatic Pilot behavior prediction method.
10. a kind of computer readable storage medium, which is characterized in that the computer-readable recording medium storage has computer journey
Sequence, the computer program make when being executed by a processor the processor execute as it is as claimed in any one of claims 1 to 6 from
Dynamic driving behavior prediction technique.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910527673.5A CN110398957A (en) | 2019-06-18 | 2019-06-18 | Automatic Pilot behavior prediction method, apparatus, computer equipment and storage medium |
| PCT/CN2019/103467 WO2020252926A1 (en) | 2019-06-18 | 2019-08-30 | Method and device for prediction of automatic driving behaviors, computer device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910527673.5A CN110398957A (en) | 2019-06-18 | 2019-06-18 | Automatic Pilot behavior prediction method, apparatus, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110398957A true CN110398957A (en) | 2019-11-01 |
Family
ID=68323246
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910527673.5A Pending CN110398957A (en) | 2019-06-18 | 2019-06-18 | Automatic Pilot behavior prediction method, apparatus, computer equipment and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN110398957A (en) |
| WO (1) | WO2020252926A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111914482A (en) * | 2020-07-27 | 2020-11-10 | 武汉中海庭数据技术有限公司 | Driving condition generation method and system for automatic driving test |
| CN111988622A (en) * | 2020-08-20 | 2020-11-24 | 深圳市商汤科技有限公司 | Video prediction method and device, electronic equipment and storage medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107590438A (en) * | 2017-08-16 | 2018-01-16 | 中国地质大学（武汉） | A kind of intelligent auxiliary driving method and system |
-
2019
- 2019-06-18 CN CN201910527673.5A patent/CN110398957A/en active Pending
- 2019-08-30 WO PCT/CN2019/103467 patent/WO2020252926A1/en active Application Filing
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107590438A (en) * | 2017-08-16 | 2018-01-16 | 中国地质大学（武汉） | A kind of intelligent auxiliary driving method and system |
Non-Patent Citations (2)
| Title |
|---|
| DAVID HA等: "World Models", 《ARXIV》 * |
| EDER SANTANA等: "Learning a Driving Simulator", 《ARXIV》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111914482A (en) * | 2020-07-27 | 2020-11-10 | 武汉中海庭数据技术有限公司 | Driving condition generation method and system for automatic driving test |
| CN111988622A (en) * | 2020-08-20 | 2020-11-24 | 深圳市商汤科技有限公司 | Video prediction method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2020252926A1 (en) | 2020-12-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109978893B (en) | Training method, device, equipment and storage medium of image semantic segmentation network | |
| US20210158023A1 (en) | System and Method for Generating Image Landmarks | |
| EP3992846A1 (en) | Action recognition method and apparatus, computer storage medium, and computer device | |
| US11922569B2 (en) | Generating realistic point clouds | |
| EP4390881A1 (en) | Image generation method and related device | |
| CN111476192B (en) | Intercepted image synthesis method based on intelligent traffic and big data cloud server | |
| CN112862023B (en) | Object density determination method and device, computer equipment and storage medium | |
| CN110472516A (en) | A kind of construction method, device, equipment and the system of character image identifying system | |
| CN115064020B (en) | Intelligent teaching method, system and storage medium based on digital twin technology | |
| CN115131849A (en) | Image generation method and related device | |
| CN110287848A (en) | The generation method and device of video | |
| US20220366244A1 (en) | Modeling Human Behavior in Work Environment Using Neural Networks | |
| CN110398957A (en) | Automatic Pilot behavior prediction method, apparatus, computer equipment and storage medium | |
| CN114463837A (en) | Human behavior recognition method and system based on self-adaptive space-time convolution network | |
| CN114565891A (en) | Smoke and fire monitoring method and system based on graph generation technology | |
| CN113011320A (en) | Video processing method and device, electronic equipment and storage medium | |
| CN110766131A (en) | Data processing device and method and electronic equipment | |
| CN112269943B (en) | Information recommendation system and method | |
| CN115565146A (en) | Perception model training method and system for acquiring aerial view characteristics based on self-encoder | |
| CN114462491A (en) | Behavior analysis model training method, behavior analysis method and equipment thereof | |
| CN111539420B (en) | Panoramic image saliency prediction method and system based on attention perception features | |
| CN110333517A (en) | Method for barrier perception, device and storage medium | |
| CN113762047A (en) | Method and device for predicting remaining usable time of image acquisition device and electronic equipment | |
| CN115457482A (en) | Image recognition method, device and system and computer equipment | |
| Lange et al. | Self-supervised Multi-future Occupancy Forecasting for Autonomous Driving |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191101 |