CN114399743A - Method for generating future track of obstacle - Google Patents
Method for generating future track of obstacle Download PDFInfo
- Publication number
- CN114399743A CN114399743A CN202111506962.0A CN202111506962A CN114399743A CN 114399743 A CN114399743 A CN 114399743A CN 202111506962 A CN202111506962 A CN 202111506962A CN 114399743 A CN114399743 A CN 114399743A
- Authority
- CN
- China
- Prior art keywords
- track
- future
- predicted
- historical
- tracks
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000005070 sampling Methods 0.000 claims abstract description 31
- 230000002452 interceptive effect Effects 0.000 claims abstract description 18
- 238000012549 training Methods 0.000 claims description 24
- 239000013598 vector Substances 0.000 claims description 13
- 230000014509 gene expression Effects 0.000 claims description 10
- 235000004522 Pentaglottis sempervirens Nutrition 0.000 claims description 7
- 230000003993 interaction Effects 0.000 claims description 7
- 240000004050 Pentaglottis sempervirens Species 0.000 claims description 5
- 238000010606 normalization Methods 0.000 claims description 4
- 230000000452 restraining effect Effects 0.000 claims description 2
- 239000000126 substance Substances 0.000 claims description 2
- 208000020442 loss of weight Diseases 0.000 claims 1
- 230000006399 behavior Effects 0.000 abstract description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000007613 environmental effect Effects 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 2
- 230000036962 time dependent Effects 0.000 description 2
- 238000012356 Product development Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Probability & Statistics with Applications (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention discloses a method for generating a future track of an obstacle. In order to overcome the problem of serious track prediction distortion caused by neglecting the interference of other traffic participants, the invention provides a multi-track generation method by combining CVAE and GAN, the coding characteristics of historical tracks are combined with the interactive characteristics of surrounding traffic participants, future tracks are generated after decoding of a generation model, the generated tracks are judged to be true and false by a judgment model, meanwhile, the generated future tracks are restrained by combining the information of a travelable area, and the generation of unreasonable tracks is avoided. The method for generating the multiple tracks by combining the multiple tracks with the interactive characteristics of surrounding traffic participants is obtained through multiple random sampling, the influence of other traffic participants is considered, the network automatically learns the multi-intention behaviors of the predicted target in different environments, the uncertainty of the future tracks is fully expressed, and meanwhile, the future tracks of the predicted target can be predicted under a complex scene by the constraint of a travelable area on the future tracks.
Description
Technical Field
The invention relates to the field of automatic control of intelligent automobiles, in particular to a method for generating future trajectories of obstacles.
Background
The intelligent driving automobile senses the surrounding environment and obstacle information through a plurality of sensors, and the sensed result is sent to a motion planning system to plan the motion of the intelligent driving automobile, so that the automatic control of the automobile is realized. The behavior prediction is that the future movement track of the surrounding traffic participants is predicted, the track of the surrounding traffic participants can be accurately predicted, collision can be effectively avoided, the intelligent driving automobile can be ensured to run safely in a complex environment, and the stability and the comfort degree of planning control can be improved.
In the existing research, the main points are as follows: the method based on the physical model and the data driving predicts the future track according to the historical track based on the kinematics principle, but the method only considers the motion information of the predicted target and can only predict for a short time, and the prediction for a long time is inaccurate; a proper machine learning or deep learning model is designed based on a data-driven method, namely a machine learning or deep learning method, and prediction is carried out in a data training mode. The method based on data driving can be used for carrying out more accurate long-term prediction by combining target historical information, environmental information and surrounding target interaction information. However, since predicting the trajectory of the surrounding traffic participants is not a deterministic task, and is influenced by the environment, the intention of the driver, and the driving habits, how to accurately predict the future trajectory of the surrounding traffic participants to improve the driving safety and comfort of the intelligent driving vehicle is a current problem and difficulty to be solved urgently.
For example, a proposed vehicle intention and trajectory prediction method, which is disclosed in publication No. CN112347567A, simplifies the prediction of human driving behavior into which region the vehicle is predicted to be inserted, determines the region in which the vehicle is finally inserted from the information of the vehicle itself and neighbors, identifies the vehicle intention based on a hidden markov model, and predicts the trajectory in combination with a vehicle dynamics model.
A vehicle track prediction model with a semantic map and an LSTM with publication number CN113128326A provides a track prediction method combining the semantic map, wherein the LSTM model is adopted to extract dynamic characteristics of a target obstacle, and the CNN model is adopted to learn semantic map characteristics of a driving environment in a part of an image map. And obtaining track prediction information through MLP by combining the dynamic characteristics of the target obstacle and the semantic map characteristics of the driving environment. And applying a neural network to carry out iterative prediction on the positions of the moving obstacles at the continuous time points of the prediction interval so as to complete the prediction of all the time points.
A trajectory prediction method and apparatus for an obstacle, disclosed in publication No. CN112364997B, predict a probability that an intention of a target obstacle belongs to the intention type and a predicted trajectory of the target obstacle under the intention type, based on environmental information around the target obstacle, historical trajectory data of the target obstacle, and predetermined reference trajectories. The method comprises the steps that a predetermined reference track divides a real track into a plurality of environment types according to the real tracks of a plurality of obstacles acquired in advance and environmental information around each real track, each real track under the environment is clustered aiming at each environment type, the clustered tracks are used as each reference track under the environment type, and each reference track corresponds to an intention type.
Disclosure of Invention
The invention mainly solves the problem of serious track prediction distortion caused by neglecting the interference of other traffic participants in the prior art; a method for generating a future trajectory of an obstacle is provided.
The technical problem of the invention is mainly solved by the following technical scheme:
the method comprises a training stage and a prediction stage, wherein the training stage simultaneously runs a track generation model and a track judgment model, the track generation model generates a series of predicted future tracks with constraints, and the track judgment model is responsible for judging the truth of the generated predicted future tracks; the training stage only runs during development, and when the prediction stage is applied in an actual scene, namely the product is trained during product development, the prediction accuracy is improved, and the trajectory prediction can be performed after the accuracy meets the requirement.
The training phase performs the following steps:
a1, acquiring historical tracks and real future tracks of a predicted target and other obstacles around the predicted target, coding the historical tracks and the real future tracks to obtain coding features of the historical tracks and the real future tracks, and acquiring the coding features of the historical tracks subjected to distributed sampling; extracting interactive features of obstacles around the predicted target, and generating a predicted future trajectory of the predicted target; the obstacles may be all types of surrounding traffic participants, including pedestrians, motor vehicles, non-motor vehicles; in the training phase, the time that the prediction occurs is adopted, namely the historical track of the predicted target and other obstacles and the future track at that time are confirmed, the confirmed future track is called as a real future track, and the predicted future track is the future track generated by the generation model and comes in and goes out of the real future track; after the interactive features of the surrounding obstacles are extracted, the influence of the surrounding obstacles is used as one of indexes for predicting future tracks, and the prediction precision is improved.
A2, acquiring a flying view of a travelable area of a scene where a predicted target is located, and constraining the predicted future track of the predicted target; in an actual traffic scene, some places such as a one-way road, a bus lane, a red light intersection and the like cannot be driven, and the predicted future track can be in accordance with the actual situation only by restricting the predicted track by using the conditions.
A3, alternately updating the weights of the discriminant model and the generated model;
the prediction phase performs the following steps:
b1: coding the historical tracks of the predicted target and other obstacles around the predicted target to obtain historical track coding features of the predicted target and other obstacles around the predicted target, and obtaining interaction features of the obstacles around the predicted target according to the historical track coding features of the other obstacles around the predicted target; in the prediction stage, the track prediction is carried out in a real application scene, so that no real future track exists; acquiring the historical track coding features subjected to distributed sampling, and acquiring the predicted future track of the predicted target according to the historical track coding features subjected to distributed sampling;
b2: projecting the predicted future trajectory of the predicted target into the travelable area aerial view to constrain the predicted future trajectory of the predicted target;
b3: step B1 is performed until enough predicted future trajectories for the plurality of predicted targets are generated, which refers to K predicted future trajectories.
Preferably, if there are n obstacles in the scene and the ith obstacle is taken as a prediction target, the coordinate expression of the history track of the prediction target at the t-th time is as follows:
wherein tau is tau times before the t time as a reference,
the coordinate of the target t moment in a world coordinate system is predicted; the real future trajectory of the predicted target at the time t is:
wherein tau 'is tau' moments after the t moment serving as a reference; hist history trackiAnd the true Future trajectory FutureiThe target trajectory is a series of trajectories formed by coordinate points of a predicted target in a world coordinate system at a plurality of moments, and these coordinate points are referred to as trajectory points.
Preferably, the training phase uses the historical track HistiObtaining the encoding characteristics of the historical track and the Future track Future after encodingiObtaining future track coding characteristics after coding; after the historical track coding features and the future track coding features are superposed, the mean value mu and the variance sigma are obtained through two full-connected layers2For learning scoreCloth Z (. mu.,. sigma.)2)～P(Z|Histi，Futurei) Wherein Z is modeled as a Gaussian distribution and Z-N (0, I); calculating Z (. mu.,. sigma.)2) The distance penalty from the gaussian distribution N (0, I) is: l1(i) ═ KLD (Z (μ, σ)2) N (0, I)); for Z (mu, sigma)2) Carrying out random sampling to obtain a group of hidden variables, obtaining feature vectors with the same dimensionality as the historical track coding features by the hidden variables through a multilayer perceptron, and then multiplying the feature vectors with the historical track coding features through normalization to obtain the historical track coding features subjected to distributed sampling; the training phase adopts the distribution Z (mu, sigma) of KL divergence metric learning2) Distance loss from gaussian distribution N (0, I).
Preferably, the method for generating future obstacle trajectories according to claim 2 is characterized in that the historical trajectories of all obstacles around the predicted target are encoded to obtain encoding features of surrounding obstacles, and the encoding features of the surrounding obstacles are averaged to obtain the interaction features of the obstacles around the predicted target; the method comprises the steps of superposing historical track coding features subjected to distributed sampling and interactive features of obstacles around a predicted target to obtain final coding features, decoding the final coding features of the predicted future track to obtain an expression of the predicted future track of the ith predicted target
Wherein the Prediction isiTo predict future trajectories.
Preferably, the method for generating the future trajectory of the obstacle according to claim 3, wherein the distribution Z (μ, σ) is generated2) Carrying out random sampling for K times to generate K groups of predicted future tracks of predicted targets, wherein the expression of the predicted future track of the K group of predicted targets is
Wherein (K ∈ K); in the training stage, K groups are used for predicting the future track Prediction of the ith obstacleikAnd said real Future trajectory FutureiCalculating to obtain the mean square error lossComprises the following steps:
preferably, a bird's eye view of a travelable area of a scene where the automatic driving vehicle is located is obtained; projecting points in the predicted future track into a driving-capable area aerial view, and constraining track points of the predicted future track according to the driving-capable area; the aerial view of the travelable area can be obtained by data identification obtained by a vision or laser sensor, unreasonable tracks in the predicted future tracks are removed through the constraint of the travelable area, and the predicted future tracks are optimized.
Preferably, the track points in the predicted future track are sequentially projected to a bird's-eye view, and the shortest distance from the track points to the travelable area is calculated according to the travelable area bird's-eye view
Wherein the content of the first and second substances,
corresponding to a certain future moment, if the track point is projected in the travelable area, then
Computing constraint loss
Wherein, alpha, beta and gamma are hyper-parameter constants,
for calculating distance-dependent constraint values, beta and gamma are used for adjusting the constraint strength, the greater the distance is, the more deviation from the travelable region is shown, the greater the penalty is, namely the loss is,
for the purpose of a time-dependent adjustment factor,
the penalty is larger and larger with smaller influence, and the loss is larger, so that the track points predicted closer to the current moment are expected to be more accurate.
Preferably, the updating of the weight of the discriminant model includes: splicing the historical track and the predicted future track into a complete track, sending the complete track into a discrimination model, and obtaining a probability value pred of track truth and falseness by a multilayer perceptron after the complete track is codedikAnd a label gtikValue computation cross entropy loss, corresponding label gtikIs false; splicing the historical track and the real future track, sending the spliced historical track and the real future track into a discrimination model again, coding the complete track, and obtaining a probability value pred of whether the track is true or false through a multilayer perceptroni_realAnd a label gti_realValue computation cross entropy loss, corresponding label gti_realIs true; the discrimination loss L4(i) is calculated as
Total Loss of discriminant modelD_iL4 (i); the updating of the weights of the generative model comprises: splicing the historical track and the generated track into a complete track, sending the complete track into a discrimination model, and obtaining a probability value pred of whether the track is true or false through a multilayer perceptron after the complete track is codedikAnd a label gtikValue computation cross entropy loss, at which point the corresponding label gtikIs true; the discriminant loss at this time is expressed as:
the total generative model loss requires a weighted sum of the generative model and the discriminant model losses: lossG_i＝α1L1(i)+α2L2(i)+α3L3(i)+α4L4(i), wherein α1，α2，α3，α4Is the corresponding loss weight.
Preferably, the prediction stage encodes the historical trajectories of the predicted target and the obstacles around the predicted target to obtain the encoding characteristics of the historical trajectories of the predicted target and the obstacles around the predicted target, and averages the encoding characteristics to obtain the predictionMeasuring the interactive characteristics of surrounding obstacles of the target; according to the learned distribution Z (mu, sigma)2) Obtaining a characteristic vector by using a hidden variable obtained after random sampling, and obtaining a historical track coding characteristic after the characteristic vector is normalized and multiplied by the historical track coding characteristic to be subjected to distributed sampling; the final coded feature is obtained by overlapping the historical track coding feature of distributed sampling and the interactive feature of the obstacle around the predicted target, and the coordinate expression of the predicted future track is obtained after decoding
The predicted future trajectory PredictioniProjecting the point in the space to the aerial view of the travelable area, restraining the predicted future track, and removing the track beyond the travelable area; again according to the distribution Z (μ, σ)2) Carrying out random sampling, and repeating the process until K predicted future tracks are generated, wherein the K predicted future tracks are used as predicted target future possible driving tracks; the distribution of the prediction phase learning is the same as the step of the training phase learning distribution.
The invention has the beneficial effects that:
1. the generation model fully considers the historical track characteristics of the predicted target and the interactive characteristics of surrounding vehicles, and the prediction precision is improved;
2. the network can obtain a more real track through the learning of the discrimination model;
3. by carrying out travelable area constraint on the generated track, filtering unreasonable tracks and obtaining more reasonable and effective tracks;
4. the method is suitable for all types of surrounding traffic participants, including pedestrians, motor vehicles and non-motor vehicles;
5. in the track generation stage, the influence of other traffic participants is considered, multi-intention behaviors of the predicted target in different environments are automatically learned through a network, and the uncertainty of future tracks is fully expressed.
Drawings
FIG. 1 is a schematic flow chart of a training phase of a method for generating a future trajectory of an obstacle according to the present invention;
fig. 2 is a schematic flow chart of a prediction stage of a method for generating a future obstacle trajectory according to the present invention.
Detailed Description
The technical scheme of the invention is further specifically described by the following embodiments and the accompanying drawings.
Example (b):
the method for generating the future trajectory of the obstacle of the embodiment is mainly applied to the automatic driving automobile as shown in fig. 1 and fig. 2, and combines a multi-trajectory generation method of CVAE and GAN, and comprises a training phase and a prediction phase, wherein the training phase is operated only when a product is developed; after the product is really put into use, only a prediction stage is implemented to predict the future track; in the training stage, a track generation model and a track judgment model are simultaneously operated, the track generation model learns Gaussian distribution of track coding characteristics by taking the coding characteristics of historical tracks and the coding characteristics of future tracks as conditions, a series of tracks with constraints are generated after the coding characteristics of the historical tracks and the interactive characteristics of surrounding traffic vehicles are combined according to Gaussian distribution sampling, and the judgment model is responsible for judging the truth and falseness of the generated tracks; and in the prediction stage, only the generating model is operated, the prediction future track is generated after the Gaussian distribution learned in the training stage is sampled for multiple times and combined with the historical track coding characteristics and the interactive characteristics of surrounding traffic participants through a decoder, and the final multiple tracks are obtained after the prediction future track is projected and the tracks outside the travelable region are removed.
In order to clearly express the trajectory, the predicted target may be regarded as traveling on a world coordinate system, i.e., the trajectory of the predicted target is formed by coordinate points on the world coordinate system.
Assuming that the current time is the t-th time, n other traffic participants exist in the traffic scene of the automatically driven vehicle, and the other traffic participants are regarded as the obstacles, the historical track of the ith obstacle at the t-th time is as follows:
tau is tau times before t time, tau is shownThe length of the historical track is obtained; the future trajectory at time t is:
where τ ' is τ ' from time t as a reference, τ ' indicates the length of the future trace.
In the training stage, a sensor on the automatic driving vehicle is used for obtaining a historical track and a future track of the automatic driving vehicle and surrounding vehicles, the historical track and the future track can be obtained by data identification obtained by a vision or laser sensor, and a specific identification method is not described in detail in the embodiment; for the ith prediction target, the historical track HistiObtaining the encoding characteristics of the historical track and the Future track Future through the encoder 1iObtaining future track coding characteristics through the encoder 2, obtaining a mean value mu and a variance sigma through two full-connected layers after the superposition of the historical track coding characteristics and the future track coding characteristics2For learning the distribution Z (μ, σ)2)～P(Z|Histi，Futurei) Z is modeled as a Gaussian distribution, Z-N (0, I); the training phase adopts the distribution Z (mu, sigma) of KL divergence metric learning2) Distance loss from gaussian N (0, I) L1(I) ═ KLD (Z (μ, σ)2) N (0, I)); and then according to the learned distribution Z (mu, sigma)2)～P(Z|Histi，Futurei) And carrying out random sampling to obtain a group of hidden variables, obtaining a feature vector with the same dimensionality as the historical track coding feature by the hidden variables through a multilayer perceptron, and multiplying the feature vector with the historical track coding feature through softmax normalization to obtain the historical track coding feature subjected to distributed sampling.
Meanwhile, the interactive features of the vehicles around the ith target are extracted, and the specific method is as follows: coding features of historical tracks of all obstacles around the ith prediction target are obtained by passing the historical tracks of all the obstacles around the ith prediction target through the coder 3 respectively, and then the average value of the coding features of the historical tracks of the obstacles is obtained to obtain interactive features of the obstacles around the ith prediction target; the finally coded characteristics are obtained by overlapping the history track coding characteristics subjected to distributed sampling and the interactive characteristics of the obstacles, and then the finally coded characteristics are decodedObtaining a representation of coordinates of the predicted future trajectory
K random sampling is carried out on the learned distribution to generate K groups of tracks,
in the training phase, the K groups generate a trajectory PredictionikWith the true Future trajectory FutureiCalculating mean square error loss
Acquiring a travelable area aerial view of a scene where the automatic driving vehicle is located, wherein the travelable area aerial view can be obtained by data identification obtained by a vision or laser sensor, and the specific identification method is not described in detail in the embodiment; will generate the trajectory PredictionikThe point in the space is projected to a fly-eye view of a travelable area, and the track points for generating the track are constrained according to the travelable area, and the specific method comprises the following steps: will predict future trajectory Prediction in turnikThe track point in the target area is projected into the aerial view, and the shortest distance from the track point to the travelable area is calculated according to the travelable area aerial view
Corresponding to a future time, wherein if the track point is projected in the travelable area, then
According to the shortest distance from the track point to the travelable area
And a point in time in the future
Computing constraint lossConstraint loss based on travelable region
Wherein, alpha, beta and gamma are hyper-parameter constants,
calculating distance-related constraint values, wherein beta and gamma are used for adjusting constraint intensity, the greater the distance is, the more deviation from a travelable region is shown, the greater the penalty is, namely, the greater the loss is,
for the purpose of a time-dependent adjustment factor,
the penalty is larger and larger with smaller influence, and the loss is larger, so that the track points predicted closer to the current moment are expected to be more accurate.
The discrimination model consists of an encoder 4 and a multilayer perceptron; in the whole training process, the weight of the discriminant model and the weight of the generated model are alternately updated.
The process of updating the discriminant model is as follows: hist history trackiAnd predicting future trajectory PredictionikSplicing into a complete track, sending the complete track into a discrimination model, and obtaining a probability value pred of whether the track is true or false through a multilayer perceptron after the complete track is coded by a coder 4ikAnd a label gtikValue computation cross entropy loss, corresponding label gtikIs false; meanwhile, history track HistiSplicing with a real future track Futurei, sending the real future track Futurei into the discrimination model again, coding the complete track by a coder 4, and obtaining a probability value pred of whether the track is true or false by a multilayer perceptroni_realAnd a label gti_realValue computation cross entropy loss, corresponding label gti_realThe result is true; at this time, the expression for the discrimination loss is obtained as:
total Loss of discriminant modelD_iThe total discriminant model loss is used for L4(i)And the weight corresponding to the new discriminant model.
The process of updating the generative model is as follows: hist history trackiAnd generating a trajectory PredictonikSplicing into a complete track, sending the complete track into a discrimination model, and obtaining a probability value pred of whether the track is true or false through a multilayer perceptron after the complete track is coded by a coder 4ikAnd a label gtikValue computation cross entropy loss, at which point the corresponding label gtikIs true; at this time, the discriminant loss is expressed as:
the total loss of the generative model needs to be weighted and summed with the loss of the generative model and the discriminant model: lossG_i＝α1L1(i)+a2 L2(i)+α3 L3(i)+α4L4(i), wherein α1，α2，α3，α4For the corresponding weight of loss, the generative model total loss is used to update the weight corresponding to the generative model.
Wherein, the above mentioned encoders 1, 2, 3, 4 can be one-dimensional convolution network or RNN series cyclic neural network such as LSTM, GRU, etc., and the decoder can be RNN series cyclic neural network (LSTM, GRU, etc.).
When the prediction stage is executed, the automatic driving automobile predicts the track of the surrounding obstacles in a real driving scene, and only historical track information is available at the moment, but no future track information is available. The specific process of the prediction stage is as follows: for the ith prediction target, the historical track HistiHistorical track coding characteristics are obtained through the coder 1, coding characteristics of surrounding obstacles are obtained through the coder 3 for historical tracks of the obstacles around the ith prediction target, and then the interactive characteristics of the obstacles around the ith prediction target are obtained after the coding characteristics of the obstacles are averaged.
According to the distribution Z (mu, sigma)2) Random sampling is carried out to obtain a group of hidden variables, the hidden variables are subjected to multi-layer perceptron to obtain characteristic vectors with the same dimensionality as the coding characteristics of the historical track, the characteristic vectors are subjected to softmax normalization and then multiplied by the coding characteristics of the historical track to obtain the historical track subjected to distributed samplingA coding feature; the final coded feature is obtained by superposing the historical track coding feature subjected to distributed sampling and the interactive feature of surrounding obstacles, and then the coordinate expression of the predicted future track is obtained by a decoder as follows:
will predict future trajectory PredictioniProjecting the points in the image to a bird's-eye view of the travelable area, constraining track points for generating tracks according to the travelable area, and removing the tracks corresponding to the track points beyond the travelable area; again according to the distribution Z (μ, σ)2) Random sampling is carried out, and the process is repeated until K tracks are generated:
the above-mentioned K value may adopt different K values according to actual needs, and in this embodiment, the K value generally takes 6, 8, and 10.
Future trajectories of other obstacles around the current autonomous vehicle can be predicted and generated according to the prediction method of the ith prediction target.
In the present embodiment, by a multi-track generation method combining CVAE and GAN, a data-driven manner is adopted to learn a generative model in which a distribution Z (μ, σ) under given historical track and future track characteristics is learned and a discriminant model2) The method comprises the steps of performing multiple random sampling on distribution through learning to generate multiple tracks, fully considering historical track characteristics of a predicted target and interaction characteristics of surrounding vehicles through a generation model, and enabling a network to obtain a more real track through learning of a discrimination model; meanwhile, unreasonable tracks are filtered by restricting the driving areas of the generated tracks, and more reasonable and effective tracks are obtained.
In addition, the method in the embodiment can be applied to all types of surrounding traffic participants, including pedestrians, motor vehicles and non-motor vehicles, and the method can be used only by obtaining track information of corresponding types of drivable areas and targets in a scene; if only the track information of the target can generate multiple tracks, only the constraint of the travelable area is lacked.
It should be understood that the examples are for illustrative purposes only and are not intended to limit the scope of the present invention. Further, it should be understood that various changes or modifications of the present invention may be made by those skilled in the art after reading the teaching of the present invention, and such equivalents may fall within the scope of the present invention as defined in the appended claims.
Claims (9)
1. The method for generating the future trajectory of the obstacle is characterized by comprising a training stage and a prediction stage, wherein the training stage simultaneously runs a trajectory generation model and a trajectory discrimination model, the trajectory generation model generates a series of predicted future trajectories with constraints, and the trajectory discrimination model is responsible for discriminating the truth of the generated predicted future trajectories;
the training phase performs the following steps:
a1: acquiring historical tracks and real future tracks of a predicted target and other obstacles around the predicted target, coding the historical tracks and the real future tracks to obtain coding features of the historical tracks and the real future tracks, and acquiring the coding features of the historical tracks subjected to distributed sampling; extracting interactive features of obstacles around the predicted target, and generating a predicted future trajectory of the predicted target;
a2: acquiring a navigable area aerial view of a scene where a predicted target is located, and constraining a predicted future track of the predicted target;
a3: alternately updating the weights of the discrimination model and the generation model;
the prediction phase performs the following steps:
b1: coding the historical tracks of the predicted target and other obstacles around the predicted target to obtain historical track coding features of the predicted target and other obstacles around the predicted target, and obtaining interaction features of the obstacles around the predicted target according to the historical track coding features of the other obstacles around the predicted target; acquiring the historical track coding features subjected to distributed sampling, and acquiring the predicted future track of the predicted target according to the historical track coding features subjected to distributed sampling;
b2: projecting the predicted future trajectory of the predicted target into the travelable area aerial view to constrain the predicted future trajectory of the predicted target;
b3: step B1 is performed until a predicted future trajectory is generated for enough of the plurality of predicted targets.
2. The method according to claim 1, wherein if there are n obstacles in the scene and the ith obstacle is taken as a prediction target, the coordinate expression of the historical trajectory at the t-th time of the prediction target is:
i is less than or equal to n, wherein tau is tau times before the t time,
the coordinate of the target t moment in a world coordinate system is predicted; the real future trajectory of the predicted target at the time t is:
wherein τ 'is τ' from time t as a reference.
3. Method for generating future trajectory of obstacle according to claim 2, wherein said training phase combines the historical trajectories HistiObtaining the encoding characteristics of the historical track and the Future track Future after encodingiObtaining future track coding characteristics after coding; overlapping the historical track coding features and the future track coding features, and obtaining a mean value mu and a variance sigma through two full-connected layers2For learning the distribution Z (mu, sigma)2)～P(Z|Histi，Futurei) Wherein Z is modeled as a Gaussian distribution and Z-N (0, I); calculating Z (. mu.,. sigma.)2) And Gaussian distribution N: (Distance loss of 0, I) is: l1(i) ═ KLD (Z (μ, σ)2) N (0, I)); for Z (mu, sigma)2) Carrying out random sampling to obtain a group of hidden variables, obtaining feature vectors with the same dimensionality as the historical track coding features through the hidden variables by a multilayer perceptron, and then multiplying the feature vectors with the historical track coding features through normalization to obtain the historical track coding features subjected to distributed sampling.
4. The method according to claim 2, wherein the historical trajectories of all obstacles around the predicted target are encoded to obtain the encoding characteristics of surrounding obstacles, and the average of the encoding characteristics of the surrounding obstacles is obtained to obtain the interaction characteristics of the obstacles around the predicted target; overlapping the historical track coding features subjected to distributed sampling and the interactive features of obstacles around the predicted target to obtain the final coding features of the predicted future track, and decoding the final coding features to obtain the expression of the predicted future track of the ith predicted target
Wherein the Prediction isiTo predict future trajectories.
5. A method for generating future trajectory of obstacle according to claim 3, characterized by distributing Z (μ, σ)2) Carrying out random sampling for K times to generate K groups of predicted future tracks of predicted targets, wherein the expression of the predicted future track of the K group of predicted targets is
Wherein (K ∈ K); in the training stage, K groups are used for predicting the future track Prediction of the ith obstacleikWith the true Future trajectory FutureiThe mean square error loss is calculated as:
6. the method for generating the future obstacle trajectory according to claim 1, wherein a bird's eye view of a travelable area of a scene where the autonomous vehicle is located is obtained; and projecting the points in the predicted future track to the aerial view of the travelable area, and constraining the track points of the predicted future track according to the travelable area.
7. The method according to claim 1, wherein the track points in the predicted future track are projected into the bird's-eye view in sequence, and the shortest distance from the track points to the travelable area is calculated according to the travelable area bird's-eye view
Wherein the content of the first and second substances,
corresponding to the future time, if the track point is projected in the travelable area, then
Computing constraint loss
Wherein, alpha, beta and gamma are hyper-parameter constants.
8. The method according to claim 1, wherein the weight updating of the discriminant model comprises: splicing the historical track and the predicted future track into a complete track, sending the complete track into a discrimination model, and obtaining a probability value pred of track truth and falseness by a multilayer perceptron after the complete track is codedikAnd a label gtikValue computation cross entropy loss, corresponding label gtikIs false; splicing the historical track and the real future track, sending the spliced historical track and the real future track into a discrimination model again, coding the complete track, and obtaining whether the track is true or false through a multilayer perceptronProbability value predi_realAnd a label gti_realValue computation cross entropy loss, corresponding label gti_realIs true; the discrimination loss L4(i) is calculated as
Total Loss of discriminant modelD_i＝L4(i)；
The updating of the weights of the generative model comprises: splicing the historical track and the generated track into a complete track, sending the complete track into a discrimination model, and obtaining a probability value pred of whether the track is true or false through a multilayer perceptron after the complete track is codedikAnd a label gtikValue computation cross entropy loss, at which point the corresponding label gtikIs true; the discriminant loss at this time is expressed as:
the total generative model loss requires a weighted sum of the generative model and the discriminant model losses: lossG_i＝α1L1(i)+α2L2(i)+α3L3(i)+α4L4(i), wherein α1，α2，α3，α4Is the loss of weight.
9. The method according to claim 2, wherein in the prediction stage, the historical trajectories of the predicted target and the obstacles around the predicted target are encoded to obtain the encoding characteristics of the historical trajectories of the predicted target and the obstacles around the predicted target, and the average of the encoding characteristics is obtained to obtain the interaction characteristics of the obstacles around the predicted target; according to the learned distribution Z (mu, sigma)2) Carrying out random sampling for K times to obtain a hidden variable to obtain a feature vector, normalizing the feature vector, and multiplying the normalized feature vector by the historical track coding features to obtain the historical track coding features subjected to distributed sampling; the final coded features are obtained by overlapping the historical track coding features of distributed sampling and the interactive features of obstacles around the predicted target, and K predicted future track expressions are obtained after decoding
The predicted future trajectory PredictionikAnd (5) projecting the points in the space to the aerial view of the travelable area, restraining the predicted future track and removing the track beyond the travelable area.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111506962.0A CN114399743B (en) | 2021-12-10 | 2021-12-10 | Method for generating future track of obstacle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111506962.0A CN114399743B (en) | 2021-12-10 | 2021-12-10 | Method for generating future track of obstacle |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114399743A true CN114399743A (en) | 2022-04-26 |
| CN114399743B CN114399743B (en) | 2024-05-28 |
Family
ID=81227946
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111506962.0A Active CN114399743B (en) | 2021-12-10 | 2021-12-10 | Method for generating future track of obstacle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114399743B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115148025A (en) * | 2022-06-28 | 2022-10-04 | 重庆长安汽车股份有限公司 | Traffic target track prediction method and readable storage medium |
| CN115293297A (en) * | 2022-10-08 | 2022-11-04 | 武汉理工大学 | Method for predicting track of ship driven by intention |
| CN116152611A (en) * | 2023-04-14 | 2023-05-23 | 山东省凯麟环保设备股份有限公司 | Multistage multi-scale point cloud completion method, system, equipment and storage medium |
| CN116152782A (en) * | 2023-04-18 | 2023-05-23 | 苏州魔视智能科技有限公司 | Obstacle track prediction method, device, equipment and storage medium |
| CN116309689A (en) * | 2023-05-17 | 2023-06-23 | 上海木蚁机器人科技有限公司 | Obstacle track prediction method, device, equipment and medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010228740A (en) * | 2009-03-05 | 2010-10-14 | Nissan Motor Co Ltd | Travel path creating device, method of creating travel path, and operation supporting device |
| CN108875818A (en) * | 2018-06-06 | 2018-11-23 | 西安交通大学 | Based on variation from code machine and confrontation network integration zero sample image classification method |
| CN112651374A (en) * | 2021-01-04 | 2021-04-13 | 东风汽车股份有限公司 | Future trajectory prediction method based on social information and automatic driving system |
| CN112721950A (en) * | 2021-01-14 | 2021-04-30 | 肖玉连 | Autonomous vehicle control method and system based on deep learning |
-
2021
- 2021-12-10 CN CN202111506962.0A patent/CN114399743B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010228740A (en) * | 2009-03-05 | 2010-10-14 | Nissan Motor Co Ltd | Travel path creating device, method of creating travel path, and operation supporting device |
| CN108875818A (en) * | 2018-06-06 | 2018-11-23 | 西安交通大学 | Based on variation from code machine and confrontation network integration zero sample image classification method |
| CN112651374A (en) * | 2021-01-04 | 2021-04-13 | 东风汽车股份有限公司 | Future trajectory prediction method based on social information and automatic driving system |
| CN112721950A (en) * | 2021-01-14 | 2021-04-30 | 肖玉连 | Autonomous vehicle control method and system based on deep learning |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115148025A (en) * | 2022-06-28 | 2022-10-04 | 重庆长安汽车股份有限公司 | Traffic target track prediction method and readable storage medium |
| CN115148025B (en) * | 2022-06-28 | 2023-10-20 | 重庆长安汽车股份有限公司 | Traffic target track prediction method and readable storage medium |
| CN115293297A (en) * | 2022-10-08 | 2022-11-04 | 武汉理工大学 | Method for predicting track of ship driven by intention |
| CN115293297B (en) * | 2022-10-08 | 2023-01-20 | 武汉理工大学 | Method for predicting track of ship driven by intention |
| CN116152611A (en) * | 2023-04-14 | 2023-05-23 | 山东省凯麟环保设备股份有限公司 | Multistage multi-scale point cloud completion method, system, equipment and storage medium |
| CN116152611B (en) * | 2023-04-14 | 2023-08-25 | 山东省凯麟环保设备股份有限公司 | Multistage multi-scale point cloud completion method, system, equipment and storage medium |
| CN116152782A (en) * | 2023-04-18 | 2023-05-23 | 苏州魔视智能科技有限公司 | Obstacle track prediction method, device, equipment and storage medium |
| CN116309689A (en) * | 2023-05-17 | 2023-06-23 | 上海木蚁机器人科技有限公司 | Obstacle track prediction method, device, equipment and medium |
| CN116309689B (en) * | 2023-05-17 | 2023-07-28 | 上海木蚁机器人科技有限公司 | Obstacle track prediction method, device, equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114399743B (en) | 2024-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Lee et al. | Convolution neural network-based lane change intention prediction of surrounding vehicles for ACC | |
| CN112347567B (en) | Vehicle intention and track prediction method | |
| CN114399743B (en) | Method for generating future track of obstacle | |
| US11816901B2 (en) | Multi-agent trajectory prediction | |
| CN112734808B (en) | Trajectory prediction method for vulnerable road users in vehicle driving environment | |
| CN111930110A (en) | Intent track prediction method for generating confrontation network by combining society | |
| CN111222630A (en) | Autonomous driving rule learning method based on deep reinforcement learning | |
| Cho et al. | Deep predictive autonomous driving using multi-agent joint trajectory prediction and traffic rules | |
| Emuna et al. | Deep reinforcement learning for human-like driving policies in collision avoidance tasks of self-driving cars | |
| CN111661045B (en) | Training generator unit and discriminator unit for collision-aware trajectory prediction | |
| CN113076599A (en) | Multimode vehicle trajectory prediction method based on long-time and short-time memory network | |
| CN113942524B (en) | Vehicle running control method, system and computer readable storage medium | |
| CN114312830A (en) | Intelligent vehicle coupling decision model and method considering dangerous driving conditions | |
| Ye et al. | GSAN: Graph self-attention network for learning spatial–temporal interaction representation in autonomous driving | |
| CN117141517A (en) | Method for constructing vehicle track prediction model by combining data driving and knowledge guiding | |
| CN115547040A (en) | Driving behavior prediction method based on inner neural network under safety potential field | |
| CN117325865A (en) | Intelligent vehicle lane change decision method and system for LSTM track prediction | |
| CN114368387B (en) | Attention mechanism-based driver intention recognition and vehicle track prediction method | |
| Jeong | Predictive lane change decision making using bidirectional long shot-term memory for autonomous driving on highways | |
| Zhang et al. | Predictive trajectory planning for autonomous vehicles at intersections using reinforcement learning | |
| CN114179830B (en) | Autonomous overtaking method and system for automatic driving vehicle | |
| Masmoudi et al. | Autonomous car-following approach based on real-time video frames processing | |
| Hegde et al. | Vehicle trajectory prediction using gan | |
| Basile et al. | Ddpg based end-to-end driving enhanced with safe anomaly detection functionality for autonomous vehicles | |
| Yoon et al. | Trajectory Prediction Using Graph-Based Deep Learning for Longitudinal Control of Autonomous Vehicles: A Proactive Approach for Autonomous Driving in Urban Dynamic Traffic Environments |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |