CN114723784A - Pedestrian motion trajectory prediction method based on domain adaptation technology - Google Patents
Pedestrian motion trajectory prediction method based on domain adaptation technology Download PDFInfo
- Publication number
- CN114723784A CN114723784A CN202210364770.9A CN202210364770A CN114723784A CN 114723784 A CN114723784 A CN 114723784A CN 202210364770 A CN202210364770 A CN 202210364770A CN 114723784 A CN114723784 A CN 114723784A
- Authority
- CN
- China
- Prior art keywords
- domain
- training
- matrix
- pedestrian
- input sample
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000006978 adaptation Effects 0.000 title claims abstract description 62
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000005516 engineering process Methods 0.000 title claims abstract description 15
- 238000012549 training Methods 0.000 claims abstract description 140
- 238000012545 processing Methods 0.000 claims abstract description 19
- 239000000523 sample Substances 0.000 claims description 98
- 239000011159 matrix material Substances 0.000 claims description 93
- 238000012360 testing method Methods 0.000 claims description 64
- 230000003044 adaptive effect Effects 0.000 claims description 38
- 239000013598 vector Substances 0.000 claims description 24
- 238000007781 pre-processing Methods 0.000 claims description 14
- 230000008569 process Effects 0.000 claims description 11
- 238000012546 transfer Methods 0.000 claims description 9
- 238000006073 displacement reaction Methods 0.000 claims description 8
- 230000006870 function Effects 0.000 claims description 6
- 239000013074 reference sample Substances 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 6
- 230000008859 change Effects 0.000 claims description 3
- 238000000354 decomposition reaction Methods 0.000 claims description 3
- 239000000284 extract Substances 0.000 claims description 3
- 238000000605 extraction Methods 0.000 claims description 3
- 230000017105 transposition Effects 0.000 claims description 3
- 230000004927 fusion Effects 0.000 abstract description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30241—Trajectory
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Multimedia (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to the technical field of automatic driving, and particularly discloses a pedestrian motion trail prediction method based on a domain adaptation technologyThen based onAnd X' constructing r time sequence convolution networks for training to obtain r prediction models, and then carrying out training on the r prediction modelsInputting the predicted path into r prediction models to predict r predicted paths of N pedestrians, and finally performing path fusion to obtain the optimal predicted track of each pedestrian. The invention enables the data distribution of the training scene and the actual application scene to be approximately consistent through the domain adaptation processing, enhances the generalization capability of the prediction model, and maps the observation track into the training input tracks with different scales through changing the domain adaptation parameters so as to utilize the deeper information of the observation track, thereby enabling the prediction track to be more accurate.
Description
Technical Field
The invention relates to the technical field of automatic driving, in particular to a pedestrian motion trail prediction method based on a domain adaptation technology.
Background
With the popularization of intelligent transportation, predicting the motion trajectory of pedestrians becomes an increasingly important issue. In unmanned or assisted driving, accurate prediction of pedestrian trajectories facilitates a driving system to plan ahead the motion trajectory of a vehicle in a traffic environment. Currently, research results in the field mainly focus on the following aspects: firstly, modeling is carried out by using various social force models based on social attributes of pedestrian motion in the early stage; in recent years, modeling is performed by combining a deep learning method, such as Social long-short term memory network (Social LSTM), Social generation countermeasure network (Social GAN), graph neural network and other methods, but the following problems are faced:
(1) the construction of the social force model is subjective, and the complex actual motion rule of the pedestrian is difficult to accurately describe;
(2) the deep learning related method depends on massive training data, but the training data scene is not matched with the actual application scene, so that the model generalization capability is poor;
(3) the observation track generally adopts fixed time interval sampling, and the multi-scale and multi-resolution depth information of sampling points with different time intervals is difficult to utilize.
Therefore, the invention carries out the domain adaptation processing on the training scene and the testing scene to ensure that the distribution of the training scene and the testing scene is approximately consistent, and fuses the sequences of sampling points at different time intervals, thereby more accurately predicting the pedestrian track.
Disclosure of Invention
The invention provides a pedestrian motion trail prediction method based on a domain adaptation technology, which solves the technical problems that: how to more accurately predict pedestrian trajectories in crowded environments.
In order to solve the above technical problems, the present invention provides a method for predicting a pedestrian motion trajectory based on a domain adaptation technique, comprising the steps of:
s1, acquiring the motion trail of the pedestrian under a training scene and a testing scene to generate a corresponding training data set and a corresponding testing data set, preprocessing the training data set to generate a training input sample set X and a training output reference set X', and preprocessing the testing data set to generate a testing input sample set Y;
s2, based on r different domain adaptive parameters, carrying out domain adaptive processing on the training input sample set X and the test input sample set Y to obtain r different domain adaptive training input sample setsAdapting a test input sample set to r different domains
S3 adaptive training input sample set based on r fieldsConstructing r time series convolution networks for training by a training output reference set X' to obtain r corresponding different prediction models;
s4, fitting r fields into the test input sample setThe method comprises the steps of inputting the prediction information into r prediction models correspondingly for prediction to obtain r prediction paths of each pedestrian;
and S5, fusing the r predicted paths of each pedestrian to obtain the predicted track of each pedestrian.
Further, the step S2 specifically includes the steps of:
s21, inputting training into the abscissa component X of the sample set XxAnd testing the abscissa component Y of the input sample set YxExtracted and respectively regarded as a source domain and a target domain for domain adaptation processing, and r groups of domain adaptation abscissa source domain matrixes are obtained on the basis of r different main characteristic vector numbers dSum-domain adaptive abscissa target domain matrix
S22, inputting training into the ordinate component X of the sample set XyAnd testing the ordinate component Y of the input sample set YyExtracted and respectively regarded as a source domain and a target domain for domain adaptation processing, and r groups of domain adaptation ordinate source domain matrixes are obtained based on r different main characteristic vector numbers dDome adaptive ordinate object domain matrix
S23, adapting the fields of the same group to the source field matrix of abscissaDome-adaptive ordinate source domain matrixMerging to obtain r field adaptive training input sample setsAdapting the fields of the same group to the abscissa object field matrixDome adaptive ordinate object domain matrixTo carry out the combinationAnd, r groups of domain adaptation test input sample sets are obtained together
Step S21 does not have to be in the order of step S22.
The domain adaptation processing procedure in step S21 specifically includes the steps of:
s211, combining the source domain matrix XxAnd the target domain matrix YxSplicing to obtain a nuclear matrixT at the upper right corner of the matrix represents matrix transposition;
M is the number of samples in the training input sample set X, namely the training output reference set X', N is the total number of pedestrians, K is the number of samples in the test input sample set Y,is a matrixF norm of (d);
s213, constructing a central matrixE is an MN + KN dimensional unit matrix, and 1 is an MN + KN dimensional all-1 row vector;
s214, matrix pair (KMK + mu E)-1KHK carries out eigenvalue decomposition, and extracts the first d main eigenvectors to construct a transfer matrix W, wherein mu is a balance factor;
s215, extracting WTMN column vectors before K form MN x d domain adaptive abscissa source domain matrixExtraction of WTKN column vectors after K form a KN x d domain adaptive abscissa target domain matrix
S216, changing the main feature vector number d for r-1 times, reconstructing the transfer matrix W according to the steps S214 and S215 after each change, and constructing a new domain adaptive abscissa source domain matrix based on the reconstructed transfer matrix WSum-domain adaptive abscissa target domain matrixThereby obtaining r groups of different domain adaptive abscissa source domain matrixes aiming at r different main characteristic vector numbers dDomain-adapted abscissa target domain matrix
The domain adaptation processing procedure in step S22 is the same as step S211 to step S216.
Further, in step S1, the training data set S includes F frame position coordinates obtained by sampling the motion trajectories of N pedestrians at equal time intervals for a long time, that is, the position coordinates are obtainedWherein 1 frame dataThe spatial positions of N pedestrians at a certain moment in time, i.e.Wherein the pedestrian is nThe spatial position of the frame sequence f is marked asF is more than or equal to 1 and less than or equal to F is a frame number, N is more than or equal to 1 and less than or equal to N is a pedestrian number, and (x, y) is a two-dimensional plane coordinate point of a pedestrian;
time period [ Delta T, T + Delta T]T comprising N pedestrians0+T1Frame sequential position coordinates, front T0The frame data can construct a multi-dimensional training input sampleRear T1The frame data can construct a multi-dimensional training output reference sampleWherein training input samples for the nth pedestrianTraining output reference sample of nth pedestrian(xn,i,yn,i) The position coordinates of the nth pedestrian in the ith frame are shown, i is 1,2, …, T0+T1;
By varying the starting time Δ t, i.e. Δ t ═ mTfraM is more than or equal to 0 and less than or equal to (M-1), wherein TfraFor adjacent frame time intervals, obtaining M multi-dimensional training input samples to form a training input sample setAnd M multi-dimensional training output reference samples form a training output reference set
Preprocessing a test data set to generate a test input sample setTo training dataThe same process is used for generating the training input sample set X by preprocessing the set S.
Further, in step S211, the abscissa component X of the input sample set X is trainedxI.e. the source domain matrix XxComprises the following steps:
testing the abscissa component Y of the input sample set YxI.e. the target domain matrix YxComprises the following steps:
in step S215, the corresponding domain is adapted to the abscissa source domain matrixComprises the following steps:
similarly, the domain adaptation in step S22 is based on the vertical coordinate source domain matrixComprises the following steps:
in step S23, mergeAndresulting set of domain-adapted training input samplesSample of (1)Comprises the following steps:
can be obtained by the same way, combinedAndresulting set of domain-adapted test input samplesSample of (1)Comprises the following steps:
further, the step S3 specifically includes the steps of:
s31, adapting the first domain to the training input sample setSample of (1)Rearranging construct setsWherein the sample
S32, inputTraining the L-layer time sequence convolution network, taking the average displacement error between the output of the L-layer time sequence convolution network and a training output reference set X' as a loss function for calculating a training error in the training process, learning a weight value by using a random gradient descent algorithm, and storing a trained prediction model when the training termination condition is that the maximum training period number is met; wherein the convolution process of the time series convolution network is described as follows:
inIs a2 XN dimensional matrix, willInputting L-layer time sequence convolution network to obtain first-layer output1≤t≤d,1≤t′≤T1I is more than or equal to 1 and less than or equal to 2, j is more than or equal to 1 and less than or equal to N and higher layer output1≤t′<T1L is more than or equal to 1 and less than L, and correspondingly obtainedFirst layer output of (2)And higher layer outputThe weights of the convolution kernels of the different layers, η the scale of the convolution kernel,representing the input of the L-layer time series convolution network;
s33, remaining r-1 field adaptation training input sample setAnd repeating the steps S31 to S32 to finally obtain r trained prediction models.
Further, the step S4 specifically includes the steps of:
s41, fitting r fields into the test input sample setSample of (1)Rearranging r sets of structureWherein
S42, collecting rThe predicted paths are input into r prediction models trained in step S33, and r different predicted paths Tra (1), Tra (2), …, Tra (r) are obtained for each pedestrian.
Further, for r different predicted paths Tra (1), Tra (2), …, Tra (r) of the pedestrian n, the step S5 is performed according to Tra ═ α1Tra(1)+α2Tra(2)+…+αrTra (r) to obtain a predicted trajectory Tra of the pedestrian n,αjdenotes the weight of the first predicted path tra (j), j being 1,2, …, r.
The invention provides a pedestrian motion trail prediction method based on a domain adaptation technology, which comprises the steps of firstly acquiring a training data set and a test data set, preprocessing the training data set and the test data set to obtain a training input sample set X, a training output reference set X' and a test input sample set Y, then performing domain adaptation on the two input sample sets X and Y based on r different domain adaptation parameters to obtain a domain adapted data setThen based onAnd X' constructing r time series convolution networks for training to obtain r prediction models, and then training the r prediction modelsInputting the prediction path into r prediction models for prediction to obtain r prediction paths of N pedestrians, and finally fusing the r prediction paths of each pedestrian to obtain the optimal prediction track of each pedestrian. In the invention, the training scene data and the actual application scene data do not meet the independent and same-distribution statistical characteristic, and the training scene data and the actual application scene data are approximately consistent through domain adaptation processing, so that the generalization capability of a prediction model is enhanced; the invention takes the multi-scale and multi-resolution depth information of the observation track into consideration, maps the observation track into different scale training input tracks by changing the domain adaptive parameters, thereby utilizingAnd the deeper information of the track is observed, so that the final predicted track is more accurate as a whole.
Drawings
Fig. 1 is a flowchart of a method for predicting a pedestrian motion trajectory based on a domain adaptation technology according to an embodiment of the present invention.
Detailed Description
The embodiments of the present invention will be described in detail below with reference to the accompanying drawings, which are given solely for the purpose of illustration and are not to be construed as limitations of the invention, including the drawings which are incorporated herein by reference and for illustration only and are not to be construed as limitations of the invention, since many variations thereof are possible without departing from the spirit and scope of the invention.
In order to predict the pedestrian trajectory more accurately, referring to the flowchart in fig. 1, an embodiment of the present invention provides a method for predicting a pedestrian motion trajectory based on a domain adaptation technique, which generally includes the steps of:
s1, acquiring the motion trail of the pedestrian under a training scene and a testing scene to generate a corresponding training data set and a corresponding testing data set, preprocessing the training data set to generate a training input sample set X and a training output reference set X', and preprocessing the testing data set to generate a testing input sample set Y;
s2, carrying out domain adaptation processing on the training input sample set X and the test input sample set Y to obtain r different domain adaptation training input sample setsAdapting a test input sample set to r different domains
S3 adaptive training input sample set based on r fieldsConstructing r time series convolution networks for training by a training output reference set X' to obtain r corresponding different prediction models;
s4, correspondingly inputting r field adaptation test input sample sets Y into r prediction models for prediction to obtain r prediction paths of each pedestrian;
and S5, fusing the r predicted paths of each pedestrian to obtain the predicted track of each pedestrian.
In detail, in step S1, the training data set S includes F-frame position coordinates obtained by sampling the motion trajectories of N pedestrians at equal time intervals over a long period of time, that is, the F-frame position coordinatesWherein 1 frame dataThe spatial positions of N pedestrians at a certain moment in time, i.e.Wherein the spatial position of the pedestrian n in the frame sequence f is marked asF is more than or equal to 1 and less than or equal to F is a frame number, N is more than or equal to 1 and less than or equal to N is a pedestrian number, and (x, y) is a two-dimensional plane coordinate point of a pedestrian;
time period [ Delta T, T + Delta T]T comprising N pedestrians0+T1Frame sequential position coordinates, front T0The frame data can construct a multi-dimensional training input sampleRear T1The frame data can construct a multi-dimensional training output reference sampleTraining input sample of nth pedestrianTraining output reference sample of nth pedestrian(xn,i,yn,i) The position coordinates of the nth pedestrian in the ith frame are shown, i is 1,2, …, T0+T1;
By varying the starting time Δ t, i.e. Δ t ═ mTfraM is more than or equal to 0 and less than or equal to (M-1), wherein TfraFor adjacent frame time intervals, obtaining M multi-dimensional training input samples to form a training input sample setAnd M multi-dimensional training output reference samples form a training output reference set
Preprocessing a test data set to generate a test input sample setThe process of (2) is the same as the process of preprocessing the training data set S to generate the training input sample set X.
Step S2 specifically includes the steps of:
s21, inputting training into the abscissa component X of the sample set XxAnd testing the abscissa component Y of the input sample set YxExtracted and respectively regarded as a source domain and a target domain for domain adaptation processing, and r groups of domain adaptation abscissa source domain matrixes are obtained on the basis of r different main characteristic vector numbers dSum-domain adaptive abscissa target domain matrix
S22, inputting training into the ordinate component X of the sample set XyAnd testing the ordinate component Y of the input sample set YyExtracted and respectively regarded as a source domain and a target domain for domain adaptation processing, and r groups of domain adaptation ordinate source domain matrixes are obtained based on r different main characteristic vector numbers dSum-domain adaptive ordinate object domain matrix
S23, adapting the domains of the same group to the abscissa source domain matrixDome-adaptive ordinate source domain matrixMerging to obtain r field adaptive training input sample setsAdapting the fields of the same group to the abscissa object field matrixDome adaptive ordinate object domain matrixMerging to obtain r groups of domain adaptation test input sample sets
Step S21 does not have to be in the order of step S22.
The domain adaptation processing procedure in step S21 specifically includes the steps of:
s211, combining the source domain matrix XxAnd the target domain matrix YxSplicing to obtain a nuclear matrixT at the upper right corner of the matrix represents matrix transposition;
M is the number of samples in the training input sample set X, namely the training output reference set X', N is the total number of pedestrians, K is the number of samples in the test input sample set Y,is a matrixF norm of (d);
s213, constructing a central matrixE is an MN + KN dimensional unit matrix, and 1 is an MN + KN dimensional all-1 row vector;
s214, matrix pair (KMK + μ E)-1KHK carries out eigenvalue decomposition, and extracts the first d main eigenvectors to construct a transfer matrix W, wherein mu is a balance factor;
s215, extracting WTMN column vectors before K form MN x d domain adaptive abscissa source domain matrixExtraction of WTKN column vectors after K form a KN x d domain adaptive abscissa target domain matrixS216, changing the main feature vector number d for r-1 times, reconstructing the transfer matrix W according to the steps S214 and S215 after each change, and constructing a new domain adaptive abscissa source domain matrix based on the reconstructed transfer matrix WSum-domain adaptive abscissa target domain matrixThereby aiming at r different main characteristicsVector number d obtains r groups of different domain adaptive abscissa source domain matrixesDomain-adapted abscissa target domain matrix
The domain adaptation processing procedure in step S22 is the same as that of steps S211 to S216.
Further illustratively, in step S211, the abscissa component X of the input sample set X is trainedxI.e. the source domain matrix XxComprises the following steps:
testing the abscissa component Y of the input sample set YxI.e. the target domain matrix YxComprises the following steps:
in step S215, the corresponding domain is adapted to the abscissa source domain matrixComprises the following steps:
similarly, the domain adaptation ordinate in step S22Source domain matrixComprises the following steps:
in step S23, mergeAndresulting set of domain-adapted training input samplesSample of (1)Comprises the following steps:
can be obtained by the same way, combinedAndresulting set of domain-adapted test input samplesSample of (1)Comprises the following steps:
based on this, step S3 specifically includes the steps of:
s31, adapting the first domain to the training input sample setSample of (1)Rearranging construct setsIn which the sample
S32, inputTraining the L-layer time sequence convolution network, taking the average displacement error between the output of the L-layer time sequence convolution network and a training output reference set X' as a loss function for calculating a training error in the training process, learning a weight value by using a random gradient descent algorithm, and storing a trained prediction model when the training termination condition is that the maximum training period number is met; wherein the convolution process of the time series convolution network is described as follows:
inIs a2 XN dimensional matrix, willInputting L layers of time sequence convolution network to obtain first layer output1≤t≤d,1≤t′≤T1I is more than or equal to 1 and less than or equal to 2, j is more than or equal to 1 and less than or equal to N and higher layer output1≤t′<T1L is more than or equal to 1 and less than L, and correspondingly obtainedFirst layer output of (2)And higher layer outputThe weights of the convolution kernels of the different layers, η the scale of the convolution kernel,representing the input of an L-layer time series convolutional network;
s33, remaining r-1 field adaptation training input sample setAnd repeating the steps S31 to S32 to finally obtain r trained prediction models.
Step S4 specifically includes the steps of:
s41, fitting r fields into the test input sample setSample of (1)Rearranging r sets of structureWherein
S42, collecting rThe predicted paths are input into r prediction models trained in step S33, and r different predicted paths Tra (1), Tra (2), …, Tra (r) are obtained for each pedestrian.
Then, r different predicted paths Tra (1), Tra (2), …, Tra (r) for the pedestrian n, step S5 is performed based on Tra ═ α1Tra(1)+α2Tra(2)+…+αrTra (r) to obtain a predicted trajectory Tra of the pedestrian n,αjdenotes the weight of the first predicted path tra (j), j being 1,2, …, r. And obtaining the predicted tracks of the N pedestrians in the same way.
The embodiment of the invention provides a pedestrian motion trail prediction method based on a domain adaptation technology, which comprises the steps of firstly acquiring a training data set and a test data set, carrying out preprocessing to obtain a training input sample set X, a training output reference set X' and a test input sample set Y, then carrying out domain adaptation on the two input sample sets X and Y based on r different domain adaptation parameters to obtain a domain adapted data setThen based onAnd X' constructing r time series convolution networks for training to obtain r prediction models, and then training the r prediction modelsInputting the prediction into r prediction models for prediction to obtain r prediction paths of N pedestrians, and finally fusing the r prediction paths of each pedestrian to obtain the optimal prediction track of each pedestrian. In the invention, the training scene data and the actual application scene data do not meet the independent and identically distributed statistical characteristics, and the training scene data and the actual application scene data are approximately consistent through domain adaptive processing, so that the generalization capability of a prediction model is enhanced; the invention takes the multi-scale and multi-resolution depth information of the observation track into consideration, and maps the observation track into different-scale training input tracks by changing the domain adaptive parameters, thereby utilizing the deeper information of the observation track and integrally enabling the final predicted track to be more accurate.
In a specific experiment, the training data set in step S1 includes two data sets (eth, hotel) of eth and 3 data sets (univ, zara1, zara2) of UCY, which are 5 different scene data sets. Table 1 shows an example of a single-frame data segment, a training data set is composed of multi-frame data, each frame of data is marked with a pedestrian number, a frame number, a pedestrian x coordinate and a pedestrian y coordinate, the frame interval is 0.4 second, an observation sample is a pedestrian track of 3.2 seconds and corresponds to T0The next 4.8 seconds of trajectory is predicted, corresponding to T, for 8 frames of image112 frames of pictures.
Table 1 single frame data fragment example
Frame number | Pedestrian numbering | Pedestrian x coordinate | Y coordinate of pedestrian |
10 | 1.0 | 10.7867577985 | 3.67631555479 |
10 | 2.0 | 10.9587077931 | 3.15460523261 |
10 | 3.0 | 10.9993275592 | 2.64673717882 |
The balance factor μ in the domain adaptation algorithm in step S2 is 0.01, the first domain adaptation main feature vector number d is 12, the first time d is 10, and the third time d is 6, that is, the parameter is changed three times r is 3.
Activation function of time series convolutional network in step S3Where a is 0.25, the convolution kernel scale is 2 × 2, and the total number of layers L is 5. The model implementation is based on a Pythrch library, each layer of convolution is realized by depending on a two-dimensional convolution function Conv2d, the loss function is the average displacement error ADE, each batch of training samples is 128, the model is trained for 100 cycles by using random gradient descent (SGD), and the learning rate is 0.01.
Fusion parameter α in step S41=0.2,α2=0.3,α3=0.5。
In step S1, the test data set is also taken from scene 1 to scene 5, and 5 alternative training test modes are adopted to avoid overlapping of the training set and the test set. 1. Training set (hotel, univ, zara1, zara2), test set (eth); 2. training set (eth, univ, zara1, zara2), test set (hotel); 3. training set (eth, hotel, zara1, zara2), test set (univ); 4. training set (eth, hotel, univ, zara2), test set (zara 1); 5. training set (eth, hotel, univ, zara1), test set (zara 2).
And finally, calculating the average displacement error ADE and the final displacement error FDE value of the optimal planning track and the actual track in the test set for evaluating the classification effect. The expressions for the average displacement error ADE and the final displacement error FDE are as follows:
wherein Respectively predicting coordinates and actual coordinates of the t frame of the pedestrian n,the predicted coordinates and the actual coordinates are respectively the last frame of the trajectory of the pedestrian n.
The average ADE FDE values for 5 different scenes of the present invention were compared to other mainstream methods (Linear, S-LSTM, S-GAN-P, SoPhie) and the results are shown in Table 2.
TABLE 2 comparison of ADE/FDE measurements with existing mainstream results
eth | hotel | univ | zara1 | zara2 | Average | |
Linear | 1.33/2.94 | 0.39/0.72 | 0.82/1.59 | 0.62/1.21 | 0.77/1.48 | 0.79/1.59 |
S-LSTM | 1.09/2.35 | 0.79/1.76 | 0.67/1.40 | 0.47/1.00 | 0.56/1.17 | 0.72/1.54 |
S-GAN-P | 0.87/1.62 | 0.67/1.37 | 0.76/1.52 | 0.35/0.68 | 0.42/0.84 | 0.61/1.21 |
SoPhie | 0.70/1.43 | 0.76/1.67 | 0.54/1.24 | 0.30/0.63 | 0.38/0.78 | 0.54/1.15 |
The invention | 0.70/1.28 | 0.39/0.63 | 0.49/0.90 | 0.39/0.66 | 0.32/0.50 | 0.45/0.79 |
As can be seen from table 2, the average ADE value FDE values of the present invention are significantly better than the 4 main stream methods in 5 different scenes.
The above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and all such changes, modifications, substitutions, combinations, and simplifications are intended to be included in the scope of the present invention.
Claims (8)
1. A pedestrian motion trail prediction method based on a domain adaptation technology is characterized by comprising the following steps:
s1, acquiring the motion trail of the pedestrian under a training scene and a testing scene to generate a corresponding training data set and a corresponding testing data set, preprocessing the training data set to generate a training input sample set X and a training output reference set X', and preprocessing the testing data set to generate a testing input sample set Y;
s2, based on r different domain adaptive parameters, carrying out domain adaptive processing on the training input sample set X and the test input sample set Y to obtain r different domain adaptive training input sample setsAdapting a test input sample set to r different domains
S3, training input sample set based on r field adaptationR time series convolution networks are constructed by the training output reference set X' for training to obtain r corresponding different prediction models;
s4, fitting r fields into the test input sample setThe method comprises the steps of inputting the prediction information into r prediction models correspondingly for prediction to obtain r prediction paths of each pedestrian;
and S5, fusing the r predicted paths of each pedestrian to obtain the predicted track of each pedestrian.
2. The method for predicting the motion trail of the pedestrian based on the domain adaptation technology as claimed in claim 1, wherein the step S2 specifically comprises the steps of:
s21, inputting training into the abscissa component X of the sample set XxAnd testing the abscissa component Y of the input sample set YxExtracted and respectively regarded as a source domain and a target domain for domain adaptation processing, and r groups of domain adaptation abscissa source domain matrixes are obtained on the basis of r different main characteristic vector numbers dSum-domain adaptive abscissa target domain matrix
S22, inputting training into the ordinate component X of the sample set XyAnd testing the ordinate component Y of the input sample set YyExtracted and respectively regarded as a source domain and a target domain for domain adaptation processing, and r groups of domain adaptation ordinate source domain matrixes are obtained based on r different main characteristic vector numbers dDome adaptive ordinate object domain matrix
S23, adapting the fields of the same group to the source field matrix of abscissaDome-adaptive ordinate source domain matrixMerging to obtain r field adaptive training input sample setsAdapting the fields of the same group to the abscissa object field matrixDome adaptive ordinate object domain matrixMerging to obtain r groups of domain adaptation test input sample sets
Step S21 does not have to be in the order of step S22.
3. The method for predicting the motion trail of the pedestrian based on the domain adaptation technology as claimed in claim 2, wherein the domain adaptation processing procedure in the step S21 specifically comprises the steps of:
s211, combining the source domain matrix XxAnd the target domain matrix YxSplicing to obtain a nuclear matrixT at the upper right corner of the matrix represents matrix transposition;
M is the number of samples in the training input sample set X, namely the training output reference set X', N is the total number of pedestrians, K is the number of samples in the test input sample set Y,is a matrixF norm of (d);
s213, constructing a central matrixE is an MN + KN dimensional unit matrix, and 1 is an MN + KN dimensional all-1 row vector;
s214, matrix pair (KMK + mu E)-1KHK carries out eigenvalue decomposition, and extracts the first d main eigenvectors to construct a transfer matrix W, wherein mu is a balance factor;
s215, extracting WTMN column vectors before K form MN x d domain adaptive abscissa source domain matrixExtraction of WTKN column vectors after K form a KN x d domain adaptive abscissa target domain matrix
S216, changing the main feature vector number d for r-1 times, reconstructing the transfer matrix W according to the steps S214 and S215 after each change, and constructing a new domain adaptive abscissa source domain matrix based on the reconstructed transfer matrix WSum-domain adaptive abscissa object domain matrixThereby obtaining r groups of different domain adaptive abscissa source domain matrixes aiming at r different main characteristic vector numbers dDomain-adapted abscissa target domain matrix
The domain adaptation processing procedure in step S22 is the same as that of steps S211 to S216.
4. The method for predicting pedestrian motion trail according to claim 3, wherein in the step S1, the training data set S comprises F-frame position coordinates obtained by sampling N pedestrians 'motion trail at equal time intervals for a longer time, that is, N pedestrians' motion trailWherein 1 frame dataThe spatial positions of N pedestrians at a certain moment in time, i.e.Wherein the spatial position of the pedestrian n in the frame sequence f is identified asF is more than or equal to 1 and less than or equal to F is a frame number, N is more than or equal to 1 and less than or equal to N is a pedestrian number, and (x, y) is a two-dimensional plane coordinate point of a pedestrian;
time period [ Delta T, T + Delta T]T comprising N pedestrians0+T1Frame sequential position coordinates, front T0The frame data can construct a multi-dimensional training input sampleRear T1The frame data can construct a multi-dimensional training output reference sampleWherein training input samples for the nth pedestrianTraining output reference sample of nth pedestrian(xn,i,yn,i) The position coordinates of the nth pedestrian in the ith frame are shown, i is 1,2, …, T0+T1;
By varying the starting time Δ t, i.e. Δ t ═ mTfraM is more than or equal to 0 and less than or equal to (M-1), wherein TfraFor adjacent frame time intervals, obtaining M multi-dimensional training input samples to form a training input sample setAnd M multi-dimensional training output reference samples form a training output reference set
5. The method for predicting the motion trail of the pedestrian based on the domain adaptation technology as claimed in claim 4, wherein in step S211, the abscissa component X of the input sample set X is trainedxI.e. the source domain matrix XxComprises the following steps:
testing the abscissa component Y of the input sample set YxI.e. the target domain matrix YxComprises the following steps:
in step S215, the corresponding domain is adapted to the abscissa source domain matrixComprises the following steps:
similarly, the domain adaptation in step S22 is based on the vertical coordinate source domain matrixComprises the following steps:
in step S23, mergeAndresulting set of domain-adapted training input samplesSample of (1)Comprises the following steps:
can be obtained by the same way, combinedAndresulting set of domain-adapted test input samplesSample of (1)Comprises the following steps:
6. the method for predicting the motion trail of the pedestrian based on the domain adaptation technology as claimed in claim 5, wherein the step S3 specifically comprises the steps of:
s31, adapting the first domain to the training input sample setSample of (1)Rearranging construct setsIn which the sample
S32, inputTraining the L-layer time sequence convolution network, taking the average displacement error between the output of the L-layer time sequence convolution network and a training output reference set X' as a loss function for calculating a training error in the training process, learning a weight value by using a random gradient descent algorithm, and storing a trained prediction model when the training termination condition is that the maximum training period number is met; wherein the convolution process of the time series convolution network is described as follows:
inIs a2 XN dimensional matrix, willInputting L-layer time sequence convolution network to obtain first-layer outputAnd higher layer outputCorrespond to obtainFirst layer output of (2)And higher layer output The weights of the convolution kernels of the different layers, η the scale of the convolution kernel,representing the input of the L-layer time series convolution network;
7. The method for predicting the motion trail of the pedestrian based on the domain adaptation technology as claimed in claim 6, wherein the step S4 specifically comprises the steps of:
s41, fitting r fields into the test input sample setSample of (1)Rearranging r sets of structureWherein
8. The method for predicting the motion trail of the pedestrian based on the domain adaptation technology as claimed in claim 7, wherein: r different predicted paths Tra (1), Tra (2), …, Tra (r) for the pedestrian n, in step S5, based on Tra ═ α1Tra(1)+α2Tra(2)+…+αrTra (r) to obtain a predicted trajectory Tra of the pedestrian n,αjdenotes the weight of the first predicted path tra (j), j being 1,2, …, r.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210364770.9A CN114723784B (en) | 2022-04-08 | 2022-04-08 | Pedestrian motion trail prediction method based on domain adaptation technology |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210364770.9A CN114723784B (en) | 2022-04-08 | 2022-04-08 | Pedestrian motion trail prediction method based on domain adaptation technology |
Publications (2)
Publication Number | Publication Date |
---|---|
CN114723784A true CN114723784A (en) | 2022-07-08 |
CN114723784B CN114723784B (en) | 2024-06-14 |
Family
ID=82242090
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210364770.9A Active CN114723784B (en) | 2022-04-08 | 2022-04-08 | Pedestrian motion trail prediction method based on domain adaptation technology |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114723784B (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115272395A (en) * | 2022-07-11 | 2022-11-01 | 哈尔滨工业大学重庆研究院 | Cross-domain migratable pedestrian trajectory prediction method based on depth map convolutional network |
CN115345390A (en) * | 2022-10-19 | 2022-11-15 | 武汉大数据产业发展有限公司 | Behavior trajectory prediction method and device, electronic equipment and storage medium |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101930072A (en) * | 2010-07-28 | 2010-12-29 | 重庆大学 | Multi-feature fusion based infrared small dim moving target track starting method |
CN113316788A (en) * | 2021-04-20 | 2021-08-27 | 深圳市锐明技术股份有限公司 | Method and device for predicting pedestrian motion trail, electronic equipment and storage medium |
CN113689470A (en) * | 2021-09-02 | 2021-11-23 | 重庆大学 | Pedestrian motion trajectory prediction method under multi-scene fusion |
CN113920170A (en) * | 2021-11-24 | 2022-01-11 | 中山大学 | Pedestrian trajectory prediction method and system combining scene context and pedestrian social relationship and storage medium |
-
2022
- 2022-04-08 CN CN202210364770.9A patent/CN114723784B/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101930072A (en) * | 2010-07-28 | 2010-12-29 | 重庆大学 | Multi-feature fusion based infrared small dim moving target track starting method |
CN113316788A (en) * | 2021-04-20 | 2021-08-27 | 深圳市锐明技术股份有限公司 | Method and device for predicting pedestrian motion trail, electronic equipment and storage medium |
CN113689470A (en) * | 2021-09-02 | 2021-11-23 | 重庆大学 | Pedestrian motion trajectory prediction method under multi-scene fusion |
CN113920170A (en) * | 2021-11-24 | 2022-01-11 | 中山大学 | Pedestrian trajectory prediction method and system combining scene context and pedestrian social relationship and storage medium |
Non-Patent Citations (2)
Title |
---|
CAO JIANAO: ""Multi-UAVs Cooperative Target Tracking"", 《2021 33RD CHINESE CONTROL AND DECISION CONFERENCE (CCDC)》, 30 November 2021 (2021-11-30) * |
高铭: ""基于深度学习的复杂交通环境下目标跟踪与轨迹预测研究"", 《中国博士学位论文全文数据库 工程科技II辑》, 15 August 2020 (2020-08-15) * |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115272395A (en) * | 2022-07-11 | 2022-11-01 | 哈尔滨工业大学重庆研究院 | Cross-domain migratable pedestrian trajectory prediction method based on depth map convolutional network |
CN115345390A (en) * | 2022-10-19 | 2022-11-15 | 武汉大数据产业发展有限公司 | Behavior trajectory prediction method and device, electronic equipment and storage medium |
Also Published As
Publication number | Publication date |
---|---|
CN114723784B (en) | 2024-06-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109344736B (en) | Static image crowd counting method based on joint learning | |
Varma et al. | Transformers in self-supervised monocular depth estimation with unknown camera intrinsics | |
CN111581966B (en) | Context feature-fused aspect-level emotion classification method and device | |
CN110263912A (en) | A kind of image answering method based on multiple target association depth reasoning | |
CN113610126A (en) | Label-free knowledge distillation method based on multi-target detection model and storage medium | |
CN112348849A (en) | Twin network video target tracking method and device | |
CN114332578A (en) | Image anomaly detection model training method, image anomaly detection method and device | |
CN114723784A (en) | Pedestrian motion trajectory prediction method based on domain adaptation technology | |
CN110334589A (en) | A kind of action identification method of the high timing 3D neural network based on empty convolution | |
CN111047078B (en) | Traffic characteristic prediction method, system and storage medium | |
CN115100574A (en) | Action identification method and system based on fusion graph convolution network and Transformer network | |
CN113920170B (en) | Pedestrian track prediction method, system and storage medium combining scene context and pedestrian social relationship | |
CN110110372B (en) | Automatic segmentation prediction method for user time sequence behavior | |
CN111508000A (en) | Deep reinforcement learning target tracking method based on parameter space noise network | |
CN111052128A (en) | Descriptor learning method for detecting and locating objects in video | |
CN115311502A (en) | Remote sensing image small sample scene classification method based on multi-scale double-flow architecture | |
CN113688765A (en) | Attention mechanism-based action recognition method for adaptive graph convolution network | |
CN114692732A (en) | Method, system, device and storage medium for updating online label | |
Qiao et al. | SF-Net: Learning scene flow from RGB-D images with CNNs | |
CN103839280B (en) | A kind of human body attitude tracking of view-based access control model information | |
CN111179272A (en) | Rapid semantic segmentation method for road scene | |
Son et al. | Partial convolutional LSTM for spatiotemporal prediction of incomplete data | |
Artemov et al. | Subsystem for simple dynamic gesture recognition using 3DCNNLSTM | |
CN116311504A (en) | Small sample behavior recognition method, system and equipment | |
Zhang et al. | Mobile robot localization based on gradient propagation particle filter network |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |