CN110737968A - Crowd trajectory prediction method and system based on deep convolutional long and short memory network - Google Patents
Crowd trajectory prediction method and system based on deep convolutional long and short memory network Download PDFInfo
- Publication number
- CN110737968A CN110737968A CN201910857181.2A CN201910857181A CN110737968A CN 110737968 A CN110737968 A CN 110737968A CN 201910857181 A CN201910857181 A CN 201910857181A CN 110737968 A CN110737968 A CN 110737968A
- Authority
- CN
- China
- Prior art keywords
- model
- deep
- conv
- pedestrian
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention discloses a crowd track prediction method and system based on a deep convolutional long and short memory network, and the method comprises the steps of determining a pedestrian track data set, determining a training set and a testing set according to the pedestrian track data set, building a deep convolutional long and short memory network Conv-LSTM model again, determining parameters of the deep Conv-LSTM model according to the training set and the testing set, and finally performing crowd track prediction according to the deep Conv-LSTM model with the parameters.
Description
Technical Field
The invention relates to the technical field of traffic simulation and robot path planning, in particular to crowd trajectory prediction methods based on deep convolutional long and short memory networks.
Background
Meanwhile, in the field of intelligent robots, although the existing robots are developed in breakthrough in aspects of life communication, motion stability and the like, the path planning of the robots is still a big problem .
Currently, pedestrian trajectory prediction methods can be briefly divided into two categories: physical motion modeling and data-driven trajectory series prediction.
Physical motion modeling is to regard pedestrians as physical entities, adapted to specific mechanical rules, with the goal of finding the most suitable geometric-physical force model for an actual pedestrian. Although the predicted track avoids the detected obstacle in the walking process by adopting physical motion modeling, for the dynamic environment containing the moving obstacle, particularly dense people, the algorithms cannot combine the moving rule of the pedestrian to make an early response, or the problem of non-optimal path is caused by error avoidance, so that the flexibility and intelligence of the predicted track are greatly reduced.
Although the researches combine the movement law of the pedestrians, the phenomena that the individual pedestrians can pass through obstacles or track abnormity and the like can occur in a test scene due to unreasonable network design and the fact that the network only depends on data trained in a single scene, and particularly in an emergency evacuation scene, the prediction capability of the neural network tracks is not high.
Disclosure of Invention
The invention aims to provide crowd trajectory prediction methods and systems based on a deep convolutional long and short memory network, and improve the precision of crowd trajectory prediction.
In order to achieve the purpose, the invention provides the following scheme:
the invention provides crowd trajectory prediction methods based on deep convolutional long and short memory networks, which comprise the following steps:
the invention provides crowd trajectory prediction methods based on deep convolutional long and short memory networks, which comprise the following steps:
step S1: determining a pedestrian trajectory data set; the pedestrian trajectory data set comprises: coordinate information of destination, position information, speed information and quantity information of pedestrians in a surrounding set range, and position information of fixed obstacles;
step S2: determining a training set and a testing set according to the pedestrian trajectory data set;
step S3: constructing a Conv-LSTM model of a deep convolution long-time memory network; the deep convolution long-time and short-time memory network model comprises a deep Conv-LSTM space-time memory layer, a global spatial feature plus deep layer and a fully-connected regression layer;
step S4: determining parameters of a deep Conv-LSTM model according to the training set and the test set;
step S5: and predicting the crowd track according to the deep Conv-LSTM model with the parameters.
Optionally, the determining parameters of the deep Conv-LSTM model according to the training set and the test set specifically includes
Step S41: substituting part of labeled data in the training set into a deep Conv-LSTM model to determine a model loss value;
step S42: judging whether the magnitude of the model loss value is within a set range; if the magnitude of the model loss value is within the set range, substituting various scene data in the test set into a deep Conv-LSTM model, determining the average relative displacement error of the model, and executing step S43; if the magnitude of the model loss value is not in the set range, taking the current model loss value as an expected loss value, utilizing an Adam optimization algorithm to update parameters in a deep Conv-LSTM model according to the expected loss value, and then executing step S41;
step S43: judging whether the average relative displacement error of the model is less than or equal to a set value or not; if the average relative displacement error of the model is less than or equal to a set value, outputting parameters of the deep Conv-LSTM model; and if the model average relative displacement error is larger than the set value, increasing the layer number of the deep Conv-LSTM space-time memory layer in the deep Conv-LSTM model, and returning to the step S41.
Optionally, the formula for determining the model average relative displacement error is as follows:
where n denotes the time of the predicted trajectory, Preal(t, i) and Preal(t + Δ t, i) represents the real displacement of the ith person in the scene at time t and time t + Δ t, respectively, Ppredicted(t, i) represents the displacement predicted by the model at time t for the ith pedestrian.
Optionally, the determining the pedestrian trajectory data set specifically includes:
a high-definition camera is adopted to record crowd evacuation and convection videos;
and tracking the track of each pedestrian in the video by using a video tracking algorithm, and determining a pedestrian track data set.
The invention also provides crowd trajectory prediction systems based on the deep convolutional long and short memory network, the system includes:
the pedestrian trajectory data set determining module is used for determining a pedestrian trajectory data set; the pedestrian trajectory data set comprises: coordinate information of destination, position information, speed information and quantity information of pedestrians in a surrounding set range, and position information of fixed obstacles;
a training set and test set determining module for determining a training set and a test set according to the pedestrian trajectory data set;
the model building module is used for building a Conv-LSTM model of the deep convolution time memory network; the deep convolution long-time and short-time memory network model comprises a deep Conv-LSTM space-time memory layer, a global spatial feature plus deep layer and a fully-connected regression layer;
a parameter determination module for determining parameters of the deep Conv-LSTM model according to the training set and the test set;
and the crowd track prediction module is used for predicting the crowd track according to the deep Conv-LSTM model with the parameters.
Optionally, the parameter determining module specifically includes
The model loss value determining unit is used for substituting part of labeled data in the training set into a deep Conv-LSTM model to determine a model loss value;
judging unit, which is used to judge whether the order of the model loss value is in the setting range, if the order of the model loss value is in the setting range, the various scene data in the test set is substituted into the deep Conv-LSTM model, the average relative displacement error of the model is determined, the second judging unit is executed, if the order of the model loss value is not in the setting range, the current model loss value is taken as the expected loss value, the parameter in the deep Conv-LSTM model is updated according to the expected loss value by using Adam optimization algorithm, and then the model loss value is returned to the model loss value determining unit;
the second judging unit is used for judging whether the average relative displacement error of the model is less than or equal to a set value or not; if the average relative displacement error of the model is less than or equal to a set value, outputting parameters of the deep Conv-LSTM model; and if the average relative displacement error of the model is larger than a set value, increasing the layer number of the deep Conv-LSTM space-time memory layer in the deep Conv-LSTM model, and returning to the model loss value determining unit.
Optionally, the formula for determining the model average relative displacement error is as follows:
where n denotes the time of the predicted trajectory, Preal(t, i) and Preal(t + Δ t, i) represents the real displacement of the ith person in the scene at time t and time t + Δ t, respectively, Ppredicted(t, i) represents the displacement predicted by the model at time t for the ith pedestrian.
Optionally, the pedestrian trajectory data set determining module specifically includes:
the video acquisition unit is used for recording crowd evacuation and convection videos by adopting a high-definition camera;
and the pedestrian track data set determining unit is used for tracking the track of each pedestrian in the video by utilizing a video tracking algorithm and determining a pedestrian track data set.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
the method respectively represents the speed information of pedestrians, the position information of fixed obstacles, the number information of the pedestrians, the relative position information and the coordinate information of destination in a fixed range of the surrounding in a two-dimensional vector mode, integrates the information to represent the data information of the pedestrians in an evacuation scene at the moment, constructs a deep convolution duration memory network Conv-LSTM model according to the information, predicts the crowd track by utilizing the deep Conv-LSTM model, improves the complexity of the network and the precision of the forecast of the crowd track in aspect, improves the time and space memory of the crowd history track by the deep convolution network, adopts context semantic space information output by the convolution network to the Conv-LSTM layer in aspect, and further improves the regression capability of the network in step.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a flowchart of a method for predicting a trajectory of a crowd according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of a deep Conv-LSTM model according to an embodiment of the present invention;
FIG. 3 is a block diagram of a system for predicting a trajectory of a population according to an embodiment of the present invention;
FIG. 4 is a grid-based room evacuation map according to an embodiment of the present invention;
FIG. 5 is a graph of velocity information in a grid map according to an embodiment of the present invention;
FIG. 6 is a diagram of fixed obstacle information in a scenario according to an embodiment of the present invention;
FIG. 7 is a graph of relative location information of objects in a population according to an embodiment of the present invention;
FIG. 8 is a diagram of destination information for a target pedestrian according to an embodiment of the present invention;
FIG. 9 is a representation of integrated information in a scene for a routine implemented by the present invention;
FIG. 10 is a test scenario diagram according to an embodiment of the present invention;
FIG. 11 is a comparison graph of real tracks according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only partial embodiments of of the present invention, rather than all embodiments.
The invention aims to provide crowd trajectory prediction methods and systems based on a deep convolutional long and short memory network, and improve the precision of crowd trajectory prediction.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, a more detailed description is provided below in conjunction with the accompanying drawings and the detailed description.
Fig. 1 is a flowchart of a crowd trajectory prediction method according to an embodiment of the present invention, and as shown in fig. 1, the present invention provides crowd trajectory prediction methods based on a deep convolutional long and short memory network, where the method includes:
step S1: determining a pedestrian trajectory data set; the pedestrian trajectory data set comprises: coordinate information of destination, position information, speed information and quantity information of pedestrians in a surrounding set range, and position information of fixed obstacles;
step S2: determining a training set and a testing set according to the pedestrian trajectory data set;
step S3: constructing a Conv-LSTM model of a deep convolution long-time memory network;
step S4: determining parameters of a deep Conv-LSTM model according to the training set and the test set;
step S5: and predicting the crowd track according to the deep Conv-LSTM model with the parameters.
The following detailed analysis of each step was performed:
step S2: determining a training set and a testing set according to the pedestrian trajectory data set; the method specifically comprises the following steps:
step S21: dividing the scenes in the experiment into a grid graph according to the floor area (0.4m by 0.4m) of each individual unit;
step S22: adopting a two-dimensional matrix of 5 × 5 to represent the transverse speed of the target pedestrian and all transverse speed information of other pedestrians in the range of 2m × 2m around the target pedestrian; adopting a two-dimensional matrix of 5 × 5 to represent the longitudinal speed of the target pedestrian and all longitudinal speed information of other pedestrians in the range of 2m × 2m around the target pedestrian; representing fixed barrier information around the target pedestrian by using a two-dimensional matrix of 5 x 5; adopting a two-dimensional matrix of 5 x5 to represent the relative position information of the target pedestrian and the pedestrians around the target pedestrian; adopting a two-dimensional matrix of 5 × 5 to represent the transverse displacement information of the target pedestrian and the pedestrians around the target pedestrian relative to the destination; adopting a two-dimensional matrix of 5 x5 to represent the longitudinal displacement information of the target pedestrian and the pedestrians around the target pedestrian relative to the destination; combining six 5-by-5 two-dimensional matrix information of the target pedestrian into 6 channels of deep tensor convolution input high grade facing to crowd track prediction;
step S23: the processed data are randomly selected from each scene to be 70% as a training set, and the rest 30% as a testing set.
Step S3: constructing a Conv-LSTM model of a deep convolution long-time memory network; as shown in particular in fig. 2.
The deep convolution long-time and short-time memory network model comprises a deep Conv-LSTM space-time memory layer, a global spatial feature plus deep layer and a fully-connected regression layer.
1. The deep Conv-LSTM spatiotemporal memory layer is constructed as follows:
the deep Conv-LSTM space-time memory layer is composed of layers 1-3, each layer has 8 Conv-LSTMs, the input of which is a sequence of 8 observed pedestrian trajectories from time t to t +8, receives 8 consecutive 6@5 tensors and initial hidden layer inputs of 32@5, each Conv-LSTM outputs hidden layer tensors of @5 and element tensors of @ 32@5, the second layer receives 8 consecutive 32@5 hidden layer tensors and 64@5 initial hidden layer inputs, each Conv-LSTM outputs hidden layer inputs of @5 and element tensors of @ 64, 3 receives 8 consecutive 64@5 hidden layer tensors and element tensors of @5, and 128 element tensors of @ 5.
2. The structure of the global spatial feature deepening layer is as follows:
taking the last hidden layer outputs of Conv-LSTM in the deep Conv-LSTM space-time memory layer as the input of the second partial convolution operation, and performing convolution operation with 256 × 3 convolution kernels to obtain 256 × 3 convolution layers;
3. the construction of the fully connected regression layer is as follows:
setting the total number of nodes of the first two fully-connected layers as 256;
setting the retention probability of the dropout layer to be 0.5;
the node number of the third full-connection layer is set to be 2, and the predicted speed v of the pedestrian at the time t +9 is respectively representedx,vy。
Step S4: determining parameters of the deep Conv-LSTM model according to the training set and the test set specifically comprises
Step S41: substituting 30 labeled data in the training set into a deep Conv-LSTM model, and determining a model loss value by using a mean square error loss value formula;
step S42: judging whether the magnitude of the model loss value is within a set range; if the magnitude of the model loss value is within the set range, substituting various scene data in the test set into a deep Conv-LSTM model, determining the average relative displacement error of the model, and executing step S43; if the magnitude of the model loss value is not in the set range, taking the current model loss value as an expected loss value, utilizing an Adam optimization algorithm to update parameters in a deep Conv-LSTM model according to the expected loss value, and then executing step S41;
step S43: judging whether the average relative displacement error of the model is less than or equal to a set value or not; if the average relative displacement error of the model is less than or equal to a set value, outputting parameters of the deep Conv-LSTM model; and if the model average relative displacement error is larger than the set value, increasing the layer number of the deep Conv-LSTM space-time memory layer in the deep Conv-LSTM model, and returning to the step S41.
The method further includes setting an initial value of the current expected loss value to zero before step S41.
The formula for determining the average relative displacement error of the model is as follows:
where n denotes the time of the predicted trajectory, Preal(t, i) and Preal(t + Δ t, i) represents the real displacement of the ith person in the scene at time t and time t + Δ t, respectively,Ppredicted(t, i) represents the displacement predicted by the model at time t for the ith pedestrian.
Step S1: the determining the pedestrian trajectory data set specifically includes:
step S11: a high-definition camera is adopted to record crowd evacuation and convection videos;
step S12: and tracking the track of each pedestrian in the video by using a video tracking algorithm, and determining a pedestrian track data set.
Fig. 3 is a structure diagram of a crowd trajectory prediction system according to an embodiment of the present invention, and as shown in fig. 3, the present invention further provides crowd trajectory prediction systems based on a deep convolutional long and short memory network, where the system includes:
the pedestrian trajectory data set determining module 1 is used for determining a pedestrian trajectory data set; the pedestrian trajectory data set comprises: coordinate information of destination, position information, speed information and quantity information of pedestrians in a surrounding set range, and position information of fixed obstacles;
a training set and test set determining module 2, configured to determine a training set and a test set according to the pedestrian trajectory data set;
the model building module 3 is used for building a Conv-LSTM model of the deep convolution long-time memory network; the deep convolution long-time and short-time memory network model comprises a deep Conv-LSTM space-time memory layer, a global spatial feature plus deep layer and a fully-connected regression layer;
a parameter determining module 4, configured to determine parameters of a deep Conv-LSTM model according to the training set and the test set;
and the crowd track prediction module 5 is used for predicting the crowd tracks according to the deep Conv-LSTM model with the parameters.
The various modules are discussed in detail below:
the parameter determination module 4 specifically comprises
The model loss value determining unit is used for substituting part of labeled data in the training set into a deep Conv-LSTM model to determine a model loss value;
judging unit, which is used to judge whether the order of the model loss value is in the setting range, if the order of the model loss value is in the setting range, the various scene data in the test set is substituted into the deep Conv-LSTM model, the average relative displacement error of the model is determined, the second judging unit is executed, if the order of the model loss value is not in the setting range, the current model loss value is taken as the expected loss value, the parameter in the deep Conv-LSTM model is updated according to the expected loss value by using Adam optimization algorithm, and then the model loss value is returned to the model loss value determining unit;
the second judging unit is used for judging whether the average relative displacement error of the model is less than or equal to a set value or not; if the average relative displacement error of the model is less than or equal to a set value, outputting parameters of the deep Conv-LSTM model; and if the average relative displacement error of the model is larger than a set value, increasing the layer number of the deep Conv-LSTM space-time memory layer in the deep Conv-LSTM model, and returning to the model loss value determining unit.
The formula for determining the average relative displacement error of the model is as follows:
where n denotes the time of the predicted trajectory, Preal(t, i) and Preal(t + Δ t, i) represents the real displacement of the ith person in the scene at time t and time t + Δ t, respectively, Ppredicted(t, i) represents the displacement predicted by the model at time t for the ith pedestrian.
The pedestrian trajectory data set determining module 1 specifically includes:
the video acquisition unit is used for recording crowd evacuation and convection videos by adopting a high-definition camera;
and the pedestrian track data set determining unit is used for tracking the track of each pedestrian in the video by utilizing a video tracking algorithm and determining a pedestrian track data set.
The method respectively represents the speed information of pedestrians, the position information of fixed obstacles, the number information of the pedestrians, the relative position information and the coordinate information of destination in a fixed range of the surrounding in a two-dimensional vector mode, integrates the information to represent the data information of the pedestrians in an evacuation scene at the moment, constructs a deep convolution duration memory network Conv-LSTM model according to the information, predicts the crowd track by utilizing the deep Conv-LSTM model, improves the complexity of the network and the precision of the forecast of the crowd track in aspect, improves the time and space memory of the crowd history track by the deep convolution network, adopts context semantic space information output by the convolution network to the Conv-LSTM layer in aspect, and further improves the regression capability of the network in step.
The crowd trajectory prediction method and system of the present invention does not require any pooling/regularization terminology because the present invention learns the single framework for all pedestrians in the scene instead of designing single trajectory models, which means much lower complexity and strong spatial context modeling capability, unlike the traditional LSTM method, the end-to-end method of the present invention can better handle temporal dependencies.
The invention is further described with reference to the following drawings:
according to the floor area of a single pedestrian, the whole map is divided into a mesh map with the size of a single grid of 0.4 x 0.4 (unit m), and the feeling of the pedestrian to the surroundings is also represented by the mesh map. In the following, the deep multi-channel convolution input tensor facing the crowd trajectory prediction designed by the invention is introduced by taking a pedestrian evacuation scene as an example.
As shown in FIG. 4, (a) is a grid diagram, and (b) is a real experimental scene corresponding to the map, the experiment is carried out in fields of Beijing aerospace university, the invention divides the whole scene into 36 grid diagrams, each grid size is 0.4 (unit: m), black grids are the distribution of walls and obstacles, gray grids are the destination, gray circles are the positions of pedestrians in the scene, and the likeThe map can show the distribution state of all pedestrians at a certain moment and the mutual information of the spatial positions of the pedestrians over time, the position information of the scene at t can be used as a matrix
To represent it.
The present invention summarizes that, as shown in fig. 5, (a) and (b) represent the lateral and longitudinal velocities of each pedestrians arriving at the location in an evacuation scene at a certain time, respectively, the pedestrian perception in real life is -bounded, and by questionnaire survey, the present invention concludes that pedestrians generally observe a range of about 2 × 2 (unit: m) around themselves when making motion decisions, so the present invention takes a 5 × 5-sized network around a single pedestrian as effective data for a single pedestrian, in fig. 5, the light gray grid represents the lateral and longitudinal velocity values of pedestrians in this grid at the current time, and if the present invention is to obtain effective data information labeled with dark gray, then a 5 × 5-sized peripheral (5 × 5) and other information (5 a) surrounding pedestrian matrix are obtained, and if the present invention is to obtain corresponding information for a pedestrian, the pedestrian is also obtained for the relevant sensing of the velocity in the front-and front-5-sized matrix (5 × 5).
The present invention relates to a method for representing obstacles by accessibility, and a grid with obstacles is defined as an inaccessible area with a value of 1, and a grid without obstacles is defined as an accessible area with a value of 0, and a grid without obstacles is defined as an inaccessible area with a value of 0, according to the above rules, the present invention obtains obstacle data information as shown in fig. 6, and a grid with a value of 0 is an area that can be reached by pedestrians, and a grid with a value of 1 is an area that cannot be reached by pedestrians, and also the present invention defines a grid with a pedestrian's accessibility of 2: (unit: m) to the surroundings, and a pedestrian's 1 area that is felt within the range is accessible, and a pedestrian's 2 felt presence of a wall makes a corresponding avoidance action, and a pedestrian's 3 is defined as a grid with a pedestrian's accessibility of 0, and a grid with a value of 635 as a small accessibility, and a single destination point 365 represents a single obstacle vector capable of representing pedestrians as a small number of being felt by pedestrians and a pedestrian.
In questionnaires, most sample pedestrians react to the number and relative positions of surrounding pedestrians in a crowd and have great influence on the decision of the motion trail of the pedestrians, people can reduce the traveling speed in a dense crowd, the speed of the pedestrians is relatively high in a sparse crowd, and people can search a low-density area as the approximate traveling direction of steps, so that the pedestrian density information around the pedestrians in a scene is very important, and the relative position distribution of the surrounding pedestrians can influence the specific motion direction of the pedestrians.
For anyIn the evacuation scene of emergency escape, the pedestrian certainly knows where the exit of the destination of the pedestrian is when the pedestrian escapes in advance, and when the pedestrian passes through a traffic light intersection of a road, the pedestrian also knows that the destination of the pedestrian is opposite to the road instead of purposeless walking, so that the information of effectively representing the destination is very important. The invention uses the displacement vector from horizontal and vertical directions to the destination
And the two ratios of the maximum distance W and the maximum distance H in the transverse and longitudinal directions of the scene respectively represent the difference between the pedestrian and the target from the two directions. The closer the two values are to 0, the closer the pedestrian is to the destination.
μXAnd muYThe representation being grouped into 
And
this facilitates faster convergence of the network during training.
And
for indicating horizontal and vertical displacement to the destination. W and H represent the maximum distances in the horizontal and vertical directions of the scene.
As shown in fig. 8, the small flag is the destination, and according to the rule of the coordinate diagram, the present invention can easily find that the destination direction of the pedestrian above the scene is lower right, the destination direction of the pedestrian in the middle is horizontal, and the destination direction of the pedestrian below is upper right, so that the present invention can effectively represent the destination information of the pedestrian.
The pedestrian speed information in the certain range of the surrounding , the position information of fixed obstacles, the pedestrian number information, the relative position information and the destination coordinate information are respectively represented in a two-dimensional vector form, as shown in FIG. 9, X1 represents the transverse speed of the pedestrian at the moment and the transverse speed of the surrounding pedestrian, X2 represents the longitudinal speed of the pedestrian at the moment and the longitudinal speed of the surrounding pedestrian, X3 represents that no obstacle exists around the pedestrian at the moment, X4 represents the relative position information of 5 people around the pedestrian at the moment, X5 represents that the pedestrian should walk to the right in the transverse direction, and X6 represents that the pedestrian does not need to be displaced in the longitudinal direction, the data information of the six aspects of X1-X6 can be represented by integrating the information of the pedestrian at the moment in an evacuation scene, and the deep layer tensor data form obtained by integrating the information of the six aspects can return the speed information of at the moment in the transverse direction and the longitudinal direction under the pedestrian.
Building deep convolutional long-short time memory network models as shown in FIG. 3, wherein the structures of the models are deep Conv-LSTM spatiotemporal memory layer → global spatial features plus deep layers in sequence;
the deep Conv-LSTM spatiotemporal memory layer is constructed as follows:
the deep Conv-LSTM space-time memory layers are composed of layers 1-3, each layer has 8 Conv-LSTMs, the input of which is a sequence of 8 observed pedestrian trajectories from time t to t +8, receives 8 consecutive 6@5 tensors and initial hidden layer inputs of 32@5, each Conv-LSTM outputs hidden layer tensors of 32@5 and element tensors of 32@5, the second layer receives 8 consecutive 32@5 hidden layer tensors and 64@5 initial hidden layer inputs, each Conv-LSTM outputs hidden layer inputs of 64@5 and element tensors of 64@5, the 3 rd receives 8 consecutive 64@5 hidden layer inputs and 128@5 initial hidden layer inputs, and each Conv-LSTM outputs 128 element tensors of 128@5 and 128@5 hidden layer outputs.
The structure of the global spatial feature deepening layer is as follows:
the invention takes the last hidden layer outputs of Conv-LSTM space-time memory layers of the deep layer as the input of the second partial convolution operation, and obtains 256 × 3 convolution layers by performing convolution operation with 256 × 3 convolution kernels.
The construction of the fully connected regression layer is as follows:
setting the total number of nodes of the first two fully-connected layers as 256;
setting the retention probability of the dropout layer to be 0.5;
the node number of the third full-connection layer is set to be 2, and the node numbers respectively represent the predicted speed v of the pedestrian at the time t + 9x，vy。
For the problem of predicting the spatiotemporal sequence of the pedestrian trajectory, the current output is times correlated with the previous time, so the invention needs sequence data as the input of the network, the experimental data of the invention adopts a camera of 30FPS, in combination with the fact that the invention needs to observe pedestrians for about 1 to 2 seconds in practice, the invention extracts frames of data from 65 frames every 12 frames, and arranges the data into 8 sequences according to times in sequence as the input of the network (the invention gives the reason for extracting the data in the experimental part), the invention acquires 20 groups of evacuation scenes and 4 groups of convection scenes as the source of network training data, and the invention prepares the training data according to the format of the data described above.
The present invention employs Adam optimization algorithm during network training, Adam algorithm differs from conventional SGD random gradient descent maintains a single learning rate (i.e. alpha) updating all weights, the learning rate does not change during training, Adam designs independent adaptive learning rates for different parameters by computing the order moment estimate and the second order moment estimate of the gradient, the proposer of Adam algorithm describes it as a set of advantages of two random gradient descent expansions, i.e. adaptive gradient algorithm (AdaGrad) maintains learning rates for each parameters to boost performance on sparse gradients (i.e. natural language and computer vision problems), root mean square transfer (RMSProp) adaptively maintains learning rates for each parameters based on the mean of the nearest magnitude of the gradient weight, which means the algorithm has excellent performance on unsteady and on-line problems, Adam algorithm obtains both the optimal points of AdaGrad and rmrop algorithms at the same time as the mean of the average gradient weight gradient, which means that the algorithm has excellent performance on the unsteady and on-line problems, which is calculated by using the mean deviation of the average gradient estimate of the average gradient, which is close to the mean deviation of the average of the adaptive gradient estimate of the moving average gradient (i.e. the adaptive gradient), thus the adaptive gradient estimate of the adaptive gradient deviation of the adaptive gradient score of Adam algorithm 3580) and the adaptive gradient (367) and the adaptive gradient estimate, which is calculated by using the mean deviation of the mean shift of the average gradient, the average gradient estimate, the average gradient of the average gradient, the average gradient estimate of the average gradient, the adaptive gradient of the average gradient estimate.
The initial use of 1 x10 in the training of the model herein-3The network finds a lower Loss value in a faster time with a larger learning rate, and then the learning rate is gradually reduced, so that the final Loss value of the model is stabilized at 1 × 10-6This magnitude represents that the model parameters of the invention are relatively optimal values.
The effect of the present invention is further described in with the simulation experiment:
1. simulation experiment conditions are as follows:
the present invention was tested using 4 evacuation scenario data (16802 samples) and 2 convection data (3925 samples), which were collected under a variety of scenarios, including actual evacuation and convection. Note that the present invention herein uses only scenarios that do not occur in the training set as test scenarios. The map of these scenes is shown in fig. 10, a) rooms with vertical obstacles are evacuated, b) rooms with horizontal obstacles are evacuated, c) convection, i.e. zebra stripes, d) rooms with two vertical obstacles are evacuated, e) rooms are evacuated with two horizontal obstacles. The hardware platform of the simulation experiment is Intel KuRui i77700K @4.2GHzCPU, 32GB RAM, NVIDIA Geforce GTX1080Ti GPU, and the software platform of the simulation experiment is Python 3.5 and Pyorch 1.0.0.
2. Simulation experiment content and result analysis:
the invention compares with the real trace map in the test scene, mainly compare the performance of various algorithms under from the macroscopic trend overall, the invention adopts the same number of people with real scene in the experiment, these people all begin from the same starting point at the same time, figure 11 shows the contrast of 3 algorithms of depth Conv-LSTM, SF, Social-LSTM and real trace in the competitive and non-competitive two kinds of scenes, it is not difficult to find from figure 11 the invention, depth Conv-LSTM is similar to real trace in various scenes to a greater extent, obviously superior to other two kinds of algorithms.
In the invention, a multi-channel tensor is used to represent spatial information about a pedestrian in a relatively dense population. For each pedestrian, the information includes a situational map, pedestrian neighbors, obstacles, relative positions of the crowd and the destination. Using the real data, an end-to-end convolutional LSTM network is designed and trained. In addition, to reduce the trajectory prediction error, the network is deepened and the number of layers is optimized. Experimental results show that compared with Social-LSTM and Social force models, the proposed network can produce more realistic trajectories.
This approach holds great promise as an better trajectory prediction model to help autonomous vehicles avoid collisions with pedestrians.
The deep convolution long and short memory network model (Social-Conv-LSTM) designed by the invention is used for simulating the flowing state of pedestrians in crowds, the track pedestrian track prediction model is combined with the advantages of the long and short memory (LSTM) and the convolution network, the invariance of the convolution space characteristic of the convolution network is fully utilized, the rolling machine network is used for replacing the full-connection network in the traditional long and short memory, and meanwhile, the network deepening structure is designed, so that the complexity and the distinguishing capability of the network are guaranteed, and meanwhile, the track prediction of the pedestrians is more accurate. The result shows that the crowd motion simulation performed by the model is more accurate and real, and the position deviation is smaller.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.
The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core idea of the present invention, and to those skilled in the art with variations in the specific embodiments and applications of the invention.
Claims (8)
1. The crowd trajectory prediction method based on the deep convolutional long and short memory network is characterized by comprising the following steps:
step S1: determining a pedestrian trajectory data set; the pedestrian trajectory data set comprises: coordinate information of destination, position information, speed information and quantity information of pedestrians in a surrounding set range, and position information of fixed obstacles;
step S2: determining a training set and a testing set according to the pedestrian trajectory data set;
step S3: constructing a Conv-LSTM model of a deep convolution long-time memory network; the deep convolution long-time and short-time memory network model comprises a deep Conv-LSTM space-time memory layer, a global spatial feature plus deep layer and a fully-connected regression layer;
step S4: determining parameters of a deep Conv-LSTM model according to the training set and the test set;
step S5: and predicting the crowd track according to the deep Conv-LSTM model with the parameters.
2. The method for predicting the crowd trajectory based on the deep convolutional long and short memory network as claimed in claim 1, wherein the determining the parameters of the deep Conv-LSTM model according to the training set and the test set specifically comprises
Step S41: substituting part of labeled data in the training set into a deep Conv-LSTM model to determine a model loss value;
step S42: judging whether the magnitude of the model loss value is within a set range; if the magnitude of the model loss value is within the set range, substituting various scene data in the test set into a deep Conv-LSTM model, determining the average relative displacement error of the model, and executing step S43; if the magnitude of the model loss value is not in the set range, taking the current model loss value as an expected loss value, utilizing an Adam optimization algorithm to update parameters in a deep Conv-LSTM model according to the expected loss value, and then executing step S41;
step S43: judging whether the average relative displacement error of the model is less than or equal to a set value or not; if the average relative displacement error of the model is less than or equal to a set value, outputting parameters of the deep Conv-LSTM model; and if the model average relative displacement error is larger than the set value, increasing the layer number of the deep Conv-LSTM space-time memory layer in the deep Conv-LSTM model, and returning to the step S41.
3. The deep convolutional long and short memory network-based crowd trajectory prediction method according to claim 2, wherein the formula for determining the model average relative displacement error is as follows:
where n denotes the time of the predicted trajectory, Preal(t, i) and Preal(t + Δ t, i) represents the real displacement of the ith person in the scene at time t and time t + Δ t, respectively, Ppredicted(t, i) represents the displacement predicted by the model at time t for the ith pedestrian.
4. The method for predicting the pedestrian trajectory based on the deep convolutional long and short memory network as claimed in claim 1, wherein the determining the pedestrian trajectory data set specifically comprises:
a high-definition camera is adopted to record crowd evacuation and convection videos;
and tracking the track of each pedestrian in the video by using a video tracking algorithm, and determining a pedestrian track data set.
5. Crowd's orbit prediction system based on long and short memory network of deep convolution, its characterized in that, the system includes:
the pedestrian trajectory data set determining module is used for determining a pedestrian trajectory data set; the pedestrian trajectory data set comprises: coordinate information of destination, position information, speed information and quantity information of pedestrians in a surrounding set range, and position information of fixed obstacles;
the training set and test set determining module is used for determining a training set and a test set according to the pedestrian trajectory data set;
the model building module is used for building a Conv-LSTM model of the deep convolution time memory network; the deep convolution long-time and short-time memory network model comprises a deep Conv-LSTM space-time memory layer, a global spatial feature plus deep layer and a fully-connected regression layer;
a parameter determination module for determining parameters of the deep Conv-LSTM model according to the training set and the test set;
and the crowd track prediction module is used for predicting the crowd track according to the deep Conv-LSTM model with the parameters.
6. The system according to claim 5, wherein the parameter determination module specifically comprises
The model loss value determining unit is used for substituting part of labeled data in the training set into a deep Conv-LSTM model to determine a model loss value;
an judging unit, which is used for judging whether the magnitude of the model loss value is in a set range or not, if the magnitude of the model loss value is in the set range, substituting various scene data in the test set into a deep Conv-LSTM model, determining the average relative displacement error of the model, and executing a second judging unit, if the magnitude of the model loss value is not in the set range, taking the current model loss value as an expected loss value, updating the parameters in the deep Conv-LSTM model according to the expected loss value by using an Adam optimization algorithm, and then returning to the model loss value determining unit;
the second judging unit is used for judging whether the average relative displacement error of the model is less than or equal to a set value or not; if the average relative displacement error of the model is less than or equal to a set value, outputting parameters of the deep Conv-LSTM model; and if the average relative displacement error of the model is larger than a set value, increasing the layer number of the deep Conv-LSTM space-time memory layer in the deep Conv-LSTM model, and returning to the model loss value determining unit.
7. The deep convolutional long and short memory network-based crowd trajectory prediction system of claim 6, wherein the formula for determining the model average relative displacement error is as follows:
where n denotes the time of the predicted trajectory, Preal(t, i) and Preal(t + Δ t, i) represents the real displacement of the ith person in the scene at time t and time t + Δ t, respectively, Ppredicted(t, i) represents the displacement predicted by the model at time t for the ith pedestrian.
8. The system according to claim 5, wherein the pedestrian trajectory data set determining module specifically includes:
the video acquisition unit is used for recording crowd evacuation and convection videos by adopting a high-definition camera;
and the pedestrian track data set determining unit is used for tracking the track of each pedestrian in the video by utilizing a video tracking algorithm and determining a pedestrian track data set.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910857181.2A CN110737968B (en) | 2019-09-11 | 2019-09-11 | Crowd trajectory prediction method and system based on deep convolutional long and short memory network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910857181.2A CN110737968B (en) | 2019-09-11 | 2019-09-11 | Crowd trajectory prediction method and system based on deep convolutional long and short memory network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110737968A true CN110737968A (en) | 2020-01-31 |
| CN110737968B CN110737968B (en) | 2021-03-16 |
Family
ID=69267812
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910857181.2A Active CN110737968B (en) | 2019-09-11 | 2019-09-11 | Crowd trajectory prediction method and system based on deep convolutional long and short memory network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110737968B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111403021A (en) * | 2020-03-11 | 2020-07-10 | 中国电子工程设计院有限公司 | Monitoring method and device |
| CN111459168A (en) * | 2020-04-23 | 2020-07-28 | 上海交通大学 | Fused automatic-driving automobile pedestrian crossing track prediction method and system |
| CN111553232A (en) * | 2020-04-22 | 2020-08-18 | 陕西师范大学 | Gate loop unit network pedestrian trajectory prediction method based on scene state iteration |
| CN111639591A (en) * | 2020-05-28 | 2020-09-08 | 深圳地平线机器人科技有限公司 | Trajectory prediction model generation method and device, readable storage medium and electronic equipment |
| CN111767475A (en) * | 2020-03-20 | 2020-10-13 | 吉利汽车研究院（宁波）有限公司 | Target pedestrian trajectory prediction model training method and device, electronic equipment and storage medium |
| CN112100327A (en) * | 2020-08-31 | 2020-12-18 | 北京航空航天大学 | Dialog abstract generation method and system based on comprehensive feature extraction |
| CN112163705A (en) * | 2020-09-29 | 2021-01-01 | 重庆邮电大学 | Epidemic situation prediction and early warning method based on CNN + SEIR and LSTM |
| CN112417752A (en) * | 2020-10-29 | 2021-02-26 | 济南大学 | Cloud layer track prediction method and system based on convolution LSTM neural network |
| CN113408047A (en) * | 2021-08-05 | 2021-09-17 | 江苏大学 | Vehicle dynamics prediction model based on time-lag feedback neural network, training data acquisition method and training method |
| CN117252449A (en) * | 2023-11-20 | 2023-12-19 | 水润天府新材料有限公司 | Full-penetration drainage low-noise pavement construction process and system |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106682371A (en) * | 2017-03-06 | 2017-05-17 | 山东师范大学 | Method and system for people evacuation simulation under emergencies |
| CN108172025A (en) * | 2018-01-30 | 2018-06-15 | 东软集团股份有限公司 | A kind of auxiliary driving method, device, car-mounted terminal and vehicle |
| CN108428243A (en) * | 2018-03-07 | 2018-08-21 | 北京航空航天大学 | A kind of pedestrian movement's speed predicting method based on artificial neural network |
| CN108537818A (en) * | 2018-03-07 | 2018-09-14 | 上海交通大学 | Crowd's trajectory predictions method based on cluster pressure LSTM |
| CN108805080A (en) * | 2018-06-12 | 2018-11-13 | 上海交通大学 | Multi-level depth Recursive Networks group behavior recognition methods based on context |
| US20190050693A1 (en) * | 2017-08-14 | 2019-02-14 | Ehsan Taghavi | Generating labeled data for deep object tracking |
| CN109344725A (en) * | 2018-09-04 | 2019-02-15 | 上海交通大学 | A kind of online tracking of multirow people based on space-time attention rate mechanism |
| CN109635793A (en) * | 2019-01-31 | 2019-04-16 | 南京邮电大学 | A kind of unmanned pedestrian track prediction technique based on convolutional neural networks |
| CN109670270A (en) * | 2019-01-11 | 2019-04-23 | 山东师范大学 | Crowd evacuation emulation method and system based on the study of multiple agent deeply |
| CN109766790A (en) * | 2018-12-24 | 2019-05-17 | 重庆邮电大学 | A kind of pedestrian detection method based on self-adaptive features channel |
| CN110135314A (en) * | 2019-05-07 | 2019-08-16 | 电子科技大学 | A kind of multi-object tracking method based on depth Trajectory prediction |
-
2019
- 2019-09-11 CN CN201910857181.2A patent/CN110737968B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106682371A (en) * | 2017-03-06 | 2017-05-17 | 山东师范大学 | Method and system for people evacuation simulation under emergencies |
| US20190050693A1 (en) * | 2017-08-14 | 2019-02-14 | Ehsan Taghavi | Generating labeled data for deep object tracking |
| CN108172025A (en) * | 2018-01-30 | 2018-06-15 | 东软集团股份有限公司 | A kind of auxiliary driving method, device, car-mounted terminal and vehicle |
| CN108428243A (en) * | 2018-03-07 | 2018-08-21 | 北京航空航天大学 | A kind of pedestrian movement's speed predicting method based on artificial neural network |
| CN108537818A (en) * | 2018-03-07 | 2018-09-14 | 上海交通大学 | Crowd's trajectory predictions method based on cluster pressure LSTM |
| CN108805080A (en) * | 2018-06-12 | 2018-11-13 | 上海交通大学 | Multi-level depth Recursive Networks group behavior recognition methods based on context |
| CN109344725A (en) * | 2018-09-04 | 2019-02-15 | 上海交通大学 | A kind of online tracking of multirow people based on space-time attention rate mechanism |
| CN109766790A (en) * | 2018-12-24 | 2019-05-17 | 重庆邮电大学 | A kind of pedestrian detection method based on self-adaptive features channel |
| CN109670270A (en) * | 2019-01-11 | 2019-04-23 | 山东师范大学 | Crowd evacuation emulation method and system based on the study of multiple agent deeply |
| CN109635793A (en) * | 2019-01-31 | 2019-04-16 | 南京邮电大学 | A kind of unmanned pedestrian track prediction technique based on convolutional neural networks |
| CN110135314A (en) * | 2019-05-07 | 2019-08-16 | 电子科技大学 | A kind of multi-object tracking method based on depth Trajectory prediction |
Non-Patent Citations (3)
| Title |
|---|
| KAI CHEN等: "An Integrated Deep Learning Framework for Occluded Pedestrian Tracking", 《DIGITAL OBJECT IDENTIFIER》 * |
| THARINDU FERNANDO等: "Soft + Hardwired attention: An LSTM framework for human trajectory prediction and abnormal event detection", 《NEURAL NETWORKS》 * |
| 李昱科: "人群行为理解中的时空信息处理及研究", 《中国博士学位论文全文数据库 信息科技辑》 * |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111403021B (en) * | 2020-03-11 | 2023-12-05 | 中国电子工程设计院有限公司 | Monitoring method and device |
| CN111403021A (en) * | 2020-03-11 | 2020-07-10 | 中国电子工程设计院有限公司 | Monitoring method and device |
| CN111767475A (en) * | 2020-03-20 | 2020-10-13 | 吉利汽车研究院（宁波）有限公司 | Target pedestrian trajectory prediction model training method and device, electronic equipment and storage medium |
| CN111553232A (en) * | 2020-04-22 | 2020-08-18 | 陕西师范大学 | Gate loop unit network pedestrian trajectory prediction method based on scene state iteration |
| CN111459168B (en) * | 2020-04-23 | 2021-12-10 | 上海交通大学 | Fused automatic-driving automobile pedestrian crossing track prediction method and system |
| CN111459168A (en) * | 2020-04-23 | 2020-07-28 | 上海交通大学 | Fused automatic-driving automobile pedestrian crossing track prediction method and system |
| CN111639591B (en) * | 2020-05-28 | 2023-06-30 | 深圳地平线机器人科技有限公司 | Track prediction model generation method and device, readable storage medium and electronic equipment |
| CN111639591A (en) * | 2020-05-28 | 2020-09-08 | 深圳地平线机器人科技有限公司 | Trajectory prediction model generation method and device, readable storage medium and electronic equipment |
| CN112100327B (en) * | 2020-08-31 | 2022-05-31 | 北京航空航天大学 | Dialog abstract generation method and system based on comprehensive feature extraction |
| CN112100327A (en) * | 2020-08-31 | 2020-12-18 | 北京航空航天大学 | Dialog abstract generation method and system based on comprehensive feature extraction |
| CN112163705A (en) * | 2020-09-29 | 2021-01-01 | 重庆邮电大学 | Epidemic situation prediction and early warning method based on CNN + SEIR and LSTM |
| CN112163705B (en) * | 2020-09-29 | 2022-07-22 | 重庆邮电大学 | Epidemic situation prediction and early warning method based on CNN + SEIR and LSTM |
| CN112417752A (en) * | 2020-10-29 | 2021-02-26 | 济南大学 | Cloud layer track prediction method and system based on convolution LSTM neural network |
| CN113408047A (en) * | 2021-08-05 | 2021-09-17 | 江苏大学 | Vehicle dynamics prediction model based on time-lag feedback neural network, training data acquisition method and training method |
| CN113408047B (en) * | 2021-08-05 | 2024-03-22 | 江苏大学 | Vehicle dynamics prediction model based on time-lag feedback neural network, training data acquisition method and training method |
| CN117252449A (en) * | 2023-11-20 | 2023-12-19 | 水润天府新材料有限公司 | Full-penetration drainage low-noise pavement construction process and system |
| CN117252449B (en) * | 2023-11-20 | 2024-01-30 | 水润天府新材料有限公司 | Full-penetration drainage low-noise pavement construction process and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110737968B (en) | 2021-03-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110737968B (en) | Crowd trajectory prediction method and system based on deep convolutional long and short memory network | |
| Song et al. | Pedestrian trajectory prediction based on deep convolutional LSTM network | |
| Rehder et al. | Pedestrian prediction by planning using deep neural networks | |
| CN111061277B (en) | Unmanned vehicle global path planning method and device | |
| Lei et al. | Dynamic path planning of unknown environment based on deep reinforcement learning | |
| Ivanovic et al. | Generative modeling of multimodal multi-human behavior | |
| Fernando et al. | Deep inverse reinforcement learning for behavior prediction in autonomous driving: Accurate forecasts of vehicle motion | |
| CN113313947B (en) | Road condition evaluation method of short-term traffic prediction graph convolution network | |
| JP2022547611A (en) | Simulation of various long-term future trajectories in road scenes | |
| Wei et al. | Learning motion rules from real data: Neural network for crowd simulation | |
| CN113515131B (en) | Mobile robot obstacle avoidance method and system based on condition variation automatic encoder | |
| CN114155270A (en) | Pedestrian trajectory prediction method, device, equipment and storage medium | |
| Bordallo et al. | Counterfactual reasoning about intent for interactive navigation in dynamic environments | |
| Bahamid et al. | A review on crowd analysis of evacuation and abnormality detection based on machine learning systems | |
| CN113505924A (en) | Information propagation prediction method and system based on cascade spatiotemporal features | |
| CN110956684B (en) | Crowd movement evacuation simulation method and system based on residual error network | |
| Chen et al. | Pedestrian behavior prediction model with a convolutional LSTM encoder–decoder | |
| Zhang et al. | Direction-decision learning based pedestrian flow behavior investigation | |
| Hossain et al. | SFMGNet: A physics-based neural network to predict pedestrian trajectories | |
| Turkay et al. | Integrating information theory in agent-based crowd simulation behavior models | |
| Yang et al. | IST-PTEPN: an improved pedestrian trajectory and endpoint prediction network based on spatio-temporal information | |
| CN116562449A (en) | Pedestrian flow prediction method and system based on convolution of knowledge graph and space-time graph | |
| Sighencea et al. | Pedestrian Trajectory Prediction in Graph Representation Using Convolutional Neural Networks | |
| CN113869170B (en) | Pedestrian track prediction method based on graph division convolutional neural network | |
| Wang et al. | Path planning model of mobile robots in the context of crowds |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |