CN116578661A - Vehicle track time-space reconstruction method and system based on attention mechanism - Google Patents
Vehicle track time-space reconstruction method and system based on attention mechanism Download PDFInfo
- Publication number
- CN116578661A CN116578661A CN202310578357.7A CN202310578357A CN116578661A CN 116578661 A CN116578661 A CN 116578661A CN 202310578357 A CN202310578357 A CN 202310578357A CN 116578661 A CN116578661 A CN 116578661A
- Authority
- CN
- China
- Prior art keywords
- track
- time
- space
- tracks
- historical
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 230000007246 mechanism Effects 0.000 title claims abstract description 41
- 238000012549 training Methods 0.000 claims abstract description 10
- 230000014509 gene expression Effects 0.000 claims abstract description 9
- 238000012360 testing method Methods 0.000 claims abstract description 4
- 239000010410 layer Substances 0.000 claims description 68
- 230000002776 aggregation Effects 0.000 claims description 19
- 238000004220 aggregation Methods 0.000 claims description 19
- 238000012545 processing Methods 0.000 claims description 16
- 238000010606 normalization Methods 0.000 claims description 15
- 230000004931 aggregating effect Effects 0.000 claims description 13
- 238000013528 artificial neural network Methods 0.000 claims description 10
- 230000000694 effects Effects 0.000 claims description 9
- 238000013507 mapping Methods 0.000 claims description 7
- 230000015654 memory Effects 0.000 claims description 5
- 230000007787 long-term memory Effects 0.000 claims description 4
- 230000006403 short-term memory Effects 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 239000002355 dual-layer Substances 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 210000002569 neuron Anatomy 0.000 claims description 3
- 230000008569 process Effects 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 230000008901 benefit Effects 0.000 description 8
- 230000029305 taxis Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 5
- 238000012544 monitoring process Methods 0.000 description 5
- 230000000306 recurrent effect Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000000903 blocking effect Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000007774 longterm Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000002123 temporal effect Effects 0.000 description 2
- DLMYFMLKORXJPO-UHFFFAOYSA-N 2-amino-3-[(triphenylmethyl)thio]propanoic acid Chemical compound C=1C=CC=CC=1C(C=1C=CC=CC=1)(SCC(N)C(O)=O)C1=CC=CC=C1 DLMYFMLKORXJPO-UHFFFAOYSA-N 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012886 linear function Methods 0.000 description 1
- 230000033001 locomotion Effects 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/29—Geographical information databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
- G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Databases & Information Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Remote Sensing (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to a vehicle track space-time reconstruction method and system based on an attention mechanism, and belongs to the technical field of space-time reconstruction of automatic vehicle identification tracks. The method comprises the following steps: s1, aiming at a space track of a current track, carrying out track embedding expression on the space track; s2, embedding and expressing various historical tracks; s3, predicting a time sequence corresponding to the current track by adopting a time sequence prediction model based on an attention mechanism, and combining the time sequence into a space-time track; and S4, training the whole model by using real data, reconstructing the track in the test set, and finally evaluating and analyzing the prediction error according to the prediction result and the actual data. Compared with the prior art, the technical scheme has higher accuracy and efficiency, and has wide application prospect.
Description
Technical Field
The invention belongs to the technical field of space-time reconstruction of automatic vehicle identification tracks, and relates to a vehicle track space-time reconstruction method and system based on an attention mechanism.
Background
Automatic vehicle identification technology (Automatic Vehicle Identification, AVI) is widely used in urban traffic monitoring, and accurate and comprehensive trajectory data greatly facilitates the development and application of intelligent traffic systems. The biggest advantage of AVI data over traditional trajectory data is that it can cover almost all travel vehicles in urban traffic, while it also has faster recognition speed and more accurate recognition capability. However, due to the constraint of economy and environmental cost, the distribution of AVI monitoring points in a city is quite sparse and uneven, so that the space-time positions in the AVI track are sparse, and great uncertainty is brought to the interior of the track, and the use value of the AVI track is greatly damaged by the uncertainty, so that the reconstruction of the sparse AVI track into a track with fine granularity becomes a key problem of AVI track mining, and has important theoretical and practical significance.
Automated vehicle identification data has many advantages over conventional GPS tracks. First, the accuracy of the automatic identification data of the vehicle is high, and the accuracy is high, and the method comprises two aspects, namely, the uncertainty on the position point in the track is eliminated because the identification data of the vehicle is collected at a specific position. Secondly, the vehicle can be accurately identified and the passing time of the vehicle can be recorded, and particularly, the automatic vehicle identification system based on the radio frequency identification technology (RFID) can achieve almost 100% of identification accuracy for the vehicle. Secondly, the automatic vehicle identification data has a large data volume, which is shown in three aspects, firstly, the automatic vehicle identification system can identify all types of vehicles in the city, including private cars, taxis, network-bound cars, trucks and the like, and the automatic vehicle identification system has the greatest advantage and can cover almost all traveling vehicles. Secondly, the vehicle recognition technology can be applied to various scenes, such as toll stations, urban roads and the like. Thirdly, the information is rich. The vehicle identification data can acquire more information such as a vehicle ID, a vehicle type, a vehicle use, and the like. The data volume of the automatic vehicle identification system is far beyond the scenes and vehicles covered by the GPS.
However, the automatic vehicle identification data has some disadvantages, the biggest point is that the sampling rate of the automatic vehicle identification data is very low, because the automatic vehicle identification equipment cannot record the vehicle position in real time, it can only record the vehicle information when the vehicle passes through the acquisition points, and the acquisition points in the urban road network are often arranged sparsely due to the limitation of environment, economy and the like, the distance between two adjacent acquisition points is hundreds of meters and kilometers, and the vehicle track based on the automatic vehicle identification data is very sparse. However, the existing researches tell us that the sparse vehicle track can cause the detail in the track to be lost and cause serious influence on the subsequent research and application, so in order to fully play the advantages and roles of the automatic vehicle identification track in the construction of the intelligent traffic system, the sparse automatic vehicle identification track must be reconstructed.
Disclosure of Invention
In view of the above, the present invention aims to provide a vehicle track space-time reconstruction method and system based on an attention mechanism, so as to accurately reconstruct a space-time track of a vehicle.
In order to achieve the above purpose, the present invention provides the following technical solutions:
a vehicle track space-time reconstruction method based on an attention mechanism, the method comprising the steps of: s1, aiming at a space track of a current track, carrying out track embedding expression on the space track; s2, embedding and expressing various historical tracks; s3, predicting a time sequence corresponding to the current track by adopting a time sequence prediction model based on an attention mechanism, and combining the time sequence into a space-time track; and S4, training the whole model by using real data, reconstructing the track in the test set, and finally evaluating and analyzing the prediction error according to the prediction result and the actual data.
Further, in step S1, the spatial track of the current track is expressed by track embedding, which specifically includes: the processing of the current track is divided into two parts, namely, space reconstruction is carried out on the current track and track embedding is carried out on the reconstructed space track; for one travel on adjacent pairs of points (C A ,C B ) The locus [ (as) A ,at A ),(as B ,as B )]According to itThe spatial trajectory is represented as an m-dimensional vector s= [ S ] as a result of spatial trajectory reconstruction 1 ,s 2 ,…,s m ]Wherein s is i Representing an ith spatial point in the spatial trajectory;
for the current trajectory, a time state vector S is used t To its travel total time feature, the travel total time (at A -at B ) Expanded to an m-dimensional vector (of the same dimension as the space vector S),S t the calculation method of (2) is as follows:
S t ＝(at B -at A )×S
to represent the effect of vehicle departure time on time-space relationship, a period vector is used to represent vehicle driving period, the time of day is divided into C time slices, and then C is used S To represent the departure time at of the journey A At a time-slice position during the day, the cycle vector for that trip may be represented as S C And can be found by the following formula:
according to the above-mentioned current track embedding manner, the embedded current track C may be expressed as:
C＝Concat(S,S t ,S c )。
further, in step S2, embedding expressions are performed on a plurality of history tracks, for a given pair of adjacent acquisition point pairs (C A ,C B ) A determined path between them, the historical track set of which is denoted as T H ＝{Tr 1 ,Tr 2 ,…,Tr N One of the tracks Tr is considered i The density of the space-time positions in the method is far lower than that of the path space points, and the track aggregation method is used for aggregating all the historical tracks into a complete space-time track, and the method specifically comprises the following steps:
s21, historical track aggregation:
by separating the spatio-temporal positions in the track by time slices, then taking the position of the highest probability within each time slice as the position of the aggregated track on that time slice. The method is similar to the method, and the difference is that the method divides the space position in the path according to the space slice instead of the time slice, then finds all the time points on the time slice for a determined space slice, and uses the average time of the track points as the time of the track after aggregation on the space slice; thus, for the historical track set T H Polymerized trajectoriesThe method can be expressed as follows:
wherein Tr is i Representing a historical track set T H In the above, N represents the number of tracks in the track set, and the No. represents the track aggregation operation, specifically, for a fixed-position point, find all the times of passing through the position, and use the average value of the times of passing through the position as the time of the point; after the time of aggregation of each position point, sequencing the position points according to the position sequence to obtain an aggregated track;
s22, aggregating historical tracks in the same period:
the definition of the history trace for the same period is as follows: given a pair of adjacent points (C A ,C B ) And a set of historical trajectories T within a path therebetween H ＝{Tr 1 ,Tr 2 ,…,Tr N For a track [ (as) to be reconstructed ] A ,at A ),(as B ,at B )]Find all the departure times at from the track A Historical tracks of the same period, the set of which, i.e. the historical tracks of the same period, is denoted as T C ＝{Tr 1 ,Tr 2 ,…,Tr M Where m represents the number of tracks in the set, tr i Representing one track in the set, and aggregating the history tracks in the same period, wherein the operation formula is as follows:
s23, aggregation of historical tracks in the same state:
the definition of the history trace for the same state is as follows: given a pair of adjacent points (C A ,C B ) And a set of historical trajectories T within a path therebetween H ＝{Tr 1 ,Tr 2 ,…,Tr N -a }; for a track [ (as) to be reconstructed A ,at A ),(as B ,at B )]First according to the travel time (at B -at A ) Calculate the traffic state of the journey and then for T H Calculating the traffic state of each track according to the travel time of the track, and representing a track set with the same state as the track to be reconstructed as T S ＝{Tr 1 ,Tr 2 ,…,Tr O Where O represents the number of tracks in the set, tr i Representing one track in the set, aggregating the history tracks in the same state, wherein the track aggregation represented by the following formula is shown as follows:
s24, fusing historical tracks:
for the three aggregated historical tracks, in order to enable the model to capture the space-time relationship inside the tracks and the space-time relationship between the tracks, a splicing operation is used for fusing the tracks and inputting H as the historical tracks of the model, and the specific operation is as follows:
as can be seen from the above three types of history tracks, the history track set includes a history track set with the same period and a history track set with the same state, and the history track set is separately extracted to strengthen the weights of the two tracks in the neural network, because the track to be reconstructed may have higher similarity with the two tracks.
Further, in step S3, in order to reconstruct the time sequence of the current spatial trajectory, considering the space-time relationship in the learned trajectory from the historical trajectory, that is, the input of the model is the historical space-time trajectory and the current spatial trajectory, and the output is the time sequence corresponding to the current spatial trajectory; aiming at the time sequence generating task, adopting an encoder-decoder architecture, wherein the encoder is responsible for processing the historical space-time track, and the model encodes the historical space-time track by using a multi-head self-attention mechanism to extract the space-time relation in the historical track; in the decoder, the spatial track is input, the spatial features are still extracted by using a multi-head self-attention mechanism, and a cross attention (cross attention) joint encoder and decoder are used in the decoder, so that the space-time relationship in the track is learned from the output of the encoder; a long-short-term memory network is used at the end of the model to capture the time relationship between tracks, and finally the required time sequence is output.
Further, the model is divided into two parts, namely an encoder and a decoder, the encoder is responsible for encoding the historical track, and the decoder is responsible for processing the current track and generating a time sequence by combining the output of the encoder, and the method specifically comprises the following steps:
an encoder: the input of the encoder is a historical track H, and in order for the encoder to better extract the characteristics in the historical track, a linear layer is used for embedding and expressing the historical track, and the historical track is mapped into a higher-dimensional hidden space:
H h ＝Linear(H)
wherein H is h Output as a linear layer; in a spatio-temporal track, its spatio-temporal relationship is related not only to the state of the track but also to the time period in which it is located, and its spatio-temporal relationship is highly related to the total time it takes to traverse the path, therefore, a multi-headed attentiveness mechanism is usedCapturing the time-space relation, firstly inputting the sequence H h Mapping to Q h ,K h And V h Three subspaces:
Q H ＝W Q H h
K H ＝W K H h
V H ＝W V H h
wherein W is Q 、W K 、W V Weight matrices of Q, K, V, respectively, i.e. H h Mapping to three hidden spaces Q, K and V; in this regard, the multi-headed self-attention operation is defined as:
MultiHead(Q,K,V)＝[Head 1 ,Head 2 ,...,Head h ]W O
Head i ＝Attention(QW i Q ,KW i K ,VW i V )
where h is the number of attention heads, W O Is the final output weight matrix; after multi-headed self-attention, the model then uses the residual connection and layer normalization layer, specifically formulated as follows:
LayerNorm(X+MultiHeadAttention(X))
wherein residual connection refers to x+ MultiHeadAttention (X), X refers to multi-headed self-attention input; layerNorm refers to layer normalization, which converts the mean and variance of the input of neurons of each layer into unity, the multi-head self-attention and residual connection & layer normalization module in the encoder part is repeated L times, the output of the upper layer is used as the input of the next layer until the output of the last layer of the encoder, and the output of the last layer is mapped to K, V hidden spaces by using an attention formula for the decoder to inquire;
a decoder: the input of the decoder is a current track C, and the current track C comprises space information, state information, cycle information and travel time information of the current track; in order to make it easier for the decoder to extract the information therein, the current track C is embedded and expressed using a linear layer, so as to map it into a higher-dimensional hidden space, which is specifically as follows:
C h ＝Linear(C)
C h output as a linear layer; after embedding the track into the representation, a dual-layer self-attention mechanism is used in the decoder to capture the spatio-temporal relationship within and between tracks, self-attention module and residual connection at the first layer&In layer normalization, the processing of the spatial trajectories is consistent with the self-attention mechanism in the encoder;
in the second-layer multi-head self-attention module, a cross attention mechanism (cross attention) is used in order to let the output in the encoder, depending on the input in the decoder, ultimately output different values. In the layer, the final output of the encoder is mapped into two hidden spaces K and V, the output of the upper layer of the decoder is mapped into a hidden space Q, and multi-head self-attention operation is carried out on the mapped Q, K and V; then aiming at the output of the cross attention mechanism, in order to solve the problems in model training and make the model converge faster, a residual error connection & layer normalization is still used for processing the output of the cross attention mechanism; meanwhile, the encoder also uses a multi-layer structure, with the output of the upper layer decoder as the input of the lower layer decoder, up to the output of the last layer.
Finally, in order to enable the model to capture the time-space relationship between tracks, a long-short-term memory network is used to process the output of the decoder, and before that, in order to enable the LSTM to capture the time relationship between tracks better, the output of the decoder is combined with, i.e. connected to, the original spatial tracks. The output of the final LSTM is the required time sequence:
T＝LSTM(Concat(O dec ,C))
wherein T represents the final generated time series, O dec Representing the output of the decoder, C representing the embedded spatial trajectory;
after the time sequence is generated, the time sequence and the unembossed space track are spliced together according to the position, so that the reconstructed space-time track can be obtained.
The invention has the beneficial effects that:
compared with the prior art, the technical scheme of the invention has higher accuracy and efficiency, and has wide application prospect.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objects and other advantages of the invention may be realized and obtained by means of the instrumentalities and combinations particularly pointed out in the specification.
Drawings
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in the following preferred detail with reference to the accompanying drawings, in which:
FIG. 1 is a model diagram of the present invention;
FIG. 2 is a block diagram of a time series prediction module according to the present invention;
FIG. 3 is a graph showing the experimental effect of the present invention.
Detailed Description
The technical scheme of the invention is described in detail below with reference to the accompanying drawings.
Fig. 1 is a model diagram of the present invention, and as shown in the figure, the method for reconstructing a vehicle track space-time based on an attention mechanism provided by the present invention mainly includes the following steps: s1, aiming at a space track of a current track, carrying out track embedding expression on the space track; s2, embedding and expressing various historical tracks; s3, predicting a time sequence corresponding to the current track by adopting a time sequence prediction model based on an attention mechanism, and combining the time sequence into a space-time track; and S4, training the whole model by using real data, reconstructing the track in the test set, and finally evaluating and analyzing the prediction error according to the prediction result and the actual data.
In this embodiment, step S1 performs track embedding expression on the spatial track of the current track. In the model of the invention, the processing of the current track is very important, because the original current track only comprises the space-time position information of the starting point and the ending point, the information of countless points in the middle of the track is difficult to generate by means of the information of the two points, and therefore, the method of reconstructing the space track firstly and then reconstructing the space-time track according to the space track is adopted, and therefore, the processing of the current track is divided into two parts, namely, the space reconstruction of the current track and the track embedding of the reconstructed space track are carried out; for one travel on adjacent pairs of points (C A ,C B ) The locus [ (as) A ,at A ),(as B ,at B )]According to the space track reconstruction result, the space track is expressed as an m-dimensional vector S= [ S ] 1 ,s 2 ,…,s m ]Wherein s is i Representing an ith spatial point in the spatial trajectory;
for the current trajectory, a time state vector S is used t To its travel total time feature, the travel total time (at A -at B ) Expanded to an m-dimensional vector (of the same dimension as the space vector S),S t the calculation method of (2) is as follows:
S t ＝(at B -at A )×S
to represent the effect of vehicle departure time on time-space relationship, a period vector is used to represent vehicle driving period, the time of day is divided into C time slices, and then C is used S To represent the departure time at of the journey A At a time-slice position during the day, the cycle vector for that trip may be represented as S C And can be found by the following formula:
according to the above-mentioned current track embedding manner, the embedded current track C may be expressed as:
C＝Concat(S,S t ,S c )。
in step S2, embedding expressions are performed on the plurality of history trajectories, and for a given pair of adjacent acquisition point pairs (C A ,C B ) A determined path between them, the historical track set of which is denoted as T H ＝{Tr 1 ,Tr 2 ,…,Tr N One of the tracks Tr is considered i The density of the space-time positions in the method is far lower than that of the path space points, and the track aggregation method is used for aggregating all the historical tracks into a complete space-time track, and the method specifically comprises the following steps:
s21, historical track aggregation:
by separating the spatio-temporal positions in the track by time slices, then taking the position of the highest probability within each time slice as the position of the aggregated track on that time slice. The method is similar to the method, and the difference is that the method divides the space position in the path according to the space slice instead of the time slice, then finds all the time points on the time slice for a determined space slice, and uses the average time of the track points as the time of the track after aggregation on the space slice; thus, for the historical track set T H Polymerized trajectoriesThe method can be expressed as follows:
wherein Tr is i Representing a historical track set T H In the above, N represents the number of tracks in the track set, and the No. represents the track aggregation operation, specifically, for a fixed-position point, find all the times of passing through the position, and use the average value of the times of passing through the position as the time of the point; after aggregating time for each position point, the position points are sequenced according to the positionSequencing to obtain the aggregated track;
s22, aggregating historical tracks in the same period:
taking the influence of the periodicity of the tracks on the path on the motion law of the vehicle into consideration, extracting the historical tracks with the same period from the historical tracks, and taking the aggregated tracks as the input of a model; the definition of the history trace for the same period is as follows: given a pair of adjacent points (C A ,C B ) And a set of historical trajectories T within a path therebetween H ＝{Tr 1 ,Tr 2 ,…,Tr N For a track [ (as) to be reconstructed ] A ,at A ),(as B ,at B )]Find all the departure times at from the track A Historical tracks of the same period, the set of which, i.e. the historical tracks of the same period, is denoted as T C ＝{Tr 1 ,Tr 2 ,…,Tr M Where m represents the number of tracks in the set, tr i Representing one track in the set, and aggregating the history tracks in the same period, wherein the operation formula is as follows:
s23, aggregation of historical tracks in the same state:
considering that the rule of the vehicle when moving on the road is influenced by the passing state (quick passing/blocking), the passing state of the vehicle can be calculated according to the time of the vehicle moving between two adjacent points, namely blocking or quick passing, and when the passing time of the vehicle is more than 1.5 times of the average time, the travel is considered to be blocked; the definition of the history trace for the same state is as follows: given a pair of adjacent points (C A ,C B ) And a set of historical trajectories T within a path therebetween H ＝{Tr 1 ,Tr 2 ,…,Tr N -a }; for a track [ (as) to be reconstructed A ,at A ),(as B ,at B )]First according to the travel time (at B -at A ) Calculate the traffic state of the journey and then for T H All of (3)The track, according to its travel time, calculates the traffic state of each track, and represents the track set of all the same states as the track to be reconstructed as T S ＝{Tr 1 ,Tr 2 ,…,Tr O Where O represents the number of tracks in the set, tr i Representing one track in the set, aggregating the history tracks in the same state, wherein the track aggregation represented by the following formula is shown as follows:
s24, fusing historical tracks:
for the three aggregated historical tracks, in order to enable the model to capture the space-time relationship inside the tracks and the space-time relationship between the tracks, a splicing operation is used for fusing the tracks and inputting H as the historical tracks of the model, and the specific operation is as follows:
as can be seen from the above three types of history tracks, the history track set includes a history track set with the same period and a history track set with the same state, and the history track set is separately extracted to strengthen the weights of the two tracks in the neural network, because the track to be reconstructed may have higher similarity with the two tracks.
In step S3, the time sequence generating module in the model is a sequence-to-sequence mode, i.e. the input is one sequence and the output is another sequence; in the model, in order to reconstruct the time sequence of the current space track, the time-space relation in the track is learned from the history track, namely, the input of the model is the history time-space track and the current space track, and the output is the time sequence corresponding to the current space track; aiming at the time sequence generating task, adopting an encoder-decoder architecture, wherein the encoder is responsible for processing the historical space-time track, and the model encodes the historical space-time track by using a multi-head self-attention mechanism to extract the space-time relation in the historical track; in the decoder, the spatial track is input, the spatial features are still extracted by using a multi-head self-attention mechanism, and a cross attention (cross attention) joint encoder and decoder are used in the decoder, so that the space-time relationship in the track is learned from the output of the encoder; a long-short-term memory network is used at the end of the model to capture the time relationship between tracks, and finally the required time sequence is output. Fig. 2 is a block diagram of a time series prediction according to the present invention.
The model is divided into an encoder and a decoder, wherein the encoder is responsible for encoding a historical track, and the decoder is responsible for processing a current track and generating a time sequence by combining the output of the encoder, and the model specifically comprises the following steps:
an encoder: the input of the encoder is a historical track H, and in order for the encoder to better extract the characteristics in the historical track, a linear layer is used for embedding and expressing the historical track, and the historical track is mapped into a higher-dimensional hidden space:
H h ＝Linear(H)
wherein H is h Output as a linear layer; in a spatio-temporal track, its spatio-temporal relationship is related not only to the state of the track but also to the time period in which it is located and its spatio-temporal relationship is highly related to the total time it takes to traverse the path, therefore, a multi-head attention mechanism is used to capture its spatio-temporal relationship, first the input sequence H is entered h Mapping to Q h ,K h And V h Three subspaces:
Q H ＝W Q H h
K H ＝W K H h
V H ＝W V H h
wherein W is Q 、W K 、W V Weight matrices of Q, K, V, respectively, i.e. H h Mapping to three hidden spaces Q, K and V; in this regard, the multi-headed self-attention operation is defined as:
MultiHead(Q,K,V)＝[Head 1 ,Head 2 ,...,Head h ]W O
Head i ＝Attention(QW i Q ,KW i K ,VW i V )
where h is the number of attention heads, W O Is the final output weight matrix; the plurality of attention heads create a plurality of subspaces, input data are mapped onto the subspaces, and then characterization information of the subspaces is concerned jointly, so that space-time relation information in the track is effectively captured;
after multi-headed self-attention, the model then uses residual connection and layer normalization layers, with the aim of solving the multi-layer network training problem and speeding up convergence. The specific formula is as follows:
LayerNorm(X+MultiHeadAttention(X))
wherein residual connection refers to x+ MultiHeadAttention (X), X refers to multi-headed self-attention input; residual connection is often used in residual networks, which allows the network to focus on only the current difference part, usually for solving the problem of multi-layer network training, where this layer is used to solve the problem of model training; layerNorm refers to layer normalization, which transforms the mean and variance of the inputs of neurons of each layer to unity, which can speed up convergence; the multi-head self-attention and residual connection & layer normalization module in the encoder part is repeated for L times, the output of the upper layer is used as the input of the lower layer until the output of the last layer of the encoder, and then the attention formula is used for mapping the output of the last layer of the encoder to K, V hidden spaces for the decoder to inquire;
a decoder: the input of the decoder is a current track C, and the current track C comprises space information, state information, cycle information and travel time information of the current track; in order to make it easier for the decoder to extract the information therein, the current track C is embedded and expressed using a linear layer, so as to map it into a higher-dimensional hidden space, which is specifically as follows:
C h ＝Linear(C)
C h output as a linear layer; after embedding the track into the representation, a dual-layer self-attention mechanism is used in the decoder to capture the spatio-temporal relationship within and between tracks, self-attention module and residual connection at the first layer&In layer normalization, the processing of the spatial trajectories is consistent with the self-attention mechanism in the encoder;
in the second-layer multi-head self-attention module, a cross attention mechanism (cross attention) is used in order to let the output in the encoder, depending on the input in the decoder, ultimately output different values. In the layer, the final output of the encoder is mapped into two hidden spaces K and V, the output of the upper layer of the decoder is mapped into a hidden space Q, and multi-head self-attention operation is carried out on the mapped Q, K and V; then aiming at the output of the cross attention mechanism, in order to solve the problems in model training and make the model converge faster, a residual error connection & layer normalization is still used for processing the output of the cross attention mechanism; meanwhile, the encoder also uses a multi-layer structure, with the output of the upper layer decoder as the input of the lower layer decoder, up to the output of the last layer.
Finally, in order to enable the model to capture the time-space relationship between tracks, a long-short-term memory network is used to process the output of the decoder, and before that, in order to enable the LSTM to capture the time relationship between tracks better, the output of the decoder is combined with, i.e. connected to, the original spatial tracks. The output of the final LSTM is the required time sequence:
T＝LSTM(Concat(O dec ,C))
wherein T represents the final generated time series, O dec Representing the output of the decoder, C representing the embedded spatial trajectory;
after the time sequence is generated, the time sequence and the unembossed space track are spliced together according to the position, so that the reconstructed space-time track can be obtained.
Experimental results in this example:
experimental data set: the real automatic vehicle identification data set of Chongqing city is used, the data set comprises 330 RFID monitoring points in Chongqing city main urban area, the monitoring points are distributed unevenly in space, the adjacent relation can be known from the track data of the monitoring points, the distance between two adjacent detection points is hundreds of meters short, kilometers long, the road structure is complex, and the track uncertainty between the adjacent points is large. Another data set contains GPS data of more than 15000 taxis in Chongqing city for one month in 10 months (2021.10.1-2021.10.31) of 2021, the GPS device of taxis in Chongqing city records the position of taxis every 10-15 seconds, and the total number of taxis in one month is about 15 hundred million, and the total number of taxis in the one month is about 1200 ten thousand passenger routes after the route of taxis is extracted for training a model.
Evaluation index:
space-time linear combining distance (Spatiotemporal Linear Combine Distance, STLC). The thinking of space-time linear combination distance is that one track is divided into a time sequence and a space sequence, the distance between the time sequences and the distance between the space sequences are respectively considered when the distance is calculated with the other track, and then the time distance and the space distance are combined according to a certain weight value to be used as the overall track distance.
The space-time longest common subsequence (Spatiotemporal Longest Common Sub-Sequence, STLCSS). Similar to the longest common subsequence (LCSS), however in the spatio-temporal longest common subsequence, a time factor is additionally considered. Only one space error range is originally set, and after space factors are considered, the time error range of the points is additionally considered, and the points can be considered to be matched only by simultaneous matching of the time error and the space error
Experimental results:
fig. 3 is an experimental effect diagram of the present invention, because of the extremely sparse characteristic of the AVI track, in the model of the present invention, the space-time reconstruction of the track is divided into two steps, the spatial features of the track are reconstructed first, and then the time attribute of the track is reconstructed according to the spatial features of the track, so in the comparative experiment, the method is also divided into two parts, namely, the spatial reconstruction method of the track and the time prediction method of the track. The specific method is as follows:
the space reconstruction method mainly comprises the following three steps: RICK, MPR and HMM.
The temporal prediction method mainly comprises the following steps, wherein History and Linear belong to the traditional statistical learning-based method, and RNN and LSTM belong to the deep learning method.
Linear model (Linear): the linear model is a simple statistical-based learning method, and a linear function is used to fit the relationship between space and time in the track, i.e. for each spatial point, a linear relationship between time and space is considered to exist, the linear relationship is learned from historical data, and the time of each spatial point is predicted according to the linear relationship.
Historical highest frequency access (History): historical highest frequency access is a simple prediction method based on historical data, namely, for any fixed space point, a time point of accessing the highest frequency of the space point is used as the time of passing through the space point.
Recurrent Neural Network (RNN): the circulating neural network uses a neural network to model the space-time relationship in the same track, namely, a space sequence of the track is input, a time sequence of the track is output, and the space-time relationship between different positions in the track can be effectively extracted by using the circulating neural network to model the track.
Long and short term memory network (LSTM): long and short term memory networks are a variant of recurrent neural networks, but unlike recurrent neural networks, long and short term memory networks have the advantage of effectively preserving long-term dependencies in track sequences, which are better than recurrent neural networks when the sequences are longer.
TABLE 1
As a result of further analysis of the experimental results shown in table 1, it was found that when the influence of different spatial reconstruction methods on temporal spatial reconstruction was analyzed using the fixed time prediction method, respectively, it was found from the experimental results that the RICK method performed better in the experiment than MPR and HMM, and MPR was slightly better than HMM. Compared with the traditional methods, the method has better effect and no small improvement. When the fixed space reconstruction method is used for reconstructing the track by using different time prediction methods respectively, the effect of the History method is slightly better than that of the Linear method, on the basis, the RNN and LSTM of the method based on deep learning and considering the space-time law in the track are better than those of the History and the Linear method, and the LSTM can capture the long-term dependence property in the track, and the experimental effect of the LSTM is slightly better than that of the RNN. The time prediction method provided by the invention not only can consider the space-time dependence in the track, but also can learn the space-time relationship from the track of the historical track, so that the experimental effect of the method is optimal.
Finally, it is noted that the above embodiments are only for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention may be modified without departing from the spirit and scope of the technical solution, and all such modifications are included in the scope of the claims of the present invention.
Claims (6)
1. A vehicle track time-space reconstruction method based on an attention mechanism is characterized by comprising the following steps of: the method comprises the following steps:
s1, aiming at a space track of a current track, carrying out track embedding expression on the space track;
s2, embedding and expressing various historical tracks;
s3, predicting a time sequence corresponding to the current track by adopting a time sequence prediction model based on an attention mechanism, and combining the time sequence into a space-time track;
and S4, training the whole model by using real data, reconstructing the track in the test set, and finally evaluating and analyzing the prediction error according to the prediction result and the actual data.
2. A vehicle track space-time reconstruction method based on an attention mechanism as claimed in claim 1, wherein: in step S1, the track embedding expression is performed on the spatial track of the current track, which specifically includes:
the processing of the current track is divided into two parts, namely, space reconstruction is carried out on the current track and track embedding is carried out on the reconstructed space track; for one travel on adjacent pairs of points (C A ，C B ) The locus [ (as) A ，at A )，(as B ，at B )]According to the space track reconstruction result, the space track is expressed as an m-dimensional vector S= [ S ] 1 ，s 2 ，...，s m ]Wherein s is i Representing an ith spatial point in the spatial trajectory;
for the current trajectory, a time state vector S is used t To its travel total time feature, the travel total time (at A -at B ) Is expanded into an m-dimensional vector which is used for the data processing,S t the calculation method of (2) is as follows:
S t ＝(at B -at A )×S
to represent the effect of vehicle departure time on time-space relationship, a period vector is used to represent vehicle driving period, the time of day is divided into C time slices, and then C is used S To represent the departure time at of the journey A At a time-slice position during the day, the cycle vector for that trip may be represented as S C And can be found by the following formula:
according to the above-mentioned current track embedding manner, the embedded current track C may be expressed as:
C＝Concat(S，S t ，S c )。
3. a vehicle track space-time reconstruction method based on an attention mechanism as claimed in claim 2, wherein: in step S2, embedding expressions are performed on the plurality of history trajectories, and for a given pair of adjacent acquisition point pairs (C A ，C B ) A determined path between them, the historical track set of which is denoted as T H ＝{Tr 1 ，Tr 2 ，...，Tr N One of the tracks Tr is considered i The density of the space-time positions in the method is far lower than that of the path space points, and the track aggregation method is used for aggregating all the historical tracks into a complete space-time track, and the method specifically comprises the following steps:
s21, historical track aggregation:
dividing the space position in the path according to the space slice, then finding all time points on the time slice for a determined space slice, and using the average time of the track points as the time of the track on the space slice after aggregation; thus, for the historical track set T H Polymerized trajectoriesThe method can be expressed as follows:
wherein Tr is i Representing a historical track set T H The ith track in (c), N represents the number of tracks in the track set,representing track aggregation operations, in particular, for a fixed point, finding all times of passing the point, using the average of the times of passing the pointTime for the point; after the time of aggregation of each position point, sequencing the position points according to the position sequence to obtain an aggregated track;
s22, aggregating historical tracks in the same period:
the definition of the history trace for the same period is as follows: given a pair of adjacent points (C A ，C B ) And a set of historical trajectories T within a path therebetween H ＝{Tr 1 ，Tr 2 ，...，Tr N For a track [ (as) to be reconstructed ] A ，at A )，(as B ，at B )]Find all the departure times at from the track A Historical tracks of the same period, the set of which, i.e. the historical tracks of the same period, is denoted as T C ＝{Tr 1 ，Tr 2 ，...，Tr M Where m represents the number of tracks in the set, tr i Representing one track in the set, and aggregating the history tracks in the same period, wherein the operation formula is as follows:
s23, aggregation of historical tracks in the same state:
the definition of the history trace for the same state is as follows: given a pair of adjacent points (C A ，C B ) And a set of historical trajectories T within a path therebetween H ＝{Tr 1 ，Tr 2 ，...，Tr N -a }; for a track [ (as) to be reconstructed A ，at A )，(as B ，at B )]First according to the travel time (at B -at A ) Calculate the traffic state of the journey and then for T H Calculating the traffic state of each track according to the travel time of the track, and representing a track set with the same state as the track to be reconstructed as T S ＝{Tr 1 ，Tr 2 ，...，Tr O Where O represents the number of tracks in the set, tr i Representing one track in the collection, and aggregating the history tracks in the same stateThe operation is as shown in the following formula, in whichThe represented trajectories are aggregated:
s24, fusing historical tracks:
for the three aggregated historical tracks, in order to enable the model to capture the space-time relationship inside the tracks and the space-time relationship between the tracks, a splicing operation is used for fusing the tracks and inputting H as the historical tracks of the model, and the specific operation is as follows:
as can be seen from the above three types of history tracks, the history track set includes a history track set with the same period and a history track set with the same state, and the history track set is separately extracted to strengthen the weights of the two tracks in the neural network, because the track to be reconstructed may have higher similarity with the two tracks.
4. A vehicle track space-time reconstruction method based on an attention mechanism according to claim 3, wherein: in step S3, in order to reconstruct the time sequence of the current spatial track, considering the space-time relationship in the track learned from the historical track, that is, the input of the model is the historical space-time track and the current spatial track, and the output is the time sequence corresponding to the current spatial track; aiming at the time sequence generating task, adopting an encoder-decoder architecture, wherein the encoder is responsible for processing the historical space-time track, and the model encodes the historical space-time track by using a multi-head self-attention mechanism to extract the space-time relation in the historical track; in the decoder, the spatial track is input, the spatial features in the spatial track are still extracted by using a multi-head self-attention mechanism, and the cross-attention joint encoder and decoder are used in the spatial track, so that the space-time relationship in the track is learned from the output of the encoder; a long-short-term memory network is used at the end of the model to capture the time relationship between tracks, and finally the required time sequence is output.
5. The method for reconstructing a vehicle trajectory space-time based on an attention mechanism of claim 4, wherein: the model is divided into an encoder and a decoder, wherein the encoder is responsible for encoding a historical track, and the decoder is responsible for processing a current track and generating a time sequence by combining the output of the encoder, and the model specifically comprises the following steps:
an encoder: the input of the encoder is a historical track H, and a linear layer is used for embedding and expressing the historical track, and the historical track is mapped into a higher-dimensional hidden space:
H h ＝Linear(H)
wherein H is h Output as a linear layer; using a multi-head attention mechanism to capture its time-space relationship, the input sequence H is first entered h Mapping to Q h ，K h And V h Three subspaces:
Q H ＝W Q H h
K H ＝W K H h
V H ＝W V H h
wherein W is Q 、W K 、W V Weight matrices of Q, K, V, respectively, i.e. H h Mapping to three hidden spaces Q, K and V; in this regard, the multi-headed self-attention operation is defined as:
MultiHead(Q，K，V)＝[Head 1 ，Head 2 ，...，Head h ]W O
Head i ＝Attention(QW i Q ，KW i K ，VW i V )
where h is the number of attention heads, W O Is the final output weight matrix;
after multi-headed self-attention, the model then uses the residual connection and layer normalization layer, specifically formulated as follows:
LayerNorm(X+MultiHeadAttention(X))
wherein residual connection refers to x+ MultiHeadAttention (X), X refers to multi-headed self-attention input; layerNorm refers to layer normalization, which converts the mean and variance of the input of neurons of each layer into unity, the multi-head self-attention and residual connection & layer normalization module in the encoder part is repeated L times, the output of the upper layer is used as the input of the next layer until the output of the last layer of the encoder, and the output of the last layer is mapped to K, V hidden spaces by using an attention formula for the decoder to inquire;
a decoder: the input of the decoder is a current track C, and the current track C comprises space information, state information, cycle information and travel time information of the current track; embedding and expressing the current track C by using a linear layer, so as to map the current track C into a hidden space with higher dimension, wherein the specific operation is as follows:
C h ＝Linear(C)
C h output as a linear layer; after embedding the track into the representation, a dual-layer self-attention mechanism is used in the decoder to capture the spatio-temporal relationship within and between tracks, self-attention module and residual connection at the first layer&In layer normalization, the processing of the spatial trajectories is consistent with the self-attention mechanism in the encoder;
in the second layer multi-head self-attention module, a cross attention mechanism is used, in the layer, the final output of the encoder is mapped into two hidden spaces of K and V, the output of the upper layer of the decoder is mapped into a hidden space Q, and multi-head self-attention operation is carried out on the mapped Q, K and V; then aiming at the output of the cross attention mechanism, the output of the cross attention mechanism is processed by using a residual connection & layer normalization;
finally, a long and short term memory network is used to process the decoder output, and finally the LSTM output is the required time sequence:
T＝LSTM(Concat(O dec ，C))
wherein T represents the final generated time series, O dec Representing the output of the decoder, C representing the embedded spatial trajectory;
after the time sequence is generated, the time sequence and the unembossed space track are spliced together according to the position, so that the reconstructed space-time track can be obtained.
6. A vehicle trajectory space-time reconstruction system based on an attention mechanism, characterized in that: the system employs a method as claimed in claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310578357.7A CN116578661A (en) | 2023-05-22 | 2023-05-22 | Vehicle track time-space reconstruction method and system based on attention mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310578357.7A CN116578661A (en) | 2023-05-22 | 2023-05-22 | Vehicle track time-space reconstruction method and system based on attention mechanism |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116578661A true CN116578661A (en) | 2023-08-11 |
Family
ID=87537435
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310578357.7A Pending CN116578661A (en) | 2023-05-22 | 2023-05-22 | Vehicle track time-space reconstruction method and system based on attention mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116578661A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117831287A (en) * | 2023-12-29 | 2024-04-05 | 北京大唐高鸿数据网络技术有限公司 | Method, device, equipment and storage medium for determining highway congestion index |
-
2023
- 2023-05-22 CN CN202310578357.7A patent/CN116578661A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117831287A (en) * | 2023-12-29 | 2024-04-05 | 北京大唐高鸿数据网络技术有限公司 | Method, device, equipment and storage medium for determining highway congestion index |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108629978B (en) | Traffic track prediction method based on high-dimensional road network and recurrent neural network | |
| CN111832225B (en) | Method for constructing driving condition of automobile | |
| Zhang et al. | Deeptravel: a neural network based travel time estimation model with auxiliary supervision | |
| CN111539454B (en) | Vehicle track clustering method and system based on meta-learning | |
| CN111832814A (en) | Air pollutant concentration prediction method based on graph attention machine mechanism | |
| CN113487066B (en) | Long-time-sequence freight volume prediction method based on multi-attribute enhanced graph convolution-Informer model | |
| CN110570651A (en) | Road network traffic situation prediction method and system based on deep learning | |
| CN110909909A (en) | Short-term traffic flow prediction method based on deep learning and multi-layer spatiotemporal feature map | |
| CN114202120A (en) | Urban traffic travel time prediction method aiming at multi-source heterogeneous data | |
| CN110836675A (en) | Decision tree-based automatic driving search decision method | |
| CN116578661A (en) | Vehicle track time-space reconstruction method and system based on attention mechanism | |
| CN113762338A (en) | Traffic flow prediction method, equipment and medium based on multi-graph attention mechanism | |
| CN111461426A (en) | High-precision travel time length prediction method based on deep learning | |
| WO2024068527A1 (en) | A computer-implemented method for forecasting traffic indicators in a road network | |
| CN114973657A (en) | Urban traffic noise pollution analysis and evaluation method based on trajectory data | |
| CN114882069A (en) | Taxi track abnormity detection method based on LSTM network and attention mechanism | |
| CN116523177A (en) | Vehicle energy consumption prediction method and device integrating mechanism and deep learning model | |
| CN114091581A (en) | Vehicle operation behavior type identification method based on sparse track | |
| CN112819218B (en) | High-resolution urban mobile source pollution space-time prediction method, system and storage medium | |
| CN112927507A (en) | Traffic flow prediction method based on LSTM-Attention | |
| CN115984634A (en) | Image detection method, apparatus, device, storage medium, and program product | |
| Zhang et al. | UT-ATD: Universal Transformer for Anomalous Trajectory Detection by Embedding Trajectory Information. | |
| Fan et al. | Multimodal traffic travel time prediction | |
| Neto et al. | Predicting the next location for trajectories from stolen vehicles | |
| Sun et al. | Alleviating data sparsity problems in estimated time of arrival via auxiliary metric learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |