CN113157846A

CN113157846A - Intention and track prediction method and device, computing equipment and storage medium

Info

Publication number: CN113157846A
Application number: CN202110462326.6A
Authority: CN
Inventors: 张景淮; 张世权; 方良骥; 蒋沁宏; 刘毅成; 周博磊; 李樊
Original assignee: Bozhi Perceptual Interaction Research Center Co ltd; Sensetime Group Ltd
Current assignee: Bozhi Perceptual Interaction Research Center Co ltd; Sensetime Group Ltd
Priority date: 2021-04-27
Filing date: 2021-04-27
Publication date: 2021-07-23

Abstract

The present disclosure provides an intent and trajectory prediction method, apparatus, computing device and storage medium, wherein the intent prediction method comprises: determining a target key point pair in the track to be marked based on the track to be marked and the target map; determining first intention labeling information corresponding to each target key point pair; determining first intention marking information of each position point on the track to be marked based on the target map and the first intention marking information corresponding to the target key point pairs; or, the trajectory prediction method comprises: acquiring a motion track and a prefabricated map of a second target object; determining at least one piece of movement intention information of the movement track in a future preset time period based on the movement track and the prefabricated map; based on the at least one movement intention information, at least one predicted trajectory of the second target object within a future preset time period is determined. The method and the device for determining the first intention marking information can determine the first intention marking information corresponding to the track to be marked, and efficiency of determining the first intention marking information is improved.

Description

Intention and track prediction method and device, computing equipment and storage medium

Technical Field

The present disclosure relates to the field of automatic driving technologies, and in particular, to an intention and trajectory prediction method, apparatus, computing device, and storage medium.

Background

Most of intention labeling information in the prior art is obtained based on a manual labeling mode, certain labor cost is consumed, and the problem of low efficiency exists. Furthermore, the intention label information obtained by the manual label method is coarse-grained intention information, which neglects the transient transformativeness of the behavior of the target object, and thus the real intention corresponding to the target cannot be accurately reflected.

Disclosure of Invention

The embodiment of the disclosure at least provides an intention and track prediction method, an intention and track prediction device, a computing device and a storage medium.

In a first aspect, an embodiment of the present disclosure provides an intent prediction method, including:

determining a target key point pair in the track to be marked based on the track to be marked and a target map;

determining first intention labeling information corresponding to each target key point pair;

and determining first intention labeling information of each position point on the track to be labeled based on the target map and the first intention labeling information corresponding to the target key point pairs.

Under the condition that the tracks to be marked correspond to different road positions in the target map, the target key point pairs are determined in different modes, so that the target key point pairs can be determined in a mode matched with the road positions on the basis of the target map, and the accuracy of the determined target key point pairs is improved. Under the condition that the target key point pair exists in the track to be marked, the intention of changing lanes, turning directions, passing through intersections and the like of the object generating the track to be marked is shown, so that after the first intention marking information corresponding to the determined target key point pair exists, the first intention marking information of all position points on the track in the target key point pair in the track to be marked can be determined, and therefore the first intention marking information of each position point on the track to be marked can be accurately determined. In addition, since the first intention label information is the intention label information of each position point, it is possible to accurately reflect the fine intention information of the first target object at each time when the trajectory to be labeled is generated. In addition, the process of determining the first intention labeling information can be automatically completed, so that the consumption of labor cost is reduced, and the intention labeling efficiency can be improved.

In a possible implementation manner, the determining, based on the target map and the first intention labeling information corresponding to the target key point pair, the first intention labeling information of each position point on the track to be labeled includes:

determining different sub-tracks corresponding to different preset road types and included in the track to be marked based on the target map;

and determining first intention labeling information of each position point on the track to be labeled based on the first intention labeling information corresponding to the target key point pair and the preset road category corresponding to the sub-track.

The method for determining the first intention labeling information of each position point on different sub-tracks can be determined based on the preset road category, and the accuracy of determining the first intention labeling information of each position point on the track to be labeled can be improved by means of the intention determining methods corresponding to different sub-tracks.

In a possible implementation manner, the determining, based on the target map, different sub-tracks included in the track to be labeled and corresponding to different preset road categories includes:

taking a sub-track corresponding to a first preset road category in the track to be marked as a first track;

the target key point pairs comprise channel-changing key point pairs; the determining a target key point pair in the track to be marked based on the track to be marked and the target map comprises the following steps:

determining a lane change point of the first track where the lane identification changes based on the target map;

and determining a lane change starting point and a lane change ending point in the first track based on the lane change point and the first track, and taking the lane change starting point and the lane change ending point as a lane change key point pair of the first track.

The first track generated on the common straight line road in the track to be marked can be screened out through the first preset road category, then the position of the road where the first target object generating the first track runs can be determined to change based on the change of the road mark, the point with the changed road mark is used as the lane changing point of the first target object, the accuracy of the determined lane changing point can be ensured, and then the accurate determination of the lane changing key point pair can be realized based on the determined lane changing point.

In a possible implementation, the determining a lane change starting point and a lane change ending point in the first track based on the lane change point and the first track includes:

determining starting time information corresponding to a preset starting point and ending time information corresponding to a preset ending point based on the time information corresponding to the lane changing point and a preset time interval;

taking a position point of the first track with the time information as the starting time information as a preset starting point, and taking a position point of the first track with the time information as the ending time information as a preset ending point;

determining a lane change starting point in the first track based on the preset starting point and the lane change point;

and determining a lane change end point in the first track based on the preset end point and the lane change point.

The preset track changing track of the first target object can be determined from the first track based on the preset starting point and the preset ending point determined by the preset time interval, and then the track changing starting point and the track changing ending point of the first target object can be accurately determined by utilizing the determined preset track changing track, the preset starting point and the preset ending point.

In a possible implementation manner, the determining, based on the preset starting point and the lane change point, a lane change starting point in the first track includes:

connecting the preset starting point and the lane changing point to obtain a target connecting line;

determining the vertical distance from at least part of position points between the preset starting point and the lane changing point in the first track to the target connecting line;

and taking the position point corresponding to the maximum vertical distance as the lane change starting point.

When the vertical distance is the largest, the first target object can be ensured to be farthest away from a lane changing line where a lane changing point is located, and when the vertical distance is the farthest away, the probability that the first target object starts to change lanes is the largest, so that the position point corresponding to the largest vertical distance is used as a lane changing starting point, and the reasonability and the accuracy of the determined lane changing starting point are improved.

In a possible implementation manner, after the taking the position point corresponding to the maximum vertical distance as the lane change starting point, the method further includes:

and taking the lane change starting point with the vertical distance larger than a preset threshold value as a final lane change starting point.

For the lane change starting point filtered by the preset threshold, when the first target object generates the first track corresponding to the lane change starting point, the first target object most probably moves on the lane change line where the lane change point is located, namely, whether the first target object has the real intention of lane change can be determined by the preset threshold, so that the reasonability of the determined lane change starting point is improved.

In one possible embodiment, the determining the first intention labeling information corresponding to each of the target key point pairs includes:

determining the displacement direction of the first target object between the lane change starting point and the lane change ending point of each lane change key point pair;

and determining first intention labeling information corresponding to the channel changing key point pairs based on the displacement direction.

The displacement direction can accurately reflect the intention of the first target object, and the first intention labeling information corresponding to the channel changing key point pair can be accurately determined by utilizing the displacement direction.

In a possible implementation manner, the determining, based on the first intention labeling information corresponding to the target key point pair and the preset road category corresponding to the sub-track, the first intention labeling information of each position point on the track to be labeled includes:

and for each first track, taking the first intention labeling information of the channel changing key point pair corresponding to the first track as the first intention labeling information of a first position point positioned between the channel changing key point pairs corresponding to the first track in the first track.

Each first position point located between the track-changing key point pair corresponding to the first track is a position point on the track-changing track generated by the first target object executing the intention corresponding to the first intention marking information of the track-changing key point pair, so that the first intention marking information of the track-changing key point pair can reflect the first intention marking information of the first position point, and further, the first intention marking information of each first position point is accurately determined.

for each first track, determining a second position point which is not located between the channel change key point pair corresponding to the first track in the first track;

and determining first intention marking information of the second position point based on the speed information corresponding to the second position point.

The first target object located at the second position point has no intention of changing lanes, turning, passing through intersections and the like, so whether the first target object moves on the second position point can be accurately determined by using the speed information, and further, the first intention marking information of each second position point can be accurately determined.

taking a sub-track corresponding to a second preset road type in the track to be marked as a second track;

the target key point pairs comprise intersection key point pairs; the determining a target key point pair in the track to be marked based on the track to be marked and the target map comprises the following steps:

determining intersection entrance point information and intersection exit point information in the second track based on the target map;

and taking an entry point corresponding to the intersection entry point information in the second track and an exit point corresponding to the intersection exit point information in the second track as an intersection key point pair of the second track.

And if the first target object is positioned at the intersection, the second track generated in the intersection is started at the intersection entry point and ended at the intersection exit point, so that the intersection key point pair can be accurately determined according to the determined intersection entry point information and the intersection exit point information.

aiming at each intersection key point pair, determining included angle information between a road where an exit point corresponding to the intersection key point pair is located and a road where an entry point is located;

and determining first intention labeling information corresponding to the intersection key point pairs based on the included angle information.

The included angle information between the roads can accurately reflect the intention of the first target object, and the first intention labeling information corresponding to the lane changing key point pair can be accurately determined by using the included angle information.

and regarding each second track, taking the first intention labeling information of the intersection key point pair corresponding to the second track as the first intention labeling information of a third position point positioned between the intersection key point pairs corresponding to the second track in the second track.

The second track generated by the first target object when passing through the intersection is bound to start at the intersection entry point and end at the intersection exit point, and the second track is determined by the intersection entry point and the intersection exit point, so that the first intention marking information of each position point in the second track is bound to be consistent with the first intention marking information of the intersection key point corresponding to the second track, and therefore the first intention marking information of each third position point can be accurately determined.

taking a position point which does not belong to the first track and does not belong to the second track in the track to be marked as a fourth position point;

and determining first intention marking information of the fourth position point based on the speed information corresponding to the fourth position point.

The first target object located at the fourth position point has no intention of changing lanes, turning, passing through intersections and the like, so that whether the first target object moves on the fourth position point can be accurately determined by using the speed information, and further, the first intention marking information of each fourth position point can be accurately determined.

In a possible implementation manner, before determining the target key point pair in the track to be annotated, the method further includes:

under the condition that the to-be-annotated track is discontinuous, determining a track expression of the known track based on the known track in the to-be-annotated track;

completing the track to be marked based on the track expression to obtain a completed track to be marked;

taking the completed track to be marked which accords with the preset completion limiting information as a candidate track to be marked;

and taking the known track in the candidate tracks to be marked as a final track to be marked.

The curve corresponding to the track expression can be well attached to the part of the tracks to be marked after completion, unqualified tracks to be marked after completion can be screened out by utilizing preset completion limiting information, the part of discontinuous candidates can be reserved, and then the quantity of the first intention marking information can be increased.

In one possible embodiment, the preset completion limit information includes:

the completed track to be marked is positioned on a road in the target map; and the supplemented track to be marked conforms to the driving rule of the road where the supplemented track to be marked is located.

And by utilizing the preset completion limiting information, the completed track to be marked which does not accord with the road driving rule and is not positioned on the road in the target map can be screened out.

In a possible implementation manner, in the case that the trajectory to be annotated is discontinuous, determining a trajectory expression of a known trajectory based on the known trajectory in the trajectory to be annotated includes:

and under the condition that the tracks to be marked are discontinuous, screening known tracks with the track length larger than the preset length from the known tracks in the tracks to be marked, and determining a track expression of the known tracks.

In this way, a shorter known trajectory can be screened out that does not have the determined intended annotation information.

In a second aspect, an embodiment of the present disclosure provides a trajectory prediction method, including:

acquiring a motion track and a prefabricated map of a second target object;

determining at least one motion intention message of the motion trail in a future preset time period based on the motion trail and the prefabricated map;

determining at least one predicted trajectory of the second target object within a future preset time period based on the at least one movement intent information.

Based on the motion trail of the target object, the motion intention information of the target object in the process of generating the motion trail can be determined; based on the high-precision prefabricated map, it can be determined that the movement locus corresponds to road regulation information of roads in the prefabricated map, and the road regulation information can play a role of indicating the movement intention of the target object, for example, a left-turn road can indicate the target object to make a left turn. Furthermore, by combining the movement intention information and the road rule information in the process of generating the movement track, the movement intention information of the target object in a future preset time period can be accurately predicted; and, because the predicted track is generated based on the movement intention information, the generated predicted track can be matched with the movement intention information, and the generated predicted track has higher accuracy on the basis of the accuracy of the movement intention information. In addition, the movement intention information is generated according to the pre-prepared map, so that the generated movement intention information may include a plurality of movement intention information when the generated movement intention information conforms to the road rule information, furthermore, when the generated movement intention information includes a plurality of movement intention information, the prediction tracks matched with each piece of movement intention information can be respectively generated, the diversity and difference of the generated prediction tracks are increased, the various prediction tracks can better fit various potential driving tracks of the target object, and the safety of automatic driving is further improved.

In one possible embodiment, the determining at least one piece of movement intention information of the movement track in a future preset time period based on the movement track and the prefabricated map includes:

determining depth feature information of a scene where the second target object is located based on state information corresponding to the motion track and the prefabricated map;

determining at least one motion intention information of the motion trail in a future preset time period based on the depth feature information;

the determining, based on the at least one movement intention information, at least one predicted trajectory of the second target object within a future preset time period comprises:

determining at least one predicted trajectory of the second target object within a future preset time period based on the depth feature information and the at least one movement intention information.

The state information comprises information such as speed, acceleration, a road where the second target object is located, a position on the road and the like, the prefabricated map comprises detailed information of the road in the state information corresponding to the motion track, the depth characteristic information can reflect characteristic information of a scene where the second target object is located, for example, position information of other objects adjacent to the second target object, appearance, orientation and the like of the second target object, therefore, the depth characteristic information of the scene where the second target object is located can be accurately determined based on the prefabricated map and the state information, furthermore, a predicted track which is in accordance with the motion intention and is matched with the scene where the second target object is located can be generated based on the accurate depth characteristic information and the determined motion intention information, and the reasonability of determining the predicted track is improved.

In one possible embodiment, the exercise intention information includes an exercise intention and a probability corresponding to the exercise intention;

the determining, based on the depth feature information and the at least one movement intention information, at least one predicted trajectory of the second target object within a future preset time period comprises:

for each piece of movement intention information, determining at least one predicted trajectory of the second target object within a future preset time period based on the movement intention, the probability corresponding to the movement intention and the depth feature information.

The probability corresponding to the movement intention can represent the possibility of the second target object having the movement intention, the higher the probability is, the higher the possibility is, and based on the movement intention and the probability corresponding to the movement intention, more prediction tracks corresponding to the movement intention with the higher probability can be generated, so that the accuracy of the generated prediction tracks is improved.

In one possible embodiment, the predicted trajectory is output by a trained predictive neural network; the prediction neural network is obtained by training sample tracks and sample maps.

The trained prediction neural network has higher reliability, so that a prediction track with high accuracy can be generated.

In one possible embodiment, the predictive neural network is trained by the following steps:

inputting a part of sample tracks of the sample objects and a sample map into a prediction neural network to be trained;

determining prediction intent information for the sample object based on a portion of the sample trajectories and the sample map;

determining intention marking information of the sample object starting from the partial track within a preset time period in the future;

and training the predictive neural network to be trained based on the predicted intention information and the intention marking information in the future preset time period to obtain the trained predictive neural network.

The loss function related to the intention prediction can be determined by utilizing the prediction intention information and the intention labeling information, the loss function is utilized to train the prediction neural network to be trained, the accuracy of the trained prediction neural network to the intention prediction can be improved, in addition, a sample prediction track which accords with the prediction intention information can be generated based on the prediction intention information with higher accuracy, the loss function related to the track is constructed by utilizing the sample prediction track and the sample track, the loss function is utilized to train the prediction neural network to be trained, and finally, the accuracy of the pre-stored track determined by the trained prediction neural network can be improved.

In one possible embodiment, the intention label information includes first intention label information;

the determining of the intention labeling information of the sample object starting from the partial track within a preset time period in the future includes:

with the intention prediction method in the first aspect, first intention labeling information of the sample object starting from the partial trajectory within a preset time period in the future is determined.

In this way, the rationality and accuracy of the determined loss function related to the intent prediction can be further improved by using the first intent marking information and the prediction intent information which can reflect the real intent of the sample object at each position point, and the accuracy of the trained prediction neural network for the intent prediction can be improved by training the prediction neural network to be trained by using the loss function.

In one possible embodiment, the intention label information includes second intention label information;

determining first intention marking information of the sample object starting from the partial track within a preset time period in the future by using the intention prediction method in the first aspect;

and determining the second intention labeling information according to the first intention labeling information and the intention conversion rule.

In this way, the predicted neural network is trained using the second intention label information determined based on the first intention label information and the intention conversion rule, and the amount of data to be processed is reduced as compared with a scheme in which the predicted neural network is trained directly using the first intention label information, thereby making it possible to improve the training speed of the predicted neural network.

In one possible embodiment, the intention label information includes third intention label information labeled manually.

Therefore, the scheme of training the predictive neural network by directly utilizing the artificially labeled third intention labeling information avoids the step of determining the first intention labeling information or the second intention labeling information, and is beneficial to improving the training speed of the predictive neural network.

In a possible implementation, after determining at least one predicted trajectory of the second target object within a future preset time period, the method further includes:

and controlling a driving device for acquiring the motion trail of the second target object or sending prompt information based on at least one predicted trail of the second target object in a future preset time period.

Thus, the driving device can be controlled to avoid the second target object or give prompt information in the driving process of the driving device, so that the safety of automatic driving is improved.

In a third aspect, an embodiment of the present disclosure further provides an intent prediction apparatus, including:

the first determination module is used for determining a target key point pair in the track to be marked based on the track to be marked and a target map;

the second determining module is used for determining the first intention labeling information corresponding to each target key point pair;

and the third determining module is used for determining the first intention labeling information of each position point on the track to be labeled based on the target map and the first intention labeling information corresponding to the target key point pairs.

In a fourth aspect, an embodiment of the present disclosure further provides a trajectory prediction apparatus, including:

the acquisition module is used for acquiring a motion track and a prefabricated map of a second target object;

the fourth determination module is used for determining at least one piece of movement intention information of the movement track in a future preset time period based on the movement track and the prefabricated map;

a fifth determination module, configured to determine, based on the at least one movement intention information, at least one predicted trajectory of the second target object within a future preset time period.

In a fifth aspect, this disclosure also provides a computer device, a processor, a memory, and a computer program product, where the memory stores machine-readable instructions executable by the processor, and the processor is configured to execute the machine-readable instructions stored in the memory, and when the machine-readable instructions are executed by the processor, the machine-readable instructions are executed by the processor to perform the steps in the first aspect, or any one of the possible implementations of the first aspect, or the second aspect.

In a sixth aspect, alternative implementations of the present disclosure also provide a computer-readable storage medium having stored thereon a computer program which, when executed, performs the steps of the first aspect described above, or any one of the possible implementations of the first aspect, or which, when executed, performs the steps of the second aspect described above.

For the description of the effects of the intention and trajectory predicting device, the computer apparatus, and the computer readable storage medium, reference is made to the description of the intention and trajectory predicting method, which is not repeated herein.

In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly described below, and the drawings herein incorporated in and forming a part of the specification illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the technical solutions of the present disclosure. It is appreciated that the following drawings depict only certain embodiments of the disclosure and are therefore not to be considered limiting of its scope, for those skilled in the art will be able to derive additional related drawings therefrom without the benefit of the inventive faculty.

FIG. 1 illustrates a flow chart of an intent prediction method provided by an embodiment of the present disclosure;

FIG. 2 is a flow chart illustrating a trajectory prediction method provided by an embodiment of the present disclosure;

FIG. 3a is a schematic diagram illustrating a method for predicting a trajectory of a neural network output target object provided by an embodiment of the present disclosure;

FIG. 3b is a schematic diagram illustrating a predicted image of a predicted neural network output with respect to a predicted trajectory of a second target object provided by an embodiment of the present disclosure;

FIG. 4 illustrates a flow chart of a method of training a predictive neural network to be trained provided by an embodiment of the present disclosure;

FIG. 5 is a schematic diagram of an intent prediction apparatus provided by an embodiment of the present disclosure;

FIG. 6 is a schematic diagram of a trajectory prediction device provided by an embodiment of the present disclosure;

fig. 7 shows a schematic structural diagram of a computer device provided by an embodiment of the present disclosure.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, not all of the embodiments. The components of embodiments of the present disclosure, as generally described and illustrated herein, may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the disclosure, as claimed, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making creative efforts, shall fall within the protection scope of the disclosure.

Furthermore, the terms "first," "second," and the like in the description and in the claims, and in the drawings described above, in the embodiments of the present disclosure are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein.

Reference herein to "a plurality or a number" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.

Research shows that most of intention labeling information in the prior art is obtained based on a manual labeling mode, certain labor cost is consumed, and the problem of low efficiency exists. Moreover, the intention labeling information obtained by the manual labeling method is coarse-grained intention information, which neglects the transient transformativeness of the behavior of the target object, thereby causing that the real intention corresponding to the target cannot be accurately reflected.

Based on the above research, the present disclosure provides an intention and trajectory prediction method, apparatus, computing device, and storage medium, where, in a case where a trajectory to be annotated corresponds to different road positions in a target map, the target key point pairs are determined in different manners, so that, based on the target map, the target key point pairs can be determined in a manner matching the road positions, and accuracy of the determined target key point pairs is improved. Under the condition that the target key point pair exists in the track to be marked, the intention of changing lanes, turning directions, passing through intersections and the like of the object generating the track to be marked is shown, so that after the first intention marking information corresponding to the determined target key point pair exists, the first intention marking information of all position points on the track in the target key point pair in the track to be marked can be determined, and therefore the first intention marking information of each position point on the track to be marked can be accurately determined. In addition, since the first intention label information is the intention label information of each position point, it is possible to accurately reflect the fine intention information of the first target object at each time when the trajectory to be labeled is generated. In addition, the process of determining the first intention labeling information can be automatically completed, so that the consumption of labor cost is reduced, and the intention labeling efficiency can be improved.

The above-mentioned drawbacks are the results of the inventor after practical and careful study, and therefore, the discovery process of the above-mentioned problems and the solutions proposed by the present disclosure to the above-mentioned problems should be the contribution of the inventor in the process of the present disclosure.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.

To facilitate understanding of the embodiment, a detailed description is first given of an intent and trajectory prediction method disclosed in the embodiments of the present disclosure, where an execution subject of the intent and trajectory prediction method provided in the embodiments of the present disclosure is generally a computer device with certain computing capability, and in some possible implementations, the trajectory prediction and intent annotation method may be implemented by a processor calling a computer-readable instruction stored in a memory.

The trajectory prediction and intent annotation method provided by the embodiments of the present disclosure is described below by taking an execution subject as a computer device as an example.

As shown in fig. 1, a flowchart of an intent prediction method provided for an embodiment of the present disclosure may include the following steps:

s101: and determining a target key point pair in the track to be marked based on the track to be marked and the target map.

Here, the target map may be a high-precision map generated in advance about a road on which the first target object is located when the track to be noted is generated, and for example, the target map may be a city road map of a city corresponding to the road on which the first target object is located when the track to be noted is generated.

The track to be marked is a complete motion track generated by the acquired first target object, the target key point pairs can comprise a lane changing key point pair and an intersection key point pair, and under the condition that the target key point pair exists in the track to be marked, the intention of lane changing, turning, crossing and the like of the object generating the track to be marked can be explained.

In specific implementation, according to the target map, it can be determined that each position point on the track to be marked corresponds to a target position point in the target map. Specific road information, such as a road identifier, a preset road category, and the like, may exist in each target location point. And further, determining a target key point pair in the track to be marked according to the road information corresponding to each position point on the determined track to be marked.

S102: and determining the corresponding first intention labeling information of each target key point pair.

Here, for each determined target key point pair, the first intention label information corresponding to the target key point pair may be determined by an intention determination method corresponding to a preset road type based on the preset road type in the road information corresponding to the target key point pair. For example, if the preset road category corresponding to the target key point pair is a left turn intersection, the left turn may be used as the first intention labeling information corresponding to the target key point pair.

And the first intention labeling information is fine-grained intention labeling information.

S103: and determining first intention marking information of each position point on the track to be marked based on the first intention marking information corresponding to the target map and the target key point pairs.

After the first intention labeling information of each target key point pair is determined, according to the target position point corresponding to each target key point pair and the target position point corresponding to each position point on the track to be labeled, a first target position point between the target position points corresponding to the target key point pair in the target position points corresponding to each position point on the track to be labeled and a second target position point outside the target position points corresponding to the target key point pair can be determined.

Then, for each first location point, the first intention labeling information of the target key point pair corresponding to the first target location point may be used as the first intention labeling information of the location point on the to-be-labeled track corresponding to the first target location point.

For each second target position point, the first intention labeling information of the position point on the to-be-labeled track corresponding to the second target position point can be determined according to the speed information of the second target position point and the like.

Therefore, the first intention labeling information of each position point on the track to be labeled can be determined.

Therefore, under the condition that the tracks to be marked correspond to different road positions in the target map, the target key point pairs are determined in different modes, so that the target key point pairs can be determined in a mode matched with the road positions on the basis of the target map, and the accuracy of the determined target key point pairs is improved. Under the condition that the target key point pair exists in the track to be marked, the intention of changing lanes, turning directions, passing through intersections and the like of the object generating the track to be marked is shown, so that after the first intention marking information corresponding to the determined target key point pair exists, the first intention marking information of all position points on the track in the target key point pair in the track to be marked can be determined, and therefore the first intention marking information of each position point on the track to be marked can be accurately determined. In addition, since the first intention label information is the intention label information of each position point, it is possible to accurately reflect the fine intention information of the first target object at each time when the trajectory to be labeled is generated. In addition, the process of determining the first intention labeling information can be automatically completed, so that the consumption of labor cost is reduced, and the intention labeling efficiency can be improved.

In one embodiment, for S103, the first intention labeling information of each position point on the trajectory to be labeled can be determined according to the following steps:

the method comprises the following steps that firstly, different sub-tracks corresponding to different preset road types and included in a track to be marked are determined based on a target map;

and secondly, determining first intention labeling information of each position point on the track to be labeled based on the first intention labeling information corresponding to the target key point pairs and the preset road category corresponding to the sub-track.

In specific implementation, based on the target location point and the target map corresponding to the to-be-labeled track, the preset road category corresponding to each target location point, that is, the preset road category corresponding to the location point on the to-be-labeled track corresponding to each target location point, may be determined.

Furthermore, the track to be marked can be segmented according to the preset road type corresponding to the position point on the track to be marked, and different sub-tracks corresponding to different preset road types and included in the track to be marked are determined.

Then, after determining the first intention labeling information of each target key point pair, the first intention labeling information of each position point on the sample track may be determined according to the preset road category corresponding to each sub-track in different sub-tracks included in the determined track to be labeled and the first intention labeling information of the corresponding key point pair in the sub-track.

Therefore, intention labeling can be carried out on each position point on the to-be-labeled track in a fine-grained manner, and the real intention of the first target object on each position point on the sample track can be reflected.

In one embodiment, the target key point pair may include a lane change key point pair, the lane change key point pair includes a lane change starting point and a lane change ending point, and the preset road category includes a first preset road category, where the first preset road category may be a general straight road category. Aiming at the first step, a target map can be used for screening out sub-tracks corresponding to the first preset road type in the track to be marked, the sub-tracks are used as first tracks, then, a track curve corresponding to the first tracks can be processed by using a B-spline interpolation method, and middle points on the track curve are supplemented. Therefore, the continuity of the track curve can be enhanced, and the accuracy of the determined channel-changing key point pairs can be improved by using the track curve with good continuity.

After B-spline interpolation is performed on the trajectory curve corresponding to the first trajectory, road information of a road on which a first target object generating the first trajectory is located in the target map may be determined by using the target map.

Based on the road identifier in the road information, a first track with the changed road identifier may be determined, and then a position point in the first track corresponding to the changed road identifier may be determined, and the position point is used as a lane change point of the first track, and a time when the road identifier is changed is used as time information of the lane change point. Then, determining the time which is before the time corresponding to the time information of the lane changing point and has the preset time interval with the time corresponding to the time information of the lane changing point by using the preset time interval, and taking the time as the starting time information corresponding to the preset starting point; then, the time after the determined time corresponding to the time information of the lane change point and having the preset time interval with the time corresponding to the time information of the lane change point can be used as the end time information corresponding to the preset end point. Here, the preset time interval may represent a time interval from when the lane change starts to when the lane change point is reached, or from a time corresponding to the lane change point to when the lane change ends, and in general, the first target object may complete the lane change for a time length corresponding to two preset time intervals.

Further, based on the time information of each position point in the track to be labeled, the position point in the first track, where the time information is the start time information, may be taken as a preset starting point, and the position point in the first track, where the time information is the end time information, may be taken as a preset end point. And then, determining a preset lane changing track of the first target object from the first track according to the preset starting point and the preset end point. Therefore, the preset lane changing track of the first target object can be determined more accurately based on the preset starting point, the lane changing point and the preset ending point.

In one embodiment, after determining the preset starting point, the lane change point and the preset ending point, a lane change key point location algorithm may be used to determine a real lane change starting point and a real lane change ending point of the first target object from the preset lane change trajectory, and further determine a real lane change trajectory of the first target object, wherein the lane change key point location algorithm may be a Distance-to-finger algorithm.

In specific implementation, for a lane change starting point, in a preset lane change trajectory curve corresponding to a preset lane change trajectory, the determined preset lane change starting point and a lane change point may be connected to obtain a target connecting line, then, perpendicular lines may be drawn to the target connecting line with each of at least some position points in the first trajectory as a starting point, a distance of each perpendicular line is determined, the distance of the perpendicular line corresponding to each position point is taken as a vertical distance corresponding to each position point, and then, a position point corresponding to the largest vertical distance may be selected as the lane change starting point. Here, at least a part of the position points in the first track are all or part of the position points in the preset track changing track between the preset track changing starting point and the track changing point.

In addition, after determining the lane change starting point, a preset threshold may be used to screen out a lane change starting point whose vertical distance is greater than the preset threshold, and take such a lane change starting point as a final lane change starting point, and for a lane change starting point whose vertical distance is less than the preset threshold, the lane change starting point is discarded, and it is determined that no lane change key point pair exists in the first track corresponding to the lane change starting point, in a specific implementation, the preset threshold may be 0.8 m, which is less than half of the width of the medium-small first target object. Therefore, for the lane change starting point filtered by the preset threshold, when the first target object generates the first track corresponding to the lane change starting point, the first target object most probably moves on the lane change line where the lane change point is located, namely, whether the first target object has the real intention of lane change can be determined by the preset threshold, and therefore the reasonability of the determined lane change starting point is improved.

For the lane change end point, the determined preset lane change end point and the determined lane change point can be connected to obtain a target connecting line corresponding to the lane change end point, then the vertical distance between each position point in at least part of position points in the first track and the target connecting line corresponding to the lane change end point can be determined, and the position point corresponding to the maximum vertical distance is selected as the lane change end point. Here, at least a part of the position points in the first trajectory are all position points or a part of position points in the preset switch trajectory between the preset switch end point and the switch point.

In addition, after determining the lane change end point, a lane change end point with a vertical distance greater than a preset threshold corresponding to the lane change end point may be screened out by using a preset threshold corresponding to the lane change end point, and such a lane change end point may be used as a final lane change end point, and for a lane change end point with a vertical distance less than the preset threshold corresponding to the lane change end point, the lane change end point is discarded, and it is determined that no lane change key point pair exists in the first track corresponding to the lane change end point, and in a specific implementation, the preset threshold corresponding to the lane change end point may be 0.64 meters.

Based on the above process, the lane change starting point and the lane change ending point in the first track can be determined, and the lane change starting point and the lane change ending point in the first track are used as the pair of lane change key points in the first track.

Then, for the lane change key point pair, based on the target map, a displacement direction of the first target object between a lane change starting point and a lane change ending point in the lane change key point pair may be determined, and then, intention information corresponding to the displacement direction may be determined and used as the first intention labeling information of the lane change key point pair.

In one embodiment, step two can be implemented as follows:

for the determined first track corresponding to each first preset category, all first location points located between the lane change key point pairs corresponding to the first track may be determined first, where the first location points are necessarily located in the lane change track, and therefore, the first intention label information of the first location points is necessarily the same as the first intention label information of the lane change key point pairs, and further, the first intention label information of the lane change key point pairs corresponding to the first track may be used as the first intention label information of the first location points located between the lane change key point pairs corresponding to the first track in the first track.

In another embodiment, for the case that the length of the lane change track corresponding to the first track is smaller than the length of the first track, there must be a position point in the first track that is not located between the pair of lane change key points corresponding to the first track, and therefore, when determining the first intention marking information of each position point in the sample track, it is also necessary to determine the first intention marking information of the second position point that is not located between the pair of lane change key points corresponding to the first track.

In a specific implementation, for each of the second location points, the first intention marking information of the location point may be determined according to the speed information of the location point. For example, if the speed corresponding to the speed information of the location point is less than the preset speed threshold, it indicates that the first target object at the location point is in a stationary state, the first intention marking information of the location point may be determined as the stationary intention, and if the speed corresponding to the speed information of the location point is greater than the preset speed threshold, it indicates that the first target object at the location point is in a straight-moving state, the first intention marking information of the location point may be determined as the straight-moving intention.

In addition, for a first track without a lane change key point pair, first intention labeling information of a fifth position point in the first track needs to be determined. In specific implementation, for each position point in the fifth position points, the first intention labeling information of the position point may still be determined according to the speed information of the position point, which is not described herein again.

In one embodiment, the target key point pairs may further include intersection key point pairs, the intersection key point pairs include entry points and exit points, and the preset road categories include a second preset road category, where the second preset road category may be a road category including intersections, U-shaped intersections, and the like. Aiming at the first step, a target map can be utilized to screen out sub-tracks corresponding to a second preset road type in the tracks to be marked, the sub-tracks are used as second tracks, then the road positions of the second tracks in the target map can be determined by utilizing the target map, and intersection entrance point information and intersection exit point information in the second tracks can be determined based on the determined road positions.

Further, a position point corresponding to the intersection entry point information in the second track may be determined and used as the entry point, a position point corresponding to the intersection exit point information in the second track may be determined and used as the exit point, and the determined entry point and the exit point may be used as the intersection key point pair of the second track.

Then, aiming at each intersection key point pair, the included angle information between the road where the exit point of the exit point corresponding to the intersection key point pair is located and the road where the entry point is located can be determined, wherein the included angle information comprises the included angle and the included angle direction (clockwise direction and anticlockwise direction), the included angle direction is the direction from the road where the exit point is located to the road where the entry point is located, and the included angle is the angle between the roads to which the included angle direction points. Based on the angle and the direction of the included angle corresponding to the included angle information, intention information corresponding to the included angle information can be determined, for example, if the direction of the included angle corresponding to the included angle information is clockwise and the angle of the included angle is 30 degrees to 150 degrees, the intention information of the included angle information is a left turn intention; if the direction of the included angle corresponding to the included angle information is clockwise or counterclockwise and the included angle exceeds 150 degrees, the intention information of the included angle information is a U-shaped turning intention, and if the included angle corresponding to the included angle information is within plus or minus 30 degrees, the intention information of the included angle information is a straight intention.

Furthermore, the corresponding first intention labeling information of the intersection key point pair can be determined based on the included angle information.

In one embodiment, step two can be implemented as follows:

for the second track, if the first target object is located at the intersection, the second track generated in the intersection is bound to start at the intersection entry point and end at the intersection exit point, and the second track is determined by the intersection entry point and the intersection exit point, so that the first intention label information of each position point in the second track is bound to be consistent with the first intention label information of the intersection key point corresponding to the second track, and therefore, the first intention label information of the intersection key point pair corresponding to the second track can be used as the first intention label information of the third position point located between the intersection key point pairs corresponding to the second track in the second track.

In addition, the position point which does not belong to the first track and does not belong to the second track in the track to be marked is taken as a fourth position point. For each position point in the fourth position point, the first intention labeling information of the position point can still be determined according to the speed information of the position point, and details are not repeated here.

Based on the first intention labeling information of each position point in the track to be labeled can be determined.

In an embodiment, before determining a target key point pair in a track to be annotated, for a discontinuous track to be annotated, a continuous known track in the track to be annotated may be determined, and then, for the determined known track, the track to be annotated may be completed in a manner of determining a track expression of the known track, and after the completed track to be annotated is obtained. And then, screening the completed track to be marked by using preset completion limit information and a target map to obtain a final and reasonable candidate track to be marked. The preset completion limiting information may include that the completed track to be marked is located on the road in the target map, and the completed track to be marked conforms to the driving rule of the road where the completed track to be marked is located, where the driving rule may include turning left and right, turning off head, and the like. In specific implementation, based on the target map, it can be determined that the supplemented to-be-annotated track corresponds to road information in the target map, then according to road identifiers in the road information, a track where the supplemented to-be-annotated track breaks away from a road in the target map can be screened out, and the situation that the first target object breaks away from the road does not exist in the driving process, so that the supplemented to-be-annotated track which breaks away from the road in the target map is an unreasonable track and has no reference value, and the supplemented to-be-annotated track is discarded.

According to the preset completion limit information that the completed track to be marked conforms to the driving rule of the road where the completed track to be marked is located, the completed track to be marked which does not conform to the driving rule can be screened out, for example, the completed track to be marked is a track of a first target object which moves in a reverse direction on the road corresponding to the track, and the first target object follows the driving rule of the road where the first target object is located in the driving process, so that the screened completed track to be marked which does not conform to the driving rule is also an unreasonable track, has no reference value, and the completed track to be marked is discarded.

Then, based on the step of determining the first intention labeling information of each position point on the to-be-labeled track, the position point on the known track in the candidate to-be-labeled track is subjected to intention labeling by using the completion position point in the candidate to-be-labeled track, wherein the completion position point in the candidate to-be-labeled track is predicted and may not be accurate, so that the position point on the known track is subjected to intention labeling only by using the completion position point, and the intention labeling is not performed on the completion position point. Then, after determining the first intention labeling information of the position point on the known track in the candidate to-be-labeled tracks, the known track in the candidate to-be-labeled tracks can be used as the final to-be-labeled track. Therefore, partial discontinuous tracks to be marked can be reserved, intention marking is carried out on each position point on the reserved tracks to be marked through complementing the position points, the number of the first intention marking information is increased, and the reasonability of the determined first intention marking information can be improved.

In another embodiment, after the completed track to be annotated is obtained, intention annotation may also be performed on each position point on the completed track to be annotated, so as to determine first intention annotation information of each position point on the completed track to be annotated.

In addition, after the continuous known tracks in the tracks to be marked are determined, the track expressions of the known tracks with shorter track lengths are not unique, the tracks corresponding to all points to be complemented in the tracks to be marked cannot be accurately determined, and further, the first intention marking information of each position point on the known tracks cannot be determined by utilizing the complementing position points, so that the known tracks with shorter track lengths have no reference value, and the known tracks are discarded, so that the known tracks with shorter track lengths and no determined intention marking information can be screened out.

During specific implementation, the known track with the track length larger than the preset length can be screened out according to the track length of the known track, then the track expression of the known track and the curve corresponding to the track expression are determined according to the screened out known track, and the track to be marked can be completed by utilizing the curve, so that the completed track to be marked is obtained. The track expression may be a cubic expression, a quartic expression, or the like.

In addition, the embodiment of the present disclosure provides a trajectory prediction method, which may output a predicted trajectory of a second target object, where the predicted trajectory of the second target object may be output by a trained predictive neural network, and the trained predictive neural network is obtained by training intent marking information obtained based on the intent prediction method.

As shown in fig. 2, a flowchart of a trajectory prediction method provided in an embodiment of the present disclosure may include the following steps:

s201: and acquiring a motion track and a prefabricated map of a second target object.

Here, the second target object may be a vehicle having various traveling intentions on the road. The motion trajectory is a trajectory that the second target object has generated during a certain time interval in the past, for example, the motion trajectory may be a trajectory that the target object has generated during the past 5 seconds. The prefabricated map may be a high-precision map generated in advance about a road where the target object is located, and in specific implementation, the prefabricated map may be an urban road map of a city corresponding to the road where the target object is located.

In one embodiment, the trajectory prediction method provided by the embodiment of the disclosure may be applied to a driving device, for example, the driving device may be an autonomous vehicle. The motion trajectory may be determined according to state information corresponding to the second target object, where the state information may include information such as a speed, an acceleration, a road where the second target object is located and a position on the road, appearance information of the second target object (e.g., information about a head and a tail of the target object, etc.). The state information of the second target object may be information related to a scene in which the second target object is located, which is captured by various sensors installed while the traveling apparatus travels, and in addition, the state information of the second target object may further include state information of other objects adjacent to the second target object.

The pre-map may be stored in a storage system in the traveling apparatus in advance, or may be acquired by the traveling apparatus on demand through a network according to a real-time traveling position, and is not limited herein.

S202: and determining at least one piece of movement intention information of the movement track in a future preset time period based on the movement track and the prefabricated map.

Here, the movement intention information is information for representing the movement intention of the second target object within a preset period in the future from the end point of the movement trajectory and a probability to which the movement intention corresponds, and for example, the movement intention may include left-right turn, u-turn, straight line, still, and the like.

After the motion track and the pre-prepared map of the second target object are obtained, depth feature information of a scene where the second target object is located may be determined according to state information and the pre-prepared map corresponding to the motion track, where the depth feature information may reflect depth features of the scene where the second target object is located, for example, depth features corresponding to motion tracks and positions of other objects adjacent to the second target object, depth features of roads corresponding to the motion tracks, and the like. During specific implementation, according to the state information corresponding to the motion track and the pre-fabricated map, it can be determined that the motion track corresponds to road information in the pre-fabricated map, and the road information may include coordinate information of a road route, a road identifier of a road on which the second target object runs, and a driving rule of the road. And based on the depth feature information and the prefabricated map, it may also be determined that the motion trajectories of other objects adjacent to the second target object correspond to road information, positions, and the like in the prefabricated map, where the coordinate information of the road route may include coordinates of a center line of a lane, coordinates of a sideline (sidewalk), and the like, the road identifier may include a road name and a name or a serial number of each lane on the road, and the driving rule may include rules of going straight, turning left and right, dropping head, and the like, and then the depth feature of the scene where the second target object is located may be determined according to the above information. The future preset time period may be determined according to the road information or may be preset by a developer, for example, the future preset time period may be 10 seconds in the future. The determination method of the future preset period is not limited herein.

Then, based on the road information of the second target object and other objects, the motion trajectories of the second target object and other objects, and appearance information, positions, and the like of the second target object and other objects in the determined depth feature information, at least one piece of movement intention information of the second target object in a future preset period may be determined. Here, since there may be potentially various kinds of movement intentions of the second target object during the traveling, in order to achieve accurate prediction of the movement intention information of the second target object, a plurality of different movement intention information may be determined in combination with the above information.

In a specific implementation, the movement intention information corresponding to the road rule may be determined according to the road rule corresponding to the determined road information, the movement intention information of other objects may be determined according to the movement tracks, positions and the like of other objects adjacent to the second target object, and then at least one piece of movement intention information of the second target object in a future preset time period may be determined by combining the movement intention information of the second target object when the movement track is generated.

In addition, the depth feature information can predict the output of any trajectory prediction feature extractor in the neural network, and the motion intention information can be output by an intention prediction branch in the neural network.

S203: based on the at least one movement intention information, at least one predicted trajectory of the second target object within a future preset time period is determined.

Here, different pieces of movement intention information may be used to guide the generation of different predicted trajectories, and the same piece of movement intention information may also be used to guide the generation of a plurality of different predicted trajectories, each predicted trajectory corresponding to one piece of movement intention information. Therefore, for any one of the determined at least one motion intention information, at least one predicted trajectory that conforms to the motion intention information may be determined based on the motion intention information. In the case where the determined exercise intention information includes a plurality of pieces, at least one predicted trajectory corresponding to each of the plurality of pieces of exercise intention information may be determined.

However, in order to better fit various possible driving trajectories corresponding to various potential movement intentions of the second target object and improve the safety of automatic driving, the trajectory prediction method provided by the embodiment of the disclosure may simultaneously generate a plurality of predicted trajectories corresponding to each movement intention information.

In addition, when implemented, the trajectory prediction method may be performed by a trained predictive neural network, and the predicted trajectory may be output by a trajectory prediction branch in the trained predictive neural network.

In this way, based on the motion trail of the target object, the motion intention information of the target object in the process of generating the motion trail can be determined; based on the high-precision prefabricated map, it can be determined that the movement locus corresponds to road regulation information of roads in the prefabricated map, and the road regulation information can play a role of indicating the movement intention of the target object, for example, a left-turn road can indicate the target object to make a left turn. Furthermore, by combining the movement intention information and the road rule information in the process of generating the movement track, the movement intention information of the target object in a future preset time period can be accurately predicted; and, because the predicted track is generated based on the movement intention information, the generated predicted track can be matched with the movement intention information, and the generated predicted track has higher accuracy on the basis of the accuracy of the movement intention information. In addition, the movement intention information is generated according to the pre-prepared map, so that the generated movement intention information may include a plurality of movement intention information when the generated movement intention information conforms to the road rule information, furthermore, when the generated movement intention information includes a plurality of movement intention information, the prediction tracks matched with each piece of movement intention information can be respectively generated, the diversity and difference of the generated prediction tracks are increased, the various prediction tracks can better fit various potential driving tracks of the target object, and the safety of automatic driving is further improved.

In one possible embodiment, for S203, determining the predicted trajectory of the second target object further requires incorporating depth feature information of the scene. Therefore, after determining the depth feature information of the scene in which the second target object is located, at least one predicted trajectory of the second target object within a future preset time period may be determined according to the depth feature information of the scene and the determined movement intention information.

In addition, after generating at least one predicted trajectory of the second target object within a future preset period, the predictive neural network may further output a predicted image with respect to the predicted trajectory of the second target object.

In one possible embodiment, the movement intention information may include a movement intention of the second target object and a probability corresponding to the movement intention, and in particular, the output form of the movement intention information may be a form of a probability distribution of the movement intention.

In S203, for each piece of determined at least one piece of movement intention information, at least one predicted trajectory of the second target object within a future preset time period may be determined based on the movement intention included in the movement intention information, the probability corresponding to the movement intention, and the depth feature information.

In a specific implementation, in the case that the determined motion intention information includes a plurality of pieces, the motion intention and the probability corresponding to the motion intention, which may be determined by the intention prediction branch in the prediction neural network, included in each piece of motion intention information may be based on the motion intention and the probability corresponding to the motion intention, the motion intention with the highest probability is selected as the motion intention that requires outputting the predicted trajectory, and then at least one predicted trajectory corresponding to the motion intention is generated according to the motion intention with the highest probability and by combining the depth feature information.

Or the probabilities corresponding to the movement intentions in each piece of determined movement intention information may be ranked, the movement intentions corresponding to the probabilities that the ranking order satisfies the preset ranking order are selected, and then at least one predicted track corresponding to each screened movement intention is generated by combining the depth feature information.

Still alternatively, the determined probabilities corresponding to the movement intentions included in each piece of movement intention information may be ranked, and the predicted trajectory corresponding to each movement intention may be generated based on the ranking order and the depth feature information, where the number of predicted trajectories corresponding to movement intentions that are generated before the ranking order may be greater than the number of predicted trajectories corresponding to movement intentions that are generated after the ranking order.

In an embodiment, after determining at least one predicted trajectory of the second target object within a future preset time period, the driving device that obtains the motion trajectory of the second target object may be further controlled to avoid the second target object during the next driving process based on the at least one predicted trajectory of the second target object within the future preset time period, or the driving device may be controlled to send out prompt information to prompt a driver and/or a passenger in the second target object, so as to improve the safety of the automatic driving.

As shown in fig. 3a, the schematic diagram of the Trajectory of the target object output by the neural network for prediction provided by the embodiment of the present disclosure includes state information reflecting Motion trajectories of the second target object and other objects adjacent to the second target object and related information of a scene in which the second target object is located, a high-precision Map corresponding to a road in which the second target object is located, which is represented by a, determined Motion Trajectory and appearance information (Motion) of the second target object, information (Social) of positions and Motion trajectories of other objects adjacent to the target object, an arbitrary Trajectory prediction feature extractor Encoder, extracted depth feature information (Agent Emb), a prediction Stage Evaluation Stage, an Intention prediction Branch Evaluation Branch, a predicted Motion Intention information Evaluation, probabilities (Evaluation on-hot vectors) corresponding to the Motion Intention, a Trajectory prediction Branch Evaluation Branch, the generated predicted Trajectory track. The Encoder is a Deep Neural Network (Deep Neural Network) for extracting depth feature information Agent Emb, and the target Branch is used for generating a predicted track of the target object. As shown in fig. 3b, a schematic diagram of a predictive image of a predicted track of a second target object is output for a predictive neural network provided by the embodiment of the present disclosure, where a line segment a, a line segment b, a line segment c, and a line segment d represent predicted tracks generated by the predictive neural network.

In addition, since the predicted trajectory is output by the trained predicted neural network, the method provided by the embodiment of the present disclosure further includes a method for training the predicted neural network to be trained by using the sample trajectory and the sample map, as shown in fig. 4, a flowchart of the method for training the predicted neural network to be trained provided by the embodiment of the present disclosure may include the following steps:

s401: and inputting a part of sample tracks of the sample objects and a sample map into a prediction neural network to be trained.

S402: based on the partial track in the sample track and the sample map, prediction intention information of the sample object is determined.

S403: and determining the intention marking information of the sample object in a future preset time period from the partial track.

S404: and training the predictive neural network to be trained based on the predictive intention information and the intention marking information in the future preset time period to obtain the trained predictive neural network.

The sample track is a motion track which is generated by an acquired sample object, a partial track which is input into the prediction neural network to be trained must have a subsequent track in the sample track, and the subsequent track cannot be a partial track containing the tail end track of the sample track. The sample map can be a high-precision map corresponding to the road where the sample object is located when the sample track is generated.

In specific implementation, part of the sample trajectory and scene information may be input in the form of information that can be directly recognized by the predictive neural network to be trained.

After acquiring a partial track and a sample map in the sample track, the predictive neural network to be trained may determine the prediction intention information of the sample object according to the partial track and the sample map, where the prediction intention information may include the prediction intention and a probability corresponding to the prediction intention. In addition, after the sample track is obtained, the intention labeling information corresponding to the sample track can be determined, and based on the intention labeling information corresponding to the sample track, the sample track and the partial track, the intention labeling information of the sample object in a future preset time period from the partial track can be determined.

Furthermore, according to the prediction intention information and the intention label information in the future preset time period, the intention label information in the future preset time period is used as a true value of training, a first loss value between the prediction intention information and the intention label information is determined, network parameters related to intention prediction in the prediction neural network to be trained are adjusted based on the first loss value until a preset training cut-off condition is met, the trained prediction neural network can be obtained, and therefore the trained prediction neural network can output accurate prediction intention information.

Thereafter, the predictive neural network to be trained may generate a sample predicted trajectory of the sample object based on the determined prediction intent information. Based on the sample predicted trajectory and the sample trajectory generated by the sample object in a future preset time period after the sample predicted trajectory and the partial trajectory in the sample trajectory are generated, taking the sample trajectory corresponding to the future preset time period as a true value of training, determining a second loss value between the sample predicted trajectory and the sample trajectory corresponding to the future preset time period, adjusting network parameters related to trajectory prediction in the predictive neural network to be trained based on the second loss value until a preset training cutoff condition is met, and obtaining the trained predictive neural network, so that the trained predictive neural network can output an accurate predicted trajectory.

In addition, if the number of position points missing on the sample track generated within a future preset time period after the sample object generates the partial track in the sample track is greater than a preset position point threshold, the sample prediction track corresponding to the sample track is ignored and is not used for constructing the prediction loss function. And if the number of the missing position points is less than a preset position point threshold value, after the corresponding sample prediction track in the sample track is determined, determining the predicted position point corresponding to the missing position point in the sample prediction track, and constructing a prediction loss function by using the position points in the sample track except the predicted position point and the position points on the sample track generated by the sample object in a future preset period.

In an embodiment, regarding the process of training the predictive neural network to be trained by using the intention loss function and the predictive loss function, the predictive neural network to be trained may be trained by using the intention loss function first, and after it is determined that the trained predictive neural network can output accurate predictive intention information, the predictive loss function is constructed to train the predictive neural network to be trained, so as to obtain the trained predictive neural network. Alternatively, the intention loss function and the prediction loss function may be simultaneously constructed, and the intention loss function and the prediction loss function are used to train the to-be-trained prediction neural network synchronously, so as to obtain the trained prediction neural network, which is not limited herein.

In one embodiment, the intention label information includes first intention label information, wherein the first label information can be determined by the intention prediction method provided by the embodiment of the disclosure.

For S403, after obtaining the sample trajectory of the sample object, the first intention labeling information of the sample object starting from the partial trajectory within a preset time period in the future may be determined by using the intention prediction method provided in the embodiments of the present disclosure. In addition, in the case where the intention icon information includes the first intention label information, the predicted intention information may be fine-grained intention information that reflects intention information of each position point on a trajectory generated by the second target object after a future preset time period has elapsed.

In another embodiment, the intention label information includes second intention label information.

For S403, after obtaining the sample trajectory of the sample object, the first intention labeling information of the sample object starting from the partial trajectory within a preset time period in the future may be determined by using the intention prediction method provided in the embodiments of the present disclosure. Then, the second intention label information may be determined according to an intention converting rule between the second intention label information and the first intention label information. For example, the intention conversion rule may be that the first intention label information of a certain position point in the sample track is taken as the intention label information of the sample object after a preset period (for example, 3 seconds) at the time corresponding to the position point. Furthermore, for any position point in each position point on the sample track corresponding to the future preset time period, the second intention marking information of the sample track corresponding to the future preset time period may be determined according to the intention conversion rule and the determined first intention marking information, that is, the second intention marking information in the future preset time period may be determined.

The second intention labeling information can reflect the intention of the sample object on a certain section of sample track, and the intention of each position point on the section of sample track is the same; the first intention labeling information may reflect the intention of the sample object at each location point on the sample trajectory, wherein the intention of each location point may not be the same.

In the case where the intention annotation information includes second intention tagging information, the predicted intention information may be a coarse-grained intention information.

In addition, the intention label letter can also comprise third intention label information labeled manually. In this way, the predictive neural network to be trained can be trained directly using the third intention labeling information.

Regarding to determining the first intention labeling information of each position point on the sample track, in the prior art, a manual labeling method is adopted, only the second intention labeling information corresponding to the sample track can be determined, and the first intention labeling information of each position point on the sample track cannot be obtained. In addition, the manual labeling method not only consumes a large amount of labor cost and reduces the labeling efficiency, but also has the condition of labeling errors, thereby influencing the accuracy of the trained predicted trajectory output by the prediction neural network.

Therefore, in order to solve the above problem, the trajectory prediction method provided by the embodiment of the present disclosure further includes a step of automatically determining the first intention label information of each location point on the sample trajectory:

step one, determining key point pairs in a sample track based on the sample track and a sample map;

determining first intention labeling information corresponding to each key point pair;

and thirdly, determining first intention marking information of each position point on the sample track based on the sample map and the first intention marking information corresponding to the key point pairs.

After the sample track and the sample map are obtained, sample sub-tracks corresponding to different preset road categories and included in the sample track can be determined according to the sample map, then, key point pairs in each sample sub-track can be determined by using a key point pair determination method corresponding to the preset road category based on the preset road categories corresponding to the different sample sub-tracks, and first intention labeling information of each key point pair is determined by using an intention determination method corresponding to the preset road category.

After the first intention labeling information of each key point pair is determined, the first intention labeling information of each position point on the sample track can be determined according to the preset road category corresponding to each sample sub-track in different sample sub-tracks included in the determined sample track and the first intention labeling information of the corresponding key point pair in the sample sub-track. Therefore, the intention labeling of each position point on the sample track can be realized in a fine-grained manner, and the real intention of each position point of the sample object on the sample track can be reflected, so that the accuracy of predicting the predicted track output by the neural network is improved, in addition, the whole process can be automatically finished, and the labeling efficiency is improved. In addition, after the first intention labeling information of each position point on the sample track is determined, second intention labeling information of the sample track can be determined based on an intention conversion rule, and the to-be-trained predictive neural network is trained by using the second intention labeling information of the sample track, so that the trained predictive neural network is obtained.

In one embodiment, the key point pair may include a lane change key point pair, the lane change key point pair includes a lane change starting point and a lane change ending point, and the preset road category includes a first preset road category, where the first preset road category may be a general straight road category. Aiming at the first step, a sample sub-track corresponding to the first preset road category in the sample track can be screened out by using a sample map, the sample sub-track is used as a first sample track, and then a track curve corresponding to the first sample track can be processed by using a B-spline interpolation method to supplement a middle point on the track curve. Therefore, the continuity of the track curve can be enhanced, and the accuracy of the determined channel-changing key point pairs can be improved by using the track curve with good continuity.

After B-spline interpolation is performed on the trajectory curve corresponding to the first sample trajectory, road information of a road in the sample map where the sample object generating the first sample trajectory is located may be determined by using the sample map.

Based on the road identifier in the road information, a first sample track with the changed road identifier may be determined, and then a position point corresponding to the first sample track when the road identifier is changed may be determined, and the position point is used as a lane change point of the first sample track, and a time when the road identifier is changed is used as time information of the lane change point. Then, determining the time which is before the time corresponding to the time information of the lane changing point and has the preset time interval with the time corresponding to the time information of the lane changing point by using the preset time interval, and taking the time as the starting time information corresponding to the preset starting point; then, the time after the determined time corresponding to the time information of the lane change point and having the preset time interval with the time corresponding to the time information of the lane change point can be used as the end time information corresponding to the preset end point. Here, the preset time interval may represent a time interval from when the lane change starts to when the lane change point is reached, or from a time corresponding to the lane change point to when the lane change ends, and in general, the sample object may complete the lane change for a time length corresponding to two preset time intervals.

Further, based on the time information of each position point in the sample track, a position point in the sample track where the time information is the start time information is used as a preset starting point, and a position point in the first sample track where the time information is the end time information is used as a preset end point. And then, determining a preset lane changing track of the sample object from the first sample track according to the preset starting point and the preset end point. Therefore, the lane changing track of the sample object can be determined more accurately based on the preset starting point, the lane changing point and the preset end point.

In one embodiment, after determining the preset starting point, the lane change point and the preset ending point, the lane change key point positioning algorithm may be used to determine the real lane change starting point and the real lane change ending point of the sample object from the preset lane change track, and further determine the real lane change track of the sample object, wherein the lane change key point positioning algorithm may be a Distance-to-finger algorithm.

In specific implementation, for a lane change starting point, in a preset lane change trajectory curve corresponding to a preset lane change trajectory, the determined preset lane change starting point and a lane change point may be connected to obtain a target connecting line, then, perpendicular lines may be drawn to the target connecting line with each position point of at least part of position points in the first sample trajectory as a starting point, a distance of each perpendicular line is determined, the distance of the perpendicular line corresponding to each position point is taken as a vertical distance corresponding to each position point, and then, a position point corresponding to the largest vertical distance may be selected as the lane change starting point. Here, at least a part of the position points in the first sample track are all or part of the position points in the preset lane change track between the preset lane change starting point and the lane change point.

In addition, after determining the lane change starting point, a preset threshold may be used to screen out a lane change starting point whose vertical distance is greater than the preset threshold, and take such a lane change starting point as a final lane change starting point, and for a lane change starting point whose vertical distance is less than the preset threshold, the lane change starting point is discarded, and it is determined that no lane change key point pair exists in the first sample trajectory corresponding to the lane change starting point, in a specific implementation, the preset threshold may be 0.8 m, which is less than half of the width of the small and medium-sized sample object. Therefore, the target object corresponding to the lane change starting point filtered by the preset threshold value is most likely to move on the lane change line where the lane change point is located, namely, whether the sample object has the real lane change intention or not can be determined by the preset threshold value, and therefore the reasonability of the determined lane change starting point is improved.

For the lane change end point, the determined preset lane change end point and the determined lane change point can be connected to obtain a target connecting line corresponding to the lane change end point, then the vertical distance between each position point in at least part of position points in the first sample track and the target connecting line corresponding to the lane change end point can be determined, and the position point corresponding to the maximum vertical distance is selected as the lane change end point. Here, at least a part of the position points in the first sample trajectory are all or part of the position points in the preset switch trajectory between the preset switch end point and the switch point.

In addition, after determining the lane change end point, a lane change end point with a vertical distance greater than a preset threshold corresponding to the lane change end point may be screened out by using a preset threshold corresponding to the lane change end point, and such a lane change end point may be used as a final lane change end point, and for a lane change end point with a vertical distance less than the preset threshold corresponding to the lane change end point, the lane change end point is discarded, and it is determined that no lane change key point pair exists in the first sample trajectory corresponding to the lane change end point, and in a specific implementation, the preset threshold corresponding to the lane change end point may be 0.64 meters. Therefore, the target object corresponding to the lane change end point, which is filtered by the preset threshold corresponding to the lane change end point, most probably moves on the lane change line where the lane change point is located, and then whether the sample object has the real lane change intention or not can be determined by the preset threshold, so that the reasonability of the determined lane change end point is improved.

Based on the above process, a lane change start point and a lane change end point in the first sample trajectory may be determined. After determining the lane change starting point and the lane change ending point in the first sample track, a displacement direction between the lane change starting point and the lane change ending point may be determined based on the sample map, and then, intention information corresponding to the displacement direction may be determined.

Further, the lane change starting point and the lane change ending point in the first sample track may be used as a pair of lane change key points in the first sample track, and the intention information corresponding to the determined displacement direction between the lane change starting point and the lane change ending point may be used as the first intention marking information of the pair of lane change key points.

In one embodiment, the key point pairs may further include intersection key point pairs, where the intersection key point pairs include entry points and exit points, and the preset road categories include a second preset road category, where the second preset road category may be a road category including intersections, U-shaped intersections, and the like. And aiming at the first step, a sample sub-track corresponding to a second preset road category in the sample track can be screened out by using a sample map, the sample sub-track is used as a second sample track, then the road position of the second sample track in the sample map can be determined by using the sample map, and the intersection entrance point information and the intersection exit point information in the second sample track can be determined based on the determined road position.

Further, a position point corresponding to the intersection entrance point information in the second sample track may be determined, and the point may be used as the entrance point, a position point corresponding to the intersection exit point information in the second sample track may be determined, and the point may be used as the exit point, and then, included angle information between a road where the exit point is located and a road where the entrance point is located may be determined, where the included angle information includes an included angle and a direction of the included angle (clockwise and counterclockwise), the direction of the included angle is a direction from the road where the exit point is located to the road where the entrance point is located, and the angle of the included angle is an angle between the roads to which the direction of the included angle points. Then, based on the angle and the direction of the included angle corresponding to the included angle information, intention information corresponding to the included angle information can be determined, for example, if the direction of the included angle corresponding to the included angle information is a clockwise direction and the angle of the included angle is 30 degrees to 150 degrees, the intention information of the included angle information is a left turn intention; if the direction of the included angle corresponding to the included angle information is clockwise or counterclockwise and the included angle exceeds 150 degrees, the intention information of the included angle information is a U-shaped turning intention, and if the included angle corresponding to the included angle information is within plus or minus 30 degrees, the intention information of the included angle information is a straight intention.

Furthermore, the entry point and the exit point in the second sample track may be used as a key point pair of the intersection in the second sample track, and the intention information corresponding to the determined included angle information between the entry point and the exit point may be used as the first intention label information of the key point pair of the intersection.

Then, after determining the first intention labeling information corresponding to the key point pair, the first intention labeling information of each position point on the sample track can be determined according to the following steps:

determining different sample sub-tracks corresponding to different preset road types and included in a sample track based on a sample map;

and secondly, determining first intention labeling information of each position point on the sample track based on the first intention labeling information corresponding to the key point pairs and the preset road category corresponding to the sample sub-track.

In specific implementation, based on a sample map, a first sample track and a second sample track included in a sample track may be determined, and then, for each first sample track, all sixth location points located between the lane change key point pairs corresponding to the first sample track in the first sample track may be determined first, where the sixth location points are necessarily located in the lane change track, and therefore, the first intention tagging information of the sixth location points is necessarily the same as the first intention tagging information of the lane change key point pairs, and further, the first intention tagging information of the lane change key point pairs corresponding to the first sample track may be used as the first intention tagging information of the sixth location points located between the lane change key point pairs corresponding to the first sample track in the first sample track.

For each second sample track, if the sample object is located at the intersection, the second track generated in the intersection is bound to start at the intersection entry point and end at the intersection exit point, and the second sample track is determined by the intersection entry point and the intersection exit point, so that the first intention labeling information of each position point in the second sample track is bound to be consistent with the first intention labeling information of the intersection key point corresponding to the second sample track, and therefore, the first intention labeling information of the intersection key point pair corresponding to the second sample track can be used as the first intention labeling information of each position point located between the intersection key point pairs corresponding to the second sample track in the second sample track.

For the case that the length of the track change track corresponding to the first sample track is smaller than the length of the first sample track, the first sample track necessarily has a position point which is not located between the track change key point pair corresponding to the first sample track, and therefore, when determining the first intention labeling information of each position point in the sample track, it is further necessary to determine the first intention labeling information of the seventh position point which is not located between the track change key point pair corresponding to the first sample track, the first intention labeling information of the eighth position point which is not belonging to the first sample track and not belonging to the second sample track in the sample track, and the first intention labeling information of the ninth position point in the first sample track which is not the track change key point pair. In particular, for each of the seventh location point, the eighth location point and the ninth location point, the first intention marking information of the location point can be determined according to the speed information of the location point. For example, if the speed corresponding to the speed information of the position point is less than the preset speed threshold, it indicates that the sample object at the position point is in a stationary state, the first intention marking information of the position point may be determined as the stationary intention, and if the speed corresponding to the speed information of the position point is greater than the preset speed threshold, it indicates that the sample object at the position point is in a straight-moving state, the first intention marking information of the position point may be determined as the straight-moving intention.

Based on this, the first intention marking information of each position point in the sample track can be determined.

In an embodiment, before determining key point pairs in a sample trajectory, for a discontinuous sample trajectory, a continuous known trajectory in the sample trajectory may be determined, then, for determining the known trajectory, the sample trajectory may be completed in a manner of determining a trajectory expression of the known trajectory, and after obtaining the completed sample trajectory, the completed sample trajectory may be screened by using preset completion limit information and a sample map, so as to obtain a final and reasonable candidate sample trajectory. The preset completion limit information may include that the completed sample trajectory is located on a road in the sample map, and the completed sample trajectory conforms to a driving rule of the road where the completed sample trajectory is located, where the driving rule may include turning left and right, dropping head, and the like. In specific implementation, based on the sample map, the supplemented sample track can be determined to correspond to the road information in the sample map, then according to the road identification in the road information, the track of the supplemented sample track separated from the road in the sample map can be screened out, and the sample object can not be separated from the road in the driving process, so that the supplemented sample track separated from the track of the road in the sample map is an unreasonable track and has no reference value, and the supplemented sample track is discarded.

According to the preset completion limit information that the completed sample track conforms to the driving rule of the road where the completed sample track is located, the completed sample track which does not conform to the driving rule can be screened out, for example, the completed sample track is a track where the sample object moves in a reverse direction on the road corresponding to the track, and the sample object conforms to the driving rule of the road where the sample object is located in the driving process, so that the screened completed sample track which does not conform to the driving rule is also an unreasonable track, has no reference value, and the completed sample track is discarded.

Then, based on the step of determining the first intention labeling information of each position point on the sample track, the intention labeling of the position point on the known track in the candidate sample track may be performed by using the complementing position point in the candidate sample track, where, since the complementing position point in the candidate sample track is predicted and may be inaccurate, the intention labeling is performed only on the position point on the known track by using the complementing position point, and the intention labeling is not performed on the complementing position point. Then, after determining the first intention tagging information of the position point on the known one of the candidate sample trajectories, the known one of the candidate sample trajectories may be taken as a final sample trajectory. In this way, partial discontinuous sample tracks can be reserved, and each position point on the reserved sample tracks is subjected to intention marking through the completion position point, so that the number of first intention marking information is increased, and the reasonability of the determined first intention marking information can be improved; and according to the first intention labeling information and the intention conversion rule of each position point on the sample track, second intention labeling information corresponding to the sample track can be determined, so that the number of the second intention labeling information for training is increased, and the training effect of training the predictive neural network to be trained is improved.

In another embodiment, after the completed sample track is obtained, intention labeling may also be performed on each position point on the completed sample track to determine first intention labeling information of each position point on the completed sample track, and then, second intention labeling information corresponding to the point on the completed sample track may be determined. Then, the completed sample trajectory may be used as a final sample trajectory, and the corresponding second intention labeling information may be used as a true value of the intention training, so as to train the predictive neural network to be trained.

In addition, after the continuous known tracks in the sample tracks are determined, the track expressions of the known tracks with shorter track lengths are not unique, the tracks corresponding to all the points to be complemented in the sample tracks cannot be accurately determined, and further the first intention labeling information of each position point on the known tracks cannot be determined by utilizing the complemented position points, so that the known tracks with shorter track lengths have no reference value, and the known tracks are discarded, so that the known tracks with shorter track lengths and without the determined intention labeling information can be screened out.

In specific implementation, a known track with a track length larger than a preset length can be screened out according to the track length of the known track, then a track expression of the known track and a curve corresponding to the track expression are determined according to the screened known track, and the curve can be used for completing a sample track to obtain a completed sample track. The track expression may be a cubic expression, a quartic expression, or the like.

It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.

Based on the same inventive concept, the embodiment of the present disclosure further provides an intention and trajectory prediction apparatus corresponding to the intention and trajectory prediction method, and since the principle of the apparatus in the embodiment of the present disclosure for solving the problem is similar to the intention and trajectory prediction method described above in the embodiment of the present disclosure, the implementation of the apparatus may refer to the implementation of the method, and repeated details are not described again.

As shown in fig. 5, a schematic diagram of an intent prediction apparatus provided in an embodiment of the present disclosure includes:

a first determining module 501, configured to determine, based on a track to be annotated and a target map, a target key point pair in the track to be annotated;

a second determining module 502, configured to determine first intention labeling information corresponding to each of the target key point pairs;

a third determining module 503, configured to determine, based on the target map and the first intention labeling information corresponding to the target key point pair, first intention labeling information of each location point on the track to be labeled.

In a possible implementation manner, the third determining module 503 is configured to determine, based on the target map, different sub-tracks included in the track to be annotated and corresponding to different preset road categories;

In a possible implementation manner, the third determining module 503 is configured to use a sub-track corresponding to a first preset road category in the track to be labeled as a first track;

the target key point pairs comprise channel-changing key point pairs; the first determining module 501 is configured to determine, based on the target map, a lane change point in the first track where a road sign changes;

In a possible implementation manner, the first determining module 501 is configured to determine, based on the time information corresponding to the lane change point and a preset time interval, start time information corresponding to a preset start point and end time information corresponding to a preset end point;

In a possible implementation manner, the first determining module 501 is configured to connect the preset starting point and the lane changing point to obtain a target connection line;

In a possible implementation manner, the first determining module 501 is further configured to, after the position point corresponding to the maximum vertical distance is taken as the lane change starting point, take the lane change starting point where the vertical distance is greater than a preset threshold as a final lane change starting point.

In a possible implementation manner, the second determining module 502 is configured to determine, for each lane change key point pair, a displacement direction of the first target object between a lane change starting point and a lane change ending point of the lane change key point pair;

In a possible implementation manner, the third determining module 503 is configured to, for each first track, use the first intention labeling information of the lane changing key point pair corresponding to the first track as the first intention labeling information of the first position point located between the lane changing key point pairs corresponding to the first track in the first track.

In a possible implementation manner, the third determining module 503 is configured to determine, for each first track, a second location point in the first track that is not located between a lane change key point pair corresponding to the first track;

In a possible implementation manner, the third determining module 503 is configured to use a sub-track corresponding to a second preset road category in the track to be labeled as a second track;

the target key point pairs comprise intersection key point pairs; the first determining module 501 is configured to determine intersection entrance point information and intersection exit point information in the second track based on the target map;

In a possible implementation manner, the second determining module 502 is configured to determine, for each intersection key point pair, included angle information between a road where an exit point corresponding to the intersection key point pair is located and a road where an entry point is located;

In a possible implementation manner, the third determining module 503 is configured to, for each second track, use the first intention labeling information of the intersection key point pair corresponding to the second track as the first intention labeling information of the third location point located between the intersection key point pairs corresponding to the second track in the second track.

In a possible implementation manner, the third determining module 503 is configured to use a position point, which does not belong to the first track and does not belong to the second track, in the track to be labeled as a fourth position point;

In one possible implementation, the apparatus further includes a completion module 504;

the completion module 504 is configured to, before the first determining module 501 determines the target key point pair in the to-be-annotated trajectory, determine a trajectory expression of the known trajectory based on a known trajectory in the to-be-annotated trajectory when the to-be-annotated trajectory is discontinuous;

In one possible embodiment, the preset completion limit information includes:

In a possible implementation manner, the completion module 504 is configured to, in a case that the to-be-annotated track is discontinuous, screen a known track with a track length of a known track being greater than a preset length from known tracks in the to-be-annotated track, and determine a track expression of the known track.

As shown in fig. 6, a schematic diagram of a trajectory prediction apparatus provided in an embodiment of the present disclosure includes:

an obtaining module 601, configured to obtain a motion trajectory and a prefabricated map of a second target object;

a fourth determining module 602, configured to determine at least one piece of movement intention information of the movement track in a future preset time period based on the movement track and the prefabricated map;

a fifth determining module 603, configured to determine, based on the at least one motion intention information, at least one predicted trajectory of the second target object within a future preset time period.

In a possible implementation manner, the fourth determining module 602 is configured to determine depth feature information of a scene where the second target object is located, based on the state information corresponding to the motion trajectory and the prefabricated map;

the fourth determining module 602 is configured to determine at least one predicted trajectory of the second target object within a future preset time period based on the depth feature information and the at least one motion intention information.

the fourth determining module 602 is configured to determine, for each piece of movement intention information, at least one predicted trajectory of the second target object within a future preset time period based on the movement intention, the probability corresponding to the movement intention, and the depth feature information.

In one possible implementation, the apparatus further comprises a training module 604;

the training module 604 is configured to train a predictive neural network to be trained according to the following steps:

the training module 604 is configured to determine, by using the intention prediction method in the foregoing embodiment, first intention marking information of the sample object in a future preset time period from the partial trajectory.

the training module 604 is configured to determine, by using the intention prediction method in the foregoing embodiment, first intention labeling information of the sample object in a future preset time period from the partial trajectory;

In one possible implementation, the apparatus further comprises a control module 605;

the control module 605 is configured to, after the fifth determining module 603 determines at least one predicted trajectory of the second target object in a future preset time period, control a driving device that acquires a motion trajectory of the second target object or send a prompt message based on the at least one predicted trajectory of the second target object in the future preset time period.

The description of the processing flow of each module in the device and the interaction flow between the modules may refer to the related description in the above method embodiments, and will not be described in detail here.

An embodiment of the present disclosure further provides a computer device, as shown in fig. 7, which is a schematic structural diagram of a computer device provided in an embodiment of the present disclosure, and includes:

a processor 71 and a memory 72; the memory 72 stores machine-readable instructions executable by the processor 71, the processor 71 being configured to execute the machine-readable instructions stored in the memory 72, the processor 71 performing the following steps when the machine-readable instructions are executed by the processor 71: s101: determining a target key point pair in the track to be marked based on the track to be marked and the target map; s102: determining first intention labeling information corresponding to each target key point pair, and S103: and determining first intention marking information of each position point on the track to be marked based on the first intention marking information corresponding to the target map and the target key point pairs.

Alternatively, processor 71 performs the following steps: s201: acquiring a motion track and a prefabricated map of a second target object; s202: determining at least one motion intention information of the motion trail in a future preset time period based on the motion trail and the prefabricated map, and S203: based on the at least one movement intention information, at least one predicted trajectory of the second target object within a future preset time period is determined.

The memory 72 includes a memory 721 and an external memory 722; the memory 721 is also referred to as an internal memory, and temporarily stores operation data in the processor 71 and data exchanged with an external memory 722 such as a hard disk, and the processor 71 exchanges data with the external memory 722 through the memory 721.

For the specific execution process of the instruction, reference may be made to the steps of the trajectory prediction and intent tagging method in the embodiment of the present disclosure, which are not described herein again.

The embodiments of the present disclosure also provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program performs the steps of the trajectory prediction and intent annotation method in the above method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.

The computer program product of the trajectory prediction and intent annotation method provided in the embodiments of the present disclosure includes a computer-readable storage medium storing a program code, where instructions included in the program code may be used to execute the steps of the trajectory prediction and intent annotation method described in the above method embodiments, which may be referred to in the above method embodiments specifically, and are not described herein again.

The computer program product may be embodied in hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present disclosure. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

Finally, it should be noted that: the above-mentioned embodiments are merely specific embodiments of the present disclosure, which are used for illustrating the technical solutions of the present disclosure and not for limiting the same, and the scope of the present disclosure is not limited thereto, and although the present disclosure is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive of the technical solutions described in the foregoing embodiments or equivalent technical features thereof within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present disclosure, and should be construed as being included therein. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

1. An intent prediction method, comprising:

2. The method according to claim 1, wherein the determining the first intention labeling information of each position point on the track to be labeled based on the first intention labeling information corresponding to the target map and the target key point pairs comprises:

3. The method according to claim 2, wherein the determining, based on the target map, different sub-tracks included in the track to be labeled and corresponding to different preset road categories comprises:

4. The method of claim 3, wherein determining a lane change start point and a lane change end point in the first trajectory based on the lane change point and the first trajectory comprises:

5. The method of claim 4, wherein determining a lane change start point in the first trajectory based on the preset start point and the lane change point comprises:

6. The method according to claim 5, wherein after the taking the position point corresponding to the largest vertical distance as the lane change starting point, further comprising:

7. The method according to any one of claims 3 to 6, wherein the determining the first intention labeling information corresponding to each of the target key point pairs comprises:

8. The method according to any one of claims 3 to 7, wherein the determining the first intention label information of each position point on the track to be labeled based on the first intention label information corresponding to the target key point pair and the preset road category corresponding to the sub-track comprises:

9. The method according to claim 8, wherein the determining the first intention labeling information of each position point on the track to be labeled based on the first intention labeling information corresponding to the target key point pair and the preset road category corresponding to the sub-track comprises:

10. The method according to claim 3, wherein the determining, based on the target map, different sub-tracks included in the track to be labeled and corresponding to different preset road categories comprises:

11. The method according to claim 10, wherein the determining the corresponding first intention labeling information of each of the target key point pairs comprises:

12. The method according to claim 10, wherein the determining the first intention labeling information of each position point on the to-be-labeled track based on the first intention labeling information corresponding to the target key point pair and the preset road category corresponding to the sub-track comprises:

13. The method according to claim 12, wherein the determining the first intention labeling information of each position point on the to-be-labeled track based on the first intention labeling information corresponding to the target key point pair and the preset road category corresponding to the sub-track comprises:

14. The method according to claim 1, before determining the target key point pair in the track to be annotated, further comprising:

15. The method of claim 14, wherein the pre-set completion limit information comprises:

16. The method according to claim 14, wherein in a case that the trajectory to be labeled is discontinuous, determining a trajectory expression of a known trajectory based on the known trajectory in the trajectory to be labeled comprises:

17. A trajectory prediction method, comprising:

acquiring a motion track and a prefabricated map of a second target object;

18. The method of claim 17, wherein the determining at least one movement intention information of the movement track within a future preset time period based on the movement track and the prefabricated map comprises:

19. The method of claim 18, wherein the athletic intent information includes an athletic intent and a probability corresponding to the athletic intent;

20. The method of claim 17, wherein the predicted trajectory is output by a trained predictive neural network; the prediction neural network is obtained by training sample tracks and sample maps.

21. The method of claim 20, wherein the predictive neural network is trained using the steps of:

22. The method of claim 21, wherein the intent tag information comprises first intent tag information;

determining first intention marking information of the sample object starting from the partial trajectory within a future preset time period by using the intention prediction method of any one of claims 1 to 16.

23. The method of claim 21, wherein the intent tag information comprises second intent tag information;

determining first intention marking information of the sample object starting from the partial track within a preset time period in the future by using the intention prediction method of any one of claims 1 to 16;

24. The method of claim 21, wherein the intention tagging information comprises third intention tagging information that is manually tagged.

25. The method of any one of claims 17 to 24, further comprising, after determining at least one predicted trajectory of the second target object over a future preset time period:

26. An intent prediction device, comprising:

27. A trajectory prediction device, comprising:

28. A computer device, comprising: a processor, a memory storing machine readable instructions executable by the processor for executing the machine readable instructions stored in the memory, the machine readable instructions when executed by the processor, the processor performing the steps of the intent prediction method of any of claims 1-16 or the processor performing the steps of the trajectory prediction method of any of claims 17-25.

29. A computer-readable storage medium, characterized in that a computer program is stored thereon, which, when being executed by a computer device, performs the steps of the intent prediction method according to any one of claims 1 to 16, or performs the steps of the trajectory prediction method according to any one of claims 17 to 25.