WO2021093335A1 - Method for automatically labeling lane changing intention based on high-noise trajectory data of vehicle - Google Patents
Method for automatically labeling lane changing intention based on high-noise trajectory data of vehicle Download PDFInfo
- Publication number
- WO2021093335A1 WO2021093335A1 PCT/CN2020/098164 CN2020098164W WO2021093335A1 WO 2021093335 A1 WO2021093335 A1 WO 2021093335A1 CN 2020098164 W CN2020098164 W CN 2020098164W WO 2021093335 A1 WO2021093335 A1 WO 2021093335A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- vehicle
- lane
- frame
- lane change
- trajectory data
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
- G01C21/34—Route searching; Route guidance
- G01C21/36—Input/output arrangements for on-board computers
- G01C21/3626—Details of the output of route guidance instructions
- G01C21/3658—Lane guidance
Definitions
- the present disclosure belongs to the technical field of unmanned driving vehicles, and in particular relates to a method for automatically labeling a lane changing intention based on trajectory data of a vehicle.
- Autonomous vehicles also referred to as unmanned vehicles, may be driven partly, or entirely, without the assistance of a human driver.
- the autonomous vehicle needs to be able to predict the future intentions and trajectories of surrounding vehicles.
- the traveling intentions of the surrounding vehicles have an important impact on decision-making and planning of the autonomous vehicle, and driving intentions of them may affect each other.
- Human drivers can make predictions about the traveling intentions and future trajectories of surrounding vehicles based on the driver’s own experience and observations. In this way, based upon these predictions, human drivers can make important traveling decisions such as overtaking, decelerating, or changing lanes.
- Existing driving assistance systems usually lack such a predictive capability, which leaves traveling decisions entirely to human drivers.
- it is desirable for unmanned driving vehicles to have such decision-making capability which requires the ability to predict the future intention and trajectory of surrounding vehicles.
- a rule-based algorithm is the “gap acceptance model” which assumes that a lane changing motivation of a driver is based on lead and lag gaps of a target lane. In this method, it is assumed that when the gap reaches the minimum acceptable value, the driver tends to make a lane change.
- the method is characterized by simple and convenient determination of a vehicle intention, such a method requires a large amount of cumbersome and time-consuming fine-tuning of parameters.
- learning-based algorithms use a function or a neural network which may be trained or modeled using a large amount of data. Learning based algorithms require a large amount of training data in order to perfect the model.
- One approach is to manually label public data sets (such as NGSIM US101) , but this requires a large amount of labor, and such public data sets are generally collected by using fixed sensors on the road which is very different from the type of sensor data collected by the sensors of an autonomous vehicle in practice.
- an actual angle of the vehicle is very small during a lane change, and the change of angle is difficult to recognize in a noisy data set; and 2. in this method, a threshold needs to be fixed for determining the change of the angle, but vehicles have different positions and speeds during lane changes, so the method is not suitable for changeable road conditions.
- the present application proposes a method for automatically labeling a lane change intention of a vehicle based on the vehicle’s trajectory data.
- the vehicle whose lane change intention is to be labeled may be a vehicle in the surroundings of an autonomous vehicle and thus may be referred to as a surrounding vehicle.
- the method comprises:
- the labeled frame is labeled as a lane changing frame or a lane keeping frame.
- the tracking trajectory data is collected by a sensor system of an autonomous vehicle in a test drive of the autonomous vehicle. This way the tracking and trajectory data may accurately reflect the type of data an autonomous vehicle will receive in practice.
- This method may be performed for a plurality of surrounding vehicles. In this way tracking trajectory data of a plurality of vehicles can be identified and classified or labeled, thereby generating training data for a learning based vehicle intention and trajectory prediction model.
- the output of the method may comprise labeled tracking and trajectory data including a plurality of frames, with each frame relating to a surrounding vehicle at a point in time, and each frame being labeled as a lane changing frame or a lane keeping frame.
- the approach of this method may be able to cope with noise and is fairly robust as it is able to detect an essence of lane changing behaviors which may then be labeled.
- the method may be implemented by a computing device or processor executing machine readable instructions.
- a second aspect of the present disclosure provides a non-transitory machine readable storage medium storing instructions which are executable by a processor to perform the method of the first aspect of the present disclosure.
- preprocessing of the tracking trajectory data comprises screening the tracking and trajectory data to remove one or more of the following types of tracking and trajectory data: data relating to a surrounding vehicle which cannot be located on a known map, data relating to a surrounding vehicle which is stationary; and data relating to a surrounding vehicle which is more than a predetermined distance away from an autonomous vehicle which collected the tracking and trajectory data.
- data relating to a surrounding vehicle which cannot be located on a known map, data relating to a surrounding vehicle which is stationary; and data relating to a surrounding vehicle which is more than a predetermined distance away from an autonomous vehicle which collected the tracking and trajectory data.
- the method comprises, after determining a starting point of lane change and before outputting at least one labeled frame, verifying an effective lane change lane changing trajectory of the surrounding vehicle.
- verifying the effective lane changing trajectory may comprise discarding lane change trajectories in which a period of time, or number of frames, between the starting point of the lane change and the point of lane change is below a minimum threshold or above a maximum threshold.
- finding the point of lane change of the surrounding vehicle comprises identifying a position of the surrounding vehicle in a plurality of frames corresponding to successive points in time, finding an (n-1) th frame where the surrounding vehicle is in a first lane and a nth frame where the surrounding vehicle is in a second lane and defining the nth frame as the point of lane change of the surrounding vehicle.
- the method calculates an average value of a distance between a vehicle and the road center over time and determines the start of a lane change based on changes in this average value. This use of an average value may further enhance the ability of the method to cope with noise.
- determining the starting point of lane change comprises:
- the present disclosure is extremely robust to extracting lane change trajectories from noisy tracking trajectory data of vehicles, a generalized definition is proposed for lane changing behaviors of vehicles, and labeling validity is verified during a labeling process, so that a large number of correct and effective lane change frames and lane keeping frames can be labeled without human intervention.
- Fig. 1 is a flowchart of a method for automatically labeling a lane changing intention of the present disclosure.
- Fig. 2 is a schematic diagram of a lane keeping trajectory and a lane change trajectory in the tracking data of the vehicle of the present disclosure, where the lane keeping trajectory is on the left side and the lane changing trajectory is on the right side.
- Fig. 3 is a schematic diagram of a Cartesian coordinate system and a Frenet coordinate system.
- Fig. 4 shows determination of a lane changing frame in the method for automatically labeling a lane changing intention of the present disclosure.
- Fig. 5 shows determination of a lane keeping frame in the method for automatically labeling a lane changing intention of the present disclosure.
- Fig. 6 is a schematic diagram of labeling intention of a vehicle on an NGSIM data set in an automatic labeling method in the prior art.
- Fig. 1 is a flowchart of a method for automatically labeling a lane changing intention based on high-noise trajectory data of a vehicle according to the present disclosure, and the method for automatically labeling a lane changing intention includes the following blocks:
- labeled output frames may be generated as a training set for input into a neural network, which may be then be trained to identify a lane change intention of a vehicle.
- This method is fairly robust and may be able to label even high-noise trajectory data.
- High noise trajectory data is data in which the trajectory of a vehicle is not smooth. In practice vehicles often meander rather than traveling in a straight line. As shown in Fig. 2, not every change in angle ends with a lane change, while sometimes a vehicle moves left and right within a lane before finally changing lane. Such data may be considered to be high-noise.
- Example implementations of blocks S1 to S5 will now be discussed.
- Block S1 comprises collecting and preprocessing tracking trajectory data of surrounding vehicles.
- the tracking trajectory data of the surrounding vehicles may be generated by a sensing system of a vehicle during a test drive of the vehicle.
- the tracking trajectory data may include one or more of a map with road information, physical information of the vehicle (for example, a vehicle type, a vehicle size such as a length, a width and a height) , a position of the vehicle (based on a Frenet road coordinate system, or a Cartesian world coordinate system) , a lane in which the vehicle is located, a speed (in one embodiment, the speed may be based on the Frenet road coordinate system) , an orientation of the vehicle (in one embodiment, the orientation may be based on the Frenet road coordinate system) , etc.; which are generated after tracking process is performed on the vehicle after data sensed by a camera, a laser radar, a millimeter wave radar, etc.
- fuse refers to a calculation which combines data derived from separate sources.
- camera and radar data may be fused together so that both an object (such as a surrounding vehicle) and a distance to the object may be detected.
- the tracking trajectory data of the surrounding vehicles may be high-noise tracking trajectory data.
- Fig. 2 shows example of high noise tracking and trajectory data, in which the left side of Fig. 2 depicts a lane keeping trajectory and the right side of Fig. 2 depicts a lane changing trajectory.
- the Frenet road coordinate system may be used.
- the Cartesian coordinate system is usually used to describe a position of an object, but it is not the optimal choice for unmanned driving. Unmanned driving cars usually make acceleration or lane changing decisions based on the road.
- the Frenet coordinate system uses a center line of the road as a reference line, and uses a tangential direction and a normal direction of the reference line to define the coordinate system. As shown in Fig. 3, a distance in the tangential direction S and a lateral distance L of the vehicle relative to the center line of the road are coordinates of the vehicle in the Frenet coordinate system.
- the preprocessing is performed on the tracking trajectory data of the surrounding vehicles.
- the pre-processing is to screen out data and may include:
- Block S2 comprises finding a lane changing point.
- a time point at which the vehicle crosses different lanes may be found, e.g., the location of the vehicle at the (n-1) th frame may be defined as a lane A, and the position of the vehicle at the nth frame may be defined as a lane B, and the nth frame may be defined as the lane changing point of the vehicle.
- a vehicle may have one or more lane changing points along the entire trajectory. In other examples, lane keeping behavior may be maintained along the entire trajectory such that no lane changing points exist.
- a left side of Fig. 4 shows tracking trajectory data of a lane change in a road 400 between a first lane and a second lane.
- Reference numeral 401 indicates a center line of the first lane
- reference numeral 402 indicates a center line of the second lane.
- a trajectory 410 of a vehicle changing lane includes a plurality of points or frames indicated by dots and reference numerals 411.
- the point of lane change 440 is denoted by a triangle in Fig. 4 and has already been determined in block S2 of the method.
- Block S3 comprises determining a starting point of the lane change.
- the method of determining the starting point of lane change may comprise a basic model (a) and a modification of the basic model (b) to enable the model to better cope with noise.
- a basic model is made for a behavior of the lane changing vehicle in the present disclosure: according to this model when the lane changing behavior of the vehicle starts, the vehicle will gradually move towards a target lane, i.e., a lateral distance L of the vehicle from a current lane increases and a lateral distance L of the vehicle from the target lane decreases.
- a target lane i.e., a lateral distance L of the vehicle from a current lane increases and a lateral distance L of the vehicle from the target lane decreases.
- a rhombus 420 in Fig. 4 is an actual starting point of lane change, while positions corresponding to rectangles 430 are misjudged as starting points of lane change according to this basic model. Therefore, the basic model can be improved.
- the improved model (b) is discussed below.
- the present disclosure proposes a method for determining a movement trend of a vehicle.
- An average value Lmean for a plurality of frames (e.g. 5 frames) in coordinates of direction L is used as a basis for determination, which greatly increases robustness of determination of a lane changing behavior.
- the smoother curve 450 on the right in Fig. 4 is a trajectory graph drawn with the alternative Lmean. Compared with the original trajectory 410 adjacent to the curve, it can be seen that the new trajectory graph is more robust to noise.
- Block S4 verifies an effective lane changing trajectory. For example, a lane changing trajectory which is too short or too long (for example, t ⁇ 10 or t > 100) is discarded and is not considered, where t corresponds to a number of frames in the lane changing trajectory. Whether the vehicle is still in the current lane is verified in each frame in (b) in block S3. If another lane change occurs during a lane changing backstepping process, the backstepping is immediately terminated and t is recorded. If t is too short or too long, the track is discarded. This verification can effectively filter out excessive noise or incomplete data.
- a lane changing trajectory which is too short or too long for example, t ⁇ 10 or t > 100
- t corresponds to a number of frames in the lane changing trajectory.
- FIG. 5 shows an example of a lane changing trajectory 510 between a first lane and a second lane of a road 500, in which a center of the first lane is denoted by reference numeral 501 and a center of the second lane is denoted by a reference numeral 502
- a triangle point 520 at the front end is an actual point of lane change
- the other triangles 510 are points of lane change caused by noises.
- the right side of Fig. 5 depicts a lane keeping trajectory 520 on the same road, in which triangles 510 are points of lane change which are falsely identified due to noise.
- lane changing trajectories shown in Fig. 5 are discarded and are not considered, thus ensuring the correctness of the training data. For example, through the back stepping process which traverses frames prior to an identified point of lane change (e.g. one of the triangles) it may be found that the lane changing trajectories shown in Fig. 5 are too short, so they may be discarded.
- an identified point of lane change e.g. one of the triangles
- Block S5 outputs at least one of a labeled lane changing frame and labeled lane keeping frame.
- all frames [n-t, n] from the start of the lane change of the vehicle to the point of lane change are defined as lane changing frames of the vehicle, and the others are lane keeping frames.
- an ending point of lane change is not set, where the ending point of lane change is the point of lane change, because an actual behavior of the vehicle after the point of lane change is lane keeping until the next lane change.
- the method may be performed on a computing device, by executing instructions by a processor of the computing device.
- a non-transitory machine readable storage medium storing instructions which are executable by a processor to perform any of the above methods is provided.
- the technical solutions discussed above are robust in extracting lane changing trajectories from noisy tracking trajectory data of vehicles. They use a generalized definition for lane changing behaviors of vehicles, and validity of a labeling is verified during the labeling process, so that a large number of correct and effective lane changing frames and lane keeping frames can be labeled without human intervention.
- a change in an average value of a distance between a vehicle and the road center over time is used as to determine the start of a lane change. In this way the influence of noise may be reduced and a robust model provided which focuses on an essence of lane changing behaviors.
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Automation & Control Theory (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Traffic Control Systems (AREA)
Abstract
A method for automatically labeling a lane changing intention based on high-noise trajectory data of a vehicle is provided. The method includes: collecting and preprocessing tracking trajectory data of a surrounding vehicle (S1); finding a point of lane change (S2); determining a starting point of lane change (S3); verifying an effective trajectory (S4); and outputting at least one labeled frame of the vehicle (S5).
Description
The present disclosure belongs to the technical field of unmanned driving vehicles, and in particular relates to a method for automatically labeling a lane changing intention based on trajectory data of a vehicle.
Autonomous vehicles, also referred to as unmanned vehicles, may be driven partly, or entirely, without the assistance of a human driver. In order for such unmanned or autonomous driving technologies to be able to safely and efficiently pass through complex traffic scenarios and make optimal decisions, the autonomous vehicle needs to be able to predict the future intentions and trajectories of surrounding vehicles. The traveling intentions of the surrounding vehicles have an important impact on decision-making and planning of the autonomous vehicle, and driving intentions of them may affect each other. Human drivers can make predictions about the traveling intentions and future trajectories of surrounding vehicles based on the driver’s own experience and observations. In this way, based upon these predictions, human drivers can make important traveling decisions such as overtaking, decelerating, or changing lanes. Existing driving assistance systems usually lack such a predictive capability, which leaves traveling decisions entirely to human drivers. However, it is desirable for unmanned driving vehicles to have such decision-making capability which requires the ability to predict the future intention and trajectory of surrounding vehicles.
Existing intention prediction algorithms are generally divided into two types: a rule-based algorithm and a learning-based algorithm. One example of a rule-based algorithm, is the “gap acceptance model” which assumes that a lane changing motivation of a driver is based on lead and lag gaps of a target lane. In this method, it is assumed that when the gap reaches the minimum acceptable value, the driver tends to make a lane change. Although the method is characterized by simple and convenient determination of a vehicle intention, such a method requires a large amount of cumbersome and time-consuming fine-tuning of parameters. In contrast, learning-based algorithms, use a function or a neural network which may be trained or modeled using a large amount of data. Learning based algorithms require a large amount of training data in order to perfect the model.
Training data needs to be labeled and how to obtain a large amount of labeled data is one of the difficulties of such methods. One approach is to manually label public data sets (such as NGSIM US101) , but this requires a large amount of labor, and such public data sets are generally collected by using fixed sensors on the road which is very different from the type of sensor data collected by the sensors of an autonomous vehicle in practice.
In an article published in 2018 and titled “Learning Vehicle Cooperative Lane-changing Behavior from Observed Trajectories in the NGSIM Dataset” (with authors Su, Shuang &Mülling, Katharina &Dolan, John &Palanisamy, Praveen &Mudalige, Priyantha) , an automatic labeling method is used to perform vehicle intention labeling on an NGSIM data set. As shown in Fig. 6, a change in an angle between a vehicle orientation and the road is used as a start sign of a lane change, but this is impractical in actual application, which has the following two problems: 1. an actual angle of the vehicle is very small during a lane change, and the change of angle is difficult to recognize in a noisy data set; and 2. in this method, a threshold needs to be fixed for determining the change of the angle, but vehicles have different positions and speeds during lane changes, so the method is not suitable for changeable road conditions.
SUMMARY
Accordingly the present application proposes a method for automatically labeling a lane change intention of a vehicle based on the vehicle’s trajectory data. The vehicle whose lane change intention is to be labeled may be a vehicle in the surroundings of an autonomous vehicle and thus may be referred to as a surrounding vehicle.
In one example the method comprises:
collecting and preprocessing tracking trajectory data of a surrounding vehicle;
finding a point of lane change of the surrounding vehicle;
determining a starting point of the lane change; and
outputting at least one labeled frame of the surrounding vehicle, wherein the labeled frame is labeled as a lane changing frame or a lane keeping frame.
In one example the tracking trajectory data is collected by a sensor system of an autonomous vehicle in a test drive of the autonomous vehicle. This way the tracking and trajectory data may accurately reflect the type of data an autonomous vehicle will receive in practice. This method may be performed for a plurality of surrounding vehicles. In this way tracking trajectory data of a plurality of vehicles can be identified and classified or labeled, thereby generating training data for a learning based vehicle intention and trajectory prediction model. The output of the method may comprise labeled tracking and trajectory data including a plurality of frames, with each frame relating to a surrounding vehicle at a point in time, and each frame being labeled as a lane changing frame or a lane keeping frame.
The approach of this method may be able to cope with noise and is fairly robust as it is able to detect an essence of lane changing behaviors which may then be labeled.
Due to the large volumes of data, it may not be possible for a human to label the data accurately in a reasonable amount of time and a human labeler may miss subtle indications of lane change, which may be detectable by a computer. Accordingly the method may be implemented by a computing device or processor executing machine readable instructions.
A second aspect of the present disclosure provides a non-transitory machine readable storage medium storing instructions which are executable by a processor to perform the method of the first aspect of the present disclosure.
In one example, preprocessing of the tracking trajectory data, comprises screening the tracking and trajectory data to remove one or more of the following types of tracking and trajectory data: data relating to a surrounding vehicle which cannot be located on a known map, data relating to a surrounding vehicle which is stationary; and data relating to a surrounding vehicle which is more than a predetermined distance away from an autonomous vehicle which collected the tracking and trajectory data. In this way the amount of data to be labeled is reduced thus saving processing power and memory. Furthermore, a richer and more useful training set may be obtained as the most useful portions of the collected data are retained.
In one example, the method comprises, after determining a starting point of lane change and before outputting at least one labeled frame, verifying an effective lane change lane changing trajectory of the surrounding vehicle. This may help to eliminate data which is ambiguous and which may confound a training model. For example, verifying the effective lane changing trajectory may comprise discarding lane change trajectories in which a period of time, or number of frames, between the starting point of the lane change and the point of lane change is below a minimum threshold or above a maximum threshold.
In one example finding the point of lane change of the surrounding vehicle comprises identifying a position of the surrounding vehicle in a plurality of frames corresponding to successive points in time, finding an (n-1) th frame where the surrounding vehicle is in a first lane and a nth frame where the surrounding vehicle is in a second lane and defining the nth frame as the point of lane change of the surrounding vehicle.
In one example, the method calculates an average value of a distance between a vehicle and the road center over time and determines the start of a lane change based on changes in this average value. This use of an average value may further enhance the ability of the method to cope with noise.
In one example, determining the starting point of lane change comprises:
a) determining, for each frame of a plurality of frames, a lane in which the surrounding vehicle is located;
b) determining, for each frame of the plurality of frames, lateral distance L of the surrounding vehicle relative to the lane in which the surrounding vehicle is located;
c) determining, for each frame, an average value of the lateral distance, Lmean, based on an average of the lateral distance L of the current frame a number of previous frames; and
d) determining the starting point of lane change, by traversing frames going backwards in time from the point of lane change, and selecting as the starting point of lane change: a first frame in which the average value of the lateral distance, Lmean, has not decreased relative to the previously traversed frame, or a first frame in which the average value of the lateral distance, Lmean, is at the center of the previous lane.
The advantages of the present disclosure lie in that: the present disclosure is extremely robust to extracting lane change trajectories from noisy tracking trajectory data of vehicles, a generalized definition is proposed for lane changing behaviors of vehicles, and labeling validity is verified during a labeling process, so that a large number of correct and effective lane change frames and lane keeping frames can be labeled without human intervention.
Fig. 1 is a flowchart of a method for automatically labeling a lane changing intention of the present disclosure.
Fig. 2 is a schematic diagram of a lane keeping trajectory and a lane change trajectory in the tracking data of the vehicle of the present disclosure, where the lane keeping trajectory is on the left side and the lane changing trajectory is on the right side.
Fig. 3 is a schematic diagram of a Cartesian coordinate system and a Frenet coordinate system.
Fig. 4 shows determination of a lane changing frame in the method for automatically labeling a lane changing intention of the present disclosure.
Fig. 5 shows determination of a lane keeping frame in the method for automatically labeling a lane changing intention of the present disclosure.
Fig. 6 is a schematic diagram of labeling intention of a vehicle on an NGSIM data set in an automatic labeling method in the prior art.
DETAILED DESCRIPTION OF PREFERRED IMPLEMENTATIONS
The following further describes a method for automatically labeling a lane changing intention based on high-noise trajectory data of a vehicle in conjunction with Figs. 1 to 5. It should be noted that the embodiments described below with reference to the drawings are exemplary and are used to explain the present disclosure, but cannot be construed as limiting the present disclosure.
Fig. 1 is a flowchart of a method for automatically labeling a lane changing intention based on high-noise trajectory data of a vehicle according to the present disclosure, and the method for automatically labeling a lane changing intention includes the following blocks:
S1, collecting and preprocessing tracking trajectory data of surrounding vehicles;
S2, finding a lane change point;
S3, determining a lane change starting point;
S4, verifying an effective lane change trajectory; and
S5, outputting at least one of a labeled lane change frame and a labeled lane keeping frame.
In this way labeled output frames may be generated as a training set for input into a neural network, which may be then be trained to identify a lane change intention of a vehicle. This method is fairly robust and may be able to label even high-noise trajectory data. High noise trajectory data is data in which the trajectory of a vehicle is not smooth. In practice vehicles often meander rather than traveling in a straight line. As shown in Fig. 2, not every change in angle ends with a lane change, while sometimes a vehicle moves left and right within a lane before finally changing lane. Such data may be considered to be high-noise. Example implementations of blocks S1 to S5 will now be discussed.
Block S1 comprises collecting and preprocessing tracking trajectory data of surrounding vehicles. The tracking trajectory data of the surrounding vehicles may be generated by a sensing system of a vehicle during a test drive of the vehicle. The tracking trajectory data may include one or more of a map with road information, physical information of the vehicle (for example, a vehicle type, a vehicle size such as a length, a width and a height) , a position of the vehicle (based on a Frenet road coordinate system, or a Cartesian world coordinate system) , a lane in which the vehicle is located, a speed (in one embodiment, the speed may be based on the Frenet road coordinate system) , an orientation of the vehicle (in one embodiment, the orientation may be based on the Frenet road coordinate system) , etc.; which are generated after tracking process is performed on the vehicle after data sensed by a camera, a laser radar, a millimeter wave radar, etc. is fused. As described herein and in the claims, “fuse” refers to a calculation which combines data derived from separate sources. For example, camera and radar data may be fused together so that both an object (such as a surrounding vehicle) and a distance to the object may be detected. The tracking trajectory data of the surrounding vehicles may be high-noise tracking trajectory data. Fig. 2 shows example of high noise tracking and trajectory data, in which the left side of Fig. 2 depicts a lane keeping trajectory and the right side of Fig. 2 depicts a lane changing trajectory.
In some examples of the present disclosure, the Frenet road coordinate system may be used. The Cartesian coordinate system is usually used to describe a position of an object, but it is not the optimal choice for unmanned driving. Unmanned driving cars usually make acceleration or lane changing decisions based on the road. As an alternative solution to the Cartesian coordinate system, the Frenet coordinate system uses a center line of the road as a reference line, and uses a tangential direction and a normal direction of the reference line to define the coordinate system. As shown in Fig. 3, a distance in the tangential direction S and a lateral distance L of the vehicle relative to the center line of the road are coordinates of the vehicle in the Frenet coordinate system.
In block S1, the preprocessing is performed on the tracking trajectory data of the surrounding vehicles. In one example the pre-processing is to screen out data and may include:
(1) only selecting trajectory data with map information, wherein when a vehicle cannot be positioned on a known map, a trajectory of the vehicle is not selected;
(2) not selecting a trajectory of a stationary vehicle because a lane changing behavior and a lane keeping behavior of a vehicle are only meaningful to moving vehicles;
(3) only selecting a tracking trajectory of a vehicle within a certain distance (such as, within 100 meters) from an unmanned driving vehicle. Because noises in tracking data will be too large in the case of more than 100 meters, and the tracking data is not suitable for being used as training data.
Block S2 comprises finding a lane changing point. For example, taking the vehicle as a unit, a time point at which the vehicle crosses different lanes may be found, e.g., the location of the vehicle at the (n-1) th frame may be defined as a lane A, and the position of the vehicle at the nth frame may be defined as a lane B, and the nth frame may be defined as the lane changing point of the vehicle. A vehicle may have one or more lane changing points along the entire trajectory. In other examples, lane keeping behavior may be maintained along the entire trajectory such that no lane changing points exist.
A left side of Fig. 4 shows tracking trajectory data of a lane change in a road 400 between a first lane and a second lane. Reference numeral 401 indicates a center line of the first lane, while reference numeral 402 indicates a center line of the second lane. A trajectory 410 of a vehicle changing lane includes a plurality of points or frames indicated by dots and reference numerals 411. The point of lane change 440 is denoted by a triangle in Fig. 4 and has already been determined in block S2 of the method.
Block S3 comprises determining a starting point of the lane change. The method of determining the starting point of lane change may comprise a basic model (a) and a modification of the basic model (b) to enable the model to better cope with noise.
(a) A basic model is made for a behavior of the lane changing vehicle in the present disclosure: according to this model when the lane changing behavior of the vehicle starts, the vehicle will gradually move towards a target lane, i.e., a lateral distance L of the vehicle from a current lane increases and a lateral distance L of the vehicle from the target lane decreases. However, this is not necessarily true in noisy tracking data. There may be a large number of “jumps” in positioning of the vehicle in the Frenet road coordinate system, which includes jumps in both the direction S and in the direction L. A rhombus 420 in Fig. 4 is an actual starting point of lane change, while positions corresponding to rectangles 430 are misjudged as starting points of lane change according to this basic model. Therefore, the basic model can be improved. The improved model (b) is discussed below.
(b) The present disclosure proposes a method for determining a movement trend of a vehicle. An average value Lmean for a plurality of frames (e.g. 5 frames) in coordinates of direction L is used as a basis for determination, which greatly increases robustness of determination of a lane changing behavior. Starting from the nth frame of the point of lane change of the vehicle (the triangle on the left in Fig. 4) and sequentially calculating Lmean of the (n-1) th frame, the (n-2) th frame, . . ., etc. of the vehicle relative to the current lane, wherein according to the definition in (a) , a distance of the vehicle in direction L at the (n-1) th frame relative to the current lane should be less than a distance of the vehicle in direction L at the nth frame relative to the current lane, a distance in the direction L for the (n-2) th frame should be less than a distance in direction L for the (n-1) th frame, and so on, until calculation is performed on the (n-t) th frame; and determining the (n-t) th frame as the starting point of lane change of the vehicle when Lmean no longer decreases progressively or reaches a center line of the lane in which the vehicle is located (in this time, L=0) . The smoother curve 450 on the right in Fig. 4 is a trajectory graph drawn with the alternative Lmean. Compared with the original trajectory 410 adjacent to the curve, it can be seen that the new trajectory graph is more robust to noise.
Block S4, verifies an effective lane changing trajectory. For example, a lane changing trajectory which is too short or too long (for example, t < 10 or t > 100) is discarded and is not considered, where t corresponds to a number of frames in the lane changing trajectory. Whether the vehicle is still in the current lane is verified in each frame in (b) in block S3. If another lane change occurs during a lane changing backstepping process, the backstepping is immediately terminated and t is recorded. If t is too short or too long, the track is discarded. This verification can effectively filter out excessive noise or incomplete data. The left side of Fig. 5 shows an example of a lane changing trajectory 510 between a first lane and a second lane of a road 500, in which a center of the first lane is denoted by reference numeral 501 and a center of the second lane is denoted by a reference numeral 502 As shown in the left side of Fig. 5, in the lane changing trajectory, a triangle point 520 at the front end is an actual point of lane change, and the other triangles 510 are points of lane change caused by noises. The right side of Fig. 5 depicts a lane keeping trajectory 520 on the same road, in which triangles 510 are points of lane change which are falsely identified due to noise. In block S4, both trajectories shown in Fig. 5 are discarded and are not considered, thus ensuring the correctness of the training data. For example, through the back stepping process which traverses frames prior to an identified point of lane change (e.g. one of the triangles) it may be found that the lane changing trajectories shown in Fig. 5 are too short, so they may be discarded.
Block S5, outputs at least one of a labeled lane changing frame and labeled lane keeping frame. For example, all frames [n-t, n] from the start of the lane change of the vehicle to the point of lane change are defined as lane changing frames of the vehicle, and the others are lane keeping frames. In the present disclosure, in one example, an ending point of lane change is not set, where the ending point of lane change is the point of lane change, because an actual behavior of the vehicle after the point of lane change is lane keeping until the next lane change.
The method may be performed on a computing device, by executing instructions by a processor of the computing device. In one example, a non-transitory machine readable storage medium storing instructions which are executable by a processor to perform any of the above methods is provided.
The technical solutions discussed above are robust in extracting lane changing trajectories from noisy tracking trajectory data of vehicles. They use a generalized definition for lane changing behaviors of vehicles, and validity of a labeling is verified during the labeling process, so that a large number of correct and effective lane changing frames and lane keeping frames can be labeled without human intervention.
Specific parameters in the present disclosure can be adjusted, which is also applicable to low-noise trajectory data of vehicle and not limited to tracking data.
In the present disclosure, a change in an average value of a distance between a vehicle and the road center over time is used as to determine the start of a lane change. In this way the influence of noise may be reduced and a robust model provided which focuses on an essence of lane changing behaviors.
The embodiment described above is used to explain the present disclosure, rather than limit the present disclosure. Within the spirit of the present disclosure and the protection scope of the claims, any modifications and changes made to the present disclosure fall into the protection scope of the present disclosure.
Claims (17)
- A method for automatically labeling a lane changing intention of a vehicle based on trajectory data of the vehicle, the method comprising:collecting and preprocessing tracking trajectory data of a surrounding vehicle;finding a point of lane change of the surrounding vehicle;determining a starting point of lane change;andoutputting at least one labeled frame of the vehicle, wherein the labeled frame is labeled as a lane changing frame or a lane keeping frame.
- The method of claim 1, wherein the tracking trajectory data of the surrounding vehicle is generated by a sensing system of an autonomous vehicle during a test drive of the autonomous vehicle, and wherein the tracking trajectory data includes one or more of: a map with road information, physical information of the surrounding vehicle, a position of the surrounding vehicle, a lane in which the surrounding vehicle is located, a speed of the surrounding vehicle, and an orientation of the surrounding vehicle.
- The method of claim 1 or 2, wherein the preprocessing of the tracking trajectory data comprises screening the tracking and trajectory data to remove one or more of the following types of tracking and trajectory data:data relating to a surrounding vehicle which cannot be located on a known map,data relating to a surrounding vehicle which is stationary; anddata relating to a surrounding vehicle which is more than a predetermined distance away from an autonomous vehicle which collected the tracking and trajectory data.
- The method of claim 1 or 2 wherein pre-processing of the tracking and trajectory data includes selecting tracking and trajectory data which relates to a surrounding vehicle which is moving and can be positioned on a known map and is within a predetermined distance of an autonomous vehicle which collected the tracking and trajectory data.
- The method of any of the above claims, wherein finding the point of lane change of the surrounding vehicle comprises identifying a position of the surrounding vehicle in a plurality of frames corresponding to successive points in time, finding an (n-1) th frame where the surrounding vehicle is in a first lane and a n th frame where the surrounding vehicle is in a second lane and defining the n th frame as the point of lane change of the surrounding vehicle.
- The method of any of the above claims wherein determining the starting point of lane change comprises:a) determining, for each frame of a plurality of frames, a lane in which the surrounding vehicle is located;b) determining, for each frame of the plurality of frames, lateral distance L of the surrounding vehicle relative to the lane in which the surrounding vehicle is located;c) determining, for each frame, an average value of the lateral distance, L mean, based on an average of the lateral distance L of the current frame a number of previous frames; andd) determining the starting point of lane change, by traversing frames going backwards in time from the point of lane change, and selecting as the starting point of lane change: a first frame in which the average value of the lateral distance, L mean, has not decreased relative to the previously traversed frame, or a first frame in which the average value of the lateral distance, L mean, is at the center of the previous lane.
- The method of any of claims 1-5, wherein determining the starting point of lane change comprises:using an average value L mean which is an average of the coordinates of the lane changing vehicle relative to the current lane in a lateral direction L taken over 5 frames, as a basis for determining a starting point of the lane change; calculating L mean starting from a n th frame which is the lane changing point of the lane changing vehicle and sequentially calculating L mean of previous frames including a (n-1) th frame, a (n-2) th frame, ..., etc. of the lane changing vehicle, until calculation is performed on the (n-t) th frame; and determining the (n-t) th frame as the starting point of the lane change of the lane changing vehicle, wherein the (n-t) th frame is a frame in which L mean no longer decreases progressively compared to the previously calculated frame or is a frame in which L mean reaches a center line of the lane in which the lane changing vehicle is located.
- The method of any one of the above claims further comprising, after determining a starting point of lane change and before outputting at least one labeled frame, verifying an effective lane change lane changing trajectory of the surrounding vehicle.
- The method of claim 8, wherein in verifying the effective lane changing trajectory of the surrounding vehicle, comprises discarding lane change trajectories in which a period of time or number of frames between the starting point of the lane change and the point of lane change is below a minimum threshold or above a maximum threshold.
- A non-transitory computer readable storage medium storing instructions which are executable by a processor to perform the method of any of claims 1-9.
- A method for automatically labeling a lane change intention based on high-noise vehicle trajectory data, comprising the following steps:S1, collecting and preprocessing tracking trajectory data of surrounding vehicles;S2, finding a lane change point;S3, determining a lane change starting point;S4, verifying an effective lane change trajectory; andS5, outputting a labeled lane change frame and lane keeping frame.
- The method for automatically labeling a lane change intention based on high-noise vehicle trajectory data of claim 11, wherein in step S1, the tracking trajectory data of the surrounding vehicles is generated by an unmanned driving sensing system during a drive test, which comprises a map with road information, vehicle physical information, a vehicle position, a lane in which the vehicle is located, a speed, and a vehicle orientation.
- The method for automatically labeling a lane change intention based on high-noise vehicle trajectory data of claim 11, wherein in step S1, the preprocessing of the tracking trajectory data comprises the following data screening:(1) only selecting trajectory data with map information, wherein when a vehicle cannot be positioned on a known map, a trajectory of the vehicle is not selected;(2) skipping selecting a trajectory of a stationary vehicle;(3) only selecting a tracking trajectory of a vehicle within a certain distance from an unmanned driving vehicle.
- The method for automatically labeling a lane change intention based on high-noise vehicle trajectory data of claim 11, wherein in step S2, taking the vehicle as a unit, a time point for the vehicle to cross different lanes is found, i.e., the vehicle at the (n-1) th frame is defined as a lane A, and the vehicle at the n th frame is defined as a lane B, and the n th frame is the lane change point of the vehicle.
- The method for automatically labeling a lane change intention based on high-noise vehicle trajectory data of claim 11, wherein step S3 further comprises:(a) making a setting for a behavior of the lane changing vehicle: when a lane changing behavior of the vehicle starts, the vehicle will gradually move towards a target lane, i.e., an L lateral distance of the vehicle from a current lane increases and an L lateral distance of the vehicle from the target lane decreases;(b) using an average value L mean for the first five frames of L direction coordinates as a basis for determination; starting from the n th frame of the lane change point of the vehicle and sequentially calculating L mean of the (n-1) th frame, the (n-2) th frame, ..., etc. of the vehicle relative to the current lane, wherein according to the definition in (a) , an L direction distance of the vehicle at the (n-1) th frame relative to the current lane should be less than an L direction distance of the vehicle at the n th frame relative to the current lane, an L direction distance for the (n-2) th frame should be less than an L direction distance for the (n-3) th frame, and so on, until calculation is performed on the (n-t) th frame; and determining the (n-t) th frame as the lane change starting point of the vehicle when L mean no longer decreases progressively or reaches a center line of the lane in which the vehicle is located.
- The method for automatically labeling a lane change intention based on high-noise vehicle trajectory data of claim 15, wherein in step S4, a too short or too long lane change trajectory is discarded and is not considered.
- A non-transitory computer readable storage medium storing instructions which are executable by a processor to perform the method of any of claims 11-16.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201911113106.1 | 2019-11-14 | ||
CN201911113106.1A CN110986994B (en) | 2019-11-14 | 2019-11-14 | Automatic lane change intention marking method based on high-noise vehicle track data |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2021093335A1 true WO2021093335A1 (en) | 2021-05-20 |
Family
ID=70084361
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/098164 WO2021093335A1 (en) | 2019-11-14 | 2020-06-24 | Method for automatically labeling lane changing intention based on high-noise trajectory data of vehicle |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN110986994B (en) |
WO (1) | WO2021093335A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115482659A (en) * | 2022-08-18 | 2022-12-16 | 浙江工商大学 | Intelligent agent autonomous decision-making method based on deep reinforcement learning |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110986994B (en) * | 2019-11-14 | 2021-08-03 | 苏州智加科技有限公司 | Automatic lane change intention marking method based on high-noise vehicle track data |
CN111599182B (en) * | 2020-07-22 | 2020-10-20 | 中汽院汽车技术有限公司 | Automatic lane changing behavior extraction method based on natural driving data |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130345895A1 (en) * | 2012-06-20 | 2013-12-26 | Trimble Navigation Limited | Lane change monitoring |
CN107967486A (en) * | 2017-11-17 | 2018-04-27 | 江苏大学 | A kind of nearby vehicle Activity recognition method based on V2V communications with HMM-GBDT mixed models |
US20190084558A1 (en) * | 2017-09-19 | 2019-03-21 | Toyota Jidosha Kabushiki Kaisha | Vehicle surrounding monitoring device |
CN109886304A (en) * | 2019-01-22 | 2019-06-14 | 江苏大学 | A kind of complex road condition following peripheral vehicle behavior recognition methods based on HMM-SVM bilayer improved model |
CN109946688A (en) * | 2019-03-18 | 2019-06-28 | 中国汽车工程研究院股份有限公司 | Lane-change contextual data extracting method, device and server |
CN110146100A (en) * | 2018-02-13 | 2019-08-20 | 华为技术有限公司 | Trajectory predictions method, apparatus and storage medium |
CN110986994A (en) * | 2019-11-14 | 2020-04-10 | 苏州智加科技有限公司 | Automatic lane change intention marking method based on high-noise vehicle track data |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103823382B (en) * | 2014-02-27 | 2016-08-03 | 浙江省科威工程咨询有限公司 | A kind of lane-change track optimizing based on vehicle and speed and the Visual Implementation method |
KR101617345B1 (en) * | 2014-12-03 | 2016-05-02 | 현대모비스 주식회사 | System and method for estimating driving lane of vehicle |
US10077050B2 (en) * | 2016-05-24 | 2018-09-18 | GM Global Technology Operations LLC | Automated driving system for evaluating lane cut-out and method of using the same |
CN107545612A (en) * | 2016-06-23 | 2018-01-05 | 戚艳红 | Make video recording startup method and the activation system of making video recording of a kind of drive recorder |
JP7006093B2 (en) * | 2017-09-28 | 2022-02-10 | トヨタ自動車株式会社 | Driving support device |
CN108693883B (en) * | 2018-06-06 | 2021-12-24 | 苏州智加科技有限公司 | High-precision parking method and system for automatic driving of vehicle |
CN110103955B (en) * | 2019-04-28 | 2021-04-23 | 浙江吉利控股集团有限公司 | Vehicle early warning method and device and electronic equipment |
CN110304075B (en) * | 2019-07-04 | 2020-06-26 | 清华大学 | Vehicle track prediction method based on hybrid dynamic Bayesian network and Gaussian process |
CN110329263B (en) * | 2019-07-05 | 2020-08-21 | 西南交通大学 | Self-adaptive track changing planning method for automatic driving vehicle |
-
2019
- 2019-11-14 CN CN201911113106.1A patent/CN110986994B/en active Active
-
2020
- 2020-06-24 WO PCT/CN2020/098164 patent/WO2021093335A1/en active Application Filing
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130345895A1 (en) * | 2012-06-20 | 2013-12-26 | Trimble Navigation Limited | Lane change monitoring |
US20190084558A1 (en) * | 2017-09-19 | 2019-03-21 | Toyota Jidosha Kabushiki Kaisha | Vehicle surrounding monitoring device |
CN107967486A (en) * | 2017-11-17 | 2018-04-27 | 江苏大学 | A kind of nearby vehicle Activity recognition method based on V2V communications with HMM-GBDT mixed models |
CN110146100A (en) * | 2018-02-13 | 2019-08-20 | 华为技术有限公司 | Trajectory predictions method, apparatus and storage medium |
CN109886304A (en) * | 2019-01-22 | 2019-06-14 | 江苏大学 | A kind of complex road condition following peripheral vehicle behavior recognition methods based on HMM-SVM bilayer improved model |
CN109946688A (en) * | 2019-03-18 | 2019-06-28 | 中国汽车工程研究院股份有限公司 | Lane-change contextual data extracting method, device and server |
CN110986994A (en) * | 2019-11-14 | 2020-04-10 | 苏州智加科技有限公司 | Automatic lane change intention marking method based on high-noise vehicle track data |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115482659A (en) * | 2022-08-18 | 2022-12-16 | 浙江工商大学 | Intelligent agent autonomous decision-making method based on deep reinforcement learning |
CN115482659B (en) * | 2022-08-18 | 2023-09-19 | 浙江工商大学 | Intelligent autonomous decision-making method based on deep reinforcement learning |
Also Published As
Publication number | Publication date |
---|---|
CN110986994A (en) | 2020-04-10 |
CN110986994B (en) | 2021-08-03 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP7106664B2 (en) | Intelligent driving control method and device, electronic device, program and medium | |
WO2021093335A1 (en) | Method for automatically labeling lane changing intention based on high-noise trajectory data of vehicle | |
US10255812B2 (en) | Method and apparatus for preventing collision between objects | |
CN110532916B (en) | Motion trail determination method and device | |
CN106647776B (en) | Method and device for judging lane changing trend of vehicle and computer storage medium | |
US9352746B2 (en) | Lane relative position estimation method and system for driver assistance systems | |
CN111907521B (en) | Transverse control method and device for automatic driving vehicle and storage medium | |
KR102456626B1 (en) | Apparatus and method for traffic lane recognition in automatic steering control of vehilcles | |
CN112793576B (en) | Lane change decision method and system based on rule and machine learning fusion | |
CN112530170A (en) | Vehicle driving state detection method and device, electronic equipment and storage medium | |
US11080544B1 (en) | Method and device for calibrating pitch of camera on vehicle and method and device for continual learning of vanishing point estimation model to be used for calibrating the pitch | |
JP4937844B2 (en) | Pedestrian detection device | |
CN113127466B (en) | Vehicle track data preprocessing method and computer storage medium | |
JP2018147399A (en) | Target detection device | |
JP5573780B2 (en) | Course evaluation device and course evaluation method | |
CN111066024B (en) | Method and device for recognizing a lane, driver assistance system and vehicle | |
JP5682302B2 (en) | Traveling road estimation device, method and program | |
CN113581168B (en) | Bad-sight-distance curve vehicle-meeting safety method and system based on artificial intelligence | |
JP2020119526A (en) | Main object selection for assist function or automatic driving function of driver assist system or driving system of power vehicle | |
US20240020964A1 (en) | Method and device for improving object recognition rate of self-driving car | |
CN115649174B (en) | Intelligent overtaking event processing method and related product | |
CN115641569B (en) | Driving scene processing method, device, equipment and medium | |
WO2021166893A1 (en) | Target object recognition device | |
CN118230593A (en) | Parking space detection method, electronic equipment, storage medium and vehicle | |
CN114461671A (en) | Data processing method, vehicle, and computer-readable storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20888488 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20888488 Country of ref document: EP Kind code of ref document: A1 |