WO2022105394A1

WO2022105394A1 - Simulation method and system, device, readable storage medium, and platform for autonomous driving

Info

Publication number: WO2022105394A1
Application number: PCT/CN2021/118205
Authority: WO
Inventors: Pingliang HAN; Bohan SHANG; Wei Wang
Original assignee: Suzhou Zhijia Science & Technologies Co., Ltd.
Priority date: 2020-11-17
Filing date: 2021-09-14
Publication date: 2022-05-27
Also published as: CN112286206A; CN112286206B

Abstract

Disclosing a simulation method and system, a device, a readable storage medium, and a platform for autonomous driving, and relates to the field of simulated driving. The method comprises: obtaining a reference driving data set; running a simulated driving platform according to reference land data; obtaining simulated vehicle attitude data of a simulated vehicle in real time; determining a deviation in position between the simulated vehicle attitude data and reference vehicle attitude data; and planning a driving trajectory of the simulated vehicle according to the deviation in position and a correspondence between reference traffic data and the reference vehicle attitude data to obtain closed-loop simulation data of simulated driving. It can improve the accuracy of a simulated driving test and the authenticity of the simulated driving test.

Description

SIMULATION METHOD AND SYSTEM, DEVICE, READABLE STORAGE MEDIUM, AND PLATFORM FOR AUTONOMOUS DRIVING

CROSS REFERENCE TO RELATED APPLICATION

This disclosure claims the benefits of priority to Chinese application number 202011286050.2, filed on 17 November 2020 and entitled “Simulation method and system, device, readable storage medium, and platform for automated driving” which is hereby incorporated by reference in its entirety.

TECHNICAL FIELD

Embodiments of the present application relate to the field of simulated driving, and in particular to a simulation method and system, a device, a readable storage medium, and a platform for autonomous driving.

BACKGROUND

Autonomous driving is a technology that enables a vehicle to travel in an autonomous manner without a human driver. Before being launched on the market, an autonomous driving system needs to be subjected to a large number of tests to ensure the safety and reliability of the system. It is generally believed that the autonomous driving system needs at least 11 billion miles of test mileage to reach the safety and reliability requirements.

The foregoing tests can be completed by actual road tests. However, it is often difficult for the actual road tests to provide the required test amount. In addition, the actual road tests are often restricted by actual road conditions, which makes it difficult to test the autonomous driving system for special scenes.

It is also possible to test the autonomous driving system on a simulation platform based on a constructed simulated driving scene and by simulating the simulated driving of an autonomous driving vehicle on the simulation platform. Here, by simulating possible scene environments, especially by simulating some special scenes such as dangerous scenes and extreme scenes, the test of the autonomous driving system under extreme driving conditions can be completed. The simulation platform can also use actual road driving data collected from the real world for testing. However, due to the need to convert the actual road driving data into related scenes, it is easy to cause scene selection errors or scene loss problems. In addition, the cost of collecting and labeling the actual road driving data is also quite high.

A method is needed to solve at least one or more of the foregoing problems.

SUMMARY

Embodiments of the present application provide a simulation method and system, a device, a readable storage medium, and a platform for autonomous driving. The embodiments of the present application can at least improve the test efficiency of autonomous driving algorithms, for example, can at least improve the accuracy of test results of the autonomous driving algorithms, reduce the time required to complete the test, or reduce the test amount, so as to improve the overall reliability and safety of the autonomous driving system.

In one aspect, there is provided a computer implemented simulation method for autonomous driving, the method being executed by one or more processors and comprising:

obtaining a reference driving data set, by a processor of a computer, the reference driving data set comprising a reference lane data subset, a reference traffic data subset, and a reference vehicle attitude data subset, wherein reference lane data in the reference lane data subset, reference traffic data in the reference traffic data subset, and reference vehicle attitude data in the reference vehicle attitude data subset have a correspondence with each other;

running a simulated driving platform based on the reference lane data, so that a simulated vehicle performs simulated autonomous driving on the simulated driving platform;

obtaining by the processor simulated vehicle attitude data of the simulated vehicle in real time;

determining, by the processor, a deviation in position between the simulated vehicle attitude data and the reference vehicle attitude data, and determining, target traffic data corresponded to the simulated vehicle attitude data from the reference traffic data according to the deviation in position and the correspondence between the reference traffic data and the reference vehicle attitude data, the deviation in position being used to indicate a difference in geographic positions on the road between a reference vehicle corresponding to the reference driving data set and the simulated vehicle; and

planning, by the processor, a driving trajectory of the simulated vehicle according to the reference lane data and the target traffic data to obtain closed-loop simulation data of simulated driving.

In an optional embodiment, the reference traffic data comprises position data of a road participating vehicle; and

the step of planning by the processor a driving trajectory of the simulated vehicle according to the reference lane data and the target traffic data to obtain closed-loop simulation data of simulated driving further comprises:

predicting by the processor an intended trajectory of the road participating vehicle according to the reference lane data and the position data of the road participating vehicle, to obtain prediction data of the participating vehicle;

planning by the processor the driving trajectory of the simulated vehicle according to the reference lane data and the prediction data of the participating vehicle, to obtain trajectory planning data; and

determining by the processor the closed-loop simulation data of simulated driving according to the trajectory planning data.

In an optional embodiment, the step of determining the closed-loop simulation data of simulated driving according to the trajectory planning data comprises:

receiving by the processor feedback data returned by the simulated vehicle, the feedback data being used to indicate a simulated driving condition of the simulated vehicle;

generating by the processor a control command according to the feedback data and the trajectory planning data;

running the simulated vehicle according to the control command, and generating updated simulated vehicle attitude data according to an operation result of the simulated vehicle; and

repeating by the processor, according to the updated simulated vehicle attitude data, steps of determining the deviation in position between the simulated vehicle attitude data and the reference vehicle attitude data and determining target traffic data corresponding to the simulated vehicle attitude data from the reference traffic data according to the deviation in position and the correspondence between the reference traffic data and the reference vehicle attitude data, to obtain the closed-loop simulation data of simulated driving.

In an optional embodiment, the method further comprises below steps after the closed-loop simulation data of simulated driving is obtained:

obtaining by the processor key test indicators from the closed-loop simulation data of simulated driving; and

evaluating by the processor the key test indicators to obtain a platform evaluation result of the simulated driving platform.

In an optional embodiment, the step of evaluating the key test indicators to obtain a platform evaluation result of the simulated driving platform further comprises:

obtaining by the processor an indicator evaluation result from the key test indicators; and

obtaining by the processor time positioning of the indicator evaluation result being failure during the autonomous driving in response to the indicator evaluation result being failure, to obtain the platform evaluation result.

In an optional embodiment, the key test indicators comprise at least one of a braking indicator, an acceleration indicator, and an inter-vehicle distance indicator,

the braking indicator corresponding to a braking acceleration of the simulated vehicle;

the acceleration indicator corresponding to a speed-up acceleration of the simulated vehicle; and

the inter-vehicle distance indicator corresponding to a distance between the simulated vehicle and another simulated vehicle on the road in a designated driving stage.

In an optional embodiment, the method further comprises:

determining by the processor that the braking indicator fails when the braking acceleration reaches a first acceleration requirement;

determining by the processor that the acceleration indicator fails when the speed-up acceleration reaches a second acceleration requirement; and

determining by the processor that the inter-vehicle distance indicator fails when the distance between the simulated vehicle and the another simulated vehicle on the road in the designated driving stage less than a distance threshold.

In an optional embodiment, the step of obtaining a reference driving data set comprises:

obtaining by the processor a data packet list, the data packet list comprising data packets corresponding to different driving time periods, and the data packets being arranged in ascending order of the driving time periods; and

reading by the processor the reference driving data set in the data packets from the data packet list in sequence.

In an optional embodiment, the data packets comprise the reference driving data set corresponding to image frames in a reference driving video; and

the step of reading the reference driving data set in the data packets from the data packet list in sequence comprises:

reading the data packets from the data packet list in sequence; and

obtaining the reference driving data corresponding to the image frames frame by frame from the data packets and caching same.

In another aspect, there is provided a simulation system for autonomous driving, including a processor configured to execute the following modules:

an obtaining module configured to obtain a reference driving data set, the reference driving data set comprising a reference lane data subset, a reference traffic data subset, and a reference vehicle attitude data subset, wherein reference lane data in the reference lane data subset, reference traffic data in the reference traffic data subset, and reference vehicle attitude data in the reference vehicle attitude data subset have a correspondence with each other;

an operating module configured to run a simulated driving platform based on the reference lane data, so that a simulated vehicle performs simulated autonomous driving on the simulated driving platform;

the obtaining module is further configured to obtain simulated vehicle attitude data of the simulated vehicle in real time;

a determination module configured to determine a deviation in position between the simulated vehicle attitude data and the reference vehicle attitude data, and determining, target traffic data corresponded to the simulated vehicle attitude data from the reference traffic data according to the deviation in position and the correspondence between the reference traffic data and the reference vehicle attitude data, the deviation in position being used to indicate a difference in geographic positions on the road between a reference vehicle corresponding to the reference driving data set and the simulated vehicle; and

a planning module configured to plan a driving trajectory of the simulated vehicle according to the reference lane data and the target traffic data to obtain closed-loop simulation data of simulated driving.

the planning module comprises:

a prediction unit configured to predict an intended trajectory of the road participating vehicle according to the reference lane data and the position data of the road participating vehicle, to obtain prediction data of the rode participating vehicle;

a planning unit configured to plan the driving trajectory of the simulated vehicle according to the reference lane data and the prediction data of the road participating vehicle, to obtain trajectory planning data; and

a determination unit configured to determine the closed-loop simulation data of simulated driving according to the trajectory planning data.

In an optional embodiment, the determination unit is further configured to receive feedback data returned by the simulated vehicle, the feedback data being used to indicate a simulated driving condition of the simulated vehicle; and generate a control command according to the feedback data and the trajectory planning data; and

the determination unit is further configured to run the simulated vehicle according to the control command, and generating updated simulated vehicle attitude data according to an operation result of the simulated vehicle; and repeat, according to the updated simulated vehicle attitude data, steps of determining the deviation in position between the simulated vehicle attitude data and the reference vehicle attitude data and determining target traffic data corresponding to the simulated vehicle attitude data from the reference traffic data according to the deviation in position and the correspondence between the reference traffic data and the reference vehicle attitude data, to obtain the closed-loop simulation data of simulated driving.

In an optional embodiment, the obtaining module is further configured to obtain processor key test indicators from the closed-loop simulation data of simulated driving; and

the determination module is further configured to evaluate the key test indicators to obtain a platform evaluation result of the simulated driving platform.

In an optional embodiment, the obtaining module is further configured to an indicator evaluation result from the key test indicators; and obtain time positioning of the indicator evaluation result being failure during the autonomous driving in response to the indicator evaluation result being failure, to obtain the platform evaluation result.

In an optional embodiment, the key indicators comprise at least one of a braking indicator, an acceleration indicator, and an inter-vehicle distance indicator,

In an optional embodiment, the determination module is further configured to determine the braking indicator fails when the braking acceleration reaches a first acceleration requirement.

the determination module is further configured to determine the acceleration indicator fails when the speed-up acceleration reaches a second acceleration requirement; and

the determination module is further configured to determine the inter-vehicle distance indicator fails when the distance between the simulated vehicle and the another simulated vehicle on the road in the designated driving stage less than a distance threshold.

In an optional embodiment, the obtaining module is further configured to obtain a data packet list, the data packet list comprising data packets corresponding to different driving time periods, and the data packets being arranged in ascending order of the driving time periods; and read the reference driving data set in the data packets from the data packet list in sequence.

In an optional embodiment, the data packets comprise the reference driving data corresponding to image frames in a reference driving video; and the obtaining module is further configured to read the data packets from the data packet list in sequence; and obtain the reference driving data corresponding to the image frames frame by frame from the data packets and caching same.

In another aspect, there is provided a computer device for simulating autonomous driving. The computer device comprises a processor and a memory configured to store at least one instruction, at least one program, a code set or an instruction set, executable by the processor, wherein the processor when executing the at least one instruction, the at least one program, the code set or the instruction set implements the simulation method for autonomous driving described in any of the foregoing embodiments of the present application.

In another aspect, there is provided a non-transitory computer-readable storage medium having at least one instruction, at least one program, a code set or an instruction set stored thereon that are executable by a computing device to implement the simulation method for autonomous driving described in any of the foregoing embodiments of the present application.

In another aspect, there is provided a computer program product or a computer program, wherein the computer program product or the computer program comprises computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device implements the simulation method for autonomous driving described in any of the foregoing embodiments.

In yet another aspect, there is provided a simulated driving platform, wherein the simulated driving platform comprises a processor and a controller; and the processor and the controller are configured to control a simulated vehicle to implement the simulation method for autonomous driving described in any of the foregoing embodiments.

The beneficial effects brought by the technical solutions provided in the embodiments of the present application comprise at least the following:

Closed-loop simulation can be performed on an autonomous driving process of a simulated vehicle based on reference lane data, reference traffic data, and reference vehicle attitude data in a reference driving data set, and a driving trajectory plan of the simulated vehicle can be determined according to the reference lane data and the reference traffic data in real time, which can at least improve the test efficiency of autonomous driving algorithms, for example, can at least improve the accuracy of test results of the autonomous driving algorithms, reduce the time required to complete the test, or reduce the test amount, so as to improve the overall reliability and safety of the autonomous driving system.

The present application can at least alleviate the deficiency that no accurate simulated driving results can be obtained when the autonomous driving is simulated according to a reference driving condition in an open-loop simulated driving test process. In an open-loop simulated driving test, since the position of the simulated vehicle needs to be constantly adjusted according to the position of a real/reference driving vehicle in an open-loop simulated driving process, a complete simulated driving process cannot be obtained. The simulation method for autonomous driving provided in the present application improves the accuracy of the simulated driving test and improves the authenticity of the simulated driving test.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments will be briefly described below. Apparently, the drawings in the following description are only some embodiments of the present application, and a person of ordinary skill in the art can obtain other drawings according to the drawings without any creative work.

FIG. 1 is a schematic diagram of an implementation environment of a simulation method for autonomous driving according to an embodiment of the present application;

FIG. 2 is a flowchart of a simulation method for autonomous driving according to an exemplary embodiment of the present application;

FIG. 3 is a structural block diagram of a simulated driving system according to an exemplary embodiment of the present application;

FIG. 4 is a flowchart of a simulation method for autonomous driving according to another exemplary embodiment of the present application;

FIG. 5 is a structural block diagram of a simulation apparatus for autonomous driving according to an exemplary embodiment of the present application;

FIG. 6 is a structural block diagram of a simulation apparatus for autonomous driving according to another exemplary embodiment of the present application; and

FIG. 7 is a structural block diagram of a server according to an exemplary embodiment of the present application.

DETAILED DESCRIPTION

In order to make the objective, technical solutions, and advantages of the present application clearer, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

Unmanned vehicle: it has a full name of unmanned driving vehicle, also known as autonomous-driving vehicle, and wheeled mobile robot, which mainly relies on intelligent driving instruments based on a computer system in the vehicle to realize the purpose of unmanned driving. An unmanned vehicle may be an intelligent vehicle that senses a road environment through an onboard sensing system, plans a driving route in an autonomous manner, and controls the vehicle to reach a predetermined target. An unmanned vehicle may utilize an onboard sensor to sense the surrounding environment of the vehicle, and controls the steering and speed of the vehicle according to the sensed information about the road, vehicle position, and obstacles, so that the vehicle can safely and reliably travel on the road. An unmanned vehicle may be integrated with many cutting-edge technologies such as automatic control, architecture, artificial intelligence, and visual computing. It is a product of the highly developed computer science, pattern recognition and intelligent control technology.

Internet of Vehicles (IoV) : it is the Internet of Things for Vehicles, which uses travelling vehicles as information sensing objects. IoV may thus refer to a communication of transport related information between vehicles or between a vehicle and surrounding transport, road or city infrastructure and/or a communication protocol for such communications. With the aid of a new generation of information and communication technology, network connections between a vehicle and a vehicle, a person, the road, a service platform, or other objects can be realized, which can improve the overall intelligent driving level of the vehicles, to provide users with safe, comfortable, intelligent, and efficient driving experience and traffic services, while improving the efficiency of traffic operation and improving the intelligent level of social traffic services. Optionally, a vehicle-mounted device on the vehicle effectively utilizes dynamic information of all vehicles in an information network platform through a wireless communication technology to provide different functional services during vehicle operation. The Internet of Vehicles usually exhibits the following characteristics: the Internet of Vehicles can provide protection for a distance between vehicles and reduce the probability of vehicle collision accidents; the Internet of Vehicles can help vehicle owners navigate in real time and improve the efficiency of traffic operation through communication with other vehicles and network systems.

Autonomous driving simulation or similar terms refer to the simulation of autonomous driving by a computer. Autonomous driving simulation technology thus applies computer simulation technology to the automotive field. Autonomous driving simulation may be more complex in research and development terms than the simulation system of traditional ADAS (Advanced Driving Assistance System) , and may have high requirements of the system architecture. An autonomous driving simulation system may digitally restore and generalize the real world through mathematical modelling, and establishing a relatively accurate, reliable and effective simulation model (i.e., path planning model) is a key factor for ensuring that the simulation results have high credibility. One basic principle of simulation technology is to simulate a controller of a vehicle by an algorithm in a simulation scene, and achieve the test and verification of the autonomous driving algorithm with reference to sensor simulation and other technologies.

Generally, in a process of automatically generating a road test simulation scene, vehicle obstacles may be artificially set at certain positions on a simulation scene map with a given speed, attitude, and other information to generate false vehicle obstacle perceiving signals, or position points of a lane line are automatically sampled in a real environment and false lane line perceiving signals are generated for corresponding position points in the simulation scene map, to simulate the real road condition scene. Optionally, a simulation scene resembling a real environment can be made based on a graphics processing unit (GPU) . The simulation scene is similar to an animation in the real environment, and perceiving information is calculated based on an algorithm all over again therein.

However, in the above process, a motion state of vehicle obstacles, lane lines, and other perceiving information cannot be truly reflected during the actual road test, and artificially designed vehicle obstacles, lane lines, or other information usually ignore noise interference, making the simulation scene unable to better reproduce the real road condition, resulting in poor simulation effects of the autonomous driving simulation system, and making an autonomous driving algorithm used in a path planning model unable to perform faster and accurate iterative updates, which affects the accuracy of the autonomous driving algorithm and affects the intelligence level of autonomous vehicles.

In view of this, embodiments of the present application provide a simulation method for autonomous driving. Closed-loop simulation can be performed on an autonomous driving process of a simulated vehicle based on reference lane data, reference traffic data, and reference vehicle attitude data in a reference driving data set (here, the reference driving data set is a data set obtained based on data collected from a driving process of a reference vehicle when the reference/real vehicle is travelling on the road, and is used to indicate driving conditions of the reference vehicle at different positions on the road, such as traffic conditions, lane conditions, and vehicle attitude conditions) , and a driving trajectory plan of the simulated vehicle can be determined according to the reference lane data and the reference traffic data in real time, which can at least improve the test efficiency of autonomous driving algorithms, for example, can at least improve the accuracy of test results of the autonomous driving algorithms, reduce the time required to complete the test, or reduce the test amount, so as to improve the overall reliability and safety of the autonomous driving system. The present application can at least alleviate the deficiency that no accurate simulated driving results can be obtained when the autonomous driving is simulated according to a reference driving condition in an open-loop simulated driving test process. In an open-loop simulated driving test, since the position of the simulated vehicle needs to be constantly adjusted according to the position of a real/reference driving vehicle in an open-loop simulated driving process, a complete simulated driving process cannot be obtained.

FIG. 1 is a schematic diagram of an implementation environment of a simulation method for autonomous driving according to an embodiment of the present application. Referring to FIG. 1, a carrier 101 and a computer device 102 are comprised in the implementation environment.

The carrier 101 is configured to collect road test data in an actual travelling/driving process. Optionally, the carrier 101 is equipped with functional modules such as an onboard sensor, a positioning component, a camera component, a controller, a data processor, and an autonomous driving system. The functional modules can utilize modern mobile communication and network technologies such as the Internet of Vehicles, 5th Generation Mobile Networks (5G) and Vehicle To X (V2X, wireless communication technology for vehicles) to implement interchange and sharing between traffic participants, so as to have the functions of sensing and perception, decision-making and planning, and control and execution in complex environments.

Optionally, the carrier 101 comprises traditional vehicles, intelligent vehicles, unmanned vehicles, electric vehicles, bicycles, motorcycles, and other transportation means. The carrier 101 can be manually driven by a driver, or can be driven by an autonomous driving system to implement unmanned driving.

Optionally, the onboard sensor comprises a laser radar, a millimeter wave radar sensor, an acceleration sensor, a gyroscope sensor, a proximity sensor, a pressure sensor, and other data collection units.

In some embodiments, the road test data is a rosbag data packet backhauled by a robot operating system (ROS) during a road test of the carrier 101, and information collected by functional modules such as the camera component and the onboard sensor during the road test of the carrier 101 is stored in the rosbag packet, which is used to sense and track the position and movement attitude of obstacles and lane lines. Optionally, positioning data collected by a positioning component based on a global positioning system (GPS) is also stored in the rosbag packet. Optionally, estimation of a vehicle attitude of the carrier 101 itself from an inertial measurement unit (IMU, also referred to as inertial sensor) is also stored in the rosbag packet. Optionally, timestamps of the foregoing various information are also stored in the rosbag packet.

The carrier 101 and the computer device 102 can be directly or indirectly connected through wired or wireless communication. For example, the carrier 101 and the computer device 102 are wirelessly connected through a vehicle network, which is not limited in the embodiment of the present application.

The computer device 102 is configured to debug parameters of a simulated driving platform to iteratively update the simulated driving platform. Optionally, the computer device 102 comprises at least one of one server, a plurality of servers, a cloud computing platform, or a virtualisation centre. Optionally, the computer device 102 undertakes the main calculation work, while the carrier 101 undertakes the secondary calculation work; alternatively, the computer device 102 undertakes the secondary calculation work, while the carrier 101 undertakes the main calculation work; alternatively, a distributed computing architecture is adopted between the carrier 101 and the computer device 102 to perform collaborative computing.

Optionally, the carrier 101 generally refers to one of a plurality of vehicles. The carrier 101 is equipped with a terminal device for communication connection to the computer device 102. The types of the terminal device include but are not limited to: at least one of a vehicle-mounted terminal, a smart phone, a tablet computer, a smart watch, a smart speaker, an e-book reader, a Moving Picture Experts Group Audio Layer III (MP3) player, a Moving Picture Experts Group Audio Layer IV (MP4) player, a laptop computer or a desktop computer. An autonomous driving system is configured on the terminal device, and the autonomous driving system can plan travelling parameters of the carrier 101 based on a path planning model debugged by the computer device 102.

A person skilled in the art would have known that the number of the foregoing carriers 101 may be larger or smaller. For example, there may be only one carrier 101, or there may be dozens or hundreds of carriers 101 or more carriers. The embodiment of the present application does not limit the number and the device type of carriers 101.

FIG. 2 is a flowchart of a simulation method for autonomous driving according to an exemplary embodiment of the present application. Taking the method being applied to a computer device as an example for description, as shown in FIG. 2, the method comprises:

Step 201. a reference driving data set is obtained, the reference driving data set comprising a reference lane data subset, a reference traffic data subset, and a reference vehicle attitude data subset.

The reference driving data set may be a data set obtained by collecting data from a driving process of a reference vehicle when the reference vehicle is driven on the road, and is used to indicate driving conditions of the reference vehicle at different positions on the road, such as traffic conditions, lane conditions, and vehicle attitude conditions.

Here, the reference lane data subset may comprise reference lane data, the reference traffic data subset may comprise reference traffic data, and the reference vehicle attitude data subset may comprise reference vehicle attitude data. The reference lane data comprises at least one of lane identification information, speed limit information, road material information, and other information. The lane identification information represents information about which lane of a current road the reference vehicle is located in, the speed limit information represents speed limit information corresponding to the lane where the reference vehicle is currently located, and the road material information represents a material of the ground where the reference vehicle is currently located. The reference traffic data comprises at least one of road participating vehicle information, traffic light information, obstacle information, and other information, wherein the information about the road participating vehicle represents information about other vehicles located around the participating vehicle when the participating vehicle travels to a certain position, the traffic light information represents traffic lights that the participating vehicle passes by when travelling on the road, and the indications of the traffic lights (for example, the red light indicates stop, the yellow light indicates slowing down, and the green light indicates passage) , and the obstacle information represents obstacles appearing on the road, such as speed bumps, road curbs, etc. The reference vehicle attitude data comprises at least one of steering wheel information, speed information, acceleration information, vehicle position information, and other information, wherein the steering wheel information is used to represent the current direction control condition of the reference vehicle, the speed information is used to represent a current speed of the reference vehicle, the acceleration information is used to represent a current acceleration of the vehicle, such as an acceleration during a start acceleration stage or a braking stage, and the vehicle position information is used to represent a current distance of the reference vehicle from a start point or an end point. Here, the reference lane data in the reference lane data subset, the reference traffic data in the reference traffic data subset, and the reference vehicle attitude data in the reference vehicle attitude data subset have a correspondence with each other. A correspondence between two or more data sets means that the data in the different data sets is related to each other. For instance the reference lane data, reference traffic data, reference vehicle attitude data may relate to data obtained when the vehicle was in the same position or may relate to data obtained at a same point or points in time; accordingly there is considered to be a correspondence between the data in the different data sets. In an illustrative simulation example, in which a vehicle having the specific reference vehicle attitude data is traveling on a lane having the reference lane data and the traffic on the lane has the reference traffic data at a specific time or in a time range, the correspondence means the these data in the scene correspond to and are connected with each other. Other types of correspondence which serve the purpose the claimed invention all fall within the protective scope of the present application.

The reference driving data set can be obtained in at least one of the following methods:

In one embodiment, the reference vehicle is a real vehicle, and the reference driving data set is obtained by collecting real driving data generated by the reference vehicle, that is, the real vehicle, travelling on a real road. The real driving data may comprise attitude data of the vehicle itself, traffic data, lane data, etc. Here, the traffic data and lane data can be manually input (for example, recognized by a system engineer) , or the traffic data and lane data can be generated by automatically recognizing driving images generated in the driving process.

It is assumed that a human driver drives a reference vehicle from city A and travels to city B to end. Here, vehicle attitude data is data of the travelling condition of the vehicle obtained according to the driving process by the human driver, comprising at least one of steering wheel information, speed information, acceleration information, vehicle position information, and other information of the reference vehicle when the reference vehicle is at different positions of the road in the process of travelling from city A to city B. Environmental images around the vehicle in the driving process by the human driver can be collected to generate a driving video and the driving video is recognized to obtain the traffic data and the lane data, comprising lane detection information, obstacle inspection information, information about a road participating vehicle, etc. when the reference vehicle is at different positions of the road in the process of travelling from city A to city B.

In a second method, the reference vehicle is a vehicle model in a vehicle driving application. A player controls the vehicle model in the vehicle driving application to generate driving data. Here, the driving data comprises attitude data of the vehicle model, traffic data, lane data, etc., wherein the traffic data and lane data are generated according to a three-dimensional virtual environment where the vehicle model is located, or the traffic data and lane data are generated by automatically recognizing driving images generated in the travelling process of the vehicle model.

Exemplarily, the vehicle model departs from a start point of the three-dimensional virtual environment, and the player controls the vehicle model to drive from the start point to the end point on a terminal, and vehicle attitude data is generated according to a control operation of the player on the terminal, comprising at least one of steering wheel data, speed data, acceleration data, vehicle position data, and other information of the vehicle model when the vehicle model is at different positions between the start point and the end point. During the player controlling the vehicle model, images are collected from the three-dimensional virtual environment around the vehicle model, to generate a driving video, and the driving video is recognized to obtain traffic data and lane data.

In an embodiment, the reference driving data set may be obtained according to a real driving video of a real driving process. The real driving video is a video obtained through image collection of a real driving environment in the driving process of a reference vehicle. For example, data can be obtained from image frames of a real driving video to form the reference driving data set. There is a correspondence between the driving video and the data in the reference driving data set.

In some embodiments, the reference lane data subset in the reference driving data set may comprise the following reference lane data: the reference lane data corresponds to an image frame in a driving video, for example: an n ^th frame in the driving video corresponds to reference lane data which is used to represent a lane condition when the reference vehicle is driven to a position corresponding to the n ^th image frame, where n is a positive integer.

In some embodiments, the reference driving data set comprises a reference traffic data subset, the reference traffic data subset comprises reference traffic data, and the reference traffic data corresponds to an image frame in the driving video, for example: an n ^th frame in the driving video corresponds to reference traffic data which is used to represent a traffic condition when the reference vehicle is driven to a position corresponding to the n ^th image frame.

In some embodiments, the reference driving data set comprises a reference vehicle attitude data subset, the reference vehicle attitude data subset comprises reference vehicle attitude data, and the reference vehicle attitude data corresponds to an image frame in the driving video, for example: an n ^th frame in the driving video corresponds to reference vehicle attitude data which is used to represent a vehicle attitude condition of the reference vehicle when the reference vehicle is driven to a position corresponding to the n ^th image frame.

That is, in conjunction with the relationship between each data subset in the reference driving data set and the image frames in the driving video, the reference lane data, the reference traffic data, and the reference vehicle attitude data also have a correspondence with each other, and the correspondence is used to represent a lane condition, a traffic condition, and a attitude condition of the vehicle itself when the vehicle is driven to a certain position. When the reference vehicle is driven to a certain position to correspond to a certain frame or a certain group of image frames (image frames between two adjacent key frames) in the driving video, there is a frame or a group of image frames corresponding to a group of reference lane data, reference traffic data, and reference vehicle attitude data in the driving data set.

In some embodiments, a data packet list is obtained, wherein the data packet list comprises data packets corresponding to different driving time periods, and the data packets are arranged in ascending order of the driving time periods; and the reference driving data set in the data packets is sequentially read from the data packet list.

Step 202. a simulated driving platform is operated according to the reference lane data, so that a simulated vehicle performs autonomous simulated driving on the simulated driving platform.

In some embodiments, the reference lane data is uploaded to the simulated driving platform, and the simulated vehicle uses the reference lane data as the lane for simulated driving when performing simulated driving.

The simulated driving platform is a platform used to complete the path planning and behaviour decision for the simulated vehicle based on actual driving data of the reference vehicle on the basis of unmanned driving and control. Generally, the simulated driving platform completes path planning and behaviour decision through a plurality of functional modules or a plurality of units in one functional module. In some embodiments, before the simulated driving platform is operated to perform autonomous simulated driving of the simulated vehicle, it is also necessary to input basic parameters of the simulated vehicle in the simulated driving platform, for example: a vehicle weight, a load capacity, a top speed, an acceleration per hundred kilometers, the number of compartments, braking sensitivity, throttle sensitivity, etc. Thus, the simulated driving platform can perform simulated control on the simulated vehicle.

Exemplarily, the reference driving data set may comprise a plurality of groups of simulated lane data, wherein the first group of simulated lane data represents data of the lane where the simulated driving platform controls the simulated vehicle to start travelling. For example, the first group of simulated lane data may be data on the second lane of four lanes, which means that the simulated driving platform controls the simulated vehicle to start travelling on the second lane.

Since operating of the simulated driving platform is a cyclic process, that is, in the initial stage, the simulated vehicle is operated to travel on the lane corresponding to the reference lane data according to the reference lane data, and in the subsequent process, the data generated in the driving process and the reference driving data set from the real vehicle are obtained in real time according to the driving condition of the simulated vehicle, so as to continue to control the simulated vehicle.

Step 203. simulated vehicle attitude data of the simulated vehicle is obtained in real time.

The simulated vehicle attitude data is used to represent the vehicle travelling condition of the simulated vehicle in the simulation process of autonomous driving, comprising steering wheel information, speed information, acceleration information, and vehicle position information of the simulated vehicle. Here, the vehicle position information is used to represent a current position of the simulated vehicle, and the vehicle position information can be expressed by a distance from the start point, or by coordinates in a coordinate system constructed between the start point and the end point, which is not limited in the embodiment of the present application.

Here, the vehicle position information may be obtained according to real-time positioning of the simulated vehicle, or the vehicle position information may be inferred according to the speed information, acceleration information, and steering wheel information of the simulated vehicle in the previous simulated driving.

Exemplarily, a current driving speed of the simulated vehicle and a driving distance within a certain period of time can be determined according to the speed information. The change in the driving speed of the simulated vehicle can be determined according to the acceleration information, and the change in the driving direction of the simulated vehicle, the driving distances in different driving directions, and the change between different lanes can also be determined according to the steering wheel information.

Step 204. a deviation in position between the simulated vehicle attitude data and the reference vehicle attitude data is determined, and target traffic data corresponding to the simulated vehicle attitude data is determined from the reference traffic data according to the deviation in position and a correspondence between the reference traffic data and the reference vehicle attitude data.

Optionally, the deviation in position is used to indicate a difference in geographic positions between the simulated vehicle and the reference vehicle at the same time point on the time axis. For example, assuming that a geographic position of the simulated vehicle when the simulated vehicle travels to 00: 10: 00 is a first geographic position, and a geographic position of the reference vehicle when the reference vehicle travels to 00: 10: 00 is a second geographic position, it can be determined that a deviation in position between the simulated vehicle and the reference vehicle at 00: 10: 00 is a difference between the first geographic position and the second geographic position. Here, the difference between the first geographic position and the second geographic position can be represented by a distance. Assuming that there is a first distance between the first geographic position and the start point, and there is a second distance between the second geographic position and the start point, it is determined that the difference between the first geographic position and the second geographic position is a distance difference between the first distance and the second distance.

After the deviation in position between the simulated vehicle and the reference vehicle is determined, the reference traffic data (referred to as “target traffic data” ) of the reference vehicle at a position corresponding to the first geographic position can be determined from the reference traffic data according to the deviation in position, and the target traffic data indicates a road traffic condition of the reference vehicle at the position.

In some embodiments, the reference traffic data comprises position data of a road participating vehicle, that is, the target traffic data comprises the position data of the road participating vehicle when the reference vehicle moves to the first position, for example: the position of the participating vehicle around the reference vehicle when the reference vehicle moves to the first position.

Exemplarily, when the simulated vehicle is driven to minute m (00: m: 00) , current position A of the simulated vehicle is determined, where m is a positive number, and position B of the reference vehicle when the reference vehicle is driven to minute m is determined. A moment when the reference vehicle is at position A is determined according to a deviation in position between position A and position B, and accordingly corresponding target traffic data at that moment is determined. In some embodiments, it is also necessary to determine target lane data in the reference lane data that is corresponding to the target traffic data, that is, information of a lane where the reference vehicle is located at that moment.

Cached reference traffic data is searched according to the deviation in position between the simulated vehicle and the reference vehicle, and in conjunction with the lane where the simulated vehicle is located and the relative distance between the simulated vehicle and the reference vehicle, to determine the target traffic data.

Step 205. a driving trajectory of the simulated vehicle is planned according to the reference lane data and the target traffic data to obtain closed-loop simulation data of simulated driving.

Optionally, the driving trajectory of the simulated vehicle is planned in conjunction with the reference lane data and the position data of the participating vehicle in the target traffic data, to obtain the closed-loop simulation data of simulated driving, wherein the planning process is performed in real time according to the current position of the simulated vehicle. That is, the simulated vehicle modifies a vehicle attitude thereof according to the planned driving trajectory, and continues planning according to the modified vehicle attitude until the simulated vehicle completes the autonomous driving process, to obtain the final closed-loop simulation data of simulated driving.

Optionally, in the process of planning the driving trajectory, an intentional driving trajectory of the road participating vehicle is predicted according to the reference lane data and the position data of the road participating vehicle in the reference traffic data, to obtain driving trajectory prediction data of the participating vehicle. The driving trajectory of the simulated vehicle is planned according to the reference lane data and the driving trajectory prediction data of the participating vehicle, to obtain driving trajectory planning data, and the closed-loop simulation data of simulated driving is obtained according to the driving trajectory planning data. Here, the driving planning of the simulated vehicle also comprises behaviour planning, and the behaviour planning may result in the planning of the driving trajectory to a certain extent.

Here, planning the driving trajectory according to the reference lane data, the simulated vehicle attitude data (which can be used to determine the relationship between the simulated vehicle and the lane) , and the relationship between the simulated vehicle and the lane can at least avoid the occurrence of the simulated vehicle being on the solid lane line, or avoid undesired travelling events such as the simulated vehicle travelling between two lanes for a long time. In some embodiments, in planning the driving trajectory according to the reference lane data, an obstacle on the lane can also be identified, and whether to control the simulated vehicle to avoid the obstacle is determined according to the size, shape, and other parameters of the obstacle. For example, when the obstacle is large, the simulated vehicle is controlled to avoid the obstacle; alternatively, when the obstacle is small but has a shape with a sharp tip, the simulated vehicle is controlled to avoid the obstacle.

In some embodiments, feedback data returned by the simulated vehicle can be obtained, and a control command can be generated according to the feedback data and the driving trajectory planning data. The feedback data is used to indicate the simulated driving condition of the simulated vehicle, for example: the driving direction and driving distance of the simulated vehicle according to the most recent control command. The simulated vehicle is operated according to the generated control command, and the simulated vehicle attitude data is updated according to an operation result of the simulated vehicle. The following steps are repeated according to the updated simulated vehicle attitude data: determining the deviation in position between the simulated vehicle attitude data and the reference vehicle attitude data and determining target traffic data corresponding to the simulated vehicle attitude data from the reference traffic data according to the deviation in position and the correspondence between the reference traffic data and the reference vehicle attitude data, to obtain the closed-loop simulation data of simulated driving.

The phrase closed-loop simulation data of simulated driving means that the simulated driving process is performed in a closed loop. The reference driving data set generated by the reference vehicle is used to indicate road information and traffic information for the closed-loop driving process of the simulated vehicle , thereby assisting vehicle control of the reference vehicle in the closed-loop simulation process. In the overall driving process of the simulated vehicle, the driving trajectory planning and control of the simulated vehicle is performed according to the lane and traffic conditions in the reference driving data, so as to complete the overall simulated driving/travelling process, and the efficiency of the simulated driving test on the simulated driving platform is determined according to the vehicle control condition in the simulated driving/travelling process.

In some examples, closed-loop simulation of simulated driving may include performing the driving trajectory planning and control of the simulated vehicle according to the reference driving data from the real driving process and the position feedback of the simulated vehicle in the simulated driving process. In other words, the process in which the closed-loop simulation platform for simulated driving makes decision on the driving trajectory planning and control of the simulated vehicle according to the reference driving data from the real driving process does not involve adjusting and correcting the position of the simulated vehicle according to the position of the vehicle in the corresponding real driving process.

In summary, according to the simulation method for autonomous driving provided in this embodiment, closed-loop simulation can be performed on an autonomous driving process of a simulated vehicle based on reference lane data, reference traffic data, and reference vehicle attitude data in a reference driving data set, and a driving trajectory plan of the simulated vehicle can be determined according to the reference lane data and the reference traffic data in real time. This can avoid the case where the real and complete driving process of the simulated driving cannot be obtained because real-time adjustment is performed on the simulated driving condition according to the reference driving condition in an open-loop simulated driving test process, and accordingly accurate driving simulation results cannot be obtained, and this can improve the test efficiency of autonomous driving algorithms, for example, can at least improve the accuracy of test results of the autonomous driving algorithms, reduce the time required to complete the test, or reduce the test amount, so as to improve the overall reliability and safety of the autonomous driving system.

Illustratively, FIG. 3 is a structural block diagram of a simulated driving system according to an exemplary embodiment of the present application. As shown in FIG. 3, the system comprises: a data packet list unit 310, a playback and packaging unit 320, a prediction unit 330, a planning unit 340, a control unit 350, and a vehicle model 360.

The playback and packaging unit 320 may comprise the following modules:

1) A frame information caching module 321 is configured to dynamically adjust, according to an overall simulation progress, a rate of reading data. In the closed-loop simulation process of the simulated vehicle, the playback and packaging unit 320 continues to read data from the data packet list unit 310. When the driving speed is fast, the rate of reading data can be increased, and in contrast, when the driving speed is slow, the rate of reading data can be decreased. The data corresponding to each frame of image that is read from data packets can be cached, which may include static information such as lane information and vehicle information, and may also include dynamic information such as traffic information and key information of the vehicle itself. Here, it should be understood that the frame information caching module is not necessary for the implementation of the embodiment.

2) A vehicle attitude deviation management module 322 is configured to generate a deviation of vehicle attitudes from a simulated vehicle attitude obtained in the closed-loop simulation process and the cached reference vehicle attitude, wherein the simulated vehicle attitude is used to represent the vehicle attitude condition of the simulated vehicle, and the reference vehicle attitude is used to represent the vehicle attitude condition of the reference vehicle.

3) A frame information scheduling module 323 searches cached traffic information according to the deviation of the vehicle attitudes that is determined by the vehicle attitude deviation management module 322, and in conjunction with a relative distance between the simulated vehicle and the lane and a relative distance between the simulated vehicle and the reference vehicle, to obtain target traffic data corresponding to the current simulated vehicle.

4) An indicator evaluation module 324 is configured to generate key test indicators. The test indicators refer to preset indicators that need to be evaluated for their completion statuses, such as emergency braking, sudden acceleration, a distance to the vehicle ahead when starting and stopping, or a jitter of the vehicle itself. These indicators are used to determine whether the test cases satisfy normal driving requirements, so as to identify problems in the simulated driving process.

It should be noted that the foregoing functions of the frame information caching module 321, the vehicle attitude deviation management module 322, the frame information scheduling module 323, and the indicator evaluation unit 324 can be implemented in the playback and packaging unit 320, or can be split into a plurality of sub-units for implementation, which is not limited in the embodiment of the present application.

The playback and packaging unit 320 also interacts with other units in the simulated driving system as follows.

1. The playback and packaging unit 320 sends the cached lane information, other vehicle information, and modified vehicle attitude information to the prediction unit 330. The prediction unit 330 predicts the intention and trajectory for behaviour of road participants according to the information received in real time, that is, predicts the intentional trajectory of the road participating vehicle, and the prediction unit 330 sends a prediction result to the planning unit 340.

2. The playback and packaging unit 320 sends the cached lane information and modified vehicle attitude information to the planning unit 340. The planning unit 340 plans the behaviour and trajectory of the simulated vehicle according to the lane information and the modified vehicle attitude information, and according to prediction information obtained from the prediction unit 330. Optionally, the planning unit 340 sends the simulated planned trajectory to the control unit 350. Optionally, the planning unit 340 also feeds back the simulated planned trajectory to the playback and packaging unit 320.

3. The playback and packaging unit 320 sends the modified vehicle attitude information to the control unit 350. The control unit 350 receives the simulated planned trajectory sent by the planning unit 340, generates a control command based on a vehicle feedback returned by the vehicle model 360 of the simulated vehicle, and sends the control command to the vehicle model 360. Here, the vehicle feedback comprises position information, speed information, acceleration information, steering wheel information, or the like of the current simulated vehicle.

4. The vehicle model 360 is controlled according to the control command sent by the control unit 350, to simulate the operation of the simulated vehicle, and then the movement condition of the vehicle is sent to the playback and packaging unit 320.

It should be noted that the playback and packaging unit 320, the prediction unit 330, the planning unit 340, the control unit 350, and the vehicle model 360 can be respectively implemented as different units, or can be implemented as different units in one functional module, which is not limited in the embodiment of the present application.

In some embodiments, after the closed-loop simulation data of simulated driving is obtained, it is also necessary to obtain key test indicators from the closed-loop simulation data of simulated driving. FIG. 4 is a flowchart of a simulation method for autonomous driving according to another exemplary embodiment of the present application. Taking the method being applied to a computer device as an example, as shown in FIG. 4, the method comprises:

Step 401. a reference driving data set is obtained, the reference driving data set comprising a reference lane data subset, a reference traffic data subset, and a reference vehicle attitude data subset.

Here, the reference lane data in the reference lane data subset, the reference traffic data in the reference traffic data subset, and the reference vehicle attitude data in the reference vehicle attitude data subset have a correspondence with each other.

In some embodiments, there is an correspondence between the foregoing reference driving data set and the driving video in the driving process. Exemplarily, the driving video is a video obtained through image collection of a driving environment in the driving process of the reference vehicle, and the data in the reference driving data set is data generated in the driving process of the reference vehicle. Therefore, there is correspondence between the image frames in the driving video and the data in the reference driving data set.

In some embodiments, a data packet list is obtained, wherein the data packet list comprises data packets corresponding to different driving time periods, and the data packets are arranged in ascending order of the driving time periods; and the reference driving data set in the data packets is sequentially read from the data packet list. In some embodiments, the data packets comprise the reference driving data set corresponding to image frames in a reference driving video. Since the data packets comprise the reference driving video, and the reference driving data set is correspondingly stored according to the image frames in the reference driving video, the data packets are sequentially read from the data packet list, and the reference driving data corresponding to the image frames is obtained frame by frame from the data packets and cached. Here, since the reference driving video is a video recorded according to the process of the reference vehicle, and the arrangement sequence of the image frames in the reference driving video corresponds to the generation sequence of the reference driving data in the driving process of the reference vehicle, the reference driving data corresponding to the image frames are obtained sequentially and cached.

Step 402. a simulated driving platform is operated according to the reference lane data, so that a simulated vehicle performs autonomous simulated driving on the simulated driving platform.

Since operating of the simulated driving platform is a cyclic process, that is, in the initial stage, the simulated vehicle is operated to travel on the lane corresponding to the reference lane data according to the reference lane data, and in the subsequent process, the feedback data generated in the driving process and the reference driving data set are obtained in real time according to the driving condition of the simulated vehicle, so as to continue to control the simulated vehicle.

Step 403. simulated vehicle attitude data of the simulated vehicle is obtained in real time.

The simulated vehicle attitude data is used to represent the vehicle travelling condition of the simulated vehicle in the simulation process of autonomous driving, comprising steering wheel information, speed information, acceleration information, and vehicle position information of the simulated vehicle. Here, the vehicle position information is used to represent a current position of the simulated vehicle, and the vehicle position information can be expressed by a distance from the start point, or by coordinates in a coordinate system constructed between the start point and the end point, which is not limited in the embodiment of the present application. Step 404. a deviation in position between the simulated vehicle attitude data and the reference vehicle attitude data is determined, and target traffic data corresponding to the simulated vehicle attitude data is determined from the reference traffic data according to the deviation in position and a correspondence between the reference traffic data and the reference vehicle attitude data.

Optionally, when the deviation in position between the simulated vehicle attitude data and the reference vehicle attitude data is determined, the deviation in position is used to indicate a difference in geographic positions between the simulated vehicle and the reference vehicle at the same time point on the time axis.

After the deviation in position between the first position and the second position is determined, the corresponding target traffic data in the reference traffic data when the reference vehicle is driven to the first position is determined according to the deviation in position, that is, representing the traffic condition on the road when the reference vehicle is driven to the first position.

In some embodiments, the reference traffic data comprises position data of a road participating vehicle, that is, the target traffic data comprises the position data of the road participating vehicle when the reference vehicle is driven to the first position, for example: the position of the participating vehicle around the reference vehicle when the reference vehicle is driven to the first position.

Step 405. a driving trajectory of the simulated vehicle is planned according to the reference lane data and the target traffic data to obtain closed-loop simulation data of simulated driving.

Optionally, the driving trajectory of the simulated vehicle is planned in conjunction with the reference lane data and the position data of the participating vehicle in the target traffic data, to obtain the closed-loop simulation data of simulated driving, wherein the planning process is performed in real time according to the position of the simulated vehicle. That is, the simulated vehicle modifies a vehicle attitude thereof according to the planning, and continues planning according to the modified vehicle attitude until the simulated vehicle completes the autonomous driving, to obtain the final closed-loop simulation data of simulated driving.

Optionally, in the planning process, an intentional trajectory of the road participating vehicle is predicted according to the reference lane data and the position data of the road participating vehicle, to obtain prediction data of the participating vehicle. The driving trajectory of the simulated vehicle is planned by using the reference lane data and the prediction data of the participating vehicle, to obtain trajectory planning data, and the closed-loop simulation data of simulated driving is determined according to the trajectory planning data. Here, when the driving trajectory is planned according to the reference lane data, the relationship between the simulated vehicle and the lane is determined according to the simulated vehicle attitude data, and the driving trajectory is planed according to the relationship with the lane, thereby avoiding the simulated vehicle being on the solid lane line, or avoiding the simulated vehicle being driven between two lanes for a long time.

Step 406. key test indicators are obtained from the closed-loop simulation data of simulated driving.

The key test indicators comprise at least one of a braking indicator, an acceleration indicator, and an inter-vehicle distance indicator. Here, the braking indicator corresponds to a braking acceleration of the simulated vehicle, the acceleration indicator corresponds to a speed-up acceleration of the simulated vehicle, and the inter-vehicle distance indicator corresponds to a distance between the simulated vehicle and another simulated vehicle on the road in a designated driving stage.

Optionally, the key test indicators also comprise a lane deviation indicator, a vehicle condition indicator, etc., wherein the lane deviation indicator is used to indicate a deviation between the simulated vehicle and the central lane line, and the vehicle condition indicator is used to indicate a vehicle condition of the simulated vehicle itself, for example: oil content and tire pressure. In this embodiment, the key test indicators comprising a braking indicator, an acceleration indicator, and an inter-vehicle distance indicator is taken as an example for description.

Optionally, in an autonomous simulated driving process, key test indicators are generated according to the vehicle attitude condition of the vehicle itself, the lane condition, and the traffic condition.

Here, the braking indicator is used to determine whether the acceleration of the simulated vehicle during braking satisfies an appropriate first acceleration requirement for braking, so as to avoid an adverse inertia effect caused by excessive braking. The acceleration indicator is used to determine whether the acceleration of the simulated vehicle during speeding up satisfies an appropriate second acceleration requirement for speeding up, so as to avoid the excessive speed increase causing significant discomfort of the ride (for example, the feeling of pushing back) . The inter-vehicle distance indicator is used to determine that the simulated vehicle is not too close to other vehicles in the designated driving stage, for example: when the simulated vehicle stops, a distance to the vehicle ahead is not too short.

Step 407. the key test indicators are evaluated to obtain a platform evaluation result of the simulated driving platform.

In some embodiments, the indicator evaluation result is obtained from the key test indicators, and time positioning of the indicator evaluation result being failure during the autonomous driving is obtained in response to the indicator evaluation result being failure, to obtain the platform evaluation result.

Here, the indicator evaluation result comprises at least one of the following indicator evaluations:

First, it is determined that the braking indicator fails when the braking acceleration reaches the first acceleration requirement.

Exemplarily, when the braking acceleration is less than -6 m/s ², it is determined that the braking indicator fails, for example: in the driving process of the simulated vehicle, when the simulated vehicle is driven to 00: 12: 20, the braking acceleration is -8m/s ², it is determined that the braking indicator fails when the simulated vehicle is driven to 00: 12: 20.

Second, when the speed-up acceleration reaches the second acceleration requirement, it is determined that the acceleration indicator fails.

Exemplarily, when the speed-up acceleration is greater than 3 m/s ², it is determined that the acceleration indicator fails, for example: in the driving process of the simulated vehicle, when the simulated vehicle is driven to 00: 13: 20, the speed-up acceleration is 6/s ², it is determined that the acceleration indicator fails when the simulated vehicle is driven to 00: 13: 20.

Third, when the distance between the simulated vehicle and the another simulated vehicle on the road in the designated driving stage is less than a distance threshold, it is determined that the inter-vehicle distance indicator fails.

Exemplarily, in response to the distance between the simulated vehicle and the vehicle ahead being less than 0.5 meter when the simulated vehicle stops, it is determined that the inter-vehicle distance indicator fails, for example: in the driving process of the simulated vehicle, when the simulated vehicle is driven to 00: 14: 20, the speed of the simulated vehicle being 0 m/s, and the distance between the simulated vehicle and the vehicle ahead being 0.3 meter, it is determined that when the simulated vehicle stops after having been driven to 00: 14: 20, the distance to the vehicle ahead is too short, and the inter-vehicle distance fails.

It should be noted that the values of the first acceleration requirement, the second acceleration requirement, and a distance threshold are only exemplary distances. In an actual autonomous simulated driving test, the first acceleration requirement, the second acceleration requirement, and the distance threshold are determined based on a designer’s settings, or the first acceleration requirement, the second acceleration requirement, and the distance threshold are determined according to analysis of the autonomous simulated driving process. The embodiment of the present application does not limit the values of the first acceleration requirement, the second acceleration requirement, and the distance threshold.

The method provided in this embodiment provides an end-to-end closed-loop simulation platform based on big data, which can directly use data collected by road tests to quickly perform end-to-end closed-loop simulation.

In the method provided in this embodiment, since the simulated driving platform is a closed-loop simulation platform, the simulated driving platform is easy to expand and facilitates the introduction of more modules. For example, if closed-loop simulation needs to be performed on a lane line, a lane line module can be added to the simulated driving platform, and the interface between the playback and packaging unit and the prediction module can be modified, which improves the flexibility and adaptability of the autonomous driving simulation.

The method provided in this embodiment is extended to support distributed operation. The data packet list can be segmented, and then is operated on a plurality of machines in a distributed way. Then the operating results thereof are aggregated, which improves the operation efficiency of the autonomous simulated driving test.

In the method provided in this embodiment, when algorithm improvement iterations are tested by using actual road driving data playback for closed-loop simulation, the data set is directly used for simulation verification. This eliminates the need for the step of converting the data set into related scenes and then simulating same, and avoids the problem of insufficient test coverage caused by incomplete scene selection, thereby improving the speed and reliability of the test.

FIG. 5 is a structural block diagram of a simulation system for autonomous driving according to an exemplary embodiment of the present application. As shown in FIG. 5, the system comprises: an obtaining module 510, an operating module 520, a determination module 530, and a planning module 540.

The obtaining module 510 is configured to obtain a reference driving data set, the reference driving data set comprising a reference lane data subset, a reference traffic data subset, and a reference vehicle attitude data subset, wherein reference lane data in the reference lane data subset, reference traffic data in the reference traffic data subset, and reference vehicle attitude data in the reference vehicle attitude data subset have a correspondence with each other.

The operating module 520 is configured to operate a simulated driving platform according to the reference lane data, so that a simulated vehicle performs autonomous simulated driving on the simulated driving platform.

The obtaining module 510 is further configured to obtain simulated vehicle attitude data of the simulated vehicle in real time.

The determination module 530 is configured to determine a deviation in position between the simulated vehicle attitude data and the reference vehicle attitude data, and determine target traffic data corresponding to the simulated vehicle attitude data from the reference traffic data according to the deviation in position and a correspondence between the reference traffic data and the reference vehicle attitude data, the deviation in position being used to indicate a difference in geographic positions on the road between the simulated vehicle and a reference vehicle corresponding to the reference driving data set.

The planning module 540 is configured to plan a driving trajectory of the simulated vehicle according to the reference lane data and the target traffic data to obtain closed-loop simulation data of simulated driving.

In an optional embodiment, the reference traffic data comprises position data of a road participating vehicle.

As shown in FIG. 6, the planning module 540 comprises:

a prediction unit 541 configured to predict an intentional trajectory of the road participating vehicle according to the reference lane data and the position data of the road participating vehicle, to obtain prediction data of the participating vehicle;

a planning unit 542 configured to plan the driving trajectory of the simulated vehicle according to the reference lane data and the prediction data of the participating vehicle, to obtain trajectory planning data; and

a determination unit 543 configured to determine the closed-loop simulation data of simulated driving according to the trajectory planning data.

In an optional embodiment, the determination unit 543 is further configured to receive feedback data returned by the simulated vehicle, the feedback data being used to indicate a simulated driving condition of the simulated vehicle; and generate a control command according to the feedback data and the trajectory planning data.

The determination unit 543 is further configured to operate the simulated vehicle according to the control command, and generate updated simulated vehicle attitude data according to an operation result of the simulated vehicle; and repeat, according to the updated simulated vehicle attitude data, steps of determining the deviation in position between the simulated vehicle attitude data and the reference vehicle attitude data and determining target traffic data corresponding to the simulated vehicle attitude data from the reference traffic data according to the deviation in position and the correspondence between the reference traffic data and the reference vehicle attitude data, to obtain the closed-loop simulation data of simulated driving.

In an optional embodiment, the obtaining module 510 is further configured to obtain key test indicators from the closed-loop simulation data of simulated driving; and

the determination module 530 is further configured to evaluate the key test indicators to obtain a platform evaluation result of the simulated driving platform.

In an optional embodiment, the obtaining module 510 is further configured to obtain an indicator evaluation result from the key test indicators; and obtain time positioning of the indicator evaluation result being failure during the autonomous driving in response to the indicator evaluation result being failure, to obtain the platform evaluation result.

In an optional embodiment, the key indicators comprise at least one of a braking indicator, an acceleration indicator, and an inter-vehicle distance indicator.

the braking indicator is corresponding to a braking acceleration of the simulated vehicle;

the acceleration indicator is corresponding to a speed-up acceleration of the simulated vehicle; and

the inter-vehicle distance indicator is corresponding to a distance between the simulated vehicle and another simulated vehicle on the road in a designated driving stage.

In an optional embodiment, the determination module 530 is further configured to, determine that the braking indicator fails when the braking acceleration reaches a first acceleration requirement;

the determination module 530 is further configured to determine that the acceleration indicator fails when the speed-up acceleration reaches a second acceleration requirement; and

the determination module 530 is further configured to determine that the inter-vehicle distance indicator fails when the distance between the simulated vehicle and the another simulated vehicle on the road in the designated driving stage is less than a distance threshold.

In an optional embodiment, the obtaining module 510 is further configured to obtain a data packet list, the data packet list comprising data packets corresponding to different driving time periods, and the data packets being arranged in ascending order of the driving time periods; and read the reference driving data set in the data packets from the data packet list in sequence.

In an optional embodiment, the data packets comprise the reference driving data corresponding to image frames in a reference driving video; and

the obtaining module 510 is further configured to read the data packets from the data packet list in sequence; and obtain the reference driving data corresponding to the image frames frame by frame from the data packets and cache same.

In summary, according to the simulation system for autonomous driving provided in this embodiment, the reference driving data set is obtained, closed-loop simulation can be performed on an autonomous driving process of a simulated vehicle based on reference lane data, reference traffic data, and reference vehicle attitude data in a reference driving data set, and a driving trajectory plan of the simulated vehicle can be determined according to the reference lane data and the reference traffic data in real time. This can avoid the case where the real and complete driving process of the simulated driving cannot be obtained because real-time adjustment is performed on the simulated driving condition according to the reference driving condition in an open-loop simulated driving test process, and accordingly accurate driving simulation results cannot be obtained, and this can improve the accuracy of a simulated driving test and the authenticity of the simulated driving test.

It should be noted that the simulation system for autonomous driving provided in the foregoing embodiments is only exemplified by the division of the foregoing functional modules. In practical applications, the foregoing functions can be allocated to different functional modules according to requirements, that is, the internal structure of the device is divided into different functional modules to complete all or some of the functions described above. In addition, the simulation system for autonomous driving provided in the foregoing embodiments and the simulation method for autonomous driving belong to the same idea, and for specific implementation processes of the simulation system for autonomous driving, reference can be made to the method embodiments, which will not be described in detail herein.

FIG. 7 is a schematic structural diagram of a server according to an exemplary embodiment of the present application. Details are as follows:

A server 700 comprises a central processing unit (CPU) 701, a system memory 704 comprising a random access memory (RAM) 702 and a read only memory (ROM) 703, and a system bus 705 that connects the system memory 704 and the central processing unit 701. The server 700 also comprises a mass storage device 706 for storing an operating system 713, an application program 714, and other program modules 715.

The mass storage device 706 is connected to the central processing unit 701 through a mass storage controller (not shown) connected to the system bus 705. The mass storage device 706 and its associated computer-readable medium provide non-volatile storage for the server 700. That is, the mass storage device 706 may comprise a computer-readable medium (not shown) such as a hard disk or a compact disc read only memory (CD-ROM) drive.

Without loss of generality, the computer-readable medium may comprise a non-transitory computer-readable storage medium and a communication medium. The computer storage medium may comprise any of volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. The computer storage medium comprises a RAM, a ROM, an erasable programmable read only memory (EPROM) , an electrically erasable programmable read only memory (EEPROM) , a flash memory or other solid-state storage technologies, a CD-ROM, a digital versatile disc (DVD) or other optical storage, a tape cassette, a magnetic tape, a disk storage or other magnetic storage devices. Certainly, a person skilled in the art may know that the computer storage medium is not limited to the above-mentioned types. The system memory 704 and the mass storage device 706 can be collectively referred to as memories.

According to various embodiments of the present application, the server 700 may also be connected to a remote computer on a network via the Internet or other networks to run. That is, the server 700 can be connected to a network 712 through a network interface unit 711 connected to the system bus 705, or the network interface unit 711 can also be used to connect to other types of networks or remote computer systems (not shown) .

The foregoing memory also comprises one or more programs, and the one or more programs are stored in the memory and configured to be executed by the CPU.

An embodiment of the present application further provides a computer device, the computer device comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the simulation method for autonomous driving provided in the foregoing method embodiments.

An embodiment of the present application further provides a non-transitory computer-readable storage medium, wherein the computer-readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the simulation method for autonomous driving provided in the foregoing method embodiments.

An embodiment of the present application further provides a computer program product or computer program, wherein the computer program product or the computer program comprises computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device implements the simulation method for autonomous driving described in any of the foregoing embodiments.

Optionally, the computer-readable storage medium may comprise: a read only memory (ROM) , a random access memory (RAM) , solid state drives (SSD) , an optical disc, etc. Here, the random access memory may comprise a resistance random access memory (ReRAM) and a dynamic random access memory (DRAM) . The serial numbers of the foregoing embodiments of the present application are only for description, and do not represent the superiority or inferiority of the embodiments.

A person of ordinary skill in the art can understand that all or some of the steps in the foregoing embodiments can be completed by hardware, or completed by a program instructing relevant hardware. The program can be stored in a computer-readable storage medium. The storage medium mentioned can be a read-only memory, a magnetic disk or an optical disc.

The above description merely relates to optional embodiments of the present application, and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included within the scope of protection of the present application.

Claims

A simulation method for autonomous driving, comprising:

obtaining a reference driving data set, by a processor of a computer system, the reference driving data set comprising a reference lane data subset, a reference traffic data subset, and a reference vehicle attitude data subset, wherein reference lane data in the reference lane data subset, reference traffic data in the reference traffic data subset, and reference vehicle attitude data in the reference vehicle attitude data subset have a correspondence with each other;

running a simulated driving platform based on the reference lane data, so that a simulated vehicle performs simulated autonomous driving on the simulated driving platform;

obtaining, by the processor, simulated vehicle attitude data of the simulated vehicle in real time;

determining, by the processor, a deviation in position between the simulated vehicle attitude data and the reference vehicle attitude data, and determining, target traffic data corresponding to the simulated vehicle attitude data from the reference traffic data according to the deviation in position and the correspondence between the reference traffic data and the reference vehicle attitude data, the deviation in position being used to indicate a difference in geographic positions on the road between a reference vehicle corresponding to the reference driving data set and the simulated vehicle; and

planning, by the processor, a driving trajectory of the simulated vehicle according to the reference lane data and the target traffic data to obtain closed-loop simulation data of simulated driving.
The method according to claim 1, wherein the reference traffic data comprises position data of a road participating vehicle; and

the step of planning by the processor a driving trajectory of the simulated vehicle according to the reference lane data and the target traffic data to obtain closed-loop simulation data of simulated driving further comprises:

predicting by the processor an intended trajectory of the road participating vehicle according to the reference lane data and the position data of the road participating vehicle, to obtain prediction data of the participating vehicle;

planning by the processor the driving trajectory of the simulated vehicle according to the reference lane data and the prediction data of the participating vehicle, to obtain trajectory planning data; and

determining by the processor the closed-loop simulation data of simulated driving according to the trajectory planning data.
The method according to claim 2, wherein the step of determining the closed-loop simulation data of simulated driving according to the trajectory planning data comprises:

receiving by the processor feedback data returned by the simulated vehicle, the feedback data being used to indicate a simulated driving condition of the simulated vehicle;

generating by the processor a control command according to the feedback data and the trajectory planning data;

running the simulated vehicle according to the control command, and generating updated simulated vehicle attitude data according to an operation result of the simulated vehicle; and

repeating by the processor, according to the updated simulated vehicle attitude data, steps of determining the deviation in position between the simulated vehicle attitude data and the reference vehicle attitude data and determining target traffic data corresponding to the simulated vehicle attitude data from the reference traffic data according to the deviation in position and the correspondence between the reference traffic data and the reference vehicle attitude data, to obtain the closed-loop simulation data of simulated driving.
The method according to any one of claims 1 to 3, further comprising below steps after the closed-loop simulation data of simulated driving is obtained:

obtaining by the processor key test indicators from the closed-loop simulation data of simulated driving; and

evaluating by the processor the key test indicators to obtain a platform evaluation result of the simulated driving platform.
The method according to claim 4, wherein the step of evaluating the key test indicators to obtain a platform evaluation result of the simulated driving platform further comprises:

obtaining by the processor an indicator evaluation result from the key test indicators; and

obtaining by the processor time positioning of the indicator evaluation result being failure during the autonomous driving in response to the indicator evaluation result being failure, to obtain the platform evaluation result.
The method according to claim 4, wherein

the key test indicators comprise at least one of a braking indicator, an acceleration indicator, and an inter-vehicle distance indicator,

the braking indicator corresponding to a braking acceleration of the simulated vehicle;

the acceleration indicator corresponding to a speed-up acceleration of the simulated vehicle; and

the inter-vehicle distance indicator corresponding to a distance between the simulated vehicle and another simulated vehicle on the road in a designated driving stage.
The method according to claim 6, further comprising:

determining by the processor that the braking indicator fails when the braking acceleration reaches a first acceleration requirement;

determining by the processor that the acceleration indicator fails when the speed-up acceleration reaches a second acceleration requirement; and

determining by the processor that the inter-vehicle distance indicator fails when the distance between the simulated vehicle and the another simulated vehicle on the road in the designated driving stage less than a distance threshold.
The method according to any one of claims 1 to 3, wherein the step of obtaining a reference driving data set comprises:

obtaining by the processor a data packet list, the data packet list comprising data packets corresponding to different driving time periods, and the data packets being arranged in ascending order of the driving time periods; and

reading by the processor the reference driving data set in the data packets from the data packet list in sequence.
The method according to claim 8, wherein the data packets comprise the reference driving data set corresponding to image frames in a reference driving video; and

the step of reading the reference driving data set in the data packets from the data packet list in sequence comprises:

reading the data packets from the data packet list in sequence; and

obtaining the reference driving data corresponding to the image frames frame by frame from the data packets and caching same.
A simulation system for autonomous driving, including a processor configured to execute the following modules:

an obtaining module configured to obtain a reference driving data set, the reference driving data set comprising a reference lane data subset, a reference traffic data subset, and a reference vehicle attitude data subset, wherein reference lane data in the reference lane data subset, reference traffic data in the reference traffic data subset, and reference vehicle attitude data in the reference vehicle attitude data subset have a correspondence with each other;

an operating module configured to run a simulated driving platform based on the reference lane data, so that a simulated vehicle performs simulated autonomous driving on the simulated driving platform;

the obtaining module is further configured to obtain simulated vehicle attitude data of the simulated vehicle in real time;

a determination module configured to determine a deviation in position between the simulated vehicle attitude data and the reference vehicle attitude data, and determining, target traffic data corresponded to the simulated vehicle attitude data from the reference traffic data according to the deviation in position and the correspondence between the reference traffic data and the reference vehicle attitude data, the deviation in position being used to indicate a difference in geographic positions on the road between a reference vehicle corresponding to the reference driving data set and the simulated vehicle; and

a planning module configured to plan a driving trajectory of the simulated vehicle according to the reference lane data and the target traffic data to obtain closed-loop simulation data of simulated driving.
The system according to claim 10, wherein the reference traffic data comprises position data of a road participating vehicle; and

the planning module comprises:

a prediction unit configured to predict an intended trajectory of the road participating vehicle according to the reference lane data and the position data of the road participating vehicle, to obtain prediction data of the rode participating vehicle;

a planning unit configured to plan the driving trajectory of the simulated vehicle according to the reference lane data and the prediction data of the road participating vehicle, to obtain trajectory planning data; and

a determination unit configured to determine the closed-loop simulation data of simulated driving according to the trajectory planning data.
The system according to claim 11, wherein

the determination unit is further configured to receive feedback data returned by the simulated vehicle, the feedback data being used to indicate a simulated driving condition of the simulated vehicle; and generate a control command according to the feedback data and the trajectory planning data; and

the determination unit is further configured to run the simulated vehicle according to the control command, and generating updated simulated vehicle attitude data according to an operation result of the simulated vehicle; and repeat, according to the updated simulated vehicle attitude data, steps of determining the deviation in position between the simulated vehicle attitude data and the reference vehicle attitude data and determining target traffic data corresponding to the simulated vehicle attitude data from the reference traffic data according to the deviation in position and the correspondence between the reference traffic data and the reference vehicle attitude data, to obtain the closed-loop simulation data of simulated driving.
The system according to any one of claims 10 to 12, wherein

the obtaining module is further configured to obtain processor key test indicators from the closed-loop simulation data of simulated driving; and

the determination module is further configured to evaluate the key test indicators to obtain a platform evaluation result of the simulated driving platform.
The system according to claim 13, wherein

the obtaining module is further configured to an indicator evaluation result from the key test indicators; and obtain time positioning of the indicator evaluation result being failure during the autonomous driving in response to the indicator evaluation result being failure, to obtain the platform evaluation result.
The system according to claim 13, wherein

the key test indicators comprise at least one of a braking indicator, an acceleration indicator, and an inter-vehicle distance indicator,

the braking indicator corresponding to a braking acceleration of the simulated vehicle;

the acceleration indicator corresponding to a speed-up acceleration of the simulated vehicle; and

the inter-vehicle distance indicator corresponding to a distance between the simulated vehicle and another simulated vehicle on the road in a designated driving stage.
The system according to claim 15, wherein

the determination module is further configured to determine the braking indicator fails when the braking acceleration reaches a first acceleration requirement;

the determination module is further configured to determine the acceleration indicator fails when the speed-up acceleration reaches a second acceleration requirement; and

the determination module is further configured to determine the inter-vehicle distance indicator fails when the distance between the simulated vehicle and the another simulated vehicle on the road in the designated driving stage less than a distance threshold.
The system according to any one of claims 10 to 12, wherein

the obtaining module is further configured to obtain a data packet list, the data packet list comprising data packets corresponding to different driving time periods, and the data packets being arranged in ascending order of the driving time periods; and read the reference driving data set in the data packets from the data packet list in sequence.
The system according to claim 17, wherein the data packets comprise the reference driving data corresponding to image frames in a reference driving video; and

the obtaining module is further configured to read the data packets from the data packet list in sequence; and obtain the reference driving data corresponding to the image frames frame by frame from the data packets and caching same.
A computer device for simulating autonomous driving, comprising a processor and a memory configured to store at least one instruction, at least one program, a code set or an instruction set, executable by the processor, wherein the processor when executing the at least one instruction, the at least one program, the code set or the instruction set implements the simulation method for autonomous driving according to any one of claims 1 to 9.
A non-transitory computer-readable storage medium having at least one instruction, at least one program, a code set or an instruction set stored thereon that are executable by a computing device to implement the simulation method for autonomous driving according to any one of claims 1 to 9.
A simulated driving platform, wherein the simulated driving platform comprises a processor and a controller; and the processor and the controller are configured to control a simulated vehicle to implement the simulation method for autonomous driving according to any one of claims 1 to 9.