CN112965917A

CN112965917A - Test method, device, equipment and storage medium for automatic driving

Info

Publication number: CN112965917A
Application number: CN202110404324.1A
Authority: CN
Inventors: 顾天宇; 李亨通; 张博; 沙翔; 沈浴竹
Original assignee: Beijing Voyager Technology Co Ltd
Current assignee: Beijing Voyager Technology Co Ltd
Priority date: 2021-04-15
Filing date: 2021-04-15
Publication date: 2021-06-15

Abstract

According to example embodiments of the present disclosure, test methods, apparatuses, devices, and computer-readable storage media for autonomous driving are provided. The test method for autonomous driving includes setting at least one test scenario for a virtual vehicle based on travel data of an autonomous vehicle in a real environment. Each test scenario includes a driving state of the virtual vehicle and a test environment in which the virtual vehicle is located. The method also includes obtaining configuration information for the trajectory planning, the configuration information specifying a plurality of factors related to travel costs of the trajectory. The method also includes generating a planned trajectory for the virtual vehicle in the at least one test scenario based on the configuration information. The method also includes presenting a first visual representation of the planned trajectory and a second visual representation of the travel cost of the planned trajectory. In this way, it is possible to support adjusting the cost configuration related to the cost of the trajectory in the test, thereby optimizing the trajectory planning function of the autonomous driving system.

Description

Test method, device, equipment and storage medium for automatic driving

Technical Field

Embodiments of the present disclosure relate generally to the field of automated driving, and more particularly, to a test method, apparatus, device, computer-readable storage medium, and program product for automated driving.

Background

The automated driving is a technique of sensing the surroundings of a vehicle, planning a motion trajectory of the vehicle, and controlling the vehicle to reach a specified target by using a computer instead of or in addition to a human driver. An autopilot system in its broadest sense generally comprises two parts, namely a software system and a hardware system. The hardware system includes various sensors for sensing the environment and actuators for causing the vehicle to perform a driving action. The software system comprises various modules for information fusion, path planning, behavior decision and motion control. An important function of the software system is to generate trajectories for autonomous vehicles. Therefore, during the development of a software system, testing is required to enable the software system to generate efficient and secure traces.

Disclosure of Invention

According to an example embodiment of the present disclosure, a test solution for autonomous driving is provided.

In a first aspect of the present disclosure, a test method for autonomous driving is provided. The method includes setting at least one test scenario for the virtual vehicle based on travel data of the autonomous vehicle in the real environment. Each test scenario in the at least one test scenario includes a driving state of the virtual vehicle and a test environment in which the virtual vehicle is located. The method also includes obtaining configuration information for the trajectory planning, the configuration information specifying a plurality of factors related to travel costs of the trajectory. The method further includes generating a planned trajectory for the virtual vehicle in the at least one test scenario based on the configuration information. The method further includes presenting a first visual representation of the planned trajectory and a second visual representation of the travel cost of the planned trajectory.

In a second aspect of the present disclosure, a test device for autonomous driving is provided. The device comprises a scene setting module, a real environment setting module and a virtual vehicle testing module, wherein the scene setting module is configured to set at least one testing scene aiming at the virtual vehicle based on the running data of the automatic driving vehicle in the real environment, and each testing scene in the at least one testing scene comprises the running state of the virtual vehicle and the testing environment where the virtual vehicle is located. The apparatus also includes a configuration obtaining module configured to obtain configuration information for the trajectory planning, the configuration information specifying a plurality of factors related to a cost of travel of the trajectory. The apparatus further includes a trajectory generation module configured to generate a planned trajectory for the virtual vehicle in the at least one test scenario based on the configuration information. The apparatus further includes an information presentation module configured to present a first visual representation of the planned trajectory and a second visual representation of the cost of travel of the planned trajectory.

In a third aspect of the disclosure, an electronic device is provided that includes one or more processors; and storage means for storing the one or more programs which, when executed by the one or more processors, cause the one or more processors to carry out the method according to the first aspect of the disclosure.

In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, implements a method according to the first aspect of the present disclosure.

In a fifth aspect of the present disclosure, a computer program product is provided comprising computer executable instructions, wherein the computer executable instructions, when executed by a processor, implement a method according to the first aspect of the present disclosure.

It should be understood that the statements herein reciting aspects are not intended to limit the critical or essential features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. In the drawings, like or similar reference characters designate like or similar elements, and wherein:

FIG. 1 illustrates a schematic diagram of an example environment in which embodiments of the present disclosure can be implemented;

FIG. 2 illustrates an example of a user interface according to some embodiments of the present disclosure;

FIG. 3 illustrates a flow diagram of a testing method for autonomous driving, according to some embodiments of the present disclosure;

FIG. 4 illustrates an example of a second area of a user interface according to some embodiments of the present disclosure;

FIG. 5 illustrates a flow diagram of a method of setting up a test scenario in accordance with some embodiments of the present disclosure;

FIG. 6 illustrates another example of a second region of a user interface according to some embodiments of the present disclosure;

FIG. 7 illustrates yet another example of a user interface according to some embodiments of the present disclosure;

FIG. 8 shows a schematic block diagram of a testing device for autonomous driving according to some embodiments of the present disclosure; and

FIG. 9 illustrates a block diagram of a computing device capable of implementing various embodiments of the present disclosure.

Detailed Description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.

In describing embodiments of the present disclosure, the terms "include" and its derivatives should be interpreted as being inclusive, i.e., "including but not limited to. The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first," "second," and the like may refer to different or the same object. Other explicit and implicit definitions are also possible below.

As briefly described above, one important function of the software system in an autonomous driving system is to generate trajectories for autonomous vehicles. In the track generation, the total cost of the track needs to be considered. The total cost of the trajectory is related to the different behaviors of the autonomous vehicle, and the cost items corresponding to these behaviors constitute the total cost. For example, the cost term corresponding to the trajectory colliding with the pedestrian is a, the cost term corresponding to the trajectory pressing against the lane boundary is B, the cost term corresponding to the speed exceeding the speed limit of the speed limit area is C, and so on. The total cost of the trace is the sum of these cost terms. In the track generation, the total cost of the planned track can be continuously reduced through iteration. In other words, tracks that make fewer mistakes, i.e., tracks with lower total costs, may be determined to be legitimate tracks.

How to set cost items corresponding to different behaviors is a key issue for trajectory planning. In conventional solutions, the cost term is typically set empirically. This solution is greatly limited by subjective experience and thus does not facilitate the generation of an optimized trajectory. In addition, in conventional solutions, testing is typically performed directly on the autonomous vehicle performing the drive test, and the relationship between trajectory and cost cannot be intuitively provided to the test personnel.

In accordance with an embodiment of the present disclosure, a testing scheme for autonomous driving is presented that is directed to addressing one or more of the problems set forth above and other potential problems. In this solution, at least one test scenario for a virtual vehicle is set based on the driving data of the autonomous vehicle in the real environment. Each set at least one test scene comprises the running state of the virtual vehicle and the test environment where the virtual vehicle is located. Configuration information for trajectory planning is obtained. The configuration information specifies a plurality of factors, such as a plurality of cost items, related to the travel cost of the trajectory. Based on the configuration information, a planned trajectory is generated for the virtual vehicle in the at least one test scenario. A visual representation of the planned trajectory and a visual representation of the travel cost of the planned trajectory are presented.

According to the test solution for autonomous driving proposed herein, it is possible to test a trajectory planning function using driving data in a real environment and visually present a planned trajectory and associated cost information. Thereby, it may be supported to adjust the cost configuration related to the cost of the trajectory in the test, thereby making the calculated cost more reliable. In this way, the trajectory planning function of the autonomous driving system can be optimized. Embodiments of the present disclosure will be described below in detail with reference to the accompanying drawings.

Fig. 1 illustrates a schematic diagram of an example environment 100 in which various embodiments of the present disclosure can be implemented. In general, the example environment 100 includes a real environment 101 and a test environment (not shown) built by a computing device 102. Real environment 101 includes pedestrians 112 waiting to cross the road, roadside trees 113, and vehicles 111 (also referred to as "autonomous vehicles") deployed with on-board system 110. The vehicle 111 travels in the real environment 101. Pedestrians 112, trees 113, roads, etc. constitute the external environment in which the vehicle 111 travels. The vehicle 111 may perform a running test in the real environment 101, and may also actually run in the real environment 101. It should be understood that the real environment 101 shown in fig. 1 is merely illustrative and is not intended to limit the scope of the present disclosure.

The in-vehicle system 110 deployed on the vehicle 111 (e.g., an in-vehicle terminal or other in-vehicle device of the vehicle 111) may include at least a portion of a software system for autonomous driving. For example, in-vehicle system 110 may include various modules (not shown) for information fusion, path planning, behavioral decision-making, and motion control. The in-vehicle system 110 may generate and record environmental information related to the real environment 101 while the vehicle 111 is traveling in the real environment 101. For example, sensing devices (e.g., lidar, cameras, etc.) mounted on the vehicle 111 may sense and collect environmental data, and the in-vehicle system 110 may generate environmental information based on the environmental data. Such environment information may include various information related to the external environment of the vehicle 111 while traveling, such as information indicating the pedestrian 112, information indicating a road, and the like.

In addition to the environmental information, the in-vehicle system 110 may also generate and record travel information relating to a travel action of the vehicle 111. The travel information described herein may refer to various information required to reproduce the travel state of the vehicle 111 in the real environment 101. The travel information may include one or more travel actions made by the vehicle 111 over time during travel, start and/or end times of the one or more travel actions, trajectories followed by the one or more travel actions, triggers for the one or more travel actions, and/or the like.

The in-vehicle system 110 may generate and record the travel information 131 in various suitable manners. In some embodiments, for each travel action of the vehicle 111, travel information related to the travel action, such as a start time of the travel action, a track followed by the travel action, and the like, may be recorded. In such an embodiment, the travel information is recorded according to the travel action.

The computing device 102 may obtain or store the travel information and environmental information generated and recorded by the in-vehicle system 110 as at least a portion of the travel data 130 of the autonomous vehicle in the real environment. The computing device 102 may also obtain or store configuration information 140 for trajectory planning.

The configuration information 140 specifies at least a number of factors related to the cost of travel of the trajectory. These factors are, for example, considered as a number of cost terms that constitute the total cost of the trajectory and may therefore also be referred to herein as "cost terms". The configuration information 140 may indicate a weight for each of a plurality of factors and a manner of contribution to the total cost. The contribution may refer to the mathematical form, e.g., cubic, quadratic, linear, etc., of the corresponding factor in the calculation of the total cost.

These factors relating to the running cost can be divided into four types. The first type of factor relates to the ability of the autonomous vehicle to perform a driving maneuver. In other words, the first type of factor may be a kinematic constraint of the autonomous vehicle. By way of example, the first type of factor may include, but is not limited to, a maximum velocity, a maximum forward acceleration, a maximum lateral acceleration, a range of attitude angles, and the like.

The second type of factor is related to the consumption of the autonomous vehicle to perform a driving maneuver. For example, a second type of factor may be the energy that an autonomous vehicle will spend traveling in a trajectory. By way of example, the second type of factors may include, but is not limited to, consumption caused by planned forward acceleration, lateral acceleration, angle changes, and the like.

A third type of factor relates to a travel action that the autonomous vehicle is prohibited from performing. In other words, the third type of factor may be a strong constraint on the behavior of the autonomous vehicle. By way of example, the third type of factors may include, but is not limited to, violations of traffic regulations, required road speed limits, boundary limits for roads, and the like.

A fourth type of factor relates to a travel action that the autonomous vehicle is restricted from performing. In other words, the fourth type of factor may be a weak constraint on the behavior of the autonomous vehicle. By way of example, the fourth type of factors may include, but is not limited to, a preferred speed range, lane boundaries, comfort requirements, and the like.

In some embodiments, configuration information 140 may be obtained from in-vehicle system 110. In such an embodiment, the factors (including weights and contribution) specified by the configuration information 140 are used by the in-vehicle system 110 in generating the trajectory for the vehicle 111. In some embodiments, the configuration information 140 may be generated by any suitable algorithm or specified by a user.

A trajectory optimizer 120 is deployed at the computing device 102. The trajectory optimizer 120 may provide or draw the user interface 150 via a display unit of the computing device 102. A user, such as a developer of an autopilot system, may interact with trajectory optimizer 120 through user interface 150, for example, to adjust various cost terms.

FIG. 2 illustrates one example of a user interface 150 provided by the trajectory optimizer 120. In general, the user interface 150 may include a first area 210, a second area 220, a third area 230, and a trigger button "start".

The first region 210 is used to present a visual representation of the planned trajectory generated by the trajectory optimizer 120 based on the travel data 130 and the configuration information 140. The first region 210 may also present a visual representation of other information (e.g., constraints) related to the planned trajectory. The second area 220 is used to present options to the user regarding the test scenario and configuration information. To this end, the second area 220 includes a test scenario panel 221 and a configuration information panel 222. The third region 230 is used to present a visual representation of the cost of travel of the planned trajectory. For example, the third region 230 may display the contribution of each cost item to the travel cost individually. The trigger button "start" is used to trigger trajectory generation in response to a user click. The contents presented by the first, second, and

third regions

210, 220, and 230 will be described in detail below.

With continued reference to fig. 1. The environment 100 shown in fig. 1 is merely exemplary. Trajectory optimizer 120 may be implemented or distributed across multiple computing devices. Alternatively, trajectory optimizer 120 may receive travel data 130 and configuration information 140 from other devices. Computing device 102 may be any device with computing capabilities. By way of non-limiting example, the computing device 102 may be any type of stationary, mobile, or portable computing device, including but not limited to a desktop computer, laptop computer, notebook computer, netbook computer, tablet computer, multimedia computer, mobile phone, or the like; all or a portion of the components of the computing device 102 may be distributed in the cloud.

In order to more clearly understand the test scheme for autonomous driving provided by embodiments of the present disclosure, embodiments of the present disclosure will be further described with reference to fig. 3. FIG. 3 shows a flowchart of an example test method 300 for autonomous driving, according to an embodiment of the present disclosure. The method 300 may be implemented by the trajectory optimizer 120 of FIG. 1. For ease of discussion, the method 300 will be described in conjunction with FIG. 1.

At block 310, the trajectory optimizer 120 sets at least one test scenario for the virtual vehicle based on the travel data 130 of the autonomous vehicle in the real environment 101. Each test scenario includes a driving state of the virtual vehicle and a test environment in which the virtual vehicle is located. The travel data 130 may include data of a plurality of real scenes. The data of each real scene may include driving information and environment information in the real scene, as described with reference to fig. 1. The data of each real scene may be stored in a file. Such a file is also referred to as a scene file.

In some embodiments, the set at least one test scenario may be a reproduction of a real scenario. For example, the trajectory optimizer 120 may read data of a real scene in the scene file and set a test scene using the read data.

Refer to fig. 4. Fig. 4 illustrates an example of the second area 220 of the user interface 150 according to some embodiments of the present disclosure. When the test scenario panel 221 is presented, the second area 220 includes a sub-area 410 related to the real scenario, which in turn includes a button "load real scenario" for the user to select to load the real scenario. In response to the user clicking the button "load real scene," the trajectory optimizer 120 may present a directory including one or more scene files for selection by the user. Thus, batch import of scene files can be supported.

If it is recognized that the user selects a certain scene file, the trajectory optimizer 120 may parse the scene file to obtain data of the real scene and set the test scene based on the data. Accordingly, the name of the selected scene file may be presented in the "name" column of table 415. At this time, the "status" column may display information such as "not started".

If it is recognized that the user selects a directory including a plurality of scene files, the trajectory optimizer 120 may parse each scene file under the directory to obtain data of a corresponding real scene, and set a plurality of test scenes based on the parsed data. Accordingly, the names of the respective scene files in the selected directory may be presented in the table 415. At this time, the "status" column may display information such as "not started".

In such an embodiment, by reproducing a real scene in a real environment, driving in the real scene can be restored. In this way, debugging can be performed for specific problems in real scenes.

Alternatively or additionally, in some embodiments, the at least one set test scenario may be a simulation scenario. The simulated scene may be determined by changing the real scene. In such an embodiment, the test scenario panel 221 may include a sub-area 420 related to the simulation scenario.

Such an embodiment is described below with reference to fig. 5. FIG. 5 illustrates a flow diagram of a method 500 of setting up a test scenario according to some embodiments of the present disclosure. Method 500 may be considered as one specific implementation of block 310.

At block 510, trajectory optimizer 120 determines constraints related to the driving environment of the autonomous vehicle from driving data 130. For example, the trajectory optimizer 120 may determine constraints from a scene file selected by the user. The constraints described herein may refer to any condition or parameter that limits or affects the travel of the autonomous vehicle.

The constraints may be used to constrain the geometry of the road on which the autonomous vehicle is traveling. For example, such constraints may specify the width of the road, whether there are turns, the angle of the turn (e.g., quarter turn, S-shape), and so forth. Alternatively or additionally, the constraint may be used to constrain the speed of the vehicle on the road. For example, such a constraint may specify a speed limit, such as a maximum speed, specified by the link. Alternatively or additionally, the constraint may be used to constrain the distance of the autonomous vehicle relative to another vehicle on the road. Such constraints may specify the distance of the autonomous vehicle relative to the preceding vehicle, the speed relative to the preceding vehicle, and the like. Such a constraint may be, for example, an adaptive cruise (ACC) parameter.

At block 520, the trajectory optimizer 120 changes the constraints based on the user input. For example, the trajectory optimizer 120 may change the geometry of the road, the upper limit of the speed, parameters of the ACC, etc. based on user input.

In some embodiments, the user input may indicate to load a pre-generated simulation configuration to change the constraint. The pre-generated simulation configuration may specify a policy to change one or more constraints. Refer to fig. 4. Sub-region 420 may include a button "load simulation configuration" for the user to select to load the simulation configuration. If it is recognized that the button "load simulation configuration" is clicked, the trajectory optimizer 120 may read the pre-generated simulation configuration and change one or more constraints according to the specified policy. For example, the angle of the turn of the road may be increased strategically.

Alternatively or additionally, in some embodiments, the user input may indicate that a simulation configuration is generated to change the constraints. As shown in fig. 4, sub-region 420 may include a button "generate simulation configuration" for the user to select to generate the simulation configuration. If it is recognized that the button "generate simulation configuration" is clicked, the trajectory optimizer 120 may receive from the user the constraint condition that is desired to be changed. As an example, trajectory optimizer 120 may receive user input in table 425 specifying constraints that the user desires to change. For example, the input may specify the ACC parameter to be changed and its value.

At block 530, the trajectory optimizer 120 sets at least one test scenario based on the changed constraints. The unchanged conditions or parameters may maintain the data in the scene file. The trajectory optimizer 120 may utilize the recorded generated simulation configurations and data in the maintained scenario file to set up test scenarios.

In such embodiments, scenes not included in the real environment may be set by changing one or more constraints. In this way, a combination of test scenarios desired by the user can be simulated. In this way, all possible scenes can be covered as much as possible. Testing in more scenarios helps to fully optimize the trajectory planning function of the autopilot system.

With continued reference to fig. 3. At block 320, the trajectory optimizer 120 obtains configuration information 140 for trajectory planning. The configuration information 140 specifies a number of factors related to the cost of travel of the trajectory. These factors may be the four types of factors described with reference to fig. 1. The configuration information 140 may indicate a weight for each of a plurality of factors and a manner of contribution to the total cost (e.g., quadratic, linear, sigmoid function, etc.). As an example, the weight of the maximum speed among the first type of factors may be 5000, and the contribution manner may be quadratic. The weight of comfort in the fourth type of factor may be 25 and the contribution may be linear.

The configuration information 140 may be read from one or more configuration files. Alternatively or additionally, the configuration information 140 may be user input or modified. Refer to fig. 6. Fig. 6 illustrates another example of the second area 220 of the user interface according to some embodiments of the present disclosure. For example, after the test scenario is set, in response to the user clicking on the tab of the configuration information panel 222, the trajectory optimizer 120 may present the configuration information panel 222 as shown in FIG. 6.

The configuration information panel 222 may include a button "add configuration" for the user to select to add configuration information. For example, in response to the button being clicked, the trajectory optimizer 120 may receive configuration information entered by the user in the sub-region 610, such as weights for the respective cost terms. As another example, in response to the button being clicked, the trajectory optimizer 120 may import a file containing configuration information from the outside.

Alternatively or additionally, the configuration information panel 222 may include a button "load configuration" for the user to select to load configuration information. For example, in response to the button being clicked, the trajectory optimizer 120 may present one or more profiles to the user for selection by the user. Then. The trajectory optimizer 120 may load the configuration file selected by the user and parse the configuration information therein.

Alternatively or additionally, the configuration information panel 222 may include a button "modify configuration" for the user to select to modify the configuration information. For example, in response to the button being clicked, the trajectory optimizer 120 may receive a modification to one or more cost terms entered by the user in sub-region 610. For example, the user input may indicate to modify the weight of the maximum speed from 5000 to 3000. As another example, the user input may indicate that the comfort contribution manner is modified from linear to quadratic.

With continued reference to fig. 3. At block 330, the trajectory optimizer 120 generates a planned trajectory for the virtual vehicle in the at least one test scenario based on the configuration information 140. For example, in response to determining that the button "start" shown in FIG. 2 is clicked, the trajectory optimizer 120 begins generating planned trajectories for virtual vehicles in one or more test scenarios.

Trajectory optimizer 120 may include a trajectory generation module that is the same as or similar to the module used to generate the trajectory in-vehicle system 110. At block 330, the trajectory generation module may generate a planned trajectory for the virtual vehicle based on the configuration information 140. To utilize the trajectory generation module, trajectory optimizer 120 may load a configuration associated with the trajectory generation model, which may be obtained, for example, from in-vehicle system 110.

At block 340, the trajectory optimizer 120 presents a visual representation of the planned trajectory (also referred to as a "first visual representation") and a visual representation of the travel cost of the planned trajectory (also referred to as a "second visual representation"). The trajectory optimizer 120 may present the first and second visual representations in the user interface 150 in any suitable manner.

Refer to fig. 7. Fig. 7 illustrates yet another example of a user interface 150 according to some embodiments of the present disclosure. After generating the planned trajectory, the trajectory optimizer 120 may present a visual representation 710 of the planned trajectory in the first region 210. In general, in this example, the visual representation 710 is a line that depicts the orientation of the planned path. Further, the visual representation 710 may additionally include information such as velocity, acceleration, attitude angle, etc. at each point in the planned trajectory. For example, in response to a user clicking on a point on the visual representation 710, the trajectory optimizer 120 may display the velocity, acceleration, attitude angle, etc. at the point.

Additionally, in some embodiments, the trajectory optimizer 120 may also present a visual representation of other information related to the planned trajectory in the first region 210. Such other information may include time-varying conditions, time-invariant conditions, time-varying instructions generated for the test scenario, time-invariant instructions, etc. that make up the test scenario. In the example of fig. 7, a visual representation 711 of the boundary of the road and a visual representation 712 of obstacles on the road are shown.

After generating the planned trajectory, the trajectory optimizer 120 may present a visual representation of the cost of travel of the planned trajectory in the third region 230. The trajectory optimizer 120 may present a visual representation of the total travel cost, as well as a visual representation of a plurality of factors (i.e., a plurality of cost terms) related to the travel cost.

In the example of fig. 7, the trajectory optimizer 120 displays a graph 730 of the travel cost in the third region 230. Graph 730 may include a curve representing total travel cost and a curve representing various cost terms.

The trajectory optimizer 120 may present a plurality of interface elements corresponding to the plurality of cost terms and present, in association with each interface element of the plurality of interface elements, a contribution of the corresponding cost term to the total trajectory cost. The contribution described here may refer to the value of the corresponding cost term, or the proportion of the corresponding cost term in the total driving cost, for the planned trajectory.

In the example of FIG. 7, the trajectory optimizer 120 presents interface elements corresponding to cost items in sub-region 720. Fig. 7 shows the sub-region 720 on an enlarged scale. Interface elements 701 through 709 correspond to factor A, factor B, factor C, factor D, factor E, factor F, factor G, factor H, and factor I, respectively. In some embodiments, interface elements 701 through 709 may have different colors to more intuitively distinguish between different factors. Immediately following the interface element, the values of the corresponding factors in the planned trajectory are displayed. In this example, the values of factor C and factor G, i.e., the costs incurred by factor C and factor G, are 10 and 5, respectively, while the values of the other factors are zero. It should be understood that the values of the factors shown in fig. 7 are merely exemplary, and are not intended to limit the scope of the present disclosure.

In this way, a user (e.g., a developer of an automated driving system) may be made to intuitively feel the role of the different factors in the planned trajectory. After the user clicks on the interface element corresponding to a certain factor, the factor may also be highlighted.

Furthermore, in some embodiments, after generating a planned trajectory for a test scenario, a scenario file for setting up the test scenario may be identified accordingly. For example, the "status" column of the table 415 shown in fig. 4 may display an element such as "tested" corresponding to the scene file.

In some embodiments, the trajectory optimizer 120 may support real-time modification of the configuration information 140. For example, a user may modify one or more factors through the user interface 150, such as modifying weights, contribution ways, or other possible parameters of one or more factors.

In response to presenting a visual representation of the cost of travel, the trajectory optimizer 120 may receive user input. The user input may indicate a factor that the user desires to modify. The trajectory optimizer 120 may adjust at least one factor of the plurality of factors based on the user input. That is, the trajectory optimizer 120 may modify the factors that the user desires to modify in accordance with the user input. Next, the trajectory optimizer 120 may update the planned trajectory generated for the virtual vehicle based on the adjusted factors and present a visual representation of the difference between the pre-updated planned trajectory and the updated planned trajectory. In such an embodiment, the cost configuration may be changed in real-time and the trajectory generated in real-time. In this way, the adjustment of the cost configuration can be efficiently performed.

As an example, after viewing the contribution of the various factors presented to the total cost of travel, the user may desire to modify some factor, such as its weight, manner of contribution, or additional parameters, etc. The user may click the button "modify configuration" in the second area 220 shown in fig. 6 and enter or specify the factors desired to be modified. Upon receiving the user input, the trajectory optimizer 120 may save the modified factors to generate modified configuration information and generate a new planning trajectory based on the modified configuration information. The trajectory optimizer 120 may present a visual representation of the new planned trajectory in the first region 210. The visual representation may be displayed in superimposition with the visual representation 710 shown in fig. 7. In this way, the user can be visually presented with the difference of the two planned trajectories.

With continued reference to fig. 3. If multiple test scenarios are set at block 310, blocks 320 through 340 may be performed in sequence for the multiple test scenarios set. For example, blocks 320 through 340 may be performed in sequence in the order in which the test scenarios are arranged in table 415.

In some embodiments, the method 300 may also include additional steps or blocks. For each test scenario of the plurality of test scenarios, it is determined whether the reference attribute of the planned trajectory generated at block 330 is better than the reference attribute of the real trajectory corresponding to the test scenario. The reference attribute as used herein may be any suitable metric for assessing the goodness of a trajectory, such as lateral clearance (clearance), changes in velocity, changes in acceleration, and the like. The scope of the present disclosure is not limited in this respect.

The planned trajectory may be considered superior to the real trajectory if it is determined that the reference attribute of the planned trajectory is superior to the reference attribute of the real trajectory for more than a threshold number of the plurality of test scenarios. In this case, the trajectory optimizer 120 may output configuration information used to generate the planned trajectory for use in generating the trajectory for the autonomous vehicle. That is, such configuration information is a better performing cost configuration for most test scenarios.

The above-described process may be performed for a plurality of configurations of the running cost, i.e., a plurality of combinations of the cost items. This helps to find a cost configuration with better generalization performance. Such a cost configuration may be suitable for most autonomous driving scenarios.

As can be seen from the above description, embodiments according to the present disclosure may test a trajectory planning function using driving data in a real environment and visually present a planned trajectory and associated cost information. Thereby, it is possible to support adjusting the cost configuration in the test in order to find a more reliable cost configuration scheme. This may facilitate optimizing the trajectory planning function of the autonomous driving system. Further, it should be understood that the number and shape, relative position, values, etc. of the interface elements in the user interfaces shown in fig. 1, 2, 4, 6, and 7 are exemplary and not intended to limit the scope of the present disclosure.

Fig. 8 shows a schematic block diagram of a testing device 800 for autonomous driving according to some embodiments of the present disclosure. The apparatus 800 may be included on the computing device 102 of fig. 1. For example, the apparatus 800 may be used to implement the trajectory optimizer 120 shown in FIG. 1.

As shown in FIG. 8, the apparatus 800 includes a ring scenario setting module 810 configured to set at least one test scenario for a virtual vehicle based on travel data of an autonomous vehicle in a real environment, each test scenario of the at least one test scenario including a travel state of the virtual vehicle and a test environment in which the virtual vehicle is located. The apparatus 800 further includes a configuration obtaining module 820 configured to obtain configuration information for trajectory planning, the configuration information specifying a plurality of factors related to cost of travel of the trajectory. The apparatus 800 further comprises a trajectory generation module 830 configured to generate a planned trajectory for the virtual vehicle in the at least one test scenario based on the configuration information. The apparatus 800 further includes an information presentation module 840 configured to present a first visual representation of the planned trajectory and a second visual representation of the cost of travel of the planned trajectory.

In some embodiments, the apparatus 800 further comprises: a trajectory evaluation module configured to determine, for each of the at least one test scenario, whether a reference attribute of the planned trajectory is superior to a reference attribute of a real trajectory corresponding to each test scenario; and a configuration output module configured to output configuration information for generating a trajectory for the autonomous vehicle if it is determined that the reference attribute of the planned trajectory is better than the reference attribute of the real trajectory for more than a threshold number of the at least one test scenarios.

In some embodiments, the scenario setup module 820 includes: a condition determination module configured to determine a constraint condition related to a running environment of the autonomous vehicle from the running data; a condition changing module configured to change the constraint condition based on the first user input; and a condition utilization module configured to set at least one test scenario based on the changed constraint condition.

In some embodiments, the constraint is for constraining at least one of: the geometry of the road on which the autonomous vehicle is traveling, the speed of the vehicle on the road, or the distance of the autonomous vehicle relative to another vehicle on the road.

In some embodiments, the apparatus 800 further comprises: an input receiving module configured to receive a second user input in response to presenting the second visual representation; a factor adjustment module configured to adjust at least one factor of the plurality of factors based on a second user input; a trajectory update module configured to update the planned trajectory generated for the virtual vehicle based on the adjusted at least one factor; and a difference presentation module configured to present a third visual representation of a difference between the planned trajectory and the updated planned trajectory.

In some embodiments, the information presentation module 840 includes: an interface element presentation module configured to present a plurality of interface elements corresponding to a plurality of factors; and a numerical presentation module configured to present, in association with each interface element of the plurality of interface elements, a contribution of the corresponding factor to a cost of travel of the planned trajectory.

In some embodiments, the plurality of factors relate to at least one of: an ability of the autonomous vehicle to perform a travel action, a consumption of the travel action by the autonomous vehicle, a travel action for which the autonomous vehicle is prohibited from performing, or a travel action for which the autonomous vehicle is restricted from performing.

Fig. 9 illustrates a schematic block diagram of an example device 900 that may be used to implement embodiments of the present disclosure. Device 900 may be used to implement computing device 102 of fig. 1. As shown, device 900 includes a Central Processing Unit (CPU)901 that can perform various appropriate actions and processes in accordance with computer program instructions stored in a Read Only Memory (ROM)902 or loaded from a storage unit 908 into a Random Access Memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The CPU 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input/output (I/O) interface 905 is also connected to bus 904.

A number of components in the device 900 are connected to the I/O interface 905, including: an input unit 906 such as a keyboard, a mouse, and the like; an output unit 907 such as various types of displays, speakers, and the like; a storage unit 908 such as a magnetic disk, optical disk, or the like; and a communication unit 909 such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 909 allows the device 900 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.

The processing unit 901 performs the various methods and processes described above, such as any of the

processes

300 and 500. For example, in some embodiments, either of

processes

300 and 500 may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 900 via ROM 902 and/or communications unit 909. When the computer program is loaded into RAM 903 and executed by CPU 901, one or more steps of any of

processes

300 and 500 described above may be performed. Alternatively, in other embodiments, CPU 901 may be configured to perform any of

processes

300 and 500 in any other suitable manner (e.g., by way of firmware).

The present disclosure may be methods, apparatus, systems, and/or computer program products. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied thereon for carrying out various aspects of the present disclosure.

The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.

The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.

The computer program instructions for carrying out operations of the present disclosure may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, the electronic circuitry that can execute the computer-readable program instructions implements aspects of the present disclosure by utilizing the state information of the computer-readable program instructions to personalize the electronic circuitry, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA).

Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processing unit of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The foregoing description of the embodiments of the present disclosure has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

The embodiment of the present disclosure discloses:

ts1. a test method for autonomous driving, comprising:

setting at least one test scene aiming at a virtual vehicle based on the running data of an automatic driving vehicle in a real environment, wherein each test scene in the at least one test scene comprises the running state of the virtual vehicle and the test environment where the virtual vehicle is located;

obtaining configuration information for trajectory planning, the configuration information specifying a plurality of factors relating to travel costs of a trajectory;

generating a planned trajectory for the virtual vehicle in the at least one test scenario based on the configuration information; and

presenting a first visual representation of the planned trajectory and a second visual representation of the travel cost of the planned trajectory.

Ts2. the method of TS1, further comprising:

for each of the at least one test scenario, determining whether a reference attribute of the planned trajectory is better than the reference attribute of a real trajectory corresponding to the each test scenario; and

outputting the configuration information for use in generating a trajectory for an autonomous vehicle if it is determined that the reference attribute of the planned trajectory is better than the reference attribute of the real trajectory for more than a threshold number of the at least one test scenario.

Ts3. the method of TS1, wherein setting the at least one test scenario for the virtual vehicle comprises:

determining from the driving data a constraint condition relating to a driving environment of the autonomous vehicle;

changing the constraint condition based on a first user input; and

setting the at least one test scenario based on the changed constraint.

Ts4. the method according to TS3, wherein the constraint is for constraining at least one of:

the geometry of the road on which the autonomous vehicle is traveling,

speed of vehicles on said road, or

A distance of the autonomous vehicle relative to another vehicle on the road.

Ts5. the method of TS1, further comprising:

receiving a second user input in response to presenting the second visual representation;

adjusting at least one factor of the plurality of factors based on the second user input;

updating the planned trajectory generated for the virtual vehicle based on the adjusted at least one factor; and

presenting a third visual representation of a difference between the planned trajectory and the updated planned trajectory.

Ts6. the method of TS1, wherein presenting the second visual representation includes:

presenting a plurality of interface elements corresponding to the plurality of factors; and

presenting, in association with each interface element of the plurality of interface elements, a contribution of the corresponding factor to the cost of travel of the planned trajectory.

Ts7. the method of TS1, wherein the plurality of factors relate to at least one of:

the ability of the autonomous vehicle to perform a driving maneuver,

the autonomous vehicle performs a consumption of a travel action,

a running action which the autonomous vehicle is prohibited from performing, or

The autonomous vehicle is restricted from performing the travel action.

Ts8. a test device for autonomous driving, comprising:

the system comprises a scene setting module, a real environment setting module and a virtual vehicle monitoring module, wherein the scene setting module is configured to set at least one test scene aiming at a virtual vehicle based on running data of an automatic driving vehicle in a real environment, and each test scene in the at least one test scene comprises a running state of the virtual vehicle and a test environment where the virtual vehicle is located;

a configuration obtaining module configured to obtain configuration information for trajectory planning, the configuration information specifying a plurality of factors relating to a travel cost of a trajectory;

a trajectory generation module configured to generate a planned trajectory for the virtual vehicle in the at least one test scenario based on the configuration information; and

an information presentation module configured to present a first visual representation of the planned trajectory and a second visual representation of the travel cost of the planned trajectory.

Ts9. the apparatus of TS8, further comprising:

a trajectory evaluation module configured to determine, for each of the at least one test scenario, whether a reference attribute of the planned trajectory is better than the reference attribute of a real trajectory corresponding to the each test scenario; and

a configuration output module configured to output the configuration information for use in generating a trajectory for an autonomous vehicle if it is determined that the reference attribute of the planned trajectory is better than the reference attribute of the real trajectory for more than a threshold number of the at least one test scenarios.

Ts10. the apparatus according to TS8, wherein the scene setting module comprises:

a condition determination module configured to determine a constraint condition related to a running environment of the autonomous vehicle from the running data;

a condition changing module configured to change the constraint condition based on a first user input; and

a condition utilization module configured to set the at least one test scenario based on the changed constraint condition.

Ts11. the apparatus according to TS10, wherein the constraint is for constraining at least one of:

the geometry of the road on which the autonomous vehicle is traveling,

speed of vehicles on said road, or

A distance of the autonomous vehicle relative to another vehicle on the road.

Ts12. the apparatus of TS8, further comprising:

an input receiving module configured to receive a second user input in response to presenting the second visual representation;

a factor adjustment module configured to adjust at least one factor of the plurality of factors based on the second user input;

a trajectory update module configured to update the planned trajectory generated for the virtual vehicle based on the adjusted at least one factor; and

a difference presentation module configured to present a third visual representation of a difference between the planned trajectory and the updated planned trajectory.

Ts13. the device according to TS8, wherein the information presentation module comprises:

an interface element presentation module configured to present a plurality of interface elements corresponding to the plurality of factors; and

a numerical presentation module configured to present, in association with each interface element of the plurality of interface elements, a contribution of the corresponding factor to the travel cost of the planned trajectory.

The ts14. apparatus according to TS8, wherein the plurality of factors relate to at least one of:

the ability of the autonomous vehicle to perform a driving maneuver,

the autonomous vehicle performs a consumption of a travel action,

a running action which the autonomous vehicle is prohibited from performing, or

The autonomous vehicle is restricted from performing the travel action.

Ts15. an electronic device, the device comprising:

one or more processors; and

storage means for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to carry out the method of any one of TS 1-7.

Ts16. a computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of TS1-TS 7.

Ts17 a computer program product comprising computer executable instructions, wherein the computer executable instructions, when executed by a processor, implement a method as any one of TS1-TS 7.

Claims

1. A test method for autonomous driving, comprising:

2. The method of claim 1, further comprising:

3. The method of claim 1, wherein setting the at least one test scenario for the virtual vehicle comprises:

changing the constraint condition based on a first user input; and

setting the at least one test scenario based on the changed constraint.

4. The method of claim 3, wherein the constraint is for constraining at least one of:

the geometry of the road on which the autonomous vehicle is traveling,

speed of vehicles on said road, or

A distance of the autonomous vehicle relative to another vehicle on the road.

5. The method of claim 1, further comprising:

6. The method of claim 1, wherein presenting the second visual representation comprises:

7. A test device for autonomous driving, comprising:

8. An electronic device, the device comprising:

one or more processors; and

storage means for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to carry out the method according to any one of claims 1-7.

9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-6.

10. A computer program product comprising computer executable instructions, wherein the computer executable instructions, when executed by a processor, implement the method of any one of claims 1-6.