CN112685289A - Scene generation method, and scene-based model in-loop test method and system - Google Patents
Scene generation method, and scene-based model in-loop test method and system Download PDFInfo
- Publication number
- CN112685289A CN112685289A CN202011448739.0A CN202011448739A CN112685289A CN 112685289 A CN112685289 A CN 112685289A CN 202011448739 A CN202011448739 A CN 202011448739A CN 112685289 A CN112685289 A CN 112685289A
- Authority
- CN
- China
- Prior art keywords
- scene
- test
- model
- elements
- appointed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000010998 test method Methods 0.000 title claims abstract description 12
- 238000012360 testing method Methods 0.000 claims abstract description 289
- 238000011156 evaluation Methods 0.000 claims abstract description 40
- 238000005070 sampling Methods 0.000 claims abstract description 9
- 230000001276 controlling effect Effects 0.000 claims description 13
- 230000001105 regulatory effect Effects 0.000 claims description 5
- 230000003068 static effect Effects 0.000 claims description 5
- 238000012544 monitoring process Methods 0.000 claims description 2
- 230000000977 initiatory effect Effects 0.000 claims 1
- 238000004088 simulation Methods 0.000 abstract description 18
- 238000012795 verification Methods 0.000 abstract description 6
- 230000001133 acceleration Effects 0.000 description 18
- 238000010586 diagram Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 5
- 230000003044 adaptive effect Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 2
- 238000011990 functional testing Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 241001465754 Metazoa Species 0.000 description 1
- 238000000342 Monte Carlo simulation Methods 0.000 description 1
- 238000012614 Monte-Carlo sampling Methods 0.000 description 1
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013075 data extraction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000002405 diagnostic procedure Methods 0.000 description 1
- 238000004141 dimensional analysis Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000002347 injection Methods 0.000 description 1
- 239000007924 injection Substances 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000035484 reaction time Effects 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The embodiment of the application discloses a scene generation method, a scene-based model in-loop test method and a scene-based model in-loop test system, and relates to the technical field of automatic driving simulation tests. The method comprises the following steps: obtaining a plurality of functional scenes by combining scene elements, and determining specified scene elements in the functional scenes; extracting appointed scene elements from real traffic scene data, and counting the value distribution of the appointed scene elements; assigning the value distribution to corresponding appointed scene elements in a functional scene to obtain a plurality of logic scenes; sampling the value distribution of the specified scene elements in each logic scene to generate a plurality of value samples; and replacing the corresponding value distribution with the plurality of value samples to generate a plurality of test scenes. The test scene generated by the embodiment has wide coverage, realizes automatic setting and execution of the test case, standardizes evaluation indexes and flows, improves test efficiency, and ensures effective verification of the model function.
Description
Technical Field
The embodiment of the application relates to an automatic driving simulation test technology, in particular to a scene generation method, a scene-based model in-loop test method and a scene-based model in-loop test system.
Background
With the improvement of the automatic driving level, the conventional automobile-oriented test tool and test method cannot meet the requirement of the automatic driving automobile test. The scene-based virtual test method has great technical advantages in the aspects of test efficiency, test cost and the like, and is an important means for test and verification of the future automatic driving automobile.
Model in the Loop (MIL) is a simulation mode performed in the initial stage of the development stage and the modeling stage when the Model in the Loop is used for developing the embedded system by using Model driving. When the model is used for carrying out simulation test on the automatic driving in a loop mode, the problems that a test scene is single, coverage is not wide, evaluation indexes and processes are not standard, a test case is manually set, workload is large, and test efficiency is low exist, and effective verification of functions cannot be guaranteed finally.
Disclosure of Invention
The embodiment of the application provides a scene generation method, a scene-based model in-loop test method and a scene-based model in-loop test system, so as to ensure effective verification of the functions of the model.
In a first aspect, an embodiment of the present application provides a method for generating a scene, including:
obtaining a plurality of functional scenes by combining scene elements, and determining specified scene elements in the functional scenes; the scene elements comprise static elements, dynamic elements and weather elements;
extracting appointed scene elements from real traffic scene data, and counting the value distribution of the appointed scene elements;
assigning the value distribution to corresponding appointed scene elements in a functional scene to obtain a plurality of logic scenes;
sampling the value distribution of the specified scene elements in each logic scene to generate a plurality of value samples;
and replacing the corresponding value distribution with the plurality of value samples to generate a plurality of test scenes.
In a second aspect, an embodiment of the present application further provides a method for testing a model in a ring based on a scene, including:
starting the test operation of a scene test case;
loading a corresponding logic scene, a value sample of an appointed scene element and a model evaluation index applicable to the logic scene through the scene test case;
assigning the value sample of the appointed scene element to the corresponding appointed scene element in the logic scene to generate a test scene; controlling the model to be tested to test in the test scene, and reading test data;
comparing the model evaluation index with the test data through the scene test case to obtain and output a test result;
and starting the test operation of the next scene test case until the test of all the scene test cases is completed.
In a third aspect, an embodiment of the present application further provides a model in-loop test system based on a scene, including: the system comprises a test module, a scene building module and a model to be tested;
the test module is used for starting the test operation of a scene test case; loading a model evaluation index applicable to a corresponding logic scene and a value sample of an appointed scene element through the scene test case, and loading the logic scene from the scene building module; assigning the value sample of the appointed scene element to the corresponding appointed scene element in the logic scene to generate a test scene; controlling the model to be tested to test in the test scene, and reading test data; comparing the model evaluation index with the test data through the scene test case to obtain and output a test result; starting the test operation of the next scene test case until the test of all the scene test cases is completed;
the model to be tested is used for testing in the test scene and transmitting test data to the test module;
the scene building module comprises a plurality of logic scenes.
In the embodiment, the value sample of the specified scene element is assigned to the corresponding specified scene element in the logic scene to generate the test scene, so that the probability distribution of the original scene element can be truly reflected by the parameters in the test scene, and a plurality of test scenes can be automatically generated in batch to realize the full coverage of the scene; moreover, the embodiment provides a model-in-loop test method for sequentially and automatically starting and executing a plurality of scene test cases, manual intervention is not needed, and the test efficiency is effectively improved; the execution process of each scene test case comprises three parts of scene generation, test and comparison, wherein model evaluation indexes are integrated into the scene test cases, the evaluation indexes and the flow are standardized, the test efficiency is improved, and the effective verification of the functions of the model is ensured.
Drawings
Fig. 1a is a schematic flowchart of a scene generation method according to an embodiment of the present invention;
FIG. 1b is a schematic diagram of a cut-in scenario in a test scenario provided by an embodiment of the present invention;
FIG. 2 is a flowchart of a scenario-based model-in-loop testing method according to an embodiment of the present invention;
FIG. 3a is a schematic structural diagram of a model-in-loop test system based on a scene according to an embodiment of the present invention;
FIG. 3b is an interaction diagram of a scenario-based model-in-the-loop test system according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating a display effect of a test scenario according to an embodiment of the present invention;
FIG. 5 is a graphical representation of test results provided by an embodiment of the present invention.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the application and are not limiting of the application. It should be further noted that, for the convenience of description, only some of the structures related to the present application are shown in the drawings, not all of the structures.
The embodiment of the application provides a scene generation method, a flowchart of which is shown in fig. 1a, and which is applicable to a situation of generating a scene for an unmanned test. With reference to fig. 1a, the method provided in this embodiment specifically includes:
s110, obtaining a plurality of functional scenes by combining scene elements, and determining appointed scene elements in the functional scenes; the scene elements include a static element, a dynamic element, and a weather element.
Specifically, all possible elements and combinations of the scenes are defined first, and a plurality of functional scenes are constructed according to a combination method. Optionally, the dynamic elements include, but are not limited to, uniform speed, acceleration, deceleration, cut-in, cut-out, and traverse of the moving object. Static elements include, but are not limited to, road type, number of lanes, lane width, road surface condition, straight lanes, curves, ramps, object lane division patterns, line types, traffic sign conditions, special areas. The weather elements comprise sunny days, rain, snow and fog, visibility, daytime, night and backlight.
The specified scene element is an element that has a significant influence on the model test, and may be all or part of the above elements.
And S120, extracting the appointed scene elements from the real traffic scene data, and counting the value distribution of the appointed scene elements.
The test scene is established on the basis of the real traffic scene, and data of the real traffic scene, such as a track, a speed and the like, needs to be acquired.
Optionally, scene marking and screening are performed on real traffic scene data to obtain different real scenes; and extracting appointed scene elements according to real traffic scene data under different scenes. And after the appointed scene elements are extracted from the real traffic scene data, carrying out data processing on the appointed scene elements to obtain value distribution. The value distribution comprises a value range and a distribution mode, the distribution mode comprises average distribution and normal distribution, and the normal distribution comprises a mean and a variance value.
And S130, assigning the value distribution to corresponding appointed scene elements in the functional scene to obtain a plurality of logic scenes.
And S140, sampling the value distribution of the specified scene elements in each logic scene to generate a plurality of value samples.
After assignment, the value of a designated scene element is a group of data, but the group of data is derived from real traffic scene data and has certain one-sidedness. Therefore, the value distribution of the designated scene elements in each logic scene also needs to be sampled, and the sampling method can be a Monte Carlo method, so that a plurality of value samples are obtained, the probability distribution of the original scene elements can be truly reflected, and the full coverage of the scene is realized.
S150, replacing the corresponding value distribution with the plurality of value samples to generate a plurality of test scenes.
Optionally, a plurality of logical scenes may be stored in the scene library, and a plurality of value samples may be stored in the data table. And when the subsequent automatic test is carried out, the value sample can be read from the data table and assigned to the corresponding appointed scene element in the logic scene, and a test scene is generated.
Optionally, a plurality of test scenarios may also be integrally stored in the scenario library and participate in any scenario-based model-in-loop test method.
According to the method, through large-scale real traffic scene data extraction, scene labeling and scene screening, a scene library which is suitable for different automatic driving system tests and can represent traffic characteristics of typical areas in China can be obtained, and based on multi-dimensional analysis (types and ranges of elements) and Monte Carlo sampling of scene elements, a large number of random test samples can be obtained, and meanwhile, real reflection of the test scene on probability distribution of original scene elements can be guaranteed. Fig. 1b is a schematic diagram of a cut-in scenario in a test scenario provided in an embodiment of the present invention.
In the embodiment, a plurality of functional scenes are obtained in a scene element combination mode, and then a plurality of logic scenes are obtained by combining the value distribution of the appointed scene elements in the real traffic scene data, so that the scene coverage is initially expanded; and further sampling the value distribution of the specified scene elements in each logic scene to generate a plurality of value samples, replacing the corresponding value distribution with the plurality of value samples to generate a plurality of test scenes, namely further expanding the scene coverage by a sampling mode and ensuring the effective verification of the model function.
In the above-described embodiments and the following-described embodiments, the logical scene includes at least one of a standard regulation scene, a natural driving scene, a dangerous scene, a corner scene, and a combination scene. Optionally, the logical scenes generated by the method provided in fig. 1a are natural driving scenes and dangerous scenes.
Specifically, the standard regulation scenario is a scenario built according to the standard regulation. Dangerous scenes mainly cover three major scenes of severe weather environment, complex road traffic and typical traffic accidents. The corner scene refers to a scene containing special corners in a logic scene, such as a scene with animals appearing in a high-speed kilometer, a raised ground, a roadside parking, a giant billboard above a road and the like.
The combined scene is obtained by counting, analyzing and combining scene elements in a standard regulation scene, a natural driving scene, a dangerous scene and a corner scene. Standard and regulation scenes, natural driving scenes, dangerous scenes, corner scenes and combined scenes can be stored in a scene library, and element values are stored in a data table; or may be stored as a whole in the scenario library as a test scenario.
In this embodiment, the scene element values of the standard regulation scene, the natural driving scene, the dangerous scene, the corner scene, and the combination scene may be a plurality of values satisfying one distribution, such as the natural driving scene in the above embodiment; or may be a fixed value, such as a standard regulatory scenario; but also a few values specified, such as a combined scene. And assigning the scene element values to corresponding scene elements to generate corresponding test scenes.
In a specific embodiment, a scene library is constructed according to advanced driver assistance function regulatory standards and functional requirements. Element extraction and statistics are carried out based on real traffic scene data, an ACC (Adaptive Cruise Control) and AEB (automatic Braking system) scene library is constructed, and a total of 212 test scenes including 45 standard and regulation scenes, 86 natural driving scenes, 40 dangerous scenes and corner scenes, 41 combined scenes and the like are covered. An example scenario is as follows:
TABLE 1 test scenarios
Fig. 2 is a flowchart of a scenario-based model-in-loop test method provided in an embodiment of the present invention, which is suitable for performing an in-loop test on a model to be tested in different test scenarios. As shown in fig. 2, the method specifically includes the following steps:
s210, starting the test operation of a scene test case.
A scenario test case is used for executing a test operation in a test scenario. The test scenario includes a logical scenario and values of specified scenario elements therein.
Optionally, the logic scenarios in the test cases in different scenarios are different; or when the test cases in different scenes adopt the same logic scene, the value samples of the designated scene elements are not completely the same, so that the test cases in different scenes can test different logic scenes, or test different value samples, and realize multi-level test.
S220, loading a corresponding logic scene, a value sample of the appointed scene element and a model evaluation index applicable to the logic scene through the scene test case.
Optionally, loading a logic scene corresponding to the scene test case identifier from a scene library through the scene test case; loading the value sample of the appointed scene element from a data table through the scene test case; the value sample of the specified scene element corresponds to the scene test case identifier; and loading the model evaluation index corresponding to the scene test case identifier from a model evaluation index library through the scene test case.
The scene test case identification is used for uniquely identifying one scene test case. Specifically, the logic scene is stored in the scene library in advance, and the corresponding relationship between the logic scene and the scene test case identifier is established in the scene library. The data table stores value samples of the appointed scene elements in each logic scene, and the value samples correspond to the scene test case identifications. And storing the model evaluation indexes into a model evaluation index library in advance, and establishing a corresponding relation between the model evaluation indexes and the scene test case identifications in the model evaluation index library. Illustratively, according to the regulatory standard requirements and the ACC and AEB functional requirements, a functional test passing model evaluation index is formulated, and the following are exemplified:
TABLE 2 evaluation index of model
When the scenario test case is executed, a logic scenario corresponding to the self identification is loaded from a scenario library. Then, the value sample of the corresponding designated scene element is loaded from the data table. And loading corresponding model evaluation indexes from the model evaluation index library so as to combine the logic scene, the test specification, the indexes and the element values.
S230, assigning the value sample of the appointed scene element to the corresponding appointed scene element in the logic scene to generate a test scene; and controlling the model to be tested to test in the test scene, and reading test data.
Optionally, replacing the value distribution of the corresponding specified scene element in the logic scene with the value sample of the specified scene element through the scene test case to generate a test scene; the value distribution of the specified scene elements is obtained by counting the specified scene elements in the real traffic scene data, and the value sample of the specified scene elements is obtained by sampling the value distribution of the specified scene elements. For details, reference is made to the description of the above embodiments, which are not repeated herein.
It is worth noting that the alternative embodiment is applicable to the logical scenario generated by the method provided in fig. 1 a. However, the values of the designated scene elements in some logic scenarios (such as standard and regulatory scenarios, combination scenarios, and the like) may be fixed values or designated values, and these values may be stored in the data table in advance to prepare for generating the test scenario.
And then, controlling the model to be tested to test in the test scene through the scene test case. Optionally, the test of the model to be tested is performed in the scene building module, which will be described in the following embodiments. The model to be tested is an advanced assistant driving function model, such as an adaptive cruise control algorithm model, so that an in-loop automatic test method of the advanced assistant driving function model based on a scene is realized, the progress of system functions from concept to realization is accelerated, and the integrity and the accuracy of system function logic are ensured. The test types include but are not limited to various tests such as functional tests, calibration tests, bus communication tests, diagnostic tests, fault injection and the like, and automation thereof.
And S240, comparing the model evaluation index with the test data through the scene test case to obtain a test result and outputting the test result.
And storing and processing the test result through the scene test case, and quantitatively analyzing and comparing the model evaluation index and the test data through the script to obtain the test result. And checking and confirming the related key points through the configuration script, and automatically outputting the test result in a test report form.
And S250, starting the test operation of the test case of the next scene until the test of all the test cases of the scenes is finished.
Optionally, before starting the test operation of the scenario test case, the method further includes: creating a main project and a plurality of sub projects; the main project is used for controlling the starting of each sub project, and controlling the starting of the next sub project after monitoring that one sub project obtains a test result and outputs the test result; and each sub-project is used for executing the test operation of the test case of the corresponding scene. And realizing the sequential and automatic execution of the test cases of each scene by a mode of establishing a double-layer project.
Fig. 3a is a schematic structural diagram of a model-in-loop test system based on a scene according to an embodiment of the present invention, including: the system comprises a test module, a scene building module and a model to be tested.
The test module is used for executing the scene-based model-in-loop test method provided by each embodiment, and specifically, starting the test operation of a scene test case; loading a model evaluation index applicable to a corresponding logic scene and a value sample of an appointed scene element through the scene test case, and loading the logic scene from the scene building module; assigning the value sample of the appointed scene element to the corresponding appointed scene element in the logic scene to generate a test scene; controlling the model to be tested to test in the test scene, and reading test data; comparing the model evaluation index with the test data through the scene test case to obtain and output a test result; and starting the test operation of the next scene test case until the test of all the scene test cases is completed.
The model to be tested is used for testing in the test scene and transmitting test data to the test module. Illustratively, the adaptive cruise control algorithm model includes a safe distance model, a desired acceleration model, a braking model, and an acceleration model. The safe distance model formula is as follows:
in the formula v1Is the velocity of the main vehicle, v2To the front vehicle speed, amaxAt maximum acceleration, tdFor driver reaction time, d1The distance between the two vehicles after stopping. Because of the too large safety distance during braking, the parking distance can be set to be small, here 2 m.
From the safe distance model, the calculation formula for the expected acceleration can be derived as follows:
aexp＝k1(S-D)+k2vrel-k3a1；(2)
wherein S is the actual inter-vehicle distance of the two vehicles; d is a safe distance; a is1The actual acceleration of the rear vehicle; k is a radical of1，k2，k3Is a feedback coefficient; v. ofrelThe relative speed of the two vehicles.
When the vehicle brakes, the self-cruise control system can calculate a required acceleration signal according to the expected acceleration, the acceleration signal is input to a vehicle execution mechanism to generate a braking force signal, and the vehicle braking force signal is fed back to a tested vehicle model to perform deceleration braking.
When the vehicle accelerates, the self-cruise control system can calculate a required acceleration signal according to the expected acceleration, the acceleration signal is input to a vehicle execution mechanism to generate an accelerator pedal opening degree signal, and the accelerator pedal opening degree signal is fed back to a tested vehicle model to accelerate the vehicle.
The scene building module comprises a plurality of logic scenes. Optionally, the scene building module includes a vehicle dynamics model, a sensor model, and a scene library including a plurality of logical scenes; the plurality of logic scenes are obtained by assigning the value distribution of the appointed scene elements to the corresponding appointed scene elements in the functional scenes, and the value distribution of the appointed scene elements is obtained by counting the appointed scene elements in the real traffic scene data. The functional scene is obtained by combining scene elements.
Fig. 3b is an interaction diagram of the scene-based model-in-loop test system according to the embodiment of the present invention. The test module is used for controlling simulation in a test scene, and specifically sending the tested vehicle model, the initial state of the tested vehicle model and the test scene to the scene building module. The scene building module is used for receiving the initial state of the tested vehicle model and the test scene from the test module to build a traffic scene, and receiving control parameters and road surface information from the tested model; providing the control parameters to the vehicle dynamics model and the road surface information to the sensor model; and in the traffic scene, obtaining test data of the tested vehicle model based on the sensor model and the vehicle dynamics model, and transmitting the test data to the model to be tested.
The model to be tested is used for forwarding the test data to the test module and returning the control parameters based on the test data.
The test module is also used for recording the test data so as to compare the test data with the model evaluation index; and the method is also used for controlling the parameters of the model to be tested and the simulation progress.
In the following, the operation process of the scenario-based model in the ring test system is described in detail by using a specific example.
The TEST module can be an automatic integrated TEST ECU-TEST tool, and a corresponding relation between a logic scene, a value sample, a model evaluation index and a scene TEST case is established in the ECU-TEST tool in a dragging mode. The scene building module may be a vehicle dynamics simulation platform (e.g., CarSim). The model to be tested runs in a simulation platform environment Simulink. The ECU-TEST tool, the vehicle dynamics simulation platform and the simulation platform environment are integrated and interacted to form a model-in-loop TEST system based on a scene.
The preparation work is as follows: 1) a plurality of logic scenes are built through an ECU-TEST tool and stored in a scene library, value samples of appointed scene elements in the logic scenes are stored in a data table, and model evaluation indexes are stored in a model evaluation index library. And creating sub-projects corresponding to the main project and the test cases of each scene. 2) And establishing the 212 test scene models through a vehicle dynamics simulation platform to construct static, dynamic and weather elements required by each scene test. Fig. 4 shows a display effect diagram of a test scenario. 3) And carrying out vehicle dynamics modeling and constructing a sensor model through a vehicle dynamics simulation platform. 4) A model to be tested and an executing mechanism model adapted to the model to be tested are constructed through Simulink, and a detailed description is given to the adaptive cruise control algorithm model described in the above embodiment. 5) The method comprises the steps of integrating a model to be tested with a vehicle dynamic model (a vehicle longitudinal dynamic model), an actuating mechanism model (a braking and throttle control system and a main vehicle dynamic system) and a sensor model (a radar system), and specifically developing a scene interface of a Simulink algorithm model and a CarSim vehicle. And respectively building a vehicle longitudinal dynamic model of the main vehicle and the front vehicle in the CarSim. The radar system transmits detected rear vehicle information and front vehicle information to the following distance control system, the control system calculates expected acceleration of the main vehicle according to the safety distance model, the expected acceleration is input to the longitudinal vehicle dynamics model after being adjusted by the PID controller, the longitudinal vehicle dynamics model outputs expected braking pressure and throttle opening degree based on the acceleration, the expected braking pressure and throttle opening degree are transmitted to the braking and throttle control system, and the system outputs corresponding braking force and throttle opening degree to the main vehicle dynamics system to perform braking and acceleration control, so that self-adaptive cruise function of the main vehicle is realized.
And when the scene test is required to be carried out on the model to be tested, running the main project so as to call and execute each sub-project one by one through the main project. In the execution process of the sub-engineering, loading a logic scene corresponding to a scene test case identifier from a scene library, loading a value sample of the specified scene element from a data table, loading a model evaluation index corresponding to the scene test case identifier from a model evaluation index library, and generating a test scene.
Assuming that a test scene is a low-speed test of a target vehicle, the main vehicle approaches the low-speed target vehicle, the ACC function starts to respond, decelerates and follows the target vehicle to run, and a value sample of a designated scene element in a scene test case is shown in Table 3.
TABLE 3 specifying value samples for scene elements
The test specification of the test scenario includes: the initial speed of the main vehicle and the speed of the target vehicle are respectively changed, and applicable model evaluation indexes are as follows:
TABLE 4 evaluation index of model
Then, CarSim is called to realize vehicle motion calculation, the CarSim completes scene rendering, virtual sensor simulation, simulation calculation and combined simulation with a model to be tested in Simulink, and the ECU-TEST is responsible for capturing and recording TEST data in the simulation process, comparing and analyzing the TEST data, and displaying the obtained TEST result in a chart form. The embodiment performs normative regulation on the simulation test process, including analyzing test requirements, customizing a test scheme, writing test cases, modifying a simulation test environment, and the like. And according to the model evaluation index, automatic evaluation of a test result, automatic generation/output of a test report and the like are realized through the construction of an automatic tool such as script compiling and the like. The test data will cover critical data records such as vehicle speed, distance changes, acceleration and deceleration signals, etc. Table 3 shows part of the test data.
TABLE 5 test data
And the ECU-TEST automatically completes the calculation of 12 groups of TEST cases, and the TEST result shows that the TEST cases fail, wherein the 9 th TEST case fails.
As shown in FIG. 5, the test result is checked and analyzed, and it can be seen that the vehicle speed of the host vehicle appears in an acceleration interval for a period of time after steady-state following, the vehicle speed of the host vehicle is required to be equal to the target vehicle speed (24m/s) during steady-state following, and when the vehicle speed of the host vehicle is inconsistent with the target vehicle speed, the test case fails, and the direction is indicated for algorithm model optimization.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present application and the technical principles employed. It will be understood by those skilled in the art that the present application is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the application. Therefore, although the present application has been described in more detail with reference to the above embodiments, the present application is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present application, and the scope of the present application is determined by the scope of the appended claims.
Claims (10)
1. A method for generating a scene, comprising:
obtaining a plurality of functional scenes by combining scene elements, and determining specified scene elements in the functional scenes; the scene elements comprise static elements, dynamic elements and weather elements;
extracting appointed scene elements from real traffic scene data, and counting the value distribution of the appointed scene elements;
assigning the value distribution to corresponding appointed scene elements in a functional scene to obtain a plurality of logic scenes;
sampling the value distribution of the specified scene elements in each logic scene to generate a plurality of value samples;
and replacing the corresponding value distribution with the plurality of value samples to generate a plurality of test scenes.
2. A model in-loop test method based on scenes is characterized by comprising the following steps:
starting the test operation of a scene test case;
loading a corresponding logic scene, a value sample of an appointed scene element and a model evaluation index applicable to the logic scene through the scene test case;
assigning the value sample of the appointed scene element to the corresponding appointed scene element in the logic scene to generate a test scene; controlling the model to be tested to test in the test scene, and reading test data;
comparing the model evaluation index with the test data through the scene test case to obtain and output a test result;
and starting the test operation of the next scene test case until the test of all the scene test cases is completed.
3. The method of claim 2, wherein assigning the value sample of the specified scene element to a corresponding specified scene element in the logical scene to generate a test scene comprises:
replacing the value distribution of the corresponding appointed scene element in the logic scene with the value sample of the appointed scene element through the scene test case to generate a test scene;
the value distribution of the specified scene elements is obtained by counting the specified scene elements in the real traffic scene data, and the value sample of the specified scene elements is obtained by sampling the value distribution of the specified scene elements.
4. The method according to claim 2, wherein the loading of the corresponding logic scenario, the specifying of the value sample of the scenario element, and the model evaluation index applicable to the logic scenario by the scenario test case includes:
loading a logic scene corresponding to the scene test case identification from a scene library through the scene test case;
loading the value sample of the appointed scene element from a data table through the scene test case; the value sample of the specified scene element corresponds to the scene test case identifier;
and loading the model evaluation index corresponding to the scene test case identifier from a model evaluation index library through the scene test case.
5. The method of claim 2, wherein the logical scenarios in different scenario test cases are different; alternatively, the first and second electrodes may be,
when the test cases in different scenes adopt the same logic scene, the value samples of the appointed scene elements are not completely the same.
6. The method of claim 2, wherein before the initiating the test operation of the scenario test case, further comprising:
creating a main project and a plurality of sub projects;
the main project is used for controlling the starting of each sub-project, and controlling the starting of the next sub-project after monitoring that one sub-project obtains a test result and outputs the test result;
and each sub-project is used for executing the test operation of the test case of the corresponding scene.
7. The method of claim 2, wherein the logical scenario comprises at least one of a standard regulatory scenario, a natural driving scenario, a dangerous scenario, a corner scenario, and a combination scenario.
8. A model-in-the-loop test system based on a scene, comprising: the system comprises a test module, a scene building module and a model to be tested;
the test module is used for starting the test operation of a scene test case; loading a model evaluation index applicable to a corresponding logic scene and a value sample of an appointed scene element through the scene test case, and loading the logic scene from the scene building module; assigning the value sample of the appointed scene element to the corresponding appointed scene element in the logic scene to generate a test scene; controlling the model to be tested to test in the test scene, and reading test data; comparing the model evaluation index with the test data through the scene test case to obtain and output a test result; starting the test operation of the next scene test case until the test of all the scene test cases is completed;
the model to be tested is used for testing in the test scene and transmitting test data to the test module;
the scene building module comprises a plurality of logic scenes.
9. The system of claim 8, wherein the scene building module comprises a vehicle dynamics model, a sensor model, and a scene library comprising a plurality of logical scenes;
the plurality of logic scenes are obtained by assigning the value distribution of the appointed scene elements to the corresponding appointed scene elements in the functional scenes, and the value distribution of the appointed scene elements is obtained by counting the appointed scene elements in the real traffic scene data; the functional scene is obtained by combining scene elements.
10. The system of claim 9,
the scene building module is used for receiving the initial state of the tested vehicle model and the test scene from the test module to build a traffic scene, and receiving control parameters and road surface information from the tested model;
providing the control parameters to the vehicle dynamics model and the road surface information to the sensor model; in the traffic scene, obtaining test data of the tested vehicle model based on the sensor model and the vehicle dynamics model, and transmitting the test data to the model to be tested;
the model to be tested is used for forwarding the test data to the test module and returning the control parameters based on the test data;
and the model to be tested is an advanced auxiliary driving function model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011448739.0A CN112685289A (en) | 2020-12-11 | 2020-12-11 | Scene generation method, and scene-based model in-loop test method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011448739.0A CN112685289A (en) | 2020-12-11 | 2020-12-11 | Scene generation method, and scene-based model in-loop test method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112685289A true CN112685289A (en) | 2021-04-20 |
Family
ID=75448457
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011448739.0A Pending CN112685289A (en) | 2020-12-11 | 2020-12-11 | Scene generation method, and scene-based model in-loop test method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112685289A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113268244A (en) * | 2021-05-13 | 2021-08-17 | 际络科技(上海)有限公司 | Script generation method and device of automatic driving scene library and electronic equipment |
| CN113408141A (en) * | 2021-07-02 | 2021-09-17 | 阿波罗智联(北京)科技有限公司 | Automatic driving test method and device and electronic equipment |
| CN113419942A (en) * | 2021-04-30 | 2021-09-21 | 吉林大学 | Automatic driving safety evaluation method based on natural driving data |
| CN113918475A (en) * | 2021-12-15 | 2022-01-11 | 腾讯科技（深圳）有限公司 | Test processing method and device, computer equipment and storage medium |
| CN114354219A (en) * | 2022-01-07 | 2022-04-15 | 苏州挚途科技有限公司 | Test method and device for automatic driving vehicle |
| CN114356757A (en) * | 2021-12-22 | 2022-04-15 | 重庆长安汽车股份有限公司 | Test condition configuration method based on limited automatic driving simulation scene |
| CN114371015A (en) * | 2022-01-04 | 2022-04-19 | 一汽解放汽车有限公司 | Automatic driving test method, device, computer equipment and storage medium |
| CN116358902A (en) * | 2023-06-02 | 2023-06-30 | 中国第一汽车股份有限公司 | Vehicle function testing method and device, electronic equipment and storage medium |
| CN116719748A (en) * | 2023-08-10 | 2023-09-08 | 中国船级社 | Scene generation method, device and medium of ship system |
| CN114371015B (en) * | 2022-01-04 | 2024-06-04 | 一汽解放汽车有限公司 | Automatic driving test method, automatic driving test device, computer equipment and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102017009971A1 (en) * | 2017-10-26 | 2019-05-02 | Daimler Ag | A method of testing a lane keeping assistance system for a vehicle |
| CN110263381A (en) * | 2019-05-27 | 2019-09-20 | 南京航空航天大学 | A kind of automatic driving vehicle test emulation scene generating method |
| CN110794810A (en) * | 2019-11-06 | 2020-02-14 | 安徽瑞泰智能装备有限公司 | Method for carrying out integrated test on intelligent driving vehicle |
| CN111143197A (en) * | 2019-12-05 | 2020-05-12 | 苏州智加科技有限公司 | Automatic driving test case generation method, device, equipment and storage medium |
| CN111178402A (en) * | 2019-12-13 | 2020-05-19 | 赛迪检测认证中心有限公司 | Scene classification method and device for road test of automatic driving vehicle |
| CN111797001A (en) * | 2020-05-27 | 2020-10-20 | 中汽数据有限公司 | Method for constructing automatic driving simulation test model based on SCANeR |
| CN111797003A (en) * | 2020-05-27 | 2020-10-20 | 中汽数据有限公司 | Method for building virtual test scene based on VTD software |
| CN112036001A (en) * | 2020-07-01 | 2020-12-04 | 长安大学 | Automatic driving test scene construction method, device and equipment and readable storage medium |
-
2020
- 2020-12-11 CN CN202011448739.0A patent/CN112685289A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102017009971A1 (en) * | 2017-10-26 | 2019-05-02 | Daimler Ag | A method of testing a lane keeping assistance system for a vehicle |
| CN110263381A (en) * | 2019-05-27 | 2019-09-20 | 南京航空航天大学 | A kind of automatic driving vehicle test emulation scene generating method |
| CN110794810A (en) * | 2019-11-06 | 2020-02-14 | 安徽瑞泰智能装备有限公司 | Method for carrying out integrated test on intelligent driving vehicle |
| CN111143197A (en) * | 2019-12-05 | 2020-05-12 | 苏州智加科技有限公司 | Automatic driving test case generation method, device, equipment and storage medium |
| CN111178402A (en) * | 2019-12-13 | 2020-05-19 | 赛迪检测认证中心有限公司 | Scene classification method and device for road test of automatic driving vehicle |
| CN111797001A (en) * | 2020-05-27 | 2020-10-20 | 中汽数据有限公司 | Method for constructing automatic driving simulation test model based on SCANeR |
| CN111797003A (en) * | 2020-05-27 | 2020-10-20 | 中汽数据有限公司 | Method for building virtual test scene based on VTD software |
| CN112036001A (en) * | 2020-07-01 | 2020-12-04 | 长安大学 | Automatic driving test scene construction method, device and equipment and readable storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 张剑锋;马玲;李广召;: "自动驾驶运动控制算法的模型在环测试", 汽车工程师, no. 01 * |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113419942A (en) * | 2021-04-30 | 2021-09-21 | 吉林大学 | Automatic driving safety evaluation method based on natural driving data |
| CN113268244A (en) * | 2021-05-13 | 2021-08-17 | 际络科技(上海)有限公司 | Script generation method and device of automatic driving scene library and electronic equipment |
| CN113408141A (en) * | 2021-07-02 | 2021-09-17 | 阿波罗智联(北京)科技有限公司 | Automatic driving test method and device and electronic equipment |
| CN113408141B (en) * | 2021-07-02 | 2024-04-26 | 阿波罗智联(北京)科技有限公司 | Automatic driving test method and device and electronic equipment |
| CN113918475A (en) * | 2021-12-15 | 2022-01-11 | 腾讯科技（深圳）有限公司 | Test processing method and device, computer equipment and storage medium |
| CN114356757A (en) * | 2021-12-22 | 2022-04-15 | 重庆长安汽车股份有限公司 | Test condition configuration method based on limited automatic driving simulation scene |
| CN114371015A (en) * | 2022-01-04 | 2022-04-19 | 一汽解放汽车有限公司 | Automatic driving test method, device, computer equipment and storage medium |
| CN114371015B (en) * | 2022-01-04 | 2024-06-04 | 一汽解放汽车有限公司 | Automatic driving test method, automatic driving test device, computer equipment and storage medium |
| CN114354219A (en) * | 2022-01-07 | 2022-04-15 | 苏州挚途科技有限公司 | Test method and device for automatic driving vehicle |
| CN116358902A (en) * | 2023-06-02 | 2023-06-30 | 中国第一汽车股份有限公司 | Vehicle function testing method and device, electronic equipment and storage medium |
| CN116358902B (en) * | 2023-06-02 | 2023-08-22 | 中国第一汽车股份有限公司 | Vehicle function testing method and device, electronic equipment and storage medium |
| CN116719748A (en) * | 2023-08-10 | 2023-09-08 | 中国船级社 | Scene generation method, device and medium of ship system |
| CN116719748B (en) * | 2023-08-10 | 2024-02-20 | 中国船级社 | Scene generation method, device and medium of ship system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112685289A (en) | Scene generation method, and scene-based model in-loop test method and system | |
| CN111795832B (en) | Intelligent driving vehicle testing method, device and equipment | |
| CN117872795A (en) | Electronic simulation method and system | |
| CN110796007B (en) | Scene recognition method and computing device | |
| CN114879631A (en) | Automatic driving test system and method based on digital twin cloud control platform | |
| CN113343461A (en) | Simulation method and device for automatic driving vehicle, electronic equipment and storage medium | |
| CN113484851B (en) | Simulation test system and method for vehicle-mounted laser radar and complete vehicle in-loop test system | |
| CN109726426A (en) | A kind of Vehicular automatic driving virtual environment building method | |
| CN112311773B (en) | Implementation method for intelligent automobile sensor interface system | |
| CN112035951A (en) | Simulation platform and simulation method for automatic driving algorithm verification | |
| CN113777952A (en) | Automatic driving simulation test method for interactive mapping of real vehicle and virtual vehicle | |
| CN115099051A (en) | Automatic driving simulation test scene generation method and device, vehicle and storage medium | |
| CN116580271A (en) | Evaluation method, device, equipment and storage medium for perception fusion algorithm | |
| CN115392015A (en) | Automatic driving deduction system and deduction method based on digital twins | |
| CN114444208A (en) | Method, device, equipment and medium for determining reliability of automatic driving system | |
| CN111983934B (en) | Unmanned vehicle simulation test case generation method and system | |
| US20220406041A1 (en) | Recognition model distribution system and updating method of recognition model | |
| CN112525552A (en) | Algorithm test system and method for automobile advanced auxiliary driving system | |
| CN110134024A (en) | The construction method of distinctive mark object in Vehicular automatic driving virtual environment | |
| CN117413257A (en) | Method and system for testing driver assistance system for vehicle | |
| CN113962107A (en) | Method and device for simulating driving road section, electronic equipment and storage medium | |
| CN114326443B (en) | MIL simulation test method and system for ADAS and readable storage medium | |
| Shadrin et al. | Methods of Parameter Verification and Scenario Generation During Virtual Testing of Highly Automated and Autonomous Vehicles | |
| EP4191462A1 (en) | Method and system for quantifying vehicle requirement | |
| Cheng et al. | An Indoor Rapid Testing Platform for Autonomous Vehicles Using Vehicle-in-the-Loop Simulation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |