CN117787005A - Method and device for determining credibility of simulation scene and electronic equipment - Google Patents
Method and device for determining credibility of simulation scene and electronic equipment Download PDFInfo
- Publication number
- CN117787005A CN117787005A CN202311862605.7A CN202311862605A CN117787005A CN 117787005 A CN117787005 A CN 117787005A CN 202311862605 A CN202311862605 A CN 202311862605A CN 117787005 A CN117787005 A CN 117787005A
- Authority
- CN
- China
- Prior art keywords
- simulation
- real
- determining
- test data
- scene
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004088 simulation Methods 0.000 title claims abstract description 214
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000012360 testing method Methods 0.000 claims abstract description 200
- 238000009826 distribution Methods 0.000 claims abstract description 178
- 238000004364 calculation method Methods 0.000 claims abstract description 31
- 238000004590 computer program Methods 0.000 claims description 4
- 238000004422 calculation algorithm Methods 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 1
- 238000013100 final test Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application provides a method and a device for determining reliability of a simulation scene and electronic equipment, wherein the method comprises the following steps: acquiring real test data corresponding to a preset attention index obtained by repeatedly executing a plurality of tests in a preset real scene; constructing a simulation scene corresponding to a preset real scene; repeatedly executing multiple simulation tests in a simulation scene to obtain simulation test data corresponding to a preset attention index; carrying out probability distribution calculation on real test data and simulation test data corresponding to preset attention indexes, and determining probability distribution results, wherein the probability distribution results indicate the consistency between the real test data and the simulation test data; and determining the credibility corresponding to the simulation scene according to the probability distribution result. The consistency between the real data and the simulation data in the same automatic driving test scene can be determined through the KL divergence, so that the credibility of the simulation scene is obtained, and the accuracy of the test result of the automatic driving in the simulation scene is improved.
Description
Technical Field
The application relates to the technical field of automatic driving automobile testing, in particular to a method and a device for determining reliability of a simulation scene and electronic equipment.
Background
The automatic driving simulation test can realize the test mileage which is difficult to reach in reality in a short time, can quickly simulate any scene, helps developers traverse various scenes, greatly reduces the test mileage and reduces the test cost. However, the simulation result is adopted to replace the data acquired during real driving, the consistency of the simulation result and the real data needs to be ensured, namely, the simulation scene needs to have credibility, and the automatic driving test is directly performed on the basis of the established simulation scene at present, so that if the credibility of the simulation scene cannot be ensured, the test result of the automatic driving test is greatly influenced finally, and the accuracy of the automatic driving test is reduced.
Disclosure of Invention
Accordingly, the present application aims to provide at least a method, a device and an electronic device for determining reliability of a simulation scene, which can determine consistency between real data and simulation data in the same automatic driving test scene through KL divergence, thereby obtaining the reliability of the simulation scene and improving accuracy of test results of automatic driving in the simulation scene.
The application mainly comprises the following aspects:
in a first aspect, an embodiment of the present application provides a method for determining reliability of a simulation scene, where the method includes: acquiring real test data corresponding to a preset attention index obtained by repeatedly executing a plurality of tests in a preset real scene; constructing a simulation scene corresponding to a preset real scene; repeatedly executing multiple simulation tests in a simulation scene to obtain simulation test data corresponding to a preset attention index; carrying out probability distribution calculation on real test data and simulation test data corresponding to preset attention indexes, and determining probability distribution results, wherein the probability distribution results indicate the consistency between the real test data and the simulation test data; and determining the credibility corresponding to the simulation scene according to the probability distribution result.
In a possible embodiment, the real test data includes a real value of the preset attention index under each site test, and the simulation test data includes a simulation value of the preset attention index under each simulation test, wherein the probability distribution result is determined by: determining a data distribution interval between real test data and simulation test data; equally dividing the data distribution interval to obtain a plurality of sub data distribution intervals; determining the real data distribution probability of the preset attention index in each sub-data distribution interval according to a plurality of real values corresponding to the preset attention index; determining simulation data distribution probability of the preset attention index in each sub-data distribution interval according to a plurality of simulation values corresponding to the preset attention index; and carrying out KL divergence calculation on the real data distribution probability and the simulation data distribution probability corresponding to each sub data distribution interval, and determining a probability distribution result according to the calculation result.
In one possible implementation, the data distribution interval is determined by: determining a minimum value between a plurality of real values and a plurality of simulation values corresponding to a preset attention index, and determining the minimum value as a section lower limit value corresponding to a data distribution section; determining a maximum value between a plurality of real values and a plurality of simulation values corresponding to a preset attention index, and determining the maximum value as a section upper limit value corresponding to a data distribution section; and forming a data distribution section by the section lower limit value and the section upper limit value.
In one possible implementation, the probability distribution result includes a KL divergence between the real test data and the simulated test data, wherein the KL divergence between the real test data and the simulated test data is determined by the following formula:
in the formula, D KL (x|y) represents the KL divergence between the real test data X and the simulation test data Y, M represents the number of sub-data distribution intervals, p (x=i) represents the real data distribution probability of the preset attention index in the i-th sub-data distribution interval, and p (y=i) represents the simulation data distribution probability of the preset attention index in the i-th sub-data distribution interval.
In one possible implementation manner, the step of determining the credibility of the simulation scene according to the probability distribution result includes: comparing the KL divergence between the real test data and the simulation test data with a preset threshold value; determining that the KL divergence is smaller than or equal to a preset threshold value, and determining that the simulation scene is credible; and determining that the KL divergence is larger than a preset threshold value, and determining that the simulation scene is not credible.
In one possible embodiment, the method further comprises: if the simulation scene is credible, verifying an automatic driving algorithm by using the simulation scene to replace a real scene; if the simulation scene is not trusted, returning to reconstruct the simulation scene corresponding to the preset test scene.
In a second aspect, an embodiment of the present application further provides a device for determining reliability of a simulation scene, where the device includes: the acquisition module is used for acquiring real test data corresponding to a preset attention index obtained by repeatedly executing the test for a plurality of times under a preset real scene; the building module is used for building a simulation scene corresponding to the preset real scene; the simulation module is used for repeatedly executing the simulation test for a plurality of times under the simulation scene to obtain simulation test data corresponding to the preset attention index; the probability distribution calculation module is used for carrying out probability distribution calculation on the real test data and the simulation test data corresponding to the preset attention index, determining a probability distribution result, wherein the probability distribution result indicates the consistency between the real test data and the simulation test data; and the credibility determining module is used for determining the credibility corresponding to the simulation scene according to the probability distribution result.
In one possible implementation manner, the real test data includes a real value of the preset attention index under each site test, and the simulation test data includes a simulation value of the preset attention index under each simulation test, where the probability distribution calculation module is further configured to: determining a data distribution interval between real test data and simulation test data; equally dividing the data distribution interval to obtain a plurality of sub data distribution intervals; determining the real data distribution probability of the preset attention index in each sub-data distribution interval according to a plurality of real values corresponding to the preset attention index; determining simulation data distribution probability of the preset attention index in each sub-data distribution interval according to a plurality of simulation values corresponding to the preset attention index; and carrying out KL divergence calculation on the real data distribution probability and the simulation data distribution probability corresponding to each sub data distribution interval, and determining a probability distribution result according to the calculation result.
In a third aspect, embodiments of the present application further provide an electronic device, including: the system comprises a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, when the electronic device is running, the processor and the memory are communicated through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the simulation scene reliability determination method in the first aspect or any possible implementation manner of the first aspect.
In a fourth aspect, the embodiments of the present application further provide a computer readable storage medium, on which a computer program is stored, which when executed by a processor performs the steps of determining the reliability of the simulation scenario in the first aspect or any one of the possible implementation manners of the first aspect.
The embodiment of the application provides a method and a device for determining the credibility of a simulation scene and electronic equipment, wherein the method comprises the following steps: acquiring real test data corresponding to a preset attention index obtained by repeatedly executing a plurality of tests in a real scene corresponding to a preset test scene; constructing a simulation scene corresponding to a preset test scene; repeatedly executing multiple simulation tests in a simulation scene to obtain simulation test data corresponding to a preset attention index; carrying out probability distribution calculation on real test data and simulation test data corresponding to preset attention indexes, and determining probability distribution results, wherein the probability distribution results indicate the consistency between the real test data and the simulation test data; and determining the credibility corresponding to the simulation scene according to the probability distribution result. The consistency between the real data and the simulation data in the same automatic driving test scene can be determined through the KL divergence, so that the credibility of the simulation scene is obtained, and the accuracy of the test result of the automatic driving in the simulation scene is improved.
In order to make the above objects, features and advantages of the present application more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered limiting the scope, and that other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart illustrating a method for determining reliability of a simulation scenario provided in an embodiment of the present application;
FIG. 2 is a functional block diagram of a determining device for a trusted pair of a simulation scenario provided in an embodiment of the present application;
fig. 3 shows a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it should be understood that the accompanying drawings in the present application are only for the purpose of illustration and description, and are not intended to limit the protection scope of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. A flowchart, as used in this application, illustrates operations implemented according to some embodiments of the present application. It should be appreciated that the operations of the flow diagrams may be implemented out of order and that steps without logical context may be performed in reverse order or concurrently. Moreover, one or more other operations may be added to the flow diagrams and one or more operations may be removed from the flow diagrams as directed by those skilled in the art.
In addition, the described embodiments are only some, but not all, of the embodiments of the present application. The components of the embodiments of the present application, which are generally described and illustrated in the figures herein, may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, as provided in the accompanying drawings, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, are intended to be within the scope of the present application.
The automatic driving simulation test can realize the test mileage which is difficult to reach in reality in a short time, can quickly simulate any scene, helps developers traverse various scenes, greatly reduces the test mileage, reduces the test cost, but adopts simulation to replace real test, needs to ensure the credibility of the simulation environment, the credibility of the simulation environment is reflected in the consistency of the simulation result and real data, if the automatic driving test is carried out by using an unreliable simulation environment, the accuracy of the automatic driving test result is reduced, the final test result is also unreliable, and the evaluation method is lacking at present how to evaluate the consistency of the simulation performance and the real driving performance of the automatic driving.
Based on this, the embodiment of the application provides a method, a device and an electronic device for determining the reliability of a simulation scene, which can determine the consistency between real data and simulation data in the same automatic driving test scene through KL divergence, thereby obtaining the reliability of the simulation scene and improving the accuracy of the test result of automatic driving in the simulation scene, and the method comprises the following specific steps:
referring to fig. 1, fig. 1 is a flowchart illustrating a method for determining reliability of a simulation scenario according to an embodiment of the present application. As shown in fig. 1, the method provided in the embodiment of the present application includes the following steps:
s100, acquiring real test data corresponding to a preset attention index obtained by repeatedly executing the test for a plurality of times under a preset real scene.
S200, constructing a simulation scene corresponding to the preset real scene.
And S300, repeatedly executing the simulation test for a plurality of times in the simulation scene to obtain simulation test data corresponding to the preset attention index.
S400, probability distribution calculation is carried out on the real test data and the simulation test data corresponding to the preset attention index, and a probability distribution result is determined.
The probability distribution results indicate the consistency between the real test data and the simulated test data.
S500, determining the credibility corresponding to the simulation scene according to the probability distribution result.
Before step S100, the user may determine a preset real scene according to the test requirement, set an attention index, where the attention index is a vehicle speed, a vehicle distance, and the like, and then repeatedly perform the site test multiple times under the preset real scene, to determine real test data corresponding to the preset attention index.
In step S100 to step S500, after obtaining the real test data corresponding to the preset attention index, creating a simulation scene corresponding to the preset real scene, and repeatedly executing the simulation test on the simulation scene for a plurality of times to obtain simulation test data corresponding to the preset attention index, wherein the number of times of executing the simulation test is the same as that of executing the field test, then performing probability distribution calculation on the real test data and the simulation test data corresponding to the preset attention index, determining a probability distribution result, performing probability distribution calculation, namely verifying whether the real test data and the simulation test data obey the same probability distribution by using a KL divergence method, if the real test data and the simulation test data obey the same probability distribution, determining that the real test data and the simulation test data are consistent, and further determining the reliability corresponding to the simulation scene according to the probability distribution result.
In the application, the consistency comparison result between the real test data and the simulation test data can be determined through probability distribution calculation based on the KL divergence method, so that the reliability of the simulation scene is further determined according to the consistency comparison result.
The real test data includes the real value of the preset attention index under each site test, and the simulation test data includes the simulation value of the preset attention index under each simulation test, for example, if the site test is performed n times under the preset real scene, the preset attention is obtainedTrue test data x= (X) corresponding to index 1 ,x 2 ,x 3 ,…,x n )，x n Representing the real value of the preset attention index obtained by carrying out the nth field test under the preset real scene, and obtaining simulation test data Y= (Y) corresponding to the preset attention index by carrying out the field test for n times under the simulation scene 1 ,y 2 ,y 3 ,…,y n )，y n And representing the simulation value of the preset attention index obtained by carrying out the nth field test under the preset real scene.
In a preferred embodiment, the probability distribution result is determined by:
determining a data distribution interval between real test data and simulation test data, performing equal division processing on the data distribution interval to obtain a plurality of sub-data distribution intervals, determining real data distribution probability of a preset attention index in each sub-data distribution interval according to a plurality of real values corresponding to the preset attention index, determining simulation data distribution probability of the preset attention index in each sub-data distribution interval according to a plurality of simulation values corresponding to the preset attention index, performing KL divergence calculation on the real data distribution probability and the simulation data distribution probability corresponding to each sub-data distribution interval, and determining a probability distribution result according to a calculation result.
In specific implementation, in the application, based on the divided sub-data distribution intervals, the probability (real data distribution probability and simulation data distribution probability) that the real test data X and the simulation test data Y fall in the sub-data distribution intervals is calculated respectively, then, based on the calculated real data distribution probability and simulation data distribution probability corresponding to each sub-data distribution interval, the divergence calculation is performed, and further, the probability distribution result is truly determined, and since the KL divergence can be used for measuring the difference degree between the two distributions, the consistency between the real test data and the simulation test data can be further determined by adopting the divergence calculation.
In a preferred embodiment, the data distribution interval is determined by:
the method comprises the steps of determining a minimum value between a plurality of real values and a plurality of simulation values corresponding to a preset attention index, determining a section lower limit value corresponding to a data distribution section, determining a maximum value between the plurality of real values and the plurality of simulation values corresponding to the preset attention index, determining the maximum value as a section upper limit value corresponding to the data distribution section, and forming the data distribution section by the section lower limit value and the section upper limit value.
In an example, for the real test data X and the simulation test data Y, the data distribution interval a= [ min (X, Y), max (X, Y) ], where min (X, Y) is a lower limit value corresponding to the data distribution interval, and max (X, Y) is an upper limit value corresponding to the data distribution interval, and is a maximum value corresponding to the real test data X and the simulation test data Y.
After determining the data distribution section A, the data distribution section A is equally divided, e.g., into M sub-data distribution sections { a } 1 ,a 2 ,…a M Then, determining the real data distribution probability and the simulation data distribution probability of the preset attention index in each sub-data distribution interval according to the multiple real values and the multiple simulation values corresponding to the preset attention index, for example, for the real test data X, the real test data X falls in the sub-data distribution interval a 1 K true values in the data distribution interval a, the true test data X is in the sub-data distribution interval a 1 The lower real data distribution probability p (x=1) =k/n, n is the number of real values in the real test data X, that is, the number of tests, and the real data distribution probability falling in other intervals is the same, which is not described herein in detail, and the simulation data distribution probability under each sub-data distribution interval of the simulation test data Y is obtained in the same manner, which is not described herein in detail.
The probability distribution result includes the KL divergence between the real test data and the simulated test data.
In a preferred embodiment, the KL divergence between the real test data and the simulated test data is determined by the following formula:
in the formula, D KL (x|y) represents the KL divergence between the real test data X and the simulation test data Y, M represents the number of sub-data distribution intervals, p (x=i) represents the real data distribution probability of the preset attention index in the i-th sub-data distribution interval, and p (y=i) represents the simulation data distribution probability of the preset attention index in the i-th sub-data distribution interval.
In a statistical sense, the KL divergence can be used to measure the difference degree between two distributions, the smaller the KL divergence is, the smaller the difference between the two distributions is, and vice versa, in theory, when the two distributions are consistent, the KL divergence is 0, that is, the KL divergence reflects the consistency between the two distributions, so in the application, after the KL divergence between the data obtained by the preset attention index under the real scene and the simulation scene is determined through the above manner, the reliability corresponding to the simulation scene can be determined further based on the consistency of the response of the preset attention index.
In a preferred embodiment, step S500 includes: comparing the KL divergence between the real test data and the simulation test data with a preset threshold value, determining that the KL divergence is smaller than or equal to the preset threshold value, determining that the simulation scene is credible, and determining that the KL divergence is larger than the preset threshold value, determining that the simulation scene is not credible.
In one example, in the present application, if the KL divergence is 0, which indicates that the distribution between the real test data and the simulation test data is completely consistent, but this is an ideal situation, the present application determines a preset threshold c before the KL divergence is actually verified, when D KL If the (X|Y) is less than or equal to c, determining that the distribution between the real test data and the simulation test data is consistent, determining that the simulation scene is credible for testing the preset attention index, and if D is the following point KL (X|Y)>And c, determining that the distribution between the real test data and the simulation test data is inconsistent, and determining that the simulation scene is not credible for testing the preset attention index.
In an embodiment of the present application, the method further includes:
if the simulation scene is credible, the simulation scene is used for replacing the real scene to verify the automatic driving algorithm, and if the simulation scene is not credible, the simulation scene corresponding to the preset test scene is reconstructed.
In the method, under the condition that the credibility of the simulation scene is determined, the simulation scene is used for replacing the real scene to test the automatic driving, so that the accuracy of the subsequent automatic driving test can be further improved.
Based on the same application conception, the embodiment of the application also provides a device for determining the trusted pair of the simulation scene, which corresponds to the method for determining the trusted pair of the simulation scene provided by the embodiment, and because the principle of solving the problem by the device in the embodiment of the application is similar to that of the method for determining the trusted pair of the simulation scene in the embodiment of the application, the implementation of the device can refer to the implementation of the method, and the repetition is omitted.
Referring to fig. 2, fig. 2 is a functional block diagram of a determining device for determining a trusted pair of a simulation scenario according to an embodiment of the present application. As shown in fig. 2, the apparatus includes:
the obtaining module 600 is configured to obtain real test data corresponding to a preset attention index obtained by repeatedly performing multiple tests in a preset real scene.
The construction module 610 is configured to construct a simulation scene corresponding to the preset real scene.
The simulation module 620 is configured to repeatedly perform a plurality of simulation tests in a simulation scenario, so as to obtain simulation test data corresponding to a preset attention index.
The probability distribution calculation module 630 performs probability distribution calculation on the real test data and the simulation test data corresponding to the preset attention index, and determines a probability distribution result, where the probability distribution result indicates consistency between the real test data and the simulation test data.
The reliability determining module 640 is configured to determine, according to the probability distribution result, the reliability corresponding to the simulation scene.
Preferably, the real test data includes a real value of the preset attention index under each site test, and the simulation test data includes a simulation value of the preset attention index under each simulation test.
Wherein the probability distribution calculation module is further configured to: determining a data distribution interval between real test data and simulation test data, performing equal division processing on the data distribution interval to obtain a plurality of sub-data distribution intervals, determining real data distribution probability of a preset attention index in each sub-data distribution interval according to a plurality of real values corresponding to the preset attention index, determining simulation data distribution probability of the preset attention index in each sub-data distribution interval according to a plurality of simulation values corresponding to the preset attention index, performing KL divergence calculation on the real data distribution probability and the simulation data distribution probability corresponding to each sub-data distribution interval, and determining a probability distribution result according to a calculation result.
Based on the same application concept, please refer to fig. 3, fig. 3 shows a schematic structural diagram of an electronic device according to an embodiment of the present application. The electronic device 700 includes: processor 710, memory 720 and bus 730, memory 720 storing machine-readable instructions executable by processor 710, which when executed by processor 710 performs the steps of a method for determining the trustworthiness of a simulation scenario as provided in any of the above embodiments, when electronic device 700 is in operation, processor 710 and memory 720 are in communication via bus 730.
Based on the same application conception, the embodiment of the application also provides a computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, and the computer program is executed by a processor to execute the steps of the method for determining the reliability of the simulation scene provided by the embodiment.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described system and apparatus may refer to corresponding procedures in the foregoing method embodiments, which are not described herein again. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. The above-described apparatus embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and there may be other manners of division in actual implementation, and for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some communication interface, device or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer readable storage medium executable by a processor. Based on such understanding, the technical solutions of the present application may be embodied in essence or a part contributing to the prior art or a part of the technical solutions, or in the form of a software product, which is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The foregoing is merely a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes or substitutions are covered in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A method for determining reliability of a simulation scene, the method comprising:
acquiring real test data corresponding to a preset attention index obtained by repeatedly executing a plurality of tests in a preset real scene;
constructing a simulation scene corresponding to the preset real scene;
repeatedly executing simulation tests for a plurality of times under the simulation scene to obtain simulation test data corresponding to the preset attention index;
carrying out probability distribution calculation on the real test data and the simulation test data corresponding to the preset attention index, and determining a probability distribution result, wherein the probability distribution result indicates the consistency between the real test data and the simulation test data;
and determining the credibility corresponding to the simulation scene according to the probability distribution result.
2. The method of claim 1, wherein the real test data comprises a real value of the predetermined attention index under each site test, the simulated test data comprises a simulated value of the predetermined attention index under each simulated test,
wherein the probability distribution result is determined by:
determining a data distribution interval between the real test data and the simulation test data;
equally dividing the data distribution interval to obtain a plurality of sub data distribution intervals;
determining the real data distribution probability of the preset attention index in each sub-data distribution interval according to a plurality of real values corresponding to the preset attention index;
determining simulation data distribution probability of the preset attention index in each sub-data distribution interval according to a plurality of simulation values corresponding to the preset attention index;
and carrying out KL divergence calculation on the real data distribution probability and the simulation data distribution probability corresponding to each sub data distribution interval, and determining the probability distribution result according to the calculation result.
3. The method according to claim 2, characterized in that the data distribution interval is determined by:
determining a minimum value between a plurality of real values and a plurality of simulation values corresponding to a preset attention index, and determining the minimum value as a section lower limit value corresponding to the data distribution section;
determining a maximum value between a plurality of real values and a plurality of simulation values corresponding to a preset attention index, and determining the maximum value as an interval upper limit value corresponding to the data distribution interval;
and forming the data distribution interval by the interval lower limit value and the interval upper limit value.
4. The method according to claim 2, wherein the probability distribution result comprises KL divergence between real test data and the simulated test data,
wherein the KL divergence between the real test data and the simulated test data is determined by the following formula:
in the formula, D KL (x|y) represents a KL divergence between the real test data X and the simulation test data Y, M represents the number of sub data distribution sections, p (x=i) represents a real data distribution probability of a preset attention index in an i-th sub data distribution section, and p (y=i) represents a simulation data distribution probability of a preset attention index in the i-th sub data distribution section.
5. The method of claim 4, wherein determining the confidence level for the simulated scene based on the probability distribution result comprises:
comparing the KL divergence between the real test data and the simulation test data with a preset threshold value;
determining that the KL divergence is smaller than or equal to the preset threshold value, and determining that the simulation scene is credible;
and determining that the KL divergence is larger than the preset threshold value, and determining that the simulation scene is not credible.
6. The method according to claim 1, wherein the method further comprises:
if the simulation scene is credible, verifying an automatic driving algorithm by using the simulation scene to replace a real scene;
and if the simulation scene is not credible, returning to reconstruct the simulation scene corresponding to the preset real scene.
7. A device for determining reliability of a simulation scene, the device comprising:
the acquisition module is used for acquiring real test data corresponding to a preset attention index obtained by repeatedly executing the test for a plurality of times under a preset real scene;
the construction module is used for constructing a simulation scene corresponding to the preset real scene;
the simulation module is used for repeatedly executing multiple simulation tests in the simulation scene to obtain simulation test data corresponding to the preset attention index;
the probability distribution calculation module is used for carrying out probability distribution calculation on the real test data and the simulation test data corresponding to the preset attention index, and determining a probability distribution result, wherein the probability distribution result indicates the consistency between the real test data and the simulation test data;
and the credibility determining module is used for determining the credibility corresponding to the simulation scene according to the probability distribution result.
8. The apparatus of claim 7, wherein the real test data comprises a real value of the predetermined attention index at each site test, the simulation test data comprises a simulation value of the predetermined attention index at each simulation test,
wherein the probability distribution calculation module is further configured to:
determining a data distribution interval between the real test data and the simulation test data;
equally dividing the data distribution interval to obtain a plurality of sub data distribution intervals;
determining the real data distribution probability of the preset attention index in each sub-data distribution interval according to a plurality of real values corresponding to the preset attention index;
determining simulation data distribution probability of the preset attention index in each sub-data distribution interval according to a plurality of simulation values corresponding to the preset attention index;
and carrying out KL divergence calculation on the real data distribution probability and the simulation data distribution probability corresponding to each sub data distribution interval, and determining the probability distribution result according to the calculation result.
9. An electronic device, comprising: a processor, a memory and a bus, said memory storing machine readable instructions executable by said processor, said processor and said memory communicating via said bus when the electronic device is running, said machine readable instructions when executed by said processor performing the steps of the method for determining trustworthiness of a simulation scenario according to any one of claims 1 to 6.
10. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, performs the steps of the method for determining the trustworthiness of a simulation scenario according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311862605.7A CN117787005A (en) | 2023-12-29 | 2023-12-29 | Method and device for determining credibility of simulation scene and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311862605.7A CN117787005A (en) | 2023-12-29 | 2023-12-29 | Method and device for determining credibility of simulation scene and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117787005A true CN117787005A (en) | 2024-03-29 |
Family
ID=90394428
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311862605.7A Pending CN117787005A (en) | 2023-12-29 | 2023-12-29 | Method and device for determining credibility of simulation scene and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117787005A (en) |
-
2023
- 2023-12-29 CN CN202311862605.7A patent/CN117787005A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113485931B (en) | Test method, test device, electronic equipment and computer readable storage medium | |
| CN113326210B (en) | Method and device for determining automatic driving test scene | |
| JP2019105927A (en) | Failure probability calculation device, failure probability calculation method and program | |
| CN114879613A (en) | Industrial control system information security attack risk assessment method and system | |
| US20160292307A1 (en) | Temporal logic robustness guided testing for cyber-physical systems | |
| CN117216454B (en) | Reliability assessment method and device based on fuzzy non-probability, terminal and storage medium | |
| CN105159826B (en) | A kind of method and apparatus of wrong sentence in positioning target program | |
| CN111679600B (en) | Comparison method of control system, control terminal and computer readable storage medium | |
| CN116627847A (en) | Browser compatibility testing method and system | |
| CN117787005A (en) | Method and device for determining credibility of simulation scene and electronic equipment | |
| CN112817855A (en) | System crowd test method, device and computer storage medium | |
| CN115146491B (en) | Test method and device for automatic driving system, storage medium and electronic device | |
| CN111679298B (en) | Integrity monitoring method and device of navigation system and electronic equipment | |
| CN109471766B (en) | Sequential fault diagnosis method and device based on testability model | |
| CN112905445A (en) | Log-based test method and device and computer system | |
| CN117891566B (en) | Reliability evaluation method, device, equipment, medium and product of intelligent software | |
| CN111507639B (en) | Financing risk analysis method and device | |
| CN116915513B (en) | False data injection attack detection method and device | |
| CN117971721B (en) | Module verification method and device of multi-core processor and electronic equipment | |
| CN116484596A (en) | Calculation method and device for test space coverage | |
| CN116702521B (en) | Automatic driving scene consistency comparison method and device and electronic equipment | |
| CN116881088B (en) | System monitoring method and device, storage medium and electronic device | |
| CN115147700B (en) | Target recognition rate parameter calibration method and device for image recognition system | |
| CN118093434B (en) | Performance evaluation and test method and system for intelligent information system | |
| CN114816888B (en) | Method, device and storage medium for testing and evaluating persistent memory |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |