CN116680517B - Method and device for determining failure probability in automatic driving simulation test - Google Patents

Abstract

The application provides a method and a device for determining failure probability in automatic driving simulation test, and relates to the technical field of automatic driving, wherein the method comprises the following steps: performing high coverage uniform sampling in a high-dimensional parameter space to obtain a first test scene set; performing simulation test on each first test scene in the first test scene set to obtain a first test index value; determining scene weight of each first test scene, and determining first test failure probability according to the scene weight and the first test index value; if the first test failure probability does not meet the accuracy requirement, performing sensitivity analysis on the first scene parameters influencing the first test index value to determine sensitive scene parameters, and reducing the high-dimensional parameter space to a low-dimensional parameter space formed by the sensitive scene parameters; reliability analysis is performed on the low-dimensional parameter space to determine the final failure probability. By adopting the method and the device for determining the failure probability in the automatic driving simulation test, the problem of low calculation efficiency of the existing failure probability determination method is solved.

Description

Method and device for determining failure probability in automatic driving simulation test

Technical Field

The application relates to the technical field of automatic driving, in particular to a method and a device for determining failure probability in automatic driving simulation test.

Background

With the rapid development of automated driving systems in recent years, it becomes important how to verify and verify the reliability of automated driving vehicles and systems. The risk that an autopilot system is at can be measured by the probability that the system is accident, i.e. failure, verifying whether the failure probability of the autopilot system is low to an acceptable level is an important goal of autopilot verification. At present, simulation test has become an important means for verifying the reliability of an automatic driving system, in the simulation test, data sampling is directly performed in a parameter space by means of Monte Carlo or Latin hypercube, sobol sequences and the like, and then failure probability is determined by performing the simulation test on a specific scene obtained by sampling.

However, for mature automatic driving systems and vehicles, test failure is usually a very rare event, if the probability of occurrence of the rare event is estimated by adopting a mode of Monte Carlo sampling or Latin hypercube, sobol sequence and the like, a large amount of sampling is required on the premise of meeting the accuracy requirement, the simulation test cost is high, and the problem of low calculation efficiency is caused.

Disclosure of Invention

Therefore, the application aims to provide a method and a device for determining failure probability in automatic driving simulation test, so as to solve the problem of low calculation efficiency of the existing failure probability determination method.

In a first aspect, an embodiment of the present application provides a method for determining failure probability in an autopilot simulation test, including:

high coverage uniform sampling is carried out in a high-dimensional parameter space corresponding to the automatic driving logic scene, and a first test scene set is obtained;

performing simulation test on each first test scene in the first test scene set to obtain a first test index value of a tested system of the tested vehicle in each first test scene;

determining scene weights of each first test scene in a high-dimensional parameter space, and determining first test failure probability of the tested system according to the scene weights and the first test index values;

if the first test failure probability does not meet the accuracy requirement, performing sensitivity analysis on the first scene parameters affecting the first test index value to determine sensitive scene parameters, and reducing the dimension of the high-dimension parameter space to a low-dimension parameter space formed by the sensitive scene parameters;

and carrying out reliability analysis on the low-dimensional parameter space, determining a second test failure probability of the tested system, and taking the second test failure probability as a final failure probability.

Optionally, determining scene weights of each first test scene in the high-dimensional parameter space includes: substituting the value of a first scene parameter corresponding to each first test scene into a joint density function to obtain a joint density value of the first test scene; taking the product of the joint density value and the probability cutoff value as the scene weight of the first test scene.

Optionally, determining the first test failure probability of the tested system according to the scene weight and the first test index value includes: taking a first test scene with the first test index value in a preset failure domain as a first test failure scene; and taking the ratio of the sum of scene weights corresponding to all the first test failure scenes to the total number of the first test scenes as the first test failure probability.

Optionally, performing sensitivity analysis on the first scene parameter affecting the first test index value to determine a sensitive scene parameter includes: the method comprises the steps of determining a sensitivity index value corresponding to each first scene parameter by carrying out model-free sensitivity analysis on a test index corresponding to a first test index value and the first scene parameter; sequencing all the first scene parameters according to the sequence from the high sensitivity index value to the low sensitivity index value; selecting a plurality of first target scene parameters according to the arrangement sequence of the first scene parameters, fitting the test indexes on the models of each first target scene parameter, and calculating posterior sensitivity indexes corresponding to the fitted models; and when the posterior sensitivity index meets the fitting stopping condition, the selected first target scene parameter is used as the sensitive scene parameter.

Optionally, performing reliability analysis on the low-dimensional parameter space to determine a second test failure probability of the system under test, including: converting the low-dimensional parameter space into a standard normal distribution space; taking the obtained actual sampling function as a target sampling function, and carrying out multi-round sampling and simulation test on the standard normal distribution space by utilizing the target sampling function to obtain a plurality of second test index values; and determining the second test failure probability according to the plurality of second test index values and the scene weight corresponding to each second test index value.

Optionally, performing multiple rounds of sampling and simulation testing on the standard normal distribution space by using the target sampling function to obtain a plurality of second test index values, including: in the current round of sampling, sampling a standard normal distribution space by utilizing a target sampling function to obtain a plurality of second test scenes; performing simulation test on each second test scene in the current round of sampling to obtain a second test index value of the tested system in each second test scene in the current round of sampling; determining whether an iteration stop condition is met according to the magnitude of the second test index value; if the iteration stopping condition is met, stopping iteration, and taking the second test index value at the moment as the second test index value obtained through multiple rounds of sampling and simulation testing.

Optionally, after determining whether the iteration stop condition is satisfied according to the size of the second test index value, the method further includes: if the iteration stopping condition is not met, sequencing the second test index values corresponding to the plurality of second test scenes in the round of sampling according to the sequence from large to small, and selecting the second test scene corresponding to the second test index value in a preset proportion as a preferable failure test scene; and clustering the second scene parameters corresponding to the optimal failure test scenes, updating the target sampling function according to the clustering result to obtain a new target sampling function, taking the next round of sampling, and returning to execute the step of sampling the standard normal distribution space by using the target sampling function to obtain a plurality of second test scenes.

Optionally, determining the second test failure probability according to the plurality of second test index values and the scene weight corresponding to each second test index value includes: selecting a test index value smaller than or equal to the test index failure threshold value from the plurality of second test index values as a second target test index value; and taking the second test scene corresponding to the second target test index value as a second target test scene, and taking the ratio of the sum of scene weights of a plurality of second target test scenes to the set number as the second test failure probability.

Optionally, after determining the first test failure probability of the tested system according to the scene weight and the first test index value, the method further includes: and if the first test failure probability meets the accuracy requirement, taking the first test failure probability as the final failure probability of the tested system.

In a second aspect, an embodiment of the present application further provides a device for determining a failure probability in an autopilot simulation test, where the device includes:

the high-dimensional sampling module is used for carrying out high-coverage uniform sampling in a high-dimensional parameter space corresponding to the automatic driving logic scene to obtain a first test scene set;

the simulation test module is used for performing simulation test on each first test scene in the first test scene set to obtain a first test index value of a tested system of the tested vehicle in each first test scene;

the first probability determining module is used for determining scene weights of each first test scene in the high-dimensional parameter space, and determining first test failure probability of the tested system according to the scene weights and the first test index values;

the parameter space dimension reduction module is used for carrying out sensitivity analysis on the first scene parameters influencing the first test index value to determine sensitive scene parameters if the first test failure probability does not meet the accuracy requirement, and reducing the dimension of the high-dimension parameter space to a low-dimension parameter space formed by the sensitive scene parameters;

And the second probability determining module is used for carrying out reliability analysis on the low-dimensional parameter space, determining the second test failure probability of the tested system and taking the second test failure probability as the final failure probability.

The embodiment of the application has the following beneficial effects:

according to the method and the device for determining the failure probability in the automatic driving simulation test, which are provided by the embodiment of the application, high coverage sampling can be carried out in a high-dimensional parameter space to obtain a plurality of first test scenes, whether the accuracy requirement is met or not is determined according to the simulation test result of each first test scene, if the accuracy requirement is not met, the high-dimensional parameter space is subjected to dimension reduction processing, and reliability analysis is carried out in a low-dimensional parameter space to calculate more accurate failure probability, so that the calculation efficiency is improved.

In order to make the above objects, features and advantages of the present application more comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flowchart of a method for determining failure probability in an autopilot simulation test according to an embodiment of the present application;

fig. 2 is a schematic structural diagram of a device for determining failure probability in an autopilot simulation test according to an embodiment of the present application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. The components of the embodiments of the present application generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the application, as presented in the figures, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. Based on the embodiments of the present application, every other embodiment obtained by a person skilled in the art without making any inventive effort falls within the scope of protection of the present application.

It is important to note how to verify and verify the reliability of the autonomous vehicle and system becomes critical as the automated driving system is advanced in recent years before the present application is proposed. The risk that an autopilot system is at can be measured by the probability that the system is accident, i.e. failure, verifying whether the failure probability of the autopilot system is low to an acceptable level is an important goal of autopilot verification. At present, simulation test has become an important means for verifying the reliability of an automatic driving system, in the simulation test, data sampling is directly performed in a parameter space by means of Monte Carlo or Latin hypercube, sobol sequences and the like, and then failure probability is determined by performing the simulation test on a specific scene obtained by sampling. However, for mature automatic driving systems and vehicles, test failure is usually a very rare event, and if the probability of occurrence of the rare event is estimated by adopting a Monte Carlo sampling or Latin hypercube, sobol sequence and the like, a large amount of sampling is required on the premise of meeting the accuracy requirement, so that the problem of low calculation efficiency is caused.

Based on the above, the embodiment of the application provides a method for determining the failure probability in the automatic driving simulation test, so as to improve the calculation efficiency of the failure probability.

First, terms used in the present application will be explained.

Parameters: elements in the simulation test scenario, for example: the running speed of the main vehicle and the scene weather are rainy days, the values of the parameters obey the specified distribution function, and the space formed by all the parameters can be understood as a parameter space (also called a test space) formed by a group of random variables.

Functional scene: the most abstract level of operation scenario described by semantics, i.e. the entities within a scenario area and the relationships between entities are described by language scenario symbols, for example: and (5) automatically parking and changing lanes. The functional scenario is used for project definition, risk analysis and risk assessment of the conceptual stage, and in the simulation test process, the functional scenario often needs to be converted into a logic scenario and into a data format which can be used for a corresponding simulation environment.

Logic scenario: by defining the range of values of the variables in the parameter space, for example: the running speed of the main vehicle is 50 km/h-100 km/h to express the relation between the physical characteristics and the entities. The logical scenario is a further detailed description of the functional scenario based on state space variables for generating requirements in the project development stage. Any number of specific scenes can be derived for each logical scene with a continuous range of values.

Test scenario (specific scenario): the simulation test scene can be directly carried out, namely, the scene obtained after all parameters in the logic scene are assigned.

Referring to fig. 1, fig. 1 is a flowchart of a method for determining failure probability in an autopilot simulation test according to an embodiment of the present application. As shown in fig. 1, the method for determining failure probability in an autopilot simulation test provided by the embodiment of the present application includes:

step S101, high coverage uniform sampling is carried out in a high-dimensional parameter space corresponding to an automatic driving logic scene, and a first test scene set is obtained.

In this step, the autopilot logic scene may refer to a scene obtained by defining a range of values of parameters in a parameter space.

The high-dimensional parameter space may refer to a parameter space formed by a plurality of scene parameters and distribution functions corresponding to the scene parameters, and the parameters in the high-dimensional parameter space may be, for example, a speed of a host vehicle and a distribution function obeyed by the speed, a scene weather is a rainy day and rainfall obeying distribution function, a speed of an environmental vehicle and a distribution function obeyed by the speed, and a running road surface is a highway.

High coverage uniform sampling may refer to uniform sampling in a set step size throughout a high dimensional parameter space, and may be, for example, latin hypercube sampling or sobol sequence sampling.

The first test scene set may refer to a set of a plurality of first test scenes, where the first test scenes in the first test scene set correspond to sample points obtained after high coverage uniform sampling.

In the embodiment of the application, the whole automatic driving system of the automatic driving vehicle or the subsystem in the automatic driving system is tested by utilizing the automatic driving simulation test, the automatic driving vehicle is the tested vehicle, the automatic driving system or the subsystem in the automatic driving system is the tested system, and the performance of the tested system of the tested vehicle in the high-dimensional parameter space, namely the probability of test failure, is determined by the test results in different test scenes corresponding to the high-dimensional parameter space.

Here, a data sampling manner of latin hypercube sampling or sobol sequence sampling may be utilized to perform high coverage uniform sampling on each scene parameter in the high-dimensional parameter space, so as to obtain a plurality of sample points. One sample point is an N-dimensional array, the dimension of the array is determined by the number of scene parameters, namely, each sample point contains all parameters in a high-dimensional parameter space and values of all parameters, one sample point corresponds to one first test scene, if 1000 sample points are acquired, 1000 first test scenes are correspondingly acquired, and a first test scene set is formed by the 1000 first test scenes.

Step S102, performing simulation test on each first test scene in the first test scene set to obtain a first test index value of a tested system of the tested vehicle in each first test scene.

In this step, the first test index value may refer to a value of a test index obtained by performing a simulation test in the first test scenario.

The test index may refer to an evaluation index of the system under test, and may be, for example, a collision time, or a safe distance.

In the embodiment of the application, assuming that 1000 first test scenes are in total in the first test scene set, parallel simulation test is performed on each first test scene to obtain a test result of a tested system of a tested vehicle in each first test scene. Taking the test index as the collision time as an example, 1000 values of the collision time can be obtained, and the 1000 values of the collision time are the first test index value.

Step S103, determining scene weights of each first test scene in a high-dimensional parameter space, and determining first test failure probability of the tested system according to the scene weights and the first test index values.

In this step, the scene weight may refer to the weight of the first test scene in the high-dimensional parameter space.

Since uniform sampling is adopted when sampling the high-dimensional parameter space, and in practice, different values of the same scene parameter may have different weights according to the distribution function, the first test scene should have different scene weights so as to achieve the purpose of unbiased estimation failure probability.

The first test failure probability may refer to a failure probability of a system under test of the vehicle under test when performing a simulation test in the first test scenario.

In the embodiment of the application, whether the first test scene is a first test failure scene or not can be determined according to the first test index value corresponding to each first test scene, and the first test failure probability is calculated according to the number of the first test failure scenes and the scene weight.

In an alternative embodiment, determining scene weights for each first test scene in the high-dimensional parameter space includes: substituting the value of a first scene parameter corresponding to each first test scene into a joint density function to obtain a joint density value of the first test scene; taking the product of the joint density value and the probability cutoff value as the scene weight of the first test scene.

Here, the joint density function may refer to a joint distribution function, that is, a joint probability density function corresponding to a scene parameter distribution in a high-dimensional parameter space, and according to the joint density function, an edge probability corresponding to a specific scene parameter may be obtained.

The probability cutoff value may refer to a value obtained by cutting in a high-dimensional parameter space and then designating a probability interval, and the probability cutoff value may be calculated by using the following formula:

；

in the above-mentioned method, the step of,representing a distribution function corresponding to the ith scene parameter; />To->Namely, a designated probability interval; />Representing a probability cutoff.

Specifically, taking a single first test scene as an example, taking the value of the scene parameter corresponding to the first test scene as the input of the joint density function, so as to obtain the edge probability corresponding to the first test scene, namely the joint density value according to the calculation result of the joint density function. Then, the joint density value is multiplied by the probability cutoff value, and the result obtained by multiplication is used as the scene weight of the first test scene.

In an alternative embodiment, determining the first test failure probability of the tested system according to the scene weight and the first test index value includes: taking a first test scene with the first test index value in a preset failure domain as a first test failure scene; and taking the ratio of the sum of scene weights corresponding to all the first test failure scenes to the total number of the first test scenes as the first test failure probability.

Specifically, determining a failure domain corresponding to a test index, where the failure domain is a preset failure domain, for example, setting the failure domain as a first test index value less than or equal to 0, determining whether the first test index value is in the failure domain for a first test index value corresponding to each first test scene, and if so, determining the first test scene corresponding to the first test index value as a first test failure scene. And counting all first test failure scenes, taking the sum of scene weights of all first test failure scenes as a numerator, taking the total number of the first test scenes, namely the total number of sample points obtained in high coverage uniform sampling, as a denominator, and taking the ratio of the numerator to the denominator as the first test failure probability.

Step S104, if the first test failure probability does not meet the accuracy requirement, performing sensitivity analysis on the first scene parameters affecting the first test index value to determine the sensitive scene parameters, and reducing the high-dimensional parameter space to a low-dimensional parameter space composed of the sensitive scene parameters.

In this step, whether the first test failure probability meets the accuracy requirement may be determined according to a comparison result of the first test failure probability and the set failure probability.

In the embodiment of the application, if the first test failure probability is smaller than or equal to the set failure probability, it is determined that the first test failure probability does not meet the accuracy requirement, the first test failure probability may be a smaller value, the failure probability of the tested vehicle or the tested system in the simulation test in the high-dimensional parameter space cannot be accurately predicted by adopting the first test failure probability obtained by the sampling method, and further calculation of the failure probability is required to improve the calculation accuracy.

When further calculation is performed, a sensitive scene parameter which influences the first test index value needs to be determined, wherein the sensitive scene parameter can be selected from a plurality of first scene parameters by using a sensitive analysis method, and a low-dimensional parameter space is constructed by the selected sensitive scene parameter, so that a second test failure probability can be obtained by performing simulation test in the low-dimensional parameter space, the complexity of the low-dimensional parameter space is far lower than that of an original test space, and the calculation efficiency and the estimation accuracy can be improved.

In an alternative embodiment, after determining the first test failure probability of the tested system according to the scene weight and the first test index value, the method further includes: and if the first test failure probability meets the accuracy requirement, taking the first test failure probability as the final failure probability of the tested system.

Specifically, if the first test failure probability is greater than the set failure probability, determining that the first test failure probability meets the accuracy requirement, and considering the first test failure probability obtained after sampling the high-dimensional parameter space, the failure probability of the tested system of the tested vehicle in the simulation test in the test scene corresponding to the high-dimensional parameter space can be accurately calculated without further failure probability calculation. Then, the calculated first test failure probability is directly taken as the final failure probability.

The setting failure probability may be 20/total number of sample points, and a person skilled in the art may determine a specific value of the setting failure probability according to the actual situation.

In an alternative embodiment, performing a sensitivity analysis on the first scene parameter affecting the first test index value to determine a sensitive scene parameter includes: the method comprises the steps of determining a sensitivity index value corresponding to each first scene parameter by carrying out model-free sensitivity analysis on a test index corresponding to a first test index value and the first scene parameter; sequencing all the first scene parameters according to the sequence from the high sensitivity index value to the low sensitivity index value; selecting a plurality of first target scene parameters according to the arrangement sequence of the first scene parameters, fitting the test indexes on the models of each first target scene parameter, and calculating posterior sensitivity indexes corresponding to the fitted models; and when the posterior sensitivity index meets the fitting stopping condition, the selected first target scene parameter is used as the sensitive scene parameter.

Specifically, in order to obtain a better global sensitivity analysis result and overcome potential errors as much as possible, a sensitivity index delta based on conditional probability is selected as a sensitivity index in model free sensitivity analysis.

Here, in calculating the sensitivity index, the sensitivity index of each first scene parameter in the high-dimensional parameter space may be calculated for the first scene parameter, and the interval sensitivity index may be obtained by the following calculation formula：

；

In the above formula, Y represents a dependent variable, namely a test index; x represents an argument, i.e. a first scene parameter,probability density function representing dependent variable, +.>Probability density function representing an argument +.>Representing a conditional probability density function.

Taking the dependent variable Y as an example, the overall probability density and the conditional probability density can be calculated by using a kernel density estimation method, wherein the calculation formula of the overall probability density is as follows:

；

in the above formula, h represents bandwidth; k%) Representing a kernel function; y represents an argument, < >>Representing a specific value; n represents the number of values, i.e. a total of more and less observations.

The conditional probability density is calculated as follows:

；

in the above-mentioned method, the step of,representing bandwidth; k (/ -)>) Representing a kernel function; y represents an independent variable; / >Representing a specific value; />Representing the number of values, namely a total of more and less observed values; />Representing the set of values of x.

The theoretical value of the sensitivity index obtained through calculation is between 0 and 1, and the larger the value of the sensitivity index is, the larger the value influence of the first scene parameter corresponding to the test index is, and otherwise, the smaller the value influence of the first scene parameter corresponding to the test index is. And then, according to the order of the values of the sensitivity indexes from large to small, carrying out sensitivity sorting on the first scene parameters corresponding to the sensitivity indexes, wherein the influence of the first scene parameters with the higher sorting on the test indexes is larger, and the influence of the first scene parameters with the lower sorting on the test indexes is smaller.

After no model sensitivity analysis, a model based sensitivity analysis was performed. Here, various fitting models may be employed, such as: linear regression model, neural network model, kriging model, polynomial regression model to fit the model of the test index with respect to the first scene parameter, model-based sensitivity analysis automatically finds the optimal model among all models. Model fitting can be performed as follows: according to the arrangement sequence of the first scene parameters, selecting the first scene parameters arranged in the first position as first target scene parameters, for example: and selecting the vehicle running speed of the main vehicle as a first target scene parameter, fitting a model of the test index about the vehicle running speed of the main vehicle, calculating a posterior sensitivity index corresponding to the fitted model at the moment, and determining whether the posterior sensitivity index corresponding to the first fitting meets the stop fitting condition.

If the stop fitting condition is not satisfied, continuing to select the first scene parameters ranked in the first position and the second position as the first target scene parameters, for example: and selecting the vehicle running speed of the main vehicle arranged at the first position and the vehicle running speed of the environmental vehicle arranged at the second position as first target scene parameters, fitting a model of the test index about the vehicle running speed of the environmental vehicle, calculating a posterior sensitivity index corresponding to the fitted model at the moment, and determining whether the posterior sensitivity index corresponding to the second fitting meets the stop fitting condition.

And then, continuously selecting the first scene parameters arranged in the first position, the second position and the third position as first target scene parameters, and taking a plurality of first scene parameters in the first target scene parameters selected at the moment as sensitive scene parameters and taking the rest first scene parameters which are not selected as the first target scene parameters as insensitive scene parameters if the posterior sensitivity index corresponding to the Nth fitting meets the fitting stopping condition. For example: and in the third fitting, selecting the first scene parameters arranged in the first position, the second position and the third position as first target scene parameters, and if the fitting stopping condition is met at the moment, taking the first scene parameters arranged in the first position, the second position and the third position as sensitive scene parameters.

The condition of stopping fitting is that the posterior sensitivity index is not improved, that is, the posterior sensitivity index obtained by the last fitting is smaller than or equal to the posterior sensitivity index obtained by the previous fitting. The posterior sensitivity index is used to evaluate the prediction quality of the model, and may refer to the posterior prediction index of the model, i.e., to the key index prediction coefficient (Coefficient of Prognosis, coP) that characterizes the prediction quality. The posterior sensitivity index can be calculated by the following calculation formula:

；

in the above-mentioned method, the step of,representing the observed value, i.e. the sampled value of the parameter space, < >>Representing the fitting value, i.e. the output result of the fitting model,/->Representing the expected value, an average value of the plurality of observed values may be used as the expected value; text represents the test set.

In this way, the selected sensitive scene parameters can be considered to determine the performance of the tested system of the tested vehicle in the high-dimensional parameter space, and the insensitive scene parameters have little influence on the performance of the tested vehicle. The original high-dimensional parameter space may then be compressed into a low-dimensional parameter space consisting of sensitive scene parameters, such as: and reducing the 100-dimensional high-dimensional parameter space to a 5-dimensional low-dimensional parameter space, namely selecting 5 first scene parameters as sensitive scene parameters. For the insensitive scene parameter, the average value of the insensitive scene parameter in the sample can be used as the value of the insensitive scene parameter, so that the value can be used for simulation test.

Step S105, reliability analysis is carried out on the low-dimensional parameter space, the second test failure probability of the tested system is determined, and the second test failure probability is taken as the final failure probability.

In this step, reliability analysis may refer to an analysis method based on adaptive importance sampling of the mixture gaussian distribution.

The covariance of the Gaussian mixture distribution is forcedly set to 0, so that the algorithm is prevented from being trapped into local optimum, and meanwhile, the standard deviation of the Gaussian mixture distribution is also set to a lower limit value, so that the exploration capacity of the algorithm to the space is ensured.

The second test failure probability may refer to a test failure probability obtained by performing the simulation test in the low-dimensional parameter space.

In the embodiment of the application, multiple rounds of simulation tests are carried out in the low-dimensional parameter space, a second test index value after each round of simulation tests is obtained, and a test failure scene in the low-dimensional parameter space is determined according to the second test index value, so that the second test failure probability is determined according to the determined test failure scene.

In an alternative embodiment, reliability analysis is performed on the low-dimensional parameter space to determine a second test failure probability of the system under test, including: converting the low-dimensional parameter space into a standard normal distribution space; taking the obtained actual sampling function as a target sampling function, and carrying out multi-round sampling and simulation test on the standard normal distribution space by utilizing the target sampling function to obtain a plurality of second test index values; and determining the second test failure probability according to the plurality of second test index values and the scene weight corresponding to each second test index value.

Here, the second scene parameter refers to a scene parameter in the low-dimensional parameter space.

The second test index value may refer to a value of a test index obtained by performing a simulation test in the second test scenario.

The actual sampling function may refer to a proposed distribution function, which is used to sample the converted standard northlye distribution space.

Specifically, the low-dimensional parameter space is initialized, i.e. the low-dimensional parameter space is converted into a standard normal distribution space. And when the space conversion is carried out, different methods are used for the space conversion according to whether the determined second scene parameters are mutually independent.

The rosenberg transform (Rosenblatt transformation) method is used if the different second scene parameters are determined to be mutually independent, and the natto transform (Nataf transformation) method is used if the different second scene parameters are determined to be not mutually independent.

Then, the actual sampling function is set to a mixture gaussian distribution (the first round of circulation is a standard normal distribution) to match the standard normal distribution space, and the standard normal distribution space is sampled by using the multidimensional standard normal distribution function. When sampling, the sampling number of each round of sampling can be set to obtain a corresponding number of sample points, and each sample point corresponds to a second test scene. After multiple rounds of sampling and simulation testing, a plurality of second test index values can be obtained, and the second test failure probability is determined according to the obtained second test index values and the scene weights of the second test scenes corresponding to the second test index values.

In an alternative embodiment, the sampling and simulation test are performed on the standard normal distribution space by using the target sampling function in multiple rounds, so as to obtain a plurality of second test index values, including: in the current round of sampling, sampling a standard normal distribution space by utilizing a target sampling function to obtain a plurality of second test scenes; performing simulation test on each second test scene in the current round of sampling to obtain a second test index value of the tested system in each second test scene in the current round of sampling; determining whether an iteration stop condition is met according to the magnitude of the second test index value; if the iteration stopping condition is met, stopping iteration, and taking the second test index value at the moment as the second test index value obtained through multiple rounds of sampling and simulation testing.

Taking single-round sampling and simulation test as an example, obtaining a plurality of sample points after the sampling of the round, wherein each sample point corresponds to a plurality of second test scenes of the round, and carrying out parallel simulation on the plurality of second test scenes of the round to obtain a second test index value corresponding to each second test scene of the round. And determining whether the second test index value is in a preset failure domain or not according to each second test index value, and if the second test index value is in the preset failure domain, taking a second test scene corresponding to the second test index value as a second test failure scene. Counting the number of second test failure scenes in the round, if the proportion of the number of the second test failure scenes to the total number of all the second test scenes is larger than or equal to a preset proportion or the iteration number reaches a preset upper limit, determining that an iteration stop condition is met, stopping iteration if the iteration stop condition is met, and taking the second test index value in the round as a final second test index value obtained after multiple rounds of sampling and simulation test.

In an alternative embodiment, after determining whether the iteration stop condition is satisfied according to the size of the second test index value, the method further includes: if the iteration stopping condition is not met, sequencing the second test index values corresponding to the plurality of second test scenes in the round of sampling according to the sequence from large to small, and selecting the second test scene corresponding to the second test index value in a preset proportion as a preferable failure test scene; and clustering the second scene parameters corresponding to the optimal failure test scenes, updating the target sampling function according to the clustering result to obtain a new target sampling function, taking the next round of sampling, and returning to execute the step of sampling the standard normal distribution space by using the target sampling function to obtain a plurality of second test scenes.

Specifically, if the iteration stop condition is not met, the plurality of second test index values in the simulation test of the round are ordered in the order from big to small, if the preset failure domain is that the second test index value is less than or equal to 0, the second test index value ranked at the last 10% is used as the preferable second test index value, if the preset failure domain is that the preset failure domain is >0, the second test index value ranked at the first 10% is used as the preferable second test index value, and the second test scene corresponding to the preferable second test index value is used as the preferable failure test scene. Wherein 10% is the preset proportion, and the second test index value of the first 10% or the last 10% is selected as the preferred second test index value, which is determined by the range of the preset failure domain.

Clustering values of the second scene parameters corresponding to the preferable failure test scenes by using a DBSCAN algorithm to obtain a plurality of classified preferable failure test scene sets, and determining the category to which each preferable failure test scene belongs. After clustering, a clustering result of the present clustering can be obtained, and the clustering result includes but is not limited to: the number of the mixed Gaussian distributions, which mixed Gaussian distribution the sample point corresponding to each second test scene belongs to, and the weight of each mixed Gaussian distribution. Wherein for a first round of sampling the original distribution is a standard normal distribution and for a second round of sampling the original distribution is an updated target sampling function obtained after the first round of sampling.

The DBSCAN algorithm is a clustering algorithm without inputting the clustering number in advance, the DBSCAN algorithm automatically sets the category number according to the density and connectivity of the sample points and performs clustering, the DBSCAN algorithm is taken as a clustering algorithm for example, and other clustering algorithms without inputting the clustering number in advance can be adopted by a person skilled in the art to perform clustering.

According to the obtained clustering result, the objective sampling function is iteratively updated by using an Expectation-maximization (EM) algorithm, the EM algorithm performs iterative computation on the weights of the mixed gaussian distributions and the parameters of each gaussian distribution, so as to update the weights and the parameters of each mixed gaussian distribution to obtain an updated objective sampling function, and the updated objective sampling function is still the mixed gaussian distribution function, only the numerical value of a specific parameter is changed, for example: a desired, covariance matrix, weights for each gaussian distribution.

And after the updated target sampling function is obtained, next sampling and test simulation are carried out, namely, the updated target sampling function is utilized to carry out next sampling and test simulation on the standard normal distribution space, after the multi-sampling and test simulation are carried out, when iteration stop conditions are met, a second test index value is obtained at the moment and is used as a final second test index value, and the second failure probability is calculated by utilizing the final second test index value.

In an alternative embodiment, determining the second test failure probability according to the plurality of second test index values and the scene weights corresponding to each of the second test index values includes: selecting a value smaller than or equal to a test index failure threshold value from the plurality of second test index values as a second target test index value; and taking the second test scene corresponding to the second target test index value as a second target test scene, and taking the ratio of the sum of scene weights of a plurality of second target test scenes to the set number as the second test failure probability.

Here, the set number may refer to the number of samples per sampling in the low-dimensional parameter space, i.e. the number of second test scenes after each sampling, since the number of samples collected at each sampling is the same, for example: all 1500 samples, and therefore the set number is also a fixed value of 1500.

Specifically, the test index failure threshold is noted as: t, the value of the test index failure threshold may be set, for example: and setting a test index failure threshold value to be 0, namely T=0, determining whether the second test index value is smaller than or equal to 0 for each second test index value, if so, taking the second test index value as a second target test index value, taking a second test scene corresponding to the second target test index value as a second target test scene, taking the ratio of the sum of scene weights of all the second target test scenes to 1500 as second test failure probability, and calculating to obtain the second test failure probability which is the determined final failure probability. The specific value of the failure threshold of the test index can be determined by a person skilled in the art according to the actual situation, and the present application is not limited herein.

Compared with the method for determining the failure probability in the automatic driving simulation test in the prior art, the method provided by the application has the advantages that high coverage sampling can be carried out in a high-dimensional parameter space to obtain a plurality of first test scenes, whether the accuracy requirement is met or not is determined according to the simulation test result of each first test scene, if the accuracy requirement is not met, the high-dimensional parameter space is subjected to dimension reduction processing, and reliability analysis is carried out in a low-dimensional parameter space to calculate more accurate failure probability, so that the calculation efficiency is improved, and the problem of low calculation efficiency of the conventional failure probability determination method is solved.

Based on the same inventive concept, the embodiment of the application also provides a device for determining failure probability in the automatic driving simulation test, which corresponds to the method for determining failure probability in the automatic driving simulation test.

Referring to fig. 2, fig. 2 is a schematic structural diagram of a device for determining failure probability in an autopilot simulation test according to an embodiment of the present application. As shown in fig. 2, the apparatus 200 for determining failure probability in the autopilot simulation test includes:

the high-dimensional sampling module 201 is configured to perform high coverage uniform sampling in a high-dimensional parameter space corresponding to an autopilot logic scene, so as to obtain a first test scene set;

the simulation test module 202 is configured to perform a simulation test on each first test scene in the first test scene set, and obtain a first test index value of a tested system of the tested vehicle in each first test scene;

the first probability determining module 203 is configured to determine a scene weight of each first test scene in the high-dimensional parameter space, and determine a first test failure probability of the tested system according to the scene weight and the first test index value;

The parameter space dimension reduction module 204 is configured to perform sensitivity analysis on the first scene parameter affecting the first test index value to determine a sensitive scene parameter if the first test failure probability does not meet the accuracy requirement, and reduce the dimension of the high-dimension parameter space to a low-dimension parameter space formed by the sensitive scene parameter;

the second probability determining module 205 is configured to perform reliability analysis on the low-dimensional parameter space, determine a second test failure probability of the tested system, and take the second test failure probability as a final failure probability.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.

In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. The above-described apparatus embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and there may be other manners of division in actual implementation, and for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some communication interface, device or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

Finally, it should be noted that: the above examples are only specific embodiments of the present application, and are not intended to limit the scope of the present application, but it should be understood by those skilled in the art that the present application is not limited thereto, and that the present application is described in detail with reference to the foregoing examples: any person skilled in the art may modify or easily conceive of the technical solution described in the foregoing embodiments, or perform equivalent substitution of some of the technical features, while remaining within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application, and are intended to be included in the scope of the present application. Therefore, the protection scope of the application is subject to the protection scope of the claims.

Claims (10)

1. The method for determining the failure probability in the automatic driving simulation test is characterized by comprising the following steps of:

high coverage uniform sampling is carried out in a high-dimensional parameter space corresponding to the automatic driving logic scene, and a first test scene set is obtained;

performing simulation test on each first test scene in the first test scene set to obtain a first test index value of a tested system of a tested vehicle in each first test scene;

Determining scene weights of each first test scene in the high-dimensional parameter space, and determining first test failure probability of the tested system according to the scene weights and the first test index values;

if the first test failure probability does not meet the accuracy requirement, performing sensitivity analysis on the first scene parameters affecting the first test index value to determine sensitive scene parameters, and reducing the dimension of the high-dimension parameter space to a low-dimension parameter space formed by the sensitive scene parameters;

and carrying out reliability analysis on the low-dimensional parameter space, determining second test failure probability of the tested system, and taking the second test failure probability as final failure probability.

2. The method of claim 1, wherein determining scene weights for each first test scene in the high-dimensional parameter space comprises:

substituting the value of a first scene parameter corresponding to each first test scene into a joint density function to obtain a joint density value of the first test scene;

and taking the product of the joint density value and the probability cutoff value as the scene weight of the first test scene.

3. The method of claim 1, wherein determining a first test failure probability of the system under test based on the scene weight and the first test index value comprises:

taking a first test scene of which the first test index value is in a preset failure domain as a first test failure scene;

and taking the ratio of the sum of scene weights corresponding to all the first test failure scenes to the total number of the first test scenes as the first test failure probability.

4. The method of claim 1, wherein said sensitivity analysis of the first scene parameters affecting the first test index value to determine sensitive scene parameters comprises:

the method comprises the steps of determining a sensitivity index value corresponding to each first scene parameter by carrying out model-free sensitivity analysis on a test index corresponding to a first test index value and the first scene parameter;

sequencing all the first scene parameters according to the sequence from the high sensitivity index value to the low sensitivity index value;

selecting a plurality of first target scene parameters according to the arrangement sequence of the first scene parameters, fitting the test indexes on the models of each first target scene parameter, and calculating posterior sensitivity indexes corresponding to the fitted models;

And when the posterior sensitivity index meets the fitting stopping condition, the first target scene parameter is selected as the sensitive scene parameter.

5. The method of claim 1, wherein said performing reliability analysis on said low-dimensional parameter space to determine a second test failure probability for said system under test comprises:

converting the low-dimensional parameter space into a standard normal distribution space;

taking the obtained actual sampling function as a target sampling function, and carrying out multi-round sampling and simulation test on the standard normal distribution space by utilizing the target sampling function to obtain a plurality of second test index values;

and determining a second test failure probability according to the second test index values and scene weights corresponding to the second test index values.

6. The method of claim 5, wherein the performing the sampling and simulation test on the standard normal distribution space by using the target sampling function for multiple rounds to obtain a plurality of second test index values comprises:

in the current round of sampling, sampling the standard normal distribution space by utilizing the target sampling function to obtain a plurality of second test scenes;

performing simulation test on each second test scene in the current round of sampling to obtain a second test index value of the tested system in each second test scene in the current round of sampling;

Determining whether an iteration stop condition is met according to the size of the second test index value;

if the iteration stopping condition is met, stopping iteration, and taking the second test index value at the moment as the second test index value obtained through multiple rounds of sampling and simulation testing.

7. The method of claim 6, further comprising, after said determining whether the iteration stop condition is satisfied based on the magnitude of the second test index value:

if the iteration stopping condition is not met, sequencing the second test index values corresponding to the plurality of second test scenes in the round of sampling according to the sequence from large to small, and selecting the second test scene corresponding to the second test index value in a preset proportion as a preferable failure test scene;

and clustering the second scene parameters corresponding to the optimal failure test scene, updating the target sampling function according to a clustering result to obtain a new target sampling function, taking the next round of sampling, and returning to execute the step of sampling the standard normal distribution space by using the target sampling function to obtain a plurality of second test scenes.

8. The method of claim 5, wherein determining a second test failure probability according to the plurality of second test index values and the scene weight corresponding to each second test index value comprises:

Selecting a test index value smaller than or equal to a test index failure threshold value from the plurality of second test index values as a second target test index value;

and taking the second test scene corresponding to the second target test index value as a second target test scene, and taking the ratio of the sum of scene weights of a plurality of second target test scenes to the set number as a second test failure probability.

9. The method of claim 1, further comprising, after said determining a first test failure probability for said system under test based on said scene weight and said first test index value:

and if the first test failure probability meets the accuracy requirement, taking the first test failure probability as the final failure probability of the tested system.

10. A device for determining failure probability in an automatic driving simulation test, comprising:

the high-dimensional sampling module is used for carrying out high-coverage uniform sampling in a high-dimensional parameter space corresponding to the automatic driving logic scene to obtain a first test scene set;

the simulation test module is used for performing simulation test on each first test scene in the first test scene set to obtain a first test index value of a tested system of the tested vehicle in each first test scene;

The first probability determining module is used for determining scene weights of each first test scene in the high-dimensional parameter space, and determining first test failure probability of the tested system according to the scene weights and the first test index values;

the parameter space dimension reduction module is used for carrying out sensitivity analysis on the first scene parameters influencing the first test index value to determine sensitive scene parameters if the first test failure probability does not meet the accuracy requirement, and reducing the dimension of the high-dimension parameter space to a low-dimension parameter space formed by the sensitive scene parameters;

and the second probability determining module is used for carrying out reliability analysis on the low-dimensional parameter space, determining second test failure probability of the tested system and taking the second test failure probability as final failure probability.