CN111881520B

CN111881520B - Anomaly detection method and device for automatic driving test, computer equipment and storage medium

Info

Publication number: CN111881520B
Application number: CN202010761506.XA
Authority: CN
Inventors: 王亚亮; 黄星尧; 谭伟华; 韩旭
Original assignee: Guangzhou Jingqi Technology Co ltd; Guangzhou Weride Technology Co Ltd
Current assignee: Guangzhou Jingqi Technology Co ltd; Guangzhou Weride Technology Co Ltd
Priority date: 2020-07-31
Filing date: 2020-07-31
Publication date: 2022-01-11
Anticipated expiration: 2040-07-31
Also published as: CN111881520A

Abstract

The embodiment of the invention discloses an anomaly detection method, an anomaly detection device, computer equipment and a storage medium for automatic driving test, wherein the method comprises the following steps: simulating an operation object to execute automatic driving on the virtual vehicle in the scene data so as to test an automatic driving program, and acquiring a test result, wherein the scene data is acquired when a real vehicle runs on the road surface; inquiring switching operation in the test result, wherein the switching operation represents switching from the automatic driving mode to the manual driving mode and switching from the manual driving mode to the automatic driving mode; if the switching operation is the same, determining the switching operation as a target operation; determining a state of an automatic driving program in scene data where a target operation occurs; and if the state is an abnormal state, positioning the factors of the target operation in the abnormal state according to the test result. Whether the test result of the automatic driving program in the real scene is abnormal or not is judged according to the state of the target operation, and the efficiency and the accuracy of abnormal detection are improved.

Description

Anomaly detection method and device for automatic driving test, computer equipment and storage medium

Technical Field

The embodiment of the invention relates to an automatic driving technology, in particular to an abnormity detection method and device for an automatic driving test, computer equipment and a storage medium.

Background

Automatic driving is taken as the main direction of intelligent and networking development in the fields of global automobile and transportation travel at present, and has important value in future transportation.

In order to ensure the safety of the automatic driving system of the vehicle, some simulation tests are usually performed on the automatic driving software in an unreal vehicle environment, and test results output by the tests are analyzed to locate problems occurring in the operation process of the automatic driving program.

However, because the scenarios of the automatic driving are complex and diversified, the test results based on the automatic driving programs in different scenarios are different, even though the test results of the automatic driving programs in the same scenario are different, and often a tester needs to perform a large number of tests on the automatic driving programs in each scenario to fully expose the potential abnormalities of the automatic driving programs. At present, a tester collects a large number of test results to perform post-analysis processing, analyzes and consults all the test results one by one, consumes much time and energy, and may have the situations of missed detection and false detection.

Disclosure of Invention

The invention provides an anomaly detection method, an anomaly detection device, computer equipment and a storage medium for automatic driving test, which aim to solve the problems of low detection efficiency, easy error detection and missed detection in purely manual anomaly detection operation.

In a first aspect, an embodiment of the present invention provides an abnormality detection method for an automatic driving test, including:

simulating a business object to execute automatic driving on a virtual vehicle in scene data so as to test an automatic driving program and obtain a test result, wherein the scene data is acquired when a real vehicle runs on a road surface;

inquiring switching operation in the test result, wherein the switching operation represents switching from an automatic driving mode to a manual driving mode and switching from the manual driving mode to the automatic driving mode;

if the switching operation is the same, determining the switching operation as a target operation;

determining a state of the autonomous driving program in scene data where the target operation occurs;

and if the state is an abnormal state, positioning factors of the target operation in the abnormal state according to the test result.

In a second aspect, an embodiment of the present invention further provides an abnormality detection apparatus for an automatic driving test, including:

the data acquisition module is used for simulating a service object to execute automatic driving on the virtual vehicle in scene data so as to test an automatic driving program and obtain a test result, wherein the scene data is acquired when a real vehicle runs on a road surface;

the query module is used for querying switching operation in the test result, wherein the switching operation represents switching from an automatic driving mode to a manual driving mode and switching from the manual driving mode to the automatic driving mode;

a target operation determining module, configured to determine, if the switching operations are the same, that the switching operation is a target operation;

a state determination module for determining a state of the autonomous driving program in scene data where the target operation occurs;

and the abnormal positioning module is used for positioning the factors of the target operation in the abnormal state according to the test result if the state is the abnormal state.

In a third aspect, an embodiment of the present invention further provides a computer device, where the computer device includes:

one or more processors;

a storage device for storing one or more programs,

when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the anomaly detection method as described in the first aspect.

In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the abnormality detection method according to the first aspect.

The method comprises the steps of executing automatic driving on a virtual vehicle in scene data by simulating a service object so as to test an automatic driving program, and obtaining a test result, wherein the scene data is acquired by a real vehicle when the real vehicle runs on a road surface; inquiring switching operation in the test result, wherein the switching operation represents switching from the automatic driving mode to the manual driving mode and switching from the manual driving mode to the automatic driving mode; if the switching operation is the same, determining the switching operation as a target operation; determining a state of an automatic driving program in scene data where a target operation occurs; and if the state is an abnormal state, positioning the factors of the target operation in the abnormal state according to the test result. The method comprises the steps of obtaining a test result of the automatic driving program in a real scene, automatically positioning the state of target operation from the test result, judging whether the test result of the automatic driving program is abnormal or not according to the state of the target operation, realizing an automatic abnormal detection process to a certain extent, further positioning the factor of the target operation in the abnormal state according to the abnormal state, avoiding the problems of false detection and missed detection in the automatic driving test result detected purely manually to a certain extent, carrying out efficient abnormal state detection on a large number of test results, improving the abnormal detection efficiency of the automatic driving test and ensuring the detection accuracy.

Drawings

FIG. 1 is a schematic diagram of an unmanned vehicle according to an embodiment of the present invention;

FIG. 2 is a schematic structural diagram of a test architecture according to an embodiment of the present invention;

fig. 3 is a flowchart of an anomaly detection method for an automatic driving test according to an embodiment of the present invention;

fig. 4 is a schematic diagram of a test scenario site loop setup according to an embodiment of the present invention;

fig. 5 is a flowchart of an abnormality detection method for an automatic driving test according to a second embodiment of the present invention;

fig. 6 is a schematic structural diagram of an abnormality detection apparatus for an automatic driving test according to a third embodiment of the present invention;

fig. 7 is a schematic structural diagram of a computer device according to a fourth embodiment of the present invention.

Detailed Description

The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.

In order to further understand the technical solution of the present invention, the definition of autonomous driving and the hierarchy of autonomous driving technologies are analyzed as follows:

the automatic driving is a technology for enabling an automobile to have environment perception and path planning and autonomously realize vehicle control, namely human-simulated driving or automatic driving performed by controlling the automobile through an electronic technology.

According to the control degree of a vehicle system on vehicle control tasks, the automatic driving technology is divided into five levels of L0-L5, and the system mainly plays an auxiliary function in the levels of L1-L3; when level L4 is reached, the vehicle drive will be handed to the system in its entirety, whereas L4, L5 differ in specific scenarios and full-scene applications.

The L0 level is called the non-automation level and is defined as: the driver is responsible for performing dynamic driving tasks throughout, possibly assisted by vehicle system warnings or other intervention systems.

The L1 hierarchy is called the driver assistance hierarchy, and is defined as: in a specific driving mode, a single driving assistance system controls the transverse or longitudinal driving action of the vehicle by acquiring the driving environment information of the vehicle, but a driver needs to be responsible for operating other dynamic driving tasks.

The L2 hierarchy is referred to as the partial automation hierarchy, and its definition is: in a specific driving mode, a plurality of driving assistance systems simultaneously control the transverse or longitudinal driving actions of the vehicle by acquiring the driving environment information of the vehicle, but a driver needs to be responsible for operating other dynamic driving tasks.

The L3 hierarchy is called a conditional automation hierarchy, and its definition is: under a specific driving mode, the system is responsible for executing all dynamic driving tasks of the vehicle, and a driver needs to timely respond to an intervention request provided by the system when a special condition occurs.

The L4 level is called a highly automated level, and its definition is: in a particular driving mode, the system is responsible for performing all dynamic driving tasks of the vehicle, even if the driver fails to respond to the intervention request made by the system when a particular situation occurs.

The L5 level is called full automation level and its definition is: the system is responsible for completing all-weather all-road-condition dynamic driving tasks and can be managed by drivers.

Referring to fig. 1, there is shown an unmanned vehicle 100 to which embodiments of the automated driving test method, automated driving test apparatus, of embodiments of the present invention may be applied.

As shown in fig. 1, the unmanned vehicle 100 may include a driving Control device 101, a vehicle body bus 102, an ECU (Electronic Control Unit) 103, an ECU 104, an ECU 105, a sensor 106, a sensor 107, a sensor 108, and an actuator 109, an actuator 110, and an actuator 111.

A driving control device (also referred to as an in-vehicle brain) 101 is responsible for overall intelligent control of the entire unmanned vehicle 100. The driving control device 101 may be a controller that is separately provided, such as a Programmable Logic Controller (PLC), a single chip microcomputer, an industrial controller, and the like; or the equipment consists of other electronic devices which have input/output ports and have the operation control function; but also a computer device installed with a vehicle driving control type application. The driving control device can analyze and process the data sent by each ECU and/or the data sent by each sensor received from the vehicle body bus 102, make a corresponding decision, and send an instruction corresponding to the decision to the vehicle body bus.

The body bus 102 may be a bus for connecting the driving control apparatus 101, the ECU 103, the ECU 104, the ECU 105, the sensor 106, the sensor 107, the sensor 108, and other devices of the unmanned vehicle 100, which are not shown. Since the high performance and reliability of a CAN (Controller area network) bus are widely accepted, a vehicle body bus commonly used in a motor vehicle is a CAN bus. Of course, it is understood that the body bus may be other types of buses.

The vehicle body bus 102 may transmit the instruction sent by the driving control device 101 to the ECU 103, the ECU 104, and the ECU 105, and the ECU 103, the ECU 104, and the ECU 105 analyze and process the instruction and send the instruction to the corresponding execution device for execution.

Sensors

106, 107, 108 include, but are not limited to, lidar, cameras, accelerometers, gyroscopes, magnetometers, ultrasonic radar, and the like.

The laser radar is a device for detecting and measuring distance of an object by using laser as a sensor commonly used in the field of unmanned driving, and the sensor is internally provided with a rotating structure and can send millions of light pulses to the environment every second and output point cloud data.

Cameras are generally used to take pictures of the surroundings of an unmanned vehicle and record the scene in which the vehicle is traveling.

The accelerometer is also called a gravity sensor, and the magnitude and the direction of the acceleration in the axial direction are obtained by measuring the stress condition of the component in a certain axial direction.

The gyroscope is also called a ground sensor, the output data of the gyroscope is the magnitude and direction of a certain axial upper angular velocity, a common three-axis gyroscope is used, the working principle of the three-axis gyroscope is that an included angle between a vertical axis of a gyroscope rotor in a three-dimensional coordinate system and equipment is measured, the angular velocity is calculated, and the motion state of an object in a three-dimensional space is judged through the included angle and the angular velocity.

Magnetometers, also called geomagnetic and magnetic sensors, can be used for testing the intensity and direction of magnetic field and for locating the orientation of equipment.

The working principle of the ultrasonic radar is that the distance is measured and calculated by the time difference between the time when the ultrasonic wave is sent out by the ultrasonic wave transmitting device and the time when the ultrasonic wave is received by the receiver. The two common ultrasonic radars are a reversing radar which is arranged on a front bumper and a rear bumper of an automobile and is used for measuring front and rear obstacles of the automobile, and the other ultrasonic radar which is arranged on the side face of the automobile and is used for measuring the distance between side obstacles can be applied to parking garage position detection and high-speed transverse assistance.

It should be noted that the automated driving test method provided by the embodiment of the present invention may be executed by the driving control device 101, and accordingly, the automated driving test apparatus is generally disposed in the driving control device 101, and the automated driving test provided by the embodiment of the present invention refers to a virtual simulation test, where an automated driving program used for the test is also a program running in the virtual test, and the automated driving program may be executed by other modules in the driving control device 101 in cooperation.

It should be understood that the numbers of unmanned vehicles, driving control devices, body buses, ECUs, actuators, and sensors in fig. 1 are merely illustrative. There may be any number of unmanned vehicles, driving control devices, body buses, ECUs, and sensors, as desired for implementation.

In consideration of economic cost and time, before an unmanned vehicle runs on a road for testing, a large number of simulation tests are often required to be performed on an automatic driving program, and such simulation tests require that scene data are prepared in advance, for example, a security officer drives a real vehicle to run on the road (a road testing process) and records corresponding scene data at the same time.

The scene data includes data collected by various sensors equipped on the real vehicle, such as video data, acceleration, angular velocity, three-dimensional point cloud, and the like.

In order to manage and classify a large amount of scene data and facilitate future testing of the automatic driving program, a description file can be configured for each collected scene data, and specific information related to the scene data, such as scene type, sensor type, data type and the like, can be clearly written in the description file.

Because the scene data collected by the real vehicle in actual running is very limited, in order to enrich the test diversity and supplement more scene data for testing the automatic driving program, a random generator can be adopted to simulate the real scene data to generate virtual scene data, the virtual scene data and the real scene data are fused, and derivation and expansion are carried out on single scene data, or part of the scene data can be selected from the real scene data to be replaced by the random generator, or the random generator can be used to adjust part of the real scene data.

As shown in fig. 2, the test architecture provided by the embodiment of the present invention includes a first cloud service 210 and a second cloud service 220, and the first cloud service 210 and the second cloud service 220 are associated with each other.

The first cloud service 210 and the second cloud service 220 are cloud services, the cloud services are required services obtained through a network in an on-demand and easily-extensible manner, and the cloud services are mostly used for achieving purposes of data access, operation and the like.

The first cloud service 210 is generally deployed in one network, so that the client and the first cloud service 210 are in the same network, such as a public network, also called a public cloud, and are responsible for scheduling and generating parameters for testing, and the second cloud service 220 is generally deployed in another network, such as a local area network, also called a private cloud, and is responsible for executing testing.

The first cloud service 210 includes a control system 211, a database 212, and an interface 213.

The second cloud service 220 includes a machine 221, a machine 222, and a machine 223, where a daemon 2210 runs in the machine 221, a daemon 2220 runs in the machine 222, and a daemon 2230 runs in the machine 223. The daemon process is a process running in the background and is used for executing specific system tasks. The daemon process is typically started at system boot time and runs until the system is shut down.

The control system 211 is used for controlling the overall process of the test, and is responsible for scheduling of the machine, and performing control of various preparation steps before the test starts, etc., the database 212 is used for storing information that the first cloud service 210 and the second cloud service 220 need to use for interaction, and the interface 213 is used for an external device to access the first cloud service 210.

The interface 213 of the first cloud service 210 is opened to the daemon 2210, the daemon 2220 and the daemon 2230 in the second cloud service 220 to provide specific interaction data for the daemons, the database 212 in the first cloud service 210 is connected with the control system 211, the control system 211 can access data stored in the database 212, the states of the

machines

221, 222 and 223 in the second cloud service 220 can be queried in the database 212, the database 212 in the first cloud service 210 is connected with the interface 213, the daemon 2210, the daemon 2220 and the daemon 2220 can access test detailed information in the database 212 through the interface 213, the daemon runs on each machine, interacts with data in the database 212, and executes specified programs and instructions on the machine for testing.

It should be understood that the number of interfaces of the first cloud service, the number of machines of the second cloud service in fig. 2 are merely illustrative. There may be any number of interfaces and machines, as the implementation requires.

Example one

Fig. 3 is a flowchart of an abnormality detection method for an automatic driving test according to an embodiment of the present invention, where the method is applicable to a case where an abnormality detection is performed on a test result of automatic driving, and the method may be executed by an abnormality detection device, where the abnormality detection device may be implemented by software and/or hardware, and may be configured in a computer device, for example, a server, a personal computer, and other computing devices, and the method specifically includes the following steps:

step 301, simulating the operation object to execute automatic driving on the virtual vehicle in the scene data so as to test the automatic driving program, and obtaining a test result.

Different operation objects, i.e., objects having specific business operation capabilities, may be set for different business fields, and for tests in the field of automated driving, the operation objects are objects having vehicle driving manipulation capabilities, may be represented as security officers or remote operation terminals, and so on.

In order to simulate the drive test process of a real vehicle, the internal and external parts simulating the real vehicle in the test platform construct a virtual vehicle which can execute virtual tests.

In a specific implementation, parameters for testing an automated driving program may be set in response to a request to test the automated driving program in the first cloud service; selecting a machine for testing the automatic driving program from the second cloud service; in the second cloud service, reading parameters from the first cloud service by a daemon process; and the daemon simulates an operation object and executes automatic driving on the virtual vehicle in the scene data according to the parameters so as to test the automatic driving program and obtain a test result.

A user logs in a first cloud service at a client (such as a browser), provides an automatic driving program for the first cloud service, and requests to test the automatic driving program.

In order to improve the testing efficiency, the first cloud service can meet the requirements of a plurality of testing tasks, that is, the first cloud service can process a plurality of testing tasks in the same client, and can also process testing tasks submitted successively in a plurality of clients. Specifically, a user of the client may submit a request for a test task to the first cloud service through the network, the first cloud service may respond to the request submitted by the user, store the request in the queue, and the control system in the first cloud service may read the request for testing the autopilot from the queue according to the priority of the request, analyze the request, and perform a series of test preparation operations according to the analyzed result, for example, compile a source code of the autopilot, select a test module that is associated with the request, select an initialization mode, set parameters for testing the autopilot, and the like.

The method comprises the steps of setting parameters for testing the automatic driving program, wherein the parameters comprise scene data acquired when a real vehicle runs on a road surface, and selecting the scene data for testing the automatic driving program according to the requirements of a user, for example, the user actively requires which specific scene data to test the automatic driving program, for example, the user requires to select the type of the scene data, such as an expressway, an urban road and the like, and for example, the user does not have the requirement, randomly selects the scene data, or automatically selects the scene data with higher test score.

The parameters set for testing the automatic driving program include not only scene data acquired when a real vehicle runs on a road surface, but also a place, a duration, a type of a sensor, a test frequency for the same scene data, and the like.

It should be noted that, when the first cloud service receives a large number of requests for testing the automatic driving program, if there is no priority limit, the requests stored in the queue are sequentially responded to, and this embodiment does not limit this.

Referring to fig. 2, the second cloud service 220 is used to test the autopilot in cooperation with the first cloud service 210, and the second cloud service 220 has a plurality of machines (i.e., devices with computing power) for testing the autopilot, and when the second cloud service 220 is in a normal working state, each machine has a daemon process running therein, and these

daemon processes

2210, 2220, 2230 can access the interface 213 exposed by the first cloud service 210 to obtain detailed information of the test and then perform a test operation of the autopilot.

Generally, each machine in the second cloud service executes a test at the same time, and therefore, in this embodiment, the first cloud service maintains a real-time state of each machine in the second cloud service, and selects a machine for testing an appropriate auto-driving program according to the state to test the auto-driving program.

In a specific implementation, the state of each machine in the second cloud service can be queried locally in the first cloud service, where the state includes idle and occupied, the idle indicates that the machine does not execute the test, and the occupied indicates that the machine has executed the test.

If the state of traversing a certain machine is idle, the automatic driving program can be determined to be tested in the machine, and meanwhile, the state of the machine is modified from idle to occupied, so that other tests are prevented from being executed and conflicts are avoided.

Of course, when receiving the message sent by the machine and completing the test, the state of the machine may be modified from occupied to idle, and the resource is released to wait for the next test.

It should be noted that there are many ways to select a machine for testing an automatic driving program, and this embodiment is not limited to this. In addition to the above example method, the machines for testing the automatic driving program may be selected by, for example, recording the operating state of each machine in the second cloud service into a list, where the list may update the latest state of the machine in real time, and when a request for testing the automatic driving program is received, determining the machine for performing the test by reading the state of the machine in the list; or, a flag bit is set for each machine in the second cloud service, specifically, when the machine is in a working state, the flag bit is set to 1, when the machine is in a fault, the flag bit is set to N, and when the machine is in a normal idle state, the flag bit is set to 0.

The first cloud service, upon selecting a machine, notifies a daemon process in the machine to perform a test on the autopilot, the daemon responding to the notification to request the autopilot and parameters suitable for testing the autopilot from the first cloud service.

In a specific implementation, referring to fig. 2, the first cloud service 210 has a database 212 in which parameters set for each test are stored, and an interface 213, and the daemon process in the second cloud service 220 can request parameters applied by the test thereof through the interface 213 exposed by the first cloud service 210.

After verifying the validity of the daemon process, the first cloud service 210 reads the parameters of the machine test automation program in which the daemon process is located in the database 212, and sends the parameters to the daemon process through the interface 213.

In the test process, the daemon process simulates an operation object to start an automatic driving mode, in the automatic driving mode, the daemon process simulates an input place of the operation object so that the automatic driving program generates a test route which passes through the place and is located in the scene data, and the daemon process simulates the operation object to execute starting operation so that the automatic driving program drives the virtual vehicle to run along the test route.

The daemon process is responsible for detecting the test state (completed and unfinished) of the automatic driving program, if the automatic driving program is not tested, all the places are set as non-driving, the daemon process judges whether the non-driving places remain or not, and the test is continued; if the testing of the autopilot program has been completed, a test result is generated for the autopilot program.

In the test process, the method further comprises the following steps: the daemon process determines whether the automatic driving mode is abnormal or not, if so, the daemon process simulation operation object switches the automatic driving mode to the manual driving mode, in the manual driving mode, the daemon process simulation operation object drives the virtual vehicle in the scene data until the automatic driving mode returns to normal, and when the automatic driving mode returns to normal, the daemon process simulation operation object switches the manual driving mode to the automatic driving mode. The daemon process records the detected states of the automatic driving mode in a test result, wherein the test result comprises normal and abnormal test conditions of the automatic driving mode; in addition, a first time point, a first scene, and first position data when switching from the automatic driving mode to the manual driving mode, and a second time point, a second scene, and second position data when switching from the manual driving mode to the automatic driving mode are recorded in the test result.

The daemon process simulates an operation object to test the automatic driving program in real scene data, so that the test process of the whole automatic driving program is closer to the real automatic driving drive test process, and the obtained test result is more real and reliable; furthermore, based on a more real and reliable test result, some problems existing in the automatic driving program in combination with real scene data and in the program can be tested in advance before the actual drive test, so that the automatic driving program can be adjusted in time according to the test result, and the problem that the automatic driving program is distorted in the test process in a purely virtual test environment can be solved.

After the test is finished, the test result is fed back to the test submitter in the form of a test report. The test result comprises all log information (time, place position, sensing data, sensor type, driving speed, test route, times of switching operation, time point when switching operation occurs, time ratio of manual driving mode to total test time length and the like) of the automatic driving program operated by the virtual vehicle, a description file for marking scene data type, performance information (execution efficiency of each module, system resource use conditions such as overall cpu/memory/IO and the like), and the like.

Step 302, query switching operation in the test result.

In the present embodiment, the switching operation means a complete operation of switching from the automated driving mode to the manual driving mode and switching from the manual driving mode to the automated driving mode.

Because the test results record log information such as the driving speed, the location, the number of times of the switching operation, the time point when the switching operation occurs, the time ratio of the manual driving mode to the total test duration, and the like, the test results can be screened based on the data recorded in the log information to inquire the switching operation, for example, the number of times of the switching operation in a plurality of test results is counted, for example, the time ratio of the manual driving mode to the total test duration is used as a screening condition, a large number of test results are screened to inquire the switching operation in the test results, and a plurality of test results with the switching operation are obtained.

Step 303, if the switching operation is the same, determining the switching operation as a target operation;

and in the plurality of test results with the switching operation, classifying the plurality of test results according to the types of the scene data according to the description file for marking the types of the scene data, so that the plurality of test results of the same automatic driving program under the same scene data are classified into the same class.

Because a plurality of switching operations may exist in one test result, the reason for each switching operation may be different, and whether the state of the automatic driving program is normal or abnormal may be determined according to the switching operations, it is necessary to determine whether the switching operations in a plurality of test results in the same category are the same or different, and count the same switching operations to further locate the reason for the same switching operation.

The same switching operation means that the switching operation occurs at the same timing among the plurality of test results, that is, at the same timing of switching from the automated driving mode to the manual driving mode and at the same timing of switching from the manual driving mode to the automated driving mode. The timing of the switching operation may be counted by a big data analysis method, for example, a clustering algorithm is used to identify the timing of the automatic driving mode switching to the manual driving mode and the timing of the manual driving mode switching to the automatic driving mode in each test result, so as to determine whether the timings of the switching operation are the same.

For the test results of the same category, whether the switching operation is the same or not can be determined according to the information such as the time ratio of the manual driving mode to the total test duration, the time point when the switching operation occurs, the location position and the like, and if the switching operation is the same, the switching operation is determined as the target operation. For example, a first time point when the automatic driving mode is switched to the manual driving mode in the test result is searched, a second time point when the manual driving mode is switched to the automatic driving mode is searched in the same test result, and if a time difference value between the first time point and the second time point is smaller than a preset threshold value, it is determined that the switching operation of the plurality of test results under the scene data is the same, and the switching operation is determined to be a target operation.

Specifically, a cluster analysis method may be further used to process a plurality of test results in the same scene data, count first scenes in the test results when the automatic driving mode is switched to the manual driving mode, count second scenes in the test results when the manual driving mode is switched to the automatic driving mode, classify all the first scenes, classify all the second scenes, read first position data of the plurality of first scenes, read second position data of the plurality of second scenes, compare the plurality of first position data, calculate a first position difference, compare the plurality of second position data, calculate a second position difference, and determine that a switching operation occurring in the first scene is a target operation if the first position difference and the second position difference are both smaller than a preset threshold.

Step 304, determining the state of the automatic driving program in the scene data where the target operation occurs;

in this embodiment, the target operation may be analyzed, that is, the target operation reasonableness of the automatic driving program in the scene data may be analyzed according to the frequency, time interval, confidence interval, and other data of the target operation, so as to determine whether the state of the automatic driving program in the scene data where the target operation occurs is normal or abnormal.

For example, the probability of the target operation occurring in the multiple test results in the same scene data is counted, the probability is compared with a preset threshold, if the probability is smaller than the preset threshold, the state of the automatic driving program in the scene data where the target operation occurs is determined to be a normal state, and if the probability is greater than or equal to the preset threshold, the state of the automatic driving program in the scene data where the target operation occurs is determined to be an abnormal state. Or inputting a plurality of test results under the same scene data into the clustering model for analysis and detection, and determining whether the state of the automatic driving program in the scene data with the target operation is a normal state or an abnormal state. It should be noted that the present embodiment does not set any limit to the manner for determining the state of the automatic driving program in the scene data in which the target operation occurs.

And 305, if the state is the abnormal state, positioning the factors of the target operation in the abnormal state according to the test result.

When the state of the automatic driving program in the scene data of the target operation is determined to be an abnormal state, positioning the factors of the target operation in the abnormal state according to the test result, namely, checking the data in the test result, and analyzing the reason of the switching operation, such as abnormal data detection on the position and the driving speed of a place; or performing abnormal investigation on the working state of the sensor in the test result and the raw sensing data of the sensor to locate the factor of the target operation in the abnormal state.

In an abnormal positioning mode, position data of a test route and position data of a target route can be extracted from a test result in the same scene data, wherein the target route is a route of a real vehicle when the real vehicle runs on a road surface and scene data is collected; calculating a difference between the test route and the target route; if the difference exceeds a predetermined deviation value, the factor for locating the occurrence of the abnormal target operation may be an abnormality in the path planning module of the autopilot.

In another abnormal positioning manner, abnormal investigation may be performed on the working state of the sensor and the raw sensing data of the sensor in the test result, for example, the sensor used by the autopilot before the target operation occurs is queried from the test result and is a laser radar, a camera, an accelerometer, a magnetometer, a gyroscope and an ultrasonic radar, if the magnetometer fails, the working state of the magnetometer is set to be disabled, the type of the sensor of the autopilot is selected again, only the laser radar, the camera, the accelerometer, the gyroscope and the ultrasonic radar are selected as the sensors participating in the test, and the simulation operation object performs autopilot on the virtual vehicle in the scene data to test the autopilot to obtain a new test result; if the target operation in the abnormal state is not inquired in the new test result, determining that the factor of the target operation in the abnormal state is the failure of the sensor; or, if the laser radar is found to be in fault, replacing the laser radar with the millimeter wave radar, and re-testing the automatic driving program to obtain a new test result.

By the method, the reason of the sensor fault is eliminated, and the sensing data in the test result is abnormally positioned.

Specifically, the abnormal positioning of the sensing data in the test result includes:

data recorded by the sensor before all target operations are inquired from the test results as raw sensing data, so that reference sensing data is calculated based on the raw sensing data.

Calculating an average value of the original sensing data for each time point to form reference sensing data; alternatively, the covariance of the raw sensing data is calculated for each time point as reference sensing data.

It is identified that the target sensed data is in a deviated state with respect to the reference sensed data.

The target sensing data is original sensing data corresponding to target operation in an abnormal state.

Calculating a difference value between the target sensing data and the reference sensing data for each time point as a single point difference; calculating the average value of all single-point differences as an integral difference; and if the overall difference is larger than a preset difference threshold value, determining that the target sensing data is in a deviation state relative to the reference sensing data. Of course, the deviation state may also be identified by comparing the covariance, the third moment of the target sensed data and the reference sensed data. In the present embodiment, the method of identifying the deviation state is not limited at all.

The sensor in the deviated state is prohibited from operating.

For example, if the sensor in the deviation state is a gyroscope, the operating state of the gyroscope is set to be disabled, and other sensors (such as magnetometers) may be used instead of the gyroscope, or simulation data may be used instead of sensing data of the magnetometers.

Under the condition that the sensor in the deviation state is forbidden to operate, the simulation operation object carries out automatic driving on the virtual vehicle in the scene data so as to test an automatic driving program and obtain a new test result, and the method specifically comprises the following steps:

first, the daemon simulates an operation object to start an automatic driving mode to run an automatic driving program.

The daemon process simulates an operation object to execute starting operation on the virtual vehicle, monitors the running state of the virtual vehicle at the moment, and activates the automatic driving mode of the virtual vehicle when the running state of the virtual vehicle is kept stable so as to run an automatic driving program in real test scene data.

Secondly, in the automatic driving mode, the daemon process simulates an operation object input place so that the automatic driving program generates a test route which passes through the place and is located in the scene data.

When the automatic driving mode of the virtual vehicle is in an activated state, the daemon process reads parameters from the database of the first cloud service, the parameters comprise places, and the daemon process can sequentially read preset places in the scene data so that the automatic driving program to be tested can automatically plan a driving path according to the places to generate a test route which passes through the places and is located in the scene data. The daemon process simulates an operation object input place, and can simulate the operation of the operation object on a vehicle provided with an automatic driving system and a place to be passed by in a set driving route in the process of driving test more truly.

In one method for setting the location, the locations in the parameters may be arranged in a queue, etc., and the daemon process reads the locations in the queue in sequence, and forms a test route between every two locations, where the first location (i.e., the location where the virtual vehicle is initially located in the scene data) is the same as the last location, so that multiple test routes may form a closed and trained test route, and specifically, in the automatic driving mode, the daemon process reads the data in the queue to determine whether there are remaining locations that are not driven (i.e., the location where the daemon process is set as the destination), and if so, the daemon process reads the next location that is not driven, simulates the input location of the operation object, sets the location as the destination of the next automatic driving, so that the automatic driving program generates the route from the current location to the next location that is not driven, And the test route is located in the scene data; if not, the daemon process determines to finish generating the test routes which are positioned in the scene data and circulate in all places, so that the test of the automatic driving program on the closed and trained test routes is finished once.

For example, as shown in fig. 4, the first place and the last place are the places a, the daemon drives the virtual vehicle to travel from the place a, the virtual vehicle continues to travel along the places B, C and D, when the virtual vehicle reaches the place E, the daemon determines that there are the remaining un-traveled places F, G and H, the daemon reads the places F, G and H in turn, drives the virtual vehicle to continue to travel from the place E through the places F, G and H, at which time, the autopilot has generated a test route from the places a-B-C-D-E-F-G-H until the virtual vehicle returns to the place a again, and the autopilot has generated a test route from the places a-B-C-D-E-F-G-H-a And determining the test route of the point, namely determining to finish generating the test route which is positioned in the scene data of the test and circulates at all the places. The place at the head position and the place at the tail position are set to be the same place, so that the scenes for testing the automatic driving program can be connected end to end and can be played circularly, and long-time software simulation operation is supported. Moreover, as the collected real scene data is limited, in order to meet the test requirement, the scene data of the test automatic driving program is played circularly, so that the robustness of the automatic driving program in single scene data can be tested, and the cost for collecting the scene data can be reduced.

Further, after completing the test of the automatic driving program on the closed and trained test route once, the daemon judges whether the test completion condition is met, thereby judging whether the test of the automatic driving program is completed. For example, the daemon process may determine whether to complete the test of the automatic driving program by reading a target duration and a target cycle number in preset parameters, that is, the duration of the test exceeds the target duration, or the number of tests in a closed and trained test route exceeds the target training number, that is, the test is considered to be completed, otherwise, the test is considered to be not completed.

And if the test automatic driving program is not finished, setting all the places as non-driving, and entering a test route of the next cycle. In the test route of the next cycle, the sensor type in the scene data may be changed to test the suitability of the sensor to the autopilot, for example, lidar, cameras, accelerometers and gyroscopes may be used in the last test, and lidar, ultrasonic radar, accelerometers, gyroscopes and magnetometers may be used in the next test cycle.

If the testing of the autopilot program has been completed, a test result is generated for the autopilot program.

In addition, in the automatic driving mode, the daemon process simulates an operation object to execute starting operation so that the automatic driving program drives the virtual vehicle to run along the test route, and in the test process, the daemon process detects the current test state and compares the current test state with the state of the real vehicle running on the road surface to collect scene data, so that whether the automatic driving mode is abnormal or not is determined, namely the automatic driving mode is normal if the automatic driving mode is consistent with the real vehicle running on the road surface, and the automatic driving mode is abnormal if the automatic driving mode is inconsistent with the real vehicle running on the road surface. If the automatic driving mode is abnormal, the daemon process simulates the operation object to switch the automatic driving mode to a manual driving mode, and under the manual driving mode, the daemon process simulates the operation object to drive the virtual vehicle in the scene data until the automatic driving mode returns to normal. If the automatic driving mode is normal, the daemon process simulates an operation object to switch the manual driving mode to the automatic driving mode.

The daemon process can determine that the automatic driving mode is abnormal by inquiring the output result in the automatic driving test process in real time, such as the abnormal pose data of the virtual vehicle, the abnormal output data of the sensor and the like.

It should be noted that both the normal and abnormal test conditions of the automatic driving mode in this embodiment are recorded in the test results.

And if the target operation in the abnormal state is not inquired in the new test result, determining that the factor of the target operation in the abnormal state is not adaptive between the automatic driving program and the sensor.

The method comprises the steps of executing automatic driving on a virtual vehicle in scene data by simulating an operation object to test an automatic driving program to obtain a test result, wherein the scene data is acquired by a real vehicle when the real vehicle runs on a road surface; inquiring switching operation in the test result, wherein the switching operation represents switching from the automatic driving mode to the manual driving mode and switching from the manual driving mode to the automatic driving mode; if the switching operation is the same, determining the switching operation as a target operation; determining a state of an automatic driving program in scene data where a target operation occurs; and if the state is an abnormal state, positioning the factors of the target operation in the abnormal state according to the test result. The method comprises the steps of obtaining a test result of an automatic driving program in a real scene, referring to a plurality of index data in the test result to automatically position the state of target operation, and confirming that the data volume according to the state is large and the types are diversified, so that the state of the target operation is judged reliably and accurately, whether the test result of the automatic driving program is abnormal or not is judged according to the state of the target operation, an automatic abnormal detection process is realized to a certain extent, factors of the target operation in the abnormal state are further positioned according to the abnormal state, the problems of false detection and missed detection in pure manual detection of the automatic driving test result can be avoided to a certain extent, a large amount of test results can be efficiently detected in the abnormal state, the abnormal detection efficiency of the automatic driving test can be improved, and the detection accuracy is ensured.

Example two

Fig. 5 is a flowchart of an anomaly detection method for an automatic driving test according to a second embodiment of the present invention, where the present embodiment is based on the foregoing embodiment, and supplements and refines the content of the anomaly detection method for the automatic driving test, and the method specifically includes the following steps:

step 501, simulating an operation object to execute automatic driving on the virtual vehicle in the scene data so as to test an automatic driving program, and obtaining a test result.

In practical applications, in the process of testing a vehicle equipped with an automatic driving system on a road, in order to avoid an emergency, a safety staff is usually equipped, and the safety staff can execute a dynamic driving task on the vehicle, wherein the dynamic driving task includes controlling the speed, steering, lane changing, lights, horns and the like of the vehicle, and also includes actively avoiding obstacles, planning an actual driving path, making a decision to respond to a driving environment change, monitoring the driving environment in real time, preparing in advance and the like.

Taking an operation object as an example of a security officer, a daemon acts as the role of the security officer, tests are operated according to parameters configured by a first cloud service, for example, when the test is started, the daemon simulates the security officer to start a virtual vehicle, performs a dynamic driving task on the virtual vehicle in scene data, can perform destination changing operation, steering operation, lane changing operation and the like by reading a place in the parameters, can keep the virtual vehicle running at a stable speed by reading the speed in the parameters, can start a take-over command and a command for starting an automatic driving mode in a specific scene data area, is responsible for detecting whether the test state of an automatic driving program is finished or unfinished, and simultaneously, the daemon simulates the security officer to determine whether the automatic driving mode is abnormal or not, if the automatic driving mode is abnormal, the daemon simulates the security officer to switch the automatic driving mode to a manual driving mode, in the manual driving mode, the daemon simulates a security worker to drive the virtual vehicle in the scene data until the automatic driving mode is recovered to be normal, and when the automatic driving mode is recovered to be normal, the daemon simulates the security worker to switch the manual driving mode to the automatic driving mode.

Taking an operation object as a remote operation terminal as an example, a daemon process simulates the remote operation terminal to acquire parameters required by a test from a first cloud service, the daemon process issues a remote control command to a virtual vehicle according to different configuration information in the parameters and the simulated remote operation terminal so that the virtual vehicle executes automatic driving in scene data, and the daemon process simulates the remote operation terminal to remotely monitor and intervene in an automatic driving process of the virtual vehicle so as to test an automatic driving program.

In summary, the daemon simulates the operation object, performs automatic driving on the virtual vehicle in the scene data according to the parameters to test the automatic driving program, and stores and records all information related to the test, such as all processes, intermediate generation data, output results, and the like of the test to obtain the test result.

Step 502, query switching operation in the test result.

The switching operation can be inquired in the test result by presetting an inquiry condition; the test report can also be associated with the script file by writing the script file with the screening information so as to inquire the switching operation in the test result.

Step 503, inquiring the first position of the automatic driving mode to the manual driving mode during the switching operation.

In this example, a script file may be written with the automatic driving mode switched to the manual driving mode as script information, and the position of the switching operation may be queried from a large number of test results as the first position.

Step 504, a second position at which the manual driving mode is switched to the automatic driving mode during the switching operation is queried.

The first position may be referred to inquire of the second position at which the manual driving mode is switched to the automatic driving mode in the switching operation. Or searching the position and the time of the place in the test result, generating a position-time list, reading the time point when the switching operation occurs in the test result, corresponding the time point to the position-time list, and inquiring the position when the manual driving mode is switched to the automatic driving mode in the test result to be used as the first position.

And 505, clustering the first positions to obtain a plurality of first clusters.

In this embodiment, for the plurality of first locations, the first locations may be clustered by a partition method, a hierarchical method, a density-based method, a grid-based method, a model-based method, or the like, so as to form a plurality of first clusters, where each first cluster includes one or more first locations.

Taking K-Means clustering as an example, firstly, selecting a plurality of first positions, randomly initializing respective central points of the positions, wherein the central point is a position with the same vector length as each position point; secondly, calculating the distance from each position point to the central point, and dividing the position point into which class when the position point is closest to which central point; thirdly, calculating the central point in each class as a new central point; and repeating the steps until the change degree of each type of center converges to the position after each iteration, and finally obtaining a plurality of first clusters related to the first position.

And step 506, clustering the second positions to obtain a plurality of second clusters.

In this embodiment, for the plurality of second locations, the second locations may be clustered by a partition method, a hierarchical method, a density-based method, a grid-based method, a model-based method, or the like, so as to form a plurality of second clusters, and each second cluster includes one or more second locations.

Taking the DBSCAN cluster as an example, clustering the second location with the DBSCAN cluster, determining a radius r and a minimum location point, starting from any second location point which is not visited, centering on the second location point, determining whether the number of other second location points contained in a circle with the radius r being greater than or equal to the minimum location point, if so, marking the second location point as a central point, otherwise, marking the second location point as a noise point, repeating the above steps, if a noise point exists in a circle with the radius being equal to a certain central point, marking the noise point as an edge point, otherwise, still being a noise point, repeating the previous steps until all the second location points are visited, and finally obtaining the second cluster.

And 507, if the switching operation belongs to the same first cluster and the same second cluster, determining that the switching operation is the same and belongs to the target operation.

And combining the analysis of the switching operation in the test result, if the switching operation from the automatic driving mode to the manual driving mode and the switching operation from the manual driving mode to the automatic driving mode belong to the same first cluster and the same second cluster, determining that the switching operation is the same and belongs to the target operation.

In the embodiment, the target operation is screened by the clustering method, so that the fault tolerance can be increased, and the reasonable switching operation is selected as the target operation.

Step 508, determining the state of the autonomous driving program in the scene data where the target operation occurs.

In a specific implementation, if the target operation of the automatic driving program in the scene data is abnormal, the target operation is abnormal in the scene data, and the frequency of the target operation can be counted in a plurality of test results through a large number of counted target operations;

if the frequency is less than or equal to a preset first frequency threshold value, determining that the state of the automatic driving program in the scene data with the target operation is an abnormal state;

if the frequency is greater than or equal to a preset second frequency threshold value, determining that the state of the automatic driving program in the scene data with the target operation is a normal state;

wherein the first frequency threshold is less than the second frequency threshold.

The specific implementation mode selects the first frequency threshold and the second frequency threshold as two conditions for screening test results, so that the confidence of test result screening can be enhanced, the test results which are larger than the first frequency threshold and smaller than the second frequency threshold can be regarded as results with insufficient confidence to be screened out and sent to a tester for manual detection, the test results with high confidence are sent to an automatic anomaly detection method, the test results of automatic driving are analyzed and detected by using the two modes of manual detection and automatic detection, the problems of false detection and missed detection in the automatic driving test results of pure manual detection can be avoided to a certain extent, a large amount of test results can be efficiently detected in an abnormal state, the anomaly detection efficiency of automatic driving test can be improved, and the detection accuracy is ensured.

And 509, if the state is a normal state, determining that the automatic driving program is not matched with the scene data.

In this embodiment, when it is determined that the state of the automatic driving program in the scene data in which the target operation occurs is a normal state, it is determined that the automatic driving program does not fit the scene data, and the automatic driving program is replaced with other scene data to perform retesting.

EXAMPLE III

Fig. 6 is a schematic structural diagram of an abnormality detection apparatus for an automatic driving test according to a third embodiment of the present invention, where the apparatus may specifically include the following modules:

the data acquisition module 601 is used for simulating an operation object to execute automatic driving on the virtual vehicle in scene data so as to test an automatic driving program and obtain a test result, wherein the scene data is acquired when a real vehicle runs on a road surface;

a query module 602, configured to query the test result for a switching operation, where the switching operation indicates switching from an automatic driving mode to a manual driving mode and switching from the manual driving mode to the automatic driving mode;

a target operation determining module 603, configured to determine, if the switching operations are the same, that the switching operation is a target operation;

a state determination module 604 for determining a state of the autonomous driving program in scene data where the target operation occurs;

an exception positioning module 605, configured to, if the state is an exception state, position, according to the test result, a factor of the target operation that occurs in the exception state.

In one embodiment of the present invention, the abnormality detection apparatus further includes:

and the normal positioning module is used for determining that the automatic driving program is not matched with the scene data if the state is a normal state.

In an embodiment of the present invention, the data obtaining module 601 includes:

the test request submodule is used for responding to a request for testing an automatic driving program in a first cloud service and setting parameters for testing the automatic driving program, wherein the parameters comprise scene data acquired when a real vehicle runs on a road surface;

the machine selection submodule is used for selecting a machine for testing the automatic driving program from a second cloud service, and a daemon process is operated in the machine;

a parameter reading submodule, configured to read, in the second cloud service, the parameter from the first cloud service by a daemon process in the machine;

and the automatic driving test submodule is used for simulating an operation object by the daemon process and executing automatic driving on the virtual vehicle in the scene data according to the parameters so as to test the automatic driving program.

In one embodiment of the present invention, the target operation determining module 603 comprises:

the first inquiring submodule is used for inquiring a first position of the switching operation from the automatic driving mode to the manual driving mode;

the second inquiry submodule is used for inquiring a second position where the manual driving mode is switched to the automatic driving mode in the switching operation;

a first clustering submodule, configured to cluster the first location to obtain a plurality of first clusters;

a second clustering submodule, configured to cluster the second location to obtain a plurality of second clusters;

and the target operation determining submodule is used for determining that the switching operations are the same and belong to the target operation if the switching operations belong to the same first cluster and the same second cluster.

In one embodiment of the present invention, the status determination module 604 comprises:

a frequency statistics submodule for counting a frequency at which the target operation occurs among a plurality of the test results;

an abnormal state determination submodule, configured to determine that a state of the automatic driving program in scene data where the target operation occurs is an abnormal state if the frequency is less than or equal to a preset first frequency threshold;

a normal state determination submodule, configured to determine that a state of the automatic driving program in the scene data where the target operation occurs is a normal state if the frequency is greater than or equal to a preset second frequency threshold; wherein the first frequency threshold is less than the second frequency threshold.

In one embodiment of the invention, the anomaly locating module 605 includes:

a sensor query sub-module for querying from the test results for sensors used by the autopilot before the target operation occurred;

a sensor disabling submodule for disabling the operation of the sensor if the sensor fails;

the first condition testing submodule is used for simulating an operation object to execute automatic driving on the virtual vehicle in the scene data under the condition that the sensor is forbidden to operate so as to test the automatic driving program and obtain a new testing result;

and the sensor fault determining submodule is used for determining that the factor of the target operation in the abnormal state is the sensor fault if the target operation in the abnormal state is not inquired in the new test result.

The sensing data query submodule is used for querying data recorded by the sensor before all the target operations from the test result as original sensing data if the sensor does not have a fault;

the reference sensing data calculation submodule is used for calculating reference sensing data based on the original sensing data;

the deviation identification submodule is used for identifying that target sensing data are in a deviation state relative to the reference sensing data, and the target sensing data are original sensing data corresponding to target operation in the abnormal state;

an offset sensor disabling submodule for disabling operation of a sensor in the offset state;

the second condition testing submodule is used for simulating an operation object to execute automatic driving on the virtual vehicle in the scene data under the condition that the sensor in the deviation state is forbidden to operate so as to test the automatic driving program and obtain a new testing result;

and the adaptation determining submodule is used for determining that the factor of the target operation in the abnormal state is not adapted between the automatic driving program and the sensor if the target operation in the abnormal state is not inquired in the new test result.

In an embodiment of the invention, the reference sensing data calculation sub-module further includes:

and the reference sensing data calculation unit is used for calculating the average value of the original sensing data aiming at each time point so as to form reference sensing data.

In one embodiment of the present invention, the deviation identifying submodule further comprises:

a difference calculation unit configured to calculate a difference between the target sensing data and the reference sensing data for each time point as a single-point difference;

an average value calculating unit for calculating an average value of all the single point differences as an overall difference;

a deviation state determination unit, configured to determine that the target sensing data is in a deviation state with respect to the reference sensing data if the overall difference is greater than a preset difference threshold.

The anomaly detection device provided by the embodiment of the invention can execute the anomaly detection method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.

Example four

Fig. 7 is a schematic structural diagram of a computer device according to a fourth embodiment of the present invention. As shown in fig. 7, the computer apparatus includes a processor 700, a memory 701, a communication module 702, an input device 703, and an output device 704; the number of the processors 700 in the computer device may be one or more, and one processor 700 is taken as an example in fig. 7; the processor 700, the memory 701, the communication module 702, the input device 703 and the output device 704 in the computer apparatus may be connected by a bus or other means, and fig. 7 illustrates an example of connection by a bus.

The memory 701 may be used as a computer-readable storage medium for storing software programs, computer-executable programs, and modules, such as modules corresponding to the automatic driving test method in the present embodiment (for example, a data acquisition module 601, a query module 602, a target operation determination module 603, a state determination module 604, and an abnormality localization module 605 in the abnormality detection apparatus for the automatic driving test shown in fig. 6). The processor 700 executes various functional applications and data processing of the computer device by executing software programs, instructions and modules stored in the memory 701, that is, the automatic driving test method described above is implemented.

The memory 701 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of the computer device, and the like. Further, the memory 701 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, memory 701 may further include memory located remotely from processor 700, which may be connected to a computer device through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

And the communication module 702 is used for establishing connection with the display screen and realizing data interaction with the display screen.

The input device 703 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of a computer apparatus, a camera for acquiring images and a sound pickup apparatus for acquiring audio data.

The output device 704 may include an audio device such as a speaker.

It should be noted that the specific composition of the input device 703 and the output device 704 may be set according to actual situations.

The processor 700 executes various functional applications and data processing of the device by running software programs, instructions and modules stored in the memory 701, that is, implements the above-described connection node control method of the electronic whiteboard.

The computer device provided by the embodiment of the invention can execute the automatic driving test method provided by any embodiment of the invention, and has corresponding functions and beneficial effects.

EXAMPLE five

An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements an abnormality detection method for an automatic driving test, and the method includes:

simulating an operation object to execute automatic driving on the virtual vehicle in scene data so as to test an automatic driving program, and obtaining a test result, wherein the scene data is acquired when a real vehicle runs on a road surface;

Of course, the computer readable storage medium provided by the embodiment of the present invention is not limited to the method operations described above, and may also perform related operations in the abnormality detection method for the autopilot test provided by any embodiment of the present invention.

From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.

It should be noted that, in the embodiment of the abnormality detection apparatus for automated driving test, the included units and modules are only divided according to the functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.

It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. An abnormality detection method for an automatic driving test, characterized by comprising:

inquiring switching operation in the test result, wherein the switching operation represents complete operation of switching from an automatic driving mode to a manual driving mode and switching from the manual driving mode to the automatic driving mode;

2. The method of claim 1, wherein the simulation operation object performs automatic driving on the virtual vehicle in scene data acquired by the real vehicle while traveling on the road surface to test an automatic driving program to obtain a test result, comprising:

in a first cloud service, responding to a request for testing an automatic driving program, and setting parameters for testing the automatic driving program, wherein the parameters comprise scene data acquired when a real vehicle runs on a road surface;

selecting a machine for testing the automatic driving program from a second cloud service, wherein a daemon process is operated in the machine;

in the second cloud service, reading, by a daemon process in the machine, the parameter from the first cloud service;

and the daemon simulates an operation object and executes automatic driving on the virtual vehicle in the scene data according to the parameters so as to test the automatic driving program.

3. The method according to claim 1, wherein the determining the handover operation as a target operation if the handover operations are the same comprises:

inquiring a first position where the automatic driving mode is switched to the manual driving mode in the switching operation;

inquiring a second position where the manual driving mode is switched to the automatic driving mode in the switching operation;

clustering the first location to obtain a plurality of first clusters;

clustering the second location to obtain a plurality of second clusters;

and if the switching operation belongs to the same first cluster and the same second cluster, determining that the switching operation is the same and belongs to the target operation.

4. The method of claim 1, wherein the determining the state of the autonomous driving program in the scene data in which the target operation occurred comprises:

counting a frequency at which the target operation occurs among the plurality of test results;

if the frequency is less than or equal to a preset first frequency threshold value, determining that the state of the automatic driving program in the scene data where the target operation occurs is an abnormal state;

if the frequency is greater than or equal to a preset second frequency threshold value, determining that the state of the automatic driving program in the scene data where the target operation occurs is a normal state;

5. The method of claim 1, wherein if the status is abnormal, locating the factor of the target operation in the abnormal status according to the test result comprises:

querying, from the test results, sensors used by the autopilot before the target operation occurred;

if the sensor fails, the sensor is prohibited to operate;

under the condition that the sensor is forbidden to operate, simulating an operation object to carry out automatic driving on the virtual vehicle in the scene data so as to test the automatic driving program and obtain a new test result;

and if the target operation in the abnormal state is not inquired in the new test result, determining that the factor of the target operation in the abnormal state is the sensor failure.

6. The method according to any one of claims 1 to 5, wherein if the state is an abnormal state, locating a factor of occurrence of a target operation in the abnormal state according to the test result comprises:

if the sensor does not have a fault, inquiring data recorded by the sensor before all the target operations from the test result to be used as original sensing data;

calculating reference sensing data based on the raw sensing data;

identifying that target sensing data is in a deviation state relative to the reference sensing data, wherein the target sensing data is original sensing data corresponding to target operation in the abnormal state;

inhibiting operation of the sensor in the deviated state;

under the condition that the sensor in the deviation state is forbidden to operate, simulating an operation object to carry out automatic driving on the virtual vehicle in the scene data so as to test the automatic driving program and obtain a new test result;

and if the target operation in the abnormal state is not inquired in the new test result, determining that the factor of the target operation in the abnormal state is not adapted between the automatic driving program and the sensor.

7. The method of claim 6,

the calculating reference sensing data based on the raw sensing data includes:

calculating an average value of the raw sensing data for each time point to constitute reference sensing data;

the identifying that the target sensory data is in a deviated state relative to the reference sensory data includes:

calculating a difference between the target sensing data and the reference sensing data for each time point as a single point difference;

calculating the average value of all the single-point differences as an overall difference;

and if the overall difference is larger than a preset difference threshold value, determining that the target sensing data is in a deviation state relative to the reference sensing data.

8. The method of any one of claims 1-5, further comprising:

and if the state is a normal state, determining that the automatic driving program is not matched with the scene data.

9. An abnormality detection device for an automatic driving test, characterized by comprising:

the data acquisition module is used for simulating an operation object to execute automatic driving on the virtual vehicle in scene data so as to test an automatic driving program and obtain a test result, wherein the scene data is acquired when a real vehicle runs on a road surface;

the query module is used for querying switching operation in the test result, wherein the switching operation represents complete operation of switching from an automatic driving mode to a manual driving mode and switching from the manual driving mode to the automatic driving mode;

the state determining module is used for determining the state of the automatic driving program in scene data of the target operation according to the target operation;

10. A computer device, characterized in that the computer device comprises:

one or more processors;

a storage device for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement the anomaly detection method of any one of claims 1-8.

11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the anomaly detection method according to any one of claims 1 to 8.