CN111159833B - Method and device for evaluating unmanned vehicle algorithm - Google Patents
Method and device for evaluating unmanned vehicle algorithm Download PDFInfo
- Publication number
- CN111159833B CN111159833B CN201811228268.5A CN201811228268A CN111159833B CN 111159833 B CN111159833 B CN 111159833B CN 201811228268 A CN201811228268 A CN 201811228268A CN 111159833 B CN111159833 B CN 111159833B
- Authority
- CN
- China
- Prior art keywords
- real
- data
- unmanned vehicle
- simulation
- planning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004422 calculation algorithm Methods 0.000 title claims abstract description 94
- 238000000034 method Methods 0.000 title claims abstract description 31
- 238000004088 simulation Methods 0.000 claims abstract description 119
- 238000011156 evaluation Methods 0.000 claims abstract description 81
- 230000001133 acceleration Effects 0.000 claims description 54
- 238000004590 computer program Methods 0.000 claims description 4
- 238000013459 approach Methods 0.000 abstract 1
- 230000015654 memory Effects 0.000 description 21
- 238000004891 communication Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000003491 array Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000010485 coping Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The embodiment of the invention provides an evaluation method and device of an unmanned vehicle algorithm, wherein the method comprises the following steps: setting a simulation environment, so that the simulation environment is consistent with the real environment when a driver drives a real host; operating the unmanned vehicle algorithm in the simulation environment to obtain simulation operation data; and comparing the simulation operation data with the real operation data when the driver drives the real host, and evaluating the unmanned vehicle algorithm by adopting the comparison result. The method provided by the embodiment of the invention can evaluate the degree that the unmanned vehicle algorithm approaches to the driving level of the real driver.
Description
Technical Field
The present invention relates to the field of unmanned vehicles, and in particular, to a method, apparatus, and device for evaluating an algorithm of an unmanned vehicle, and a computer readable storage medium.
Background
Currently, aiming at the evaluation of an unmanned vehicle algorithm, the main implementation mode is to run the unmanned vehicle in a simulation system and detect whether the unmanned vehicle violates traffic rules during the simulation running. Existing assessment methods cannot quantitatively assess how close an unmanned vehicle algorithm is to a skilled driver.
Disclosure of Invention
The embodiment of the invention provides an evaluation method and device of an unmanned vehicle algorithm, which are used for at least solving the technical problems in the prior art.
In a first aspect, an embodiment of the present invention provides a method for evaluating an unmanned vehicle algorithm, including:
setting a simulation environment, so that the simulation environment is consistent with the real environment when a driver drives a real host;
operating the unmanned vehicle algorithm in the simulation environment to obtain simulation operation data;
And comparing the simulation operation data with the real operation data when the driver drives the real host, and evaluating the unmanned vehicle algorithm by adopting the comparison result.
In one embodiment, the simulation environment includes: simulating a scene and simulating a position of a host vehicle;
the setting simulation environment comprises the following steps:
Setting the simulation scene to be the same as the real scene acquired when the driver drives the real host;
The simulated host vehicle position is set to be the same as a real host vehicle position when the driver drives a real host vehicle.
In one embodiment, the comparing the simulated operation data with the actual operation data when the driver drives the actual host vehicle, and evaluating the unmanned vehicle algorithm by using the comparison result includes:
And comparing at least one of planning track data, planning speed data and planning acceleration data in the simulation operation data with corresponding items in the real operation data, and evaluating the unmanned vehicle algorithm by adopting a comparison result.
In one embodiment, the comparing the planned trajectory data in the simulation running data with the corresponding items in the real running data, and evaluating the unmanned vehicle algorithm by using the comparison result includes:
aiming at more than one simulated host vehicle position, acquiring planning track data of the unmanned vehicle in each simulated host vehicle position and real track data of the real host vehicle in a corresponding real host vehicle position, and comparing the planning track data with the real track data;
and carrying out weighted summation on the comparison results to obtain an evaluation value aiming at planning track data, and evaluating the unmanned vehicle algorithm by adopting the evaluation value.
In one embodiment, the comparing the planned acceleration data in the simulation running data with the corresponding items in the real running data, and evaluating the unmanned vehicle algorithm by using the comparison result includes:
Aiming at more than one simulated host vehicle position, acquiring planning speed data of the unmanned vehicle at each simulated host vehicle position and real speed data of a real host vehicle at a corresponding real host vehicle position, and comparing the planning speed data with the real speed data;
and carrying out weighted summation on the comparison results to obtain an evaluation value aiming at planning speed data, and evaluating the unmanned vehicle algorithm by adopting the evaluation value.
In one embodiment, the comparing the planned acceleration data in the simulation running data with the corresponding items in the real running data, and evaluating the unmanned vehicle algorithm by using the comparison result includes:
Aiming at more than one simulated host vehicle position, acquiring planning acceleration data of the unmanned vehicle at each simulated host vehicle position and real acceleration data of a real host vehicle at a corresponding real host vehicle position, and comparing the planning acceleration data with the real acceleration data;
And carrying out weighted summation on the comparison results to obtain an evaluation value aiming at the planning acceleration data, and evaluating the unmanned vehicle algorithm by adopting the evaluation value.
In a second aspect, an embodiment of the present invention provides an evaluation device for an unmanned vehicle algorithm, including:
the simulation environment setting module is used for setting a simulation environment so that the simulation environment is consistent with the real environment when a driver drives a real host;
The simulation running module is used for running the unmanned vehicle algorithm in the simulation environment to obtain simulation running data;
and the evaluation module is used for comparing the simulation operation data with the real operation data when the driver drives the real host, and evaluating the unmanned vehicle algorithm by adopting the comparison result.
In one embodiment, the simulation environment set by the simulation environment setting module includes: simulating a scene and simulating a position of a host vehicle;
The simulation environment setting module includes:
The simulation scene setting sub-module is used for setting the simulation scene to be the same as the real scene acquired when the driver drives the real host;
And the simulation host vehicle position setting submodule is used for setting the simulation host vehicle position to be the same as the real host vehicle position of the driver when driving the real host vehicle.
In one embodiment, the evaluation module is to:
And comparing at least one of planning track data, planning speed data and planning acceleration data in the simulation operation data with corresponding items in the real operation data, and evaluating the unmanned vehicle algorithm by adopting a comparison result.
In one embodiment, the evaluation module comprises:
The track evaluation sub-module is used for acquiring planning track data of the unmanned vehicle at each simulated main vehicle position and real track data of the real main vehicle at the corresponding real main vehicle position aiming at more than one simulated main vehicle position, and comparing the planning track data with the real track data; and carrying out weighted summation on the comparison results to obtain an evaluation value aiming at planning track data, and evaluating the unmanned vehicle algorithm by adopting the evaluation value.
In one embodiment, the evaluation module comprises:
The speed evaluation sub-module is used for acquiring planning speed data of the unmanned vehicle at each simulated main vehicle position and real speed data of the real main vehicle at the corresponding real main vehicle position aiming at more than one simulated main vehicle position, and comparing the planning speed data with the real speed data; and carrying out weighted summation on the comparison results to obtain an evaluation value aiming at planning speed data, and evaluating the unmanned vehicle algorithm by adopting the evaluation value.
In one embodiment, the evaluation module comprises:
the acceleration evaluation sub-module is used for acquiring planning acceleration data of the unmanned vehicle at each simulated main vehicle position and real acceleration data of the real main vehicle at the corresponding real main vehicle position aiming at more than one simulated main vehicle position, and comparing the planning acceleration data with the real acceleration data; and carrying out weighted summation on the comparison results to obtain an evaluation value aiming at the planning acceleration data, and evaluating the unmanned vehicle algorithm by adopting the evaluation value.
The functions may be implemented by hardware, or may be implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the functions described above.
In one possible design, the structure of the unmanned vehicle algorithm evaluation apparatus includes a processor and a memory, the memory is used for storing a program for executing the unmanned vehicle algorithm evaluation method in the first aspect, and the processor is configured to execute the program stored in the memory. The evaluation device of the unmanned vehicle algorithm may further comprise a communication interface for communication of the evaluation device of the unmanned vehicle algorithm with other devices or a communication network.
In a third aspect, an embodiment of the present invention provides a computer readable storage medium storing computer software instructions for use by an evaluation device of an unmanned vehicle algorithm, which includes a program for executing the method of evaluating an unmanned vehicle algorithm in the first aspect, as referred to by the evaluation device of an unmanned vehicle algorithm.
One of the above technical solutions has the following advantages or beneficial effects:
According to the embodiment of the invention, the simulation environment is consistent with the real environment when the driver drives the real host vehicle, and the simulation operation data of the unmanned vehicle when the unmanned vehicle operates in the simulation environment is compared with the real operation data of the real host vehicle when the unmanned vehicle operates, so that the obtained comparison result can reflect the difference between the unmanned vehicle algorithm and the driving level of the real driver, and the unmanned vehicle algorithm can be evaluated.
The foregoing summary is for the purpose of the specification only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the present invention will become apparent by reference to the drawings and the following detailed description.
Drawings
In the drawings, the same reference numerals refer to the same or similar parts or elements throughout the several views unless otherwise specified. The figures are not necessarily drawn to scale. It is appreciated that these drawings depict only some embodiments according to the disclosure and are not therefore to be considered limiting of its scope.
FIG. 1 is a flowchart of an evaluation method implementation of an unmanned vehicle algorithm according to an embodiment of the present invention;
Fig. 2 is a flowchart illustrating implementation of step S11 in an evaluation method of an unmanned vehicle algorithm according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of an evaluation device of an unmanned vehicle algorithm according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of an evaluation device of another unmanned vehicle algorithm according to an embodiment of the present invention;
fig. 5 is a schematic diagram of an evaluation structure of an unmanned vehicle algorithm according to an embodiment of the present invention.
Detailed Description
Hereinafter, only certain exemplary embodiments are briefly described. As will be recognized by those of skill in the pertinent art, the described embodiments may be modified in various different ways without departing from the spirit or scope of the present invention. Accordingly, the drawings and description are to be regarded as illustrative in nature and not as restrictive.
The embodiment of the invention mainly provides an evaluation method and device of an unmanned vehicle algorithm. The following description of the technical solutions is made by the following examples, respectively.
Referring to fig. 1, fig. 1 is a flowchart of an implementation method of an evaluation method of an unmanned vehicle algorithm according to an embodiment of the present invention, including:
s11: setting a simulation environment, so that the simulation environment is consistent with the real environment when a driver drives a real host.
S12: and operating the unmanned vehicle algorithm in the simulation environment to obtain simulation operation data.
S13: and comparing the simulation operation data with the real operation data when the driver drives the real host, and evaluating the unmanned vehicle algorithm by adopting the comparison result.
In the unmanned vehicle simulation test, a real host vehicle is often utilized to collect a real scene in actual operation, and the real scene is set as a simulation scene in the simulation test. And the unmanned vehicle algorithm operates in the simulation scene, and the advantages and disadvantages of the unmanned vehicle algorithm are evaluated according to the operation result.
In order to compare the difference between the unmanned vehicle algorithm and the driving behavior of the real driver, the simulation environment of the unmanned vehicle algorithm may be set to be completely identical to the real environment, so as to compare the reaction of the unmanned vehicle and the real driver when coping with the same situation. The simulation environment comprises a simulation scene and a simulation host vehicle position.
In one embodiment, as shown in fig. 2, the setting simulation environment in step S11 includes:
S111: the simulation scene is set to be the same as the real scene acquired when the driver drives the real host vehicle.
S112: the simulated host vehicle position is set to be the same as the real host vehicle position when the driver drives the real host vehicle.
Here, the "scene" may be an external scene of the vehicle, including road conditions, the position of an obstacle on the road, and the like; the "host vehicle position" may be the position of the vehicle itself. Therefore, when the simulation scene in the simulation environment is the same as the real scene acquired by the real host vehicle, and the position of the simulation host vehicle in the simulation environment is the same as the position of the real host vehicle, the simulation environment can be ensured to be consistent with the real environment.
For step S112, during the simulation operation, the real host vehicle position of the driver driving the real host vehicle can be replayed, and the simulated host vehicle position is set by adopting the replayed real host vehicle position.
For the step S12, the unmanned vehicle algorithm is operated in the simulation environment, which may be to generate a control instruction of the unmanned vehicle, but the control instruction is not used to control the unmanned vehicle to operate in the simulation environment, so that the position of the simulation host vehicle of the unmanned vehicle is always consistent with the position of the real host vehicle.
In one embodiment, step S13 includes:
And comparing at least one of planning track data, planning speed data and planning acceleration data in the simulation operation data with corresponding items in the real operation data, and evaluating the unmanned vehicle algorithm by adopting a comparison result. The method specifically comprises the following steps:
firstly, comparing planning track data in simulation running data with real track data in real running data; here, the trajectory data may be specifically a host vehicle position for a fixed length of time (e.g., 8 seconds) after the current time.
Second, the planning speed data in the simulation operation data is compared with the real speed data in the real operation data.
Thirdly, the planned acceleration data in the simulation operation data are compared with the real acceleration data in the real operation data.
In other embodiments of the present invention, other operational data may be compared in the same manner as described above, and this is not explicitly recited herein.
Because the vehicle continuously generates operation data in the operation process, when the simulation operation data and the real operation data are compared, the operation data of the vehicle at each position can be selected for being respectively compared, so that a group of comparison results are obtained, and the unmanned vehicle algorithm is comprehensively evaluated.
In one embodiment, the simulation operation data and the real operation data can be extracted according to a fixed time period, and the simulation operation data and the real operation data at corresponding moments are respectively compared to obtain a comparison result.
In this embodiment, the preset time period is 1 second, and the simulation operation data of the unmanned vehicle is extracted every 1 second, including:
1) The simulation planning track can be specifically the position of a simulation host vehicle of future 8S in planning;
2) Simulating the planning speed;
3) And simulating and planning acceleration.
Thus, at the end of the simulation, a set of data is recorded for each simulation run data, such as:
Planning trajectory data= { Ps1, ps2, ps3 … … Psn }; wherein,
Ps1 is a planned track of the unmanned vehicle recorded in the first period;
ps2 is the planned track of the unmanned vehicle recorded in the second period;
Ps3 is the planned track of the unmanned vehicle recorded in the third period;
……
planning speed data= { Vs1, vs2, vs3 … … Vsn }; wherein,
Vs1 is the planned speed of the unmanned vehicle recorded in the first period;
vs2 is the planned speed of the unmanned vehicle recorded in the second period;
vs3 is the planned speed of the unmanned vehicle recorded in the third period;
……
planning acceleration data = { As1, as2, as3 … … Asn }; wherein,
As1 is the planned acceleration of the unmanned vehicle recorded in the first period;
as2 is the planned acceleration of the unmanned vehicle recorded in the second period;
As3 is the planned acceleration of the unmanned vehicle recorded in the third period;
……
for a real host vehicle driven by a skilled driver, real operation data can also be extracted according to the same period (i.e. 1 second), and a set of data is extracted for each real operation data, for example:
true trajectory data= { Pr1, pr2, pr3 … … Prn }; wherein,
Pr1 is the real track of a real host vehicle driven by a skilled driver recorded in the first period;
pr2 is the actual track of the actual host vehicle driven by the skilled driver recorded in the second period;
pr3 is the actual track of the actual host vehicle driven by the skilled driver recorded in the third period;
……
true speed data= { Vr1, vr2, vr3 … … Vrn }; wherein,
Vr1 is the actual speed of the actual host vehicle driven by the skilled driver recorded in the first period;
vr2 is the actual speed of the actual host vehicle driven by the skilled driver recorded in the second period;
Vr3 is the actual speed of the actual host vehicle driven by the skilled driver recorded in the third period;
……
true acceleration data = { Ar1, ar2, ar3 … … Arn }; wherein,
Ar1 is the real acceleration of a real host vehicle driven by a skilled driver recorded in the first period;
Ar2 is the actual acceleration of the actual host vehicle driven by the skilled driver recorded in the second period;
ar3 is the actual acceleration of the actual host vehicle driven by the skilled driver recorded in the third period;
……
For the data, the corresponding data are compared pairwise, namely:
Comparing the planned track data with the real track data, namely, comparing Ps1 with Pr1, and comparing Ps2 with Pr2, … …;
comparing the planning speed with the real speed, namely, comparing Vs1 with Vr1, and comparing Vs2 with Vr2 … …;
Comparing the planned acceleration with the true acceleration, i.e. comparing As1 with Ar1, as2 with Ar2, … …
By the above comparison, three sets of comparison results were obtained:
comparison result of planning track data and real track data = { Pc1, pc2, pc3 … … Pcn };
Comparison result of planning speed and real speed = { Vc1, vc2, vc3 … … Vcn };
Comparison result of planned acceleration and real acceleration = { Ac1, ac2, ac3 … … Acn }.
By adopting the comparison result, the difference between the simulation operation data and the real operation data can be calculated, and concretely:
And (3) carrying out weighted summation on each value in the { Pc1, pc2 and Pc3 … … Pcn } to obtain an evaluation value aiming at the planned track data, and evaluating the unmanned vehicle algorithm by adopting Pc.
And (3) carrying out weighted summation on each value in the { Vc1, vc2 and Vc3 … … Vcn } to obtain an evaluation value aiming at the planning speed data, and adopting Pc to evaluate the unmanned vehicle algorithm.
And carrying out weighted summation on each value in the { Ac1, ac2 and Ac3 … … Acn } to obtain an evaluation value aiming at the planned acceleration data, and adopting Pc to evaluate the unmanned vehicle algorithm.
Furthermore, the Pc, the Vc and the Ac can be weighted and summed to finally obtain data C reflecting the integral difference between the simulation operation data and the real operation data, and the unmanned vehicle algorithm can be evaluated by adopting the data C. The smaller the value of C, the closer the unmanned vehicle algorithm is considered to be to the skilled driver level.
In the above embodiment, the calculation mode of weighted summation is adopted to quantify the difference value of the two sets of corresponding data. In other embodiments of the present invention, other mathematical operations may be used to quantify the difference between two sets of corresponding data, for example, by calculating an average value, calculating a root mean square value, and the like, which is not limited by the present invention.
Therefore, by adopting the evaluation method of the unmanned vehicle algorithm provided by the embodiment of the invention, the advantages and disadvantages of the unmanned vehicle algorithm can be evaluated by setting the simulation environment consistent with the real environment when the driver drives the real host vehicle and comparing the simulation operation data of the unmanned vehicle in the simulation environment with the real operation data of the real host vehicle in operation, and the obtained comparison result can reflect the difference between the unmanned vehicle algorithm and the driving level of the real driver.
The embodiment of the invention also provides an evaluation device of the unmanned vehicle algorithm, referring to fig. 3, fig. 3 is a schematic structural diagram of the evaluation device of the unmanned vehicle algorithm in the embodiment of the invention, which comprises:
A simulation environment setting module 310, configured to set a simulation environment so that the simulation environment is consistent with a real environment when a driver drives a real host;
A simulation running module 320, configured to run the unmanned vehicle algorithm in the simulation environment to obtain simulation running data;
And the evaluation module 330 is configured to compare the simulated operation data with actual operation data when the driver drives the actual host vehicle, and evaluate the unmanned vehicle algorithm by using a result of the comparison.
Fig. 4 is a schematic structural diagram of an evaluation device of an unmanned vehicle algorithm according to an embodiment of the present invention, including:
A simulation environment setup module 310, a simulation run module 320, and an evaluation module 330. Wherein,
The simulation environment set by the simulation environment setting module 310 may include: simulating a scene and simulating a position of a host vehicle;
the simulation environment setting module 310 may include:
The simulation scene setting sub-module 311 is configured to set the simulation scene to be the same as a real scene acquired when the driver drives the real host;
The simulated host vehicle position setting sub-module 312 is configured to set the simulated host vehicle position to be the same as a real host vehicle position when the driver drives the real host vehicle.
The evaluation module 330 may be configured to:
And comparing at least one of planning track data, planning speed data and planning acceleration data in the simulation operation data with corresponding items in the real operation data, and evaluating the unmanned vehicle algorithm by adopting a comparison result.
The evaluation module 330 may include at least one of the following sub-modules:
the track evaluation sub-module 331 is configured to obtain, for more than one simulated host vehicle position, planned track data of the unmanned vehicle when the simulated host vehicle is at each simulated host vehicle position, and real track data of the real host vehicle when the real host vehicle is at a corresponding real host vehicle position, and compare the planned track data with the real track data; and carrying out weighted summation on the comparison results to obtain an evaluation value aiming at planning track data, and evaluating the unmanned vehicle algorithm by adopting the evaluation value.
The speed evaluation submodule 332 is used for acquiring planning speed data of the unmanned vehicle at each simulated main vehicle position and real speed data of the real main vehicle at the corresponding real main vehicle position aiming at more than one simulated main vehicle position, and comparing the planning speed data with the real speed data; and carrying out weighted summation on the comparison results to obtain an evaluation value aiming at planning speed data, and evaluating the unmanned vehicle algorithm by adopting the evaluation value.
The acceleration evaluation sub-module 333 is configured to obtain, for more than one simulated host vehicle position, planned acceleration data of the unmanned vehicle at each simulated host vehicle position and real acceleration data of the real host vehicle at the corresponding real host vehicle position, and compare the planned acceleration data with the real acceleration data; and carrying out weighted summation on the comparison results to obtain an evaluation value aiming at the planning acceleration data, and evaluating the unmanned vehicle algorithm by adopting the evaluation value.
The functions of each module in each device of the embodiments of the present invention may be referred to the corresponding descriptions in the above methods, and are not described herein again.
The embodiment of the invention also provides an evaluation device of the unmanned vehicle algorithm, as shown in fig. 5, which is a schematic structural diagram of the evaluation device of the unmanned vehicle algorithm in the embodiment of the invention, and includes:
memory 11 and processor 12, memory 11 storing a computer program executable on processor 12. The processor 12, when executing the computer program, implements the method of obtaining the optimal parameter combination of the recommendation system in the above embodiment. The number of memories 11 and processors 12 may be one or more.
The apparatus may further include:
And the communication interface 13 is used for communicating with external equipment and carrying out data exchange transmission.
The memory 11 may comprise a high-speed RAM memory or may further comprise a non-volatile memory (non-volatile memory), such as at least one disk memory.
If the memory 11, the processor 12 and the communication interface 13 are implemented independently, the memory 11, the processor 12 and the communication interface 13 may be connected to each other and perform communication with each other through buses. The bus may be an industry standard architecture (ISA, industry Standard Architecture) bus, a peripheral component interconnect (PCI, PERIPHERAL COMPONENT INTERCONNECT) bus, or an extended industry standard architecture (EISA, extended Industry Standard Architecture), among others. The buses may be classified as address buses, data buses, control buses, etc. For ease of illustration, only one thick line is shown in fig. 5, and not only one bus or one type of bus.
Alternatively, in a specific implementation, if the memory 11, the processor 12 and the communication interface 13 are integrated on a chip, the memory 11, the processor 12 and the communication interface 13 may complete communication with each other through internal interfaces.
An embodiment of the present invention provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements a method as described in any of the embodiments above.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present invention, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and further implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.
It is to be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
Those of ordinary skill in the art will appreciate that all or a portion of the steps carried out in the method of the above-described embodiments may be implemented by a program to instruct related hardware, where the program may be stored in a computer readable storage medium, and where the program, when executed, includes one or a combination of the steps of the method embodiments.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing module, or each unit may exist alone physically, or two or more units may be integrated in one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules may also be stored in a computer readable storage medium if implemented in the form of software functional modules and sold or used as a stand-alone product. The storage medium may be a read-only memory, a magnetic or optical disk, or the like.
In summary, according to the method and the device for evaluating the unmanned vehicle algorithm provided by the embodiment of the invention, a simulation environment consistent with the real environment of the driver driving the real host vehicle is set, the unmanned vehicle is operated in the simulation environment, then the simulation operation data and the real operation data are compared, and the obtained comparison result can reflect the difference between the unmanned vehicle algorithm and the driving level of the real driver, so that the unmanned vehicle algorithm can be evaluated.
The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that various changes and substitutions are possible within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (14)
1. A method of evaluating an unmanned vehicle algorithm, the method comprising:
setting a simulation environment, so that the simulation environment is consistent with the real environment when a driver drives a real host;
operating the unmanned vehicle algorithm in the simulation environment to obtain simulation operation data;
And comparing the simulation operation data with the real operation data when the driver drives the real host, and evaluating the unmanned vehicle algorithm by adopting the comparison result.
2. The method of claim 1, wherein the simulation environment comprises: simulating a scene and simulating a position of a host vehicle;
the setting simulation environment comprises the following steps:
Setting the simulation scene to be the same as the real scene acquired when the driver drives the real host;
The simulated host vehicle position is set to be the same as a real host vehicle position when the driver drives a real host vehicle.
3. The method of claim 1, wherein comparing the simulated operational data with actual operational data of a driver driving an actual host vehicle and evaluating the unmanned vehicle algorithm using a result of the comparison comprises:
And comparing at least one of planning track data, planning speed data and planning acceleration data in the simulation operation data with corresponding items in the real operation data, and evaluating the unmanned vehicle algorithm by adopting a comparison result.
4. A method according to claim 3, wherein comparing the planned trajectory data in the simulated operational data with the corresponding items in the actual operational data, and evaluating the unmanned vehicle algorithm using the results of the comparison comprises:
aiming at more than one simulated host vehicle position, acquiring planning track data of the unmanned vehicle in each simulated host vehicle position and real track data of the real host vehicle in a corresponding real host vehicle position, and comparing the planning track data with the real track data;
and carrying out weighted summation on the comparison results to obtain an evaluation value aiming at planning track data, and evaluating the unmanned vehicle algorithm by adopting the evaluation value.
5. A method according to claim 3, wherein comparing the planned acceleration data in the simulated operational data with corresponding items in the actual operational data, and using the results of said comparison to evaluate the unmanned vehicle algorithm, comprises:
Aiming at more than one simulated host vehicle position, acquiring planning speed data of the unmanned vehicle at each simulated host vehicle position and real speed data of a real host vehicle at a corresponding real host vehicle position, and comparing the planning speed data with the real speed data;
and carrying out weighted summation on the comparison results to obtain an evaluation value aiming at planning speed data, and evaluating the unmanned vehicle algorithm by adopting the evaluation value.
6. A method according to claim 3, wherein comparing the planned acceleration data in the simulated operational data with corresponding items in the actual operational data, and using the results of said comparison to evaluate the unmanned vehicle algorithm, comprises:
Aiming at more than one simulated host vehicle position, acquiring planning acceleration data of the unmanned vehicle at each simulated host vehicle position and real acceleration data of a real host vehicle at a corresponding real host vehicle position, and comparing the planning acceleration data with the real acceleration data;
And carrying out weighted summation on the comparison results to obtain an evaluation value aiming at the planning acceleration data, and evaluating the unmanned vehicle algorithm by adopting the evaluation value.
7. An evaluation device of an unmanned vehicle algorithm, the device comprising:
the simulation environment setting module is used for setting a simulation environment so that the simulation environment is consistent with the real environment when a driver drives a real host;
The simulation running module is used for running the unmanned vehicle algorithm in the simulation environment to obtain simulation running data;
and the evaluation module is used for comparing the simulation operation data with the real operation data when the driver drives the real host, and evaluating the unmanned vehicle algorithm by adopting the comparison result.
8. The apparatus of claim 7, wherein the simulation environment set by the simulation environment setting module comprises: simulating a scene and simulating a position of a host vehicle;
the simulation environment setting module includes:
The simulation scene setting sub-module is used for setting the simulation scene to be the same as the real scene acquired when the driver drives the real host;
And the simulation host vehicle position setting submodule is used for setting the simulation host vehicle position to be the same as the real host vehicle position of the driver when driving the real host vehicle.
9. The apparatus of claim 7, wherein the evaluation module is to:
And comparing at least one of planning track data, planning speed data and planning acceleration data in the simulation operation data with corresponding items in the real operation data, and evaluating the unmanned vehicle algorithm by adopting a comparison result.
10. The apparatus of claim 9, wherein the evaluation module comprises:
The track evaluation sub-module is used for acquiring planning track data of the unmanned vehicle at each simulated main vehicle position and real track data of the real main vehicle at the corresponding real main vehicle position aiming at more than one simulated main vehicle position, and comparing the planning track data with the real track data; and carrying out weighted summation on the comparison results to obtain an evaluation value aiming at planning track data, and evaluating the unmanned vehicle algorithm by adopting the evaluation value.
11. The apparatus of claim 9, wherein the evaluation module comprises:
The speed evaluation sub-module is used for acquiring planning speed data of the unmanned vehicle at each simulated main vehicle position and real speed data of the real main vehicle at the corresponding real main vehicle position aiming at more than one simulated main vehicle position, and comparing the planning speed data with the real speed data; and carrying out weighted summation on the comparison results to obtain an evaluation value aiming at planning speed data, and evaluating the unmanned vehicle algorithm by adopting the evaluation value.
12. The apparatus of claim 9, wherein the evaluation module comprises:
the acceleration evaluation sub-module is used for acquiring planning acceleration data of the unmanned vehicle at each simulated main vehicle position and real acceleration data of the real main vehicle at the corresponding real main vehicle position aiming at more than one simulated main vehicle position, and comparing the planning acceleration data with the real acceleration data; and carrying out weighted summation on the comparison results to obtain an evaluation value aiming at the planning acceleration data, and evaluating the unmanned vehicle algorithm by adopting the evaluation value.
13. An evaluation device of an unmanned vehicle algorithm, the device comprising:
One or more processors;
a storage means for storing one or more programs;
The one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-6.
14. A computer readable storage medium storing a computer program, which when executed by a processor implements the method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811228268.5A CN111159833B (en) | 2018-10-22 | 2018-10-22 | Method and device for evaluating unmanned vehicle algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811228268.5A CN111159833B (en) | 2018-10-22 | 2018-10-22 | Method and device for evaluating unmanned vehicle algorithm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111159833A CN111159833A (en) | 2020-05-15 |
| CN111159833B true CN111159833B (en) | 2024-06-18 |
Family
ID=70554469
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811228268.5A Active CN111159833B (en) | 2018-10-22 | 2018-10-22 | Method and device for evaluating unmanned vehicle algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111159833B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112567374A (en) * | 2020-10-21 | 2021-03-26 | 华为技术有限公司 | Simulated traffic scene file generation method and device |
| CN112461250A (en) * | 2020-11-16 | 2021-03-09 | 北京百度网讯科技有限公司 | Vehicle dead reckoning service evaluation method and device, simulation equipment and medium |
| CN116527536B (en) * | 2023-04-28 | 2024-04-09 | 北京永信至诚科技股份有限公司 | Test evaluation method, device and system based on parallel simulation |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106951627A (en) * | 2017-03-15 | 2017-07-14 | 北京百度网讯科技有限公司 | Emulation test method, device, equipment and the computer-readable recording medium of Vehicular automatic driving |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6997106B2 (en) * | 2016-06-27 | 2022-01-17 | モービルアイ ビジョン テクノロジーズ リミテッド | Systems, methods and computer programs for navigating vehicles |
| CN107992016A (en) * | 2016-10-26 | 2018-05-04 | 法乐第(北京)网络科技有限公司 | A kind of automatic driving vehicle analog detection method |
| CN107991898A (en) * | 2016-10-26 | 2018-05-04 | 法乐第(北京)网络科技有限公司 | A kind of automatic driving vehicle simulating test device and electronic equipment |
-
2018
- 2018-10-22 CN CN201811228268.5A patent/CN111159833B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106951627A (en) * | 2017-03-15 | 2017-07-14 | 北京百度网讯科技有限公司 | Emulation test method, device, equipment and the computer-readable recording medium of Vehicular automatic driving |
Non-Patent Citations (1)
| Title |
|---|
| 无人驾驶车辆自动避障路径规划仿真研究;刘博等;计算机仿真;第35卷(第2期);105-110 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111159833A (en) | 2020-05-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111159833B (en) | Method and device for evaluating unmanned vehicle algorithm | |
| CN111199088B (en) | Method and device for reproducing scene data | |
| CN111091739B (en) | Automatic driving scene generation method and device and storage medium | |
| EP3709231A1 (en) | Vehicle track planning method, device, computer device and computer-readable storage medium | |
| CN110686906B (en) | Automatic driving test method and device for vehicle | |
| KR101742536B1 (en) | Running performance test server, test system having the same, testing method using the same | |
| JP6850324B2 (en) | Obstacle distribution simulation method, device, terminal and program based on multi-model | |
| Moukahal et al. | Vulnerability-oriented fuzz testing for connected autonomous vehicle systems | |
| CN112667778A (en) | Information input method, device, equipment and storage medium combining RPA and AI | |
| CN111123732B (en) | Method and device for simulating automatic driving vehicle, storage medium and terminal equipment | |
| CN111199083B (en) | Method and device for reproducing accident data of unmanned vehicle | |
| CN109857118B (en) | Method, device, equipment and storage medium for planning driving strategy of unmanned vehicle | |
| CN111123729B (en) | Method and device for optimizing vehicle driving simulation, storage medium and terminal equipment | |
| KR101690280B1 (en) | Running performance test server, test system having the same, testing method using the same | |
| CN111125270A (en) | Map construction and loading method and device | |
| CN117724982A (en) | Simulation evaluation method and device, electronic equipment and storage medium | |
| CN111125857A (en) | Distributed simulation method and device | |
| CN111177878B (en) | Derived simulation scene screening method, device and terminal | |
| CN112597835B (en) | Driving behavior evaluation method and device, electronic equipment and readable storage medium | |
| CN111090269B (en) | Sensor simulation method, device and storage medium based on generation of countermeasure network | |
| CN111091581B (en) | Pedestrian track simulation method, device and storage medium based on generation countermeasure network | |
| CN115140075A (en) | Quantitative evaluation method, device, equipment and medium for safety of expected function of vehicle | |
| Feth et al. | A Context-Aware, Confidence-Disclosing and Fail-Operational Dynamic Risk Assessment Architecture | |
| CN111198774B (en) | Unmanned vehicle simulation anomaly tracking method, device, equipment and computer readable medium | |
| CN111175055B (en) | Automatic driving distributed collaborative simulation method and device and terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20211013 Address after: 105 / F, building 1, No. 10, Shangdi 10th Street, Haidian District, Beijing 100085 Applicant after: Apollo Intelligent Technology (Beijing) Co.,Ltd. Address before: 100085 Baidu Building, 10 Shangdi Tenth Street, Haidian District, Beijing Applicant before: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |