CN114579088A - Unmanned algorithm development method based on data mining and test closed loop - Google Patents
Unmanned algorithm development method based on data mining and test closed loop Download PDFInfo
- Publication number
- CN114579088A CN114579088A CN202111671857.2A CN202111671857A CN114579088A CN 114579088 A CN114579088 A CN 114579088A CN 202111671857 A CN202111671857 A CN 202111671857A CN 114579088 A CN114579088 A CN 114579088A
- Authority
- CN
- China
- Prior art keywords
- data
- mining
- software
- library
- closed loop
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 46
- 238000000034 method Methods 0.000 title claims abstract description 28
- 238000011161 development Methods 0.000 title claims abstract description 26
- 238000007418 data mining Methods 0.000 title claims abstract description 17
- 230000010354 integration Effects 0.000 claims abstract description 8
- 238000012795 verification Methods 0.000 claims abstract description 7
- 238000005065 mining Methods 0.000 claims description 23
- 230000015572 biosynthetic process Effects 0.000 claims description 3
- 238000000605 extraction Methods 0.000 claims description 3
- 230000018109 developmental process Effects 0.000 description 18
- 230000006399 behavior Effects 0.000 description 7
- 238000001514 detection method Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000012827 research and development Methods 0.000 description 2
- 241000270295 Serpentes Species 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000007670 refining Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/20—Software design
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/10—Requirements analysis; Specification techniques
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses an unmanned algorithm development method based on data mining and test closed loops, which comprises the following steps: (1) the method comprises the following steps of requirement analysis, software design, software development, software verification, software integration test, software release and mileage test; (2) continuously acquiring various data generated in the testing process during software integration testing and mileage testing; (3) digging problem data from the collected data to form a problem library; (4) and performing secondary development on the software based on the problem library to repair the problem and form a development verification closed loop. The invention adds a development verification closed loop and a demand closed loop on the basis of a conventional development mode, and iteratively upgrades the algorithm through a question library and a scene library respectively, thereby improving the reliability and the upgrading efficiency of the algorithm.
Description
Technical Field
The invention relates to an unmanned algorithm development method based on data mining and test closed loops, and belongs to the technical field of automatic driving.
Background
The unmanned algorithm has high complexity, and the general development adopts a V flow to develop and verify according to the requirements of projects. However, for the high-level unmanned algorithm, in order to reduce the development cost, a development mode of platform software and a mode of platform software plus project adaptation are considered. The development of platform software is generally test-driven, and problems found in drive tests can be input as development requirements, so that a closed loop of development and test is achieved.
At present, a pure manual mode is used for developing and testing a closed loop, and a large amount of road test mileage and manual problem judgment are relied on. This approach is highly dependent on the experience of the tester, and if the proportion of problems is found to be reduced, more testing mileage is required to make up, and the overall process is less efficient. Testing is often the bottleneck in research and development when developing new unmanned algorithms.
Disclosure of Invention
The invention aims to solve the technical problem that the existing unmanned driving algorithm is relatively inefficient in developing and testing closed loops in a pure manual mode.
In order to achieve the aim, the invention provides an unmanned algorithm development method based on data mining and test closed loop, which comprises the following steps: (1) the method comprises the following steps of requirement analysis, software design, software development, software verification, software integration test, software release and mileage test; (2) continuously collecting various data generated in the testing process during the software integration testing and the mileage testing; (3) digging problem data from the collected data to form a problem library; (4) and performing secondary development on the software based on the problem library to repair the problem and form a development verification closed loop.
Further, the step (4) is followed by the following steps: (5) carrying out scene extraction from the problem library to form a scene library; (6) mining optimizable points of an algorithm from the scene library; (7) and providing the optimizable point to a demand analysis side to form a demand closed loop.
Further, in the step (5), valuable scene data is further extracted from the full-scale library formed by the collected data and added into the scene library.
Further, the scene refining process in the step (5) is as follows: the problem is first attributed and then several typical scenarios are selected to join the scenario library for the same reason problem.
Further, the typical scene includes at least a problem phenomenon, a road category, a weather condition, a lighting condition, and a time.
Further, the problem phenomena include sudden braking, line pressing driving, unexpected sudden braking and vehicle snaking, the road categories include expressways, urban roads and ramps, the weather conditions include sunny days, cloudy days, rainy days and snowy days, and the lighting conditions include daytime and nighttime.
Further, the optimizable point in step (6) is based on the cause of formation of each problem.
Further, the mining mode of the problem data in the step (3) comprises manual problem mining, automatic problem mining and comparison data mining.
Further, the manual problem mining includes intercepting problem data in a tagged manner.
Further, the automated problem mining includes data rationality checking and comparison against data.
Compared with the prior art, the invention has the beneficial effects that:
1. the data can generate higher value, and the automatic mining algorithm can fully utilize the large calculation power of the data center to achieve the coverage rate of the test problems which cannot be achieved by manpower, namely, a plurality of problems which cannot be found by the manpower can be automatically mined.
2. A development verification closed loop and a demand closed loop are added on the basis of a conventional development mode, and the algorithm is iteratively upgraded through a problem library and a scene library respectively, so that the reliability and the upgrading efficiency of the algorithm are improved.
3. The scene library can be continuously accumulated and used as a research and development support of a higher-level iterative algorithm.
Drawings
FIG. 1 is a flow chart of one embodiment of the present invention;
FIG. 2 is a functional block diagram of automated problem mining in one embodiment of the present invention.
Detailed Description
The technical solution of the present invention is further explained with reference to the accompanying drawings and the specific embodiments.
As shown in FIG. 1, one embodiment of the present invention is an unmanned algorithm development method based on data mining and test closed loop, comprising the following steps: (1) the method comprises the following steps of requirement analysis, software design, software development, software verification, software integration test, software release and mileage test; (2) continuously acquiring various data generated in the testing process during software integration testing and mileage testing; (3) digging out problem data from the collected data to form a problem library; (4) performing secondary development on software based on the problem library to repair problems and form a development verification closed loop; (5) carrying out scene extraction from the problem library to form a scene library; (6) mining optimizable points of an algorithm from the scene library; (7) and providing the optimizable point to a demand analysis side to form a demand closed loop. The mileage test in the step (2) comprises an unmanned mileage test and a manned mileage test, wherein the manned mileage test adopts a shadow mode to collect data.
In one embodiment of the invention, in the step (5), valuable scene data is further extracted from a full-volume library formed by the collected full-volume data and added into the scene library. For example, the sensor data and the algorithm intermediate result data during the drive test for 8 hours and 8 hours are recorded as the full data.
In an embodiment of the present invention, the scene refinement in step (5) is performed by: the problem is first attributed and then several typical scenarios are selected to join the scenario library for the same reason problem.
In one embodiment of the invention, the typical scenario includes at least the following dimensions: problem phenomena, road category, weather conditions, lighting conditions and time.
In one embodiment of the invention, said problematic phenomena include sudden braking, traffic congestion, unexpected sudden braking and vehicle hunting, said road categories include expressways, urban roads and ramps, said weather conditions include sunny, cloudy, rainy and snowy days, and said lighting conditions include daytime and nighttime. The sudden braking refers to normal braking when an obstacle exists in the front, and unexpected sudden braking is caused by mistakenly recognizing the obstacle in the front.
In one embodiment of the present invention, the optimizable point in step (6) is based on the cause of formation of each problem. For example, unexpected hard braking is caused by a false recognition of an obstacle in front, possibly because the camera range is not accurate for recognizing a distant object in the near, so that the camera range recognition algorithm needs to be optimized for the scene, and the algorithm needs to be optimized after attribution for each problem.
In one embodiment of the present invention, the mining manner of the problem data in the step (3) includes manual problem mining, automatic problem mining and comparison data mining. The automatic data mining is to check the algorithm result data, and the comparison data mining is to compare the unmanned algorithm result data with the manned data.
In one embodiment of the invention, the manual problem mining includes intercepting problem data in a tagged manner. For example, when it is found that an automatically driven vehicle suddenly brakes suddenly without accident, or suddenly does not run in the middle of a road and deviates to the left or right, or the vehicle snakes, a tester can record the time and specific phenomena of the problem on the acquisition software, namely, the time and the specific phenomena are marked, so that the vehicle can be quickly found for further analysis in the following process. The problem data obtained by labeling is intercepted from the full data.
In one embodiment of the invention, the automated problem mining includes data reasonableness checking and comparison against data. Wherein the comparison data comprises truth data and driver behavior data. When the truth value data is collected, besides the vehicle data, a set of GPS equipment is installed on the target vehicle as an obstacle, and information such as the position, the speed, the acceleration and the like of the target vehicle is also recorded as the truth value. The driver behavior data comprises driving behaviors such as stepping on an accelerator, stepping on a brake and turning a steering wheel.
As shown in fig. 2, the data sources for automated problem mining include environmental raw data, algorithm result data, and control data. Sources of the comparison data include collected truth data, driver behavior data, manual annotations based on images and point clouds. There are different mining modes for different data sources.
And (3) performing target recognition and semantic segmentation on the image and point cloud data through a deep learning model by deep learning detection. Firstly, a data segment with contrast data or labeled data is selected, the difference between a target detection result and the contrast data is compared, and a part with larger difference is put into a question bank. And adding the subsequent model into a scene in the problem library for optimization, and reasoning the environmental original data corresponding to the part of data by using the new model after optimization until the difference with the control group is small enough.
The sensor comparison detection mainly aims at perception fusion, and potential problem points are found by comparing the difference of recognition results among different sensors. Different sensors have different advantages and disadvantages for the identification of different information, for example, the laser radar range finding is accurate, and the camera judges the category more accurately. This comparison may optimize the sensing algorithm performance of other sensors by more accurate sensors.
The rationality detection finds obvious and unreasonable logics for the output results of all algorithm modules, such as no output for a long time, jump of position or speed, jump of obstacles and the like. For example, an obstacle on the right before 100ms suddenly runs to the left after 100ms, which belongs to the obstacle jump phenomenon.
The problematic parts are found from the calculation power through the data center. The check rule is manually made and gradually supplemented and perfected along with the iteration of the algorithm.
The driver contrast detection mainly faces the situation that the shadow mode records the algorithm control information and the driver control information at the same time, and finds out the behavior difference between the two. The shadow mode means that the driver controls the vehicle, but the algorithm is also running synchronously (only the result is calculated, the vehicle is not controlled), and the operation behavior of the driver can be obtained and compared with the control decision of the algorithm. And finding out the possibly unreasonable algorithm control behavior by comparing the acceleration and deceleration time of the driver with the braking time, and then confirming whether the problem is a planning problem or a control problem through the verification of the planning.
All the problem mining modes need strong calculation support, and manual participation is not needed between data acquisition and problem library generation.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting the same, and although the present invention is described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications and equivalents may be made to the embodiments of the invention without departing from the spirit and scope of the invention, which is to be covered by the claims.
Claims (10)
1. An unmanned algorithm development method based on data mining and test closed loop is characterized by comprising the following steps:
(1) the method comprises the following steps of requirement analysis, software design, software development, software verification, software integration test, software release and mileage test;
(2) continuously acquiring various data generated in the testing process during software integration testing and mileage testing;
(3) digging problem data from the collected data to form a problem library;
(4) and performing secondary development on the software based on the problem library to repair the problem and form a development verification closed loop.
2. The method for developing unmanned algorithm based on data mining and test closed loop according to claim 1, wherein the step (4) is further followed by the steps of:
(5) carrying out scene extraction from the problem library to form a scene library;
(6) mining optimizable points of an algorithm from the scene library;
(7) and providing the optimizable point to a demand analysis side to form a demand closed loop.
3. The closed-loop data mining and testing-based unmanned algorithm development method according to claim 2, wherein in the step (5), valuable scene data is further extracted from a full-scale library formed by collected data and added into the scene library.
4. The method for developing unmanned algorithm based on data mining and test closed loop according to claim 2, wherein the scene refinement process in step (5) is as follows: the problem is first attributed and then several typical scenarios are selected to join the scenario library for the same reason problem.
5. The method of claim 4, wherein the typical scenario includes at least problem phenomena, road category, weather conditions, lighting conditions and time.
6. The method of claim 5, wherein the problem phenomena include hard braking, line-pressing driving, unexpected hard braking and vehicle hunting, the road categories include expressways, urban roads and ramps, the weather conditions include sunny days, cloudy days, rainy days and snowy days, and the lighting conditions include day and night.
7. The method for developing unmanned aerial vehicle algorithm based on data mining and test closed loop of claim 2, wherein the optimizable point in step (6) is based on the cause of formation of each problem.
8. The method for developing unmanned aerial vehicle algorithm based on data mining and test closed loop of claim 1, wherein the mining manner of the problem data in the step (3) comprises manual problem mining, automatic problem mining and comparison data mining.
9. The method of claim 8, wherein the manual problem mining comprises intercepting problem data in a tagged manner.
10. The method of claim 8, wherein the automated problem mining comprises data plausibility checking and comparison against data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111671857.2A CN114579088A (en) | 2021-12-31 | 2021-12-31 | Unmanned algorithm development method based on data mining and test closed loop |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111671857.2A CN114579088A (en) | 2021-12-31 | 2021-12-31 | Unmanned algorithm development method based on data mining and test closed loop |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114579088A true CN114579088A (en) | 2022-06-03 |
Family
ID=81769613
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111671857.2A Pending CN114579088A (en) | 2021-12-31 | 2021-12-31 | Unmanned algorithm development method based on data mining and test closed loop |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114579088A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117149860A (en) * | 2023-10-31 | 2023-12-01 | 安徽中科星驰自动驾驶技术有限公司 | Driving data mining method and system for automatic driving vehicle |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111143424A (en) * | 2018-11-05 | 2020-05-12 | 百度在线网络技术（北京）有限公司 | Characteristic scene data mining method and device and terminal |
| CN111741105A (en) * | 2020-06-18 | 2020-10-02 | 华域汽车***股份有限公司 | Real-time transmission system and method for problem data of intelligent driving system |
| CN112541258A (en) * | 2020-12-08 | 2021-03-23 | 特路（北京）科技有限公司 | Test scene library of automatic driving automobile test field |
| CN112965466A (en) * | 2021-02-18 | 2021-06-15 | 北京百度网讯科技有限公司 | Reduction test method, device, equipment and program product of automatic driving system |
| CN113762406A (en) * | 2021-09-15 | 2021-12-07 | 东软睿驰汽车技术（沈阳）有限公司 | Data mining method and device and electronic equipment |
-
2021
- 2021-12-31 CN CN202111671857.2A patent/CN114579088A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111143424A (en) * | 2018-11-05 | 2020-05-12 | 百度在线网络技术（北京）有限公司 | Characteristic scene data mining method and device and terminal |
| CN111741105A (en) * | 2020-06-18 | 2020-10-02 | 华域汽车***股份有限公司 | Real-time transmission system and method for problem data of intelligent driving system |
| CN112541258A (en) * | 2020-12-08 | 2021-03-23 | 特路（北京）科技有限公司 | Test scene library of automatic driving automobile test field |
| CN112965466A (en) * | 2021-02-18 | 2021-06-15 | 北京百度网讯科技有限公司 | Reduction test method, device, equipment and program product of automatic driving system |
| CN113762406A (en) * | 2021-09-15 | 2021-12-07 | 东软睿驰汽车技术（沈阳）有限公司 | Data mining method and device and electronic equipment |
Non-Patent Citations (3)
| Title |
|---|
| 51非知名小编: "清华苏研院王宝宗：挖掘测试数据及构建场景库是提高自动驾驶安全性能极其重要的一步", pages 1 - 8, Retrieved from the Internet <URL:《https://zhuanlan.zhihu.com/p/134128697》> * |
| 九章智驾: "一文读懂自动驾驶仿真测试场景与场景库", pages 1 - 19, Retrieved from the Internet <URL:《https://zhuanlan.zhihu.com/p/399032822》> * |
| 刘生;: "智能网联汽车驾驶场景数据采集的研究及应用", 汽车纵横, no. 08, 15 August 2018 (2018-08-15) * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117149860A (en) * | 2023-10-31 | 2023-12-01 | 安徽中科星驰自动驾驶技术有限公司 | Driving data mining method and system for automatic driving vehicle |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107421752B (en) | A kind of intelligent automobile test scene acceleration reconstructing method | |
| CN114077541A (en) | Method and system for validating automatic control software for an autonomous vehicle | |
| CN112950811B (en) | New energy automobile region operation risk assessment and early warning system integrating whole automobile safety | |
| CN112487905B (en) | Method and system for predicting danger level of pedestrian around vehicle | |
| CN113343461A (en) | Simulation method and device for automatic driving vehicle, electronic equipment and storage medium | |
| WO2021257429A2 (en) | Label-free performance evaluator for traffic light classifier system | |
| US20230419683A1 (en) | Method and system for automatic driving data collection and closed-loop management | |
| CN110765224A (en) | Processing method of electronic map, vehicle vision repositioning method and vehicle-mounted equipment | |
| CN111707476B (en) | Longitudinal driving capability detection method for automatic driving automobile | |
| CN114579088A (en) | Unmanned algorithm development method based on data mining and test closed loop | |
| Ikhlayel et al. | Traffic sign detection for navigation of autonomous car prototype using convolutional neural network | |
| CN112509321A (en) | Unmanned aerial vehicle-based driving control method and system for urban complex traffic situation and readable storage medium | |
| CN112525552A (en) | Algorithm test system and method for automobile advanced auxiliary driving system | |
| CN110446106B (en) | Method for identifying front camera file, electronic equipment and storage medium | |
| CN115346288A (en) | Simulation driving record acquisition method and system, electronic equipment and storage medium | |
| CN117612140B (en) | Road scene identification method and device, storage medium and electronic equipment | |
| US20240020964A1 (en) | Method and device for improving object recognition rate of self-driving car | |
| CN117612127B (en) | Scene generation method and device, storage medium and electronic equipment | |
| CN117601903B (en) | Method and device for generating vehicle driving scene, storage medium and electronic equipment | |
| US11651508B1 (en) | Methods and apparatus for depth estimation using stereo cameras in a vehicle system | |
| CN116620278B (en) | Unmanned lane keeping method | |
| CN114254737B (en) | Multi-factor neural network model management method and system | |
| CN116461540A (en) | Updating method, device, equipment and storage medium | |
| KR20240011068A (en) | Method for improving object detecting ratio of self-driving car and apparatus thereof | |
| CN116030416A (en) | Expressway vehicle monitoring method, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |