WO2022115987A1 - 一种自动驾驶数据采集和闭环管理的方法及*** - Google Patents
一种自动驾驶数据采集和闭环管理的方法及*** Download PDFInfo
- Publication number
- WO2022115987A1 WO2022115987A1 PCT/CN2020/133062 CN2020133062W WO2022115987A1 WO 2022115987 A1 WO2022115987 A1 WO 2022115987A1 CN 2020133062 W CN2020133062 W CN 2020133062W WO 2022115987 A1 WO2022115987 A1 WO 2022115987A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- data
- vehicle
- neural network
- perception
- incremental
- Prior art date
Links
- 238000013480 data collection Methods 0.000 title claims abstract description 51
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000013528 artificial neural network Methods 0.000 claims abstract description 140
- 238000012216 screening Methods 0.000 claims abstract description 81
- 238000007781 pre-processing Methods 0.000 claims abstract description 17
- 230000008447 perception Effects 0.000 claims description 114
- 230000009471 action Effects 0.000 claims description 38
- 230000002787 reinforcement Effects 0.000 claims description 36
- 230000016776 visual perception Effects 0.000 claims description 21
- 238000001914 filtration Methods 0.000 claims description 16
- 230000006870 function Effects 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 11
- 238000011084 recovery Methods 0.000 claims description 10
- 230000009467 reduction Effects 0.000 claims description 7
- 230000001953 sensory effect Effects 0.000 claims description 7
- 230000000295 complement effect Effects 0.000 claims description 5
- 230000015654 memory Effects 0.000 claims description 4
- 238000007726 management method Methods 0.000 abstract description 40
- 230000005540 biological transmission Effects 0.000 abstract description 10
- 238000013500 data storage Methods 0.000 abstract description 8
- 238000012549 training Methods 0.000 description 15
- 238000012360 testing method Methods 0.000 description 7
- 230000001133 acceleration Effects 0.000 description 4
- 230000004438 eyesight Effects 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000006403 short-term memory Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000008685 targeting Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W60/00—Drive control systems specially adapted for autonomous road vehicles
- B60W60/001—Planning or execution of driving tasks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/004—Artificial life, i.e. computing arrangements simulating life
- G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/092—Reinforcement learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C5/00—Registering or indicating the working of vehicles
- G07C5/008—Registering or indicating the working of vehicles communicating information to a remotely located station
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/06—Improving the dynamic response of the control system, e.g. improving the speed of regulation or avoiding hunting or overshoot
- B60W2050/065—Improving the dynamic response of the control system, e.g. improving the speed of regulation or avoiding hunting or overshoot by reducing the computational load on the digital processor of the control computer
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
- G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C5/00—Registering or indicating the working of vehicles
- G07C5/08—Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time
- G07C5/0841—Registering performance data
- G07C5/085—Registering performance data using electronic data carriers
Definitions
- the invention relates to the technical field of automatic driving, in particular to a method and system for automatic driving data collection and closed-loop management.
- DNN neural network performance
- the present invention is proposed to provide a method and system for automatic driving data collection and closed-loop management that overcome the above problems or at least partially solve the above problems.
- One object of the present invention is to provide a method and system for automatic driving data collection and closed-loop management that can effectively filter out high-value incremental data.
- a further object of the present invention is to train the neural network of the perception and decision-making system for autonomous driving by using the filtered incremental data, and deploy the trained neural network to update the pre-trained neural network for data collection, thereby forming an improved Incremental data closed-loop management targeting neural network perception capabilities.
- a method for automatic driving data collection and closed-loop management is provided, which is applied to the vehicle side, including:
- the pre-trained neural network is used to filter the pre-processed vehicle driving data according to preset screening rules to obtain incremental data;
- the method further includes:
- the vehicle driving data includes vehicle perception data
- the neural network is a deep neural network.
- the vehicle perception data includes vehicle visual perception data
- the preprocessing of the vehicle driving data includes:
- At least one of the following processing is performed on the vehicle-mounted visual perception data: deformation recovery, noise reduction, deblurring, white balance, image complement and recovery under strong light or low light conditions.
- the vehicle perception data further includes vehicle navigation data and vehicle status data;
- the preprocessing of the vehicle driving data further includes:
- Time synchronization is performed on the vehicle visual perception data, the vehicle navigation data and the vehicle state data to form structured vehicle perception data, wherein each structured vehicle perception data includes a timestamp, a Perceive the image, the vehicle position at the current moment, and the vehicle dynamics parameters at the current moment.
- the preset screening rules include at least one of the following:
- High-precision map and perception data comparison screening rules prediction data and perception data comparison screening rules, cloud command screening rules, adjacent frame image difference screening rules.
- the high-precision map and perception data comparison and screening rules include:
- the comparison and screening rules for the predicted data and perceptual data include:
- the actual perceived second position of the obstacle vehicle in the second number of frames in the future from the current moment is obtained through the pre-trained neural network based on the structured vehicle perception data in the second number of frames in the future from the current moment, the second quantity is greater than or equal to the first quantity;
- the first position in each frame in the future and the second position in the corresponding frame determine whether the predicted data is consistent with the perception data
- judging whether the predicted data is consistent with the perceptual data according to the first position in each future frame and the second position in the corresponding frame including:
- formula (1) represents the actual perceived second position of the obstacle vehicle in the i-th frame in the future, represents the predicted first position of the obstacle vehicle in the ith frame in the future, and n is equal to the first number;
- the cloud instruction screening rules include:
- the incremental data is obtained by filtering the structured vehicle perception data according to the data filtering instruction issued by the cloud.
- the adjacent frame image difference screening rules include:
- the difference index is mean square error
- the calculating the difference index between the perceived images in the structured vehicle perception data of the adjacent previous frame and the next frame includes:
- It (i, j ) and It +1 (i, j) represent the pixel values of two adjacent frames of perceptual images at time t and time t+1, respectively, and i and j represent the perception pixel coordinates in the image, where m and n represent the length and width of the perceived image size, respectively; or
- the difference index is structural similarity
- the calculating the difference index between the perceived images in the structured vehicle perception data of the adjacent previous frame and the next frame includes:
- x and y represent the pixel values of two adjacent frames of perceptual images
- ⁇ x is the mean of x
- ⁇ y is the mean of y
- ⁇ x is the variance of x
- ⁇ y is the variance of y
- ⁇ xy is the covariance of x and y
- c 1 and c 2 are constants to maintain stability
- c 1 (k 1 *L) 2
- c 2 (k 2 *L) 2
- L is the dynamic range of pixel values.
- the vehicle driving data includes vehicle control data
- the neural network is a deep reinforcement learning neural network.
- the vehicle control data includes current vehicle state data S t and the current control action a human of the driver of the vehicle;
- the pre-processed vehicle driving data is screened to obtain incremental data including:
- the vehicle state data S t+1 constructs a data packet as the incremental data.
- determining the reward and punishment function value r(t) of the deep reinforcement learning neural network under the current vehicle state data S t and the current control action a nn output by the deep reinforcement learning neural network includes:
- R ca (t) is the collision avoidance index
- R ma (t) is the task completion index
- R lk (t) is the lane keeping ability index
- R speed (t) is the speed rationality index
- R steer (t) Indicates the steering rationality index
- R ca (t), R ma (t), R lk (t), R speed (t) and R steer (t) are the assumptions about the current vehicle state data S t .
- the control system of the vehicle is calculated after applying the current control action a nn output by the deep reinforcement learning neural network, and b 1 , b 2 , b 3 , and b 4 represent weight coefficients.
- the current vehicle state data S t includes at least one of the following: current environment information, current vehicle speed, and current vehicle steering angle;
- the current control action a human of the driver includes at least one of the following: a first decision-making command, a steering angle, and an accelerator pedal/brake pedal opening;
- the current control action an nn output by the deep reinforcement learning neural network includes at least one of the following: a second decision command, a steering command, and a speed command.
- the method before obtaining incremental data by filtering the pre-processed vehicle driving data according to preset screening rules by a pre-trained neural network, the method further includes:
- the preprocessed vehicle driving data is cached.
- an automatic driving data collection and closed-loop management system is also provided, which is applied to the vehicle end, including:
- a data acquirer configured to acquire vehicle driving data
- a data processor includes: a data preprocessing module configured to preprocess the vehicle driving data; and a data screening module, on which a pretrained neural network is deployed, configured to Set up a screening rule to filter the preprocessed vehicle driving data to obtain incremental data; and
- a memory configured to store the incremental data
- a sending module configured to send the incremental data to the cloud.
- the data screening module is further configured to:
- the data acquirer includes one or more vehicle-mounted cameras, and the one or more vehicle-mounted cameras are configured to acquire vehicle-mounted visual perception data;
- the data preprocessing module is also configured to:
- At least one of the following processing is performed on the vehicle-mounted visual perception data: deformation recovery, noise reduction, deblurring, white balance, image complement and recovery under strong light or low light conditions.
- the data acquirer further includes:
- a navigation device configured to obtain vehicle navigation data
- Vehicle network equipment configured to obtain vehicle status data
- the data processor further includes a data synchronization module configured to perform time synchronization on the vehicle visual perception data, the vehicle navigation data and the vehicle status data to form structured vehicle perception data, wherein each of the The structured vehicle perception data includes timestamps, perception images at the current moment, vehicle position at the current moment, and vehicle dynamics parameters at the current moment.
- the data processor further includes:
- a data cache module configured to cache the preprocessed vehicle driving data.
- the storage is a vehicle-mounted mobile hard disk.
- the method and system for automatic driving data collection and closed-loop management provided by the embodiments of the present invention, after the pre-processed vehicle driving data is filtered through a pre-trained neural network according to preset screening rules to obtain incremental data, the incremental data is then stored. data or send incremental data to the cloud for neural network training.
- the solution of the present invention can effectively filter out high-value incremental data, thereby reducing the system data storage capacity and/or the requirement for data transmission bandwidth.
- the system of the present invention can be installed on the vehicle in a rear-mounted or front-mounted manner, does not depend on a specific vehicle model, and does not require the vehicle to be equipped with a high-cost real-value system such as lidar, which greatly improves the convenience of the system and is conducive to rapid and large-scale application.
- the solution of the present invention adopts at least one of the comparison and screening rules of high-precision maps and perceptual data, the comparison and screening rules of predicted data and perceptual data, the screening rules of cloud commands, the screening rules of adjacent frame image differences, etc. to collect the vehicle driving data. By filtering, more accurate and valuable incremental data can be obtained, which can greatly reduce the amount of perception data storage or transmission.
- the scheme of the present invention trains the neural network of the perception and decision-making system of automatic driving by using the filtered incremental data, and deploys the trained neural network to update the pre-trained neural network for data collection and screening, thereby forming a Incremental data closed-loop management with the goal of improving neural network perception capabilities.
- FIG. 1 is a schematic flowchart of a method for automatic driving data collection and closed-loop management according to an embodiment of the present invention
- FIG. 2 is a schematic flowchart of a method for automatic driving data collection and closed-loop management according to another embodiment of the present invention
- FIG. 3 is a schematic flowchart of a method for automatic driving data collection and closed-loop management according to yet another embodiment of the present invention.
- FIG. 4 is a schematic flowchart of a method for automatic driving data collection and closed-loop management according to still another embodiment of the present invention.
- FIG. 5 is a schematic structural diagram of an automatic driving data collection and closed-loop management system according to an embodiment of the present invention.
- FIG. 6 is a schematic structural diagram of an automatic driving data collection and closed-loop management system according to another embodiment of the present invention.
- DNN For target detection, recognition, classification and prediction. Continuously optimizing the iterative perception and decision-making system DNN through actual data, improving the performance index of DNN, and enhancing the robustness and stability of the autonomous driving system in response to various scenarios are the key to the large-scale deployment of autonomous vehicles.
- most enterprises and scientific research institutions usually use special data collection vehicles or test vehicles for centralized data collection. These data collection vehicles generally need to be equipped with real-value systems such as lidar, which require high equipment, and the data collection process is not designed.
- the screening mechanism or screening mechanism is unclear, resulting in a large data storage space requirement or a large data transmission bandwidth requirement, which not only puts forward high requirements on the hardware system, but also increases the difficulty for subsequent data mining and finding valuable data.
- an embodiment of the present invention proposes a method and system for automatic driving data collection and closed-loop management, and the method and system can be applied to the vehicle end.
- FIG. 1 shows a schematic flowchart of a method for automatic driving data collection and closed-loop management according to an embodiment of the present invention.
- the automatic driving data collection and closed-loop management method may at least include the following steps S102 to S108.
- Step S102 acquiring vehicle driving data.
- Step S104 preprocessing the vehicle driving data.
- step S106 the pre-trained neural network is used to filter the pre-processed vehicle driving data according to a preset screening rule to obtain incremental data.
- step S108 the incremental data is stored, or the incremental data is sent to the cloud.
- the incremental data may be stored locally (ie, the vehicle-side memory), and then the copy of the incremental data stored locally may be periodically transferred to the cloud (eg, a cloud computing center) for subsequent follow-up Data processing and neural network training.
- the cloud eg, a cloud computing center
- incremental data can also be directly sent to the cloud through a network communication module (such as T-box) for subsequent data processing and neural network training.
- T-box network communication module
- it can also be stored locally and uploaded to the cloud at the same time, so as to achieve multiple backups and ensure that data will not be lost.
- the automatic driving data collection and closed-loop management method provided by the embodiment of the present invention adopts a distributed incremental data collection method, which can effectively filter out high-value incremental data, thereby reducing the system data storage capacity and/or the requirement for data transmission bandwidth .
- step S110 may also be performed: receiving and deploying the trained neural network delivered by the cloud to update the pre-trained neural network, wherein the trained neural network Neural network trained on incremental data.
- the trained neural network delivered by the cloud may be received through the T-box configured on the vehicle in an OTA (Over-the-Air, over-the-air) manner.
- the neural network of the perception decision-making system for autonomous driving is trained by using the filtered incremental data, and the trained neural network is deployed to update the pre-trained neural network, and the updated neural network is used again for data collection and screening, In this way, a closed-loop management of incremental data is formed with the goal of improving the perception ability of the neural network.
- the incremental data stored locally can be periodically copied to the cloud computing center. After analyzing and labeling the incremental data, incremental training is performed on the neural network using the labelled data, and the trained neural network is obtained. Neural Networks.
- the incremental data after the incremental data is sent to the cloud, the incremental data can be directly analyzed and labeled, and the neural network can be incrementally trained using the labeled data to obtain a trained neural network.
- the data processing efficiency can be effectively improved; on this basis, using the labeled data to perform incremental training of the neural network can effectively improve the performance indicators of the neural network and improve the neural network. training and learning efficiency.
- step S105 may be performed first: caching the pre-processed vehicle driving data.
- the preprocessed vehicle driving data acquired within a preset time eg, 5 minutes
- a preset time eg, 5 minutes
- vehicle driving data may include vehicle perception data or vehicle control data.
- vehicle perception data may include vehicle perception data or vehicle control data.
- vehicle control data The collection and closed-loop management methods of these two types of data are introduced separately below.
- DNN deep neural network
- Step S302 acquiring vehicle driving data, wherein the vehicle driving data includes vehicle perception data.
- vehicle perception data may specifically include vehicle-mounted visual perception data, that is, perception data such as images, videos, and the like acquired by a vision system (specifically, one or more cameras) configured on the vehicle.
- the above step S104 may specifically include the following step S304: performing at least one of deformation restoration, noise reduction, deblurring, white balance, image complementation and restoration under strong or weak light conditions on the vehicle visual perception data, etc. A process to enhance the clarity and precision of in-vehicle visual perception data.
- the vehicle perception data may also include vehicle navigation data and vehicle status data.
- vehicle navigation data can be obtained through a navigation device configured in the vehicle, and vehicle status data can be obtained from various sensors (such as vehicle speed sensors, acceleration sensors, etc.) and/or meters of the vehicle through a vehicle network device.
- the above step S104 may further specifically include the following step S306: performing time synchronization on the vehicle visual perception data, vehicle navigation data and vehicle status data to form structured vehicle perception data, wherein each structured vehicle perception data
- the vehicle perception data includes a timestamp, a perception image at the current moment, the vehicle position at the current moment, and vehicle dynamics parameters at the current moment.
- Vehicle dynamics parameters may include vehicle speed, steering wheel angle, acceleration/deceleration, and the like.
- step S308 the pre-trained deep neural network is used to filter the pre-processed vehicle perception data according to preset screening rules to obtain incremental data, wherein the preset screening rules include high At least one of the comparison and screening rules of precision map and perceptual data, the comparison and screening rules of predicted data and perceptual data, the screening rules of cloud commands, and the screening rules of adjacent frame image differences.
- preset screening rules include high At least one of the comparison and screening rules of precision map and perceptual data, the comparison and screening rules of predicted data and perceptual data, the screening rules of cloud commands, and the screening rules of adjacent frame image differences.
- a pre-trained neural network (specifically a deep neural network) is run to identify each frame of structured vehicle perception data to obtain identification data.
- the forward distance between the vehicle and the marked target road infrastructure in the high-precision map is less than the first threshold, it is determined whether the marked target road infrastructure is included in the identification data at the current moment. If not, take the structured vehicle perception data of a specified number of frames before and after the structured vehicle perception data at the current moment as the center as the incremental data.
- a high-definition map can be imported by a vehicle-configured navigation device, and the road infrastructure (such as traffic signal poles, road static barricades, etc.) in the high-definition map can be marked.
- the first threshold can be set according to actual application requirements, for example, can be set to 50m.
- the specified number can be set according to the accuracy requirements of the actual application. For example, it can be set to 20.
- the adjacent 20 frames of structured vehicle perception data including the structured vehicle perception data at the current moment are used as the increment. volume data.
- This method can be applied to the comparison and screening of static data, and can filter out the vehicle perception data that the deep neural network cannot accurately identify. It is more effective for training the deep neural network and improving the perception ability of the deep neural network.
- the second the comparison of the prediction data and the perceptual data filtering rules.
- the first position of the obstacle vehicle in the first number of frames from the current moment is predicted, and the structured vehicle perception based on the second number of frames in the future from the current moment is passed through the pre-trained neural network.
- the data obtains the second position of the obstacle vehicle in a second number of frames in the future from the current moment of actual perception, where the second number is greater than or equal to the first number.
- it is determined whether the prediction data is consistent with the perception data. If not, take the structured vehicle perception data of a specified number of frames before and after the structured vehicle perception data at the current moment as the center as the incremental data.
- formula (1) represents the actual perceived second position of the obstacle vehicle in the i-th frame in the future, represents the predicted first position of the obstacle vehicle in the ith frame in the future, and n is equal to the first number. Then, it is determined whether the calculated distance difference d is greater than the second threshold dmin, and if so, it can be determined that the predicted data is inconsistent with the sensing data.
- the first number and the second number may be set according to actual application requirements.
- the first number may generally be set to 8 frames, and the second number may be set to 20 frames. If it is predicted that the obstacle vehicle will exceed the perceived field of view of the visual system of the vehicle in the next 8 frames, the first number may be set as the number of frames within the perceived field of view of the visual system.
- the frame All are set to a large value (for example, 1000) to ensure that the sensory data can be filtered out.
- the second threshold dmin can be set according to practical applications, which is not limited in the present invention.
- This method can be applied to the screening of non-truth-valued perception data, and can filter out the vehicle perception data that is inconsistent with the predicted data. It is more effective for training deep neural networks and improving the perception capabilities of deep neural networks.
- the third type cloud command filtering rules.
- the structured vehicle perception data is screened according to the data screening command issued by the cloud to obtain incremental data.
- data screening instructions such as collecting vehicle perception data when the braking deceleration is greater than a threshold, collecting vehicle perception data at traffic light intersections, etc.
- the command is sent to the vehicle, and the vehicle executes the data screening command received from the cloud.
- the fourth type filtering rules for image differences between adjacent frames.
- the difference index between the perceptual images in the structured vehicle perception data of the adjacent previous frame and the next frame is calculated first. Then, it is judged whether the difference index is greater than the third threshold, and if so, the structured vehicle perception data of the next frame is taken as the incremental data of a specified number of frames adjacent to the center.
- the third threshold can be set according to actual application requirements, which is not limited in the present invention.
- the limit of the specified number is as described above. For example, when the specified number is set to 20, when the difference index of the perceptual images of the previous and subsequent frames is greater than the third threshold, the front and rear images including the structured vehicle perceptual data of the subsequent frame will be included.
- the adjacent 20 frames of structured vehicle perception data are used as incremental data.
- the above-mentioned difference indicator may be the mean square error (Mean Square Error, MSE) or the structural similarity (Structural Similarity, SSIM) between the two frames of perceptual images.
- MSE mean Square Error
- SSIM structural similarity
- the mean square error MSE between the perceptual images in the structured vehicle perception data of the previous frame and the next frame can be calculated according to the following formula (2):
- It (i, j ) and It +1 (i, j) represent the pixel values of two adjacent frames of perceptual images at time t and time t+1, respectively, and i and j represent the pixel values in the perceptual image, respectively.
- the pixel coordinates of , m and n represent the length and width of the perceived image size, respectively.
- the structural similarity SSIM between the perceptual images in the structured vehicle perception data of the previous frame and the following frame can be calculated according to the following formula (3):
- x and y represent the pixel values of two adjacent frames of perceptual images
- ⁇ x is the mean of x
- ⁇ y is the mean of y
- ⁇ x is the variance of x
- ⁇ y is the variance of y
- ⁇ xy is the covariance of x and y
- c 1 and c 2 are constants to maintain stability
- c 1 (k 1 *L) 2
- c 2 (k 2 *L) 2
- L is the dynamic range of pixel values.
- This method can more effectively improve the perception ability of the deep neural network by filtering out the vehicle perception data whose image difference between adjacent frames exceeds the threshold for neural network training.
- step S310 is the same as the previous step S108, and will not be repeated here.
- the collected vehicle driving data is screened by adopting at least one of the comparison and screening rules of high-precision maps and perceptual data, the comparison and screening rules of predicted data and perceptual data, the cloud command screening rules, the adjacent frame image difference screening rules, and the like. More accurate and valuable incremental data is available, resulting in a dramatic reduction in the amount of sensed data stored or transferred.
- DRL deep reinforcement learning
- the method for automatic driving data collection and closed-loop management may include the following steps S402 to S416 .
- Step S402 acquiring vehicle control data, wherein the vehicle control data includes current vehicle state data S t and the current control action a human of the driver of the vehicle.
- the current vehicle state data S t may include current environment information (eg, image information, grid map, etc.), current vehicle speed, current vehicle steering angle, and the like.
- the driver's current control action a human may include the first decision command, steering angle, accelerator pedal/brake pedal opening, etc.
- Vehicle control data can be obtained through vehicle-configured vision systems (such as cameras), navigation equipment, and vehicle network equipment.
- Step S404 preprocessing the vehicle control data.
- the existing common processing methods can be used to preprocess the vehicle control data, and the specific selection can be made according to the actual application, which will not be repeated in this paper.
- step S106 may be specifically implemented as the following steps S406 to S414.
- Step S406 collecting the current control action an nn output by the pre-trained deep reinforcement learning neural network based on the current vehicle state data S t .
- the current control action a nn output by the deep reinforcement learning neural network may include a second decision command, a steering command, a speed command, and the like.
- Step S408 Calculate the difference between the current control action a nn output by the deep reinforcement learning neural network and the driver's current control action a human .
- Step S410 determine whether the difference is greater than a fourth threshold. If yes, step S412 is executed.
- Step S412 determine the reward and punishment function value r(t) of the deep reinforcement learning neural network under the current vehicle state data S t and the current control action a nn output by the deep reinforcement learning neural network.
- the reward and punishment function r(t) of the deep reinforcement learning neural network is calculated according to the following formula (4):
- R ca (t) represents the Collision Avoidance indicator
- R ma (t) represents the Mission Achievement indicator
- R lk (t) represents the Lane Keeping indicator
- R speed (t) ) denotes the speed rationality index
- R steer (t) denotes the steering rationality index
- R ca (t) R ma (t), R lk (t), R speed (t) and R steer (t) are the current Under the vehicle state data S t , it is assumed that the current control action a nn output by the deep reinforcement learning neural network is applied to the control system of the vehicle .
- Each weight coefficient represents the degree of attention to the corresponding indicator, which can be set according to experience.
- Step S414 use the current vehicle state data S t , the current control action a nn output by the deep reinforcement learning neural network, the driver's current control action a human , the reward and punishment function value r(t) and the vehicle state data S t at the next moment +1 for constructing the packet as incremental data.
- the vehicle state data S t+1 at the next moment refers to the state data that the vehicle state data S t is transformed into after the vehicle is subjected to the current control action a human of the driver.
- the form of the data packet constructed in this step may be [S t , a nn , a human , r(t), S t+1 ].
- step S416 the incremental data is stored, or the incremental data is sent to the cloud.
- step S416 is the same as that of the foregoing step S108, and will not be repeated here.
- the trained deep reinforcement learning neural network issued by the cloud can also be received and deployed to update the pre-trained deep reinforcement learning neural network, wherein the trained deep reinforcement learning neural network is selected after screening.
- a deep reinforcement learning neural network trained on incremental data of vehicle control data is selected after screening.
- This embodiment performs incremental data collection and closed-loop management of vehicle control data, which can greatly reduce system data storage and/or data transmission bandwidth requirements, improve data processing efficiency, and improve the perception and decision-making capability of deep reinforcement learning neural networks. .
- FIG. 5 shows a schematic structural diagram of an automatic driving data collection and closed-loop management system 10 according to an embodiment of the present invention.
- the automatic driving data collection and closed-loop management system 10 may at least include a data acquirer 100 and a data processor 200 .
- the data acquirer 100 is configured to acquire vehicle driving data.
- the data processor 200 may include a data preprocessing module 201 and a data screening module 202 .
- the data preprocessing module 201 is configured to preprocess the vehicle driving data.
- a pre-trained neural network is deployed in the data screening module 202, and is configured to screen the pre-processed vehicle driving data according to preset screening rules to obtain incremental data through the pre-trained neural network.
- the data processor 200 may be a GPU (Graphics Processing Unit, graphics processor) that can run a neural network (eg, DNN).
- the automatic driving data collection and closed-loop management system 10 may further include: a memory 300 configured to store incremental data; and/or a sending module 400 configured to send incremental data to the cloud. Specifically, the sending module 400 sends the incremental data to the cloud through the network communication module 500 (for example, a T-box, etc.).
- the data screening module 202 may also be configured to receive and deploy the trained neural network delivered by the cloud to update the pre-trained neural network, wherein the trained neural network is a neural network trained with incremental data network.
- the data acquirer 100 may include a vision system 101 (specifically, one or more in-vehicle cameras), configured to acquire in-vehicle visual perception data.
- the data preprocessing module 201 may also be configured to perform at least one of the following processing on the vehicle-mounted visual perception data: deformation recovery, noise reduction, deblurring, white balance, image complement and recovery under strong or weak light conditions.
- the data acquirer 100 may further include a navigation device 102 and a vehicle network device 103 .
- the navigation device 102 may be, for example, an integrated navigation system configured to acquire vehicle navigation data.
- the vehicle network device 103 can be, for example, a vehicle CAN network bus, a vehicle Ethernet bus, etc.
- the vehicle status data is obtained from various sensors of the vehicle (eg, vehicle speed sensor, acceleration sensor, etc.) and/or meters through the vehicle network device 103 .
- the data processor 200 may further include a data synchronization module 203 configured to perform time synchronization on the vehicle visual perception data, the vehicle navigation data and the vehicle status data to form structured vehicle perception data, wherein each structured vehicle perception data includes: Timestamp, perceived image at the current moment, vehicle position at the current moment, and vehicle dynamics parameters at the current moment.
- a data synchronization module 203 configured to perform time synchronization on the vehicle visual perception data, the vehicle navigation data and the vehicle status data to form structured vehicle perception data, wherein each structured vehicle perception data includes: Timestamp, perceived image at the current moment, vehicle position at the current moment, and vehicle dynamics parameters at the current moment.
- the data processor 200 may further include a data cache module 204 configured to cache the preprocessed vehicle driving data.
- the storage 300 may be a vehicle-mounted mobile hard disk. Due to the large capacity of the on-board mobile hard disk, it can store a large amount of incremental data obtained by screening, and then copy and transfer from the on-board mobile hard disk to the cloud computing center in batches, so that the image or video data collected by this system can be independent of the bandwidth of the wireless network. to transmit.
- the data screening module 202 may be configured to perform the data screening operation described in any of the foregoing embodiments or a combination of embodiments, which will not be repeated herein.
- the automatic driving data collection and closed-loop management system 10 provided by the embodiment of the present invention can be installed on the vehicle in a pre-installed (that is, when the vehicle is produced) or post-installed (that is, installed after the vehicle leaves the factory).
- the vehicle is equipped with a high-cost truth-value system such as lidar, and the transmission of image or video data does not depend on the bandwidth of the wireless network, which greatly improves the convenience of the system and is beneficial to the automatic driving data collection and closed-loop management system 10 Fast and large. scale application.
- the automatic driving data collection and closed-loop management system 10 in this embodiment is a front-mounted system, which reduces the sending module 400 compared to FIG. 6 , the vision system 101 uses one or more cameras, and the storage 300 is a vehicle-mounted mobile hard disk.
- the visual perception data obtained by the camera, the vehicle navigation data obtained by the navigation device 102 and the vehicle state data obtained by the vehicle network device 103 are the inputs of the data processor 200 .
- the pre-trained DNN is deployed in the data screening module 202 of the data processor 200, and the DNN can be upgraded through OTA.
- the data screening module 202 uses the high-precision map and sensory data to compare and filter rules, prediction data and sensory data to compare and filter rules, cloud command screening rules, Adjacent frame image difference screening rules and other rules are used to filter data, and the filtered structured perception data is saved to the on-board mobile hard disk.
- the DNN result output in the data processor 200 can also be used as one of the input sources of the intelligent driving system of the vehicle, so as to improve the reuse rate of the system.
- the vehicle collects and selects incremental data in different scenarios.
- the incremental data collected and screened by the vehicle is first processed and classified in the computing center, and the scene can be classified according to information such as location, vehicle speed, and acceleration. Of course, data under specific conditions can also be collected according to the filtering instruction conditions set by the cloud.
- the incremental data is annotated, and the DNN is incrementally trained using the annotated incremental data to improve the performance of the DNN.
- the iteratively updated DNN network is regularly deployed to the data processor 200 of the vehicle by means of OTA, and then a new round of incremental data collection is performed, thereby forming a data collection and closed-loop management system.
- the automatic driving data collection and closed-loop management method and system in this embodiment are used for the collection of automatic driving control incremental data for training the DRL neural network.
- the automatic driving data collection and closed-loop management system 10 is installed on the automatic driving test vehicle, and a pre-trained DRL neural network is deployed in the data processor 200 installed on the automatic driving test vehicle, and the deployed DRL
- the neural network runs in the background, that is, the action output by the DRL neural network does not directly affect the real vehicle control.
- the automatic driving data collection and closed-loop management system 10 detects the difference between the output action of the DRL neural network and the action of the human driver in real time, and obtains the incremental control data whose difference exceeds the fourth threshold. t , a nn , a human , r, S t+1 ] for storage.
- the calculation and judgment of the difference between the output action of the DRL neural network and the action of the human driver, and the construction of the incremental control data data package are as described above, and will not be repeated here.
- the DRL neural network is trained with the incremental control data.
- the trained DRL neural network is regularly deployed to the test vehicle through OTA to form an incremental data closed-loop management system with the goal of improving the KPI of the DRL neural network control capability.
- the embodiments of the present invention can achieve the following beneficial effects:
- the method and system for automatic driving data collection and closed-loop management provided by the embodiments of the present invention, after the pre-processed vehicle driving data is filtered through a pre-trained neural network according to preset screening rules to obtain incremental data, the incremental data is then stored. data or send incremental data to the cloud for neural network training.
- the solution of the present invention can effectively filter out high-value incremental data, thereby reducing the system data storage capacity and/or the requirement for data transmission bandwidth.
- the system of the present invention can be installed on the vehicle in a rear-mounted or front-mounted manner, does not depend on a specific vehicle model, and does not require the vehicle to be equipped with a high-cost real-value system such as lidar, which greatly improves the convenience of the system and is conducive to rapid and large-scale application.
- the solution of the present invention adopts at least one of the comparison and screening rules of high-precision maps and perceptual data, the comparison and screening rules of predicted data and perceptual data, the screening rules of cloud commands, the screening rules of adjacent frame image differences, etc. to collect the vehicle driving data. By filtering, more accurate and valuable incremental data can be obtained, which can greatly reduce the amount of perception data storage or transmission.
- the scheme of the present invention trains the neural network of the perception and decision-making system of automatic driving by using the filtered incremental data, and deploys the trained neural network to update the pre-trained neural network for data collection and screening, thereby forming a Incremental data closed-loop management with the goal of improving neural network perception capabilities.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Molecular Biology (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Mechanical Engineering (AREA)
- Transportation (AREA)
- Human Computer Interaction (AREA)
- Automation & Control Theory (AREA)
- Traffic Control Systems (AREA)
Abstract
Description
Claims (23)
- 一种自动驾驶数据采集和闭环管理方法,应用于车辆端,包括:获取车辆驾驶数据;对所述车辆驾驶数据进行预处理;通过预训练的神经网络根据预设筛选规则对预处理后的所述车辆驾驶数据进行筛选得到增量数据;存储所述增量数据,或将所述增量数据发送至云端。
- 根据权利要求1所述的方法,其中,在存储所述增量数据之后,还包括:接收并部署云端下发的训练后的神经网络以更新所述预训练的神经网络,其中,所述训练后的神经网络为经过所述增量数据训练后的神经网络。
- 根据权利要求1或2所述的方法,其中,所述车辆驾驶数据包括车辆感知数据,所述神经网络为深度神经网络。
- 根据权利要求3所述的方法,其中,所述车辆感知数据包括车载视觉感知数据;所述对所述车辆驾驶数据进行预处理包括:对所述车载视觉感知数据进行下列至少之一的处理:变形恢复、降噪、去模糊、白平衡、强光或弱光条件下的图像补足和恢复。
- 根据权利要求4所述的方法,其中,所述车辆感知数据还包括车辆导航数据和车辆状态数据;所述对所述车辆驾驶数据进行预处理还包括:对所述车载视觉感知数据、所述车辆导航数据和所述车辆状态数据进行时间同步,形成结构化的车辆感知数据,其中,每一所述结构化的车辆感知数据包括时间戳、当前时刻的感知图像、当前时刻的车辆位置以及当前时刻的车辆动力学参数。
- 根据权利要求5所述的方法,其中,所述预设筛选规则包括下列至少之一:高精地图与感知数据对比筛选规则、预测数据与感知数据对比筛选规则、云端指令筛选规则、相邻帧图像差异筛选规则。
- 根据权利要求6所述的方法,其中,所述高精地图与感知数据对比筛 选规则包括:运行所述预训练的神经网络对所述结构化的车辆感知数据进行识别得到识别数据;当车辆距离高精地图中已标记的目标道路基础设施的前向距离小于第一阈值时,判断当前时刻的所述识别数据中是否包括所述已标记的目标道路基础设施;若否,将以当前时刻的所述结构化的车辆感知数据为中心前后相邻的指定数量帧的结构化的车辆感知数据作为所述增量数据。
- 根据权利要求6所述的方法,其中,所述预测数据与感知数据对比筛选规则包括:预测当前时刻起未来第一数量帧中障碍车辆的第一位置;通过所述预训练的神经网络基于当前时刻起未来第二数量帧的所述结构化的车辆感知数据得到实际感知的当前时刻起未来第二数量帧中所述障碍车辆的第二位置,所述第二数量大于或等于所述第一数量;根据未来每一帧中第一位置与对应帧中的第二位置判断预测数据与感知数据是否一致;若否,将以当前时刻的所述结构化的车辆感知数据为中心前后相邻的指定数量帧的结构化的车辆感知数据作为所述增量数据。
- 根据权利要求6所述的方法,其中,所述云端指令筛选规则包括:根据云端下发的数据筛选指令对所述结构化的车辆感知数据进行筛选得到所述增量数据。
- 根据权利要求6所述的方法,其中,所述相邻帧图像差异筛选规则 包括:计算相邻的前一帧和后一帧的所述结构化的车辆感知数据中所述感知图像之间的差异指标;判断所述差异指标是否大于第三阈值;若是,则将以所述后一帧的所述结构化的车辆感知数据为中心前后相邻的指定数量帧的结构化的车辆感知数据作为所述增量数据。
- 根据权利要求11所述的方法,其中,所述差异指标为均方误差;所述计算相邻的前一帧和后一帧的所述结构化的车辆感知数据中所述感知图像之间的差异指标包括:根据下式(2)计算前一帧和后一帧的所述结构化的车辆感知数据中所述感知图像之间的均方误差MSE:式(2)中,I t(i,j)、I t+1(i,j)分别表示t时刻和t+1时刻相邻两帧感知图像的像素值,i和j分别表示所述感知图像中的像素点坐标,m和n分别表示所述感知图像尺寸的长度和宽度;或者所述差异指标为结构相似性;所述计算相邻的前一帧和后一帧的所述结构化的车辆感知数据中所述感知图像之间的差异指标包括:根据下式(3)计算前一帧和后一帧的所述结构化的车辆感知数据中所述感知图像之间的结构相似性SSIM:式(3)中,x和y分别表示相邻两帧感知图像的像素值,μ x是x的均值,μ y是y的均值,σ x是x的方差,σ y是y的方差,σ xy是x和y的协方差;c 1和c 2为维持稳定的常数,c 1=(k 1*L) 2,c 2=(k 2*L) 2,其中,k 1和k 2为常数,L是像素值的动态范围。
- 根据权利要求1或2所述的方法,其中,所述车辆驾驶数据包括车辆控制数据,所述神经网络为深度强化学习神经网络。
- 根据权利要求13所述的方法,其中,所述车辆控制数据包括当前车辆状态数据S t和所述车辆的驾驶员的当前控制动作a human;所述通过预训练的神经网络根据预设筛选规则对预处理后的所述车辆 驾驶数据进行筛选得到增量数据包括:采集所述预训练的深度强化学习神经网络基于所述当前车辆状态数据S t输出的当前控制动作a nn;计算所述深度强化学习神经网络输出的当前控制动作a nn与所述驾驶员的当前控制动作a human之间的差异;判断所述差异是否大于第四阈值;若是,则确定所述当前车辆状态数据S t和所述深度强化学习神经网络输出的当前控制动作a nn下所述深度强化学习神经网络的赏罚函数值r(t);以所述当前车辆状态数据S t、所述深度强化学习神经网络输出的当前控制动作a nn、所述驾驶员的当前控制动作a human、所述赏罚函数值r(t)和下一时刻的车辆状态数据S t+1构造数据包作为所述增量数据。
- 根据权利要求14所述的方法,其中,所述确定所述当前车辆状态数据S t和所述深度强化学习神经网络输出的当前控制动作a nn下所述深度强化学习神经网络的赏罚函数值r(t),包括:根据下式(4)计算所述深度强化学习神经网络的赏罚函数r(t):r(t)=R ca(t)+b 1R ma(t)+b 2R lk(t)+b 3R speed(t)+b 4R steer(t) (4)其中,R ca(t)表示避免碰撞指标,R ma(t)表示任务完成度指标,R lk(t)表示车道保持能力指标,R speed(t)表示速度合理性指标,R steer(t)表示转向合理性指标,R ca(t)、R ma(t)、R lk(t)、R speed(t)和R steer(t)是在所述当前车辆状态数据S t下假设对所述车辆的控制***施加所述深度强化学习神经网络输出的当前控制动作a nn后计算得到,b 1、b 2、b 3、b 4表示权重系数。
- 根据权利要求14所述的方法,其中,所述当前车辆状态数据S t包括下列至少之一:当前环境信息、当前车速、当前车辆转向角度;所述驾驶员的当前控制动作a human包括下列至少之一:第一决策指令、转向角度、加速踏板/制动踏板开度;所述深度强化学习神经网络输出的当前控制动作a nn包括下列至少之一:第二决策指令、转向指令、速度指令。
- 根据权利要求1所述的方法,其中,在通过预训练的神经网络根据预设筛选规则对预处理后的所述车辆驾驶数据进行筛选得到增量数据之前,还包括:缓存预处理后的所述车辆驾驶数据。
- 一种自动驾驶数据采集和闭环管理***,应用于车辆端,包括:数据获取器,配置为获取车辆驾驶数据;数据处理器,包括:数据预处理模块,配置为对所述车辆驾驶数据进行预处理;以及数据筛选模块,其上部署有预训练的神经网络,配置为通过所述预训练的神经网络根据预设筛选规则对预处理后的所述车辆驾驶数据进行筛选得到增量数据;以及存储器,配置为存储所述增量数据;和/或发送模块,配置为将所述增量数据发送至云端。
- 根据权利要求18所述的***,其中,所述数据筛选模块还配置为:接收并部署云端下发的训练后的神经网络以更新所述预训练的神经网络,其中,所述训练后的神经网络为经过所述增量数据训练后的神经网络。
- 根据权利要求18或19所述的***,其中,所述数据获取器包括一个或多个车载摄像头,所述一个或多个车载摄像头配置为获取车载视觉感知数据;所述数据预处理模块还配置为:对所述车载视觉感知数据进行下列至少之一的处理:变形恢复、降噪、去模糊、白平衡、强光或弱光条件下的图像补足和恢复。
- 根据权利要求20所述的***,其中,所述数据获取器还包括:导航设备,配置为获取车辆导航数据;和整车网络设备,配置为获取车辆状态数据;并且所述数据处理器还包括数据同步模块,配置为对所述车载视觉感知数据、所述车辆导航数据和所述车辆状态数据进行时间同步,形成结构化的车辆感知数据,其中,每一所述结构化的车辆感知数据包括时间戳、当前时刻的感知图像、当前时刻的车辆位置以及当前时刻的车辆动力学参数。
- 根据权利要求18所述的***,其中,所述数据处理器还包括:数据缓存模块,配置为缓存预处理后的所述车辆驾驶数据。
- 根据权利要求18所述的***,其中,所述存储器为车载移动硬盘。
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP20963836.0A EP4257443A1 (en) | 2020-12-01 | 2020-12-01 | Method and system for automatic driving data collection and closed-loop management |
US18/255,360 US20230419683A1 (en) | 2020-12-01 | 2020-12-01 | Method and system for automatic driving data collection and closed-loop management |
PCT/CN2020/133062 WO2022115987A1 (zh) | 2020-12-01 | 2020-12-01 | 一种自动驾驶数据采集和闭环管理的方法及*** |
CN202080107696.1A CN116601065A (zh) | 2020-12-01 | 2020-12-01 | 一种自动驾驶数据采集和闭环管理的方法及*** |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2020/133062 WO2022115987A1 (zh) | 2020-12-01 | 2020-12-01 | 一种自动驾驶数据采集和闭环管理的方法及*** |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022115987A1 true WO2022115987A1 (zh) | 2022-06-09 |
Family
ID=81853659
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/133062 WO2022115987A1 (zh) | 2020-12-01 | 2020-12-01 | 一种自动驾驶数据采集和闭环管理的方法及*** |
Country Status (4)
Country | Link |
---|---|
US (1) | US20230419683A1 (zh) |
EP (1) | EP4257443A1 (zh) |
CN (1) | CN116601065A (zh) |
WO (1) | WO2022115987A1 (zh) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2024119412A1 (zh) * | 2022-12-07 | 2024-06-13 | 魔门塔(苏州)科技有限公司 | 数据处理方法、装置、计算机设备、车辆和可读存储介质 |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2022138728A (ja) * | 2021-03-10 | 2022-09-26 | トヨタ自動車株式会社 | 運転診断装置及び運転診断方法 |
CN117681893B (zh) * | 2024-02-02 | 2024-05-14 | 中国科学院自动化研究所 | 面向矿区场景的自动驾驶数据构建方法和装置 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108909833A (zh) * | 2018-06-11 | 2018-11-30 | 中国科学院自动化研究所 | 基于策略迭代的智能汽车转向控制方法 |
CN109873921A (zh) * | 2019-02-14 | 2019-06-11 | 初速度(苏州)科技有限公司 | 一种图像时间戳的传输方法、解析方法和装置 |
CN110188683A (zh) * | 2019-05-30 | 2019-08-30 | 北京理工大学 | 一种基于cnn-lstm的自动驾驶控制方法 |
US20190384303A1 (en) * | 2018-06-19 | 2019-12-19 | Nvidia Corporation | Behavior-guided path planning in autonomous machine applications |
CN110796157A (zh) * | 2019-08-29 | 2020-02-14 | 腾讯科技(深圳)有限公司 | 一种图像差异识别方法、装置及存储介质 |
-
2020
- 2020-12-01 EP EP20963836.0A patent/EP4257443A1/en active Pending
- 2020-12-01 WO PCT/CN2020/133062 patent/WO2022115987A1/zh active Application Filing
- 2020-12-01 US US18/255,360 patent/US20230419683A1/en active Pending
- 2020-12-01 CN CN202080107696.1A patent/CN116601065A/zh active Pending
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108909833A (zh) * | 2018-06-11 | 2018-11-30 | 中国科学院自动化研究所 | 基于策略迭代的智能汽车转向控制方法 |
US20190384303A1 (en) * | 2018-06-19 | 2019-12-19 | Nvidia Corporation | Behavior-guided path planning in autonomous machine applications |
CN109873921A (zh) * | 2019-02-14 | 2019-06-11 | 初速度(苏州)科技有限公司 | 一种图像时间戳的传输方法、解析方法和装置 |
CN110188683A (zh) * | 2019-05-30 | 2019-08-30 | 北京理工大学 | 一种基于cnn-lstm的自动驾驶控制方法 |
CN110796157A (zh) * | 2019-08-29 | 2020-02-14 | 腾讯科技(深圳)有限公司 | 一种图像差异识别方法、装置及存储介质 |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2024119412A1 (zh) * | 2022-12-07 | 2024-06-13 | 魔门塔(苏州)科技有限公司 | 数据处理方法、装置、计算机设备、车辆和可读存储介质 |
Also Published As
Publication number | Publication date |
---|---|
EP4257443A1 (en) | 2023-10-11 |
US20230419683A1 (en) | 2023-12-28 |
CN116601065A (zh) | 2023-08-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2022115987A1 (zh) | 一种自动驾驶数据采集和闭环管理的方法及*** | |
CN112700470B (zh) | 一种基于交通视频流的目标检测和轨迹提取方法 | |
EP4152204A1 (en) | Lane line detection method, and related apparatus | |
JP6946812B2 (ja) | 学習サーバ及び支援システム | |
CN106926800A (zh) | 多摄像头适配的车载视觉感知*** | |
CN111324120A (zh) | 一种自动驾驶前车切入切出场景提取方法 | |
US20220396281A1 (en) | Platform for perception system development for automated driving system | |
CN111753639B (zh) | 感知地图生成方法、装置、计算机设备和存储介质 | |
CN113643431A (zh) | 一种用于视觉算法迭代优化的***及方法 | |
CN109874099A (zh) | 一种联网车载设备流量控制*** | |
DE112020003708T5 (de) | Informationsverarbeitungsvorrichtung, Informationsverarbeitungsverfahren und Programm | |
CN113034378A (zh) | 一种区分电动汽车与燃油汽车的方法 | |
CN117416349A (zh) | V2X环境下基于改进YOLOV7-Tiny和SS-LSTM的自动驾驶风险预判***及方法 | |
CN117115690A (zh) | 一种基于深度学习和浅层特征增强的无人机交通目标检测方法及*** | |
US11983918B2 (en) | Platform for perception system development for automated driving system | |
CN116665166A (zh) | 一种适用于不平坦路面场景的智能车辆3d目标检测方法 | |
CN116524311A (zh) | 路侧感知数据处理方法及其***、存储介质和电子设备 | |
US20230086088A1 (en) | Vehicle onboard apparatus, image recognition system, support apparatus for vehicle onboard apparatus, image recognition method, and model data transmission method | |
US20240135252A1 (en) | Lane-assignment for traffic objects on a road | |
US20240046790A1 (en) | Augmented path planning for automotive applications | |
EP4394626A1 (en) | Classification of objects present on a road | |
US20230109494A1 (en) | Methods and devices for building a training dataset | |
US20230084623A1 (en) | Attentional sampling for long range detection in autonomous vehicles | |
EP4361967A1 (en) | Situation specific perception capability for autonomous and semi-autonomous vehicles | |
CN117332302A (zh) | 一种驾驶场景的分类方法、装置、电子设备及存储介质 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20963836 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 18255360 Country of ref document: US |
|
WWE | Wipo information: entry into national phase |
Ref document number: 202080107696.1 Country of ref document: CN |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 2020963836 Country of ref document: EP Effective date: 20230703 |