CN113490940A - Scene simulator construction method and device based on deep learning and computer equipment - Google Patents
Scene simulator construction method and device based on deep learning and computer equipment Download PDFInfo
- Publication number
- CN113490940A CN113490940A CN202080003157.3A CN202080003157A CN113490940A CN 113490940 A CN113490940 A CN 113490940A CN 202080003157 A CN202080003157 A CN 202080003157A CN 113490940 A CN113490940 A CN 113490940A
- Authority
- CN
- China
- Prior art keywords
- scene
- road condition
- dangerous
- layer
- dangerous road
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000013135 deep learning Methods 0.000 title claims abstract description 67
- 238000010276 construction Methods 0.000 title claims abstract description 21
- 238000004088 simulation Methods 0.000 claims abstract description 182
- 238000012549 training Methods 0.000 claims abstract description 80
- 238000012360 testing method Methods 0.000 claims abstract description 41
- 238000000034 method Methods 0.000 claims description 30
- 230000008569 process Effects 0.000 claims description 11
- 238000000605 extraction Methods 0.000 claims description 10
- 238000013528 artificial neural network Methods 0.000 description 18
- 239000000284 extract Substances 0.000 description 13
- 238000010586 diagram Methods 0.000 description 8
- 238000003062 neural network model Methods 0.000 description 6
- 230000007613 environmental effect Effects 0.000 description 3
- 210000004556 brain Anatomy 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000033001 locomotion Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
- B60W40/06—Road conditions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Mathematical Physics (AREA)
- Biomedical Technology (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Automation & Control Theory (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- Traffic Control Systems (AREA)
Abstract
A scene simulator construction method based on deep learning comprises the following steps: acquiring driving scene data and historical dangerous road condition scene data; extracting various road condition scene characteristics in the driving scene data, and performing deep learning by using a deep learning algorithm according to the various road condition scene characteristics to obtain a scene simulation layer; extracting various dangerous road condition characteristics in historical dangerous road condition scene data, and training an initial countermeasure network according to the various dangerous road condition characteristics to obtain a dangerous road condition layer; continuously training a scene simulator by using the scene simulation layer and the dangerous road condition layer until a preset condition is met, and obtaining a trained scene simulator; the scene simulator is used for generating a simulated driving scene during simulation test.
Description
The application relates to a scene simulator construction method and device based on deep learning and computer equipment.
With the rapid development of internet technology, the unmanned technology is also rapidly developed. In the automatic driving development process, a simulation environment is generally required to be constructed to perform simulated driving under various driving conditions. Some road condition information, such as simulation information of lanes, vehicles, pedestrians and the like, can be randomly generated through the automatic driving simulator.
In a traditional mode, a simulation test scene is usually generated by adopting a plurality of preset scenes so as to pass through a road simulation test or a simulator for carrying out a virtual simulation test on a vehicle by vehicle simulation software, but the simulation test scene generated by the simulator cannot really simulate the real reaction of the vehicle in a corresponding environment, and the reality and reliability of the generated simulated driving scene are lower, so that the simulation test efficiency is low, and the accuracy of the test result is not high.
Disclosure of Invention
According to various embodiments disclosed in the application, a scene simulator construction method, a scene simulator construction device and computer equipment based on deep learning are provided.
A scene simulator construction method based on deep learning comprises the following steps:
acquiring driving scene data and historical dangerous road condition scene data;
extracting various road condition scene characteristics in the driving scene data, and performing deep learning by using a deep learning algorithm according to the various road condition scene characteristics to obtain a scene simulation layer;
extracting various dangerous road condition characteristics in historical dangerous road condition scene data, and training an initial countermeasure network according to the various dangerous road condition characteristics to obtain a dangerous road condition layer; and
continuously training a scene simulator by using the scene simulation layer and the dangerous road condition layer until a preset condition is met, and obtaining a trained scene simulator; the scene simulator is used for generating a simulated driving scene during simulation test.
A scene simulator building device based on deep learning comprises the following components:
the data acquisition module is used for acquiring driving scene data and historical dangerous road condition scene data;
the scene simulation training module is used for extracting various road condition scene characteristics in the driving scene data and carrying out deep learning by utilizing a deep learning algorithm according to the various road condition scene characteristics to obtain a scene simulation layer;
the dangerous road condition training module is used for extracting various dangerous road condition characteristics in historical dangerous road condition scene data, and training the initial countermeasure network according to the various dangerous road condition characteristics to obtain a dangerous road condition layer; and
the scene simulator construction module is used for continuously training the scene simulator by utilizing the scene simulation layer and the dangerous road condition layer until a preset condition is met, and obtaining a trained scene simulator; the scene simulator is used for generating a simulated driving scene during simulation test.
A computer arrangement comprising a memory storing a computer program and a processor implementing the steps of a remote take-over based vehicle control method as provided in any one of the embodiments of the application when the computer program is executed.
One or more non-transitory computer-readable storage media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to implement the steps of a remote takeover-based vehicle control method provided in any one of the embodiments of the present application when the readable storage media is executed.
The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below. Other features and advantages of the application will be apparent from the description and drawings, and from the claims.
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
Fig. 1 is a diagram of an application scenario of a scene simulator construction method based on deep learning according to one or more embodiments.
FIG. 2 is a schematic flow diagram of a method for building a scene simulator based on deep learning in accordance with one or more embodiments.
FIG. 3 is a flow diagram illustrating steps of training a scene simulation layer according to one or more embodiments.
FIG. 4 is a flowchart illustrating a step of training a dangerous road condition layer according to one or more embodiments.
FIG. 5 is a flowchart illustrating the steps of testing with a scene simulator in another embodiment.
Fig. 6 is a block diagram of a deep learning based scene simulator building apparatus according to one or more embodiments.
Fig. 7 is a block diagram of a scene simulator building device based on deep learning in another embodiment.
FIG. 8 is a block diagram of a computer device in accordance with one or more embodiments.
In order to make the technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The scene simulator construction method based on deep learning can be applied to the application environment shown in fig. 1. The server 102 communicates with the vehicle 104 over a network. After acquiring the driving scene data and the historical dangerous road condition scene data, the server 102 extracts various road condition scene characteristics in the driving scene data, and performs deep learning by using a deep learning algorithm according to the various road condition scene characteristics to obtain a scene simulation layer; and extracting various dangerous road condition characteristics in the historical dangerous road condition scene data, and training the initial countermeasure network according to the various dangerous road condition characteristics to obtain a dangerous road condition layer. The server 102 continuously trains the scene simulator by using the scene simulation layer and the dangerous road condition layer until a preset condition is met, and then the trained scene simulator is obtained. The server 102, in turn, generates simulated driving scenarios when performing simulated testing of the vehicle 104 using the scenario simulator. Server 102 may be implemented as a stand-alone server or as a server cluster of multiple servers, and vehicle 104 may be a variety of autonomous vehicles.
In one embodiment, as shown in fig. 2, a scene simulator construction method based on deep learning is provided, which is described by taking the method as an example applied to the server in fig. 1, and includes the following steps:
The driving scene data may be various road environment data collected in advance, and may include historical driving record data of the vehicle, such as road condition data collected by a vehicle data recorder of the vehicle, for example. The driving scene data comprises various road types, driving environment factors and the like, for example, the road types can comprise urban roads, special roads, rural roads and the like; the driving environment factors may include various environmental factors such as weather, air quality, temperature, noise, and illumination brightness. The driving scene data also comprises various scene information such as ground roads, lane lines, signal lights, landmark buildings and traffic participants, wherein the traffic participants can comprise vehicles to and from, pedestrians, motion paths and the like.
The historical dangerous road condition scene data can be historical data of various dangerous road conditions collected from one or more platforms, and the historical dangerous road condition data can be road condition scene data in a real dangerous scene. The types of risk factors may include a variety of factors, such as a barrier factor, a traffic regulation factor, a driveway vehicle factor, a pedestrian factor, an environmental factor, and the like.
The server can acquire a large amount of driving scene data and historical dangerous road condition scene data from a local database or a third-party database in advance so as to construct and train the scene simulator. The scene simulator can be a neural network model based on deep learning. The deep learning mode can be used for forming more abstract high-level representation attribute categories or features by combining low-level features to find distributed feature representations of data, so as to establish a neural network simulating the human brain for analysis and learning, and the data, such as images, sounds, texts and the like, are interpreted by a mechanism simulating the human brain. The scene simulator can also simulate the functions of the hardware processor through software, so that the computer can simulate the environment of the hardware processor.
And step 204, extracting various road condition scene characteristics in the driving scene data, and performing deep learning by using a deep learning algorithm according to the various road condition scene characteristics to obtain a scene simulation layer.
After the server acquires a large amount of driving scene data and historical dangerous road condition scene data, the characteristics of various road condition scenes in the driving scene data can be extracted, and various road condition scene characteristics in the driving scene data are extracted.
Specifically, the server may input the driving scene data and the historical dangerous road condition scene data into a pre-constructed initial neural network model, and the initial neural network model may be constructed by using a preset deep learning algorithm and a neural network structure. The initial neural network comprises a plurality of levels, for example, a scene simulation level, a dangerous road condition learning level, a dangerous road condition generation level, and the like.
The server extracts the characteristics of various road condition scenes in the driving scene data through the scene simulation layer, and extracts various road condition scene characteristics such as various road characteristics, lane characteristics, signal lamp characteristics, landmark building characteristics, pedestrian characteristics, traffic vehicle characteristics and weather characteristics. And further learning various road condition scene characteristics by the neural network corresponding to the scene simulation level, and generating a scene simulation layer according to the learned various road condition scene characteristics. The scene simulation layer can further randomly generate multiple corresponding models by using the learned multiple road condition scene features, for example, multiple scene models included in road condition scenes such as road models, models and the like, vehicle models, pedestrian models and the like can be automatically generated by the scene simulation layer.
And step 206, extracting various dangerous road condition characteristics in the historical dangerous road condition scene data, and training the initial countermeasure network according to the various dangerous road condition characteristics to obtain a dangerous road condition layer.
The server can also extract the characteristics of a large amount of historical dangerous road condition scene data, extract various dangerous road condition characteristics in the historical dangerous road condition scene data, train the initial countermeasure network according to the various dangerous road condition characteristics, and obtain a dangerous road condition layer.
Specifically, the server can learn various dangerous road condition features through the neural network corresponding to the dangerous road condition hierarchy. The neural network corresponding to the dangerous road condition hierarchy may be a countermeasure network, for example, a Generative Adaptive Network (GAN) Model, and the Generative countermeasure network Model may include mutual game learning of a Generative Model (Generative Model) and a discriminant Model (discriminant Model), so as to output image generation data with a better effect and data enhancement.
The dangerous road condition layer can comprise a dangerous road condition learning layer and a dangerous road condition generation layer, the server learns the initial confrontation network by utilizing the extracted various dangerous road condition characteristics to obtain a dangerous road condition learning level, and the server further learns and trains the initial confrontation network and the dangerous road condition learning level to obtain the dangerous road condition generation layer. The dangerous road condition generating layer can further randomly generate various dangerous road condition scene models, for example, various dangerous road condition scenes can be automatically and randomly generated through the dangerous road condition generating layer.
And after the server trains to obtain the scene simulation layer and the dangerous road condition layer, the scene simulation layer and the dangerous road condition layer are further combined and trained. Specifically, the server can further learn and train the scene simulation layer and the dangerous road condition layer by using a generative confrontation network algorithm, so that the model can randomly generate dangerous road condition scenes in the current simulation scene when generating various road condition scenes. In the process of training the model, a trained scene simulator is generated until the obtained scene simulation meets the preset conditions. The server can further generate a simulated driving scene by using the trained scene simulator when the unmanned vehicle is subjected to simulation test.
In the scene simulator construction method based on deep learning, after the server acquires the driving scene data and the historical dangerous road condition scene data, the server extracts various road condition scene characteristics in the driving scene data and performs deep learning by using a deep learning algorithm according to the various road condition scene characteristics, so that a scene simulation layer can be obtained through effective training; the server can further train the initial countermeasure network according to various dangerous road condition characteristics by extracting various dangerous road condition characteristics in historical dangerous road condition scene data, so that a dangerous road condition layer can be obtained. By training the dangerous road condition layer through the countermeasure network, the dangerous road condition scene can be generated randomly and effectively by the trained dangerous road condition layer. The server further utilizes the scene simulation layer and the dangerous road condition layer to continuously train the scene simulator until preset conditions are met, and accordingly a trained scene simulator is obtained; the scene simulator is used for generating a simulated driving scene when the vehicle is subjected to simulation test. After a scene simulation layer and a dangerous road condition layer are trained respectively, a generative countermeasure network is further utilized for combined training, so that a scene simulator with high authenticity can be effectively constructed, and a simulated driving scene with high authenticity and reliability can be effectively generated.
In one embodiment, the scene simulator comprises a scene simulation layer and a dangerous road condition layer, wherein the scene simulation layer is used for deeply learning various road condition scene scenes by using a deep learning algorithm and randomly generating the corresponding various road condition scenes by using the learned characteristics of the various road condition scenes. The scene simulation layer may further include a scene element simulation layer, a scene signal simulation layer, and a driving scene simulation layer. The scene element simulation layer is used for extracting various scene element information characteristics in driving scene data, learning and training various scene object information characteristics, and generating simulated scene elements by using the learned various scene element information characteristics. The scene signal simulation layer is used for extracting various scene signal characteristics in the driving scene data, learning and training the various scene signal characteristics, and generating a simulated scene signal by utilizing the learned various scene signal characteristics. The driving scene simulation layer is used for extracting various audio and video signal characteristics in driving scene data, learning and training the various audio and video signal characteristics, and generating a simulated driving scene by combining simulated scene elements and simulated scene signals according to the learned audio and video signal characteristics.
The dangerous road condition layer comprises a dangerous road condition learning layer and a dangerous road condition generating layer, the dangerous road condition learning layer is used for learning and training various dangerous road condition scenes according to the countermeasure network, and the dangerous road condition generating layer is used for randomly generating the various dangerous road condition scenes by utilizing the learned various dangerous road condition characteristics.
Deep learning is carried out by utilizing a deep learning algorithm according to various road condition scene characteristics, so that a scene simulation layer can be obtained through effective training; by training the dangerous road condition layer through the countermeasure network, the dangerous road condition scene can be generated at random and effectively by the trained dangerous road condition layer, and therefore the scene simulator with high authenticity can be constructed effectively.
In one embodiment, the method for obtaining the scene simulation layer by extracting various road condition scene features in the driving scene data and performing deep learning by using a deep learning algorithm according to the various road condition scene features includes: carrying out multi-level feature extraction on driving scene data, and extracting scene element information features, scene signal features and audio-video signal features in the driving scene data; and learning and training the initial network model by using a deep learning algorithm according to the scene element information characteristics, the scene signal characteristics and the audio and video signal characteristics to obtain a trained scene simulation layer.
The driving scene data comprises various road condition scene information, such as road information, lane information, vehicle information, signal light information, pedestrian information, ground feature information and the like, wherein the ground feature information refers to various objects on the ground, such as mountains, forests, buildings and the like, and non-objects, such as various types of ground feature element information of provinces, counties and the like.
After the server acquires a large amount of driving scene data and historical dangerous road condition scene data, a plurality of levels of feature extraction can be firstly carried out on various road condition scenes in the driving scene data, and various road condition scene features in the driving scene data are extracted. The traffic scene characteristics may include scene element information characteristics, scene signal characteristics, audio/video signal characteristics, and other various traffic scene characteristics. The scene element information features are information features corresponding to a plurality of entity elements included in the driving scene, and may include, for example, element information features corresponding to specific road elements, vehicle elements, pedestrian elements, signal lamp elements, and the like. The scene signal characteristics can be sensor signal information corresponding to deeper road information, vehicle information, pedestrian information, signal lamp information and other information. The audio and video signal characteristics are audio and video signals corresponding to road information, vehicle information, pedestrian information, signal lamp information and other information in an audio and video visual scene.
Specifically, the server may input the driving scene data and the historical dangerous road condition scene data into a pre-constructed initial neural network model, where the initial neural network includes multiple levels, for example, a scene simulation level, a dangerous road condition level, and the like. The server performs multi-level feature extraction on various road condition scenes in the driving scene data through a scene simulation layer, and respectively extracts a plurality of scene element information features including road element features, vehicle element features, pedestrian element features, signal lamp element features and the like; further extracting scene signal characteristics in the driving scene data according to the scene element information characteristics; and further extracting audio and video signal characteristics in the driving scene data according to the scene element information characteristics and the scene signal characteristics.
And the server further learns the corresponding neural network by using a deep learning algorithm according to the scene element information characteristics, the scene signal characteristics and the audio and video signal characteristics, generates a scene simulation layer according to various learned road condition scene characteristics, and further trains the scene simulation layer to obtain the trained scene simulation layer. The scene simulation layer can further randomly generate multiple corresponding models by using the learned multiple road condition scene features, for example, multiple scene models included in road condition scenes such as road models, models and the like, vehicle models, pedestrian models and the like can be automatically generated by the scene simulation layer. After the driving scene data is subjected to multi-level feature extraction, the scene simulation layer is trained by using the extracted multi-level features, so that the scene simulation layer can be effectively constructed.
In one embodiment, as shown in fig. 3, the scene simulation layer includes a scene element simulation layer, a scene signal simulation layer, and a driving scene simulation layer, and the step of performing deep learning by using a deep learning algorithm according to various road and scene characteristics to obtain the scene simulation layer specifically includes the following steps:
and 302, extracting various scene element information characteristics in the driving scene data, and training various scene object information characteristics to obtain a trained scene element simulation layer.
And step 304, extracting various scene signal characteristics in the driving scene data, and training the various scene signal characteristics to obtain a trained scene signal simulation layer.
And step 306, extracting various audio and video signal characteristics in the driving scene data, and training the various audio and video signal characteristics to obtain a trained driving scene simulation layer.
The scene element information includes various environment role elements, for example, various environment role information such as specific road entity information, vehicle entity information, pedestrian entity information, signal lamp entity information, and the like.
After acquiring a large amount of driving scene data and historical dangerous road condition scene data, the server inputs the driving scene data and the historical dangerous road condition scene data into a pre-constructed initial neural network model. The initial neural network model may be constructed using a pre-set deep learning algorithm and a neural network structure. The initial neural network comprises a plurality of levels, for example, a scene simulation level, a dangerous road condition learning level, a dangerous road condition generation level, and the like. The scene simulation hierarchy can further comprise a plurality of neural network layers, and specifically can comprise a scene element simulation layer, a scene signal simulation layer and a driving scene simulation layer. And the server further performs multi-level feature extraction on various road condition scenes in the driving scene data through a plurality of neural network layers in the scene simulation hierarchy respectively to extract various road condition scene features in the driving scene data.
Specifically, the server extracts various scene element information characteristics in the driving scene data through the scene element simulation layer, learns various scene object information characteristics by using a preset deep learning algorithm, trains a neural network corresponding to the scene element simulation layer according to the learned characteristics, and obtains the trained scene element simulation layer until a training condition threshold is met. The trained scene element simulation layer may then be used to generate simulated scene element information.
The server further extracts various scene signal characteristics in the driving scene data through the scene signal simulation layer, learns the various scene signal characteristics by using a preset deep learning algorithm, trains the neural network corresponding to the scene signal simulation layer according to the learned characteristics, and obtains the trained scene signal simulation layer until the training condition threshold is met. The server can further combine various scene element information characteristics and various scene signal characteristics to learn and train the neural network, so that a trained scene signal simulation layer is obtained. The trained scene signal simulation layer may then be used to generate a variety of scene elements and scene signal information.
The server extracts various comprehensive audio and video signal characteristics in the driving scene data through the driving scene simulation layer, wherein the various comprehensive audio and video signal characteristics comprise various dynamic scene element information characteristics and various scene signal characteristics. And the server further learns the characteristics of various audio and video signals by using a preset deep learning algorithm, trains the neural network corresponding to the driving scene simulation layer according to the learned characteristics, and obtains the driving scene simulation layer after training until the training condition threshold is met. The server can further combine various scene element information characteristics, various scene signal characteristics and various audio and video signal characteristics to learn and train the neural network, so that a driving scene simulation layer which is trained is obtained. The trained driving scene simulation layer can be used for generating various dynamic driving scene information.
The driving scene data are respectively subjected to multi-level feature extraction, and a plurality of scene simulation layers are trained by utilizing the extracted multi-level features, so that the scene simulation layers can be effectively constructed.
In one embodiment, as shown in fig. 4, the dangerous road condition layer includes a dangerous road condition learning layer, and the step of extracting multiple dangerous road condition features from historical dangerous road condition scene data includes the following steps:
At step 404, risk factors and risk levels for each dangerous road condition characteristic are identified.
And 406, generating a dangerous scene factor set by using the multiple dangerous road condition characteristics according to the dangerous factors and the dangerous degree.
The server can obtain a large amount of historical dangerous road condition scene data, and the historical dangerous road condition scene data comprises dangerous road condition scene information of various dangerous factors and dangerous degrees.
The server can input a large amount of historical dangerous road condition scene data into the preset neural network corresponding to the dangerous road condition layer. Specifically, the server inputs historical dangerous road condition scene data into a dangerous road condition learning layer in the dangerous road condition layer, historical dangerous road condition scene data features are extracted through the dangerous road condition learning layer, and various dangerous road condition features in the historical dangerous road condition scene data are extracted. The server further identifies the risk factors and the degree of risk for each dangerous road condition characteristic. The risk factors may be various environmental factors causing dangerous road conditions, and may include, for example, the number of lanes, lane vehicles, pedestrians, road obstacles, bad weather, vehicle failures, illegal driving, and other risk factors causing vehicles to be in dangerous states. The risk factors may also be a plurality of factor types corresponding to the plurality of risk types. The risk degree can be calculated according to the damage level, the risk complexity and the occurrence probability.
After the server extracts various dangerous road condition features in the historical dangerous road condition scene data, dangerous factors in each dangerous road condition feature, damage level, dangerous complexity, occurrence probability and the like of each dangerous road condition feature are extracted, and the dangerous degree of each dangerous road condition feature is calculated according to the damage level, the dangerous complexity and the occurrence probability. And the server further generates a dangerous scene factor set by utilizing various dangerous road condition characteristics according to the dangerous factors and the dangerous degree. The dangerous scene factor set is the dangerous road condition features including multiple dangerous factors learned by the dangerous road condition learning layer. By carrying out feature extraction and deep learning on the scene data of various historical dangerous road conditions, various dangerous road condition features can be effectively learned.
In one embodiment, the dangerous road condition layer includes a dangerous road condition generation layer, and the initial countermeasure network is trained according to various dangerous road condition characteristics to obtain the dangerous road condition layer, including: generating a risk factor random domain according to the probability value of the risk factors in the risk scene element set; carrying out various dangerous road condition characteristic training on the dangerous scene factor set by using the generative confrontation network according to the dangerous factor random field to obtain a trained dangerous road condition generation layer; the dangerous road condition generating layer is used for randomly generating dangerous road condition information.
The server extracts various dangerous road condition features in historical dangerous road condition scene data through the dangerous road condition learning layer, recognizes the dangerous factors and the dangerous degree of each dangerous road condition feature, generates a dangerous scene factor set by using the various dangerous road condition features according to the dangerous factors and the dangerous degree, and trains the dangerous road condition generating layer by further using the dangerous scene factor set.
Specifically, the server may further calculate a probability value of each risk factor in the historical dangerous road condition scene data, that is, an occurrence probability of the risk factor, through the dangerous road condition learning layer. The initial confrontation network may be a generative confrontation network, and the generative confrontation network may include a discrimination model and a generative model, that is, the dangerous road condition learning layer may be the discrimination model, and the dangerous road condition generating layer may be the generative model.
The server further utilizes the generative confrontation network to train various dangerous road condition characteristics in the dangerous scene factor set according to the dangerous factor random domain, so that a dangerous road condition generating layer can be effectively trained, and further the dangerous road condition generating layer can randomly generate various dangerous road condition scene models, for example, various dangerous road condition scenes can be automatically and randomly generated through the dangerous road condition generating layer. The dangerous road condition learning layer and the dangerous road condition generating layer are trained by utilizing the generative confrontation network algorithm, so that the dangerous road condition layer with high effectiveness and authenticity can be effectively constructed.
In one embodiment, the method for continuously training the scene simulator by using the scene simulation layer and the dangerous road condition layer until a preset condition is met to obtain the trained scene simulator includes: constructing a scene simulator according to a scene simulation layer and a dangerous road condition layer; continuously carrying out combined training on a scene simulation layer and a dangerous road condition layer by utilizing a generative confrontation network algorithm; and obtaining the trained scene simulator until the driving scene generated by the scene simulator meets the condition threshold and the generated dangerous road condition meets the probability threshold.
And after the server trains to obtain the scene simulation layer and the dangerous road condition layer, constructing a scene simulator according to the scene simulation layer and the dangerous road condition layer. The server further carries out combined training on the scene simulation layer and the dangerous road condition layer. Specifically, the server can further utilize a generative confrontation network algorithm to continuously perform combined training on the scene simulation layer and the dangerous road condition layer, and in the training process of the model, when the simulator generates various road condition scenes, the simulator can randomly generate dangerous road condition scenes in the current simulation scene according to probability values of various dangerous road conditions. And stopping training until the driving scene generated by the scene simulator meets the condition threshold and the generated dangerous road condition meets the probability threshold, thereby obtaining the trained scene simulator. The server can further generate a simulated driving scene by using the trained scene simulator when the unmanned vehicle is subjected to simulation test.
For example, when a driving scene is simulated by a simulator, a road map of a corresponding road type may be constructed, the road map includes various types of information such as various types of ground, lane lines, signal lights, and landmark objects, and traffic participant information, for example, may include vehicles and pedestrians, and corresponding movement paths, respectively, may be added to the road map. The simulator can also randomly generate dangerous road condition scenes of various dangerous factors in the simulated driving scene, such as dangerous pedestrian tracks, dangerous vehicle tracks, roadblock information, severe environment and various dangerous road condition scenes.
After a scene simulation layer and a dangerous road condition layer are trained respectively, a generative countermeasure network is further utilized for combined training, so that a scene simulator with high authenticity can be effectively constructed, and a simulated driving scene with high authenticity and reliability can be effectively generated.
In one embodiment, as shown in fig. 5, the method further includes a step of performing a test by using a scenario simulator, which specifically includes the following steps:
In step 506, vehicle driving data of the vehicle during the simulated driving process is obtained.
And step 508, generating vehicle simulation test information according to the driving scene information and the vehicle driving data.
The server utilizes the driving scene data and the historical dangerous road condition scene data to construct and train to obtain a scene simulator comprising a scene simulation layer and a dangerous road condition layer, and then the scene simulator can be utilized to generate a simulated driving scene so as to carry out simulation test on the unmanned vehicle.
Specifically, the vehicle may send a simulation test request to the server, or the vehicle monitoring platform may directly send a simulation test instruction to the server. And after the server acquires the simulation test instruction, calling the scene simulator according to the simulation driving instruction. The server further generates driving scene information through a scene simulation layer of the scene simulator, and generates dangerous road condition information in the driving scene information at random through a dangerous road condition layer of the scene simulator. Thereby enabling the vehicle to perform simulated driving in the generated simulated driving scene. Specifically, the corresponding sensors are arranged in the vehicle, so that the sensor data and the like of the vehicle can be effectively tested in the simulated driving of the vehicle.
The server may obtain vehicle driving data of the vehicle during the simulated driving process, where the vehicle driving data includes vehicle state data and road image data collected by the vehicle, for example, the vehicle state data may include vehicle operating state data, sensor data, energy consumption data, and the like, and the road image data includes a plurality of road image data affecting the collection of the collection device. The server further generates vehicle simulation test information of the vehicle according to the driving scene information and the vehicle driving data, and the vehicle simulation test information can be used for analyzing various performance indexes of the unmanned vehicle. The driving simulation scene is generated by utilizing the constructed scene simulator and the unmanned vehicle is subjected to simulation test, so that the simulated driving scene with higher authenticity and reliability can be effectively constructed, the unmanned vehicle can be effectively subjected to simulation test, and the effectiveness and reliability of the test are effectively improved.
It should be understood that although the various steps in the flow charts of fig. 2-5 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2-5 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternating with other steps or at least some of the sub-steps or stages of other steps.
In one embodiment, as shown in fig. 6, there is provided a scene simulator building apparatus based on deep learning, including: a data acquisition module 602, a scene simulation training module 604, a dangerous road condition training module 606, and a scene simulator construction module 608, wherein:
a data obtaining module 602, configured to obtain driving scene data and historical dangerous road condition scene data;
the scene simulation training module 604 is configured to extract multiple road condition scene features in the driving scene data, and perform deep learning by using a deep learning algorithm according to the multiple road condition scene features to obtain a scene simulation layer;
a dangerous road condition training module 606, configured to extract multiple dangerous road condition features in historical dangerous road condition scene data, train the initial countermeasure network according to the multiple dangerous road condition features, and obtain a dangerous road condition layer; and
a scene simulator construction module 608, configured to continuously train a scene simulator using a scene simulation layer and a dangerous road condition layer until a preset condition is met, so as to obtain a trained scene simulator; the scene simulator is used for generating a simulated driving scene when the vehicle is subjected to simulation test.
In one embodiment, the scene simulator comprises a scene simulation layer and a dangerous road condition layer, wherein the scene simulation layer is used for deeply learning the scenes of the various road conditions and scenes by using a deep learning algorithm and randomly generating the corresponding various road conditions and scenes by using the characteristics of the learned various road conditions and scenes; the dangerous road condition layer comprises a dangerous road condition learning layer and a dangerous road condition generating layer, the dangerous road condition learning layer is used for learning and training a plurality of dangerous road condition scenes according to the countermeasure network, and the dangerous road condition generating layer is used for randomly generating the plurality of dangerous road condition scenes by utilizing the learned characteristics of the plurality of dangerous road conditions.
In one embodiment, the scene simulation training module 604 is further configured to perform multi-level feature extraction on the driving scene data, and extract scene element information features, scene signal features, and audio/video signal features in the driving scene data; and learning and training the initial network model by using a deep learning algorithm according to the scene element information characteristics, the scene signal characteristics and the audio and video signal characteristics to obtain a trained scene simulation layer.
In one embodiment, the scene simulation layer includes a scene element simulation layer, a scene signal simulation layer, and a driving scene simulation layer, and the scene simulation training module 604 is further configured to extract various scene element information features in the driving scene data, train the various scene element information features, and obtain a trained scene element simulation layer; extracting various scene signal characteristics in the driving scene data, and training the various scene signal characteristics to obtain a trained scene signal simulation layer; and extracting various audio and video signal characteristics in the driving scene data, and training the various audio and video signal characteristics to obtain a driving scene simulation layer after training.
In one embodiment, the dangerous road condition layer includes a dangerous road condition learning layer, and the dangerous road condition training module 606 is further configured to extract multiple dangerous road condition features in the historical dangerous road condition scene data through the dangerous road condition learning layer; identifying the dangerous factors and the dangerous degree of each dangerous road condition characteristic; and generating a dangerous scene factor set by utilizing various dangerous road condition characteristics according to the dangerous factors and the dangerous complexity.
In one embodiment, the dangerous road condition layer includes a dangerous road condition generating layer, and the dangerous road condition training module 606 is further configured to generate a dangerous factor random field according to a probability value of a dangerous factor in the dangerous scene element set; carrying out various dangerous road condition characteristic training on the dangerous scene factor set by using the generated countermeasure network according to the dangerous factor random field to obtain a trained dangerous road condition generation layer; and generating a dangerous road condition layer for randomly generating dangerous road condition information.
In one embodiment, the scene simulator construction module 608 is further configured to construct a scene simulator according to a scene simulation layer and a dangerous road condition layer; continuously carrying out combined training on a scene simulation layer and a dangerous road condition layer by utilizing a generative confrontation network algorithm; and obtaining the trained scene simulator until the driving scene generated by the scene simulator meets the condition threshold and the generated dangerous road condition meets the probability threshold.
In one embodiment, as shown in fig. 7, the apparatus further includes a simulation testing module 610, configured to obtain a simulation testing instruction, and invoke the scene simulator according to the simulation driving instruction; generating driving scene information by using a scene simulator, and randomly generating dangerous road condition information in the driving scene information; enabling the vehicle to carry out simulated driving in the generated simulated driving scene; acquiring vehicle driving data of a vehicle in a simulated driving process; and generating vehicle simulation test information according to the driving scene information and the vehicle driving data.
For specific limitations of the scene simulator construction device based on deep learning, reference may be made to the above limitations of the scene simulator construction method based on deep learning, and details are not repeated here. The various modules in the deep learning based scene simulator construction device can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 8. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer readable instructions, and a database. The internal memory provides an environment for the operating system and execution of computer-readable instructions in the non-volatile storage medium. The database of the computer equipment is used for storing data such as driving scene data, historical dangerous road condition scene data and the like. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer readable instructions, when executed by a processor, implement a method 8.
Those skilled in the art will appreciate that the architecture shown in fig. 8 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
A computer device comprising a memory and one or more processors, the memory having stored therein computer-readable instructions, which, when executed by the processors, cause the one or more processors, when executed, to carry out the steps of the above-described method embodiments.
One or more non-transitory computer-readable storage media storing computer-readable instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of the above-described method embodiments.
It will be understood by those of ordinary skill in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware associated with computer readable instructions, which can be stored in a non-volatile computer readable storage medium, and when executed, can include processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (20)
- A scene simulator construction method based on deep learning comprises the following steps:acquiring driving scene data and historical dangerous road condition scene data;extracting various road condition scene characteristics in the driving scene data, and performing deep learning by using a deep learning algorithm according to the various road condition scene characteristics to obtain a scene simulation layer;extracting various dangerous road condition characteristics in historical dangerous road condition scene data, and training an initial countermeasure network according to the various dangerous road condition characteristics to obtain a dangerous road condition layer; andcontinuously training a scene simulator by using the scene simulation layer and the dangerous road condition layer until a preset condition is met, and obtaining a trained scene simulator; the scene simulator is used for generating a simulated driving scene when a vehicle is subjected to simulation test.
- The method according to claim 1, wherein the scene simulator comprises a scene simulation layer and a dangerous road condition layer, wherein the scene simulation layer is used for deep learning the multiple road condition scenes by using a deep learning algorithm, and randomly generating the corresponding multiple road condition scenes by using the learned multiple road condition scene features; the dangerous road condition layer comprises a dangerous road condition learning layer and a dangerous road condition generating layer, the dangerous road condition learning layer is used for learning and training the various dangerous road condition scenes according to the countermeasure network, and the dangerous road condition generating layer is used for randomly generating the various dangerous road condition scenes by utilizing the learned various dangerous road condition characteristics.
- The method according to claim 1, wherein the extracting of the multiple road condition scene features in the driving scene data and the deep learning according to the multiple road condition scene features by using a deep learning algorithm to obtain a scene simulation layer comprises:performing multi-level feature extraction on the driving scene data, and extracting scene element information features, scene signal features and audio and video signal features in the driving scene data; andand learning and training the initial network model by using a deep learning algorithm according to the scene element information characteristics, the scene signal characteristics and the audio and video signal characteristics to obtain a trained scene simulation layer.
- The method according to claim 3, wherein the scene simulation layer comprises a scene element simulation layer, a scene signal simulation layer and a driving scene simulation layer, and the deep learning by using a deep learning algorithm according to the various road condition scene characteristics to obtain the scene simulation layer comprises:extracting various scene element information characteristics in the driving scene data, and training the various scene element information characteristics to obtain a trained scene element simulation layer;extracting various scene signal characteristics in the driving scene data, and training the various scene signal characteristics to obtain a trained scene signal simulation layer; andand extracting various audio and video signal characteristics in the driving scene data, and training the various audio and video signal characteristics to obtain a trained driving scene simulation layer.
- The method according to claim 1, wherein the dangerous road condition layer comprises a dangerous road condition learning layer, and the extracting of the plurality of dangerous road condition features from the historical dangerous road condition scene data comprises:extracting various dangerous road condition features in the historical dangerous road condition scene data through the dangerous road condition learning layer;identifying the dangerous factors and the dangerous degree of each dangerous road condition characteristic; andand generating a dangerous scene factor set by utilizing the various dangerous road condition characteristics according to the dangerous factors and the dangerous complexity.
- The method as claimed in claim 5, wherein the dangerous road condition layer comprises a dangerous road condition generation layer, and the training of the initial countermeasure network according to the various dangerous road condition features to obtain the dangerous road condition layer comprises:generating a risk factor random domain according to the probability value of the risk factors in the risk scene element set; andcarrying out multiple dangerous road condition characteristic training on the dangerous scene factor set by using a generative confrontation network according to the dangerous factor random field to obtain a trained dangerous road condition generation layer; and the dangerous road condition generation layer is used for randomly generating dangerous road condition information.
- The method according to any one of claims 1 to 6, wherein the continuously training the scene simulator by using the scene simulation layer and the dangerous road condition layer until a preset condition is met to obtain a trained scene simulator comprises:constructing a scene simulator according to the scene simulation layer and the dangerous road condition layer;continuously performing combined training on the scene simulation layer and the dangerous road condition layer by using a generative confrontation network algorithm; andand obtaining the trained scene simulator until the driving scene generated by the scene simulator meets a condition threshold and the generated dangerous road condition meets a probability threshold.
- The method of claim 1, further comprising:acquiring a simulation test instruction, and calling a scene simulator according to the simulation driving instruction;generating driving scene information by using the scene simulator, and randomly generating dangerous road condition information in the driving scene information; enabling the vehicle to carry out simulated driving in the generated simulated driving scene;acquiring vehicle driving data of the vehicle in a simulated driving process; andand generating vehicle simulation test information according to the driving scene information and the vehicle driving data.
- A scene simulator building device based on deep learning comprises the following components:the data acquisition module is used for acquiring driving scene data and historical dangerous road condition scene data;the scene simulation training module is used for extracting various road condition scene characteristics in the driving scene data and carrying out deep learning by utilizing a deep learning algorithm according to the various road condition scene characteristics to obtain a scene simulation layer;the dangerous road condition training module is used for extracting various dangerous road condition characteristics in historical dangerous road condition scene data, and training the initial countermeasure network according to the various dangerous road condition characteristics to obtain a dangerous road condition layer; andthe scene simulator construction module is used for continuously training the scene simulator by utilizing the scene simulation layer and the dangerous road condition layer until a preset condition is met, and obtaining a trained scene simulator; the scene simulator is used for generating a simulated driving scene when a vehicle is subjected to simulation test.
- The device according to claim 9, wherein the dangerous road condition layer comprises a dangerous road condition learning layer, and the dangerous road condition training module is further configured to extract multiple dangerous road condition features in the historical dangerous road condition scene data through the dangerous road condition learning layer; identifying the dangerous factors and the dangerous degree of each dangerous road condition characteristic; and generating a dangerous scene factor set by utilizing the various dangerous road condition characteristics according to the dangerous factors and the dangerous complexity.
- The apparatus according to claim 9, wherein the dangerous road condition layer comprises a dangerous road condition generation layer, and the dangerous road condition training module is further configured to generate a dangerous factor random field according to a probability value of a dangerous factor in the dangerous scene element set; carrying out various dangerous road condition characteristic training on the dangerous scene factor set by using a generating type confrontation network according to the dangerous factor random field to obtain a trained dangerous road condition generation layer; and the dangerous road condition generation layer is used for randomly generating dangerous road condition information.
- The apparatus of claim 11, wherein the scene simulator construction module is further configured to construct a scene simulator according to the scene simulation layer and the dangerous road conditions layer; continuously performing combined training on the scene simulation layer and the dangerous road condition layer by using a generative confrontation network algorithm; and obtaining the trained scene simulator until the driving scene generated by the scene simulator meets a condition threshold and the generated dangerous road condition meets a probability threshold.
- The device of claim 9, further comprising a simulation test module, configured to obtain a simulation test instruction, and invoke a scene simulator according to the simulation driving instruction; generating driving scene information by using the scene simulator, and randomly generating dangerous road condition information in the driving scene information; enabling the vehicle to carry out simulated driving in the generated simulated driving scene; acquiring vehicle driving data of the vehicle in a simulated driving process; and generating vehicle simulation test information according to the driving scene information and the vehicle driving data.
- A computer device comprising a memory and one or more processors, the memory having stored therein computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to perform the steps of:acquiring driving scene data and historical dangerous road condition scene data;extracting various road condition scene characteristics in the driving scene data, and performing deep learning by using a deep learning algorithm according to the various road condition scene characteristics to obtain a scene simulation layer;extracting various dangerous road condition characteristics in historical dangerous road condition scene data, and training an initial countermeasure network according to the various dangerous road condition characteristics to obtain a dangerous road condition layer; andcontinuously training a scene simulator by using the scene simulation layer and the dangerous road condition layer until a preset condition is met, and obtaining a trained scene simulator; the scene simulator is used for generating a simulated driving scene when a vehicle is subjected to simulation test.
- The computer device of claim 14, wherein the processor, when executing the computer readable instructions, further performs the steps of:performing multi-level feature extraction on the driving scene data, and extracting scene element information features, scene signal features and audio and video signal features in the driving scene data; andand learning and training the initial network model by using a deep learning algorithm according to the scene element information characteristics, the scene signal characteristics and the audio and video signal characteristics to obtain a trained scene simulation layer.
- The computer device of claim 15, wherein the scene simulation layers include a scene element simulation layer, a scene signal simulation layer, and a driving scene simulation layer, the processor when executing the computer readable instructions further performing the steps of:extracting various scene element information characteristics in the driving scene data, and training the various scene element information characteristics to obtain a trained scene element simulation layer;extracting various scene signal characteristics in the driving scene data, and training the various scene signal characteristics to obtain a trained scene signal simulation layer; andand extracting various audio and video signal characteristics in the driving scene data, and training the various audio and video signal characteristics to obtain a trained driving scene simulation layer.
- The computer device of claim 14, wherein the dangerous road conditions layer comprises a dangerous road conditions learning layer, and wherein the processor when executing the computer readable instructions further performs the steps of:extracting various dangerous road condition features in the historical dangerous road condition scene data through the dangerous road condition learning layer;identifying the dangerous factors and the dangerous degree of each dangerous road condition characteristic; andand generating a dangerous scene factor set by utilizing the various dangerous road condition characteristics according to the dangerous factors and the dangerous complexity.
- The computer device of claim 17, wherein the dangerous road conditions layer comprises a dangerous road conditions generating layer, and wherein the processor when executing the computer readable instructions further performs the steps of:generating a risk factor random domain according to the probability value of the risk factors in the risk scene element set;carrying out various dangerous road condition characteristic training on the dangerous scene factor set by using a generating type confrontation network according to the dangerous factor random field to obtain a trained dangerous road condition generation layer; and the dangerous road condition generation layer is used for randomly generating dangerous road condition information.
- The computer device of claim 14, wherein the processor, when executing the computer readable instructions, further performs the steps of:acquiring a simulation test instruction, and calling a scene simulator according to the simulation driving instruction;generating driving scene information by using the scene simulator, and randomly generating dangerous road condition information in the driving scene information; enabling the vehicle to carry out simulated driving in the generated simulated driving scene;acquiring vehicle driving data of the vehicle in a simulated driving process; andand generating vehicle simulation test information according to the driving scene information and the vehicle driving data.
- One or more non-transitory computer-readable storage media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of any of claims 1-8.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2020/073469 WO2021146905A1 (en) | 2020-01-21 | 2020-01-21 | Deep learning-based scene simulator construction method and apparatus, and computer device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113490940A true CN113490940A (en) | 2021-10-08 |
| CN113490940B CN113490940B (en) | 2024-07-23 |
Family
ID=76992631
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202080003157.3A Active CN113490940B (en) | 2020-01-21 | 2020-01-21 | Scene simulator construction method and device based on deep learning and computer equipment |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113490940B (en) |
| WO (1) | WO2021146905A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115240409A (en) * | 2022-06-17 | 2022-10-25 | 上海智能网联汽车技术中心有限公司 | Method for extracting dangerous scene based on driver model and traffic flow model |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114296424B (en) * | 2021-12-06 | 2024-05-28 | 苏州挚途科技有限公司 | Simulation test system and method |
| CN114771576A (en) * | 2022-05-19 | 2022-07-22 | 北京百度网讯科技有限公司 | Behavior data processing method, control method of automatic driving vehicle and automatic driving vehicle |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106652645A (en) * | 2017-03-16 | 2017-05-10 | 百度在线网络技术（北京）有限公司 | Vehicle driving training device, as well as operation method and device of vehicle driving training device |
| CN108345869A (en) * | 2018-03-09 | 2018-07-31 | 南京理工大学 | Driver's gesture recognition method based on depth image and virtual data |
| CN108595901A (en) * | 2018-07-09 | 2018-09-28 | 黄梓钥 | A kind of autonomous driving vehicle normalized security simulating, verifying model data base system |
| US20180307967A1 (en) * | 2017-04-25 | 2018-10-25 | Nec Laboratories America, Inc. | Detecting dangerous driving situations by parsing a scene graph of radar detections |
| CN108897313A (en) * | 2018-05-23 | 2018-11-27 | 清华大学 | A kind of end-to-end Vehicular automatic driving system construction method of layer-stepping |
| KR20180127245A (en) * | 2017-05-19 | 2018-11-28 | 한국과학기술원 | Method for Predicting Vehicle Collision Using Data Collected from Video Games |
| CN109190648A (en) * | 2018-06-26 | 2019-01-11 | Oppo（重庆）智能科技有限公司 | Simulated environment generation method, device, mobile terminal and computer-readable storage medium |
| CN109597317A (en) * | 2018-12-26 | 2019-04-09 | 广州小鹏汽车科技有限公司 | A kind of Vehicular automatic driving method, system and electronic equipment based on self study |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110569916B (en) * | 2019-09-16 | 2022-03-22 | 电子科技大学 | Confrontation sample defense system and method for artificial intelligence classification |
| CN110647839B (en) * | 2019-09-18 | 2022-08-12 | 深圳信息职业技术学院 | Method and device for generating automatic driving strategy and computer readable storage medium |
-
2020
- 2020-01-21 CN CN202080003157.3A patent/CN113490940B/en active Active
- 2020-01-21 WO PCT/CN2020/073469 patent/WO2021146905A1/en active Application Filing
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106652645A (en) * | 2017-03-16 | 2017-05-10 | 百度在线网络技术（北京）有限公司 | Vehicle driving training device, as well as operation method and device of vehicle driving training device |
| US20180307967A1 (en) * | 2017-04-25 | 2018-10-25 | Nec Laboratories America, Inc. | Detecting dangerous driving situations by parsing a scene graph of radar detections |
| KR20180127245A (en) * | 2017-05-19 | 2018-11-28 | 한국과학기술원 | Method for Predicting Vehicle Collision Using Data Collected from Video Games |
| CN108345869A (en) * | 2018-03-09 | 2018-07-31 | 南京理工大学 | Driver's gesture recognition method based on depth image and virtual data |
| CN108897313A (en) * | 2018-05-23 | 2018-11-27 | 清华大学 | A kind of end-to-end Vehicular automatic driving system construction method of layer-stepping |
| CN109190648A (en) * | 2018-06-26 | 2019-01-11 | Oppo（重庆）智能科技有限公司 | Simulated environment generation method, device, mobile terminal and computer-readable storage medium |
| CN108595901A (en) * | 2018-07-09 | 2018-09-28 | 黄梓钥 | A kind of autonomous driving vehicle normalized security simulating, verifying model data base system |
| CN109597317A (en) * | 2018-12-26 | 2019-04-09 | 广州小鹏汽车科技有限公司 | A kind of Vehicular automatic driving method, system and electronic equipment based on self study |
Non-Patent Citations (1)
| Title |
|---|
| 周颖;严利鑫;吴青;高嵩;吴超仲;: "基于虚拟现实技术的典型动态交通场景的设计与实现", 交通信息与安全, vol. 31, no. 01, 20 February 2013 (2013-02-20), pages 128 - 132 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115240409A (en) * | 2022-06-17 | 2022-10-25 | 上海智能网联汽车技术中心有限公司 | Method for extracting dangerous scene based on driver model and traffic flow model |
| CN115240409B (en) * | 2022-06-17 | 2024-02-06 | 上智联(上海)智能科技有限公司 | Method for extracting dangerous scene based on driver model and traffic flow model |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021146905A1 (en) | 2021-07-29 |
| CN113490940B (en) | 2024-07-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102440938B1 (en) | Method and device for creating traffic scenario with domain adaptation on virtual driving environment for testing, validating, and training autonomous vehicle | |
| CN113490940B (en) | Scene simulator construction method and device based on deep learning and computer equipment | |
| CN109902018B (en) | Method for acquiring test case of intelligent driving system | |
| CN113343461A (en) | Simulation method and device for automatic driving vehicle, electronic equipment and storage medium | |
| WO2020060480A1 (en) | System and method for generating a scenario template | |
| US20230331250A1 (en) | Method and apparatus for configuring deep learning algorithm for autonomous driving | |
| US20200242478A1 (en) | Learning method and learning device for updating hd map by reconstructing 3d space by using depth estimation information and class information on each object, which have been acquired through v2x information integration technique, and testing method and testing device using the same | |
| CN112164227A (en) | Illegal parking vehicle warning method and device, computer equipment and storage medium | |
| CN112257545A (en) | Violation real-time monitoring and analyzing method and device and storage medium | |
| CN115830399A (en) | Classification model training method, apparatus, device, storage medium, and program product | |
| CN116403174A (en) | End-to-end automatic driving method, system, simulation system and storage medium | |
| Xu et al. | Reliability of gan generated data to train and validate perception systems for autonomous vehicles | |
| CN116189063B (en) | Key frame optimization method and device for intelligent video monitoring | |
| CN116341648A (en) | Method, device, equipment and medium for training environment awareness capability of automatic driving vehicle | |
| Li | A scenario-based development framework for autonomous driving | |
| CN116776288A (en) | Optimization method and device of intelligent driving perception model and storage medium | |
| CN112529116B (en) | Scene element fusion processing method, device and equipment and computer storage medium | |
| Lambertenghi et al. | Assessing Quality Metrics for Neural Reality Gap Input Mitigation in Autonomous Driving Testing | |
| CN114743262A (en) | Behavior detection method and device, electronic equipment and storage medium | |
| Deng et al. | Generalizing deep models for overhead image segmentation through Getis-Ord Gi* pooling | |
| CN113592902A (en) | Target tracking method and device, computer equipment and storage medium | |
| Hela et al. | CarParkingVQA: Visual Question Answering application on Car parking occupancy detection | |
| CN110674764A (en) | Method, device and system for detecting exposed earthwork of construction site | |
| CN110333517A (en) | Method for barrier perception, device and storage medium | |
| Ogawa et al. | Identifying Parking Lot Occupancy with YOLOv5 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |