CN115743178A - Automatic driving method and system based on scene self-adaptive recognition - Google Patents
Automatic driving method and system based on scene self-adaptive recognition Download PDFInfo
- Publication number
- CN115743178A CN115743178A CN202211497070.3A CN202211497070A CN115743178A CN 115743178 A CN115743178 A CN 115743178A CN 202211497070 A CN202211497070 A CN 202211497070A CN 115743178 A CN115743178 A CN 115743178A
- Authority
- CN
- China
- Prior art keywords
- scene
- module
- path planning
- threshold
- operation information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 88
- 230000005856 abnormality Effects 0.000 claims abstract description 43
- 238000013528 artificial neural network Methods 0.000 claims abstract description 36
- 238000004088 simulation Methods 0.000 claims abstract description 34
- 230000007613 environmental effect Effects 0.000 claims abstract description 17
- 230000006870 function Effects 0.000 claims description 20
- 230000003044 adaptive effect Effects 0.000 claims description 16
- 230000008447 perception Effects 0.000 claims description 12
- 230000000007 visual effect Effects 0.000 claims description 9
- 238000012549 training Methods 0.000 claims description 7
- 230000001133 acceleration Effects 0.000 claims description 6
- 230000003278 mimic effect Effects 0.000 claims description 4
- 238000005457 optimization Methods 0.000 description 18
- 230000009471 action Effects 0.000 description 7
- 230000008901 benefit Effects 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 6
- 239000011159 matrix material Substances 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000013519 translation Methods 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 241000282414 Homo sapiens Species 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 210000002569 neuron Anatomy 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000010485 coping Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012821 model calculation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W60/00—Drive control systems specially adapted for autonomous road vehicles
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
- G01C21/34—Route searching; Route guidance
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- Mathematical Physics (AREA)
- Human Computer Interaction (AREA)
- General Physics & Mathematics (AREA)
- Traffic Control Systems (AREA)
Abstract
The application provides an automatic driving method and system based on scene self-adaptive recognition, wherein the method comprises the following steps: acquiring environmental information and previous-time operation information in a driving scene; the path planning module determines a path planning track in the current driving scene based on the operation information at the last moment; determining scene complexity according to the parameter space of the path planning track; the simulation learning module obtains uncertainty distribution of decision by a neural network according to the environmental information; determining scene abnormality degree according to the uncertainty distribution; and the decision module determines an automatic driving method according to the scene complexity and the scene abnormality. The scheme improves the real-time performance and the accuracy performance of the automatic driving.
Description
Technical Field
The invention belongs to the technical field of information, and particularly relates to an automatic driving method and system based on scene adaptive recognition.
Background
With the continuous upgrading of automobile intellectualization and electromotion, the automatic driving of automobiles becomes a great trend of automobile industry revolution. The development potential of the automatic driving automobile is huge, the automatic driving technology becomes an important part of the strategic emerging industry, the rapid development of the automatic driving technology can deeply affect the flowing modes of people, resources and products, and the life style of human beings is subversively changed.
Currently, a commonly used simulated learning algorithm learns an input-output pair in a data set through a neural network, and continuously optimizes neuron parameters to approximate characteristics and logic of the data set, so that the neural network can finally obtain output conforming to the logic according to the input. However, the simulation learning depends heavily on the data set, and the decision confidence of scenes which do not appear in the data set is not enough, so that the simulation learning is suitable for a single scene with similar characteristics. The traditional path planning method is to calculate an optimal collision-free track through mathematical reasoning according to the initial position, the target position and the environmental information, and then to approximate the track to the current position to obtain the position of the next moment, so as to calculate the dynamic parameters. However, the path planning is time-consuming to calculate in some scenes, and may be difficult to solve, and cannot meet the requirement of the real-time performance of automatic driving.
Disclosure of Invention
The embodiment of the specification aims to provide an automatic driving method and system based on scene adaptive recognition.
In order to solve the above technical problem, the embodiments of the present application are implemented as follows:
in a first aspect, the present application provides an automatic driving method based on scene adaptive recognition, including:
acquiring environmental information and previous-time operation information in a driving scene;
the path planning module determines a path planning track in the current driving scene based on the operation information at the last moment;
determining scene complexity according to the parameter space of the path planning track;
the simulation learning module obtains uncertainty distribution of decision by a neural network according to the environmental information;
determining scene abnormality degree according to the uncertainty distribution;
and the decision module determines an automatic driving method according to the scene complexity and the scene abnormality.
In one embodiment, the path planning module adopts a model predictive control method, including:
and predicting the motion state and the track of the vehicle in a preset time period according to the dynamic model at the current moment, and optimizing the control track at each specific moment under the condition of considering the constraint so as to ensure the optimal solution at each specific moment.
In one embodiment, optimizing the control trajectory at each specific time is based on a cost function, an acceleration limit constraint, a speed limit constraint, an obstacle avoidance constraint, and a dynamics constraint.
In one embodiment, the complexity of the scene is determined according to the parameter space of the path planning trajectory as follows: the number of constraint equations in the parameter space of the path planning trajectory is proportional to the scene complexity.
In one embodiment, the data set of the training neural network comprises a historical aerial view and corresponding operation information of the driver at the historical moment;
the historical aerial view is formed by fusing the surrounding environment information of the corresponding historical driving at the moment, wherein the aerial view is a view which is formed by RGB camera pictures with multiple visual angles and takes the vehicle as the center.
In one embodiment, the neural network has the structure: the convolution kernels of the three layers of fully-connected networks are respectively 32, 64 and 64, and the node numbers of the four layers of fully-connected networks are 1024, 512, 128 and 21 in sequence.
In one embodiment, the mimic learning module derives an uncertainty distribution of decisions from a neural network based on environmental information, comprising:
the perception result output by the neural network is as follows:
y * ＝argmax y P(y|s,w)
wherein, P (y | s, w) represents the probability that the perception model w of the neural network generates a result y after observing a scene s;
uncertainty distribution of scene U(s):
U(s)＝1-P(y * |s,w)。
in one embodiment, the scene outliers are determined from the uncertainty distribution as: the uncertainty distribution is proportional to the scene outliers.
In one embodiment, the automatic driving method is determined according to scene complexity and scene abnormality, and comprises the following steps:
if the scene complexity is larger than a first threshold and the scene abnormality degree is smaller than or equal to a second threshold, adopting the operation information determined by the simulation learning module to control the execution module;
if the scene complexity is smaller than or equal to a first threshold and the scene abnormality is larger than a second threshold, the operation information determined by the path planning module is adopted to control the execution module;
if the scene complexity is greater than a first threshold and the scene abnormality is greater than a second threshold, if a first difference value between the scene complexity and the first threshold is less than a second difference value between the scene abnormality and the second threshold, the operation information determined by the path planning module is adopted to control the execution module, and if the first difference value is greater than the second difference value, the operation information determined by the simulation learning module is adopted to control the execution module;
if the scene complexity is smaller than a first threshold and the scene abnormality is smaller than a second threshold, if the first difference is smaller than a second difference, the operation information determined by the path planning module is adopted to control the execution module, and if the first difference is larger than the second difference, the operation information determined by the simulation learning module is adopted to control the execution module.
In a second aspect, the present application provides an automatic driving system based on scene adaptive recognition, the system comprising:
the acquisition module is used for acquiring environmental information and operation information at the previous moment in a driving scene;
the path planning module is used for determining a path planning track in the current driving scene based on the operation information at the last moment;
the first determining module is used for determining scene complexity according to the parameter space of the path planning track;
the simulation learning module is used for obtaining uncertainty distribution of decision by the neural network according to the environment information;
the second determination module is used for determining scene abnormality according to the uncertainty distribution;
and the decision module is used for determining the automatic driving method according to the scene complexity and the scene abnormality.
As can be seen from the technical solutions provided in the embodiments of the present specification, the solutions: the method combines the advantages of the path planning method and the simulation learning method, can adaptively identify and analyze scene complexity and scene abnormality according to different scenes, and intelligently selects to adopt the path planning method to solve constraint or adopt the simulation learning method to calculate by a neural network. The advantages and disadvantages of the two methods in automatic driving are comprehensively considered, and the real-time performance and the accuracy of the automatic driving are improved.
Drawings
In order to more clearly illustrate the embodiments of the present specification or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the description below are only some embodiments described in the present specification, and for those skilled in the art, other drawings may be obtained according to these drawings without creative efforts.
FIG. 1 is a general block diagram of a system for automatic driving based on adaptive scene recognition provided by the present application;
FIG. 2 is a schematic flowchart of an automatic driving method based on scene adaptive recognition provided in the present application;
FIG. 3 is a schematic view of an aerial view provided herein;
fig. 4 is a schematic structural diagram of an automatic driving system based on scene adaptive recognition provided in the present application.
Detailed Description
In order to make those skilled in the art better understand the technical solutions in the present specification, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only a part of the embodiments of the present specification, and not all of the embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments in the present specification without any inventive step should fall within the scope of protection of the present specification.
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
It will be apparent to those skilled in the art that various modifications and variations can be made in the specific embodiments described herein without departing from the scope or spirit of the application. Other embodiments will be apparent to the skilled person from the description of the present application. The specification and examples are exemplary only.
As used herein, the terms "comprising," "including," "having," "containing," and the like are open-ended terms that mean including but not limited to.
In the present application, "parts" are in parts by mass unless otherwise specified.
At present, most of the driving methods which are commercially used in the field of automatic driving adopt a path planning method, and the path planning methods can be roughly divided into four types, namely methods based on graph searching, sampling, interpolation and optimization. The graph search is to search the optimal path by constructing an environment map, the sampling method represents the environment map by a sampling mode, the interpolation method is to interpolate and generate a track according to the existing reference point, and the optimization method is to construct a planning problem into an optimization problem and solve the optimization problem. The control parameters and the track of vehicle navigation can be solved based on an optimized path planning method, and meanwhile, the constraint of obstacle avoidance is considered.
A commonly used simulation learning method in the automatic driving at present is that a data set is formed by collecting environmental information (such as a camera and a laser radar) and behavior actions (such as an accelerator, steering and braking) of a driver, a large number of data sets are delivered to a neural network for training, the neural network updates neuron parameters through gradient descent, back propagation and the like, the network is enabled to continuously fit input and output pairs, and finally output according with the logic of the driver can be given according to the input.
The existing optimization-based method mainly has two defects, the first is that the final solution problem is mostly non-convex, the non-convex problem causes difficulty in solution on one hand, the optimal solution is difficult to solve in some complex scenes, and meanwhile, the calculation time is higher, so that the real-time performance cannot meet the application requirement. Some approaches solve the non-convex constraint by linearization, however, this transformation does not guarantee convergence. The second is that most current methods treat cars or obstacles as particle models or circles and do not take into account multi-dimensional shapes, such as modeling vehicles as ellipses and obstacles as polygons, which are relatively common. This limits the application of the method in some special scenarios. For example, when a vehicle reverses between two cars, it is not reasonable to treat the car as an oval.
The main defects of the existing simulation learning-based method are that the simulation learning has high limitation on scenes, the model can only well deal with scenes appearing in a data set, and scene models except the data set lose judgment, so that the simulation learning is determined to depend on the diversity of the data set seriously, and the driving behavior under the whole scene is difficult to popularize.
Based on the defects, the application provides an automatic driving method based on scene adaptive mode identification. The method is characterized in that an optimal collision-free path from a starting point to a target position is navigated based on a path planning method during navigation, a simulation learning method is used for calculating a decision in a current scene, and then scene analysis comprehensively considers the output of path planning and the output of simulation learning and selects a proper scheme. The decision scheme fully considers the advantages of the two methods.
The present invention will be described in further detail with reference to the accompanying drawings and examples.
Referring to fig. 1, a general block diagram of a system suitable for an automatic driving method based on scene adaptive recognition provided by an embodiment of the present application is shown. As shown in fig. 1, the system mainly includes: the system comprises a perception module, a path planning module, a simulation learning module and a decision module.
The sensing module is used for collecting data of the sensor and extracting environmental information required by a follow-up module. Common sensors include cameras (i.e., cameras in fig. 1), radars, uwb positioning systems (i.e., positioning in fig. 1), and the like. The camera can capture visual information of environment such as color, brightness, objects and the like; rays emitted by the radar return when hitting on an object, and the distance between the object and the radar can be calculated through the returning time, so that depth distance information is obtained; the positioning system can obtain coordinate information of the position of the positioning system. By combining the sensors and corresponding algorithms, a map of the surrounding environment and coordinates of obstacles can be constructed and processed by a subsequent module.
The path planning module is used for determining the optimal track of the current driving scene by adopting an optimal path planning method according to the operation information (including an accelerator, a brake, steering and the like) of the driver at the last moment.
The simulation learning module calculates uncertainty distribution of decision by a neural network according to the environment information acquired from the sensing module.
The decision module intelligently evaluates the scene abnormity degree caused by uncertainty distribution and the scene complexity obtained by the parameter space of the path planning track, and decides which decision scheme is selected.
Referring to fig. 2, a flow chart diagram of an automatic driving method based on scene adaptive recognition provided by an embodiment of the present application is shown.
As shown in fig. 2, the automatic driving method based on scene adaptive recognition may include:
and S210, obtaining environmental information and operation information at the previous moment in a driving scene.
Specifically, the environment information includes visual information captured by a camera, depth distance information obtained by radar measurement, self-position coordinate information obtained by a positioning system, and the like.
The last-time operation information includes information such as a linear velocity, a heading angle, and a vehicle steering angle related to an accelerator, a brake, a steering wheel, and the like.
S220, the path planning module determines a path planning track in the current driving scene based on the operation information at the last moment.
In the path planning module, the obstacles and the vehicles are modeled into a convex set, the convex set is constructed, the robot and the environment model can be considered in an optimization equation, the solving speed is accelerated, and the subsequent path planning part is based on convex set planning. The convex set may construct the shape and position of obstacles and vehicles as a generalized linear inequality: o = { x | Ax ≦ kB }, where matrices a and B are determined by the shape and size of the obstacle or robot, and the points that satisfy the inequality constitute a convex set representing the obstacle or vehicle. Unlike obstacles, the vehicle is moving at all times, so the convex set of the vehicle at each time is calculated based on the current position. The common method is to calculate the initial convex set of the vehicle (using the constructed generalized linear inequality as the initial convex set of the vehicle), and then convert the convex set according to the current position of the vehicle by a rotation matrix and a translation matrix, where the two matrices are determined by the orientation and position of the current vehicle. For example, assume the translation matrix is:
the translation is converted into: [ x, y, 1]] new ＝[x,y,1] old *Translation x,y
Similarly, the rotation is converted into: [ x, y, 1]] new ＝[x,y,1] old *Rotation x,y Concrete Rotation x,y There are different forms depending on whether the conversion is in the x-axis or in the y-axis.
Optionally, the path planning module adopts a model predictive control method, including:
and predicting the motion state and the track of the vehicle in a preset time period according to the dynamic model at the current moment, and optimizing the control track at each specific moment under the condition of considering the constraint so as to ensure the optimal solution at each specific moment.
In particular, model predictive control (model predictive control) is more commonly used in control algorithms for automatic driving, which has the advantage of enabling the control program to meet certain constraints, such as taking into account various dynamics and kinematics as constraints. The main idea of model predictive control is to predict the motion state and trajectory of a vehicle in a time window in the future based on the current dynamics model. The control trajectory at each particular time is then optimized to ensure an optimal solution at that time, taking into account the constraints. The optimization problem is divided into five parts: cost function, acceleration limit constraint, speed limit constraint, obstacle avoidance constraint and dynamics constraint. The final desired solution results are the optimal control commands and the predicted optimal trajectory over this time. And transmits the predicted control command to the vehicle for execution. In addition, the method adopts a hot start mode to save calculation time, namely, the solution for solving the problem every time can be used as the initial value of the problem at the next moment, and the solution for solving the problem every time has little difference because the movement of the vehicle in a small time unit is not too large.
The cost function is a core part of an optimization problem, the setting of the cost function determines the solving direction of the optimization equation, and the final purpose of optimization is to solve the value which enables the cost function to be minimum. The cost function is:
wherein s is a state variable of the vehicle, including coordinates and direction; u is a control variable (also called operation information) of the vehicle, including a linear speed, a steering angle and the like, s, v, a reference track and a reference speed, P and Q are weight matrixes, the weights of the vehicle running along the reference track s, the reference speed v and the reference speed can be adjusted, the larger the two values are, the more the vehicle runs according to the reference values, it can be understood that P and Q corresponding to different navigation tasks are different, N is a predicted time length, and a subscript t refers to a t-th time. This cost function represents the difference between the state and control variables of the robot and the reference values, and the direction of optimization is to minimize the difference.
The obstacle avoidance constraint is a core constraint of a planning optimization problem, the constraint ensures that the track of the vehicle cannot collide with an obstacle, and the constraint is established based on a convex set given by environmental information of a sensing module. The basis for judging whether the collision is caused is the minimum distance between the vehicle and the surrounding obstacles, and is recorded as
In order to ensure obstacle avoidance, the minimum distance needs to be controlled within a safe range, and the mathematical form is as follows:
this is the obstacle avoidance constraint of the optimization equation, and collision can be avoided by ensuring the obstacle avoidance constraint.
Different robots have different kinetic models. The Ackerman model is commonly used for vehicles and is characterized in that the Ackerman model cannot move transversely, the track only comprises a combination of straight lines and arc lines, and the radius of each arc line depends on the minimum turning radius r of the vehicle min And the turning radius is determined by the distance between the centers of the front and rear wheels of the vehicle and the maximum steering angle. The control commands of the ackerman vehicle mainly comprise linear speed and steering angle, and the dynamic model of the ackerman vehicle comprises the following components:
wherein e and
linear velocity and heading angle, respectively, alpha is the vehicle steering angle, L is the front-rear track of the vehicle, and S t And S t+1 Are states of the vehicle at different times. The constraint of the dynamic model can ensure the smoothness and feasibility of the required track.
The speed u and the acceleration a of the vehicle are constrained by the maximum value and the minimum value of the actual physical model, namely the speed u and the acceleration a are optimized within a certain range, and meanwhile, the definition domain is reduced, so that the optimization equation is convenient to solve.
In summary, the planning problem of the path planning module is to require that the vehicle can get as close to the ideal trajectory as possible while avoiding collision. The problem can be abstracted as:
aiming at the problem, the following solver is designed for iterative optimization, each iteration is divided into four steps, optimization is executed circularly, and the optimal linear speed, the optimal orientation angle and the optimal vehicle steering angle are obtained:
the method comprises the following steps: taking a control command output by a solver at the previous moment and a state of the vehicle after the vehicle executes the action as initial points; and the rapid solution of the subsequent steps is facilitated.
Step two: dynamic adjustment of safe distance d by using L1 paradigm sparsity safe (ii) a For sparse environments d safe Larger values will tend to be preferred and vice versa.
Step three: converting the constraint group into a constraint condition in a summation form by using a penalty function, and eliminating a non-convex constraint condition to ensure that all constraints of the problem are linear; the linear constraint is easier to solve, and facilitates subsequent calculation.
Step four: for the non-convex cost function, calculating the upper bound of the cost function by using an inequality method, and inputting the upper bound into an interior point method solver as a proxy function. Thus, the original non-convex problem is converted into a convex problem, and the convex problem is easier to solve. The function of the proxy function is the function of replacing the original function by another function form, so that the solution is convenient. In this step, for the problem that a is not easy to solve and is not more than b, the upper bound of a can be firstly solved as c, and the original inequality can be converted into c which is not more than b. This is the proxy function of the original inequality.
And S230, determining the scene complexity according to the parameter space of the path planning track, wherein the number of constraint equations in the parameter space of the path planning track is in direct proportion to the scene complexity.
Specifically, the scene complexity is judged by a method for researching a parameter space of path planning. The solution of the path planning depends on the dependency relationship of the established constraint equations, so that the more constraint equations (parameter quantities) represent the higher scene complexity, that is, the higher the computational solution difficulty.
And S240, the simulation learning module obtains uncertainty distribution of decision by the neural network according to the environment information.
Specifically, the simulation learning module adopts a pre-trained neural network for control.
In one embodiment, the data set for training the neural network comprises a historical aerial view and corresponding operation information of the driver at the historical moment;
the historical aerial view is formed by fusing the surrounding environment information of the corresponding historical moment driving, wherein the aerial view is a view which is formed by RGB camera pictures with multiple visual angles and takes the vehicle as the center.
Specifically, the data acquisition module mainly applies various sensors in the sensing module to acquire environmental information to produce a data set. Common sensor data are RGB cameras, depth cameras, radar, lidar, etc. The RGB camera can be used for acquiring visual information of objects around the vehicle body, and semantic information, object interaction information and the like can be extracted from the visual information. The depth camera can acquire a matrix formed by depth data of all points in a visual field, so that a depth map is constructed, and distance information of other objects can be inquired. The laser radar analyzes information such as the size of reflected energy, the amplitude, the frequency and the phase of a reflection spectrum and the like on the surface of a target object by measuring the propagation distance between the sensor transmitter and the target object, so that accurate three-dimensional structure information of the target object is presented. More sensors can bring richer information to the system, so that the system can judge the surrounding environment more accurately, but data processing and fusion among the sensors become more complex relatively, and an intelligent model is difficult to train. We consider RGB camera pictures with multiple views to form an input of a bird-view with the vehicle as the center point, as shown in fig. 3. Then an experienced human driver is arranged to perform road driving operations, and different traffic environments of different streets are selected to be driven, wherein collision is avoided and traffic rules are met. The middle holds a frequency of 15Hz, i.e. 15 data are recorded per second, each data comprising an overhead view and the corresponding driver's actions (throttle, brake, steering values). The complete drive is collected as a set of data. The larger the data size is, the more sufficient the diversity contained in the data set is, and the better the trained model is. After the acquisition is completed, the data needs to be preprocessed once. And (4) cutting and scaling the collected aerial view to enable the size of each picture to be 160-80, so that the network calculation is facilitated. For throttle, brake, steering values, the values that exceed the physical range limit are modified to the maximum/minimum values of the physical range limit, and then all data values are normalized (normalization) to have their values between-1, 1. Then processing the action value (i.e. operation value), and dividing [ -1,1] into 21 classes of 0-20 with every 0.1 value as an interval, wherein the specific calculation formula is as follows:
pred＝value*10.0+10
the neural network is then trained using the collected data set. The method adopts a convolution and full-connection structure to construct a classification network, and the network has the task of obtaining classification values corresponding to each action through input aerial view and network calculation and converting the classification values into control values for application. The specific network structure is three layers of fully-connected networks, the convolution kernels are 32, 64 and 64 respectively, four layers of fully-connected networks are provided, and the node numbers are 1024, 512, 128 and 21 in sequence. And the last layer of fully-connected network has twenty nodes, sequentially outputs probability values corresponding to all the actions, takes the node with the highest probability as a selected classification, converts the classification value into a control value through the following formula, and sends the control value to a brake system of the vehicle for execution.
value＝(pred-10)/10.0
It will be appreciated that the training goal of the mock learning is to make the strategy most approximate the driver's driving strategy, allowing the outputs at different inputs to be most closely approximated by the corresponding values in the data set. The optimization equation is as follows:
theta herein * To best approximate the driver's driving strategy parameters, s, a-D are the sampled states and actions (inputs and labels) from the data set D, loss is the error function, π (s | θ) is the output of strategy π at parameter θ given input s, and the goal of parameter optimization is to minimize the difference between the strategy of the current parameter and the strategy in the data set.
When the trained neural network is used, the current driving ambient environment information is fused into an aerial view, the aerial view is cut and scaled to the input size required by the network, and then the aerial view is input into the trained neural network to calculate so as to obtain a corresponding control result.
In one embodiment, the mimic learning module derives a decision uncertainty distribution from a neural network based on environmental information, comprising:
the perception result output by the neural network is as follows:
y * ＝argmax y P(y|s,w)
wherein, P (y | s, w) represents the probability of the perception model w of the neural network generating a result y after observing a scene s;
uncertainty distribution of scene U(s):
U(s)＝1-P(y * |s,w)。
specifically, P (ys, w) is used to represent the probability that the perceptual model w produces a result y after observing the scene s. Since the perception model selects the most probable result from all the results, the perception result finally output by the model is y * ＝argmax y P (y | s, w); the probability corresponding to this output indicates whether the perceptual model w is familiar with the observed scene. The scene uncertainty can therefore be calculated using the following formula:
U(s)＝1-P(y * |s,w)。
s250, determining scene abnormality degree according to uncertainty distribution as follows: the uncertainty distribution is proportional to the scene outliers.
Specifically, the larger the uncertainty distribution U, the more complex the scene.
In addition, the scene complexity can also be determined by the scene understanding deviation, specifically:
applying general perception complexity analysis to intelligent driving scene understanding, and preliminarily obtaining that scene understanding deviation (including classification error rate, detection error rate and tracking loss probability) and parameters of a scene understanding model are in the following functional relation:
wherein, A represents the understanding deviation of the intelligent automobile to a specific scene, (b, c) represents parameters needing fitting, (T) represents a temperature constant of Gibbs distribution, (U, V) represents a second-order derivative matrix and a first-order derivative matrix of generalization errors about a scene understanding model, and W represents the parameter quantity of the scene understanding model. The larger a represents the more complex the scene.
It can be understood that the uncertainty distribution U and the scene understanding deviation a may also be considered comprehensively to determine the scene abnormality degree, for example, the uncertainty distribution U and the scene understanding deviation a may be averaged, or a value obtained by weighted averaging or the like may also be used to determine the scene abnormality degree, which is not limited herein.
And S260, determining an automatic driving method by the decision module according to the scene complexity and the scene abnormality.
Specifically, if the scene complexity is greater than a first threshold and the scene abnormality degree is less than or equal to a second threshold, the operation information determined by the simulation learning module is adopted to control the execution module;
if the scene complexity is smaller than or equal to a first threshold and the scene abnormality is larger than a second threshold, the operation information determined by the path planning module is adopted to control the execution module;
if the scene complexity is greater than a first threshold and the scene abnormality is greater than a second threshold, if a first difference value between the scene complexity and the first threshold is less than a second difference value between the scene abnormality and the second threshold, the operation information determined by the path planning module is adopted to control the execution module, and if the first difference value is greater than the second difference value, the operation information determined by the simulation learning module is adopted to control the execution module;
if the scene complexity is smaller than a first threshold and the scene abnormality is smaller than a second threshold, if the first difference is smaller than a second difference, the operation information determined by the path planning module is adopted to control the execution module, and if the first difference is larger than the second difference, the operation information determined by the simulation learning module is adopted to control the execution module.
Specifically, the first threshold and the second threshold may be set according to actual requirements.
The two indexes of scene complexity and scene abnormality are comprehensively considered, when the scene complexity is high (namely the scene complexity is larger than a first threshold value) and the scene abnormality is low (namely the scene abnormality is smaller than or equal to a second threshold value), the result of model calculation simulating learning is adopted, so that the control can be performed more quickly and effectively, and the requirement on the real-time performance of automatic driving is met. When the scene abnormality degree is high (namely the scene abnormality degree is greater than the second threshold value) and the scene complexity is low (namely the scene complexity degree is less than or equal to the first threshold value), the current scene is a scene which is rarely appeared in the simulated learning data set, the judgment confidence of the model for the scene is low, the decision accuracy can be improved by adopting the path planning method, and possible abnormal conditions such as collision can be better avoided.
The automatic driving method based on the scene self-adaptive identification combines the advantages of the path planning method and the simulation learning method, can self-adaptively identify and analyze scene complexity and scene abnormality according to different scenes, and intelligently selects to adopt the path planning method to solve constraint or adopt the simulation learning method to calculate through a neural network. The advantages and disadvantages of the two methods in automatic driving are comprehensively considered, and the real-time performance and the accuracy of the automatic driving are improved.
The embodiment of the application combines the advantages of the two methods, can stably and reliably complete the automatic driving operation of the traffic road without collision, and has shorter computational time compared with a simple path planning method; compared with a pure simulation learning method, the method has better coping performance for training scenes with lower occurrence frequency.
Referring to fig. 4, a schematic structural diagram of an automatic driving system based on adaptive scene recognition according to an embodiment of the present application is shown.
As shown in fig. 4, the automatic driving system 400 based on scene adaptive recognition may include:
an obtaining module 410, configured to obtain environmental information in a driving scene and operation information at a previous moment;
the path planning module 420 is configured to determine a path planning track in a current driving scene based on the previous time operation information;
a first determining module 430, configured to determine scene complexity according to a parameter space of the path planning trajectory;
the simulation learning module 440 is used for obtaining uncertainty distribution of decision by the neural network according to the environment information;
a second determining module 450, configured to determine a scene abnormality degree according to the uncertainty distribution;
and the decision module 460 is used for determining an automatic driving method according to the scene complexity and the scene abnormality.
Optionally, the path planning module adopts a model predictive control method, including:
and predicting the motion state and the track of the vehicle in a preset time period according to the dynamic model at the current moment, and optimizing the control track at each specific moment under the condition of considering constraint so as to ensure the optimal solution at each specific moment.
Optionally, optimizing the control trajectory at each specific time is based on a cost function, an acceleration limit constraint, a speed limit constraint, an obstacle avoidance constraint, and a dynamics constraint.
Optionally, the number and the scene complexity of the constraint equations in the parameter space of the path planning trajectory are in direct proportion.
Optionally, the data set for training the neural network includes a historical aerial view and corresponding operation information of the driver at the historical time;
the historical aerial view is formed by fusing the surrounding environment information of the corresponding historical driving at the moment, wherein the aerial view is a view which is formed by RGB camera pictures with multiple visual angles and takes the vehicle as the center.
Optionally, the neural network has a structure: the convolution kernels of the three layers of fully connected networks are 32, 64 and 64 respectively, the nodes of the four layers of fully connected networks are 1024, 512, 128 and 21 in sequence.
Optionally, the mimic learning module 440 is further configured to:
the perception result output by the neural network is as follows:
y * ＝argmax y P(y|s,w)
wherein, P (y | s, w) represents the probability of the perception model w of the neural network generating a result y after observing a scene s;
uncertainty distribution of scene U(s):
U(s)＝1-P(y * |s,w)。
optionally, the uncertainty distribution is proportional to the scene outlier.
Optionally, the decision module 460 is further configured to:
if the scene complexity is larger than a first threshold and the scene abnormality degree is smaller than or equal to a second threshold, adopting the operation information determined by the simulation learning module to control the execution module;
if the scene complexity is smaller than or equal to a first threshold and the scene abnormality is larger than a second threshold, the operation information determined by the path planning module is adopted to control the execution module;
if the scene complexity is greater than a first threshold and the scene abnormality is greater than a second threshold, if a first difference value between the scene complexity and the first threshold is less than a second difference value between the scene abnormality and the second threshold, the operation information determined by the path planning module is adopted to control the execution module, and if the first difference value is greater than the second difference value, the operation information determined by the simulation learning module is adopted to control the execution module;
if the scene complexity is smaller than a first threshold and the scene abnormality is smaller than a second threshold, if the first difference is smaller than a second difference, the operation information determined by the path planning module is adopted to control the execution module, and if the first difference is larger than the second difference, the operation information determined by the simulation learning module is adopted to control the execution module.
The automatic driving system based on the scene adaptive recognition provided by the embodiment can execute the embodiment of the method, and the implementation principle and the technical effect are similar, and are not described herein again.
It should be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.
All the embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
Claims (10)
1. An automatic driving method based on scene adaptive recognition is characterized by comprising the following steps:
acquiring environmental information and previous-time operation information in a driving scene;
the path planning module determines a path planning track under the current driving scene based on the operation information at the last moment;
determining scene complexity according to the parameter space of the path planning track;
the simulation learning module obtains uncertainty distribution of decision by a neural network according to the environment information;
determining scene abnormality according to the uncertainty distribution;
and the decision module determines an automatic driving method according to the scene complexity and the scene abnormity degree.
2. The method of claim 1, wherein the path planning module employs a model predictive control method comprising:
and predicting the motion state and the track of the vehicle in a preset time period according to the dynamic model at the current moment, and optimizing the control track at each specific moment under the condition of considering the constraint so as to ensure the optimal solution at each specific moment.
3. The method of claim 2, wherein the optimizing the control trajectory at each particular time is based on a cost function, an acceleration limit constraint, a velocity limit constraint, an obstacle avoidance constraint, and a dynamics constraint.
4. The method of claim 1, wherein the determining the scene complexity from the parameter space of the path planning trajectory is: the number of constraint equations in the parameter space of the path planning trajectory is proportional to the scene complexity.
5. The method of claim 1, wherein the data set for training the neural network includes a historical aerial view and corresponding historical time driver operational information;
the historical aerial view is formed by fusing corresponding ambient environment information of historical driving at a moment, wherein the aerial view is a view which is formed by RGB camera pictures with multiple visual angles and takes a vehicle as a center.
6. The method of claim 1, wherein the neural network has the structure: the convolution kernels of the three layers of fully-connected networks are respectively 32, 64 and 64, and the node numbers of the four layers of fully-connected networks are 1024, 512, 128 and 21 in sequence.
7. The method of claim 1, wherein the mimic learning module derives a decision uncertainty distribution from a neural network based on the environmental information, comprising:
the perception result output by the neural network is as follows:
y * ＝argmax y P(y|s,w)
wherein, P (y | s, w) represents the probability of the perception model w of the neural network generating a result y after observing a scene s;
uncertainty distribution of scene U(s):
U(s)＝1-P(y * |s,w)。
8. the method of claim 1, wherein the determining scene outliers from the uncertainty distribution is: the uncertainty distribution is proportional to the scene outliers.
9. The method of claim 1, wherein determining an automatic driving method based on the scene complexity and the scene abnormality comprises:
if the scene complexity is larger than a first threshold and the scene abnormality degree is smaller than or equal to a second threshold, adopting the operation information determined by the simulation learning module to control the execution module;
if the scene complexity is smaller than or equal to the first threshold and the scene abnormality is larger than the second threshold, adopting the operation information determined by the path planning module to control the execution module;
if the scene complexity is greater than the first threshold and the scene abnormality is greater than the second threshold, if a first difference between the scene complexity and the first threshold is smaller than a second difference between the scene abnormality and the second threshold, the operation information determined by the path planning module is adopted to control the execution module, and if the first difference is greater than the second difference, the operation information determined by the imitation learning module is adopted to control the execution module;
if the scene complexity is smaller than the first threshold and the scene abnormality is smaller than the second threshold, if the first difference is smaller than the second difference, the operation information determined by the path planning module is adopted to control the execution module, and if the first difference is larger than the second difference, the operation information determined by the simulation learning module is adopted to control the execution module.
10. An automatic driving system based on scene adaptive recognition, characterized in that the system comprises:
the acquisition module is used for acquiring environmental information and operation information at the previous moment in a driving scene;
the path planning module is used for determining a path planning track in the current driving scene based on the operation information at the previous moment;
the first determining module is used for determining scene complexity according to the parameter space of the path planning track;
the simulation learning module is used for obtaining uncertainty distribution of decision by a neural network according to the environment information;
the second determining module is used for determining the scene abnormality degree according to the uncertainty distribution;
and the decision module is used for determining an automatic driving method according to the scene complexity and the scene abnormity degree.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211497070.3A CN115743178A (en) | 2022-11-25 | 2022-11-25 | Automatic driving method and system based on scene self-adaptive recognition |
| PCT/CN2023/133059 WO2024109763A1 (en) | 2022-11-25 | 2023-11-21 | Autonomous driving method and system based on scene adaptive recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211497070.3A CN115743178A (en) | 2022-11-25 | 2022-11-25 | Automatic driving method and system based on scene self-adaptive recognition |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115743178A true CN115743178A (en) | 2023-03-07 |
Family
ID=85338860
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211497070.3A Pending CN115743178A (en) | 2022-11-25 | 2022-11-25 | Automatic driving method and system based on scene self-adaptive recognition |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN115743178A (en) |
| WO (1) | WO2024109763A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024109763A1 (en) * | 2022-11-25 | 2024-05-30 | 中国科学院深圳先进技术研究院 | Autonomous driving method and system based on scene adaptive recognition |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10990096B2 (en) * | 2018-04-27 | 2021-04-27 | Honda Motor Co., Ltd. | Reinforcement learning on autonomous vehicles |
| CN109669461B (en) * | 2019-01-08 | 2020-07-28 | 南京航空航天大学 | Decision-making system for automatically driving vehicle under complex working condition and track planning method thereof |
| CN112418237A (en) * | 2020-12-07 | 2021-02-26 | 苏州挚途科技有限公司 | Vehicle driving decision method and device and electronic equipment |
| US12005922B2 (en) * | 2020-12-31 | 2024-06-11 | Honda Motor Co., Ltd. | Toward simulation of driver behavior in driving automation |
| CN113635909B (en) * | 2021-08-19 | 2022-07-12 | 崔建勋 | Automatic driving control method based on confrontation generation simulation learning |
| CN115743178A (en) * | 2022-11-25 | 2023-03-07 | 中国科学院深圳先进技术研究院 | Automatic driving method and system based on scene self-adaptive recognition |
-
2022
- 2022-11-25 CN CN202211497070.3A patent/CN115743178A/en active Pending
-
2023
- 2023-11-21 WO PCT/CN2023/133059 patent/WO2024109763A1/en unknown
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024109763A1 (en) * | 2022-11-25 | 2024-05-30 | 中国科学院深圳先进技术研究院 | Autonomous driving method and system based on scene adaptive recognition |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2024109763A1 (en) | 2024-05-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110834644B (en) | Vehicle control method and device, vehicle to be controlled and storage medium | |
| CN106950956B (en) | Vehicle track prediction system integrating kinematics model and behavior cognition model | |
| CN112356830B (en) | Intelligent parking method based on model reinforcement learning | |
| CN111338346A (en) | Automatic driving control method and device, vehicle and storage medium | |
| Grigorescu et al. | Neurotrajectory: A neuroevolutionary approach to local state trajectory learning for autonomous vehicles | |
| Chen et al. | Driving maneuvers prediction based autonomous driving control by deep Monte Carlo tree search | |
| US11472444B2 (en) | Method and system for dynamically updating an environmental representation of an autonomous agent | |
| Wu et al. | Deep reinforcement learning on autonomous driving policy with auxiliary critic network | |
| Aradi et al. | Policy gradient based reinforcement learning approach for autonomous highway driving | |
| Chen et al. | ES-DQN: A learning method for vehicle intelligent speed control strategy under uncertain cut-in scenario | |
| WO2024109763A1 (en) | Autonomous driving method and system based on scene adaptive recognition | |
| Xiao et al. | Vehicle trajectory prediction based on motion model and maneuver model fusion with interactive multiple models | |
| Li et al. | A deep reinforcement learning based approach for autonomous overtaking | |
| Lin et al. | Policy iteration based approximate dynamic programming toward autonomous driving in constrained dynamic environment | |
| Guidolini et al. | Neural-based model predictive control for tackling steering delays of autonomous cars | |
| Tian et al. | Personalized lane change planning and control by imitation learning from drivers | |
| Reda et al. | Path planning algorithms in the autonomous driving system: A comprehensive review | |
| Chen et al. | Automatic overtaking on two-way roads with vehicle interactions based on proximal policy optimization | |
| Jiang et al. | Event-triggered shared lateral control for safe-maneuver of intelligent vehicles | |
| Chen et al. | Deep reinforcement learning of map-based obstacle avoidance for mobile robot navigation | |
| Xiong et al. | Surrounding vehicle trajectory prediction and dynamic speed planning for autonomous vehicle in cut-in scenarios | |
| Lodhi et al. | Autonomous vehicular overtaking maneuver: A survey and taxonomy | |
| WO2023242223A1 (en) | Motion prediction for mobile agents | |
| CN116481532A (en) | Monomer unmanned aerial vehicle autonomous motion planning method based on imitation learning | |
| Hegedüs et al. | Hybrid trajectory planning for autonomous vehicles using neural networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |