CN115880654A - Vehicle lane change risk assessment method and device, computer equipment and storage medium - Google Patents
Vehicle lane change risk assessment method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN115880654A CN115880654A CN202211390972.7A CN202211390972A CN115880654A CN 115880654 A CN115880654 A CN 115880654A CN 202211390972 A CN202211390972 A CN 202211390972A CN 115880654 A CN115880654 A CN 115880654A
- Authority
- CN
- China
- Prior art keywords
- lane change
- sample data
- resnet
- lstm
- vehicle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Image Analysis (AREA)
Abstract
The application relates to a risk assessment method and device for lane change of a vehicle, computer equipment and a storage medium. The method comprises the following steps: acquiring sample data under a plurality of lane changing scenes; constructing a training data set by using the sample data, and carrying out assignment labeling on the sample data in the training data set; inputting the training data set into a ResNet-LSTM network which is set up in advance to carry out model training so as to obtain a ResNet-LSTM model; and inputting sample data corresponding to any lane change scene into the ResNet-LSTM model, and outputting a risk evaluation result. The method takes the sample data in the lane change scene as input, does not need error processing caused by methods such as environment perception, trajectory prediction and the like, extracts the spatial characteristic data based on the time dimension, and more accurately identifies the scene information, thereby more accurately evaluating the risk probability of each lane change scene.
Description
Technical Field
The present application relates to the field of vehicle technologies, and in particular, to a method and an apparatus for evaluating a risk of a vehicle lane change, a computer device, and a storage medium.
Background
With the wide application of artificial intelligence in the field of automobile automatic driving, risk assessment in various driving scenes can help an automatic driving system to predict and avoid dangerous situations.
The vehicles can meet the condition of lane changing and queue insertion of the vehicles on the side roads in the driving process. According to the big data statistics of the current traffic accidents, whether the traffic accidents are in a high-speed overhead environment or an urban road environment, the traffic accidents caused by the vehicle lane change queue insertion always account for a large proportion, so that the application of the artificial intelligence to the risk assessment of the vehicle lane change queue insertion scene is particularly wide.
Some existing technical solutions for risk assessment have the following drawbacks:
(1) In the scheme based on track and intention prediction, deep learning is utilized to carry out lane changing and track prediction on surrounding vehicles, so that whether the own vehicle has the risk of collision caused by lane changing of other vehicles in the future is judged; as shown in fig. 1, a deep learning algorithm is adopted, and an NGSIM data set is used to train and evaluate a behavior prediction model, so as to realize classification of driving intentions and obtain a trajectory probability distribution of an autonomous vehicle: constructing an intention model based on LSTM, wherein the length of a time sequence is 6, the hidden dimensionality of the LSTM is 128, the learning rate is 0.000125, and softmax cross entropy is used as training loss; and inputting the preprocessed vehicle running data and the vehicle running intention data into the track prediction model to obtain a predicted track of the vehicle, and calculating the collision time TTC, the headway TH and the enhanced collision time ETTC according to the predicted track. However, the method excessively depends on the prediction results of the vehicle intention and the track, the intermediate links are excessive, and the prediction of the intention and the track of the vehicle at present depends on the accuracy of environmental perception, so that the final risk assessment result can be overlapped with errors generated by previous links, so that the risk assessment effect is reduced, the deep learning scheme is a data-driven scheme, and collecting and mining training data of various traffic scenes from real traffic scenes is a time-consuming and labor-consuming work and needs a certain fund support.
(2) The technology adopts a single-frame original RGB image as input, performs semantic segmentation on a Mask R-CNN model trained on a COCO data set, and gives a high-resolution pixel value to a Mask part to convert the original RGB image into a Mask image, then performs feature extraction on the Mask image by using a CNN network, and further performs risk assessment and classification by using a full connection layer and an activation function. Therefore, scene recognition is performed by only depending on a single frame image, and the model cannot learn the relationship characteristics between the previous frame and the next frame, so that the model has poor robustness.
Disclosure of Invention
Based on this, it is necessary to provide a method, an apparatus, a computer device and a storage medium for risk assessment of vehicle lane change, which take a lane change frame image sequence in a lane change scene as an input, do not need to consider error processing caused by algorithms such as environmental perception and trajectory prediction, and identify scene information more accurately based on spatial feature data of a time dimension, thereby outputting a risk assessment result of each lane change scene more accurately.
The invention provides a risk assessment method for vehicle lane change, which comprises the following steps:
acquiring sample data under a plurality of lane changing scenes;
constructing a training data set by using the sample data, and carrying out assignment labeling on the sample data in the training data set;
inputting a training data set into a ResNet-LSTM network which is set up in advance, and carrying out model training to obtain a ResNet-LSTM model;
and inputting sample data corresponding to any lane change scene into the ResNet-LSTM model, and outputting a risk evaluation result.
In one embodiment, the step of obtaining sample data in several lane change scenarios includes:
based on a system and a display card environment, a UE4 engine is set up so as to load a CARLA application program by using the UE4 engine;
creating a simulation environment required by a lane change scene through a server side in a CARLA application program;
determining a first vehicle and a second vehicle through a client in a CARLA application program, wherein the first vehicle is an own vehicle body, and the second vehicle is a lane-changing vehicle relative to a side lane of the first vehicle
The lane change time, speed and degree of abruptness of the second vehicle are controlled by adjusting the lane change parameters of Python API in the CARLA application program, so that lane change frame image sequences of the second vehicle at the first vehicle view angle under different lane change scenes are generated, and the lane change frame image sequences are used as sample data under each lane change scene.
In one embodiment, the steps of constructing a training data set by using sample data and labeling the sample data in the training data set for safety or danger include:
presetting a plurality of danger grades, and endowing each danger grade with a danger coefficient;
aiming at sample data of the same lane changing scene, calibrating the danger level of the current lane changing scene for multiple times by using discrete random variables;
calculating the average value of the risk coefficient of the current lane change scene based on the preset risk coefficient of each risk level, and taking the average value of the risk coefficients as the final risk coefficient;
and obtaining the value to be labeled of the current sample data based on threshold binarization according to the intermediate values of the risk coefficients and the final risk coefficient.
In one embodiment, the step of inputting a training data set into a pre-built ResNet-LSTM network for model training to obtain a ResNet-LSTM model comprises the following steps:
performing spatial feature extraction on sample data through a ResNet structure in a ResNet-LSTM network to obtain a group of spatial feature sequences;
performing time sequence feature extraction on the spatial feature sequence through an LSTM structure in a ResNet-LSTM network to obtain spatial feature data based on time dimension;
and obtaining a safety or danger probability value by using a Softmax classifier according to the assignment of the sample data, and calculating the probability value and the assigned loss by using a cross entropy loss function.
In one embodiment, the step of performing spatial feature extraction on sample data through a ResNet structure in a ResNet-LSTM network to obtain a set of spatial feature sequences includes:
acquiring a corresponding lane change frame image sequence under each lane change scene in a training data set;
and performing spatial feature extraction on each lane change frame image in the corresponding lane change frame image sequence by using a ResBlock unit structure preset in a ResNet structure according to a preset feature dimension on the feature map so as to obtain a spatial feature sequence with the preset feature dimension.
In one embodiment, the step of performing time series feature extraction on the spatial feature sequence through an LSTM structure in a ResNet-LSTM network to obtain spatial feature data based on a time dimension includes:
acquiring a spatial feature sequence, wherein the spatial feature sequence input at each time step in the LSTM structure corresponds to the output of a ResNet structure;
iteratively updating the parameters to be learned of a hidden layer, an input gate, an output gate, a forgetting gate, a long-term memory and a short-term memory in the LSTM cell architecture and updating the cell state of the LSTM cell architecture by using the spatial feature sequence of each time step;
and performing time sequence feature extraction on the spatial feature sequence according to the updated parameters to be learned and the cell state at each time step to obtain spatial feature data based on the time dimension.
In one embodiment, the method comprises the steps of obtaining a safety or danger probability value by using a Softmax classifier according to the assignment of sample data, and calculating the probability value and the assigned loss by using a cross entropy loss function, and comprises the following steps:
acquiring spatial feature data based on a time dimension;
calculating to obtain a safety or danger probability value by using a Softmax classifier according to the assignment of the sample data;
calculating a probability value and an assigned loss using a cross entropy loss function, wherein,
wherein loss represents a loss, y i ′ Denotes the assignment of the ith sample data, y i ′ =0 for safety, y i ′ =1 means risk.
The invention provides a risk assessment device for vehicle lane change, which comprises:
the data acquisition module is used for acquiring sample data under a plurality of lane change scenes;
the data marking module is used for constructing a training data set by using the sample data and carrying out assignment marking on the sample data in the training data set;
the model training module is used for inputting a training data set into a pre-built ResNet-LSTM network for model training to obtain a ResNet-LSTM model;
and the risk evaluation module is used for inputting the sample data corresponding to any lane change scene into the ResNet-LSTM model and outputting a risk evaluation result.
The invention provides computer equipment, which comprises a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to realize the following steps:
acquiring sample data in a plurality of lane changing scenes;
constructing a training data set by using the sample data, and carrying out assignment labeling on the sample data in the training data set;
inputting the training data set into a pre-built ResNet-LSTM network for model training to obtain a ResNet-LSTM model;
and inputting sample data corresponding to any lane change scene into the ResNet-LSTM model, and outputting a risk evaluation result.
The present invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of:
acquiring sample data under a plurality of lane changing scenes;
constructing a training data set by using the sample data, and carrying out assignment and labeling on the sample data in the training data set;
inputting the training data set into a pre-built ResNet-LSTM network for model training to obtain a ResNet-LSTM model;
and inputting sample data corresponding to any lane change scene into the ResNet-LSTM model, and outputting a risk evaluation result.
In the vehicle lane change risk assessment method, the vehicle lane change risk assessment device, the computer equipment and the storage medium, the set simulation environment is utilized to obtain the sample data under a plurality of lane change scenes, so that enough model training data are provided, training data of various traffic scenes do not need to be collected and mined from real traffic scenes, the cost of manpower and material resources is reduced, the data collection process is simplified, and a target model can be trained according to the precision requirement. For example, CARLA + UE4 as a virtual data production tool can infinitely produce samples required by model training, thereby reducing the time cost and the money cost of sample collection. Furthermore, before model training, a training data set is constructed through the obtained sample data, and the sample data in the training data set is assigned and labeled so as to be applied to a Softmax classifier in an LSTM structure to calculate the probability value of safety or danger, and simplify the scene recognition process. In addition, in the step of inputting the training data set into a pre-built ResNet-LSTM network for model training to obtain a ResNet-LSTM model, a lane change frame image sequence under a lane change scene can be used as input, a track prediction result is not required to be used as input, the introduction of errors of a track prediction algorithm is avoided, the model is more independent by directly using sensor data, and the applicability is higher; and time dimension information can be increased, multi-frame input is more beneficial to a model to learn context information, the model is helpful to understand scene contents more accurately, the migration capability and robustness of the model are fully utilized, a virtual training data set can be generated infinitely theoretically, and training requirements can be met easily.
Drawings
FIG. 1 is a block diagram of a trajectory-based and intent-based prediction scheme in the prior art.
FIG. 2 is a schematic flow chart illustrating a method for assessing risk of a lane change of a vehicle according to an embodiment of the present application;
FIG. 3 is a CARLA-generated channel change frame image in an embodiment of the present application;
FIG. 4 is a diagram illustrating generation of a change frame image using CARLA in an embodiment of the present application;
FIG. 5 is a diagram of a ResNet-LSTM model in an embodiment of the present application;
FIG. 6 is a schematic diagram of a network structure for spatial feature extraction according to an embodiment of the present application;
FIG. 7 is a schematic diagram of a network architecture for timing feature extraction in an embodiment of the present application;
fig. 8 is a block diagram of a risk assessment device for a lane change of a vehicle according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The application provides a risk assessment method for vehicle lane change. Whether manual driving or automatic driving is adopted, risk assessment is carried out on a driving scene, and therefore dangerous situations can be effectively predicted and avoided. The risk assessment method for vehicle lane change, which is realized by the embodiment, can effectively break away from the dependence on the output of the environment perception algorithm and the dependence on the output of the vehicle track prediction algorithm, avoid the problem that the model cannot extract time sequence characteristics due to lack of context information, and avoid the problem that a large amount of time cost and labor cost are needed for data collection.
In one embodiment, as shown in fig. 2, a method for assessing a risk of a lane change of a vehicle is provided, which includes the following steps.
And S100, acquiring sample data in a plurality of lane change scenes.
And S200, constructing a training data set by using the sample data, and assigning and labeling the sample data in the training data set.
And step S300, inputting the training data set into a ResNet-LSTM network built in advance for model training to obtain a ResNet-LSTM model.
And S400, inputting sample data corresponding to any lane change scene into a ResNet-LSTM model, and outputting a risk evaluation result.
In one embodiment, in step S100, the step of acquiring sample data in several lane change scenes includes:
and step S110, building a UE4 engine based on the operating system and the video card environment, and loading the CARLA application program by using the UE4 engine. In one embodiment, the UE4 engine is built using the ubuntu20.04 system, NVIDIA RTX3090 graphics card environment. UE4 is called universal Engine 4, chinese is translated into 'Unreal Engine 4', and is a design Engine developed by Epic Games corporation. The CARLA application program is an automatic driving open source simulator, which is also called an automatic driving simulation platform, supports the control generation of sensors with various specifications, environments, dynamic and static NPCs (neutral point controller), is used for controlling a vehicle to run and test in a virtual environment by an automatic driving system, and can create an environment with a test close to a real environment in the virtual environment so as to control the vehicle to develop and test by the automatic driving system. And the cara application provides digital assets for cities, vehicles, sensor models, etc. The CARLA application program comprises a server side and two sides of a client side. Simulating a simulation environment through a server, receiving data in simulation and modifying the state of an object in simulation through a client, and transmitting an operation instruction to the server side through an IP address and a port by the client to realize modification.
And step S120, creating a simulation environment required by the lane change scene through a server side in the CARLA application program.
Step S130, determining a first vehicle and a second vehicle through a client in the CARLA application program, wherein the first vehicle is a self vehicle body, and the second vehicle is a lane changing vehicle relative to a side lane of the first vehicle.
Step S140, the lane change time, the speed and the degree of abruptness of the second vehicle are controlled by adjusting the lane change parameter of Python API in the CARLA application program, so as to generate a lane change frame image sequence of the second vehicle at the first vehicle view angle under different lane change scenes, and the lane change frame image sequence is used as sample data under each lane change scene.
All coordinate points of each object (actor) on the map can be acquired through Python API of the CARLA application program. In the embodiment, one coordinate point is randomly selected as a first vehicle generation point, and before the first vehicle generation point is determined, whether the selected first vehicle generation point meets the requirement of a lane change scene is judged, and if not, the first vehicle generation point is reselected; or judging whether the lane where the selected first vehicle generation point is located is a single lane, and if so, reselecting the first vehicle generation point. And after the first vehicle generation point meeting the conditions is selected, selecting a second vehicle as a lane change vehicle for the adjacent side lane and short distance from the first vehicle in the longitudinal direction. And adjusting parameters in Lane change in Python API, and controlling the speed and the degree of abruptness of the second vehicle when changing the Lane, thereby generating other Lane changing scenes which cause certain danger to the first vehicle. As shown in fig. 3-4, the lane change frame images generated using the cara application.
In one embodiment, in step S200, a training data set is constructed by using sample data, and the step of performing security or risk labeling on the sample data in the training data set further includes:
in step S210, a plurality of risk levels are set in advance, and a risk coefficient is given to each risk level.
Step S220, aiming at sample data of the same lane change scene, the danger level of the current lane change scene is calibrated for many times by using discrete random variables.
Step S230, calculating a risk coefficient mean value of the current lane change scene based on the preset risk coefficients of each risk level, and taking the mean value of the risk coefficients as a final risk coefficient.
Step S240, obtaining the assignment to be labeled of the current sample data based on threshold binarization according to the intermediate values of the risk coefficients and the final risk coefficient.
In one embodiment, five risk levels are set in advance, and for the sake of calculation, risk coefficients are given to the five risk levels: -2, -1, 0, 1, 2. The training data set contains sample data under a plurality of lane changing scenes, and for the sample data under the same lane changing scene, a mode of calibrating the danger level of the current lane changing scene for multiple times by using discrete random variables can be as follows: selecting N (for example, 20) drivers with different experiences to respectively calibrate the danger levels of the current lane-changing scene, and calculating the average value of the current danger coefficients according to the danger coefficients preset by the danger levels, wherein the danger coefficients preset in the embodiment are as follows: 2, -1, 0, 1, 2, so that the risk coefficient mean value greater than level 0 (since this time 0 is the middle value) can be classified as a risk and is represented by "1", and the risk coefficient mean value less than 0 (since this time 0 is the middle value) can be classified as a safe and is represented by "0", so that for the corresponding sample data of the current lane change scene, the obtained assignment is calculated based on the binarization of the threshold value and is represented as follows:
in an embodiment mode, step S300, inputting a training data set into a pre-built ResNet-LSTM network for model training, so as to obtain a ResNet-LSTM model.
Referring to fig. 5, the step of inputting the training data set into the pre-constructed ResNet-LSTM network for model training includes:
step S310, through ResNet structure in ResNet-LSTM network, space feature extraction is carried out on sample data to obtain at least one group of space feature sequence;
step S320, performing time sequence feature extraction on the spatial feature sequence through an LSTM structure in a ResNet-LSTM network to obtain spatial feature data based on a time dimension;
and S330, obtaining a safety or danger probability value by using a Softmax classifier according to the assignment of the sample data, and calculating the probability value and the assigned loss by using a cross entropy loss function.
And then, according to the probability value calculated by the cross entropy loss function and the loss of the assignment, continuously adjusting each parameter in the ResNet-LSTM network so as to achieve the aim of model training.
Referring to fig. 6, in an embodiment, in step S310, the step of performing spatial feature extraction on sample data through a ResNet structure in a ResNet-LSTM network to obtain a set of spatial feature sequences further includes:
acquiring a corresponding lane change frame image sequence under each lane change scene in a training data set;
and performing spatial feature extraction on each lane change frame image in the corresponding lane change frame image sequence by using a ResBlock unit structure preset in a ResNet structure and according to a preset feature dimension by using a feature map so as to obtain a spatial feature sequence with the preset feature dimension.
The basic unit structure of the ResNet structure in this embodiment is a ResBlock structure, and the ResNet structure represents a residual network structure, which can be understood as a preprocessing structure for a lane change frame image sequence in a ResNet-LSTM network structure, and of course, other models trained on a large classified data set may also be used. The ResBlock structure in this embodiment includes a three-layer structure, where the first layer compresses the number of channels of the lane-change frame image sequence by using a predetermined convolution kernel (e.g., 1*1 convolution) to obtain a feature map (feature map) of the compressed lane-change frame image sequence; and the second layer performs feature extraction on the compressed feature map, wherein the input dimension of the ResBlock structure is set as batch _ size _ sequence _ length _ 3 × 72 × 128, a spatial feature sequence with the spatial feature dimension of batch _ size _ sequence _ length _ 2048 × 1 is obtained after spatial feature extraction, and the third layer recovers the number of channels of the spatial feature sequence by using a predetermined convolution kernel (1*1 convolution).
In this embodiment, in the step of extracting spatial features of sample data by using a ResNet structure to obtain a set of spatial feature sequences, the ResNet structure sequentially includes an input layer, a convolution layer, a pooling layer, a ResBlock structure, a pooling layer, an over-fitting prevention layer, a first full-connection layer, a second full-connection layer, and an output layer. Further expressed as follows:
x j →C(64，1，1)→P→ResNet backbone→P→D→FC(200)→FC(50)→z j 。
wherein x is j ＝(x 1 ，x 2 ，...，x T ) Representing a lane change frame image sequence, C (64,1,1) represents a convolution step size of 64, convolution kernel size of (1,1), P represents pooling, resNet background represents spatial feature extraction using a ResBlock structure, D represents Dropout layer (to prevent overfitting), in one embodiment, dropout rate is set to 0.2, fc represents a fully connected layer.
Referring to fig. 7, in an embodiment, in step S320, the step of performing time series feature extraction on the spatial feature sequence through an LSTM (Long Short Term Memory) structure in the ResNet-LSTM network to obtain spatial feature data based on a time dimension further includes:
acquiring a spatial feature sequence, wherein the spatial feature sequence input at each time step in the LSTM structure corresponds to the output of a ResNet structure;
iteratively updating the parameters to be learned of a hidden layer, an input gate, an output gate, a forgetting gate, a long-term memory and a short-term memory in the LSTM cell architecture and updating the cell state of the LSTM cell architecture by using the spatial feature sequence of each time step;
and performing time sequence feature extraction on the spatial feature sequence according to the updated parameters to be learned and the cell state at each time step to obtain spatial feature data based on the time dimension.
Wherein, the spatial feature sequence of each time step input in the LSTM structure corresponds to the output of the ResNet structure, and can be expressed as:
y＝f LSTM (f ResNet50 (x 1 )，...，f ResNet50 (x j )，...，f ResNet50 (x T )，)。
the LSTM structure in this embodiment includes a forgetting gate, an input gate, and an output gate, and uses a Sigmoid function as an activation function, and uses a hyperbolic tangent function tanh as an activation function when generating a candidate memory, and the output of Sigmoid is between 0 and 1, which conforms to the physical definition of gating, and when the input is larger or smaller, the output will be very close to 1 or 0, thereby ensuring that the gate is opened or closed, and when generating a candidate memory, the tanh function is used because the output is between-1 and 1, which is consistent with the feature distribution being 0-centered in most scenarios. Furthermore, the tanh function has a larger gradient than the Sigmoid function when the input is 0, generally causing the model to converge faster. The Sigmoid function requires a certain amount of calculation for indexing, and a gate 0/1 can be used to enable the gate control output to be a discrete value of 0 or 1, namely when the input is less than the threshold value, the gate control output is 0; when the input is greater than the threshold, the output is 1. Thereby reducing the amount of computation in the case of insignificant performance degradation.
In this embodiment, the LSTM (Long Short Term Memory) structure receiving space feature sequence z = (z) 1 ，z 2 ，..，z T ) LSTM updates hidden layer parameter h = (h) 1 ，h 2 ，...，h T )，
And updating hidden layer parameters at each time step as follows:
g t ＝σ(W g z i，t +U g h t-1 +b g )
i t ＝σ(W i z i，t +U i h t-1 +b i )
o t ＝σ(W o z i，t +U o h t-1 +b o )
c t ＝g t .c t-1 +i t .tanh(W c z i，t +U c h t-1 +b c )
h t ＝o t .tanh(c t )
wherein sigma represents an activation function of sigmoid, W, U, b represents a parameter to be learned, g t An activation function representing a forgetting gate, o t Indicating the activation function of the output gate, i t Representing activation functions of input gates, c t Indicating the state of the cell.
In an embodiment, in step S330, according to the assignment of sample data, a Softmax classifier is used to obtain a safety or danger probability value, and then a cross entropy loss function is used to calculate a probability value and a loss of the assignment, further including:
acquiring spatial feature data based on a time dimension;
calculating to obtain a safety or danger probability value by using a Softmax classifier according to the assignment of the sample data;
the probability values and assigned penalties are calculated using a cross entropy penalty function, wherein,
wherein loss represents loss, y i ' denotes an assignment of ith sample data, y i ' =0 denotes safety, y i ' =1 denotes a danger.
Further, the operation process of the time sequence feature extraction and classification in the present embodiment is as follows:
z j →Z→LSTM(q，20)→Softmax(2)→y
wherein, LSTM (q, h) represents that the time step of the LSTM structure is q, and h represents the dimension of the hidden layer. The Softmax activation function is used for calculating to obtain the probability value y of safety or danger i ，0≤y i And (5) selecting the cross entropy as a loss function to calculate the loss of the probability value and the assignment value when the cross entropy is less than or equal to 1.
In this embodiment, the calculation of the probability value and the assigned loss according to the cross entropy loss function during training is as follows:
wherein loss represents a loss, y i ' denotes the assignment of the i-th sample, y i ' =0 denotes safety, y i ' =1 denotes a danger.
Furthermore, the ResNet-LSTM model obtained in the step takes the sample data of the lane change scene as input, and takes the risk probability of the lane change scene as output. However, in the course of training the model of the ResNet-LSTM network, it is necessary to continuously adjust each operation parameter by using the probability value calculated by the cross entropy loss function and the assigned loss. In further illustration, the sample data is represented as a set of channel change frame image sequences x i ＝(x 1 ，x 2 ，...，x T ) T represents the length of the time sequence, after the time sequence is input into a ResNet-LSTM network, the spatial feature extraction of each lane change frame image is firstly carried out, a group of spatial feature sequences based on the lane change scene is obtained, then the time sequence feature extraction is carried out on the group of spatial feature sequences, so that the dynamic change of the spatial features in the spatial feature sequences based on the spatial features under the time dimension is determined, therefore, the up-and-down dimension correlation of any spatial feature based on the time dimension is convenient, in the embodiment, the LSTM structure is selected to carry out the time sequence extraction and classification, and the ResNet-LSTM model generated by training can be understood as a preprocessing model and a deep learning model f, y i ＝(x i ) In this embodiment, an encoded one-hot array can be represented by Y,
in the vehicle lane change risk assessment method, the set simulation environment is utilized to obtain the sample data under a plurality of lane change scenes, so that enough model training data are provided, training data of various traffic scenes do not need to be collected and mined from real traffic scenes, the cost of manpower and material resources is reduced, the data collection process is simplified, and a target model can be trained according to the precision requirement. For example, CARLA + UE4 as a virtual data production tool can infinitely produce samples required by model training, and reduce the time cost and money cost of sample collection.
The model training of the embodiment is carried out based on a pytorch deep learning framework, the training optimizer selects Adam, the learning rate selects 0.0001, the attenuation index selects 0.01, the batch size selects 32, the epoch selects 1000, and the training verification proportion is 9: and 1, loading parameters of a pre-training model ResNet during training. In the actual use process, reading 20 frames of images collected by a front visible light sensor, then loading a model, pushing the images in, carrying out a series of calculations on the model, wherein the calculation process is the same as the training process, and calculating and outputting a two-dimensional array: [ safe _ score, risk _ score ], where safe _ score + risk _ score =1, the output at this time is the risk assessment result corresponding to the lane change scenario and is directly used as the output result of the module for subsequent use.
Before model training, a training data set is constructed through the obtained sample data, and the sample data in the training data set is assigned and labeled so as to be applied to a Softmax classifier in an LSTM structure to obtain a safety or danger probability value and simplify a scene recognition process.
In the step of inputting the training data set into a pre-built ResNet-LSTM network for model training to obtain a ResNet-LSTM model, a lane change frame image sequence under a lane change scene is used as input, a result of trajectory prediction is not required to be used as input, the introduction of errors of a trajectory prediction algorithm is avoided, and the model is more independent and higher in applicability due to the direct use of sensor data; and the time dimension information is increased, the multi-frame input is more beneficial to the model to learn the context information, the model is more accurately helpful to understand the scene content, the migration capability and robustness of the model are fully utilized, the virtual training data set can be generated infinitely theoretically, and the training requirement can be easily met.
In one embodiment, as shown in fig. 8, there is provided a risk assessment device for a vehicle lane change, including: a data acquisition module 100, a data annotation module 200, a model training module 300, and a risk assessment module 400, wherein:
the data obtaining module 100 is configured to obtain sample data in a plurality of lane change scenarios.
The data labeling module 200 is configured to construct a training data set by using the sample data, and perform assignment labeling on the sample data in the training data set.
The model training module 300 is used for inputting a training data set into a pre-built ResNet-LSTM network for model training to obtain a ResNet-LSTM model.
The risk evaluation module 400 is configured to input sample data corresponding to any lane change scenario into the ResNet-LSTM model, and output a risk evaluation result.
For the specific definition of the risk assessment device for vehicle lane change, reference may be made to the above definition of the risk assessment method for vehicle lane change, which is not described herein again. The modules in the vehicle lane change risk assessment device can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent of a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing risk assessment data of vehicle lane change. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method for risk assessment of a vehicle lane change.
In one embodiment, a computer device is provided, which may be a terminal. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method for risk assessment of a vehicle lane change. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
Those skilled in the art will appreciate that the foregoing description is by way of example only and is not intended to limit the scope of the invention in any way, as the term "computer device" may include more or less components than those shown, or may be combined in a different manner, or may have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having a computer program stored therein, the processor implementing the following steps when executing the computer program:
acquiring sample data in a plurality of lane changing scenes;
constructing a training data set by using the sample data, and carrying out assignment and labeling on the sample data in the training data set; inputting the training data set into a pre-built ResNet-LSTM network for model training to obtain a ResNet-LSTM model; and inputting sample data corresponding to any lane change scene into the ResNet-LSTM model, and outputting a risk evaluation result. In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, performs the steps of: acquiring sample data in a plurality of lane changing scenes; constructing a training data set by using the sample data, and carrying out safety or danger marking on the sample data in the training data set; inputting the training data set into a pre-built ResNet-LSTM network for model training to obtain a ResNet-LSTM model; and inputting sample data corresponding to any lane change scene into the ResNet-LSTM model, and outputting a risk evaluation result.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware related to instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), rambus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above examples only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A method for risk assessment of lane change of a vehicle, the method comprising:
acquiring sample data in a plurality of lane changing scenes;
constructing a training data set by using the sample data, and carrying out assignment and labeling on the sample data in the training data set;
inputting the training data set into a pre-built ResNet-LSTM network for model training to obtain a ResNet-LSTM model;
and inputting sample data corresponding to any lane change scene into the ResNet-LSTM model, and outputting a risk evaluation result.
2. The method of claim 1, wherein the step of obtaining sample data in several lane change scenarios comprises:
based on an operating system and a video card environment, a UE4 engine is built, and a CARLA application program is loaded by using the UE4 engine;
creating a simulation environment required by the lane change scene through a server side in the CARLA application program;
determining a first vehicle and a second vehicle through a client in the CARLA application program, wherein the first vehicle is an own vehicle body, and the second vehicle is a lane-changing vehicle relative to a side lane of the first vehicle;
and controlling lane change time, speed and degree of abruptness of a second vehicle by adjusting a lane change parameter of Python API in the CARLA application program to generate a lane change frame image sequence of the second vehicle at the first vehicle view angle under different lane change scenes, and taking the lane change frame image sequence as the sample data under each lane change scene.
3. The method of claim 1, wherein the step of constructing a training data set using the sample data and performing assignment labeling on the sample data in the training data set comprises:
presetting a plurality of danger grades, and endowing each danger grade with a danger coefficient;
aiming at sample data of the same lane changing scene, calibrating the danger level of the current lane changing scene for multiple times by using discrete random variables;
calculating the average value of the risk coefficients of the current lane change scene based on the risk coefficients of the risk grades, and taking the average value of the risk coefficients as a final risk coefficient;
and obtaining the assignment to be labeled of the current sample data based on threshold binarization according to the intermediate values of the risk coefficients and the final risk coefficient.
4. The method of claim 1, wherein the step of inputting the training data set into a pre-built ResNet-LSTM network for model training to obtain a ResNet-LSTM model comprises:
performing spatial feature extraction on the sample data through a ResNet structure in the ResNet-LSTM network to obtain at least one group of spatial feature sequences;
performing time sequence feature extraction on the spatial feature sequence through an LSTM structure in the ResNet-LSTM network to obtain spatial feature data based on a time dimension;
and obtaining a safety or danger probability value by using a Softmax classifier according to the assignment of the sample data, and calculating the probability value and the assigned loss by using a cross entropy loss function.
5. The method of claim 4, wherein the step of performing spatial feature extraction on the sample data through a ResNet structure in the ResNet-LSTM network to obtain a set of spatial feature sequences comprises:
acquiring the lane change frame image sequence corresponding to each lane change scene in the training data set;
and performing spatial feature extraction on each lane change frame image in the corresponding lane change frame image sequence by using a ResBlock unit structure preset in the ResNet structure and according to a preset feature dimension by using a feature map so as to obtain a spatial feature sequence with the preset feature dimension.
6. The method of claim 1, wherein the step of performing temporal feature extraction on the spatial feature sequence through an LSTM structure in the ResNet-LSTM network to obtain spatial feature data based on a time dimension comprises:
acquiring the spatial feature sequence, wherein the spatial feature sequence input at each time step in the LSTM structure corresponds to the output of the ResNet structure;
iteratively updating parameters to be learned of a hidden layer, an input gate, an output gate, a forgetting gate, a long-term memory and a short-term memory in the LSTM cell architecture and updating the cell state of the LSTM cell architecture by using the spatial feature sequence of each time step;
and performing time sequence feature extraction on the spatial feature sequence according to the updated parameters to be learned and the cell state at each time step to obtain spatial feature data based on the time dimension.
7. The method of claim 4, wherein the step of obtaining a safety or danger probability value by using a Softmax classifier according to the sample data assignment, and calculating the probability value and the assigned loss by using a cross entropy loss function comprises:
acquiring the spatial feature data based on the time dimension;
calculating to obtain a safety or danger probability value by using a Softmax classifier according to the assignment of the sample data;
calculating a probability value and an assigned loss using a cross entropy loss function, wherein,
wherein loss represents loss, y i ' denotes an assignment of ith sample data, y i ' =0 denotes safety, y i ' =1 denotes a danger.
8. A risk assessment device for a lane change of a vehicle, characterized in that the device comprises:
the data acquisition module is used for acquiring sample data under a plurality of lane change scenes;
the data labeling module is used for constructing a training data set by using the sample data and carrying out assignment labeling on the sample data in the training data set;
the model training module is used for inputting the training data set into a pre-built ResNet-LSTM network for model training to obtain a ResNet-LSTM model;
and the risk evaluation module is used for inputting the sample data corresponding to any lane change scene into the ResNet-LSTM model and outputting a risk evaluation result.
9. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps of the method of any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211390972.7A CN115880654A (en) | 2022-11-07 | 2022-11-07 | Vehicle lane change risk assessment method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211390972.7A CN115880654A (en) | 2022-11-07 | 2022-11-07 | Vehicle lane change risk assessment method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115880654A true CN115880654A (en) | 2023-03-31 |
Family
ID=85759530
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211390972.7A Pending CN115880654A (en) | 2022-11-07 | 2022-11-07 | Vehicle lane change risk assessment method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115880654A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116502117A (en) * | 2023-04-13 | 2023-07-28 | 厦门市帕兰提尔科技有限公司 | ResNet-based hazardous chemical identification method, device and equipment |
-
2022
- 2022-11-07 CN CN202211390972.7A patent/CN115880654A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116502117A (en) * | 2023-04-13 | 2023-07-28 | 厦门市帕兰提尔科技有限公司 | ResNet-based hazardous chemical identification method, device and equipment |
| CN116502117B (en) * | 2023-04-13 | 2023-12-15 | 厦门市帕兰提尔科技有限公司 | ResNet-based hazardous chemical identification method, device and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109978893B (en) | Training method, device, equipment and storage medium of image semantic segmentation network | |
| US20210342997A1 (en) | Computer Vision Systems and Methods for Vehicle Damage Detection with Reinforcement Learning | |
| Peng et al. | Uncertainty evaluation of object detection algorithms for autonomous vehicles | |
| EP3757905A1 (en) | Deep neural network training method and apparatus | |
| CN113033604B (en) | Vehicle detection method, system and storage medium based on SF-YOLOv4 network model | |
| CN111126258A (en) | Image recognition method and related device | |
| CN112487954B (en) | Pedestrian crossing behavior prediction method for plane intersection | |
| CN113468967A (en) | Lane line detection method, device, equipment and medium based on attention mechanism | |
| CN112418278A (en) | Multi-class object detection method, terminal device and storage medium | |
| CN111144304A (en) | Vehicle target detection model generation method, vehicle target detection method and device | |
| EP3816864A1 (en) | Device and method for the generation of synthetic data in generative networks | |
| CN113469088A (en) | SAR image ship target detection method and system in passive interference scene | |
| CN115546519B (en) | Matching method of image and millimeter wave radar target for extracting pseudo-image features | |
| CN114549369B (en) | Data restoration method and device, computer and readable storage medium | |
| CN111523489A (en) | Generation method of age classification network, and vehicle-mounted person detection method and device | |
| CN115880654A (en) | Vehicle lane change risk assessment method and device, computer equipment and storage medium | |
| CN116964588A (en) | Target detection method, target detection model training method and device | |
| CN113537020B (en) | Complex SAR image target identification method based on improved neural network | |
| CN114399638A (en) | Semantic segmentation network training method, equipment and medium based on patch learning | |
| CN117011616A (en) | Image content auditing method and device, storage medium and electronic equipment | |
| Zia et al. | Text-to-image generation with attention based recurrent neural networks | |
| CN114445716B (en) | Key point detection method, key point detection device, computer device, medium, and program product | |
| CN112699809B (en) | Vaccinia category identification method, device, computer equipment and storage medium | |
| CN111539420B (en) | Panoramic image saliency prediction method and system based on attention perception features | |
| CN114677662A (en) | Method, device, equipment and storage medium for predicting vehicle front obstacle state |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20230924 Address after: Room A101, Building I, No. 7 Zhongchuang Second Road, Hangzhou Bay New Area, Ningbo City, Zhejiang Province, 315335 Applicant after: Ningbo Lutes Robotics Co.,Ltd. Address before: 430056 A504, Building 3, No. 28, Chuanjiangchi Second Road, Wuhan Economic and Technological Development Zone, Wuhan, Hubei Province Applicant before: Wuhan Lotus Technology Co.,Ltd. |
|
| TA01 | Transfer of patent application right |