CN115214708A

CN115214708A - Vehicle intention prediction method and related device thereof

Info

Publication number: CN115214708A
Application number: CN202110420831.4A
Authority: CN
Inventors: 范时伟; 李向旭; 李飞
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2021-04-19
Filing date: 2021-04-19
Publication date: 2022-10-21

Abstract

The embodiment of the application provides a vehicle intention prediction method, which is applied to the field of automatic driving and comprises the following steps: acquiring the position of a target vehicle; determining a plurality of candidate lane groups which can be driven by the target vehicle within a preset distance according to the position of the target vehicle, wherein the lane direction difference of different candidate lane groups within the preset distance is greater than a target threshold value; acquiring road condition information of each candidate lane group in the plurality of candidate lane groups; and determining a target candidate lane group from the candidate lane groups according to the road condition information, wherein the lane direction of the target candidate lane group is used as the driving intention of the target vehicle. According to the method and the device, the driving intention of the target vehicle is defined as the lane direction of the lane which is most likely to run, and the driving intention of the vehicle can be more accurately represented by the lane direction compared with directional intentions such as left-turn and right-turn aiming at complex road conditions.

Description

Vehicle intention prediction method and related device thereof

Technical Field

The present application relates to the field of artificial intelligence, and in particular, to a vehicle intention prediction method and related apparatus.

Background

The driving intention may refer to a driving strategy to be performed by the vehicle in the future, and specifically, the driving intention of the vehicle may be estimated according to the road condition information, the driving state and other information of the vehicle. The vehicle trajectory prediction means predicting the position of the vehicle at each time point within a certain time in the future.

In the field of automatic driving, the driving intentions of surrounding vehicles are estimated accurately and reliably in real time, and the future driving tracks of the vehicles are predicted, so that the self-vehicle can be helped to predict the traffic condition ahead, the traffic situation around the self-vehicle is established, the importance judgment of other vehicle targets around the self-vehicle is facilitated, interactive key targets are screened, the self-vehicle can plan the path in advance, and the self-vehicle can safely pass through complex scenes.

In the prior art, driving intentions are defined as directional intentions such as straight driving, left turning, right turning and the like, but the definition mode of the driving intentions has limited representation capability in a complicated road scene and cannot cover all the driving intentions.

Disclosure of Invention

In a first aspect, the present application provides a vehicle intention prediction method, comprising:

acquiring the position of a target vehicle;

in the driving process of the self vehicle, in order to accurately predict whether other surrounding vehicles influence the driving safety of the self vehicle, influence the driving decision of the self vehicle and control the driving strategy of the self vehicle based on the surrounding vehicles, the driving intention of at least one related vehicle positioned around the self vehicle needs to be determined. The target vehicle in the embodiment of the application is any one of at least one associated vehicle located around the own vehicle;

determining a plurality of candidate lane groups which can be driven by the target vehicle within a preset distance according to the position of the target vehicle, wherein the lane direction difference of different candidate lane groups within the preset distance is greater than a target threshold value;

the lane where the target vehicle can travel can be understood as a lane where the target vehicle can travel within a preset distance on the premise of not violating constraints such as a back crossing rule;

in order to define the driving intention of the target vehicle more accurately, the lane direction of the lane which is most likely to be driven by the target vehicle in the future can be used as the driving intention of the target vehicle, so that firstly, it needs to be determined which lanes are likely to be driven by the target vehicle within a preset distance in the future, and the lane which is likely to be driven by the target vehicle within the preset distance can be used as a candidate lane;

the preset distance can be preset, and specifically depends on the driving intention of the target vehicle in which distance the target vehicle is expected to be predicted; alternatively, the preset distance may be determined based on the traveling speed of the target vehicle and the predicted time required for the traveling intention;

acquiring road condition information of each candidate lane group in the plurality of candidate lane groups; and determining a target candidate lane group from the candidate lane groups according to the road condition information, wherein the lane direction of the target candidate lane group is used as the driving intention of the target vehicle.

According to the embodiment of the application, the driving intention of the target vehicle is defined as the lane direction of the lane which is most likely to run, and the driving intention of the vehicle can be more accurately represented by the lane direction compared with the directional intentions of left turning, right turning and the like for the complex road conditions.

In one possible implementation, each of the candidate lane groups includes at least one lane, and a lane direction difference of the lanes included in each of the candidate lane groups within the preset distance is smaller than the target threshold.

Since the lane direction of the lane which is most likely to be driven by the target vehicle in the future is taken as the driving intention in the embodiment of the present application, and usually a part of the lanes in the candidate lanes have the same or similar lane directions, the plurality of candidate lanes may be clustered based on the direction of the lane line to obtain a plurality of candidate lane groups, where the lane direction difference of different candidate lane groups within the preset distance is greater than the target threshold, each candidate lane group includes at least one lane, and the lane direction difference between the lanes included in each candidate lane group is less than the target threshold.

In one possible implementation, the lane direction is a direction of a lane line within the preset distance from a position of the target vehicle as a starting point. That is, the lane direction indicates an extending direction of a lane line of a lane within a preset distance, not just a direction of the lane line at a certain point. Specifically, the lane direction may refer to a direction indicated by a trajectory in which a lane line is present, for example, the lane direction may refer to a trajectory direction in which the vehicle travels along the trajectory in which the lane line is present.

In one possible implementation, the preset distance is a fixed value less than 200 meters; or the preset distance is obtained by calculating the traveling speed of the target vehicle and the intention prediction time, wherein the preset distance is positively correlated with the traveling speed of the target vehicle and the intention prediction time, and the intention prediction time is less than or equal to 7 seconds.

In one possible implementation, the method further comprises: acquiring a historical driving route of the target vehicle; should according to this road conditions information, confirm the target candidate lane group from this multiunit candidate lane group, include: and determining a target candidate lane group from the plurality of candidate lane groups according to the road condition information and the historical driving route. In addition, besides acquiring the road condition information of each candidate lane group in the plurality of candidate lane groups, the historical driving route of the target vehicle, or other driving state information may also be acquired, and the driving state information may include, but is not limited to, speed, acceleration, steering angle relative to the lane center line, and the like, and is not limited herein.

In one possible implementation, the traffic information includes at least one of the following information:

lane center line information, travelable area information, obstacle information, speed limit area information.

The lane center line information may indicate a position of a center line of the lane and a center line extending direction.

The lane may include a drivable area and an undrivable area, and the drivable area information may refer to an area that can be driven on the lane specified by the intersection rule.

The obstacle information may also be referred to as barrier information, for example, the obstacle may refer to an object that may obstruct the target vehicle from traveling, such as a motor vehicle, a non-motor vehicle, a barrier, a pedestrian, an animal, or may refer to a road end, a collapsed road section, a depressed road surface, or an intersection.

The speed-limiting area information may refer to a blind sight area of a vehicle, a speed-limiting road section such as a campus and the like, congestion information and the like.

In one possible implementation, the traffic information is represented as a rasterized image, and the rasterized image includes a plurality of image channels, each image channel being used for representing at least one of the traffic information.

According to the embodiment of the application, different image channels correspond to different road condition information in a grid image coding mode, so that the road condition information is more accurately expressed and is easier to learn by a neural network.

In one possible implementation, the method further comprises: and determining a target predicted track of the target vehicle through a neural network model according to the road condition information of the target candidate lane group.

In order to be able to predict the intention of the target vehicle more accurately, and also to be able to calculate a target predicted trajectory of the target vehicle, in one possible implementation the method further comprises: acquiring historical driving routes of the target vehicle and at least one associated vehicle positioned around the target vehicle; determining the driving influence degree of the at least one associated vehicle on the target vehicle through an attention mechanism according to the historical driving route; the step of determining the target predicted track of the target vehicle through a neural network model according to the road condition information of the target candidate lane group comprises the following steps: and determining a target predicted track of the target vehicle through a neural network model according to the road condition information of the target candidate lane group and the driving influence degree.

In one possible implementation, the determining the target predicted trajectory of the target vehicle through the neural network model according to the road condition information of the target candidate lane group includes:

determining a plurality of candidate tracks of the target vehicle and the confidence coefficient of each candidate track through the neural network model according to the road condition information of the target candidate lane group;

and determining a target predicted track of the target vehicle from the plurality of candidate tracks according to the confidence.

The neural network model may include a plurality of prediction branches, each prediction branch having a capability of predicting a different prediction track type, each of the plurality of prediction branches may determine a candidate track of the target vehicle and a confidence of the candidate track according to the road condition information of the target candidate lane group, and then the candidate track with the highest confidence may be selected as the target prediction track, which may be the predicted track of the target vehicle. The above-mentioned prediction branches may also be referred to as neural network submodels in subsequent embodiments.

In a second aspect, the present application provides a model training method, including:

acquiring a first neural network model, the position of a target vehicle and a real driving track, wherein the real driving track of the target vehicle is positioned on a target lane;

acquiring road condition information of each candidate lane group in the plurality of candidate lane groups;

and determining a target candidate lane group from the candidate lane groups through the first neural network model according to the road condition information, and training the first neural network model based on the difference between the target candidate lane group and the candidate lane group where the target lane is located to obtain a second neural network model.

In one possible implementation, the determining, according to the traffic information, a target candidate lane group from the candidate lane groups through the first neural network model, and training the first neural network model based on a difference between the target candidate lane group and a candidate lane group in which the target lane is located includes:

determining the selection probability of each candidate lane through the first neural network model according to the road condition information of each candidate lane group;

acquiring the true probability of each candidate lane, wherein the true probability of the candidate lane group in which the target lane is located is 1, and the true probability of the candidate lane group in which the target lane is not located is 0;

training the first neural network model according to a difference between the selected probability of each candidate lane and the true probability of each candidate lane.

In one possible implementation, the method further comprises:

acquiring a third neural network model;

determining a target predicted track of the target vehicle through the third neural network model according to the road condition information of the candidate lane group where the target lane is located;

and training the third neural network model based on the difference between the target predicted track and the real driving track to obtain a fourth neural network model.

In one possible implementation, the determining the predicted target trajectory of the target vehicle through the third neural network model according to the traffic information of the candidate lane group in which the target lane is located includes:

determining a plurality of predicted tracks of the target vehicle through the plurality of neural network submodels according to the road condition information of the candidate lane group where the target lane is located, wherein each neural network submodel is used for determining one predicted track;

and determining a track with a difference smaller than a threshold value from the plurality of predicted tracks as the target predicted track according to the difference between each predicted track in the plurality of predicted tracks and the real driving track.

In one possible implementation, the determining, by the plurality of neural network submodels, a plurality of predicted trajectories of the target vehicle includes:

determining a plurality of predicted trajectories of the target vehicle and a confidence level of each predicted trajectory through the plurality of neural network submodels;

the training of the third neural network model based on the difference between the target predicted trajectory and the actual driving trajectory to obtain a fourth neural network model includes:

and training the third neural network model based on the difference between the target predicted track and the real driving track and the difference between the confidence coefficient of the target predicted track and the numerical value 1 to obtain a fourth neural network model.

In a third aspect, the present application provides a vehicle intention prediction apparatus comprising:

the acquisition module is used for acquiring the position of a target vehicle; the acquisition module is further used for acquiring the road condition information of each candidate lane group in the plurality of candidate lane groups determined by the candidate determination module;

the lane determining module is used for determining a plurality of candidate lane groups which can be driven by the target vehicle within a preset distance according to the position of the target vehicle, wherein the lane direction difference of different candidate lane groups within the preset distance is greater than a target threshold value; the lane determining module is further configured to determine a target candidate lane group from the plurality of candidate lane groups according to the road condition information, and a lane direction of the target candidate lane group is used as a driving intention of the target vehicle.

In one possible implementation, the lane direction is a direction of a lane line within the preset distance from a position of the target vehicle as a starting point.

In one possible implementation, the preset distance is a fixed value less than 200 meters; or,

the preset distance is obtained by calculating the traveling speed of the target vehicle and the intention prediction time, wherein the preset distance is positively correlated with the traveling speed of the target vehicle and the intention prediction time, and the intention prediction time is less than or equal to 7 seconds.

In one possible implementation, the obtaining module is further configured to obtain a historical driving route of the target vehicle;

the lane determining module is specifically configured to determine a target candidate lane group from the multiple candidate lane groups according to the road condition information and the historical driving route.

lane center line information, travelable area information, obstacle information, and speed limit area information.

In one possible implementation, the apparatus further comprises:

and the track prediction module is used for determining a target prediction track of the target vehicle through a neural network model according to the road condition information of the target candidate lane group.

In one possible implementation, the obtaining module is further configured to obtain historical driving routes of the target vehicle and at least one associated vehicle located around the target vehicle;

the track prediction module is used for determining the driving influence degree of the at least one associated vehicle on the target vehicle through an attention mechanism according to the historical driving route;

and determining a target predicted track of the target vehicle through a neural network model according to the road condition information of the target candidate lane group and the driving influence degree.

In a possible implementation, the trajectory prediction module is specifically configured to determine, according to the road condition information of the target candidate lane group, a plurality of candidate trajectories of the target vehicle and a confidence level of each candidate trajectory through the neural network model;

In a fourth aspect, the present application provides a model training apparatus, comprising:

the acquisition module is used for acquiring the first neural network model, the position of the target vehicle and a real driving track, wherein the real driving track of the target vehicle is positioned on a target lane; the acquisition module is further used for acquiring the road condition information of each candidate lane group in the plurality of candidate lane groups determined by the lane determination module;

the lane determining module is used for determining a plurality of candidate lane groups which can be driven by the target vehicle within a preset distance according to the position of the target vehicle, wherein the lane direction difference of different candidate lane groups within the preset distance is larger than a target threshold value;

and the model training module is used for determining a target candidate lane group from the candidate lane groups through the first neural network model according to the road condition information acquired by the acquisition module, and training the first neural network model based on the difference between the target candidate lane group and the candidate lane group where the target lane is located to obtain a second neural network model.

In one possible implementation, the model training module is specifically configured to determine, according to the road condition information of each candidate lane group, a selection probability of each candidate lane through the first neural network model;

the first neural network model is trained based on a difference between the selected probability of each candidate lane and the true probability of each candidate lane.

In a possible implementation, the obtaining module is further configured to obtain a third neural network model;

the device also includes:

the track prediction module is used for determining a target prediction track of the target vehicle through the third neural network model according to the road condition information of the candidate lane group where the target lane is located;

the model training module is further configured to train the third neural network model based on a difference between the target predicted trajectory and the actual driving trajectory to obtain a fourth neural network model.

In one possible implementation, the third neural network model includes a plurality of neural network submodels, and the trajectory prediction module is specifically configured to determine, according to the road condition information of the candidate lane group where the target lane is located, a plurality of predicted trajectories of the target vehicle through the plurality of neural network submodels, where each neural network submodel is configured to determine one predicted trajectory;

In one possible implementation, the trajectory prediction module is specifically configured to determine a plurality of predicted trajectories of the target vehicle and a confidence level of each predicted trajectory through the plurality of neural network submodels;

In a fifth aspect, the present application provides a server comprising a memory and a processor; the memory stores code and the processor is configured to execute the code, and when executed, the server performs the method as described in the first aspect or any one of the possible implementations of the first aspect.

In a sixth aspect, the present application provides a server comprising a memory and a processor; the memory stores code and the processor is configured to execute the code, and when executed, the server performs the method as described in the second aspect or any one of the possible implementations of the second aspect.

In a seventh aspect, the present application provides a vehicle including the vehicle intention prediction apparatus according to the third aspect.

In an eighth aspect, the present application provides a computer storage medium storing a computer program which, when executed by a computer, causes the computer to perform the method of the first aspect or any one of the possible implementations of the first aspect.

In a ninth aspect, the present application provides a computer storage medium storing a computer program which, when executed by a computer, causes the computer to carry out the method according to the second aspect or any one of the possible implementations of the second aspect.

In a tenth aspect, the present application provides a computer program product storing instructions that, when executed by a computer, cause the computer to perform the method of the first aspect or any one of the possible implementations of the first aspect.

In an eleventh aspect, the present application provides a computer program product storing instructions that, when executed by a computer, cause the computer to perform the method as set forth in the second aspect or any one of the possible implementations of the second aspect.

In a twelfth aspect, the present application provides a chip system, which includes a processor, configured to support an executing device or a training device to implement the functions mentioned in the above aspects, for example, to transmit or process data mentioned in the above methods; or, information. In one possible design, the system-on-chip further includes a memory for storing program instructions and data necessary for the execution device or the training device. The chip system may be formed by a chip, or may include a chip and other discrete devices.

The embodiment of the application provides a vehicle intention prediction method, which comprises the following steps: acquiring the position of a target vehicle; determining a plurality of candidate lane groups which can be driven by the target vehicle within a preset distance according to the position of the target vehicle, wherein the lane direction difference of different candidate lane groups within the preset distance is greater than a target threshold value; acquiring road condition information of each candidate lane group in the plurality of candidate lane groups; and determining a target candidate lane group from the candidate lane groups according to the road condition information, wherein the lane direction of the target candidate lane group is used as the driving intention of the target vehicle. According to the embodiment of the application, the driving intention of the target vehicle is defined as the lane direction of the lane which is most likely to run, and the driving intention of the vehicle can be more accurately represented by the lane direction compared with the directional intentions of left turning, right turning and the like for the complex road conditions.

Drawings

FIG. 1a is a schematic structural diagram of an artificial intelligence body framework;

FIG. 1b is a schematic representation of a road condition;

FIG. 1c is a functional block diagram of an autopilot device having an autopilot function according to an embodiment of the present application;

FIG. 2 is a flow chart illustrating a vehicle intent prediction method according to an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of traffic information;

FIG. 4 is an illustration of one lane direction;

FIG. 5 is a data processing schematic of the attention head;

FIG. 6 is a flowchart illustrating a model training method according to an embodiment of the present disclosure;

FIG. 7 is an overall framework of a neural network provided by an embodiment of the present application;

fig. 8 is a schematic structural diagram of a vehicle intention prediction apparatus according to an embodiment of the present application;

FIG. 9 is a schematic structural diagram of a model training apparatus according to an embodiment of the present disclosure;

fig. 10 is a schematic structural diagram of an execution device according to an embodiment of the present application;

FIG. 11 is a schematic structural diagram of a training apparatus provided in an embodiment of the present application;

fig. 12 is a schematic structural diagram of a chip according to an embodiment of the present disclosure.

Detailed Description

The embodiments of the present invention will be described below with reference to the drawings. The terminology used in the description of the embodiments of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

Embodiments of the present application are described below with reference to the accompanying drawings. As can be known to those skilled in the art, with the development of technology and the emergence of new scenes, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

The terms "first," "second," and the like in the description and claims of this application and in the foregoing drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances and are merely descriptive of the manner in which objects of the same nature are distinguished in the embodiments of the application. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.

The general workflow of the artificial intelligence system will be described first, please refer to fig. 1a, fig. 1a shows a schematic structural diagram of an artificial intelligence body framework, which is explained below from two dimensions of "intelligent information chain" (horizontal axis) and "IT value chain" (vertical axis). Where "intelligent information chain" reflects a list of processes processed from the acquisition of data. For example, the process may be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision making, intelligent execution and output. In this process, the data undergoes a "data-information-knowledge-wisdom" process of consolidation. The "IT value chain" reflects the value of artificial intelligence to the information technology industry from the underlying infrastructure of human intelligence, information (provision and processing technology implementation) to the industrial ecological process of the system.

(1) Infrastructure arrangement

The infrastructure provides computing power support for the artificial intelligent system, realizes communication with the outside world, and realizes support through a foundation platform. Communicating with the outside through a sensor; the computing power is provided by intelligent chips (hardware acceleration chips such as CPU, NPU, GPU, ASIC, FPGA and the like); the basic platform comprises distributed computing framework, network and other related platform guarantees and supports, and can comprise cloud storage and computing, interconnection and intercommunication networks and the like. For example, sensors and external communications acquire data that is provided to smart chips in a distributed computing system provided by the underlying platform for computation.

(2) Data of

Data at the upper level of the infrastructure is used to represent the data source for the field of artificial intelligence. The data relates to graphs, images, voice and texts, and also relates to the data of the Internet of things of traditional equipment, including service data of the existing system and sensing data such as force, displacement, liquid level, temperature, humidity and the like.

(3) Data processing

Data processing typically includes data training, machine learning, deep learning, searching, reasoning, decision making, and the like.

The machine learning and the deep learning can be used for performing symbolized and formalized intelligent information modeling, extraction, preprocessing, training and the like on data.

Inference refers to the process of simulating human intelligent inference mode in a computer or an intelligent system, using formalized information to think and solve problems of a machine according to an inference control strategy, and the typical function is searching and matching.

Decision-making refers to a process of making a decision after reasoning intelligent information, and generally provides functions of classification, sorting, prediction and the like.

(4) General capabilities

After the above-mentioned data processing, further based on the result of the data processing, some general capabilities may be formed, such as algorithms or a general system, e.g. translation, analysis of text, computer vision processing, speech recognition, recognition of images, etc.

(5) Intelligent product and industrial application

The intelligent product and industry application refers to the product and application of an artificial intelligence system in various fields, and is the encapsulation of an artificial intelligence integral solution, the intelligent information decision is commercialized, and the landing application is realized, and the application field mainly comprises: intelligent terminal, intelligent transportation, intelligent medical treatment, autopilot, wisdom city etc..

The method and the device can be applied to the field of automatic driving, and particularly can realize vehicle driving intention prediction and driving track prediction of vehicles in the field of automatic driving.

The following describes the concept of driving intent and trajectory prediction:

the driving intention may refer to a driving strategy to be performed by the vehicle in the future, and specifically, the driving intention of the vehicle may be estimated according to the road condition information and the driving state of the vehicle. The vehicle trajectory prediction means predicting the position of the vehicle at each time point within a certain time in the future.

In the field of automatic driving, the driving intentions of surrounding vehicles are estimated accurately and reliably in real time, and the future driving track of the vehicle is predicted, so that the vehicle can be helped to predict the traffic condition ahead, the traffic situation around the vehicle is established, the importance judgment of other vehicle targets around the vehicle is facilitated, the interactive key targets are screened, the vehicle can plan the path in advance, and the complex scene can be safely passed. It should be understood that, in the embodiment of the present application, the surrounding vehicle may also be referred to as an associated vehicle located around the own vehicle.

In the prior art, driving intentions are defined as directional intentions such as straight driving, left turning, right turning, etc., for example, the driving intentions of a vehicle in an intersection scene may include straight driving, left turning, right turning, etc. However, the definition of the driving intentions is limited in complex scenes, and the directional intentions cannot cover all the driving intentions at some complex intersections or other complex lane scenes. For example, referring to fig. 1b, fig. 1b is a schematic diagram of a road condition, where lanes 1 and 2 are left-turn lanes,

lanes

3 and 4 are straight lanes, and lane 5 is an S-type lane, when a vehicle travels to lane 5, the driving intention of the vehicle cannot be accurately expressed only through the intention of turning left, right, etc., and in order to solve the above problem, the driving intention of the vehicle is defined in the lane direction in the present application.

For example, in some complex road conditions, there is a lane whose lane direction extends leftward at an arc of 30 degrees and then rightward at an arc of 50 degrees, and if it is predicted that the vehicle will travel on the lane later, the lane direction can be used as the driving intention of the vehicle, and the driving intention defined with the lane direction as the smallest scale is more accurate and wider than the driving intention of turning left and then turning right.

The embodiment of the application provides a vehicle intention prediction method which can be applied to a prediction system of automatic driving, and the prediction system can predict the driving intention and the predicted track of a vehicle based on information such as road condition information and the historical driving route of the vehicle.

In the embodiment of the present application, the prediction system may include a hardware circuit (e.g., an Application Specific Integrated Circuit (ASIC), a field-programmable gate array (FPGA), a general-purpose processor, a Digital Signal Processor (DSP), a microprocessor or a microcontroller, etc.), or a combination of these hardware circuits, for example, the prediction system may be a hardware system having a function of executing instructions, such as a CPU, a DSP, etc., or a hardware system having no function of executing instructions, such as an ASIC, an FPGA, etc., or a combination of the above hardware systems having no function of executing instructions and a hardware system having a function of executing instructions.

Specifically, the prediction system may be a hardware system having a function of executing instructions, the vehicle intention prediction method provided in the embodiment of the present application may be a software code stored in a memory, and the prediction system may acquire the software code from the memory and execute the acquired software code to implement the vehicle intention prediction method provided in the embodiment of the present application.

It should be understood that the prediction system may be a combination of a hardware system without a function of executing instructions and a hardware system with a function of executing instructions, and some steps of the vehicle intention prediction method provided by the embodiment of the present application may also be implemented by a hardware system without a function of executing instructions in the prediction system, which is not limited herein.

In the embodiment of the application, the prediction system may be deployed on a vehicle or a server on a cloud side, and then, taking the case that the prediction system is deployed on the vehicle as an example, a process of realizing the prediction of the driving intention and the predicted trajectory of the vehicle by using the prediction system is described by combining a software module and a hardware module on the vehicle.

The vehicle in the embodiment of the present application, for example, the target vehicle, the related vehicle around the target vehicle, and the like in the embodiment of the present application may refer to an internal combustion engine vehicle having an engine as a power source, a hybrid vehicle having an engine and an electric motor as power sources, an electric automobile having an electric motor as a power source, and the like.

In the embodiment of the present application, the vehicle may include an automatic driving apparatus 100 having an automatic driving function.

Referring to fig. 1c, fig. 1c is a functional block diagram of an automatic driving apparatus 100 having an automatic driving function according to an embodiment of the present application. In one embodiment, the autopilot device 100 may include various subsystems such as a travel system 102, a sensor system 104, a control system 106, one or more peripherals 108, as well as a power source 110, a computer system 112, and a user interface 116. Alternatively, the autopilot device 100 may include more or fewer subsystems and each subsystem may include multiple elements. In addition, each of the subsystems and components of the autopilot device 100 may be interconnected by wires or wirelessly.

The travel system 102 may include components that provide powered motion to the autopilot device 100. In one embodiment, the travel system 102 may include an engine 118, an energy source 119, a transmission 120, and wheels/tires 121. The engine 118 may be an internal combustion engine, an electric motor, an air compression engine, or other types of engine combinations, such as a hybrid engine of a gasoline engine and an electric motor, or a hybrid engine of an internal combustion engine and an air compression engine. The engine 118 converts the energy source 119 into mechanical energy.

Examples of energy sources 119 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electrical power. The energy source 119 may also provide energy to other systems of the autopilot device 100.

The transmission 120 may transmit mechanical power from the engine 118 to the wheels 121. The transmission 120 may include a gearbox, a differential, and a drive shaft. In one embodiment, the transmission 120 may also include other devices, such as a clutch. Wherein the drive shaft may comprise one or more shafts that may be coupled to one or more wheels 121.

The sensor system 104 may include a number of sensors that sense information about the environment surrounding the autopilot device 100. For example, the sensor system 104 may include a positioning system 122 (which may be a Global Positioning System (GPS) system, a Beidou system, or other positioning system), an Inertial Measurement Unit (IMU) 124, a radar 126, a laser rangefinder 128, and a camera 130. The sensor system 104 may also include sensors that are monitored for internal systems of the autopilot device 100 (e.g., an in-vehicle air quality monitor, a fuel gauge, an oil temperature gauge, etc.). Sensor data from one or more of these sensors may be used to detect the object and its corresponding characteristics (position, shape, orientation, velocity, etc.). Such detection and identification is a key function of the safe operation of the autonomous driving apparatus 100.

The positioning system 122 may be used to estimate the geographic location of the autonomous device 100. The IMU 124 is used to sense position and orientation changes of the autopilot device 100 based on inertial acceleration. In one embodiment, IMU 124 may be a combination of an accelerometer and a gyroscope.

Radar 126 may utilize radio signals to sense objects within the surrounding environment of autopilot device 100. In some embodiments, in addition to sensing objects, radar 126 may also be used to sense the speed and/or heading of an object.

The radar 126 may include an electromagnetic wave transmitting portion and an electromagnetic wave receiving portion. The radar 126 may be implemented as a pulse radar (pulse radar) system or a continuous wave radar (continuous wave radar) system in terms of an electric wave transmission principle. The radar 126 may be implemented in a continuous wave radar mode as a Frequency Modulated Continuous Wave (FMCW) mode or a frequency shift monitoring (FSK) mode according to a signal waveform.

The radar 126 can detect an object based on a time of flight (TOF) method or a phase-shift (phase-shift) method using an electromagnetic wave as a medium, and detect the position, distance from, and relative speed of the detected object. The radar 126 may be disposed at an appropriate position outside the vehicle in order to detect an object located in front of, behind, or to the side of the vehicle. The laser radar 126 may detect an object based on a TOF system or a phase shift system using laser light as a medium, and detect a position of the detected object, a distance to the detected object, and a relative speed.

Alternatively, in order to detect an object located in front of, behind, or to the side of the vehicle, the laser radar 126 may be disposed at an appropriate position outside the vehicle.

The laser rangefinder 128 may utilize a laser to sense objects in the environment in which the autopilot device 100 is located. In some embodiments, the laser rangefinder 128 may include one or more laser sources, laser scanners, and one or more detectors, among other system components.

The camera 130 may be used to capture multiple images of the environment surrounding the autopilot device 100. The camera 130 may be a still camera or a video camera.

Alternatively, the camera 130 may be located at a suitable position outside the vehicle in order to acquire images of the outside of the vehicle. For example, the camera 130 may be disposed in the vehicle interior near the front windshield in order to capture an image in front of the vehicle. Alternatively, the camera 130 may be disposed around the front bumper or the radiator grille. For example, the camera 130 may be disposed close to a rear window in the vehicle interior in order to capture an image behind the vehicle. Alternatively, the camera 130 may be disposed around a rear bumper, trunk, or tailgate. For example, the camera 130 may be disposed in the vehicle interior in close proximity to at least one of the side windows in order to capture an image of the side of the vehicle. Alternatively, the camera 130 may be disposed on a side mirror, fender, or door periphery.

In the embodiment of the present application, the road condition information of the target vehicle, the historical driving route of the associated vehicles located around the target vehicle, and the like may be obtained based on one or more sensors in the sensor system 104.

Control system 106 is provided to control the operation of autopilot device 100 and its components. The control system 106 may include various elements including a steering system 132, a throttle 134, a braking unit 136, a sensor fusion algorithm 138, a computer vision system 140, a route control system 142, and an obstacle avoidance system 144.

The steering system 132 is operable to adjust the heading of the autopilot device 100. For example, in one embodiment, a steering wheel system.

The throttle 134 is used to control the speed of operation of the engine 118 and thus the speed of the autopilot device 100.

The brake unit 136 is used to control the deceleration of the autopilot device 100. The brake unit 136 may use friction to slow the wheel 121. In other embodiments, the brake unit 136 may convert kinetic energy of the wheel 121 into electric current. The brake unit 136 may take other forms to slow the rotational speed of the wheels 121 to control the speed of the autopilot device 100.

The computer vision system 140 may be operable to process and analyze images captured by the camera 130 to identify objects and/or features in the environment surrounding the autonomous device 100. The objects and/or features may include traffic signals, road boundaries, and obstacles. The computer vision system 140 may use object recognition algorithms, motion from motion (SFM) algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system 140 may be used to map an environment, track objects, estimate the speed of objects, and so forth.

The route control system 142 is used to determine a travel route for the autonomous device 100. In some embodiments, the route control system 142 may combine data from the sensors 138, the positioning system 122, and one or more predetermined maps to determine a travel route for the autopilot device 100.

Obstacle avoidance system 144 is used to identify, assess, and avoid or otherwise negotiate potential obstacles in the environment of autonomous driving device 100.

Of course, in one example, the control system 106 may additionally or alternatively include components other than those shown and described. Or a portion of the components shown above may be reduced.

The autopilot device 100 interacts with external sensors, other autopilot devices, other computer systems, or users through peripherals 108. The peripheral devices 108 may include a wireless communication system 146, an in-vehicle computer 148, a microphone 150, and/or speakers 152.

In some embodiments, the peripheral device 108 provides a means for a user of the autopilot device 100 to interact with the user interface 116. For example, the onboard computer 148 may provide information to a user of the autopilot device 100. The user interface 116 may also operate the in-vehicle computer 148 to receive user input. The in-vehicle computer 148 may be operated via a touch screen. In other cases, the peripheral devices 108 may provide a means for the autopilot device 100 to communicate with other devices located within the vehicle. For example, the microphone 150 may receive audio (e.g., voice commands or other audio input) from a user of the autopilot device 100. Similarly, the speaker 152 may output audio to a user of the autopilot device 100.

The wireless communication system 146 may communicate wirelessly with one or more devices, either directly or via a communication network. For example, the wireless communication system 146 may use 3G cellular communication such as Code Division Multiple Access (CDMA), EVD0, global system for mobile communications (GSM)/General Packet Radio Service (GPRS), or 4G cellular communication such as Long Term Evolution (LTE), or 5G cellular communication. The wireless communication system 146 may communicate with a Wireless Local Area Network (WLAN) using WiFi. In some embodiments, the wireless communication system 146 may utilize an infrared link, bluetooth, or ZigBee to communicate directly with the device. Other wireless protocols, such as various autonomous device communication systems, for example, wireless communication system 146 may include one or more Dedicated Short Range Communication (DSRC) devices that may include public and/or private data communication between autonomous devices and/or roadside stations.

In one implementation, the information such as the road condition information and the historical driving track in the embodiment of the present application may be received by the vehicle from other vehicles or a cloud-side server through the wireless communication system 146.

When the prediction system is located in a server on the cloud side, the vehicle may receive driving intention information or the like for the target vehicle, which is transmitted by the server, through the wireless communication system 146.

The power supply 110 may provide power to various components of the autopilot device 100. In one embodiment, power source 110 may be a rechargeable lithium ion or lead acid battery. One or more battery packs of such batteries may be configured as a power source to provide power to the various components of the autopilot device 100. In some embodiments, the power source 110 and the energy source 119 may be implemented together, such as in some all-electric vehicles.

Some or all of the functions of the autopilot device 100 are controlled by the computer system 112. The computer system 112 may include at least one processor 113, the processor 113 executing instructions 115 stored in a non-transitory computer readable medium, such as the memory 114. The computer system 112 may also be a plurality of computing devices that control individual components or subsystems of the autopilot device 100 in a distributed manner.

The processor 113 may be any conventional processor, such as a commercially available Central Processing Unit (CPU). Alternatively, the processor may be a dedicated device such as an Application Specific Integrated Circuit (ASIC) or other hardware-based processor. Although FIG. 1c functionally illustrates processors, memories, and other elements of the computer 110 in the same block, those skilled in the art will appreciate that the processors, computers, or memories may actually comprise multiple processors, computers, or memories that may or may not be stored within the same physical housing. For example, the memory may be a hard disk drive or other storage medium located in a different housing than the computer 110. Thus, reference to a processor or computer will be understood to include reference to a collection of processors or computers or memories that may or may not operate in parallel. Rather than using a single processor to perform the steps described herein, some components, such as the steering component and the retarding component, may each have their own processor that performs only computations related to the component-specific functions.

In various aspects described herein, the processor may be located remotely from the autonomous device and in wireless communication with the autonomous device. In other aspects, some of the processes described herein are executed on a processor disposed within the autopilot device while others are executed by a remote processor, including taking the steps necessary to execute a single maneuver.

In some embodiments, the memory 114 may include instructions 115 (e.g., program logic), and the instructions 115 may be executable by the processor 113 to perform various functions of the autopilot device 100, including those described above. The memory 114 may also contain additional instructions, including instructions to send data to, receive data from, interact with, and/or control one or more of the travel system 102, the sensor system 104, the control system 106, and the peripheral devices 108.

In addition to instructions 115, memory 114 may also store data such as road maps, route information, the location, direction, speed of the autonomous device, and other such autonomous device data, as well as other information. Such information may be used by the autonomous device 100 and the computer system 112 during operation of the autonomous device 100 in autonomous, semi-autonomous, and/or manual modes.

The vehicle intention prediction method provided by the embodiment of the present application may be a software code stored in the memory 114, and the processor 113 may acquire the software code from the memory and execute the acquired software code to implement the vehicle intention prediction method provided by the embodiment of the present application. After obtaining the driving intent of the target vehicle, the driving intent may be communicated to the control system 106, and the control system 106 may make the determination of the own vehicle driving strategy based on the driving intent.

A user interface 116 for providing information to or receiving information from a user of the autopilot device 100. Optionally, the user interface 116 may include one or more input/output devices within the collection of peripheral devices 108, such as a wireless communication system 146, an in-vehicle computer 148, a microphone 150, and a speaker 152.

The computer system 112 may control the functions of the autopilot device 100 based on inputs received from various subsystems (e.g., the travel system 102, the sensor system 104, and the control system 106) and from the user interface 116. For example, the computer system 112 may utilize input from the control system 106 to control the steering unit 132 to avoid obstacles detected by the sensor system 104 and the obstacle avoidance system 144. In some embodiments, the computer system 112 is operable to provide control over many aspects of the autopilot device 100 and its subsystems.

Alternatively, one or more of these components may be mounted separately from or associated with the autopilot device 100. For example, the memory 114 may be partially or completely separate from the autopilot device 100. The above components may be communicatively coupled together in a wired and/or wireless manner.

Optionally, the above components are only an example, in an actual application, components in the above modules may be added or deleted according to an actual need, and fig. 1c should not be construed as limiting the embodiment of the present application.

In the above, the application architecture of the embodiment of the present application is introduced, and the intention prediction method provided in the embodiment of the present application is described in detail below.

Since the embodiments of the present application relate to a neural network, for the sake of understanding, the following description will be made about related terms related to the embodiments of the present application.

(1) Neural network

The neural network may be composed of neural units, which may be referred to as x _s (i.e., input data) and intercept =1, the operation unitThe output of the element may be:

wherein s =1, 2, \8230, n is natural number greater than 1, and W is _s Is x _s B is the bias of the neural unit. f is an activation function (activation functions) of the neural unit for introducing a nonlinear characteristic into the neural network to convert an input signal in the neural unit into an output signal. The output signal of the activation function may be used as an input to the next convolutional layer, and the activation function may be a sigmoid function. A neural network is a network formed by a plurality of the above-mentioned single neural units being joined together, i.e. the output of one neural unit may be the input of another neural unit. The input of each neural unit can be connected with the local receiving domain of the previous layer to extract the characteristics of the local receiving domain, and the local receiving domain can be a region composed of a plurality of neural units.

(2) A Convolutional Neural Network (CNN) is a deep neural network with a convolutional structure. The convolutional neural network comprises a feature extractor consisting of convolutional layers and sub-sampling layers, which can be regarded as a filter. The convolutional layer is a neuron layer for performing convolutional processing on an input signal in a convolutional neural network. In convolutional layers of convolutional neural networks, one neuron may be connected to only a portion of the neighbor neurons. In a convolutional layer, there are usually several characteristic planes, and each characteristic plane may be composed of several neural units arranged in a rectangular shape. The neural units of the same feature plane share weights, where the shared weights are convolution kernels. Sharing weights may be understood as the way features are extracted is location independent. The convolution kernel may be formalized as a matrix of random size, and may be learned to obtain reasonable weights during the training of the convolutional neural network. In addition, sharing weights brings the direct benefit of reducing connections between layers of the convolutional neural network, while reducing the risk of overfitting.

(3) Deep neural network

Deep Neural Networks (DNNs), also known as multi-layer Neural networks, can be understood as Neural networks having many layers of hidden layers, where "many" has no particular metric. From the DNN, which is divided by the positions of different layers, the neural networks inside the DNN can be divided into three categories: input layer, hidden layer, output layer. Generally, the first layer is an input layer, the last layer is an output layer, and the middle layers are hidden layers. The layers are all connected, that is, any neuron of the ith layer is necessarily connected with any neuron of the (i + 1) th layer. Although DNN appears complex, it is not as complex as the work of each layer, in short the following linear relational expression:

wherein,

is the input vector of the input vector,

is the output vector of the digital video signal,

is the offset vector, W is the weight matrix (also called coefficient), and α () is the activation function. Each layer is only for the input vector

Obtaining the output vector through such simple operation

Due to the large number of DNN layers, the coefficient W and the offset vector

The number of the same is large. The definition of these parameters in DNN is as follows: taking coefficient W as an example: assume that in a three-layer DNN, the line from the 4 th neuron of the second layer to the 2 nd neuron of the third layerThe coefficient of sex is defined as

The superscript 3 represents the number of layers in which the coefficient W is located, while the subscripts correspond to the third layer index 2 of the output and the second layer index 4 of the input.

The summary is that: the coefficients of the kth neuron of the L-1 th layer to the jth neuron of the L-1 th layer are defined as

Note that the input layer is without the W parameter. In deep neural networks, more hidden layers make the network more able to depict complex situations in the real world. Theoretically, the more parameters the higher the model complexity, the larger the "capacity", which means that it can accomplish more complex learning tasks. The final objective of the process of training the deep neural network, i.e., learning the weight matrix, is to obtain the weight matrix (the weight matrix formed by the vectors W of many layers) of all layers of the deep neural network that is trained.

(4) Loss function

In the process of training the deep neural network, because the output of the deep neural network is expected to be as close to the value really expected to be predicted as possible, the weight vector of each layer of the neural network can be updated according to the difference between the predicted value of the current network and the really expected target value (of course, an initialization process is usually carried out before the first updating, namely parameters are preset for each layer in the deep neural network), for example, if the predicted value of the network is high, the weight vector is adjusted to be slightly lower, and the adjustment is carried out continuously until the deep neural network can predict the really expected target value or the value which is very close to the really expected target value. Therefore, it is necessary to define in advance how to compare the difference between the predicted value and the target value, which are loss functions (loss functions) or objective functions (objective functions), which are important equations for measuring the difference between the predicted value and the target value. Taking the loss function as an example, if the higher the output value (loss) of the loss function indicates the larger the difference, the training of the deep neural network becomes the process of reducing the loss as much as possible.

(5) Back propagation algorithm

The convolutional neural network can adopt a Back Propagation (BP) algorithm to correct the size of parameters in the initial super-resolution model in the training process, so that the reconstruction error loss of the super-resolution model is smaller and smaller. Specifically, error loss occurs when an input signal is transmitted in a forward direction until the input signal is output, and parameters in an initial super-resolution model are updated by reversely propagating error loss information, so that the error loss is converged. The back propagation algorithm is a back propagation motion with error loss as a dominant factor, aiming at obtaining the optimal parameters of the super-resolution model, such as a weight matrix.

(6) Attention mechanism (attention mechanism)

The attention mechanism simulates the internal process of biological observation behavior, namely a mechanism for aligning internal experience and external feeling so as to increase the observation fineness of partial areas, and can rapidly screen out high-value information from a large amount of information by using limited attention resources. Attention mechanism can quickly extract important features of sparse data, and thus is widely used for natural language processing tasks, especially machine translation. The self-attentive mechanism (self-attentive mechanism) is an improvement of the attentive mechanism, which reduces the dependence on external information and is better at capturing internal correlation of data or features. The essential idea of the attention mechanism can be rewritten as the following formula:

the equation meaning means that a constituent element in the Source is imagined to be composed of a series of data pairs, at this time, a certain element Query in the Target is given, a weight coefficient of Value corresponding to each Key is obtained by calculating similarity or correlation between the Query and each Key, and then the Value is subjected to weighted summation, so that a final Attentition numerical Value is obtained. So in essence the Attention mechanism is to perform a weighted summation on the Value values of the elements in Source, and Query and Key are used to calculate the weight coefficients of the corresponding Value. Conceptually, attention can be understood as selectively screening out a small amount of important information from a large amount of information and focusing on the important information, ignoring most of the less important information. The focusing process is embodied in the calculation of the weight coefficient, the greater the weight is, the more the weight is focused on the Value corresponding to the weight coefficient, namely, the weight represents the importance of the information, and the Value is the corresponding information. The self-Attention mechanism may be understood as internal Attention (entry), the entry mechanism occurs between all elements in the Source and the Target element Query, or the entry mechanism occurs between the Source internal elements, or the Attention calculation mechanism in the special case of Target = Source, and the specific calculation process is the same, only the calculation object is changed.

(7) Center line of lane

The lane center line is located in the middle of the two boundaries of the lane, and may be virtual or actually present.

Referring to fig. 2, fig. 2 is a flowchart illustrating a vehicle intention prediction method according to an embodiment of the present application, and as shown in fig. 2, the vehicle intention prediction method according to the embodiment of the present application includes:

201. the position of the target vehicle is acquired.

In the embodiment of the application, the target vehicle may be a vehicle which needs to be subjected to driving intention prediction.

In one implementation, in order to accurately predict whether other surrounding vehicles influence the driving safety of the own vehicle, influence the driving decision of the own vehicle, and control the driving strategy of the own vehicle based on the surrounding vehicles during the driving process of the own vehicle, the driving intention of at least one related vehicle positioned around the own vehicle needs to be determined. The target vehicle in the embodiment of the present application is any one of at least one associated vehicle located around the own vehicle.

It should be understood that the "associated vehicle" may be understood as a vehicle within a certain preset range of distance from the own vehicle, that is, determining which vehicles have an association relationship with the own vehicle based on the distance recently, and then regarding those vehicles having an association relationship as the associated vehicle of the own vehicle; in addition, the "associated vehicle" may also be understood as a vehicle that may affect the driving state decision of the vehicle in the future, that is, determine which vehicles have an association relationship with the vehicle based on whether the vehicle will affect the driving strategy of the vehicle in the future, and then take the vehicle having the association relationship as the associated vehicle of the vehicle.

In one implementation, the processor of the host vehicle may control the relevant sensors on the host vehicle to acquire the driving state information of the surrounding vehicles based on the software code in the memory 114 related to step 201, and determine which vehicles are related vehicles based on the acquired driving state information, that is, determine which vehicles need to perform the intention prediction.

Alternatively, the process of determining the target vehicle may be determined by another vehicle or a server on the cloud side, which is not limited herein.

In order to clearly predict the future driving intention of the target vehicle, driving state information, road condition information and the like of the target vehicle need to be acquired, and the information can be used as a basis for predicting the driving intention of the target vehicle. The driving state information may include a position of the target vehicle.

The position of the target vehicle may be an absolute position of the target vehicle in a map, or a relative position with the host vehicle, and the absolute position of the target vehicle may be determined based on the absolute position of the host vehicle and the relative position between the target vehicle and the host vehicle.

Taking the associated vehicle as the target vehicle as an example, in the embodiment of the present application, driving state information of the target vehicle may be obtained, where the driving state information may include a position of the target vehicle, and specifically, the position of the target vehicle may be sensed by a sensor carried by the vehicle, or the position of the target vehicle may be obtained through interaction with other vehicles and a server on a cloud side.

Taking the associated vehicle as the target vehicle as an example, in one implementation, the position of the target vehicle may be obtained in real time, or obtained once at intervals.

202. And determining a plurality of candidate lane groups which can be driven by the target vehicle within a preset distance according to the position of the target vehicle, wherein the lane direction difference of different candidate lane groups within the preset distance is greater than a target threshold value.

In the embodiment of the application, in order to define the driving intention of the target vehicle more accurately, the lane direction of the lane which the target vehicle is most likely to travel in the future may be used as the driving intention of the target vehicle, and therefore, it is first required to determine which lanes are likely to travel by the target vehicle within the preset distance in the future, and the lane which the target vehicle is likely to travel within the preset distance may be used as the candidate lane.

It should be understood that the preset distance may be preset, depending on how far the driving intention of the target vehicle is to be predicted; alternatively, the preset distance may be determined based on the traveling speed of the target vehicle and the predicted time required for the traveling intention.

In one implementation, the predetermined distance is a fixed number less than 200 meters; or the preset distance is obtained by calculating the traveling speed of the target vehicle and the predicted time of the intention, wherein the preset distance is positively correlated with the traveling speed of the target vehicle and the predicted time required by the traveling intention, and the predicted time required by the intention is less than or equal to 7 seconds.

It should be understood that the lane where the target vehicle can travel is understood as a lane where the target vehicle can travel within a preset distance without violating the constraints of the intersection regulations and the like, for example, referring to fig. 3, fig. 3 is an illustration of road condition information, where the target vehicle is located between lane 3 and lane 4, where lane 3, lane 4 and lane 5 are lanes where the target vehicle can travel within the preset distance, lane 1 and lane 2 are lanes opposite to the current traveling direction of the target vehicle, and lane 2 and lane 3 are solid lines because the target vehicle cannot change to lane 1 or lane 2 or turn around to lane 1 or lane 2 within the preset distance, and lane 1 and lane 2 are not lanes where the target vehicle can travel within the preset distance.

Next, it is described how to determine a lane in which the target vehicle can travel within a preset distance based on the position of the target vehicle:

after the position of the target vehicle is acquired, lane information around the target vehicle may be determined from map information, where the map information may contain information of positions of lane lines or virtual lane lines, travelable areas, crosswalks, lane boundaries, and the like, and stored and queried in a vector form, and the lane information may include widths of respective lanes, lane line directions, and the like.

After the position of the target vehicle is obtained, the lane on which the target vehicle can travel next can be determined from the map based on the position of the target vehicle, in one possible implementation, which lanes are associated with the target vehicle can be determined based on the similarity of the position and the traveling direction of the target vehicle to the respective lanes in a history period of time, the associated lanes can be regarded as the lanes on which the target vehicle can travel next, and specifically, the positions of the first moment and the last moment of the history of the target vehicle, that is, the positions of the target vehicle at the first moment and the last moment can be selected

And

and calculating the driving directions of the target vehicle at the first moment and the last moment, and screening the first moment t-t according to the map information _h All lanes with distance and angle difference from target vehicle within certain threshold range

The lane

All lanes L representing the lanes with which the target vehicle is associated at the first instant in time and the last historical instant in time t, with the distance and angle difference to the target vehicle being within a certain threshold range _t Lane L _t Indicating that there is an associated lane with the target vehicle at the last instant, taking the intersection of the two, which may be indicated at the first instantAnd in the time period from the moment to the historical moment, a related lane exists with the target vehicle, and the related lane can be used as a candidate lane which can be driven within a preset distance in the future of the target vehicle.

It should be appreciated that since the lane information stored in the map is a continuous sequence, where each sequence may indicate a small segment of a lane, the continuous sequence may indicate a complete lane, and thus the candidate lanes determined in the embodiments of the present application may be stored in a lane sequence.

Since the lane direction of the lane most likely to be traveled by the target vehicle in the future is required to be used as the driving intention of the target vehicle in the embodiment of the present application, and usually a part of the lanes in the candidate lanes have the same or similar lane directions, a plurality of candidate lanes may be clustered based on the direction of the lane line to obtain a plurality of candidate lane groups, where the lane direction difference of different candidate lane groups within the preset distance is greater than a target threshold, each candidate lane group includes at least one lane, and the lane direction difference between the lanes included in each candidate lane group is less than the target threshold.

It should be understood that the lane direction is a direction of a lane line within the preset distance from the position of the target vehicle, that is, the lane direction indicates an extending direction of the lane line within the preset distance of the lane, not only a direction of the lane line at a certain point. Specifically, the lane direction may refer to a direction indicated by a trajectory in which a lane line is present, for example, the lane direction may refer to a trajectory direction in which the vehicle travels along the trajectory in which the lane line is present.

Referring to fig. 4, fig. 4 is a schematic view of a lane direction in which the lane direction of the lane 5 is straight, S-shaped turn, and straight. The lane direction in the embodiment of the present application may be understood as a lane line direction or a lane center line direction.

The lane direction difference in the embodiment of the present application may be a difference between extending directions of lane lines or lane center lines, for example, if the lane direction of the lane a is extending leftward, the lane direction of the lane B is extending rightward, the lane direction of the lane C is straight, and the lane direction of the lane D is extending leftward, then the lane direction difference between the lane a and the lane D is smaller, and the lane direction difference between the lane a, the lane B, and the lane C is larger.

Referring to fig. 5, fig. 5 is a schematic diagram of a lane candidate group in the embodiment of the present application, where lane 1 and lane 2 are divided into the same lane candidate group due to the similar lane directions, lane 3 and lane 4 are divided into the same lane candidate group due to the similar lane directions, and lane 5 is divided into an independent lane candidate group due to the larger lane direction difference from other lanes.

203. And acquiring the road condition information of each candidate lane group in the plurality of candidate lane groups.

In the embodiment of the present application, after determining the plurality of candidate lane groups, it is necessary to determine a target lane group, on which the target vehicle is most likely to travel, as the target lane group from the plurality of candidate lane groups, and to take a lane direction of the target lane group as a driving intention of the target vehicle. In order to determine the most likely target lane group for the target vehicle to travel from the multiple candidate lane groups, it is necessary to obtain the road condition information of each candidate lane group according to the road condition information of each candidate lane group.

It should be understood that the traffic information of each candidate lane group may be understood as the traffic information of the respective lanes included in each candidate lane group.

In this embodiment, the traffic information may include at least one of the following information: lane center line information, travelable area information, obstacle information, speed limit area information.

In addition, besides acquiring the road condition information of each candidate lane group in the plurality of candidate lane groups, the historical driving route of the target vehicle, or other driving state information may also be acquired, and the driving state information may include, but is not limited to, speed, acceleration, steering angle relative to the lane center line, and the like, and is not limited herein.

In a possible implementation, the information obtained for making the driving intention prediction may be represented as a rasterized image, where the rasterized image includes a plurality of image channels, and each image channel is used for representing at least one type of information.

Specifically, the road condition information may be rasterized and encoded to obtain a rasterized image of the map element. The historical driving route of the target vehicle is also encoded with the same resolution and image size, and the obtained rasterized image is combined with the rasterized image of the map element to be used as input data for driving the intention of the target vehicle.

Next, how to perform the rasterization coding on the road condition information and the historical driving route of the target vehicle is described as follows:

in one possible implementation, the size of the rasterized image may be first determined, e.g., 720 × 720 pixels, and the scaling of the image to the actual location, e.g., 4 pixels per meter, then the image may represent an actual range of 180m × 180m. The method comprises the steps of establishing a coordinate system by taking the position of the last observation time of a target vehicle as a coordinate origin and the orientation of a target as a coordinate axis direction, converting all map elements and obstacles to the coordinate system, then mapping the map elements and the obstacles on a grid map, wherein the map elements and the obstacles in an actual physical space are before mapping, and pixels on the grid map are after mapping. The information coding can adopt independent channel coding, namely each kind of information is coded into different image channels, and partial information can also be coded into the same image channel. The total number of the image channels is variable, and the image channels can be added and deleted according to actual needs.

The method can be divided into three categories:

1. intention path related channel

Regarding the candidate lane group needing to be encoded, part of the traffic information of the candidate lane group is taken as the relevant channel of the intended path, such as but not limited to the lane center line channel, the drivable area channel, and the like of the candidate lane group. Specifically, taking the encoding process of the lane center line information as an example, the lane center line information of the candidate lane group may be drawn on the image channel after coordinate transformation and scaling, and the travelable region channel may draw the travelable region corresponding to the intended path on the image channel. The rendering image channel may be a binary map, i.e. the fill area value is 1, otherwise it is 0.

2. Unintended Path related Lane

And aiming at the candidate lane group needing coding, taking the road condition information which is not the candidate lane group as an unintended path related channel, and taking the unintended path related lane, wherein the coding comprises but is not limited to a lane central line channel and a travelable region channel.

3. Common information channel

Part of the traffic information of the candidate lane group may include, but is not limited to, a historical driving route channel, a historical information channel of surrounding obstacles, a speed limit zone channel, and the like. Taking the historical driving route channel as an example, the historical driving route channel can be drawn on the image channel by a curve formed by connecting the historical positions of the target vehicles, the drawn numerical value is filled into a linear numerical value, and the numerical value is larger as the distance from the current moment is closer so as to distinguish the historical driving direction of the target vehicles. The surrounding dynamic obstacle history information channel is similar to the above one and encodes all surrounding objects to the same image channel. The speed-limiting area channel can be drawn on the corresponding image channel in a binary image mode.

204. And determining a target candidate lane group from the candidate lane groups according to the road condition information, wherein the lane direction of the target candidate lane group is used as the driving intention of the target vehicle.

In the embodiment of the application, after the road condition information of each candidate lane group is obtained, a target candidate lane group can be determined from the candidate lane groups according to the road condition information, and the lane direction of the target candidate lane group is used as the driving intention of the target vehicle.

It should be understood that the target candidate lane group may also be determined from the plurality of candidate lane groups based on road condition information of the respective candidate lane groups and a historical driving route of the target vehicle.

In one implementation, the information for making the driving intention prediction may be represented as a rasterized image, and then the target candidate lane group may be determined from the plurality of candidate lane groups based on the rasterized image.

Next, how to determine a target candidate lane group from the plurality of candidate lane groups according to the traffic information will be described.

In one implementation, a neural network model may be obtained that is pre-trained and has the ability to determine the probability of selection for each candidate lane group based on road condition information.

In one implementation, the road condition information may be subjected to feature extraction through a neural network model to obtain a feature vector for expressing the road condition information of each candidate lane group, wherein the feature vector may be expanded into a one-dimensional vector, such as a one-dimensional vector with a length of 128. The feature vectors corresponding to each candidate lane group can be processed through a neural network to obtain the selection probability of each candidate lane group, the candidate lane group with the highest selection probability is selected as a target candidate lane group, and then the lane direction of the target candidate lane group is used as the driving intention of the target vehicle.

Illustratively, the road condition information may be processed by CNN to obtain a feature vector expressing the road condition information of each candidate lane group, the feature vector may be expanded into a one-dimensional vector, for example, a one-dimensional vector with a length of 128, then the one-dimensional vector is input into two fully-connected layers, and finally a vector with a length of 1 is output, and then the probability of selection of each candidate lane group is obtained by a sigmoid function. The CNN network structure can adopt, but is not limited to, such as ResNet 16.

How to train the neural network model having the capability of determining the selection probability of each candidate lane group based on the road condition information will be described in the following embodiments, and will not be described herein.

In the embodiment of the application, in order to predict the intention of the target vehicle more accurately, the target predicted track of the target vehicle can be calculated. Specifically, the traffic information of the target candidate lane group may be input into a neural network model having a capability of outputting a target predicted trajectory of the target vehicle based on the traffic information. And further determining a target predicted track of the target vehicle through a neural network model according to the road condition information of the target candidate lane group.

In one implementation, the feature vector corresponding to the traffic information of the target candidate lane group may be input into a neural network model, and the target predicted trajectory of the target vehicle may be determined through the neural network model according to the feature vector corresponding to the traffic information of the target candidate lane group.

It should be understood that, in addition to the road condition information, the driving influence degree of the related vehicles around the target vehicle on the target vehicle may also be used as an input of the neural network, specifically, the historical driving route of the target vehicle and at least one related vehicle around the target vehicle may be obtained, and the driving influence degree of the at least one related vehicle on the target vehicle may be determined by the attention mechanism according to the historical driving route, and then the target predicted trajectory of the target vehicle may be determined by the neural network model according to the road condition information of the target candidate lane group and the influence degree.

The attention mechanism is described next:

the execution subject of the attention mechanism can be a multi-head attention layer, and the multi-head attention layer can acquire N input vectors X _l Wherein each input vector corresponds to a historical driving route, specifically, the historical driving route of the target vehicle and at least one associated vehicle located around the target vehicle may be subjected to feature extraction through, but not limited to, a recurrent neural network RNN or a long-short-term memory (LSTM) network to obtain N input vectors X _l 。

N input vectors X _l The matrix X may be represented as a matrix Y, and each vector may be transformed based on the degree of correlation between vectors by the attention-driven system to obtain N output vectors. The multi-Head attention layer may include a plurality of heads Head, and specifically, the first transformation matrix Q, the second transformation matrix K, and the third transformation matrix V may be respectively used for N input vectors<X ₁ ,X ₂ ,…,X _N >Of each input vector X _i The transformation is performed to obtain a first intermediate vector (q vector), a second intermediate vector (k vector) and a third intermediate vector (v vector) corresponding to each input vector. In operation, an input matrix X composed of N input vectors may be linearly transformed by using the first transformation matrix Q, the second transformation matrix K, and the third transformation matrix V, respectively, to obtain a Q matrix, a K matrix, and a V matrix of the input matrix, and then the matrices may be split, respectively, to obtain a Q vector, a K vector, and a V vector corresponding to each input vector. For an ith input vector Xi of any of the N input vectors, a first intermediate vector (q-vector, qi) corresponding to the ith input vector is based on the ith input vector and each input vector X _j Corresponding respective second intermediate vectors (k vectors, k) _j ) Determines the ith input vector Xi and each input vector X _j The respective degrees of association of (a). Although q may also be directly substituted _i And k is _j The result of the dot multiplication of (a) is determined as the degree of correlation, but more typically, the result of the dot multiplication is first divided by a constant and then subjected to the softmax operation, which will beThe result of the operation being an input vector X _i And X _j The degree of association of (a) is:

thus, vector X may be input at this i-th order _i With respective input vector X _j Each degree of association α of _i,j As weighting factors, for each input vector X _j Corresponding third intermediate vector (v vector, v) _j ) Performing weighted combination to obtain the ith combination vector C corresponding to the ith input vector Xi _i ：

Thus, a vector sequence of N combined vectors corresponding to the N input vectors can be obtained<C ₁ ,C ₂ ,…,C _N >Or matrix C. Based on the sequence of combined vectors, N output vectors may be obtained. In particular, in one embodiment, a vector sequence of N combined vectors may be directly treated as N output vectors, i.e. Y _i ＝C _i . At this time, the output matrix Y is the combined vector matrix C, which can be written as:

the above is a description of the processing procedure of an attention head, in the MHA architecture, the MHA layer maintains m sets of transformation matrices, each set of transformation matrices includes the aforementioned first transformation matrix Q, second transformation matrix K, and third transformation matrix V, so that the above operations can be performed in parallel to obtain m combined vector sequences (i.e., m matrices C), each vector sequence includes N combined vectors obtained based on one set of transformation matrices. Under the condition, the MHA layer splices the obtained m combined vector sequences to obtain a spliced matrix; and then the splicing matrix is transformed by a fourth transformation matrix W to obtain a final output matrix Y. Splitting the output matrix Y corresponds to N output vectors < Y1, Y2, \8230;, YN >. Through the above operation process, the MHA layer performs a transform operation based on the degree of association between the N input vectors to obtain N output vectors.

And combining the N output vectors, and outputting the final interactive characteristics, namely the driving influence degree of the at least one associated vehicle on the target vehicle after passing through a full-connection network.

In the embodiment of the application, the neural network model may include a plurality of prediction branches, each prediction branch has the capability of predicting different prediction track types, and each branch of the plurality of prediction branches may determine a candidate track of the target vehicle and a confidence degree of the candidate track according to the road condition information of the target candidate lane group, where the confidence degree may represent a probability that the corresponding candidate track is a real driving track of the target vehicle.

The candidate trajectory with the highest confidence may then be selected as the target predicted trajectory, which may be the predicted trajectory of the target vehicle. The above-mentioned prediction branches may also be referred to as neural network submodels in subsequent embodiments.

It should be understood that how to train the neural network model so that it has the capability of predicting different predicted trajectory types will be described in the following embodiments and will not be described here.

In addition, when the predicted track is generated, the input road condition information related to the driving intention is input, so that the characteristics adopted by the track prediction have the definite attribute of the driving intention, the input information of the track prediction has better interpretability related to the intention, and the result of the track prediction is more accurate.

The reasoning process from the driving intention prediction is described above, and the training process of the neural network is described next.

Referring to fig. 6, fig. 6 is a flowchart illustrating a method for training a model provided in an embodiment of the present application, and as shown in fig. 6, the method for training a model provided in the embodiment of the present application includes:

601. and acquiring the first neural network model, the position of the target vehicle and a real driving track, wherein the real driving track of the target vehicle is positioned on the target lane.

In the embodiment of the present application, a first neural network model may be obtained, where the first neural network model is a neural network model to be trained, and the neural network model may include a CNN and other feature extraction networks, a full-connection network, and the like.

In the embodiment of the present application, the training sample data may include a real driving trajectory of the target vehicle.

602. And determining a plurality of candidate lane groups which can be driven by the target vehicle within a preset distance according to the position of the target vehicle, wherein the lane direction difference of different candidate lane groups within the preset distance is greater than a target threshold value.

In the embodiment of the application, a plurality of candidate lane groups that the target vehicle can travel within a preset distance may be determined according to the position of the target vehicle, where the lane direction difference of different candidate lane groups within the preset distance is greater than a target threshold.

For the detailed description of step 602, reference may be made to the description of step 202 in the foregoing embodiment, which is not described herein again.

603. And acquiring the road condition information of each candidate lane group in the plurality of candidate lane groups.

The specific description of step 603 may refer to the description of step 203 in the foregoing embodiment, and is not repeated here.

604. And determining a target candidate lane group from the candidate lane groups through the first neural network model according to the road condition information, and training the first neural network model based on the difference between the target candidate lane group and the candidate lane group where the target lane is located to obtain a second neural network model.

In the embodiment of the present application, the candidate lane group may be labeled based on whether the target lane is located in the candidate lane group, where the candidate lane group where the target lane is located may be labeled as a positive sample, the lane direction of the candidate lane group labeled as the positive sample may be regarded as a correct driving intention of the target vehicle, the candidate lane group where the target lane is not located may be labeled as a negative sample, and the lane direction of the candidate lane group labeled as the negative sample may be regarded as a wrong driving intention of the target vehicle.

In one implementation, the distance and orientation difference between the end point of the true travel trajectory of the target vehicle and all the candidate lane groups may be calculated, and if both are smaller than a certain threshold, the candidate lane group is considered as a positive sample, otherwise, the candidate lane group is considered as a negative sample. It should be understood that the number of positive samples of the candidate lane groups may be greater than 1, for example, when the target vehicle is about to enter the intersection, the real driving trajectory of the target vehicle does not reach the bifurcation point of the route, and thus does not show explicit steering information, and it cannot be determined which lane direction of the candidate lane group is the real driving intention of the target vehicle, so that lane directions of a plurality of possible candidate lane groups can be matched, and the labels of the candidate lane groups should be all positive samples.

In this embodiment, the first neural network model may process the road condition information to obtain a selection probability of each candidate lane group, and since the real trajectory of the target vehicle is located in a part of the candidate lane groups of the plurality of candidate lane groups, the real probability corresponding to the part of the candidate lane groups should be 1, and the real probability of the candidate lane group in which the real trajectory of the target vehicle is not located is 0. Accordingly, the first neural network model may be trained based on a difference between the selected probability of each candidate lane group and the true probability of each candidate lane group.

Specifically, a loss function may be constructed based on a difference between the selected probability of each candidate lane group and the true probability of each candidate lane group, and the first neural network model may be trained based on the constructed loss function. Illustratively, the loss function may be, but is not limited to, the following equation:

wherein M is the number of candidate lane groups, C ⁱ Traffic information of the ith candidate lane group, C traffic information of all candidate lane groups, X ^a For the historical driving route of the own vehicle, P (C) ⁱ |X ^a C) is the predicted long-term intention probability obtained by the first neural network,

is a tag of the set of candidate lanes,

is a cross entropy loss function.

In the embodiment of the application, a neural network model with the capability of outputting the target predicted trajectory of the target vehicle based on the road condition information can be trained.

Specifically, a third neural network model may be obtained, a target predicted trajectory of the target vehicle may be determined by the third neural network model according to the road condition information of the candidate lane group in which the target lane is located, and then the third neural network model may be trained based on a difference between the target predicted trajectory and the actual travel trajectory, so as to obtain a fourth neural network model.

It should be understood that the road condition information of the candidate lane group where the target lane is not located may not participate in the training process of the third neural network model.

It should be understood that, in addition to the road condition information, the driving influence degree of the related vehicles around the target vehicle on the target vehicle may also be used as an input of the third neural network, specifically, the historical driving route of the target vehicle and at least one related vehicle around the target vehicle may be obtained, and the driving influence degree of the at least one related vehicle on the target vehicle may be determined according to the historical driving route and through an attention mechanism, and then the target predicted trajectory of the target vehicle may be determined through the third neural network model according to the road condition information of the target candidate lane group and the influence degree.

In one possible implementation, the third neural network model includes a plurality of neural network submodels, and a plurality of predicted trajectories of the target vehicle are determined by the plurality of neural network submodels according to the road condition information of the candidate lane group where the target lane is located, where each neural network submodel is used to determine one predicted trajectory, and then a trajectory with a difference smaller than a threshold value may be determined from the plurality of predicted trajectories as the target predicted trajectory according to a difference between each predicted trajectory and the actual traveling trajectory, and the neural network submodel corresponding to the target predicted trajectory is trained based on a difference between the target predicted trajectory and the actual traveling trajectory.

In one implementation, a predicted trajectory that has the smallest difference from the actual travel trajectory among the plurality of predicted trajectories may be used as the target predicted trajectory.

Since the lane driving intention defined in the embodiment of the present application is difficult to define and label, the embodiment of the present application does not predict an explicit behavior intention, but uses a plurality of prediction branches, each branch being a predictor for identifying one kind of driving intention. When the third neural network is trained based on a plurality of predicted trajectories output by the third neural network, only one of the output predicted trajectories closest to the real driving trajectory can be selected for supervised training, that is, only the network parameters of the branch are updated, and finally different branches learn trajectories of different samples, that is, trajectories of different behaviors are predicted.

In the inference process of the model, each neural network submodel needs to process road condition information to obtain a predicted track, in order to express which predicted track needs to be used as a target predicted track, each neural network submodel also needs to output a confidence corresponding to the predicted track, and correspondingly, in the process of training the third neural network model, losses adopted for training the third neural network model are constructed based on the difference between the confidence output by each neural network submodel and the real confidence. Specifically, a plurality of predicted tracks of the target vehicle and the confidence of each predicted track may be determined by the plurality of neural network submodels, and the neural network submodel corresponding to the target predicted track is trained based on the difference between the target predicted track and the actual travel track and the difference between the confidence of the target predicted track and the value 1.

The following describes a model training method provided in an embodiment of the present application with reference to an example of training:

in order to prevent training samples from concentrating on a certain neural network submodel, two-stage training can be performed, in the first stage, supervision training is performed on all the neural network submodels, so that each neural network submodel is subjected to parameter updating, and the loss function can be as follows:

wherein,

a set of candidate lane groups with all labels as positive samples is obtained, M' is the number of the candidate lane groups of the set, K is the number of the neural network submodels, and Y is _t Is the real driving track of the target vehicle,

the predicted trace for the mth predicted trace, the kth predicted branch,

is the MSE loss function. Since the first stage trains all neural network submodels, the loss is not constructed based on the confidence level of each neural network submodel output.

In the second stage, independent neural network submodels are trained, and the loss function can be:

the confidence label of the neural network submodel closest to the real driving trajectory is 1, and the rest are 0, then the loss function may be:

wherein,

the confidence of the predicted trajectory for the neural network,

in the form of a confidence label, the user may,

is the cross entropy loss.

As shown in fig. 7, fig. 7 is an overall framework of the neural network provided in the embodiment of the present application. On the left side of fig. 7, the historical driving route of the vehicle and the road condition information of each candidate lane group are subjected to rasterization coding and then input to the CNN, so as to obtain the selection probability and the environmental characteristics of the candidate lane group. In the upper part of fig. 7, the obstacles around the vehicle and the historical driving route of the target vehicle are coded and then pass through a multi-head attention attribute to obtain multi-target interaction characteristics, the multi-target interaction characteristics are combined with environmental characteristics and then input into a neural network comprising a plurality of neural network submodels, and each neural network submodel predicts the predicted track and the corresponding confidence coefficient under the current candidate lane group. In fig. 7, the road condition information of the M candidate lane groups is respectively rasterized and encoded, and the environmental characteristics and the selected probability are calculated, and the K neural network submodels respectively predict different trajectories, so that M × K predicted trajectories and corresponding confidence levels are finally predicted.

The intention prediction method and the model training method provided in the embodiments of the present application are described above from the viewpoint of the method, and the intention prediction device and the model training device provided in the embodiments of the present application are described next from the viewpoint of the device.

Referring to fig. 8, fig. 8 is a schematic structural diagram of a vehicle intention prediction apparatus according to an embodiment of the present application, and as shown in fig. 8, the apparatus 800 includes:

an obtaining module 801, configured to obtain a position of a target vehicle; the acquisition module is further used for acquiring the road condition information of each candidate lane group in the candidate lane groups determined by the candidate determination module;

for a detailed description of the obtaining module 801, reference may be made to the description of step 201 and step 203 in the foregoing embodiment, which is not described herein again.

The lane determining module 802 is configured to determine, according to the position of the target vehicle, a plurality of candidate lane groups that the target vehicle can travel within a preset distance, where a lane direction difference of different candidate lane groups within the preset distance is greater than a target threshold; the lane determining module is further configured to determine a target candidate lane group from the plurality of candidate lane groups according to the road condition information, and a lane direction of the target candidate lane group is used as a driving intention of the target vehicle.

For a detailed description of the lane determining module 802, reference may be made to the descriptions of step 202 and step 204 in the foregoing embodiments, which are not described herein again.

In one possible implementation, each of the candidate lane groups includes at least one lane, and a lane direction difference between the lanes included in each of the candidate lane groups is less than the target threshold.

In one possible implementation, the lane direction is a lane line direction within the preset distance from the position of the target vehicle.

the preset distance is obtained by calculating the driving speed of the target vehicle and the intention prediction time, wherein the preset distance is positively correlated with the driving speed of the target vehicle and the intention prediction time, and the intention prediction time is less than or equal to 7 seconds.

In one possible implementation, the traffic information is represented as a rasterized image that includes a plurality of image channels, each image channel being used to represent at least one of the traffic information.

In one possible implementation, the apparatus further comprises:

In one possible implementation, the trajectory prediction module is specifically configured to determine, according to the road condition information of the target candidate lane group, a plurality of candidate trajectories of the target vehicle and a confidence level of each candidate trajectory through the neural network model;

Referring to fig. 9, fig. 9 is a schematic structural diagram of a model training apparatus according to an embodiment of the present application, and as shown in fig. 9, the apparatus 900 includes:

an obtaining module 901, configured to obtain a first neural network model, a position of a target vehicle, and a real driving track, where the real driving track of the target vehicle is located on a target lane; the acquisition module is further used for acquiring the road condition information of each candidate lane group in the plurality of candidate lane groups determined by the lane determination module;

for a specific description of the obtaining module 901, reference may be made to the description of step 601 and step 603 in the foregoing embodiment, and details are not described here again.

The lane determining module 902 is configured to determine, according to the position of the target vehicle, multiple candidate lane groups that the target vehicle can travel within a preset distance, where a lane direction difference of different candidate lane groups within the preset distance is greater than a target threshold;

for a detailed description of the lane determining module 902, reference may be made to the description of step 602 in the foregoing embodiment, which is not described herein again.

A model training module 903, configured to determine a target candidate lane group from the multiple candidate lane groups through the first neural network model according to the road condition information acquired by the acquiring module, and train the first neural network model based on a difference between the target candidate lane group and the candidate lane group where the target lane is located, so as to obtain a second neural network model.

For a detailed description of the model training module 903, reference may be made to the description of step 604 in the foregoing embodiment, which is not described herein again.

In a possible implementation, the model training module is specifically configured to determine, according to the road condition information of each candidate lane group, a selection probability of each candidate lane through the first neural network model;

the device also includes:

In one possible implementation, the third neural network model includes a plurality of neural network submodels, and the trajectory prediction module is specifically configured to determine a plurality of predicted trajectories of the target vehicle through the plurality of neural network submodels according to the road condition information of the candidate lane group in which the target lane is located, where each neural network submodel is configured to determine one predicted trajectory;

It should be noted that, because the contents of information interaction, execution process, and the like between the modules/units of the apparatus are based on the same concept as that of the embodiment of the method of the present application, the technical effect brought by the contents is the same as that of the embodiment of the method of the present application, and specific contents may refer to the description in the foregoing illustrated embodiment of the method of the present application, and are not repeated herein.

Referring to fig. 10, fig. 10 is a schematic structural diagram of an execution device provided in the embodiment of the present application, and the execution device 1000 may be embodied as a virtual reality VR device, a mobile phone, a tablet, a notebook computer, an intelligent wearable device, a monitoring data processing device or a server, which is not limited herein. Specifically, the execution apparatus 1000 includes: a receiver 1001, a transmitter 1002, a processor 1003 and a memory 1004 (wherein the number of processors 1003 in the execution device 1000 may be one or more, and one processor is taken as an example in fig. 10), wherein the processor 1003 may include an application processor 10031 and a communication processor 10032. In some embodiments of the present application, the receiver 1001, the transmitter 1002, the processor 1003, and the memory 1004 may be connected by a bus or other means.

The memory 1004 may include both read-only memory and random access memory and provides instructions and data to the processor 1003. A portion of memory 1004 may also include non-volatile random access memory (NVRAM). The memory 1004 stores the processor and the operating instructions, executable modules or data structures, or a subset or an expanded set thereof, wherein the operating instructions may include various operating instructions for performing various operations.

The processor 1003 controls the operation of the execution apparatus. In a particular application, the various components of the execution device are coupled together by a bus system that may include a power bus, a control bus, a status signal bus, etc., in addition to a data bus. For clarity of illustration, the various buses are referred to in the figures as a bus system.

The method disclosed in the embodiment of the present application may be applied to the processor 1003 or implemented by the processor 1003. The processor 1003 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be implemented by integrated logic circuits of hardware or instructions in the form of software in the processor 1003. The processor 1003 may be a general-purpose processor, a Digital Signal Processor (DSP), a microprocessor or a microcontroller, and may further include an Application Specific Integrated Circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, or discrete hardware components. The processor 1003 may implement or execute the methods, steps and logic blocks disclosed in the embodiments of the present application. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in the memory 1004, and the processor 1003 reads the information in the memory 1004, and completes the steps of the method in combination with the hardware thereof.

The receiver 1001 may be used to receive input numeric or character information and generate signal inputs related to performing relevant settings and function control of the device. The transmitter 1002 may be configured to output numeric or character information via a first interface; the transmitter 1002 may also be configured to send instructions to the disk groups through the first interface to modify data in the disk groups; the transmitter 1002 may also include a display device such as a display screen.

In this embodiment, in one case, the processor 1003 is configured to execute the driving intention prediction method described in the embodiment corresponding to fig. 2.

Referring to fig. 11, fig. 11 is a schematic structural diagram of a training device provided in the embodiment of the present application, a model training apparatus described in the corresponding embodiment of fig. 9 may be deployed on the training device 1100, and is used to implement functions of the model training apparatus in the corresponding embodiment of fig. 9, specifically, the training device 1100 is implemented by one or more servers, and the training device 1100 may generate relatively large differences due to different configurations or performances, and may include one or more Central Processing Units (CPUs) 1119 (e.g., one or more processors) and a memory 1132, and one or more storage media 1130 (e.g., one or more mass storage devices) that store an application program 1142 or data 1144. Memory 1132 and storage media 1130 may be, among other things, transient storage or persistent storage. The program stored on the storage medium 1130 may include one or more modules (not shown), each of which may include a sequence of instructions for operating on the exercise device. Still further, central processor 1119 may be configured to communicate with storage medium 1130 to carry out a sequence of instruction operations on storage medium 1130 on exercise device 1100.

Training apparatus 1100 may also include one or more power supplies 1126, one or more wired or wireless network interfaces 1150, one or more input-output interfaces 1158; or one or more operating systems 1141, such as Windows Server, mac OS XTM, unixTM, linuxTM, freeBSDTM, etc.

In the embodiment of the present application, the central processing unit 1119 is configured to execute the model training method provided in the embodiment corresponding to fig. 6.

Embodiments of the present application also provide a computer program product, which when executed on a computer causes the computer to perform the steps performed by the aforementioned execution device, or causes the computer to perform the steps performed by the aforementioned training device.

Also provided in an embodiment of the present application is a computer-readable storage medium, in which a program for signal processing is stored, and when the program is run on a computer, the program causes the computer to execute the steps executed by the aforementioned execution device, or causes the computer to execute the steps executed by the aforementioned training device.

The execution device, the training device, or the terminal device provided in the embodiment of the present application may specifically be a chip, where the chip includes: a processing unit, which may be for example a processor, and a communication unit, which may be for example an input/output interface, a pin or a circuit, etc. The processing unit may execute the computer execution instructions stored in the storage unit to enable the chip in the execution device to execute the data processing method described in the above embodiment, or to enable the chip in the training device to execute the data processing method described in the above embodiment. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, and the like, and the storage unit may also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or another type of static storage device that can store static information and instructions, a Random Access Memory (RAM), and the like.

Specifically, referring to fig. 12, fig. 12 is a schematic structural diagram of a chip according to an embodiment of the present disclosure, where the chip may be represented as a neural network processor NPU 1200, and the NPU 1200 is mounted on a Host CPU (Host CPU) as a coprocessor, and the Host CPU allocates tasks. The core part of the NPU is an arithmetic circuit 1203, and the controller 1204 controls the arithmetic circuit 1203 to extract matrix data in the memory and perform multiplication.

In some implementations, the arithmetic circuitry 1203 internally includes multiple processing units (PEs). In some implementations, the operational circuitry 1203 is a two-dimensional systolic array. The arithmetic circuit 1203 may also be a one-dimensional systolic array or other electronic circuit capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuitry 1203 is a general-purpose matrix processor.

For example, assume that there is an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit fetches the data corresponding to the matrix B from the weight memory 1202 and buffers it on each PE in the arithmetic circuit. The arithmetic circuit takes the matrix a data from the input memory 1201 and performs matrix operation with the matrix B, and partial results or final results of the obtained matrix are stored in an accumulator (accumulator) 1208.

The unified memory 1206 is used for storing input data and output data. The weight data is directly passed through a Memory cell Access Controller (DMAC) 1205, and the DMAC is carried into the weight Memory 1202. The input data is also carried into the unified memory 1206 by the DMAC.

The BIU is a Bus Interface Unit 1210 for the interaction of the AXI Bus with the DMAC and an Instruction Fetch memory (IFB) 1209.

A Bus Interface Unit 1210 (Bus Interface Unit, BIU for short) is configured to fetch an instruction from the external memory 1209, and to fetch the original data of the input matrix a or the weight matrix B from the external memory by the storage Unit access controller 1205.

The DMAC is mainly used to transfer input data in the external memory DDR to the unified memory 1206 or to transfer weight data into the weight memory 1202 or to transfer input data into the input memory 1201.

The vector calculation unit 1207 includes a plurality of operation processing units, and performs further processing on the output of the operation circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, magnitude comparison, and the like, if necessary. The method is mainly used for non-convolution/full-connection layer network calculation in the neural network, such as Batch Normalization, pixel-level summation, up-sampling of a feature plane and the like.

In some implementations, the vector calculation unit 1207 can store the processed output vector to the unified memory 1206. For example, the vector calculation unit 1207 may calculate a linear function; alternatively, a nonlinear function is applied to the output of the operation circuit 1203, for example, to linearly interpolate the feature plane extracted from the convolution layer, and then, for example, to accumulate the vector of values to generate the activation value. In some implementations, the vector calculation unit 1207 generates normalized values, pixel-level summed values, or both. In some implementations, the vector of processed outputs can be used as activation inputs to arithmetic circuitry 1203, e.g., for use in subsequent layers in a neural network.

An instruction fetch buffer (issue fetch buffer) 1209 connected to the controller 1204, configured to store instructions used by the controller 1204;

the unified memory 1206, the input memory 1201, the weight memory 1202, and the instruction fetch memory 1209 are On-Chip memories. The external memory is private to the NPU hardware architecture.

The processor mentioned in any of the above may be a general purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the above programs.

It should be noted that the above-described embodiments of the apparatus are merely schematic, where the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. In addition, in the drawings of the embodiments of the apparatus provided in the present application, the connection relationship between the modules indicates that there is a communication connection therebetween, and may be implemented as one or more communication buses or signal lines.

Through the above description of the embodiments, those skilled in the art will clearly understand that the present application can be implemented by software plus necessary general-purpose hardware, and certainly can also be implemented by special-purpose hardware including special-purpose integrated circuits, special-purpose CPUs, special-purpose memories, special-purpose components and the like. Generally, functions performed by computer programs can be easily implemented by corresponding hardware, and specific hardware structures for implementing the same functions may be various, such as analog circuits, digital circuits, or dedicated circuits. However, for the present application, the implementation of a software program is more preferable. Based on such understanding, the technical solutions of the present application or portions thereof contributing to the prior art may be embodied in the form of a software product, which is stored in a readable storage medium, such as a floppy disk, a usb disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, an exercise device, or a network device) to execute the method according to the embodiments of the present application.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product.

The computer program product includes one or more computer instructions. The procedures or functions described in accordance with the embodiments of the application are all or partially generated when the computer program instructions are loaded and executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, training device, or data center to another website site, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that a computer can store or a data storage device, such as a training device, data center, etc., that includes one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), among others.

Claims

1. A vehicle intent prediction method, the method comprising:

acquiring the position of a target vehicle;

and determining a target candidate lane group from the candidate lane groups according to the road condition information, wherein the lane direction of the target candidate lane group is used as the driving intention of the target vehicle.

2. The method of claim 1, wherein each of the candidate lane groups comprises at least one lane, and wherein a lane direction difference of the lanes comprised by each of the candidate lane groups within the preset distance is less than the target threshold.

3. The method according to claim 1 or 2, wherein the lane direction is a lane line direction within the preset distance from a position of the target vehicle as a starting point.

4. A method according to any one of claims 1 to 3, wherein the predetermined distance is a fixed value of less than 200 meters; or,

the preset distance is obtained by calculating the driving speed of the target vehicle and the intention prediction time, wherein the preset distance is positively correlated with the driving speed of the target vehicle and the intention prediction time, and the intention prediction time is less than or equal to 2 seconds.

5. The method of any of claims 1 to 4, further comprising:

acquiring a historical driving route of the target vehicle;

determining a target candidate lane group from the multiple candidate lane groups according to the traffic information, including:

and determining a target candidate lane group from the plurality of candidate lane groups according to the road condition information and the historical driving route.

6. The method according to any one of claims 1 to 5, wherein the traffic information comprises at least one of the following information:

7. The method according to any one of claims 1 to 6, wherein the traffic information is represented as a rasterized image, the rasterized image comprising a plurality of image channels, each image channel being configured to represent at least one of the traffic information.

8. The method of any of claims 1 to 7, further comprising:

and determining the target predicted track of the target vehicle through a neural network model according to the road condition information of the target candidate lane group.

9. The method of claim 8, further comprising:

acquiring historical driving routes of the target vehicle and at least one associated vehicle positioned around the target vehicle;

determining the driving influence degree of the at least one associated vehicle on the target vehicle through an attention mechanism according to the historical driving route;

determining a target predicted track of the target vehicle through a neural network model according to the road condition information of the target candidate lane group, wherein the determining comprises the following steps:

10. The method of claim 8, wherein determining the target predicted trajectory of the target vehicle through a neural network model according to the road condition information of the target candidate lane group comprises:

determining a target predicted trajectory of the target vehicle from the plurality of candidate trajectories according to the confidence.

11. A method of model training, comprising:

acquiring a first neural network model, a position of a target vehicle and a real driving track, wherein the real driving track of the target vehicle is positioned on a target lane;

12. The method of claim 11, wherein the determining a target candidate lane group from the candidate lane groups according to the traffic information by the first neural network model, and training the first neural network model based on a difference between the target candidate lane group and the candidate lane group in which the target lane is located comprises:

13. The method according to claim 11 or 12, characterized in that the method further comprises:

acquiring a third neural network model;

14. The method of claim 13, wherein the third neural network model comprises a plurality of neural network submodels, and the determining the target predicted trajectory of the target vehicle through the third neural network model according to the road condition information of the candidate lane group in which the target lane is located comprises:

determining a trajectory, of which the difference is smaller than a threshold value, from the plurality of predicted trajectories as the target predicted trajectory according to the difference between each of the plurality of predicted trajectories and the actual travel trajectory;

training the third neural network model based on a difference between the target predicted trajectory and the actual travel trajectory, including:

and training a neural network submodel corresponding to the target predicted track based on the difference between the target predicted track and the real running track.

15. The method of claim 14, wherein said determining a plurality of predicted trajectories of the target vehicle by the plurality of neural network submodels comprises:

determining, by the plurality of neural network submodels, a plurality of predicted trajectories for the target vehicle and a confidence for each predicted trajectory;

the training of the neural network submodel corresponding to the target predicted track based on the difference between the target predicted track and the real driving track comprises the following steps:

and training a neural network submodel corresponding to the target predicted track based on the difference between the target predicted track and the real driving track and the difference between the confidence coefficient of the target predicted track and the numerical value 1.

16. A vehicle intention prediction apparatus, characterized in that the apparatus comprises:

the acquisition module is used for acquiring the position of a target vehicle; the acquisition module is further used for acquiring the road condition information of each candidate lane group in the candidate lane groups determined by the candidate determination module;

17. The apparatus of claim 16, wherein each candidate lane group comprises at least one lane, and wherein a lane direction difference of the lanes comprised by each candidate lane group within the preset distance is less than the target threshold.

18. The apparatus according to claim 16 or 17, wherein the lane direction is a lane line direction within the preset distance from a position of the target vehicle as a starting point.

19. The apparatus according to any one of claims 16 to 18, wherein said predetermined distance is a fixed value less than 200 meters; or,

the preset distance is obtained by calculating the traveling speed of the target vehicle and the intention prediction time, wherein the preset distance is positively correlated with the traveling speed of the target vehicle and the intention prediction time, and the intention prediction time is less than or equal to 2 seconds.

20. The apparatus according to any one of claims 16 to 19, wherein the obtaining module is further configured to obtain a historical driving route of the target vehicle;

21. The apparatus according to any one of claims 16 to 20, wherein the traffic information comprises at least one of the following information:

22. The apparatus according to any one of claims 16 to 21, wherein the traffic information is represented as a rasterized image, the rasterized image comprising a plurality of image channels, each image channel being configured to represent at least one of the traffic information.

23. The apparatus of any one of claims 16 to 22, further comprising:

24. The apparatus of claim 23, wherein the obtaining module is further configured to obtain historical driving routes of the target vehicle and at least one associated vehicle located around the target vehicle;

25. The apparatus according to claim 24, wherein the trajectory prediction module is specifically configured to determine a plurality of candidate trajectories of the target vehicle and a confidence level of each candidate trajectory through the neural network model according to the road condition information of the target candidate lane group;

26. A model training apparatus, comprising:

the system comprises an acquisition module, a first neural network model, a position of a target vehicle and a real driving track, wherein the real driving track of the target vehicle is positioned on a target lane; the acquiring module is further configured to acquire road condition information of each candidate lane group in the plurality of candidate lane groups determined by the lane determining module;

27. The apparatus of claim 26, wherein the model training module is specifically configured to determine a selection probability of each candidate lane through the first neural network model according to the traffic information of each candidate lane group;

28. The apparatus of claim 26 or 27, wherein the obtaining module is further configured to obtain a third neural network model;

the device further comprises:

29. The apparatus of claim 28, wherein the third neural network model comprises a plurality of neural network submodels, and the trajectory prediction module is specifically configured to determine a plurality of predicted trajectories of the target vehicle through the plurality of neural network submodels according to the traffic information of the candidate lane group in which the target lane is located, wherein each neural network submodel is configured to determine one predicted trajectory;

determining a trajectory with a difference smaller than a threshold value from the plurality of predicted trajectories as the target predicted trajectory according to a difference between each of the plurality of predicted trajectories and the actual travel trajectory;

the model training module is specifically configured to train a neural network sub-model corresponding to the target predicted trajectory based on a difference between the target predicted trajectory and the actual travel trajectory.

30. The apparatus of claim 29, wherein the trajectory prediction module is in particular configured to determine a plurality of predicted trajectories of the target vehicle and a confidence level for each predicted trajectory by the plurality of neural network submodels;

the model training module is specifically configured to train a neural network sub-model corresponding to the target predicted trajectory based on a difference between the target predicted trajectory and the actual travel trajectory and a difference between a confidence of the target predicted trajectory and a value 1.

31. A server, comprising a memory and a processor; the memory stores code, the processor is configured to execute the code, and when executed, the server performs the method of any of claims 1 to 15.

32. A computer storage medium, characterized in that it stores a computer program which, when executed by a computer, causes the computer to carry out the method of any one of claims 1 to 15.

33. A computer program product having stored thereon instructions which, when executed by a computer, cause the computer to carry out the method of any one of claims 1 to 15.