WO2022088658A1

WO2022088658A1 - Pedestrian crossing intention estimation method and apparatus, device, and vehicle

Info

Publication number: WO2022088658A1
Application number: PCT/CN2021/095259
Authority: WO
Inventors: 范时伟; 李飞; 李向旭
Original assignee: 华为技术有限公司
Priority date: 2020-10-29
Filing date: 2021-05-21
Publication date: 2022-05-05
Also published as: CN114426030B; CN114426030A

Abstract

A pedestrian crossing intention estimation method and apparatus, a device, and a vehicle, relating to the field of smart driving. The method comprises: obtaining status information of at least one pedestrian around a vehicle (S301); according to pedestrian status information predicted at a previous time and an apparent intention of the pedestrian at the previous time, determining pedestrian status information predicted at a current time (S302); calculating a deep intention at the current time according to the pedestrian status information predicted at the current time, pedestrian status information observed at the current time and a deep intention of the pedestrian at the previous time (S303); according to status information of a first pedestrian at the current time and a deep intention of the first pedestrian at the previous time, calculating an apparent intention of the first pedestrian to cross a road (S304); and according to the status information of the pedestrian at the current time and the apparent intention at the current time, calculating a lane intention of each lane of the road where a vehicle is located (S305). By means of the method, a high-precision, real-time and accurate apparent intention estimation result is obtained, and then, whether the first pedestrian crosses the road at a first time is determined according to the apparent intention of the first pedestrian at the first time.

Description

一种行人穿行意图估计方法、装置、设备和汽车A pedestrian crossing intention estimation method, device, device and car

本申请要求于2020年10月29日提交中国国家知识产权局、申请号为202011176754.4、申请名称为“一种行人穿行意图估计方法、装置、设备和汽车”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese patent application filed on October 29, 2020 with the State Intellectual Property Office of the People's Republic of China, the application number is 202011176754.4, and the application title is "A method, device, equipment and automobile for estimating pedestrian crossing intention", all of which are The contents are incorporated herein by reference.

技术领域technical field

本发明涉及智能驾驶技术领域，尤其涉及一种行人穿行意图估计方法、装置、设备和汽车。The present invention relates to the technical field of intelligent driving, and in particular, to a method, device, device and automobile for estimating pedestrian's passing intention.

背景技术Background technique

随着智能驾驶的到来，智能车成为各大厂商重点研究的目标。对于智能驾驶来说，其实现过程主要包括定位、感知、预测、规划控制等步骤。其中，预测步骤主要作用是估计周围目标的未来位置和未来行为，以便规划控制模块作出相应的决策，从而避免交通事故的发生。With the advent of smart driving, smart cars have become the focus of research by major manufacturers. For intelligent driving, its realization process mainly includes steps such as positioning, perception, prediction, planning and control. Among them, the main function of the prediction step is to estimate the future position and future behavior of the surrounding objects, so that the planning control module can make corresponding decisions, thereby avoiding the occurrence of traffic accidents.

然而，由于行人的动态性较高、受地图和交通规则的约束较小等特点，对行人的运动轨迹和意图进行预测是具有相当大的挑战。如果预测不准确，车辆很容易发生撞人事件，这样对车辆带来很大的隐患。However, due to the characteristics of pedestrians with high dynamics and less constraints from maps and traffic rules, it is quite challenging to predict pedestrian trajectories and intentions. If the prediction is inaccurate, the vehicle is prone to collision incidents, which brings great hidden dangers to the vehicle.

发明内容SUMMARY OF THE INVENTION

为了解决上述的问题，本申请的实施例提供了一种行人穿行意图估计方法、装置、设备和汽车。In order to solve the above-mentioned problems, the embodiments of the present application provide a method, apparatus, device, and vehicle for estimating pedestrian's passing intention.

第一方面，本申请提供一种行人穿行意图估计方法，包括：获取车辆周围至少一个行人的状态信息，所述至少一个行人包括第一行人；根据第一时刻的第一行人的状态信息和第二时刻的第一行人的深层意图，计算第一时刻的第一行人的表层意图，所述第二时刻在所述第一时刻之前，所述深层意图为所述第一行人在整个生命周期内穿行道路的概率，所述表层意图为所述第一行人在当前时刻正在或即将穿行道路的概率；根据所述第一时刻的第一行人的表层意图，确定所述第一行人在第一时刻是否穿行道路。In a first aspect, the present application provides a method for estimating a pedestrian's passing intention, including: acquiring state information of at least one pedestrian around a vehicle, where the at least one pedestrian includes a first pedestrian; according to the state information of the first pedestrian at a first moment and the deep intent of the first pedestrian at the second moment, calculate the surface intent of the first pedestrian at the first moment, the second moment is before the first moment, and the deep intent is the first pedestrian The probability of crossing the road in the entire life cycle, the surface intent is the probability that the first pedestrian is or is about to cross the road at the current moment; according to the surface intent of the first pedestrian at the first moment, determine the Whether the first pedestrian crosses the road at the first moment.

在该实施方式中，通过不断获取车辆行驶的道路上的行人、道路的车道线、车道标识、周围其他车辆等信息，然后分析出当前时刻的行人的行为特征、静态交互特征、动态交互特征等状态信息后，结合上一时刻的深层意图，可以得到兼具高精度、实时性和准确性的表层意图估计结果。In this embodiment, by continuously acquiring information such as pedestrians on the road where the vehicle is traveling, lane lines, lane markings, and other surrounding vehicles, etc., the behavior characteristics, static interaction characteristics, dynamic interaction characteristics, etc. of pedestrians at the current moment are analyzed. After the state information, combined with the deep intent at the previous moment, the surface intent estimation result with high precision, real-time and accuracy can be obtained.

在一种实施方式中，在所述根据第一时刻的第一行人的状态信息和第二时刻的第一行人的深层意图，计算第一时刻的第一行人的表层意图之前，包括：根据所述第一时刻的第一行人的状态信息，计算第一时刻的第一行人的行人特征的似然概率；所述根据第一时刻的第一行人的状态信息和第二时刻的第一行人的深层意图，计算第一时刻的第一行人的表层意图，包括：根据所述第一时刻的第一行人的行人特征的似然概率和所述第二时刻的第一行人的深层意图，计算所述第一时刻的第一行人的表层意图。In an embodiment, before calculating the surface intention of the first pedestrian at the first moment according to the state information of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment, the method includes: : Calculate the likelihood probability of the pedestrian feature of the first pedestrian at the first moment according to the state information of the first pedestrian at the first moment; The deep intention of the first pedestrian at the moment, calculating the surface intention of the first pedestrian at the first moment, including: according to the likelihood probability of the pedestrian feature of the first pedestrian at the first moment and the second moment For the deep intention of the first pedestrian, the surface intention of the first pedestrian at the first moment is calculated.

在该实施方式中，车辆在获取行人的状态信息过程中，由于有时候监测的信息并不一定完全正确，可能出现错误或误差，如由于行人目标较小且运动速度慢，有时候不一定检测到行人、检测到行人速度不变等误差，这会对行人特征产生较大的影响，因此在得到行人的状态信息后，对各种检测到的信息进行融合滤波，对误差进行一定程度的平滑，使得检测到行人状态信息对应的行人特征更加真实。In this embodiment, in the process of acquiring the state information of pedestrians, sometimes the monitored information may not be completely correct, and errors or errors may occur. Errors such as pedestrian detection and pedestrian speed being detected will have a greater impact on pedestrian characteristics. Therefore, after obtaining the pedestrian status information, various detected information are fused and filtered to smooth the error to a certain extent. , so that the pedestrian features corresponding to the detected pedestrian state information are more realistic.

在一种实施方式中，所述方法还包括：当所述第一时刻的第一行人的表层意图大于设定阈值时，控制扬声器播放预警信号和/或在显示屏上显示预警信息。In one embodiment, the method further includes: when the surface intention of the first pedestrian at the first moment is greater than a set threshold, controlling the speaker to play an early warning signal and/or displaying the warning information on the display screen.

在一种实施方式中，在所述根据第一时刻的第一行人的状态信息和第二时刻的第一行人的深层意图，计算第一时刻的第一行人的表层意图之前，所述方法包括：根据第二时刻的第一行人的状态信息和第二时刻的第一行人的表层意图，计算第一时刻的预测第一行人的状态信息；根据所述第一时刻的预测第一行人的状态信息、所述第一时刻的第一行人的状态信息和第二时刻的第一行人的深层意图，计算第一时刻的第一行人的深层意图。In an implementation manner, before calculating the surface intention of the first pedestrian at the first moment according to the state information of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment, the The method includes: calculating the state information of the predicted first pedestrian at the first moment according to the state information of the first pedestrian at the second moment and the surface intention of the first pedestrian at the second moment; The state information of the first pedestrian, the state information of the first pedestrian at the first moment, and the deep intention of the first pedestrian at the second moment are predicted, and the deep intention of the first pedestrian at the first moment is calculated.

在该实施方式中，根据上一时刻的深层意图、当前时刻的行人的状态信息和当前时刻预测的行人的状态信息，对上一时刻的深层意图进行重新估计和更新，使得新深层意图可靠性随着历史观测信息的累积而增高。In this embodiment, the deep intent at the previous moment is re-estimated and updated according to the deep intent at the previous moment, the state information of the pedestrian at the current moment, and the state information of the pedestrian predicted at the current moment, so that the new deep intent is reliable. It increases with the accumulation of historical observational information.

在一种实施方式中，所述方法还包括：根据所述第一时刻的第一行人的状态信息和所述第一时刻的第一行人的表层意图，计算车辆所处道路的至少一个车道的车道意图，所述车道意图为所述第一行人穿行第一车道的概率，所述至少一个车道包括所述第一车道。In one embodiment, the method further includes: calculating at least one of the roads on which the vehicle is located according to the state information of the first pedestrian at the first moment and the surface intention of the first pedestrian at the first moment Lane intent of the lane, where the lane intent is the probability that the first pedestrian traverses the first lane, and the at least one lane includes the first lane.

在该实施方式中，通过根据所述第一时刻的第一行人的状态信息，推算出各车道的道路结构和相关车辆提取车道级的静态交互特征和动态交互特征，以及目标行人的行为特征，然后计算出行人的车道意图，让车辆能够输出行人相对各车道的穿行意图，使自动驾驶车辆提前做出应对。In this embodiment, according to the state information of the first pedestrian at the first moment, the road structure of each lane and related vehicles are calculated to extract the lane-level static interaction features and dynamic interaction features, as well as the behavior features of the target pedestrian. , and then calculate the pedestrian's lane intention, so that the vehicle can output the pedestrian's passing intention relative to each lane, so that the autonomous vehicle can respond in advance.

第二方面，本申请还提供了一种行人穿行意图估计装置，包括：获取单元，用于获取车辆周围至少一个行人的状态信息，所述至少一个行人包括第一行人；处理单元，用于根据第一时刻的第一行人的状态信息和第二时刻的第一行人的深层意图，计算第一时刻的第一行人的表层意图，所述第二时刻在所述第一时刻之前，所述深层意图为所述第一行人在整个生命周期内穿行道路的概率，所述表层意图为所述第一行人在当前时刻正在或即将穿行道路的概率；所述处理单元，还用于根据所述第一时刻的第一行人的表层意图，确定所述第一行人在第一时刻是否穿行道路。In a second aspect, the present application further provides an apparatus for estimating a pedestrian's passing intention, including: an acquisition unit for acquiring status information of at least one pedestrian around the vehicle, the at least one pedestrian including the first pedestrian; and a processing unit for According to the state information of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment, the surface intention of the first pedestrian at the first moment is calculated, and the second moment is before the first moment , the deep intention is the probability of the first pedestrian crossing the road in the entire life cycle, and the surface intention is the probability that the first pedestrian is or is about to cross the road at the current moment; the processing unit, also for determining whether the first pedestrian crosses the road at the first moment according to the surface intention of the first pedestrian at the first moment.

在一种实施方式中，所述处理单元，具体用于根据所述第一时刻的第一行人的状态信息，计算第一时刻的第一行人的行人特征的似然概率；根据所述第一时刻的第一行人的行人特征的似然概率和所述第二时刻的第一行人的深层意图，计算所述第一时刻的第一行人的表层意图。In an embodiment, the processing unit is specifically configured to calculate the likelihood probability of the pedestrian feature of the first pedestrian at the first moment according to the state information of the first pedestrian at the first moment; The likelihood probability of the pedestrian feature of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment are used to calculate the surface intention of the first pedestrian at the first moment.

在一种实施方式中，所述处理单元，还用于当所述第一时刻的第一行人的表层意图大于设定阈值时，控制扬声器播放预警信号和/或在显示屏上显示预警信息。In one embodiment, the processing unit is further configured to control the speaker to play an early warning signal and/or display the warning information on the display screen when the surface intention of the first pedestrian at the first moment is greater than a set threshold .

在一种实施方式中，所述处理单元，还用于根据第二时刻的第一行人的状态信息和第二时刻的第一行人的表层意图，计算第一时刻的预测第一行人的状态信息；根据所述第一时刻的预测第一行人的状态信息、所述第一时刻的第一行人的状态信息和第二时刻的第一行人的深层意图，计算第一时刻的第一行人的深层意图。In an embodiment, the processing unit is further configured to calculate the predicted first pedestrian at the first moment according to the state information of the first pedestrian at the second moment and the surface intention of the first pedestrian at the second moment According to the state information of the predicted first pedestrian at the first moment, the state information of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment, calculate the first moment The deep intent of the first pedestrian.

在一种实施方式中，所述处理单元，还用于根据所述第一时刻的第一行人的状态信息和所述第一时刻的第一行人的表层意图，计算车辆所处道路的至少一个车道的车道意图，所述车道意图为所述第一行人穿行第一车道的概率，所述至少一个车道包括所述第一车道。In an implementation manner, the processing unit is further configured to calculate, according to the state information of the first pedestrian at the first moment and the surface intention of the first pedestrian at the first moment, the speed of the road where the vehicle is located. Lane intent of at least one lane, the lane intent being the probability of the first pedestrian crossing the first lane, the at least one lane including the first lane.

第三方面，本申请还提供了一种设备，包括至少一个处理器，所述处理器用于执行存储器中存储的指令，以使得终端执行如第一方面各个可能实现的实施例。In a third aspect, the present application further provides a device, including at least one processor, where the processor is configured to execute instructions stored in the memory, so that the terminal executes each possible implementation of the first aspect.

第四方面，本申请还提供了一种汽车，用于执行如第一方面各个可能实现的实施例。In a fourth aspect, the present application also provides an automobile for implementing the various possible embodiments of the first aspect.

第五方面，本申请还提供了一种计算机可读存储介质，其上存储有计算机程序，当所述计算机程序在计算机中执行时，令计算机执行如第一方面各个可能实现的实施例。In a fifth aspect, the present application further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed in a computer, the computer is made to execute the various possible embodiments of the first aspect.

第六方面，本申请还提供了一种计算设备，包括存储器和处理器，其特征在于，所述存储器中存储有可执行代码，所述处理器执行所述可执行代码时，实现如第一方面各个可能实现的实施例。In a sixth aspect, the present application further provides a computing device including a memory and a processor, wherein the memory stores executable codes, and when the processor executes the executable codes, the first Aspects of various possible implementations.

附图说明Description of drawings

下面对实施例或现有技术描述中所需使用的附图作简单地介绍。The following briefly introduces the accompanying drawings required in the description of the embodiments or the prior art.

图1为本申请实施例提供的一种车辆的结构示意图；1 is a schematic structural diagram of a vehicle according to an embodiment of the present application;

图2为车辆在道路上行驶的一种场景；Fig. 2 is a kind of scene of the vehicle driving on the road;

图3为本申请实施例提供的一种行人穿行意图估计方法的过程流程图；FIG. 3 is a process flow diagram of a method for estimating pedestrian passing intention provided by an embodiment of the present application;

图4为本申请实施例提供的表层意图和深层意图通过动态贝叶斯网络结构进行建模的示意图；4 is a schematic diagram of modeling surface-level intentions and deep-level intentions through a dynamic Bayesian network structure provided by an embodiment of the present application;

图5为车辆和行人在道路上的一种场景；Figure 5 is a scene of vehicles and pedestrians on the road;

图6为本申请实施例提供的一种行人穿行意图估计装置的结构示意图。FIG. 6 is a schematic structural diagram of an apparatus for estimating a pedestrian's passing intention according to an embodiment of the present application.

具体实施方式Detailed ways

下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行描述。The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application.

本申请实施例后续将会提到两个概念，为“表层意图”和“深层意图”。其中，表层意图为行人在当前时刻正在或即将穿过道路的概率，也即行人瞬时或短暂时间考虑的穿行意图；深层意图为行人在整个生命周期内的穿过道路的概率，也即行人受环境影响较小的长期考虑的穿行意图。In the embodiments of the present application, two concepts will be mentioned later, namely "surface intent" and "deep intent". Among them, the superficial intention is the probability that the pedestrian is or is about to cross the road at the current moment, that is, the pedestrian’s intention to cross the road in an instant or a short time; Long-term consideration of travel intentions with low environmental impact.

表1表层意图和深层意图比较Table 1 Comparison of surface intent and deep intent

由于表层意图代表了行人当前时刻的穿行意图，所以表层意图对于实时性至关重要，因此本申请实施例中，通过提取行人的细节信息和交互信息，来估计表层意图。然而由于提取的细节信息和交互信息通常噪声比较大，容易影响表层意图估计的稳定性，所以本申请实施例还引入了深层意图，利用上一时刻的表层意图的数据和行人特征信息，计算出深层意图，然后再根据深层意图的数据，提高表层意图的稳定性，从而计算出的表层意图更加准确地预测出行人穿过道路的意图。Since the surface intent represents the pedestrian's current passing intent, the surface intent is very important for real-time performance. Therefore, in the embodiment of the present application, the surface intent is estimated by extracting the pedestrian's detailed information and interaction information. However, since the extracted detail information and interactive information are usually noisy, which easily affects the stability of the surface intent estimation, the embodiment of the present application also introduces a deep intent, and uses the surface intent data and pedestrian feature information at the previous moment to calculate Then, based on the data of the deep intention, the stability of the surface intention is improved, so that the calculated surface intention can more accurately predict the pedestrian's intention to cross the road.

图1为本申请实施例提供的一种车辆的框架结构图。如图1所示，该车辆100包括输入设备101、存储器102、处理器103和总线104。其中，车辆100中的输入设备101、存储器102和处理器103可以通过总线104建立通信连接。FIG. 1 is a frame structure diagram of a vehicle according to an embodiment of the present application. As shown in FIG. 1 , the vehicle 100 includes an input device 101 , a memory 102 , a processor 103 and a bus 104 . Wherein, the input device 101 , the memory 102 and the processor 103 in the vehicle 100 can establish a communication connection through the bus 104 .

输入设备101可以包括车载摄像头、车载雷达、车载导航、全球定位***(global positioning system，GPS)传感器等设备。其中，车载摄像头用于对车辆100行驶方向进行拍摄，以获取包括道路的车道线、车道标识、行人等信息的图像；车载雷达可以为激光雷达、毫米波雷达等等，用于向车辆100周围发射信号和接收返回的信号，以获取车辆100与车辆100的周围其他车辆、行人等障碍物之间的距离信息；车载导航用于根据车辆100的位置和驾驶员输入目的地位置信息，生成导航路线；GPS传感器用于获取车辆100实时的位置信息。The input device 101 may include a vehicle camera, a vehicle radar, a vehicle navigation, a global positioning system (global positioning system, GPS) sensor and other devices. Among them, the on-board camera is used to photograph the driving direction of the vehicle 100 to obtain images including information such as lane lines, lane markings, pedestrians, etc. of the road; the on-board radar can be laser radar, millimeter-wave radar, etc. Transmit signals and receive returned signals to obtain distance information between the vehicle 100 and other vehicles, pedestrians and other obstacles around the vehicle 100; in-vehicle navigation is used to generate navigation based on the location of the vehicle 100 and the destination location information input by the driver Route; GPS sensors are used to obtain real-time location information of the vehicle 100 .

存储器102可以为随机存取存储器(random-access memory，RAM)、硬盘(hard disk drive，HDD)、固态硬盘(solid state drive，SSD)等具有存储功能的设备，用于存储表层意图、深层意图、高精度地图等等，以便后续在计算表层意图和深层意图的过程中，利用历史上存储的数据进行计算。The memory 102 can be a random-access memory (RAM), a hard disk drive (HDD), a solid state drive (SSD), or other devices with storage functions, for storing surface-level intentions and deep-level intentions , high-precision maps, etc., in order to use historically stored data for subsequent calculations in the process of calculating surface intent and deep intent.

处理器103可以为车辆100的电子控制单元(electronic control unit，ECU)，先在接收到输入设备101发送的各个数据后，进行处理，得到车辆100前方的道路中和道路两侧的行人的状态信息、道路的车道线、车道标识等信息，然后根据行人的状态信息、(可选的，存储器102中存储的上一时刻行人的表层意图)和存储器102中存储的上一时刻的深层意图，计算出车辆100前方的道路中和道路两侧的行人的穿行道路的表层意图，得到该行人穿行道路的概率。当该行人穿行道路的概率大于设定的阈值时，生成警告信息，通过显示屏显示或扬声器播放等方式，以提醒驾驶员注意前方道路有行人穿行道路。The processor 103 may be an electronic control unit (ECU) of the vehicle 100. After receiving each data sent by the input device 101, the processor 103 performs processing to obtain the status of the pedestrians on the road in front of the vehicle 100 and on both sides of the road. information, road lane lines, lane markings and other information, and then according to the pedestrian state information, (optionally, the pedestrian's surface intention stored in the memory 102 at the previous moment) and the deep intention stored in the memory 102 at the previous moment, The surface intention of the pedestrian crossing the road in the road in front of the vehicle 100 and on both sides of the road is calculated, and the probability of the pedestrian crossing the road is obtained. When the probability of the pedestrian crossing the road is greater than the set threshold, a warning message is generated and displayed on the display screen or played on the speaker to remind the driver to pay attention to the pedestrian crossing on the road ahead.

处理器103还用于根据存储器102中存储的上一时刻的深层意图、当前时刻的行人的状态信息和当前时刻预测的行人的状态信息，对存储器102中已存储的上一时刻的深层意图进行重新估计和更新，得到新的当前时刻的深层意图，使得其可靠性随着历史观测信息的累积而增高。The processor 103 is further configured to, according to the deep intention of the previous moment stored in the memory 102, the state information of the pedestrian at the current moment, and the state information of the pedestrian predicted at the current moment, perform the processing on the deep intention of the previous moment stored in the memory 102. Re-estimate and update to obtain the new deep intention of the current moment, so that its reliability increases with the accumulation of historical observation information.

处理器103还用于根据接收到的行人的状态信息、道路的车道线、车道标识等信息和存储器102中已存储的新的当前时刻的深层意图，计算行人穿过每个车道的概率(后续称为“车道意图”)，从而确定当前道路上哪些车道可以行驶、哪些车道因行人穿行有危险、以及在发生危险时指示驾驶员如何避险。The processor 103 is further configured to calculate the probability of the pedestrian crossing each lane according to the received state information of the pedestrian, the lane lines of the road, lane markings and other information and the new deep intention at the current moment stored in the memory 102 (subsequent Known as "lane intent"), it determines which lanes on the current road are drivable, which lanes are dangerous for pedestrians to cross, and instructs the driver how to avoid the danger in the event of a hazard.

其中，本申请的处理器103还可以为服务器，车辆100通过通信单元与服务器进行通信，然后由服务器代替处理器103的功能。Wherein, the processor 103 of the present application may also be a server, the vehicle 100 communicates with the server through a communication unit, and then the function of the processor 103 is replaced by the server.

图2为车辆在道路上行驶的一种场景。其中，车辆100在所处的道路的内侧车道行驶，在车辆100同向行驶的道路上并无其他车辆，在车辆100行驶方向的道路的左前方有行人正欲穿行道路。下面本申请实施例以图2所示的车辆100所处的场景为例来讲述本申请的技术方案实现过程。Figure 2 is a scene of a vehicle driving on a road. The vehicle 100 is driving in the inner lane of the road where the vehicle 100 is traveling, there are no other vehicles on the road where the vehicle 100 is traveling in the same direction, and a pedestrian is about to cross the road on the left front of the road where the vehicle 100 is traveling. The following embodiments of the present application describe the implementation process of the technical solution of the present application by taking the scene where the vehicle 100 shown in FIG. 2 is located as an example.

图3为本申请实施例提供的一种行人穿行意图估计方法的过程流程图。如图3所示，处理器103具体实现过程如下：FIG. 3 is a process flow chart of a method for estimating a pedestrian's passing intention provided by an embodiment of the present application. As shown in FIG. 3 , the specific implementation process of the processor 103 is as follows:

步骤S301，处理器103获取车辆100周围至少一个行人的状态信息。本申请中，提取的行人的状态信息主要包括行人的行为特征、行人的静态交互特征和行人的动态交互特征这三类特征。Step S301 , the processor 103 acquires the status information of at least one pedestrian around the vehicle 100 . In the present application, the extracted state information of pedestrians mainly includes three types of features: behavioral features of pedestrians, static interactive features of pedestrians, and dynamic interactive features of pedestrians.

行人的行为特征表示车辆100观测到的行人细节动作，包括行人的运动状态、身体朝向、身体语言等等。示例性地，车辆100的摄像头获取到视频后，由(处理器100中的)视觉感知模块区域多人姿态估计(regional multi-person pose estimation，RMPE)等算法，先对视频中行人进行检测，得到各个行人的边界框，然后在每个边界框中检测人体关键点，并将每个边界框中各个关键点连接成每个行人的姿态。同时，处理器103根据单目测距、双目测距等算法，计算出每个行人距车辆100的距离。处理器103根据行人关键点和姿态，计算出行人相对车辆100的朝向，进而计算行人相对道路的方向，作为行人的朝向特征；处理器103利用行人关键点和姿态，推断行人的运动特性，将行人的运动或静止信息作为行人的运动特征；处理器103分析行人手部关键点，获取行人身体语言，如挥手、指向等动作，作为行人的身体语言特征。上述三个特征反应了行人的细节信息，具有较强的实时性，用于判断行人的表层意图。The behavior feature of the pedestrian represents the detailed action of the pedestrian observed by the vehicle 100 , including the pedestrian's motion state, body orientation, body language, and the like. Exemplarily, after the camera of the vehicle 100 obtains the video, the visual perception module (in the processor 100) uses algorithms such as regional multi-person pose estimation (RMPE) to first detect pedestrians in the video, The bounding box of each pedestrian is obtained, then human key points are detected in each bounding box, and each key point in each bounding box is connected into the pose of each pedestrian. At the same time, the processor 103 calculates the distance between each pedestrian and the vehicle 100 according to algorithms such as monocular ranging and binocular ranging. The processor 103 calculates the pedestrian's orientation relative to the vehicle 100 according to the pedestrian's key points and posture, and then calculates the pedestrian's relative direction to the road as the pedestrian's orientation feature; the processor 103 uses the pedestrian's key points and posture to infer the pedestrian's motion characteristics, The motion or static information of the pedestrian is used as the motion feature of the pedestrian; the processor 103 analyzes the key points of the pedestrian's hand, and obtains the pedestrian's body language, such as waving, pointing and other actions, as the body language feature of the pedestrian. The above three features reflect the detailed information of pedestrians, have strong real-time performance, and are used to judge the surface intention of pedestrians.

行人的静态交互特征表示行人与周围道路环境的相关信息。示例性地，车辆100的处理器103可以通过雷达(以毫米波雷达为例)收发毫米波信号、摄像头拍摄的图像等方式获取行人距离车辆100的距离，然后结合存储器102中存储的地图信息判断行人所处位置，确定行人与道路边界之间的距离、行人与车辆100所处车道之间的距离、行人与道路中每个车道线之间的距离等信息，然后将这些信息作为行人的静态交互特征。The static interaction features of pedestrians represent information about pedestrians and the surrounding road environment. Exemplarily, the processor 103 of the vehicle 100 can obtain the distance between the pedestrian and the vehicle 100 by sending and receiving millimeter-wave signals through radar (taking a millimeter-wave radar as an example), images captured by a camera, etc. The position of the pedestrian, determine the distance between the pedestrian and the road boundary, the distance between the pedestrian and the lane where the vehicle 100 is located, the distance between the pedestrian and each lane line in the road, and other information, and then use these information as the pedestrian's static interactive features.

行人的动态交互特征表示行人与周围动态目标，如与车辆100的交互信息。示例性地，车辆100的处理器102通过摄像头、雷达等器件获取周围的行人相对于车辆100的距离、速度、相对位置等信息，以及其它车辆相对于车辆100的距离、速度、相对位置等信息。然后预测行人和其它车辆在一段时间内的运动轨迹。最后，处理器103根据行人和其它车辆的运动轨迹，计算出各个行人与各个车辆之间最小距离、发生碰撞的时间等信息，然后将这些信息作为行人的动态交互特征。The dynamic interaction feature of the pedestrian represents the interaction information between the pedestrian and surrounding dynamic objects, such as the vehicle 100 . Exemplarily, the processor 102 of the vehicle 100 obtains information such as the distance, speed, and relative position of surrounding pedestrians relative to the vehicle 100 , and information such as the distance, speed, and relative position of other vehicles relative to the vehicle 100 through devices such as cameras and radars. . It then predicts the trajectories of pedestrians and other vehicles over a period of time. Finally, the processor 103 calculates information such as the minimum distance between each pedestrian and each vehicle, the collision time and other information according to the motion trajectories of pedestrians and other vehicles, and then uses these information as dynamic interaction characteristics of pedestrians.

步骤S304，处理器103根据当前时刻的第一行人的状态信息和上一时刻的第一行人的深层意图，计算第一行人穿行道路的表层意图。其中，第一行人即为图2中车辆100行驶方向的道路的左前方的行人，当前时刻也即对应上述的第二时刻，上一时刻也即对应上述的第一时刻。Step S304, the processor 103 calculates the surface intention of the first pedestrian crossing the road according to the state information of the first pedestrian at the current moment and the deep intention of the first pedestrian at the previous moment. The first pedestrian is the pedestrian on the left front of the road in the driving direction of the vehicle 100 in FIG. 2 , the current time corresponds to the above-mentioned second time, and the previous time corresponds to the above-mentioned first time.

在自动驾驶场景中，车辆100可以通过各种输入设备101得到周围其它车辆和障碍物的运动信息，计算其与行人的交互信息，作为行人穿行意图的判断依据。然而，输入设备101有时候监测的信息并不一定完全正确，可能出现错误或误差，如由于行人目标较小且运动速度慢，输入设备101有时候不一定检测到行人、检测到行人速度不变等误差，这会对行人特征产生较大的影响，因此本申请在得到行人的状态信息后，对各种检测到的信息进行融合滤波，对误差进行一定程度的平滑，使得检测到行人状态信息对应的行人特征更加真实。示例性地，本申请将行人的行为特征、行人的静态交互特征和行人的动态交互特征这三类特征进行组合，通过公式(1)计算出行人特征Z的似然概率，即：In an automatic driving scenario, the vehicle 100 can obtain the motion information of other vehicles and obstacles around it through various input devices 101, and calculate the interaction information between it and the pedestrian, as a basis for judging the pedestrian's passing intention. However, sometimes the information monitored by the input device 101 may not be completely correct, and errors or errors may occur. For example, because the pedestrian target is small and moving at a slow speed, the input device 101 may not necessarily detect the pedestrian, and the detected speed of the pedestrian remains unchanged. This will have a greater impact on pedestrian characteristics. Therefore, after obtaining the pedestrian's state information, this application fuses and filters various detected information, and smoothes the error to a certain extent, so that the pedestrian's state information is detected. The corresponding pedestrian features are more realistic. Exemplarily, the present application combines three types of features: pedestrian behavioral features, pedestrian static interaction features, and pedestrian dynamic interaction features, and calculates the likelihood probability of pedestrian feature Z by formula (1), that is:

其中，M _t表示行人在t时刻的穿行意图，DTC _t表示行人在t时刻的静态交互特征，D ^min _t表示行人在t时刻的动态交互特征，OR _t表示行人在t时刻的行人的朝向，MO _t表示行人在t时刻的行人的运动特征，BL _t表示行人在t时刻的行人的身体语言。各个特征的似然概率通过大量数据样本，进行最大似然估计，计算概率参数。 Among them, M _t represents the pedestrian's walking intention at time t, DTC _t represents the static interaction feature of the pedestrian at time t, D ^min _t represents the dynamic interaction feature of the pedestrian at time t, OR _t represents the orientation of the pedestrian at time t, MO _t represents the pedestrian's motion characteristics at time t, and BL _t represents the pedestrian's body language at time t. The likelihood probability of each feature is estimated by maximum likelihood through a large number of data samples, and the probability parameter is calculated.

本申请在计算当前时刻的行人穿行道路的表层意图过程中，利用动态贝叶斯网络结构，对行人的表层意图和行人的深层意图的计算进行建模，然后分别将行人特征Z、观测到的行人状态信息Y、预测的行人状态信息X作为输入信息，然后按照时间的顺序，根据深层意图和表层意图之间的相互影响，构建出如图4所示的模型。如图4所示，计算t时刻的表层意图M _t过程中，与上一个时刻的表层意图M _t-1、t时刻的行人特征Z _t和t时刻的深层意图D _t有关。 In the process of calculating the surface intention of pedestrians crossing the road at the current moment, the present application uses the dynamic Bayesian network structure to model the calculation of the surface intention of pedestrians and the deep intention of pedestrians. Pedestrian state information Y and predicted pedestrian state information X are used as input information, and then in the order of time, the model shown in Figure 4 is constructed according to the interaction between deep intent and surface intent. As shown in Figure 4, the process of calculating the surface intention M _t at time t is related to the surface intention M _t-1 at the previous time, the pedestrian feature Z _t at time t, and the deep intention D t at time _t .

示例性地，以行人状态信息为行人的位置和速度为例，处理器103计算t时刻的表层意图M _t通过公式(2)计算得到，具体为： Exemplarily, taking the pedestrian's state information as the pedestrian's position and speed as an example, the processor 103 calculates the surface intention M _t at time t and obtains it by formula (2), specifically:

其中，P(M _t|M _t-1)表示表层意图的转移概率，表层意图的转移概率代表了历史表层意图对当前表层意图的影响，起到了意图的平滑的作用；P(M _t|Z _t-1)表示行人特征的似然概率，行人特征的似然概率表示了由观测到的行人行为特征和交互特征，来推理出实时的行人穿行意图；P(D _t，D _t-1，X _t，Y _t)表示深层意图，深层意图表示了行人深层意图对表层意图的影响。 Among them, P(M _t |M _t-1 ) represents the transition probability of the surface intention, and the transition probability of the surface intention represents the influence of the historical surface intention on the current surface intention, which plays the role of smoothing the intention; P(M _t |Z _t-1 ) represents the likelihood probability of pedestrian features, and the likelihood probability of pedestrian features represents the observed pedestrian behavior characteristics and interaction characteristics to infer real-time pedestrian crossing intentions; P(D _t , D _t-1 , X _t , Y _t ) represent deep intentions, and deep intentions represent the influence of pedestrian deep intentions on surface intentions.

如果检测到的行人为初次检测时，此时存储器102中没有存储该行人的历史深层意图，则在存储器102中存储一个深层意图的初始值，比如0.5。在处理器103在计算表层意图时，选择该初始值来计算表层意图，此时计算出的表层意图主要由当前时刻的行人特征确定。If the detected pedestrian is detected for the first time and the historical deep intention of the pedestrian is not stored in the memory 102 at this time, an initial value of the deep intention, such as 0.5, is stored in the memory 102 . When the processor 103 calculates the surface intention, the initial value is selected to calculate the surface intention, and the calculated surface intention at this time is mainly determined by the characteristics of the pedestrian at the current moment.

处理器103在得到表层意图后，判断计算得到的表层意图的概率是否超过设定的阈值，如果大于设定的阈值，处理器103可以通过扬声器播放语音、通过显示屏显示提示信息等方式提醒驾驶员，也可以直接控制车辆100减速慢行、避让穿行的行人等行为，使得车辆100在当前行驶的道路上安全行驶。After obtaining the superficial intention, the processor 103 determines whether the probability of the calculated superficial intention exceeds the set threshold, and if it is greater than the set threshold, the processor 103 can play voice through the speaker, display prompt information through the display screen, etc. to remind the driver The operator can also directly control the vehicle 100 to slow down, avoid pedestrians passing by, and other behaviors, so that the vehicle 100 can drive safely on the current road.

本申请实施例中，通过车辆100上的各个类型的输入设备101不断获取车辆100行驶的道路上的行人、道路的车道线、车道标识、周围其他车辆等信息，然后分析出当前时刻的行人的行为特征、静态交互特征、动态交互特征等信息后，结合存储器102中存储的上一时刻的表层意图和上一时刻的深层意图，可以得到兼具高精度、实时性和准确性的表层意图估计结果。In the embodiment of the present application, various types of input devices 101 on the vehicle 100 are used to continuously acquire information such as pedestrians on the road where the vehicle 100 is traveling, lane lines, lane markings, and other surrounding vehicles, and then analyze the current moment of pedestrians' information. After the behavior features, static interaction features, dynamic interaction features and other information, combined with the surface intent at the last moment and the deep intent at the last moment stored in the memory 102, a surface intent estimation with high precision, real-time and accuracy can be obtained result.

其中，准确性是指利用了多方面因素对行人进行预测，并挖掘行人的深层意图，减小了行人感知中噪声的影响。时效性是指使用细节信息和交互信息等具有实时性的要素作为表层意图判断依据，将跟踪信息等具有滞后性的要素用于对深层意图的调整，在保证预测准确性的前提下提高了的时效性。Among them, the accuracy refers to the use of various factors to predict pedestrians, and to mine the deep intentions of pedestrians, reducing the impact of noise in pedestrian perception. Timeliness refers to using real-time elements such as detailed information and interactive information as the basis for judging surface intentions, and using hysteresis elements such as tracking information to adjust deep intentions, which improves the accuracy of predictions. Timeliness.

本申请实施例中，处理器103在实现步骤S304中计算当前时刻的表层意图之前，还实现如下过程：In this embodiment of the present application, the processor 103 further implements the following process before calculating the surface intent at the current moment in step S304:

步骤S302，处理器103根据上一时刻预测的行人状态信息与上一时刻行人的表层意图，确定当前时刻预测的行人状态信息。Step S302, the processor 103 determines the pedestrian state information predicted at the current time according to the pedestrian state information predicted at the previous time and the pedestrian's surface intention at the previous time.

具体地，处理器103先获取上一时刻预测的行人状态信息中的速度和位置后，然后根据上一时刻的表层意图，确定行人是否穿行道路，然后处理器103根据上一时刻预测的行人的速度和位置，以及上一时刻与当前时刻之间的时间差，计算出当前时刻预测行人的速度和位置，得到当前时刻的行人状态信息。Specifically, the processor 103 first obtains the speed and position of the pedestrian state information predicted at the last moment, and then determines whether the pedestrian is crossing the road according to the surface intention at the last moment, and then the processor 103 determines whether the pedestrian is crossing the road according to the predicted pedestrian's speed at the last moment. Speed and position, as well as the time difference between the previous moment and the current moment, calculate the speed and position of the predicted pedestrian at the current moment, and obtain the pedestrian status information at the current moment.

步骤S303，处理器103根据当前时刻预测的行人状态信息、当前时刻的观测到的行人状态信息和上一时刻的行人的深层意图，计算当前时刻的深层意图。Step S303, the processor 103 calculates the deep intention at the current moment according to the predicted pedestrian state information at the current moment, the observed pedestrian status information at the current moment, and the deep intention of the pedestrian at the previous moment.

其中，本申请所指深层意图为行人在车辆100观测到的生命周期内，行人是否真实具有穿行意图，该意图为长期稳定的意图。实际情况下，行人的深层意图为确定且不随时间改变的，而在意图估计过程中，由于深层意图的不可观测性和参考信息的可靠性影响，该节点在估计过程中是渐变的、逐渐收敛的过程，其可靠性随着历史观测信息的累积而增高。Wherein, the deep intention referred to in this application refers to whether the pedestrian actually has a passing intention within the life cycle observed by the vehicle 100 , and the intention is a long-term stable intention. In practice, the deep intention of pedestrians is deterministic and does not change with time, but in the process of intention estimation, due to the unobservability of deep intentions and the reliability of reference information, the node is gradual and convergent in the estimation process. The reliability of the process increases with the accumulation of historical observational information.

具体地，由于深层意图对稳定性和准确性要求较高，对实时性要求较低，本申请主要采用上一时刻行人的预测状态信息、当前时刻行人的实际状态信息、当前时刻的表层意图和上一时刻的行人的深层意图对当前时刻的深层意图进行估计和更新。Specifically, since the deep intent has higher requirements on stability and accuracy, and lower requirements on real-time performance, this application mainly uses the predicted state information of the pedestrian at the previous moment, the actual status information of the pedestrian at the current moment, the surface layer intent at the current moment, and The deep intention of the pedestrian at the previous moment is estimated and updated to the deep intention of the current moment.

示例性地，以行人状态信息为行人的位置和速度为例，如图4所示，处理器103利用匀速模型对行人的运动轨迹进行预测，通过将上一时刻预测的行人位置和速度信息与上一时刻行人的表层意图t输入至匀速模型，计算出当前时刻预测的行人位置和速度信息后，再根据当前时刻观测到的行人的位置和速度信息，通过公式(3)计算深层意图的校正量，即：Exemplarily, taking the pedestrian's state information as the pedestrian's position and speed as an example, as shown in Fig. 4 , the processor 103 uses the uniform speed model to predict the pedestrian's motion trajectory, and by comparing the pedestrian's position and speed information predicted at the previous moment with The surface intention t of the pedestrian at the previous moment is input into the uniform speed model, and the predicted position and speed information of the pedestrian at the current moment are calculated, and then the correction of the deep intention is calculated by formula (3) according to the position and speed information of the pedestrian observed at the current moment. amount, that is:

处理器103根据得到的深层意图的校正量，对上一时刻得到行人的深层意图进行校正，得到当前时刻的深层意图。The processor 103 corrects the deep intention of the pedestrian obtained at the previous moment according to the obtained correction amount of the deep intention, and obtains the deep intention of the current moment.

公式(3)可以以车辆100行驶的方向的道路的车道方向为参考线，将当前时刻的行人的实际位置信息和当前时刻行人的预测位置信息分别为横向和纵向两个方向，此时的计算公式为：Formula (3) can take the lane direction of the road in the direction in which the vehicle 100 travels as a reference line, and take the actual position information of the pedestrian at the current moment and the predicted position information of the pedestrian at the current moment as the horizontal and vertical directions, respectively. At this time, the calculation The formula is:

其中，x表示当前时刻的行人的预测位置，y表示当前时刻的行人的实际位置，v _x表示当前时刻的行人的预测速度，v _y表示当前时刻的行人的实际速度，L表示垂直于车道方向，S表示平行于车道方向。上式(4)表示了行人沿车道方向和垂直车道方向运动与预测运动之差对深层意图的影响。例如，当行人沿垂直车道方向运动距离大于历史意图预测的运动距离，则由此计算的深层意图为穿行的概率大于非穿行的概率，然后结合历史深层意图，可以实现深层意图的矫正，得到当前时刻的深层意图。 Among them, x represents the predicted position of the pedestrian at the current moment, y represents the actual position of the pedestrian at the current moment, v _x represents the predicted speed of the pedestrian at the current moment, v _y represents the actual speed of the pedestrian at the current moment, and L represents the direction perpendicular to the lane , S represents the direction parallel to the lane. Equation (4) above expresses the effect of the difference between the pedestrian's motion along the lane and perpendicular to the lane and the predicted motion on the deep intent. For example, when the pedestrian's movement distance along the vertical lane is greater than the movement distance predicted by the historical intention, the calculated deep intention is that the probability of passing through is greater than the probability of non-travelling. The deep intent of the moment.

另外，本申请实施例中，处理器103在计算出当前时刻行人的表层意图后，如果确定有行人穿行道路时，需要确定车辆100如何避让行人，具体实现过程如下：In addition, in the embodiment of the present application, after the processor 103 calculates the surface intention of the pedestrian at the current moment, if it is determined that there is a pedestrian crossing the road, it needs to determine how the vehicle 100 avoids the pedestrian. The specific implementation process is as follows:

步骤S305，处理器103根据当前时刻的行人的状态信息和当前时刻的表层意图，计算车辆100所处道路的各个车道的车道意图。Step S305 , the processor 103 calculates the lane intention of each lane of the road where the vehicle 100 is located according to the current state information of the pedestrian and the surface intention at the current time.

具体地，处理器103根据当前时刻的表层意图，确定该表层意图的概率大于设定的阈值时，则表明有行人即将穿行道路，继续根据行人状态信息中的行人位置、行人速度、行人距各个车道之间的距离、行人与车辆100之间的距离等信息，计算车辆100在本车道与行人发生相撞的概率、车辆100转向到其它车道上与行人发生相撞的概率，从而计算出车辆100所处道路的各个车道的车道意图。Specifically, when the processor 103 determines, according to the surface intention at the current moment, that the probability of the surface intention is greater than the set threshold, it indicates that a pedestrian is about to cross the road, and continues to use the pedestrian position, pedestrian speed, and pedestrian distance in the pedestrian status information. The distance between the lanes, the distance between the pedestrian and the vehicle 100 and other information, calculate the probability of the vehicle 100 colliding with the pedestrian in this lane, and the probability of the vehicle 100 turning to other lanes to collide with the pedestrian, so as to calculate the vehicle 100. Lane intent for each lane of the road where 100 is located.

示例性地，如图5所示，处理器103获取到行人的速度为v1、车辆100的速度v2、行人与车辆100之间竖直距离为L1、车道宽度为L2、行人距车辆100所处道路的边界距离为L3，然后判断行人到达车辆100所处道路边界的时间t1＝L3/v1是否大于t2＝L1/v2，如果大于，并结合行人运动、朝向、深层意图等其他要素计算得到的车辆100当前所处车道的车道意图小于设定阈值时，，则车辆100在当前所处车道上行驶是安全的；如果小于，并结合行人运动、朝向、深层意图等其他要素计算得到的车辆100当前所处车道的车道意图大于设定阈值时，则车辆100在当前所处车道上行驶不安全。然后判断行人到达车辆100所处车道相邻的车道边界的时间t3＝(L3+L2)/v1是否大于t2(为了方便计算，忽略车辆100从当前车道转向相邻车道的时间)，如果大于，并结合行人运动、朝向、深层意图等其他要素计算得到的车辆100当前所处车道的车道意图小于设定阈值时，则车辆100转向相邻车道上行驶是安全的；如果小于，并结合行人运动、朝向、深层意图等其他要素计算得到的车辆100当前所处车道的车道意图大于设定阈值时，则车辆100在相邻车道上行驶也不安全，此时可以控制车辆100减速慢行、或刹车停下。Exemplarily, as shown in FIG. 5 , the processor 103 obtains that the speed of the pedestrian is v1, the speed of the vehicle 100 is v2, the vertical distance between the pedestrian and the vehicle 100 is L1, the width of the lane is L2, and the distance between the pedestrian and the vehicle 100 is L1. The boundary distance of the road is L3, and then it is judged whether the time t1=L3/v1 when the pedestrian reaches the road boundary where the vehicle 100 is located is greater than t2=L1/v2, if it is greater, and combined with other factors such as pedestrian movement, orientation, and deep intention to calculate When the lane intention of the lane where the vehicle 100 is currently located is less than the set threshold, it is safe for the vehicle 100 to drive on the current lane; if it is smaller than the vehicle 100 calculated in combination with other factors such as pedestrian movement, orientation, and deep intention When the lane intention of the currently located lane is greater than the set threshold, it is unsafe for the vehicle 100 to drive in the currently located lane. Then judge whether the time t3=(L3+L2)/v1 for the pedestrian to reach the boundary of the lane adjacent to the lane where the vehicle 100 is located is greater than t2 (for the convenience of calculation, ignore the time for the vehicle 100 to turn from the current lane to the adjacent lane), if it is greater than, When the lane intention of the lane where the vehicle 100 is currently located, calculated in combination with other factors such as pedestrian movement, orientation, and deep intention, is less than the set threshold, it is safe for the vehicle 100 to turn to the adjacent lane; When the lane intention of the current lane of the vehicle 100 calculated from other elements such as , orientation, and deep intention is greater than the set threshold, it is unsafe for the vehicle 100 to drive in the adjacent lane. At this time, the vehicle 100 can be controlled to slow down, or Brake to stop.

本申请通过利用各车道的道路结构和相关车辆提取车道级的静态交互特征和动态交互特征，结合目标行人的行为特征，推理行人的车道意图，让车辆100能够输出行人相对各车道的穿行意图，使自动驾驶车辆提前做出应对。In the present application, the lane-level static interaction features and dynamic interaction features are extracted by using the road structure of each lane and related vehicles, combined with the behavioral features of the target pedestrians to infer the pedestrian's lane intention, so that the vehicle 100 can output the pedestrian's passing intention relative to each lane. Enabling autonomous vehicles to respond in advance.

图6为本申请实施例提供的一种行人穿行意图估计装置的结构示意图。如图6所示，该装置600包括获取单元601和处理单元602。FIG. 6 is a schematic structural diagram of an apparatus for estimating a pedestrian's passing intention according to an embodiment of the present application. As shown in FIG. 6 , the apparatus 600 includes an acquisition unit 601 and a processing unit 602 .

获取单元601用于获取车辆周围至少一个行人的状态信息，所述至少一个行人包括第一行人；The obtaining unit 601 is used to obtain the status information of at least one pedestrian around the vehicle, and the at least one pedestrian includes the first pedestrian;

处理单元602用于根据第一时刻的第一行人的状态信息和第二时刻的第一行人的深层意图，计算第一时刻的第一行人的表层意图，所述第二时刻在所述第一时刻之前，所述深层意图为所述第一行人在整个生命周期内穿行道路的概率，所述表层意图为所述第一行人在当前时刻正在或即将穿行道路的概率；The processing unit 602 is configured to calculate the surface intention of the first pedestrian at the first moment according to the state information of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment, where the second moment is at Before the first moment, the deep intention is the probability of the first pedestrian crossing the road in the entire life cycle, and the surface intention is the probability that the first pedestrian is or will cross the road at the current moment;

处理单元602还用于根据所述第一时刻的第一行人的表层意图，确定所述第一行人在第一时刻是否穿行道路。The processing unit 602 is further configured to determine whether the first pedestrian crosses the road at the first moment according to the surface intention of the first pedestrian at the first moment.

在一种实施方式中，处理单元602具体用于根据所述第一时刻的第一行人的状态信息，计算第一时刻的第一行人的行人特征的似然概率；根据所述第一时刻的第一行人的行人特征的似然概率和所述第二时刻的第一行人的深层意图，计算所述第一时刻的第一行人的表层意图。In an embodiment, the processing unit 602 is specifically configured to calculate the likelihood probability of the pedestrian feature of the first pedestrian at the first moment according to the state information of the first pedestrian at the first moment; The likelihood probability of the pedestrian feature of the first pedestrian at the moment and the deep intention of the first pedestrian at the second moment are used to calculate the surface intention of the first pedestrian at the first moment.

在一种实施方式中，处理单元602还用于当所述第一时刻的第一行人的表层意图大于设定阈值时，控制扬声器播放预警信号和/或在显示屏上显示预警信息。In one embodiment, the processing unit 602 is further configured to control the speaker to play a warning signal and/or display warning information on the display screen when the surface intention of the first pedestrian at the first moment is greater than a set threshold.

在一种实施方式中，处理单元602还用于根据第二时刻的第一行人的状态信息和第二时刻的第一行人的表层意图，计算第一时刻的预测第一行人的状态信息；根据所述第一时刻的预测第一行人的状态信息、所述第一时刻的第一行人的状态信息和第二时刻的第一行人的深层意图，计算第一时刻的第一行人的深层意图。In an embodiment, the processing unit 602 is further configured to calculate the predicted state of the first pedestrian at the first moment according to the state information of the first pedestrian at the second moment and the surface intention of the first pedestrian at the second moment information; according to the predicted state information of the first pedestrian at the first moment, the state information of the first pedestrian at the first moment, and the deep intention of the first pedestrian at the second moment, calculate the first pedestrian at the first moment. The deep intentions of the group.

在一种实施方式中，处理单元602还用于根据所述第一时刻的第一行人的状态信息和所述第一时刻的第一行人的表层意图，计算车辆所处道路的至少一个车道的车道意图，所述车道意图为所述第一行人穿行第一车道的概率，所述至少一个车道包括所述第一车道。In an embodiment, the processing unit 602 is further configured to calculate at least one road on which the vehicle is located according to the state information of the first pedestrian at the first moment and the surface intention of the first pedestrian at the first moment Lane intent of the lane, where the lane intent is the probability that the first pedestrian traverses the first lane, and the at least one lane includes the first lane.

本发明提供一种计算机可读存储介质，其上存储有计算机程序，当所述计算机程序在计算机中执行时，令计算机执行上述任一项方法。The present invention provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed in a computer, the computer is made to execute any one of the above methods.

本发明提供一种计算设备，包括存储器和处理器，所述存储器中存储有可执行代码，所述处理器执行所述可执行代码时，实现上述任一项方法。The present invention provides a computing device, including a memory and a processor, wherein executable codes are stored in the memory, and when the processor executes the executable codes, any one of the above methods is implemented.

本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请实施例的范围。Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Experts may use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of the embodiments of the present application.

此外，本申请实施例的各个方面或特征可以实现成方法、装置或使用标准编程和/或工程技术的制品。本申请中使用的术语“制品”涵盖可从任何计算机可读器件、载体或介质访问的计算机程序。例如，计算机可读介质可以包括，但不限于：磁存储器件(例如，硬盘、软盘或磁带等)，光盘(例如，压缩盘(compact disc，CD)、数字通用盘(digital versatile disc，DVD)等)，智能卡和闪存器件(例如，可擦写可编程只读存储器(erasable programmable read-only memory，EPROM)、卡、棒或钥匙驱动器等)。另外，本文描述的各种存储介质可代表用于存储信息的一个或多个设备和/或其它机器可读介质。术语“机器可读介质”可包括但不限于，无线信道和能够存储、包含和/或承载指令和/或数据的各种其它介质。Furthermore, various aspects or features of the embodiments of the present application may be implemented as a method, apparatus or article of manufacture using standard programming and/or engineering techniques. The term "article of manufacture" as used in this application encompasses a computer program accessible from any computer readable device, carrier or medium. For example, computer readable media may include, but are not limited to, magnetic storage devices (eg, hard disks, floppy disks, or magnetic tapes, etc.), optical disks (eg, compact discs (CDs), digital versatile discs (DVDs) etc.), smart cards and flash memory devices (eg, erasable programmable read-only memory (EPROM), card, stick or key drives, etc.). Additionally, various storage media described herein can represent one or more devices and/or other machine-readable media for storing information. The term "machine-readable medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and/or carrying instructions and/or data.

在上述实施例中，图6中行人穿行意图估计装置600可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘Solid State Disk(SSD))等。In the above embodiments, the pedestrian crossing intention estimation apparatus 600 in FIG. 6 may be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented in software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer may be a general purpose computer, special purpose computer, computer network, or other programmable device. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be downloaded from a website site, computer, server, or data center Transmission to another website site, computer, server, or data center is by wire (eg, coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes an integration of one or more available media. The usable media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, DVD), or semiconductor media (eg, Solid State Disk (SSD)), and the like.

应当理解的是，在本申请实施例的各种实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。It should be understood that, in various embodiments of the embodiments of the present application, the size of the sequence numbers of the above-mentioned processes does not mean the sequence of execution, and the execution sequence of each process should be determined by its functions and internal logic, and should not be The implementation process of the embodiments of the present application constitutes any limitation.

所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的***、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described systems, devices and units may refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

在本申请所提供的几个实施例中，应该理解到，所揭露的***、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个***，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者接入网设备等)执行本申请实施例各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。The functions, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application can be embodied in the form of software products in essence, or the parts that make contributions to the prior art or the parts of the technical solutions, and the computer software products are stored in a storage medium , including several instructions to cause a computer device (which may be a personal computer, a server, or an access network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the embodiments of this application. The aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .

以上所述，仅为本申请实施例的具体实施方式，但本申请实施例的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请实施例揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请实施例的保护范围之内。The above are only specific implementations of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Changes or substitutions that are thought of should be covered within the protection scope of the embodiments of the present application.

Claims

一种行人穿行意图估计方法，其特征在于，包括：A method for estimating pedestrian crossing intention, comprising:

获取车辆周围至少一个行人的状态信息，所述至少一个行人包括第一行人；acquiring status information of at least one pedestrian around the vehicle, where the at least one pedestrian includes the first pedestrian;

根据第一时刻的第一行人的状态信息和第二时刻的第一行人的深层意图，计算第一时刻的第一行人的表层意图，所述第二时刻在所述第一时刻之前，所述深层意图为所述第一行人在整个生命周期内穿行道路的概率，所述表层意图为所述第一行人在当前时刻正在或即将穿行道路的概率；According to the state information of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment, the surface intention of the first pedestrian at the first moment is calculated, and the second moment is before the first moment , the deep intention is the probability of the first pedestrian crossing the road in the entire life cycle, and the surface intention is the probability that the first pedestrian is or is about to cross the road at the current moment;

根据所述第一时刻的第一行人的表层意图，确定所述第一行人在第一时刻是否穿行道路。According to the surface intention of the first pedestrian at the first moment, it is determined whether the first pedestrian is crossing the road at the first moment.
根据权利要求1所述的方法，其特征在于，在所述根据第一时刻的第一行人的状态信息和第二时刻的第一行人的深层意图，计算第一时刻的第一行人的表层意图之前，包括：The method according to claim 1, wherein, calculating the first pedestrian at the first moment according to the state information of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment Before the surface intent, include:

根据所述第一时刻的第一行人的状态信息，计算第一时刻的第一行人的行人特征的似然概率；According to the state information of the first pedestrian at the first moment, the likelihood probability of the pedestrian feature of the first pedestrian at the first moment is calculated;

所述根据第一时刻的第一行人的状态信息和第二时刻的第一行人的深层意图，计算第一时刻的第一行人的表层意图，包括：The calculation of the surface intention of the first pedestrian at the first moment according to the state information of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment includes:

根据所述第一时刻的第一行人的行人特征的似然概率和所述第二时刻的第一行人的深层意图，计算所述第一时刻的第一行人的表层意图。According to the likelihood probability of the pedestrian feature of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment, the surface intention of the first pedestrian at the first moment is calculated.
根据权利要求1-2任意一项所述的方法，其特征在于，所述方法还包括：The method according to any one of claims 1-2, wherein the method further comprises:

当所述第一时刻的第一行人的表层意图大于设定阈值时，控制扬声器播放预警信号和/或在显示屏上显示预警信息。When the surface intention of the first pedestrian at the first moment is greater than the set threshold, the speaker is controlled to play an early warning signal and/or the warning information is displayed on the display screen.
根据权利要求1-3任意一项所述的方法，其特征在于，在所述根据第一时刻的第一行人的状态信息和第二时刻的第一行人的深层意图，计算第一时刻的第一行人的表层意图之前，所述方法包括：The method according to any one of claims 1-3, characterized in that, in the calculation of the first moment according to the state information of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment Before the surface intent of the first pedestrian, the method includes:

根据第二时刻的第一行人的状态信息和第二时刻的第一行人的表层意图，计算第一时刻的预测第一行人的状态信息；According to the state information of the first pedestrian at the second time and the surface intention of the first pedestrian at the second time, calculate the state information of the predicted first pedestrian at the first time;

根据所述第一时刻的预测第一行人的状态信息、所述第一时刻的第一行人的状态信息和第二时刻的第一行人的深层意图，计算第一时刻的第一行人的深层意图。Calculate the first row at the first moment according to the state information of the predicted first pedestrian at the first moment, the state information of the first pedestrian at the first moment, and the deep intention of the first pedestrian at the second moment deep intentions of man.
根据权利要求1-4任意一项所述的方法，其特征在于，所述方法还包括：The method according to any one of claims 1-4, wherein the method further comprises:

根据所述第一时刻的第一行人的状态信息和所述第一时刻的第一行人的表层意图，计算车辆所处道路的至少一个车道的车道意图，所述车道意图为所述第一行人穿行第一车道的概率，所述至少一个车道包括所述第一车道。According to the state information of the first pedestrian at the first moment and the surface intention of the first pedestrian at the first moment, calculate the lane intention of at least one lane of the road where the vehicle is located, where the lane intention is the first lane intention The probability that a pedestrian traverses a first lane, the at least one lane including the first lane.
一种行人穿行意图估计装置，其特征在于，包括：A device for estimating pedestrian's passing intention, comprising:

获取单元，用于获取车辆周围至少一个行人的状态信息，所述至少一个行人包括第一行人；an acquisition unit, configured to acquire status information of at least one pedestrian around the vehicle, where the at least one pedestrian includes the first pedestrian;

处理单元，用于根据第一时刻的第一行人的状态信息和第二时刻的第一行人的深层意图，计算第一时刻的第一行人的表层意图，所述第二时刻在所述第一时刻之前，所述深层意图为所述第一行人在整个生命周期内穿行道路的概率，所述表层意图为所述第一行人在当前时刻正在或即将穿行道路的概率；The processing unit is configured to calculate the surface intention of the first pedestrian at the first moment according to the state information of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment, where the second moment is at Before the first moment, the deep intention is the probability of the first pedestrian crossing the road in the entire life cycle, and the surface intention is the probability that the first pedestrian is or will cross the road at the current moment;

所述处理单元，还用于根据所述第一时刻的第一行人的表层意图，确定所述第一行人在第一时刻是否穿行道路。The processing unit is further configured to determine whether the first pedestrian crosses the road at the first moment according to the surface intention of the first pedestrian at the first moment.
根据权利要求6所述的装置，其特征在于，所述处理单元，具体用于根据所述第一时刻的第一行人的状态信息，计算第一时刻的第一行人的行人特征的似然概率；The device according to claim 6, wherein the processing unit is specifically configured to calculate the similarity of the pedestrian characteristics of the first pedestrian at the first moment according to the state information of the first pedestrian at the first moment probability;

根据所述第一时刻的第一行人的行人特征的似然概率和所述第二时刻的第一行人的深层意图，计算所述第一时刻的第一行人的表层意图。According to the likelihood probability of the pedestrian feature of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment, the surface intention of the first pedestrian at the first moment is calculated.
根据权利要求6-7任意一项所述的装置，其特征在于，所述处理单元，还用于当所述第一时刻的第一行人的表层意图大于设定阈值时，控制扬声器播放预警信号和/或在显示屏上显示预警信息。The device according to any one of claims 6-7, wherein the processing unit is further configured to control the speaker to play an early warning when the surface intention of the first pedestrian at the first moment is greater than a set threshold signal and/or show warning messages on the display.
根据权利要求6-8任意一项所述的装置，其特征在于，所述处理单元，还用于根据第二时刻的第一行人的状态信息和第二时刻的第一行人的表层意图，计算第一时刻的预测第一行人的状态信息；The device according to any one of claims 6-8, wherein the processing unit is further configured to use the state information of the first pedestrian at the second moment and the surface intention of the first pedestrian at the second moment , calculate the state information of the predicted first pedestrian at the first moment;

根据所述第一时刻的预测第一行人的状态信息、所述第一时刻的第一行人的状态信息和第二时刻的第一行人的深层意图，计算第一时刻的第一行人的深层意图。Calculate the first row at the first moment according to the state information of the predicted first pedestrian at the first moment, the state information of the first pedestrian at the first moment, and the deep intention of the first pedestrian at the second moment deep intentions of man.
根据权利要求6-9任意一项所述的装置，其特征在于，所述处理单元，还用于根据所述第一时刻的第一行人的状态信息和所述第一时刻的第一行人的表层意图，计算车辆所处道路的至少一个车道的车道意图，所述车道意图为所述第一行人穿行第一车道的概率，所述至少一个车道包括所述第一车道。The device according to any one of claims 6-9, characterized in that the processing unit is further configured to: according to the state information of the first pedestrian at the first moment and the first row at the first moment The surface intention of the person is to calculate the lane intention of at least one lane of the road where the vehicle is located, where the lane intention is the probability of the first pedestrian crossing the first lane, and the at least one lane includes the first lane.
一种设备，包括至少一个处理器，所述处理器用于执行存储器中存储的指令，以使得终端执行如权利要求1-5任一所述的方法。A device comprising at least one processor for executing instructions stored in a memory to cause a terminal to execute the method according to any one of claims 1-5.
一种汽车，用于执行如权利要求1-5中的任一项所述的方法。A vehicle for carrying out the method of any one of claims 1-5.
一种计算机可读存储介质，其上存储有计算机程序，当所述计算机程序在计算机中执行时，令计算机执行权利要求1-5中任一项的所述的方法。A computer-readable storage medium on which a computer program is stored, when the computer program is executed in a computer, the computer is made to perform the method of any one of claims 1-5.
一种计算设备，包括存储器和处理器，其特征在于，所述存储器中存储有可执行代码，所述处理器执行所述可执行代码时，实现权利要求1-5中任一项所述的方法。A computing device, comprising a memory and a processor, wherein executable code is stored in the memory, and when the processor executes the executable code, the processor of any one of claims 1-5 is implemented. method.