WO2020042984A1

WO2020042984A1 - Vehicle behavior detection method and apparatus

Info

Publication number: WO2020042984A1
Application number: PCT/CN2019/101807
Authority: WO
Inventors: 虞抒沁
Original assignee: 杭州海康威视数字技术股份有限公司
Priority date: 2018-08-28
Filing date: 2019-08-21
Publication date: 2020-03-05
Also published as: CN110866427A

Abstract

Disclosed are a vehicle behavior detection method and apparatus. The vehicle behavior detection method comprises: a vehicle-mounted intelligent device collects a video frame, recognizes a vehicle behavior according to the video frame so as to obtain a recognition result that the vehicle behavior is abnormal, and uploads the recognition result to a server, so that the server performs vehicle behavior alarm according to the recognition result. The present application can effectively recognize various abnormal vehicle behaviors and improves urban traffic efficiency.

Description

一种车辆行为检测方法及装置Method and device for detecting vehicle behavior

本申请要求于2018年08月28日提交中国专利局、申请号为201810986173.3，发明名称为“一种车辆行为检测方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims priority from a Chinese patent application filed with the Chinese Patent Office on August 28, 2018, with application number 201810986173.3, and the invention name is "A Vehicle Behavior Detection Method and Device", the entire contents of which are incorporated herein by reference. in.

技术领域Technical field

本申请涉及机器视觉领域，尤其涉及一种车辆行为的检测方法及装置。The present application relates to the field of machine vision, and in particular, to a method and device for detecting vehicle behavior.

背景技术Background technique

随着社会的不断进步，视频分析的应用范围越来越广。相关的数字监控***远远不能满足于许多应用场合的需要，主要体现在智能化程度不够高以及应用不够广泛。以交通事故检测领域为例，一方面，目前仅在各个卡口、路口配备监控进行简单的违章取证，不仅监控盲点很大，而且在任何地方发生事故，依然需要交警到达现场进行事故和责任鉴定，事故车辆占用行车道等待交警到达现场的时间段内会严重影响交通；另一方面，基于深度学习技术的行为识别近年来已经开始应用于一些基于人的动作识别，但是对于车辆的事故或是违章识别却没有被关注。With the continuous progress of society, the scope of application of video analysis is getting wider and wider. The related digital monitoring system is far from meeting the needs of many applications, which is mainly reflected in the lack of intelligence and the widespread application. Taking the field of traffic accident detection as an example, on the one hand, at present, monitoring is only provided at various checkpoints and intersections for simple violations of evidence. Not only is there a large blind spot in monitoring, but an accident occurs anywhere, and traffic police still need to arrive at the scene for accident and liability identification. In the time period when the accident vehicle occupies the lane and waits for the traffic police to reach the scene, traffic will be seriously affected; on the other hand, behavior recognition based on deep learning technology has begun to be applied to some human-based motion recognition in recent years, but for vehicle accidents or Illegal identification was not noticed.

目前，已有类似技术的专利申请，例如US20180121731A1，其公开了一种采用3D卷积网络对图像进行合成，采用LSTM网络实现车身图像分析和判断的方法，然而，该方法是采用外界的摄像头采集车辆等所包括的图形或者文字，识别出该图形或文字，并未公开可以采用车载终端的视频检测装置识别交通违章或者事故；再例如CN108133172A，公开了一种采用机器学习的方法采集目标车辆信息，并对视频图像和运动轨迹进行分析，从而判断出交通违章以及事故，然而，该方法未公开可以采用车载终端的视频检测装置来进行图像采集和传输。At present, there are patent applications for similar technologies, such as US20180121731A1, which discloses a method for synthesizing images using a 3D convolutional network and using the LSTM network to realize body image analysis and judgment. However, this method uses external camera acquisition The graphics or text included in the vehicle, etc., identify the graphics or text, it is not disclosed that the vehicle terminal's video detection device can be used to identify traffic violations or accidents; for example, CN108133172A, discloses a method for collecting target vehicle information using machine learning The video images and motion trajectories are analyzed to determine traffic violations and accidents. However, this method does not disclose that the video detection device of the vehicle terminal can be used to perform image acquisition and transmission.

因此，在基于视频分析的车辆行为检测领域，如何及时有效的识别各种违章驾驶、交通事故等异常车辆行为，从而提升城市交通效率，仍属于亟待解决和改进的问题。Therefore, in the field of vehicle behavior detection based on video analysis, how to timely and effectively identify various illegal vehicle behaviors such as illegal driving and traffic accidents to improve urban traffic efficiency is still an urgent problem to be solved and improved.

发明内容Summary of the Invention

本申请实施例的目的在于提供一种车辆行为的检测方法及装置，以实现及时有效地识别异常车辆行为，从而提升城市交通效率。具体技术方案如下：The purpose of the embodiments of the present application is to provide a method and a device for detecting vehicle behavior, so as to realize timely and effective identification of abnormal vehicle behavior, thereby improving urban traffic efficiency. Specific technical solutions are as follows:

第一方面，本申请实施例提供了一种车辆行为检测方法，应用于车载智能设备，所述方法包括：In a first aspect, an embodiment of the present application provides a vehicle behavior detection method, which is applied to a vehicle-mounted smart device. The method includes:

采集视频帧；Capture video frames;

基于所述视频帧，对车辆行为进行识别，得到异常车辆行为的识别结果；Identifying vehicle behavior based on the video frame to obtain a recognition result of abnormal vehicle behavior;

将所述识别结果上传至服务器，以使所述服务器根据所述识别结果，进行车辆行为报警。The recognition result is uploaded to a server, so that the server performs a vehicle behavior alarm according to the recognition result.

第二方面，本申请实施例提供了一种车辆行为检测方法，应用于服务器，所述方法包括：In a second aspect, an embodiment of the present application provides a vehicle behavior detection method, which is applied to a server, and the method includes:

接收车载智能设备发送的识别结果，所述识别结果为所述车载智能设备基于采集的视频帧，对车辆行为进行识别，得到的异常车辆行为的识别结果；Receiving a recognition result sent by a vehicle-mounted smart device, where the recognition result is a recognition result of abnormal vehicle behavior obtained by the vehicle-mounted smart device based on the collected video frames;

根据所述识别结果，进行车辆行为报警。According to the recognition result, a vehicle behavior alarm is performed.

第三方面，本申请实施例提供了一种车辆行为检测方法，应用于车载智能设备，所述方法包括：In a third aspect, an embodiment of the present application provides a vehicle behavior detection method, which is applied to a vehicle-mounted smart device, and the method includes:

采集视频帧；Capture video frames;

将所述视频帧上传至服务器，以使所述服务器基于所述视频帧，对车辆行为进行识别，得到异常车辆行为的识别结果，并根据所述识别结果，进行车辆行为报警。Uploading the video frame to a server, so that the server recognizes vehicle behavior based on the video frame, obtains a recognition result of abnormal vehicle behavior, and performs a vehicle behavior alarm according to the recognition result.

第四方面，本申请实施例提供了一种车辆行为检测方法，应用于服务器，所述方法包括：In a fourth aspect, an embodiment of the present application provides a vehicle behavior detection method, which is applied to a server, and the method includes:

接收车载智能设备发送的视频帧；Receiving video frames sent by vehicle smart devices;

第五方面，本申请实施例提供了一种车辆行为检测装置，应用于车载智能设备，所述装置包括：In a fifth aspect, an embodiment of the present application provides a vehicle behavior detection device applied to a vehicle-mounted smart device, where the device includes:

采集模块，用于采集视频帧；An acquisition module for acquiring video frames;

数据处理模块，用于基于所述视频帧，对车辆行为进行识别，得到异常车辆行为的识别结果；A data processing module, configured to identify vehicle behavior based on the video frame, and obtain a recognition result of abnormal vehicle behavior;

数据传输模块，用于将所述识别结果上传至服务器。A data transmission module, configured to upload the recognition result to a server.

第六方面，本申请实施例提供了一种车辆行为检测装置，应用于服务器，所述装置包括：According to a sixth aspect, an embodiment of the present application provides a vehicle behavior detection device, which is applied to a server, and the device includes:

接收模块，用于接收车载智能设备发送的识别结果，所述识别结果为所述车载智能设备基于采集的视频帧，对车辆行为进行识别，得到的异常车辆行为的识别结果；A receiving module, configured to receive a recognition result sent by a vehicle-mounted intelligent device, where the recognition result is a recognition result of abnormal vehicle behavior obtained by the vehicle-mounted intelligent device based on the collected video frames;

综合报警模块，用于根据所述识别结果，进行车辆行为报警。A comprehensive alarm module is configured to perform a vehicle behavior alarm according to the recognition result.

第七方面，本申请实施例提供了一种车辆行为检测装置，应用于车载智能设备，所述装置包括：In a seventh aspect, an embodiment of the present application provides a vehicle behavior detection device, which is applied to a vehicle-mounted smart device, and the device includes:

数据传输模块，用于将所述视频帧上传至服务器。A data transmission module is configured to upload the video frame to a server.

第八方面，本申请实施例提供了一种车辆行为检测装置，应用于服务器，所述装置包括：In an eighth aspect, an embodiment of the present application provides a vehicle behavior detection device, which is applied to a server, and the device includes:

接收模块，用于接收车载智能设备发送的视频帧；A receiving module, configured to receive a video frame sent by a vehicle intelligent device;

第九方面，本申请实施例提供了一种车辆行为检测***，所述***包括：车载智能设备及服务器；In a ninth aspect, an embodiment of the present application provides a vehicle behavior detection system. The system includes: a vehicle-mounted intelligent device and a server;

所述车载智能设备，用于采集视频帧；基于所述视频帧，对车辆行为进行识别，得到异常车辆行为的识别结果；将所述识别结果上传至所述服务器；The in-vehicle intelligent device is used to collect video frames; identify vehicle behaviors based on the video frames to obtain recognition results of abnormal vehicle behaviors; and upload the recognition results to the server;

所述服务器，用于接收所述车载智能设备发送的所述识别结果；根据所述识别结果，进行车辆行为报警。The server is configured to receive the recognition result sent by the in-vehicle smart device; and perform a vehicle behavior alarm according to the recognition result.

第十方面，本申请实施例提供了一种车辆行为检测***，所述***包括：车载智能设备及服务器；In a tenth aspect, an embodiment of the present application provides a vehicle behavior detection system. The system includes: a vehicle-mounted intelligent device and a server;

所述车载智能设备，用于采集视频帧；将所述视频帧上传至所述服务器；The in-vehicle smart device is used to collect video frames; upload the video frames to the server;

所述服务器，用于接收所述车载智能设备发送的所述视频帧；基于所述视频帧，对车辆行为进行识别，得到异常车辆行为的识别结果；根据所述识别结果，进行车辆行为报警。The server is configured to receive the video frame sent by the in-vehicle smart device; identify vehicle behavior based on the video frame to obtain a recognition result of abnormal vehicle behavior; and perform a vehicle behavior alarm based on the recognition result.

第十一方面，本申请实施例提供了一种车载智能设备，包括处理器和机器可读存储介质，所述机器可读存储介质存储有能够被所述处理器执行的机器可执行指令，所述处理器被所述机器可执行指令促使执行本申请实施例第一方面所提供的方法。According to an eleventh aspect, an embodiment of the present application provides a vehicle-mounted smart device, including a processor and a machine-readable storage medium, where the machine-readable storage medium stores machine-executable instructions that can be executed by the processor, and therefore The processor is caused by the machine executable instructions to execute the method provided by the first aspect of the embodiments of the present application.

第十二方面，本申请实施例提供了一种机器可读存储介质，存储有机器可执行指令，在被处理器调用和执行时，所述机器可执行指令促使所述处理器执行本申请实施例第一方面所提供的方法。In a twelfth aspect, an embodiment of the present application provides a machine-readable storage medium that stores machine-executable instructions. When called and executed by a processor, the machine-executable instructions cause the processor to execute the implementation of the present application. The method provided by the first aspect of the example.

第十三方面，本申请实施例提供了一种应用程序，其特征在于，用于在运行时执行：本申请实施例第一方面所提供的方法。In a thirteenth aspect, an embodiment of the present application provides an application program, which is characterized in that it is used to execute at runtime: the method provided in the first aspect of the embodiment of the present application.

第十四方面，本申请实施例提供了一种服务器，其特征在于，包括处理器和机器可读存储介质，所述机器可读存储介质存储有能够被所述处理器执行的机器可执行指令，所述处理器被所述机器可执行指令促使执行本申请实施例第二方面所提供的方法。In a fourteenth aspect, an embodiment of the present application provides a server, including a processor and a machine-readable storage medium, where the machine-readable storage medium stores machine-executable instructions that can be executed by the processor. The processor is caused by the machine executable instructions to execute the method provided by the second aspect of the embodiments of the present application.

第十五方面，本申请实施例提供了一种机器可读存储介质，存储有机器可执行指令，在被处理器调用和执行时，所述机器可执行指令促使所述处理器执行本申请实施例第二方面所提供的方法。In a fifteenth aspect, an embodiment of the present application provides a machine-readable storage medium that stores machine-executable instructions. When called and executed by a processor, the machine-executable instructions cause the processor to execute the implementation of the present application. The method provided by the second aspect of the example.

第十六方面，本申请实施例提供了一种应用程序，用于在运行时执行：本申请实施例第二方面所提供的方法。In a sixteenth aspect, an embodiment of the present application provides an application program for executing at runtime: the method provided in the second aspect of the embodiment of the present application.

第十七方面，本申请实施例提供了一种车载智能设备，包括处理器和机器可读存储介质，所述机器可读存储介质存储有能够被所述处理器执行的机器可执行指令，所述处理器被所述机器可执行指令促使执行本申请实施例第三方面所提供的方法。In a seventeenth aspect, an embodiment of the present application provides a vehicle-mounted smart device, including a processor and a machine-readable storage medium, where the machine-readable storage medium stores machine-executable instructions that can be executed by the processor, and The processor is caused by the machine executable instructions to execute the method provided by the third aspect of the embodiments of the present application.

第十八方面，本申请实施例提供了一种机器可读存储介质，存储有机器可执行指令，在被处理器调用和执行时，所述机器可执行指令促使所述处理器执行本申请实施例第三方面所提供的方法。In an eighteenth aspect, an embodiment of the present application provides a machine-readable storage medium that stores machine-executable instructions. When called and executed by a processor, the machine-executable instructions cause the processor to execute the implementation of the present application. The method provided by the third aspect of the example.

第十九方面，本申请实施例提供了一种应用程序，用于在运行时执行：本申请实施例第三方面所提供的方法。In a nineteenth aspect, an embodiment of the present application provides an application program for executing at runtime: the method provided in the third aspect of the embodiment of the present application.

第二十方面，本申请实施例提供了一种服务器，包括处理器和机器可读存储介质，所述机器可读存储介质存储有能够被所述处理器执行的机器可执行指令，所述处理器被所述机器可执行指令促使执行本申请实施例第四方面所提供的方法。In a twentieth aspect, an embodiment of the present application provides a server, including a processor and a machine-readable storage medium, where the machine-readable storage medium stores machine-executable instructions that can be executed by the processor, and the processing The machine is caused by the machine executable instructions to execute the method provided by the fourth aspect of the embodiments of the present application.

第二十一方面，本申请实施例提供了一种机器可读存储介质，存储有机器可执行指令，在被处理器调用和执行时，所述机器可执行指令促使所述处理器执行本申请实施例第四方面所提供的方法。In a twenty-first aspect, an embodiment of the present application provides a machine-readable storage medium that stores machine-executable instructions. When called and executed by a processor, the machine-executable instructions cause the processor to execute the application. The method provided in the fourth aspect of the embodiment.

第二十二方面，本申请实施例提供了一种应用程序，用于在运行时执行：本申请实施例第四方面所提供的方法。In the twenty-second aspect, the embodiment of the present application provides an application program for executing at runtime: the method provided in the fourth aspect of the embodiment of the present application.

本申请提出如下方案：基于车载智能设备如行车记录仪，或者智能手机进行视频采集，并进行本地处理识别异常车辆行为，将识别结果上传服务器，通过大数据分析进行车辆报警，用以加强道路行驶安全以及加速小微事故处理速度，以辅助提升城市交通效率。This application proposes the following solution: video capture based on vehicle-mounted smart devices such as driving recorders or smartphones, and local processing to identify abnormal vehicle behaviors, upload the recognition results to the server, and perform vehicle alarms through big data analysis to enhance road driving Safety and acceleration of small and micro accidents to help improve urban traffic efficiency.

本申请还提出如下方案：基于车载智能设备如行车记录仪，或者智能手机进行视频采集，将采集的视频帧上传服务器，通过大数据分析识别异常车辆行为，并发出报警信息，用以加强道路行驶安全以及加速小微事故处理速度，以辅助提升城市交通效率。This application also proposes the following solution: video capture is performed based on a vehicle-mounted smart device such as a driving recorder, or a smartphone, and the collected video frames are uploaded to the server, and abnormal vehicle behavior is identified through big data analysis, and an alarm message is issued to enhance road driving Safety and acceleration of small and micro accidents to help improve urban traffic efficiency.

附图说明BRIEF DESCRIPTION OF THE DRAWINGS

为了更清楚地说明本申请实施例和现有技术的技术方案，下面对实施例和现有技术中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to more clearly explain the embodiments of the present application and the technical solutions of the prior art, the following briefly introduces the drawings used in the embodiments and the prior art. Obviously, the drawings in the following description are only the present invention. Some embodiments of the application, for those of ordinary skill in the art, can obtain other drawings according to the drawings without paying creative labor.

图1为根据本申请一种实施方式的车辆行为检测方法流程示意图；1 is a schematic flowchart of a vehicle behavior detection method according to an embodiment of the present application;

图2为根据本申请一种实施方式的应用于车载智能设备的车辆行为检测装置的结构示意图；2 is a schematic structural diagram of a vehicle behavior detection device applied to a vehicle-mounted smart device according to an embodiment of the present application;

图3为根据本申请一种实施方式的应用于服务器的车辆行为检测装置的结构示意图；3 is a schematic structural diagram of a vehicle behavior detection device applied to a server according to an embodiment of the present application;

图4为根据本申请另一种实施方式的车辆行为检测方法流程示意图；4 is a schematic flowchart of a vehicle behavior detection method according to another embodiment of the present application;

图5为根据本申请另一种实施方式的应用于车载智能设备的车辆行为检测装置的结构示意图；5 is a schematic structural diagram of a vehicle behavior detection device applied to a vehicle-mounted smart device according to another embodiment of the present application;

图6为根据本申请另一种实施方式的应用于服务器的车辆行为检测装置的结构示意图；6 is a schematic structural diagram of a vehicle behavior detection device applied to a server according to another embodiment of the present application;

图7为相关技术中CNN编码过程示意图；7 is a schematic diagram of a CNN encoding process in the related art;

图8为根据本申请实施方式的3D CNN编码过程示意图；8 is a schematic diagram of a 3D CNN encoding process according to an embodiment of the present application;

图9为根据本申请一种实施方式的车辆行为检测***的报警过程的示意图；9 is a schematic diagram of an alarm process of a vehicle behavior detection system according to an embodiment of the present application;

图10为根据本申请另一种实施方式的车辆行为检测***的报警过程的示意图；10 is a schematic diagram of an alarm process of a vehicle behavior detection system according to another embodiment of the present application;

图11为根据本申请一种实施方式的车载智能设备的结构示意图；11 is a schematic structural diagram of a vehicle-mounted smart device according to an embodiment of the present application;

图12为根据本申请一种实施方式的服务器的结构示意图；12 is a schematic structural diagram of a server according to an embodiment of the present application;

图13为根据本申请另一种实施方式的车载智能设备的结构示意图；13 is a schematic structural diagram of a vehicle-mounted smart device according to another embodiment of the present application;

图14为根据本申请另一种实施方式的服务器的结构示意图。FIG. 14 is a schematic structural diagram of a server according to another embodiment of the present application.

具体实施方式detailed description

为使本申请的目的、技术方案、及优点更加清楚明白，以下参照附图并举实施例，对本申请进一步详细说明。显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。In order to make the purpose, technical solution, and advantages of the present application clearer and clearer, the following describes the present application in detail with reference to the accompanying drawings and examples. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

根据本申请的一种实施方式，提出一种车辆行为检测方法，如图1所示，包括：According to an embodiment of the present application, a vehicle behavior detection method is provided, as shown in FIG. 1, including:

步骤101：数据采集步骤。Step 101: a data collection step.

车载智能设备可以为行车记录仪、智能手机等具有视频采集功能的设备，车载智能设备往往安装在车辆内部的前端，由车载智能设备进行视频采集。The vehicle-mounted intelligent device may be a device with a video capture function such as a driving recorder, a smart phone, etc. The vehicle-mounted intelligent device is often installed in the front end of the vehicle, and the vehicle-mounted intelligent device performs video collection.

可选的，步骤101可以包括：车载智能设备按照固定时间间隔，采集多个视频帧。Optionally, step 101 may include: the vehicle-mounted smart device collects multiple video frames at a fixed time interval.

示例性地，车载智能设备包括行车记录仪、移动通信设备、或者其他具有数据传输功能的摄像装置。车载智能设备通过其摄像模块采集视频，获取视频文件，所述视频文件包括多个视频帧，摄像模块在进行设备采集时，可以按照固定时间间隔进行采集的多个视频帧。Exemplarily, the in-vehicle intelligent device includes a driving recorder, a mobile communication device, or another camera device having a data transmission function. The vehicle-mounted smart device collects video through its camera module to obtain a video file, where the video file includes multiple video frames. When the camera module performs device collection, the camera module can perform multiple video frames collected at fixed time intervals.

可选的，步骤101可以包括：车载智能设备以其自身所属的车辆为中心，设定检测范围，对检测范围内的一个或多个车辆拍摄得到多个视频帧。Optionally, step 101 may include: the vehicle-mounted smart device sets a detection range around the vehicle to which it belongs, and photographs one or more vehicles within the detection range to obtain multiple video frames.

步骤102：数据处理步骤。Step 102: a data processing step.

该步骤主要是基于视频帧，进行车辆行为识别的过程，该步骤可以在车载智能设备上执行。即，车载智能设备可以直接基于采集的视频帧，对车辆行为进行识别，得到异常车辆行为的识别结果。This step is mainly a process of identifying vehicle behavior based on video frames, and this step can be performed on a vehicle-mounted smart device. That is, the vehicle-mounted smart device can directly recognize the vehicle behavior based on the collected video frames, and obtain the recognition result of the abnormal vehicle behavior.

可选的，步骤102可以包括：车载智能设备将视频帧输入神经网络(neural network)，利用神经网络对视频帧进行特征抽取，生成多维的行为特征向量；对多维的行为特征向量中的每一维采用逻辑分类，得到每一维行为特征向量为不同事件类型的异常车辆行为的置信度；若任一事件类型的异常车辆行为的置信度大于预设阈值，则确定视频帧中存在该事件类型的异常车辆行为。Optionally, step 102 may include: the in-vehicle intelligent device inputs video frames into a neural network, and uses the neural network to perform feature extraction on the video frames to generate multi-dimensional behavior feature vectors; each of the multi-dimensional behavior feature vectors The dimensions are logically classified to obtain the confidence level that the behavior feature vectors of each dimension are abnormal vehicle behaviors of different event types; if the confidence level of abnormal vehicle behaviors of any event type is greater than a preset threshold, it is determined that the event type exists in the video frame Abnormal vehicle behavior.

示例性地，上述异常车辆行为的识别结果，其含义为判断得出视频帧中一个车辆是否存在异常车辆行为，且异常车辆行为的事件类型是否为危险驾驶、交通事故等类型，得出识别结果的判别规则是根据神经网络模型所输出的置信度是否大于预设阈值判断得出。即，如果逻辑分类后得到的是危险驾驶的置信度大于预设阈值，那么判断得出当前视频帧中包括事件类型为危险驾驶的异常车辆行为。如果逻辑分类后得到是交通事故的置信度大于预设阈值，那么得出当前视频帧中包括事件类型为交通事故的异常车辆行为。神经网络模型的训练过程，在此不再赘述。Exemplarily, the above recognition result of abnormal vehicle behavior means that it is determined whether a vehicle in the video frame has abnormal vehicle behavior, and whether the event type of the abnormal vehicle behavior is a type of dangerous driving, traffic accident, etc., and the identification result is obtained. The determination rule is based on whether the confidence level output by the neural network model is greater than a preset threshold. That is, if the confidence level of the dangerous driving is greater than a preset threshold after logical classification, it is determined that the current video frame includes abnormal vehicle behavior whose event type is dangerous driving. If the confidence level of the traffic accident is greater than a preset threshold after logical classification, then the current video frame includes abnormal vehicle behavior with the event type being a traffic accident. The training process of the neural network model is not repeated here.

异常车辆行为的事件类型包括危险驾驶、交通事故等。危险驾驶包括追逐竞驾、醉酒驾驶以及疲劳驾驶，其中，追逐竞驾的判断包括通过前后车位置、距离随时间变化的视频帧的情况，醉酒驾驶的判断包括通过车辆行驶轨迹的变化、司机本体变化的视频帧的情况，例如司机身体躯干的晃动情况等，疲劳驾驶的判断包括通过车辆行驶轨迹的变化、司机本体变化的视频帧的情况，例如司机眼睛的睁开与闭合情况、司机双手位置等；交通事故包括某一类型/形态的交通事故，其中，交通事故在类型上包括追尾事故、超车事故、左转弯事故、有转变事故等，在形态上包括碰撞、刮擦、碾压、翻车、坠车、失火等。The types of incidents of abnormal vehicle behavior include dangerous driving, traffic accidents, and so on. Dangerous driving includes chasing, drunk driving, and fatigue driving. Among them, the judgment of chasing driving includes passing through the video frames of the position and distance of the car in front and rear, and the distance changes with time. The situation of changing video frames, such as the shaking of the torso of the driver ’s body, and the judgment of fatigue driving includes the changes of the vehicle ’s driving trajectory and the changes of the driver ’s body, such as the opening and closing of the driver ’s eyes and the position of the driver ’s hands Traffic accidents include certain types / forms of traffic accidents. Among them, traffic accidents include rear-end accidents, overtaking accidents, left-turn accidents, and transition accidents, and morphologically include collisions, scratches, rolling, and rollovers. , Crash, fire, etc.

示例性地，神经网络包括但不限于深度神经网络(DNN)、卷积神经网络(CNN)、循环神经网络(RNN)、长短期记忆(LSTM)神经网络等。Exemplarily, the neural network includes, but is not limited to, a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory (LSTM) neural network, and the like.

示例性地，特征映射结构采用影响函数核小的sigmoid函数作为卷积网络的激活函数，sigmoid函数是二分类的,每个类都能输出独立的置信度，因而在类间差异较小的时候能同时给出几个置信度较高的类别输出，训练模型时不会因此抑制相似类别特征,从而获得较好的模型性能。Exemplarily, the feature mapping structure uses the sigmoid function with a small influence function kernel as the activation function of the convolutional network. The sigmoid function is binary classification, and each class can output independent confidence, so when the difference between classes is small It can give several class outputs with high confidence at the same time, and it will not suppress similar class features when training the model, so as to obtain better model performance.

示例性地，步骤102属于智能设备本地执行。上述智能设备本地执行，其含义为在步骤102的执行过程中，车辆本地的智能设备不需将所采集的数据上传至服务器，仅在单个车辆本地的车载智能设备中进行上述数据处理过程。Exemplarily, step 102 belongs to the smart device to execute locally. The above-mentioned smart device executes locally, which means that during the execution of step 102, the smart device that is local to the vehicle does not need to upload the collected data to the server, and performs the above-mentioned data processing process only on the on-board smart device that is local to a single vehicle.

步骤103：数据传输步骤。Step 103: a data transmission step.

可选的，步骤103可以包括：车载智能设备将存在异常车辆行为的视频帧及异常车辆行为的事件类型上传至服务器。Optionally, step 103 may include: the in-vehicle smart device uploads a video frame with an abnormal vehicle behavior and an event type of the abnormal vehicle behavior to the server.

车载智能设备可以将道路中的所有车辆行驶行为相关数据以及异常车辆行为的识别结果实时上传至服务器。The in-vehicle intelligent device can upload data related to the driving behavior of all vehicles on the road and the recognition results of abnormal vehicle behavior to the server in real time.

示例性地，将结构化分析数据及关键视频数据上传服务器。所述结构化分析数据包括以数据库形式体现的异常车辆行为的识别结果和/或中间过程数据。所述关键视频数据包括用于判断出存在异常车辆行为的多个帧。For example, the structured analysis data and key video data are uploaded to the server. The structured analysis data includes identification results of abnormal vehicle behavior and / or intermediate process data embodied in a database form. The key video data includes a plurality of frames for determining that there is abnormal vehicle behavior.

示例性地，所述数据传输的方式包括无线网络通信模式。Exemplarily, the data transmission mode includes a wireless network communication mode.

步骤104：综合报警步骤。Step 104: Comprehensive alarm step.

可选的，该步骤包括服务器进行车辆行为分析与报警。Optionally, this step includes the server performing vehicle behavior analysis and alarm.

示例性地，通过对步骤103上传的某一区域的道路中所有车辆的异常车辆行为的识别结果进行分析，进行报警和后处理。其中，所述报警包括，由服务器根据接收到的识别结果经由无线网络通过车载智能设备对该车辆发出报警信息，并经由无线网络通过车载智能设备对该车辆的周围车辆发出报警信息；后处理包括针对某一类型/形态的交通事故对事故类型进行处理，轻微事故直接给出判罚与后续清道措施，使道路交通尽快恢复正常，以辅助提升城市交通效率。Exemplarily, by analyzing the recognition results of abnormal vehicle behaviors of all vehicles on a road in a certain area uploaded in step 103, alarms and post-processing are performed. Wherein, the alarm includes that the server sends alarm information to the vehicle through the vehicle-mounted smart device via the wireless network according to the received recognition result, and sends the alarm information to the surrounding vehicles of the vehicle through the vehicle-mounted smart device via the wireless network; The type of traffic accident is dealt with according to a certain type / form. Minor accidents will be directly penalized and follow-up clearance measures will be made to restore road traffic to normal as soon as possible to help improve urban traffic efficiency.

可选的，步骤104可以包括：向存在异常车辆行为的车辆、以及与该车辆的距离在预设范围内的多个车辆发出报警信息。Optionally, step 104 may include: sending an alarm message to a vehicle having abnormal vehicle behavior, and a plurality of vehicles having a distance from the vehicle within a preset range.

示例性地，与该车辆的距离在预设范围内的多个车辆即为该车辆的周围车辆，包括在设定范围内的开启网络接收功能的所有车辆。Exemplarily, a plurality of vehicles whose distances from the vehicle are within a preset range are surrounding vehicles of the vehicle, including all vehicles with a network receiving function enabled within a set range.

上述发出报警信息的实现方式包括对驾驶员发出音频、视频、体感等多种形式的报警信息，还包括对副驾驶或者乘客发出音频、视频、体感等多种形式的报警信息。The implementation of the above-mentioned alarm information includes audio, video and somatosensory alarm messages to the driver, as well as audio, video and somatosensory alarm messages to the co-pilot or passenger.

根据本申请所提出的方案，至少能达到如下的有益效果：According to the scheme proposed in this application, at least the following beneficial effects can be achieved:

1.能够识别各种违章驾驶、交通事故等事件类型；1. Able to identify various types of incidents such as illegal driving and traffic accidents;

2.利用车载智能设备和大数据分析解决监控盲点问题，在道路全程都能监督车辆安全行驶以及在监控盲点进行事故处理；2.Using in-vehicle intelligent equipment and big data analysis to solve the problem of monitoring blind spots, which can monitor the safe driving of vehicles throughout the road and handle accidents at the monitoring blind spots;

3.充分利用车载智能设备的本地处理能力，减少服务器端的数据处理压力，可以降低对服务器的硬件需求，通过本地处理、服务器判断报警使得车辆报警有更高的实时性，实现了加强道路安全和/或加速事故的处理的技术效果。3. Make full use of the local processing capabilities of the vehicle-mounted smart device, reduce the data processing pressure on the server side, and reduce the hardware requirements for the server. Through local processing and the server judges the alarm, the vehicle alarm has a higher real-time nature, and the road safety and And / or the technical effect of speeding up the handling of the accident.

与上述步骤相应地，本申请提出一种车辆行为检测装置，如图2所示，应用于车载智能设备，该装置包括：Corresponding to the above steps, the present application proposes a vehicle behavior detection device, as shown in FIG. 2, which is applied to a vehicle-mounted intelligent device. The device includes:

采集模块201，用于采集视频帧。An acquisition module 201 is configured to acquire a video frame.

数据处理模块202，用于基于所述视频帧，对车辆行为进行识别，得到异常车辆行为的识别结果。The data processing module 202 is configured to identify a vehicle behavior based on the video frame, and obtain a recognition result of abnormal vehicle behavior.

数据传输模块203，用于将所述识别结果上传至服务器。A data transmission module 203 is configured to upload the recognition result to a server.

可选的，采集模块201，具体可以用于：按照固定时间间隔，采集多个视频帧。Optionally, the acquisition module 201 may be specifically configured to collect multiple video frames at a fixed time interval.

可选的，数据处理模块202，具体可以用于：将所述视频帧输入神经网络，利用所述神经网络对所述视频帧进行特征抽取，生成多维的行为特征向量；对所述多维的行为特征向量中的每一维采用逻辑分类，得到每一维行为特征向量为不同事件类型的异常车辆行为的置信度；若任一事件类型的异常车辆行为的置信度大于预设阈值，则确定所述视频帧中存在该事件类型的异常车辆行为。Optionally, the data processing module 202 may be specifically configured to: input the video frame into a neural network, use the neural network to perform feature extraction on the video frame, and generate a multi-dimensional behavior feature vector; Each dimension in the feature vector is logically classified to obtain the confidence level that the behavior feature vector of each dimension is an abnormal vehicle behavior of a different event type; if the confidence level of the abnormal vehicle behavior of any event type is greater than a preset threshold, determine all The abnormal vehicle behavior of this event type is described in the video frame.

可选的，数据传输模块203，具体可以用于：将存在异常车辆行为的视频帧及所述异常车辆行为的事件类型上传至服务器。Optionally, the data transmission module 203 may be specifically configured to: upload a video frame with abnormal vehicle behavior and an event type of the abnormal vehicle behavior to a server.

可选的，采集模块201，具体可以用于：以所述车载智能设备所属的车辆为中心，设定检测范围，对所述检测范围内的一个或多个车辆拍摄得到多个视频帧。Optionally, the acquisition module 201 may be specifically configured to set a detection range around the vehicle to which the vehicle-mounted smart device belongs, and shoot one or more vehicles within the detection range to obtain multiple video frames.

本申请提出一种车辆行为检测装置，如图3所示，应用于服务器，该装置包括：This application proposes a vehicle behavior detection device, as shown in FIG. 3, which is applied to a server. The device includes:

接收模块301，用于接收车载智能设备发送的识别结果，所述识别结果为所述车载智能设备基于采集的视频帧，对车辆行为进行识别，得到的异常车辆行为的识别结果；The receiving module 301 is configured to receive a recognition result sent by a vehicle-mounted smart device, where the recognition result is a recognition result of abnormal vehicle behavior obtained by the vehicle-mounted smart device based on the collected video frames;

综合报警模块302，用于根据所述识别结果，进行车辆行为报警。A comprehensive alarm module 302 is configured to perform a vehicle behavior alarm according to the recognition result.

可选的，综合报警模块302，具体可以用于：向存在异常车辆行为的车辆、以及与所述车辆的距离在预设范围内的多个车辆发出报警信息。Optionally, the comprehensive alarm module 302 may be specifically configured to: issue an alarm message to a vehicle having abnormal vehicle behavior, and a plurality of vehicles having a distance from the vehicle within a preset range.

根据本申请的另一种实施方式，提出一种车辆行为检测方法，如图4所示，包括：According to another embodiment of the present application, a vehicle behavior detection method is provided, as shown in FIG. 4, including:

步骤111：数据采集步骤。Step 111: a data collection step.

可选的，步骤111可以包括：车载智能设备按照固定时间间隔，采集多个视频。Optionally, step 111 may include: the vehicle-mounted smart device collects multiple videos at a fixed time interval.

可选的，步骤111可以包括：车载智能设备以其自身所属的车辆为中心，设定检测范围，对检测范围内的一个或多个车辆拍摄得到多个视频帧。Optionally, step 111 may include: the vehicle-mounted smart device sets a detection range around the vehicle to which it belongs, and photographs one or more vehicles within the detection range to obtain multiple video frames.

步骤112：数据传输步骤。Step 112: a data transmission step.

可选的，步骤112可以包括：车载智能设备将步骤111中所采集的视频帧数据实时上传至服务器。Optionally, step 112 may include: the vehicle-mounted smart device uploads the video frame data collected in step 111 to the server in real time.

示例性地，数据传输的方式可以包括无线网络通信模式。Exemplarily, the manner of data transmission may include a wireless network communication mode.

步骤113：数据处理步骤。Step 113: a data processing step.

可选的，步骤113可以包括：服务器将视频帧输入神经网络(neural network)，利用神经网络对视频帧进行特征抽取，生成多维的行为特征向量；对多维的行为特征向量中的每一维采用逻辑分类，得到每一维行为特征向量为不同事件类型的异常车辆行为的置信度；若任一事件类型的异常车辆行为的置信度大于预设阈值，则确定视频帧中存在该事件类型的异常车辆行为。Optionally, step 113 may include: the server inputs a video frame into a neural network, and uses the neural network to perform feature extraction on the video frame to generate a multi-dimensional behavior feature vector; and adopts each dimension in the multi-dimensional behavior feature vector. Logically classify and obtain the confidence level that the feature vector of each dimension is abnormal vehicle behavior of different event types; if the confidence level of abnormal vehicle behavior of any event type is greater than a preset threshold, it is determined that there is an abnormality of the event type in the video frame Vehicle behavior.

示例性地，上述异常车辆行为的识别结果，其含义为判断得出视频帧中一个车辆属于危险驾驶或者交通事故，得出识别结果的判别规则是根据神经网络模型所输出的置信度是否大于预设阈值判断得出。即，如果逻辑分类后得到是危险驾驶的置信度大于预设阈值，那么判断得出当前视频帧中包括危险驾驶的异常车辆行为。如果逻辑分类后得到是交通事故的置信度大于预设阈值，那么得出当前视频帧中包括交通事故的异常车辆行为。神经网络模型的训练过程，在此不再赘述。Exemplarily, the above recognition result of abnormal vehicle behavior means that it is determined that a vehicle in a video frame belongs to a dangerous driving or a traffic accident, and the determination rule for obtaining the recognition result is based on whether the confidence level output by the neural network model is greater than Judging by setting the threshold. That is, if the confidence that the dangerous driving is obtained after the logical classification is greater than a preset threshold, it is determined that the abnormal vehicle behavior of the dangerous driving is included in the current video frame. If the confidence level obtained after the logical classification is that the traffic accident is greater than a preset threshold, then the current video frame includes the abnormal vehicle behavior of the traffic accident. The training process of the neural network model is not repeated here.

异常车辆行为的事件类型包括危险驾驶和/或交通事故。危险驾驶包括追逐竞驾、醉酒驾驶以及疲劳驾驶，其中，追逐竞驾的判断包括通过前后车位置、距离随时间变化的视频帧的情况，醉酒驾驶的判断包括通过车辆行驶轨迹的变化、司机本体变化的视频帧的情况，例如司机身体躯干的晃动情况等，疲劳驾驶的判断包括通过车辆行驶轨迹的变化、司机本体变化的视频帧的情况，例如司机眼睛的睁开与闭合情况等；交通事故包括某一类型/形态的交通事故，其中，交通事故在类型上包括追尾事故、超车事故、左转弯事故、有转变事故等，在形态上包括碰撞、刮擦、碾压、翻车、坠车、失火等。Types of incidents of abnormal vehicle behavior include dangerous driving and / or traffic accidents. Dangerous driving includes chasing, drunk driving, and fatigue driving. Among them, the judgment of chasing driving includes passing through the video frames of the position and distance of the car in front and rear, and the distance changes with time. The judgment of drunk driving includes changing the driving trajectory of the vehicle, and the driver ’s body. The situation of changing video frames, such as the shaking of the torso of the driver ’s body, and the judgment of fatigue driving includes the changes of the vehicle ’s driving trajectory and the changes of the driver ’s body, such as the opening and closing of the driver ’s eyes; traffic accidents Including a certain type / form of traffic accidents, among which the types of traffic accidents include rear-end accidents, overtaking accidents, left-turn accidents, and transition accidents, and morphologically include collisions, scratches, rolling, rollovers, crashes, Fire, etc.

示例性地，特征映射结构采用影响函数核小的sigmoid函数作为卷积网络的激活函数，使得特征映射具有位移不变性。Exemplarily, the feature map structure uses a sigmoid function with a small influence function kernel as the activation function of the convolution network, so that the feature map has displacement invariance.

在服务器侧识别各种违章驾驶、交通事故，可以降低对车载智能设备的硬件需求，充分利用无线传输的便捷性与高效率，在服务器端进行完备的数据处理和分析，通过实时数据传输、服务器端数据处理使得车辆报警有更高的准确度，实现了加强道路安全和/或加速事故的处理的技术效果。Identifying various illegal driving and traffic accidents on the server side can reduce the hardware requirements for on-board smart devices, make full use of the convenience and efficiency of wireless transmission, and perform complete data processing and analysis on the server side. Through real-time data transmission, the server The terminal data processing makes the vehicle alarm have higher accuracy, and achieves the technical effect of strengthening road safety and / or speeding up the processing of accidents.

步骤114：综合报警步骤。Step 114: Comprehensive alarm step.

示例性地，通过对步骤113得出的异常车辆行为的识别结果，进行报警和后处理。其中，所述报警包括，根据识别结果经由无线网络通过车载智能设备对该车辆发出报警信息，并经由无线网络通过车载智能设备对该车辆的周围车辆发出报警信息；后处理包括针对某一类型/形态的交通事故对事故类型进行处理，轻微事故直接给出判罚与后续清道措施，使道路交通尽快恢复正常，以辅助提升城市交通效率。Exemplarily, an alarm and post-processing are performed by identifying a result of abnormal vehicle behavior obtained in step 113. Wherein, the alarm includes: sending an alarm message to the vehicle through the vehicle-mounted smart device via the wireless network according to the recognition result, and sending an alarm message to the vehicle's surrounding vehicles through the vehicle-mounted intelligent device via the wireless network; the post-processing includes targeting a certain type / Traffic accidents in various forms deal with the types of accidents. Minor accidents directly provide penalties and follow-up clearance measures to restore road traffic to normal as soon as possible to help improve urban traffic efficiency.

可选的，步骤114可以包括：向存在异常车辆行为的车辆、以及与该车辆的距离在预设范围内的多个车辆发出报警信息。Optionally, step 114 may include: sending an alarm message to a vehicle having abnormal vehicle behavior, and a plurality of vehicles having a distance from the vehicle within a preset range.

上述发出报警信息的实现方式包括对驾驶员发出音频、视频、体感等多种形式的报警信息，还包括对副驾驶或者乘客发出音频、视频、体感等多种形式的报警信息。The above-mentioned implementation manner of issuing the alarm information includes audio, video and somatosensory alarm information to the driver, and also includes audio, video and somatosensory alarm information to the co-pilot or passenger.

3.可以降低对车载智能设备的硬件需求，充分利用无线传输的便捷性与高效率，在服务器端进行完备的数据处理和分析，通过实时数据传输、服务器端数据处理使得车辆报警有更高的准确度，实现了加强道路安全和/或加速事故的处理的技术效果。3. It can reduce the hardware requirements for in-vehicle smart devices, make full use of the convenience and high efficiency of wireless transmission, and perform complete data processing and analysis on the server side. Through real-time data transmission and server-side data processing, vehicle alarms have a higher Accuracy, achieving the technical effect of enhancing road safety and / or speeding up the handling of accidents.

与上述步骤相应地，本申请提出一种车辆行为检测装置，如图5所示，应用于车载智能设备，该装置包括：Corresponding to the above steps, the present application proposes a vehicle behavior detection device, as shown in FIG. 5, which is applied to a vehicle-mounted smart device, and the device includes:

采集模块501，用于采集视频帧；An acquisition module 501, configured to acquire a video frame;

数据传输模块502，用于将所述视频帧上传至服务器。A data transmission module 502 is configured to upload the video frame to a server.

可选的，采集模块501，具体可以用于：以所述车载智能设备所属的车辆为中心，设定检测范围，对所述检测范围内的一个或多个车辆拍摄得到多个视频帧。Optionally, the acquisition module 501 may be specifically configured to set a detection range around a vehicle to which the vehicle-mounted smart device belongs, and shoot one or more vehicles within the detection range to obtain multiple video frames.

可选的，采集模块501，具体可以用于：按照固定时间间隔，采集多个视频帧。Optionally, the acquisition module 501 may be specifically configured to collect multiple video frames at a fixed time interval.

本申请提出一种车辆行为检测装置，如图6所示，应用于服务器，该装置包括：This application proposes a vehicle behavior detection device, as shown in FIG. 6, which is applied to a server. The device includes:

接收模块601，用于接收车载智能设备发送的视频帧；A receiving module 601, configured to receive a video frame sent by a vehicle-mounted smart device;

数据处理模块602，用于基于所述视频帧，对车辆行为进行识别，得到异常车辆行为的识别结果；A data processing module 602, configured to identify a vehicle behavior based on the video frame, and obtain a recognition result of abnormal vehicle behavior;

综合报警模块603，用于根据所述识别结果，进行车辆行为报警。A comprehensive alarm module 603 is configured to perform a vehicle behavior alarm according to the recognition result.

可选的，数据处理模块602，具体可以用于：将所述视频帧输入神经网络，利用所述神经网络对所述视频帧进行特征抽取，生成多维的行为特征向量；对所述多维的行为特征向量中的每一维采用逻辑分类，得到每一维行为特征向量为不同事件类型的异常车辆行为的置信度；若任一事件类型的异常车辆行为的置信度大于预设阈值，则确定所述视频帧中存在该事件类型的异常车辆行为。Optionally, the data processing module 602 may be specifically configured to: input the video frame into a neural network, and use the neural network to perform feature extraction on the video frame to generate a multi-dimensional behavior feature vector; Each dimension in the feature vector is logically classified to obtain the confidence level that the behavior feature vector of each dimension is an abnormal vehicle behavior of a different event type; if the confidence level of the abnormal vehicle behavior of any event type is greater than a preset threshold, determine all The abnormal vehicle behavior of this event type is described in the video frame.

可选的，综合报警模块603，具体可以用于：向存在异常车辆行为的车辆、以及与所述车辆的距离在预设范围内的多个车辆发出报警信息，所述异常车辆行为包括危险驾驶和/或交通事故。Optionally, the comprehensive alarm module 603 may be specifically configured to: issue alarm information to vehicles having abnormal vehicle behavior, and to multiple vehicles within a preset distance from the vehicle, where the abnormal vehicle behavior includes dangerous driving And / or traffic accidents.

本申请的可选实施例提供一种车辆行为检测方法，包括：步骤101：数据采集步骤。An optional embodiment of the present application provides a vehicle behavior detection method, including: Step 101: a data collection step.

采用车载智能设备进行视频采集，对于车载智能设备的摄像模块所采集的视频帧，每隔0.1秒将所采集的多个视频帧送入后续步骤进行计算。例如，后续步骤需要每隔0.1秒给出一次计算结果，视频全帧率是50帧/秒，则隔帧采样的结果是每隔0.1秒送5帧图片进入后续步骤。The vehicle intelligent device is used for video collection. For the video frames collected by the camera module of the vehicle intelligent device, multiple collected video frames are sent to the subsequent steps for calculation every 0.1 seconds. For example, in the subsequent steps, calculation results need to be given every 0.1 seconds, and the full frame rate of the video is 50 frames per second, and then the result of the interframe sampling is to send 5 frames of pictures every 0.1 seconds into the subsequent steps.

其中，车载智能设备包括行车记录仪、移动通信设备、或者其他具有数据传输功能的摄像装置。Among them, the vehicle-mounted smart device includes a driving recorder, a mobile communication device, or another camera device with a data transmission function.

步骤102：数据处理步骤，具体包括如下步骤：Step 102: a data processing step, which specifically includes the following steps:

步骤102-1：将采集的连续视频帧图片例如5帧图片输入神经网络，通过神经网络对连续帧图片例如5帧图片进行特征抽取，生成行为特征向量。其中，使用3D卷积网络进行视频序列的特征提取，使用LSTM网络对特征进行历史信息的融合后输出最终特征向量。Step 102-1: The collected continuous video frame pictures, such as 5 frames of pictures, are input into a neural network, and the continuous frame pictures, such as 5 frames of pictures, are subjected to feature extraction through the neural network to generate behavior feature vectors. Among them, a 3D convolutional network is used for feature extraction of a video sequence, and a LSTM network is used for historical feature fusion of features to output a final feature vector.

相关技术中，对视频的操作是将视频帧单独进行CNN特征编码，然后使用LSTM对各帧特征进行处理，如图7所示，对视频帧Image(t-2)、Image(t-1)、Image(t)分别进行CNN特征编码，然后采用LSTM对编码后的各帧特征进行处理，其中，h(x)是LSTM的隐含状态，S(t)是输出的结果向量，S(t-3)、S(t-2)、S(t-1)是过程向量，输出S(t)作为计算最终结果。In the related art, the operation of the video is to individually encode the CNN feature of the video frame, and then use LSTM to process the features of each frame. As shown in Figure 7, the video frames Image (t-2), Image (t-1) , Image (t) respectively encode CNN features, and then use LSTM to process the encoded features of each frame, where h (x) is the hidden state of LSTM, S (t) is the output result vector, and S (t -3), S (t-2), S (t-1) are process vectors, and S (t) is output as the final result of the calculation.

但是上述方法中存在如下问题：单张视频帧图片无法准确反映连续的视频片段状态，且前几张图片的LSTM网络输出无法参与训练。因此在本申请的步骤中，将图5的流程简化，如图8所示，将一段视频的5帧作为1个输入，使用3D卷积进行特征编码，然后使用LSTM对特征编码后的帧特征进行处理，其中，h(x)是LSTM的隐含状态，S(t)是输出的结果向量，S(t-5)是过程向量，输出S(t)作为本次计算最终结果。However, the above method has the following problems: a single video frame picture cannot accurately reflect the state of continuous video fragments, and the LSTM network output of the first few pictures cannot participate in training. Therefore, in the steps of this application, the process of FIG. 5 is simplified. As shown in FIG. 8, 5 frames of a video are taken as one input, 3D convolution is used to perform feature coding, and then LSTM is used to encode the feature frame features. Perform processing, where h (x) is the hidden state of LSTM, S (t) is the output result vector, S (t-5) is the process vector, and S (t) is output as the final result of this calculation.

步骤102-2：对特征向量S(t)采用逻辑分类，输出大于预设阈值的类别。Step 102-2: Logically classify the feature vector S (t) and output a category larger than a preset threshold.

采用sigmoid函数对特征向量S(t)进行最终的类别输出。The sigmoid function is used for the final class output of the feature vector S (t).

具体地，采用sigmoid函数对步骤102中的特征向量S(t)进行类别输出，sigmoid函数由以下公式定义：Specifically, the sigmoid function is used to classify the feature vector S (t) in step 102, and the sigmoid function is defined by the following formula:

其中，x为各个事件通过神经网络特征提取后产生的多维特征向量S(t)，每个所输出的类别的置信度f(x)都对应着特征向量上的一维。将f(x)与预设阈值(危险驾驶或者某一类型/形态的交通事故的区别判断标准)相比较，大于该预设阈值，则判断属于危险驾驶或者某一类型/形态的交通事故，得到异常车辆行为的识别结果。Among them, x is a multi-dimensional feature vector S (t) generated by the neural network feature extraction of each event, and the confidence f (x) of each output category corresponds to one dimension on the feature vector. Compare f (x) with a preset threshold (a criterion for distinguishing between dangerous driving or a certain type / form of traffic accidents), and if it is larger than the preset threshold, determine that it is a dangerous driving or a type / form of traffic accidents, Get the recognition result of abnormal vehicle behavior.

步骤102属于智能设备本地执行。上述智能设备本地执行，其含义为在步骤102的执行过程中，车辆本地的智能设备不需将所采集的数据上传至服务器，仅在单个车辆本地的车载智能设备中进行上述数据处理过程。Step 102 is performed locally by the smart device. The above-mentioned smart device executes locally, which means that during the execution of step 102, the smart device that is local to the vehicle does not need to upload the collected data to the server, and performs the above-mentioned data processing process only on the on-board smart device that is local to a single vehicle.

这样车载智能设备能够识别各种违章驾驶、交通事故等事件类型；如果充分利用车载智能设备的本地处理能力，车载智能设备上实现该功能，那么可以减少服务器端的数据处理压力，可以降低对服务器的硬件需求，通过本地处理、服务器判断报警使得车辆报警有更高的实时性，实现了加强道路安全和/或加速事故的处理的技术效果。In this way, the in-vehicle intelligent device can identify various types of incidents such as driving violations, traffic accidents; if the local processing capabilities of the in-vehicle intelligent device are fully utilized and this function is implemented on the in-vehicle intelligent device, the data processing pressure on the server side can be reduced, and the server's Hardware requirements, through local processing and server judgment alarms, make vehicle alarms have higher real-time performance, and achieve the technical effect of strengthening road safety and / or speeding up the handling of accidents.

步骤103：数据传输步骤。Step 103: a data transmission step.

该步骤包括将步骤102得出的识别结果，以及道路中的所有车辆行驶行为数据(结构化分析数据及关键视频数据)通过无线通信网络实时上传至服务器。结构化分析数据包括网络模型的输出f(x)所表征的事件类别，以及相关的视频帧数据、中间结果数据。This step includes uploading the recognition result obtained in step 102 and the driving behavior data (structured analysis data and key video data) of all vehicles on the road to the server in real time through the wireless communication network. The structured analysis data includes the event categories represented by the output f (x) of the network model, as well as related video frame data and intermediate result data.

步骤104：综合报警步骤。Step 104: Comprehensive alarm step.

该步骤包括在服务器，进行车辆行为分析与报警。This step includes performing a vehicle behavior analysis and alarm on the server.

通过上述识别结果，当车辆被判别为危险驾驶的时候，根据GPS以及车牌识别(车牌识别通过采样视频的中间帧图像来进行)信息的综合定位，向危险驾驶车辆周围一定范围内(示例性地，半径100米的圆周范围内)的车辆进行报警，输出危险驾驶车辆与本车的相对位置。GPS由于精度所限，不能精准地给出危险驾驶车辆与本车的相对位置，所以采用GPS给定范围，范围内的车辆通过各车所能看到的车牌在图像中的相对位置来构成整体的位置信息。Based on the above recognition results, when the vehicle is judged to be dangerous driving, based on the comprehensive positioning of GPS and license plate recognition (the license plate recognition is performed by sampling the middle frame image of the video) information, a certain range around the dangerous driving vehicle (exemplarily , Within a circle of a radius of 100 meters), an alarm is issued, and the relative position of the dangerous driving vehicle and its own vehicle is output. Due to the limited accuracy of GPS, the relative position of a dangerous driving vehicle and its own vehicle cannot be accurately given. Therefore, a given range of GPS is adopted. The vehicles in the range form the whole by the relative position of the license plate in the image that each vehicle can see. Location information.

通过上述识别结果，当车辆被判别为事故车辆的时候，根据GPS以及车牌识别(车牌识别通过采样视频的中间帧图像来进行)信息的综合定位，向事故车辆周围100米范围内的车辆进行报警，输出事故车辆与本车的相对位置。GPS信息由于精度所限，不能精准地给出事故车辆与本车的相对位置，所以采用GPS给定范围，范围内的车辆通过各车所能看到的车牌在图像中的相对位置来构成整体的位置信息。Based on the above recognition results, when a vehicle is judged to be an accident vehicle, the vehicle is alerted to vehicles within 100 meters of the accident vehicle based on the comprehensive positioning of GPS and license plate recognition (the license plate recognition is performed by sampling the middle frame image of the video). , Output the relative position of the accident vehicle and the own vehicle. Due to the limited accuracy of GPS information, the relative position of the accident vehicle and its own vehicle cannot be accurately given, so the GPS given range is used. The vehicles in the range form the whole by the relative position of the license plate in the image that each vehicle can see. Location information.

本申请的可选实施例提供一种车辆行为检测方法，包括：步骤111：数据采集步骤。An optional embodiment of the present application provides a vehicle behavior detection method, including: Step 111: a data collection step.

其中，智能设备包括行车记录仪、移动通信设备、或者其他具有数据传输功能的摄像装置。Among them, the smart device includes a driving recorder, a mobile communication device, or another camera device with a data transmission function.

步骤112：数据传输步骤。Step 112: a data transmission step.

具体地，该步骤包括将步骤111中所采集的视频帧实时上传至服务器。Specifically, this step includes uploading the video frames collected in step 111 to the server in real time.

示例性地，数据传输的方式包括无线网络通信模式。Exemplarily, the manner of data transmission includes a wireless network communication mode.

步骤113：数据处理步骤，具体包括在服务器执行如下步骤：Step 113: a data processing step, which specifically includes performing the following steps on the server:

步骤113-1：将上传的连续视频帧图片例如5帧图片输入神经网络，通过神经网络对连续帧图片例如5帧图片进行特征抽取，生成行为特征向量。其中，使用3D卷积网络进行视频序列的特征提取，使用LSTM网络对特征进行历史信息的融合后输出最终特征向量。Step 113-1: The uploaded continuous video frame pictures such as 5 frames are input to a neural network, and the continuous frame pictures such as 5 frames are subjected to feature extraction through the neural network to generate behavior feature vectors. Among them, a 3D convolutional network is used for feature extraction of a video sequence, and a LSTM network is used for historical feature fusion of features to output a final feature vector.

步骤113-2：对特征向量S(t)采用逻辑分类，输出大于预设阈值的类别。Step 113-2: Logically classify the feature vector S (t) and output a category larger than a preset threshold.

具体地，采用sigmoid函数对步骤112中的特征向量S(t)进行类别输出，sigmoid函数由以下公式定义：Specifically, the sigmoid function is used to classify the feature vector S (t) in step 112, and the sigmoid function is defined by the following formula:

步骤11：综合报警步骤。Step 11: Comprehensive alarm procedure.

根据本申请的实施方式，还提出一种车辆行为检测***，包括：车载智能设备及服务器；According to an embodiment of the present application, a vehicle behavior detection system is further provided, including: a vehicle-mounted intelligent device and a server;

示例性地，服务器可以是云分析服务器。Exemplarily, the server may be a cloud analysis server.

图9示出了该车辆行为检测***综合报警步骤的示意图，当前车辆的车载智能设备进行异常车辆行为的识别后，将识别结果发送至服务器，服务器根据识别结果，向判断为异常车辆行为的车载智能设备、判断为异常车辆行为的周围车辆的车载智能设备、120急救平台、交通管理机构平台等发出报警信息。FIG. 9 shows a schematic diagram of the comprehensive alarm procedure of the vehicle behavior detection system. After the vehicle-mounted smart device of the current vehicle recognizes abnormal vehicle behavior, it sends the recognition result to the server, and the server sends the identification to the vehicle that is determined to have abnormal vehicle behavior. Intelligent devices, on-board intelligent devices of surrounding vehicles judged to be abnormal vehicle behavior, 120 emergency platforms, traffic management agency platforms, etc. issue alarm messages.

根据本申请的另一种实施方式，提出一种车辆行为检测***，包括：车载智能设备及服务器；According to another embodiment of the present application, a vehicle behavior detection system is provided, including: a vehicle intelligent device and a server;

图10示出了该车辆行为检测***综合报警步骤的示意图，当前车辆的车载智能设备将采集的视频帧发送至服务器，服务器对异常车辆行为进行识别后，根据识别结果，向判断为异常车辆行为的车载智能设备、判断为异常车辆行为的周围车辆的车载智能设备、120急救平台、交通管理机构平台等发出报警信息。FIG. 10 shows a schematic diagram of the comprehensive alarm procedure of the vehicle behavior detection system. The on-board intelligent device of the current vehicle sends the collected video frames to the server. After the server recognizes the abnormal vehicle behavior, it judges the abnormal vehicle behavior according to the recognition result. The vehicle-mounted intelligent device, the vehicle-mounted intelligent device of the surrounding vehicles judged to be abnormal vehicle behavior, the 120 emergency platform, the traffic management agency platform, and the like issue alarm information.

本申请实施例还提供了一种车载智能设备，如图11所示，包括处理器1101和机器可读存储介质1102，其中，An embodiment of the present application further provides a vehicle-mounted smart device. As shown in FIG. 11, the device includes a processor 1101 and a machine-readable storage medium 1102.

机器可读存储介质1102，用于存储能够被处理器1101执行的机器可执行指令；A machine-readable storage medium 1102, for storing machine-executable instructions capable of being executed by the processor 1101;

处理器1101，用于被机器可读存储介质1102上所存放的机器可执行指令促使执行如下方法步骤：The processor 1101 is configured to be caused to execute the following method steps by the machine-executable instructions stored on the machine-readable storage medium 1102:

采集视频帧；Capture video frames;

可选的，所述处理器1101在执行所述采集视频帧时，具体可以执行：Optionally, when the processor 1101 executes the captured video frame, it may specifically perform:

按照固定时间间隔，采集多个视频帧。Capture multiple video frames at regular time intervals.

可选的，所述处理器1101在执行所述基于所述视频帧，对车辆行为进行识别，得到异常车辆行为的识别结果时，具体可以执行：Optionally, when the processor 1101 executes the recognition of the vehicle behavior based on the video frame and obtains the recognition result of the abnormal vehicle behavior, it may specifically perform:

将所述视频帧输入神经网络，利用所述神经网络对所述视频帧进行特征抽取，生成多维的行为特征向量；Input the video frame into a neural network, and use the neural network to perform feature extraction on the video frame to generate a multi-dimensional behavior feature vector;

对所述多维的行为特征向量中的每一维采用逻辑分类，得到每一维行为特征向量为不同事件类型的异常车辆行为的置信度；Logically classify each dimension in the multi-dimensional behavioral feature vector to obtain the confidence that the behavioral feature vector in each dimension is an abnormal vehicle behavior of a different event type;

若任一事件类型的异常车辆行为的置信度大于预设阈值，则确定所述视频帧中存在该事件类型的异常车辆行为。If the confidence level of the abnormal vehicle behavior of any event type is greater than a preset threshold, it is determined that the abnormal vehicle behavior of the event type exists in the video frame.

可选的，所述处理器1101在执行所述将所述识别结果上传至服务器时，具体可以执行：Optionally, when the processor 1101 executes the uploading the recognition result to the server, the processor 1101 may specifically perform:

将存在异常车辆行为的视频帧及所述异常车辆行为的事件类型上传至服务器。Upload the video frame with abnormal vehicle behavior and the event type of the abnormal vehicle behavior to the server.

以所述车载智能设备所属的车辆为中心，设定检测范围，对所述检测范围内的一个或多个车辆拍摄得到多个视频帧。The detection range is set with the vehicle to which the vehicle-mounted smart device belongs as a center, and multiple video frames are obtained by shooting one or more vehicles within the detection range.

机器可读存储介质1102与处理器1101之间可以通过有线连接或者无线连接的方式进行数据传输，并且车载智能设备可以通过有线通信接口或者无线通信接口与其他的设备进行通信。The machine-readable storage medium 1102 and the processor 1101 may perform data transmission through a wired connection or a wireless connection, and the vehicle-mounted intelligent device may communicate with other devices through a wired communication interface or a wireless communication interface.

本申请实施例还提供了一种机器可读存储介质，存储有机器可执行指令，在被处理器调用和执行时，所述机器可执行指令促使所述处理器执行图11所示实施例的车载智能设备所执行的步骤。An embodiment of the present application further provides a machine-readable storage medium that stores machine-executable instructions. When called and executed by a processor, the machine-executable instructions cause the processor to execute the embodiment shown in FIG. 11. Steps performed by an in-vehicle smart device.

本申请实施例还提供了一种应用程序，用于在运行时执行：图11所示实施例的车载智能设备所执行的步骤。The embodiment of the present application further provides an application program for executing during execution: the steps performed by the vehicle-mounted smart device in the embodiment shown in FIG. 11.

本申请实施例还提供了一种服务器，如图12所示，包括处理器1201和机器可读存储介质1202，其中，An embodiment of the present application further provides a server, as shown in FIG. 12, including a processor 1201 and a machine-readable storage medium 1202, where:

机器可读存储介质1202，用于存储能够被处理器1201执行的机器可执行指令；A machine-readable storage medium 1202, configured to store machine-executable instructions that can be executed by the processor 1201;

处理器1201，用于被机器可读存储介质1202上所存放的机器可执行指令促使执行如下方法步骤：The processor 1201 is configured to be caused to execute the following method steps by the machine-executable instructions stored on the machine-readable storage medium 1202:

可选的，所述处理器1201在执行所述根据所述识别结果，进行车辆行为报警时，具体可以执行：Optionally, when the processor 1201 executes the vehicle behavior alarm according to the recognition result, it may specifically perform:

向存在异常车辆行为的车辆、以及与所述车辆的距离在预设范围内的多个车辆发出报警信息。The warning information is issued to a vehicle having abnormal vehicle behavior and a plurality of vehicles having a distance from the vehicle within a preset range.

机器可读存储介质1202与处理器1201之间可以通过有线连接或者无线连接的方式进行数据传输，并且服务器可以通过有线通信接口或者无线通信接口与其他的设备进行通信。The machine-readable storage medium 1202 and the processor 1201 may perform data transmission through a wired connection or a wireless connection, and the server may communicate with other devices through a wired communication interface or a wireless communication interface.

本申请实施例还提供了一种机器可读存储介质，存储有机器可执行指令，在被处理器调用和执行时，所述机器可执行指令促使所述处理器执行图12所示实施例的服务器所执行的步骤。An embodiment of the present application further provides a machine-readable storage medium that stores machine-executable instructions. When called and executed by a processor, the machine-executable instructions cause the processor to execute the embodiment shown in FIG. 12. Steps performed by the server.

本申请实施例还提供了一种应用程序，用于在运行时执行：图12所示实施例的服务器所执行的步骤。The embodiment of the present application further provides an application program, which is used to execute at runtime: the steps performed by the server in the embodiment shown in FIG. 12.

本申请实施例还提供了一种车载智能设备，如图13所示，包括处理器1301和机器可读存储介质1302，其中，An embodiment of the present application further provides a vehicle-mounted smart device. As shown in FIG. 13, the smart device includes a processor 1301 and a machine-readable storage medium 1302, where:

机器可读存储介质1302，用于存储能够被处理器1301执行的机器可执行指令；A machine-readable storage medium 1302, configured to store machine-executable instructions that can be executed by the processor 1301;

处理器1301，用于被机器可读存储介质1302上所存放的机器可执行指令促使执行如下方法步骤：The processor 1301 is configured to be caused to execute the following method steps by the machine-executable instructions stored on the machine-readable storage medium 1302:

采集视频帧；Capture video frames;

可选的，所述处理器1301在执行所述采集视频帧时，具体可以执行：Optionally, when the processor 1301 executes the captured video frame, it may specifically perform:

机器可读存储介质1302与处理器1301之间可以通过有线连接或者无线连接的方式进行数据传输，并且车载智能设备可以通过有线通信接口或者无线通信接口与其他的设备进行通信。Data can be transmitted between the machine-readable storage medium 1302 and the processor 1301 through a wired connection or a wireless connection, and the vehicle-mounted intelligent device can communicate with other devices through a wired communication interface or a wireless communication interface.

本申请实施例还提供了一种机器可读存储介质，存储有机器可执行指令，在被处理器调用和执行时，所述机器可执行指令促使所述处理器执行图13所示实施例的车载智能设备所执行的步骤。An embodiment of the present application further provides a machine-readable storage medium that stores machine-executable instructions. When called and executed by a processor, the machine-executable instructions cause the processor to execute the embodiment shown in FIG. 13. Steps performed by an in-vehicle smart device.

本申请实施例还提供了一种应用程序，用于在运行时执行：图13所示实施例的车载智能设备所执行的步骤。The embodiment of the present application further provides an application program for executing during execution: the steps performed by the vehicle-mounted smart device in the embodiment shown in FIG. 13.

本申请实施例还提供了一种服务器，如图14所示，包括处理器1401和机器可读存储介质1402，其中，An embodiment of the present application further provides a server, as shown in FIG. 14, including a processor 1401 and a machine-readable storage medium 1402, where:

机器可读存储介质1402，用于存储能够被处理器1401执行的机器可执行指令；A machine-readable storage medium 1402, configured to store machine-executable instructions that can be executed by the processor 1401;

处理器1401，用于被机器可读存储介质1402上所存放的机器可执行指令促使执行如下方法步骤：The processor 1401 is configured to be caused to execute the following method steps by the machine-executable instructions stored on the machine-readable storage medium 1402:

可选的，所述处理器1401在执行所述基于所述视频帧，对车辆行为进行识别，得到异常车辆行为的识别结果时，具体可以执行：Optionally, when the processor 1401 executes the recognition of the vehicle behavior based on the video frame and obtains the recognition result of the abnormal vehicle behavior, it may specifically perform:

可选的，所述处理器1401在执行所述根据所述识别结果，进行车辆行为报警时，具体可以执行：Optionally, when the processor 1401 executes the vehicle behavior alarm according to the recognition result, it may specifically perform:

向存在异常车辆行为的车辆、以及与所述车辆的距离在预设范围内的多个车辆发出报警信息，所述异常车辆行为包括危险驾驶和/或交通事故。Issue warning information to vehicles that have abnormal vehicle behavior and multiple vehicles that are within a preset distance from the vehicle, where the abnormal vehicle behavior includes dangerous driving and / or traffic accidents.

机器可读存储介质1402与处理器1401之间可以通过有线连接或者无线连接的方式进行数据传输，并且服务器可以通过有线通信接口或者无线通信接口与其他的设备进行通信。The machine-readable storage medium 1402 and the processor 1401 may perform data transmission through a wired connection or a wireless connection, and the server may communicate with other devices through a wired communication interface or a wireless communication interface.

本申请实施例还提供了一种机器可读存储介质，存储有机器可执行指令，在被处理器调用和执行时，所述机器可执行指令促使所述处理器执行图14所示实施例的服务器所执行的步骤。An embodiment of the present application further provides a machine-readable storage medium that stores machine-executable instructions. When called and executed by a processor, the machine-executable instructions cause the processor to execute the embodiment shown in FIG. 14. Steps performed by the server.

本申请实施例还提供了一种应用程序，用于在运行时执行：图14所示实施例的服务器所执行的步骤。The embodiment of the present application further provides an application program for executing at runtime: the steps performed by the server in the embodiment shown in FIG. 14.

上述机器可读存储介质可以包括RAM(Random Access Memory，随机存取存储器)，也可以包括NVM(Non-volatile Memory，非易失性存储器)，例如至少一个磁盘存储器。可选的，机器可读存储介质还可以是至少一个位于远离前述处理器的存储装置。The above machine-readable storage medium may include RAM (Random Access Memory, Random Access Memory), and may also include NVM (Non-volatile Memory, non-volatile memory), such as at least one disk memory. Optionally, the machine-readable storage medium may also be at least one storage device located far from the foregoing processor.

上述处理器可以是通用处理器，包括CPU(Central Processing Unit，中央处理器)、NP(Network Processor，网络处理器)等；还可以是DSP(Digital Signal Processor，数字信号处理器)、ASIC(Application Specific Integrated Circuit，专用集成电路)、FPGA(Field-Programmable Gate Array，现场可编程门阵列)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。The above processor may be a general-purpose processor, including a CPU (Central Processing Unit), a NP (Network Processor), etc .; it may also be a DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit (ASIC), FPGA (Field-Programmable Gate Array), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

对于车辆行为检测装置、车辆行为检测***、车载智能设备、服务器、机器可读存储介质以及应用程序实施例而言，由于其所涉及的方法内容基本相似于前述的方法实施例，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。For the vehicle behavior detection device, the vehicle behavior detection system, the vehicle-mounted intelligent device, the server, the machine-readable storage medium, and the application program embodiment, since the content of the method involved is basically similar to the foregoing method embodiment, the comparison described It is simple, and the relevant part can refer to the description of the method embodiment.

需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。It should be noted that in this article, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations There is any such actual relationship or order among them. Moreover, the terms "including", "comprising", or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or device that includes a series of elements includes not only those elements but also those that are not explicitly listed Or other elements inherent to such a process, method, article, or device. Without more restrictions, the elements defined by the sentence "including a ..." do not exclude the existence of other identical elements in the process, method, article, or equipment including the elements.

本说明书中的各个实施例均采用相关的方式描述，各个实施例之间相同相似的部分互相参见即可，每个实施例重点说明的都是与其他实施例的不同之处。尤其，对于车辆行为检测装置、车辆行为检测***、车载智能设备、服务器、机器可读存储介质以及应用程序实施例而言，由于其基本相似于方法实施例，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。Each embodiment in this specification is described in a related manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, the embodiments of the vehicle behavior detection device, the vehicle behavior detection system, the in-vehicle smart device, the server, the machine-readable storage medium, and the application program are basically similar to the method embodiment, so the description is relatively simple and relevant See the description of the method embodiments.

以上所述仅为本申请的较佳实施例而已，并不用以限制本申请，凡在本申请的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本申请保护的范围之内。The above are only preferred embodiments of this application, and are not intended to limit this application. Any modification, equivalent replacement, or improvement made within the spirit and principles of this application shall be included in this application Within the scope of protection.

Claims

一种车辆行为检测方法，其特征在于，应用于车载智能设备，所述方法包括：A vehicle behavior detection method is characterized in that it is applied to a vehicle-mounted smart device, and the method includes:

采集视频帧；Capture video frames;

基于所述视频帧，对车辆行为进行识别，得到异常车辆行为的识别结果；Identifying vehicle behavior based on the video frame to obtain a recognition result of abnormal vehicle behavior;

将所述识别结果上传至服务器，以使所述服务器根据所述识别结果，进行车辆行为报警。The recognition result is uploaded to a server, so that the server performs a vehicle behavior alarm according to the recognition result.
根据权利要求1所述的方法，其特征在于，所述采集视频帧，包括：The method according to claim 1, wherein the acquiring a video frame comprises:

按照固定时间间隔，采集多个视频帧。Capture multiple video frames at regular time intervals.
根据权利要求1所述的方法，其特征在于，所述基于所述视频帧，对车辆行为进行识别，得到异常车辆行为的识别结果，包括：The method according to claim 1, wherein the identifying vehicle behavior based on the video frame to obtain a recognition result of abnormal vehicle behavior comprises:

将所述视频帧输入神经网络，利用所述神经网络对所述视频帧进行特征抽取，生成多维的行为特征向量；Input the video frame into a neural network, and use the neural network to perform feature extraction on the video frame to generate a multi-dimensional behavior feature vector;

对所述多维的行为特征向量中的每一维采用逻辑分类，得到每一维行为特征向量为不同事件类型的异常车辆行为的置信度；Logically classify each dimension in the multi-dimensional behavioral feature vector to obtain the confidence that the behavioral feature vector in each dimension is an abnormal vehicle behavior of a different event type;

若任一事件类型的异常车辆行为的置信度大于预设阈值，则确定所述视频帧中存在该事件类型的异常车辆行为。If the confidence level of the abnormal vehicle behavior of any event type is greater than a preset threshold, it is determined that the abnormal vehicle behavior of the event type exists in the video frame.
根据权利要求1所述的方法，其特征在于，所述将所述识别结果上传至服务器，包括：The method according to claim 1, wherein the uploading the recognition result to a server comprises:

将存在异常车辆行为的视频帧及所述异常车辆行为的事件类型上传至服务器。Upload the video frame with abnormal vehicle behavior and the event type of the abnormal vehicle behavior to the server.
根据权利要求4所述的方法，其特征在于，所述采集视频帧，包括：The method according to claim 4, wherein said acquiring video frames comprises:

以所述车载智能设备所属的车辆为中心，设定检测范围，对所述检测范围内的一个或多个车辆拍摄得到多个视频帧。The detection range is set with the vehicle to which the vehicle-mounted smart device belongs as a center, and multiple video frames are obtained by shooting one or more vehicles within the detection range.
一种车辆行为检测方法，其特征在于，应用于服务器，所述方法包括：A vehicle behavior detection method is characterized in that it is applied to a server, and the method includes:

接收车载智能设备发送的识别结果，所述识别结果为所述车载智能设备基于采集的视频帧，对车辆行为进行识别，得到的异常车辆行为的识别结果；Receiving a recognition result sent by a vehicle-mounted smart device, where the recognition result is a recognition result of abnormal vehicle behavior obtained by the vehicle-mounted smart device based on the collected video frames;

根据所述识别结果，进行车辆行为报警。According to the recognition result, a vehicle behavior alarm is performed.
根据权利要求6所述的方法，其特征在于，所述根据所述识别结果，进行车辆行为报警，包括：The method according to claim 6, wherein the performing a vehicle behavior alarm according to the recognition result comprises:

向存在异常车辆行为的车辆、以及与所述车辆的距离在预设范围内的多个车辆发出报警信息。The warning information is issued to a vehicle having abnormal vehicle behavior and a plurality of vehicles having a distance from the vehicle within a preset range.
一种车辆行为检测方法，其特征在于，应用于车载智能设备，所述方法包括：A vehicle behavior detection method is characterized in that it is applied to a vehicle-mounted smart device, and the method includes:

采集视频帧；Capture video frames;

将所述视频帧上传至服务器，以使所述服务器基于所述视频帧，对车辆行为进行识别，得到异常车辆行为的识别结果，并根据所述识别结果，进行车辆行为报警。Uploading the video frame to a server, so that the server recognizes vehicle behavior based on the video frame, obtains a recognition result of abnormal vehicle behavior, and performs a vehicle behavior alarm according to the recognition result.
根据权利要求8所述的方法，其特征在于，所述采集视频帧，包括：The method according to claim 8, wherein the acquiring a video frame comprises:

以所述车载智能设备所属的车辆为中心，设定检测范围，对所述检测范围内的一个或多个车辆拍摄得到多个视频帧。The detection range is set with the vehicle to which the vehicle-mounted smart device belongs as a center, and multiple video frames are obtained by shooting one or more vehicles within the detection range.
根据权利要求8所述的方法，其特征在于，所述采集视频帧，包括：The method according to claim 8, wherein the acquiring a video frame comprises:

按照固定时间间隔，采集多个视频帧。Capture multiple video frames at regular time intervals.
一种车辆行为检测方法，其特征在于，应用于服务器，所述方法包括：A vehicle behavior detection method is characterized in that it is applied to a server, and the method includes:

接收车载智能设备发送的视频帧；Receiving video frames sent by vehicle smart devices;

基于所述视频帧，对车辆行为进行识别，得到异常车辆行为的识别结果；Identifying vehicle behavior based on the video frame to obtain a recognition result of abnormal vehicle behavior;

根据所述识别结果，进行车辆行为报警。According to the recognition result, a vehicle behavior alarm is performed.
根据权利要求11所述的方法，其特征在于，所述基于所述视频帧，对车辆行为进行识别，得到异常车辆行为的识别结果，包括：The method according to claim 11, wherein the identifying vehicle behavior based on the video frame to obtain a recognition result of abnormal vehicle behavior comprises:

将所述视频帧输入神经网络，利用所述神经网络对所述视频帧进行特征抽取，生成多维的行为特征向量；Input the video frame into a neural network, and use the neural network to perform feature extraction on the video frame to generate a multi-dimensional behavior feature vector;

对所述多维的行为特征向量中的每一维采用逻辑分类，得到每一维行为特征向量为不同事件类型的异常车辆行为的置信度；Logically classify each dimension in the multi-dimensional behavioral feature vector to obtain the confidence that the behavioral feature vector in each dimension is an abnormal vehicle behavior of a different event type;

若任一事件类型的异常车辆行为的置信度大于预设阈值，则确定所述视频帧中存在该事件类型的异常车辆行为。If the confidence level of the abnormal vehicle behavior of any event type is greater than a preset threshold, it is determined that the abnormal vehicle behavior of the event type exists in the video frame.
根据权利要求11所述的方法，其特征在于，所述根据所述识别结果，进行车辆行为报警，包括：The method according to claim 11, wherein the performing a vehicle behavior alarm according to the recognition result comprises:

向存在异常车辆行为的车辆、以及与所述车辆的距离在预设范围内的多个车辆发出报警信息。The warning information is issued to a vehicle having abnormal vehicle behavior and a plurality of vehicles having a distance from the vehicle within a preset range.
一种车辆行为检测装置，其特征在于，应用于车载智能设备，所述装置包括：A vehicle behavior detection device, which is characterized in that it is applied to a vehicle-mounted smart device, and the device includes:

采集模块，用于采集视频帧；An acquisition module for acquiring video frames;

数据处理模块，用于基于所述视频帧，对车辆行为进行识别，得到异常车辆行为的识别结果；A data processing module, configured to identify vehicle behavior based on the video frame, and obtain a recognition result of abnormal vehicle behavior;

数据传输模块，用于将所述识别结果上传至服务器。A data transmission module, configured to upload the recognition result to a server.
一种车辆行为检测装置，其特征在于，应用于服务器，所述装置包括：A vehicle behavior detection device is characterized in that it is applied to a server, and the device includes:

接收模块，用于接收车载智能设备发送的识别结果，所述识别结果为所述车载智能设备基于采集的视频帧，对车辆行为进行识别，得到的异常车辆行为的识别结果；A receiving module, configured to receive a recognition result sent by a vehicle-mounted intelligent device, where the recognition result is a recognition result of abnormal vehicle behavior obtained by the vehicle-mounted intelligent device based on the collected video frames;

综合报警模块，用于根据所述识别结果，进行车辆行为报警。A comprehensive alarm module is configured to perform a vehicle behavior alarm according to the recognition result.
一种车辆行为检测装置，其特征在于，应用于车载智能设备，所述装置包括：A vehicle behavior detection device, which is characterized in that it is applied to a vehicle-mounted smart device, and the device includes:

采集模块，用于采集视频帧；An acquisition module for acquiring video frames;

数据传输模块，用于将所述视频帧上传至服务器。A data transmission module is configured to upload the video frame to a server.
一种车辆行为检测装置，其特征在于，应用于服务器，所述装置包括：A vehicle behavior detection device is characterized in that it is applied to a server, and the device includes:

接收模块，用于接收车载智能设备发送的视频帧；A receiving module, configured to receive a video frame sent by a vehicle intelligent device;

数据处理模块，用于基于所述视频帧，对车辆行为进行识别，得到异常车辆行为的识别结果；A data processing module, configured to identify vehicle behavior based on the video frame, and obtain a recognition result of abnormal vehicle behavior;

综合报警模块，用于根据所述识别结果，进行车辆行为报警。A comprehensive alarm module is configured to perform a vehicle behavior alarm according to the recognition result.
一种车辆行为检测***，其特征在于，所述***包括：车载智能设备及服务器；A vehicle behavior detection system, characterized in that the system includes: a vehicle intelligent device and a server;

所述车载智能设备，用于采集视频帧；基于所述视频帧，对车辆行为进行识别，得到异常车辆行为的识别结果；将所述识别结果上传至所述服务器；The in-vehicle intelligent device is used to collect video frames; identify vehicle behaviors based on the video frames to obtain recognition results of abnormal vehicle behaviors; and upload the recognition results to the server;

所述服务器，用于接收所述车载智能设备发送的所述识别结果；根据所述识别结果，进行车辆行为报警。The server is configured to receive the recognition result sent by the in-vehicle smart device; and perform a vehicle behavior alarm according to the recognition result.
一种车辆行为检测***，其特征在于，所述***包括：车载智能设备及服务器；A vehicle behavior detection system, characterized in that the system includes: a vehicle intelligent device and a server;

所述车载智能设备，用于采集视频帧；将所述视频帧上传至所述服务器；The in-vehicle smart device is used to collect video frames; upload the video frames to the server;

所述服务器，用于接收所述车载智能设备发送的所述视频帧；基于所述视频帧，对车辆行为进行识别，得到异常车辆行为的识别结果；根据所述识别结果，进行车辆行为报警。The server is configured to receive the video frame sent by the in-vehicle smart device; identify vehicle behavior based on the video frame to obtain a recognition result of abnormal vehicle behavior; and perform a vehicle behavior alarm based on the recognition result.
一种车载智能设备，其特征在于，包括处理器和机器可读存储介质，所述机器可读存储介质存储有能够被所述处理器执行的机器可执行指令，所述处理器被所述机器可执行指令促使执行权利要求1-5任一项所述的方法。An in-vehicle smart device, comprising a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions executable by the processor, and the processor is used by the machine The executable instructions cause the method of any one of claims 1-5 to be performed.
一种机器可读存储介质，其特征在于，存储有机器可执行指令，在被处理器调用和执行时，所述机器可执行指令促使所述处理器执行权利要求1-5任一项所述的方法。A machine-readable storage medium, characterized in that it stores machine-executable instructions, and when called and executed by a processor, the machine-executable instructions cause the processor to execute any one of claims 1-5 Methods.
一种应用程序，其特征在于，用于在运行时执行：权利要求1-5任一项所述的方法。An application program, which is configured to execute at runtime: the method according to any one of claims 1-5.
一种服务器，其特征在于，包括处理器和机器可读存储介质，所述机器可读存储介质存储有能够被所述处理器执行的机器可执行指令，所述处理器被所述机器可执行指令促使执行权利要求6或7所述的方法。A server includes a processor and a machine-readable storage medium, where the machine-readable storage medium stores machine-executable instructions executable by the processor, and the processor is executable by the machine The instructions cause the method of claim 6 or 7 to be performed.
一种机器可读存储介质，其特征在于，存储有机器可执行指令，在被处理器调用和执行时，所述机器可执行指令促使所述处理器执行权利要求6或7所述的方法。A machine-readable storage medium, characterized in that it stores machine-executable instructions, and when called and executed by a processor, the machine-executable instructions cause the processor to execute the method according to claim 6 or 7.
一种应用程序，其特征在于，用于在运行时执行：权利要求6或7所述的方法。An application program, which is used to execute the method according to claim 6 or 7 at runtime.
一种车载智能设备，其特征在于，包括处理器和机器可读存储介质，所述机器可读存储介质存储有能够被所述处理器执行的机器可执行指令，所述处理器被所述机器可执行指令促使执行权利要求8-10任一项所述的方法。An in-vehicle smart device, comprising a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions executable by the processor, and the processor is used by the machine The executable instructions cause the method according to any one of claims 8-10 to be performed.
一种机器可读存储介质，其特征在于，存储有机器可执行指令，在被处理器调用和执行时，所述机器可执行指令促使所述处理器执行权利要求8-10任一项所述的方法。A machine-readable storage medium, characterized in that it stores machine-executable instructions, and when called and executed by a processor, the machine-executable instructions cause the processor to execute any one of claims 8-10 Methods.
一种应用程序，其特征在于，用于在运行时执行：权利要求8-10任一项所述的方法。An application program, which is configured to execute at runtime: the method according to any one of claims 8-10.
一种服务器，其特征在于，包括处理器和机器可读存储介质，所述机器可读存储介质存储有能够被所述处理器执行的机器可执行指令，所述处理器被所述机器可执行指令促使执行权利要求11-13任一项所述的方法。A server includes a processor and a machine-readable storage medium, where the machine-readable storage medium stores machine-executable instructions executable by the processor, and the processor is executable by the machine The instructions cause the method of any one of claims 11-13 to be performed.
一种机器可读存储介质，其特征在于，存储有机器可执行指令，在被处理器调用和执行时，所述机器可执行指令促使所述处理器执行权利要求11-13任一项所述的方法。A machine-readable storage medium, characterized in that it stores machine-executable instructions, and when called and executed by a processor, the machine-executable instructions cause the processor to execute any one of claims 11-13 Methods.
一种应用程序，其特征在于，用于在运行时执行：权利要求11-13任一项所述的方法。An application program, which is used to execute at runtime: the method according to any one of claims 11-13.