WO2020238284A1

WO2020238284A1 - Parking space detection method and apparatus, and electronic device

Info

Publication number: WO2020238284A1
Application number: PCT/CN2020/075065
Authority: WO
Inventors: 王哲; 丁明宇; 石建萍; 何宇帆
Original assignee: 北京市商汤科技开发有限公司
Priority date: 2019-05-29
Filing date: 2020-02-13
Publication date: 2020-12-03
Also published as: JP2022510329A; CN112016349A; KR20210087070A

Abstract

A parking space detection method and apparatus, and an electronic device. The method comprises: acquiring a parking space image (S101); inputting the parking space image into a neural network to obtain regional information and/or corner information of a free parking space in the parking space image (S102); and determining, on the basis of the regional information and/or corner information of the free parking space in the parking space image, a detection result of the free parking space in the parking space image (S103).

Description

停车位的检测方法、装置与电子设备Parking space detection method, device and electronic equipment

相关申请的交叉引用Cross references to related applications

本申请基于申请号为201910458754.4、申请日为2019年5月29日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此以引入方式并入本申请。This application is based on a Chinese patent application with an application number of 201910458754.4 and an application date of May 29, 2019, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby incorporated into this application by way of introduction.

技术领域Technical field

本申请涉及人工智能技术，尤其是一种停车位的检测方法、装置与电子设备。This application relates to artificial intelligence technology, especially a parking space detection method, device and electronic equipment.

背景技术Background technique

随着人们生活水平的提高，汽车成为生活中必不可少的交通工具，随着计算机视觉的发展，智能驾驶得到了广泛的关注。智能驾驶中一个关键任务为停车位的检测，停车位的检测器目的是自动寻找空闲车位，并将车辆停放在空闲车位上。With the improvement of people's living standards, cars have become an indispensable means of transportation in life. With the development of computer vision, intelligent driving has received extensive attention. A key task in intelligent driving is the detection of parking spaces. The purpose of the parking space detector is to automatically find free parking spaces and park the vehicle on the free parking spaces.

发明内容Summary of the invention

本申请实施例提供一种停车位的检测方法、装置与电子设备。The embodiments of the present application provide a parking space detection method, device and electronic equipment.

第一方面，本申请实施例提供一种停车位的检测方法，所述方法包括：获取停车位图像；将所述停车位图像输入实例分割神经网络中，获得所述停车位图像中空闲停车位的区域信息和/或角点信息；基于所述停车位图像中空闲停车位的区域信息和/或角点信息，确定所述停车位图像中空闲停车位的检测结果。In a first aspect, an embodiment of the present application provides a parking space detection method, the method includes: obtaining a parking space image; inputting the parking space image into an instance segmentation neural network to obtain free parking spaces in the parking space image Based on the area information and/or corner information of the free parking space in the parking space image, determine the detection result of the free parking space in the parking space image.

第二方面，本申请实施例提供一种停车位的检测装置，所述装置包括：In a second aspect, an embodiment of the present application provides a parking space detection device, the device including:

第一获取模块，用于获取停车位图像；The first acquisition module is used to acquire parking space images;

处理模块，用于将所述停车位图像输入神经网络中，获得所述停车位图像中空闲停车位的区域信息和/或角点信息；A processing module, configured to input the parking space image into a neural network to obtain area information and/or corner point information of free parking spaces in the parking space image;

确定模块，用于基于所述停车位图像中空闲停车位的区域信息和/或角点信息，确定所述停车位图像中空闲停车位的检测结果。The determining module is configured to determine the detection result of the free parking space in the parking space image based on the area information and/or corner point information of the free parking space in the parking space image.

第三方面，本申请实施例提供一种电子设备，包括：存储器，用于存储计算机程序；处理器，用于执行所述计算机程序，以实现第一方面任一项所述的停车位的检测方法。In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, configured to store a computer program; a processor, configured to execute the computer program to realize the parking space detection according to any one of the first aspect method.

第四方面，本申请实施例提供一种计算机存储介质，所述存储介质中存储计算机程序，所述计算机程序在执行时实现第一方面任一项所述的停车位的检测方法。In a fourth aspect, an embodiment of the present application provides a computer storage medium in which a computer program is stored, and the computer program implements the parking space detection method described in any one of the first aspect when executed.

本申请实施例提供的停车位的检测方法、装置与电子设备，通过获取停车位图像，将停车位图像输入到神经网络中，获得所述停车位图像中空闲停车位的区域信息和/或角点信息；基于所述停车位图像中空闲停车位的区域信息和/或角点信息，确定所述停车位图像中空闲停车位的检测结果。即本申请实施例的检测方法，只需要将获得的停车位图像输入到神经网络中，即可得到准确的空闲停车位的区域信息和/或角点信息，不需要前期的图像处理，整个检测过程简单，耗时短，且基于停车位图像中空闲停车位的区域信息和/或角点信息，来确定出停车位图像中空闲停车位的检测结果，有效提高了空闲停车位的检测准确性。The parking space detection method, device and electronic equipment provided by the embodiments of the present application obtain the parking space image and input the parking space image into the neural network to obtain the area information and/or angle of the free parking space in the parking space image Point information; based on the area information and/or corner point information of the free parking space in the parking space image, determine the detection result of the free parking space in the parking space image. That is to say, the detection method of the embodiment of the present application only needs to input the obtained parking space image into the neural network to obtain the accurate area information and/or corner point information of the free parking space, without the need for pre-image processing, the entire detection The process is simple and time-consuming, and based on the area information and/or corner information of the free parking space in the parking space image, the detection result of the free parking space in the parking space image is determined, which effectively improves the detection accuracy of the free parking space .

附图说明Description of the drawings

为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。In order to more clearly describe the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

图1为本申请实施例提供的停车位检测方法的流程示意图一；FIG. 1 is a schematic flow chart 1 of a parking space detection method provided by an embodiment of this application;

图2为停车位的一种示例图；Figure 2 is an example diagram of parking spaces;

图3为本申请实施例提供的停车位检测方法的流程示意图二；FIG. 3 is a second schematic flowchart of a parking space detection method provided by an embodiment of the application;

图4a为本申请实施例使用的停车位训练图像的一种示例图；Fig. 4a is an example diagram of a parking space training image used in an embodiment of the application;

图4b为对图4a进行关键点标注后的图像；Fig. 4b is an image after the key points of Fig. 4a are marked;

图5为本申请实施例涉及的实例分割网络的一种训练流程图；FIG. 5 is a training flowchart of the instance segmentation network involved in an embodiment of this application;

图6为本申请实施例涉及的实例分割网络的一种结构示意图；FIG. 6 is a schematic structural diagram of an instance segmentation network involved in an embodiment of this application;

图7为本申请实施例涉及的停车位检测结果示意图；FIG. 7 is a schematic diagram of a parking space detection result related to an embodiment of the application;

图8为本申请实施例提供的停车位的检测装置的结构示意图一；FIG. 8 is a first structural diagram of a parking space detection device provided by an embodiment of the application;

图9为本申请实施例提供的停车位的检测装置的结构示意图二；FIG. 9 is a second structural diagram of a parking space detection device provided by an embodiment of the application;

图10为本申请实施例提供的停车位的检测装置的结构示意图三；FIG. 10 is a third structural diagram of a parking space detection device provided by an embodiment of the application;

图11为本申请实施例提供的电子设备的结构示意图。FIG. 11 is a schematic structural diagram of an electronic device provided by an embodiment of the application.

具体实施方式Detailed ways

为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments It is a part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of this application.

本申请实施例可以应用于终端设备、计算机***、服务器等电子设备，其可与众多其它通用或专用计算***环境或配置一起操作。适于与终端设备、计算机***、服务器等电子设备一起使用的众所周知的终端设备、计算***、环境和/或配置的例子包括但不限于以下至少之一：个人计算机***，服务器计算机***，瘦客户机，厚客户机，手持或膝上设备，基于微处理器、中央处理器(Central Processing Unit，CPU)、图形处理器(Graphics Processing Unit，GPU)的***，车载***，机顶盒，可编程消费电子产品，网络个人电脑，小型计算机***，大型计算机***和包括上述任何***的分布式云计算技术环境，等等。The embodiments of the present application can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with many other general or special computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments and/or configurations suitable for use with electronic devices such as terminal devices, computer systems, servers, etc. include but are not limited to at least one of the following: personal computer systems, server computer systems, thin clients Computers, thick clients, handheld or laptop devices, systems based on microprocessors, central processing units (CPU), graphics processing units (GPUs), vehicle systems, set-top boxes, programmable consumer electronics Products, network personal computers, small computer systems, large computer systems and distributed cloud computing technology environments including any of the above systems, etc.

终端设备、计算机***、服务器等电子设备可以在由计算机***执行的计算机***可执行指令(诸如程序模块)的一般语境下描述。通常，程序模块可以包括例程、程序、目标程序、组件、逻辑、数据结构等等，它们执行特定的任务或者实现特定的抽象数据类型。计算机***/服务器可以在分布式云计算环境中实施，分布式云计算环境中，任务是由通过通信网络链接的远程处理设备执行的。在分布式云计算环境中，程序模块可以位于包括存储设备的本地或远程计算***存储介质上。Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by the computer system. Generally, program modules may include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. The computer system/server can be implemented in a distributed cloud computing environment. In the distributed cloud computing environment, tasks are executed by remote processing equipment linked through a communication network. In a distributed cloud computing environment, program modules may be located on storage media of local or remote computing systems including storage devices.

在一种示例性的应用中，将上述电子设备设置在车辆上，可以与倒车***连接，用于辅助倒车***将车辆停放在空闲的停车位上。在另一种示例性的应用中，将电子设备与辅助驾驶***连接，电子设备将获得到的空闲停车位的检测结果发送给辅助驾驶***，以使辅助驾驶***根据该空闲停车位的检测结果来控制车辆行驶，例如，控制车辆停放至空闲停车位上。可选的，该电子设备还可以直接为辅助驾驶***的一部分或全部，或者为倒车***的一部分或全部。可选的，该电子设备还可以根据实际需要与其他的车辆控制***连接，本申请实施例对此不做限制。In an exemplary application, the above-mentioned electronic device is installed on a vehicle and can be connected to a reversing system to assist the reversing system to park the vehicle in a free parking space. In another exemplary application, the electronic device is connected to the driving assistance system, and the electronic device sends the obtained detection result of the free parking space to the driving assistance system, so that the driving assistance system can according to the detection result of the free parking space To control the driving of the vehicle, for example, control the vehicle to park in a free parking space. Optionally, the electronic device can also be directly part or all of the driving assistance system, or part or all of the reversing system. Optionally, the electronic device can also be connected with other vehicle control systems according to actual needs, which is not limited in the embodiment of the present application.

下面以具体地实施例对本申请的技术方案进行详细说明。下面这几个具体的实施例可以相互结合，对于相同或相似的概念或过程可能在某些实施例不再赘述。The technical solution of the present application will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

图1为本申请实施例提供的停车位的检测方法的流程示意图一。如图1所示，本实施例的方法可以包括：FIG. 1 is a schematic diagram 1 of a flow chart of a parking space detection method provided by an embodiment of the application. As shown in Figure 1, the method of this embodiment may include:

S101、获取停车位图像。S101. Acquire a parking space image.

本实施例以执行主体为电子设备为例进行说明，该电子设备可以但不限于是智能手机、计算机、车载***等。In this embodiment, the execution subject is an electronic device as an example for description. The electronic device may be, but is not limited to, a smart phone, a computer, a vehicle-mounted system, and the like.

可选的，图2为停车位的一种示例图，本实施例的电子设备还可以具有摄像头，通过摄像头能够拍摄车辆的行驶环境，例如通过摄像头可以拍摄车辆所行驶的道路周围的停车位的图像，得到停车位图像，并将该停车位图像发送给电子设备的处理器，以使处理器执行本实施例的方法，得到停车位图像中空闲停车位的检测结果。Optionally, FIG. 2 is an example diagram of a parking space. The electronic device of this embodiment may also have a camera, through which the driving environment of the vehicle can be photographed. For example, the camera can photograph the parking space around the road on which the vehicle is traveling. Image, the parking space image is obtained, and the parking space image is sent to the processor of the electronic device, so that the processor executes the method of this embodiment to obtain the detection result of the free parking space in the parking space image.

可选的，本实施例的电子设备可以与外部的摄像头连接，通过外部的摄像头拍摄车辆的行驶环境，从而获得停车位图像。Optionally, the electronic device of this embodiment may be connected to an external camera, and the driving environment of the vehicle is captured by the external camera, so as to obtain a parking space image.

可选的，本申请实施例的摄像头的成像组件可以但不限于为互补金属氧化物半导体(Complementary Metal Oxide Semiconductor，CMOS)或电荷耦合器件(Charge Coupled Device，CCD)。Optionally, the imaging component of the camera in the embodiment of the present application may be, but is not limited to, a complementary metal oxide semiconductor (Complementary Metal Oxide Semiconductor, CMOS) or a charge coupled device (Charge Coupled Device, CCD).

S102、将所述停车位图像输入神经网络中，获得所述停车位图像中空闲停车位的区域信息和/或角点信息。S102. Input the parking space image into a neural network to obtain area information and/or corner point information of an idle parking space in the parking space image.

本申请实施例的神经网络包括但不限于反向传播(Back Propagation，BP) 神经网络、径向基函数(Radial Basis Function，RBF)神经网络、感知器神经网络、线性神经网络、反馈神经网络等。The neural networks in the embodiments of the present application include but are not limited to Back Propagation (BP) neural networks, Radial Basis Function (RBF) neural networks, perceptron neural networks, linear neural networks, feedback neural networks, etc. .

可选的，上述神经网络可以实现实例分割，其中，实例分割是指不仅进行像素级别的分类，还需在具体的类别基础上区别开不同的实例。例如，停车位图像中有多个空闲停车位甲、乙、丙，实例分割可以将这3个空闲停车位识别为不同的对象。Optionally, the aforementioned neural network may implement instance segmentation, where instance segmentation refers to not only pixel-level classification, but also different instances need to be distinguished on the basis of specific categories. For example, there are multiple free parking spaces A, B, and C in the parking space image, and instance segmentation can identify these 3 free parking spaces as different objects.

本申请实施例通过神经网络可以检测出停车位图像中空闲停车位的区域信息，和/或，空闲停车位的角点信息。例如，预先通过带有空闲停车位的区域信息和/或空闲停车位的角点信息的停车位图像集对神经网络进行训练，使得神经网络学习到提取空闲停车位的区域信息和/或角点信息的能力，这样可将图2所示的停车位图像输入至神经网络中，可以通过神经网络对停车位图像的处理而输出得到的该停车位图像中空闲停车位的区域信息和/或角点信息。In the embodiment of the present application, the area information of the free parking space in the parking space image and/or the corner point information of the free parking space can be detected through the neural network. For example, the neural network is trained in advance through the parking space image set with the area information of the free parking space and/or the corner point information of the free parking space, so that the neural network learns to extract the area information and/or corner point of the free parking space Information capability, so that the parking space image shown in Figure 2 can be input to the neural network, and the area information and/or angle of the free parking space in the parking space image can be output through the neural network processing the parking space image Point information.

其中，空闲停车位的区域信息可以包括该空闲停车位的位置和大小等信息；角点信息包括空闲停车位的角点的位置信息。The area information of the free parking space may include information such as the position and size of the free parking space; the corner point information includes the position information of the corner points of the free parking space.

可选的，空闲停车位的角点信息可包括该空闲停车位的至少3个角点的角点信息。由于停车位通常为长方形，则可以根据空闲停车位的至少3个角点的角点信息确定出该空闲停车位的区域信息。Optionally, the corner point information of the free parking space may include corner point information of at least three corner points of the free parking space. Since the parking space is usually rectangular, the area information of the free parking space can be determined according to the corner point information of at least three corner points of the free parking space.

本申请实施例的检测方法，只需要将获得的停车位图像输入到神经网络中，即可得到准确的空闲停车位的区域信息和/或角点信息，不需要前期的图像处理，整个检测过程简单，耗时短。The detection method of the embodiment of the application only needs to input the obtained parking space image into the neural network to obtain the accurate area information and/or corner point information of the free parking space, without the need for pre-image processing, and the entire detection process Simple and time-consuming.

S103、基于所述停车位图像中空闲停车位的区域信息和/或角点信息，确定所述停车位图像中空闲停车位的检测结果。S103: Determine a detection result of the free parking space in the parking space image based on the area information and/or corner point information of the free parking space in the parking space image.

在一种示例中，可以将停车图像中空闲停车的区域信息作为空闲停车位的检测结果。在另一种示例中，可以将停车图像中空闲停车的角点信息作为空闲停车位的检测结果。In an example, the free parking area information in the parking image can be used as the detection result of the free parking space. In another example, the corner point information of the free parking in the parking image can be used as the detection result of the free parking space.

在又一种示例中，所述基于所述停车位图像中空闲停车位的区域信息和角点信息，确定所述停车位图像中空闲停车位的检测结果，包括：将停车位图像中空闲停车位的区域信息和角点信息进行融合，确定停车位图像中空闲停车位的检测结果。其中，所述将空闲停车位的区域信息和角点信息进行融合，确定所述空闲停车位的检测结果的方式包括但不限于如下方式：In another example, the determining the detection result of the free parking space in the parking space image based on the area information and corner point information of the free parking space in the parking space image includes: parking the free parking space in the parking space image The area information of the space and the corner point information are merged to determine the detection result of the free parking space in the parking space image. Wherein, the method of fusing the area information and corner point information of the free parking space to determine the detection result of the free parking space includes but is not limited to the following methods:

方式一，确定停车位图像中空闲停车位的角点信息构成的停车位区域信息；将停车位图像中空闲停车位的区域信息和角点信息构成的停车位区域信息进行融合，基于融合后的区域信息确定停车位图像中空闲停车位的检测结果。例如，将空闲停车位的角点信息围绕成的停车位区域信息记为停车位区域信息1，将停车位图像中空闲停车位的区域信息记为区域信息2，将停车位区域信息1与区域信息2进行融合得到一个区域信息3，例如将停车位区域信息1与区域信息2的平均值作为区域信息3，将融合后的区域信息3作为停车位图像中空闲停车位的检测结果。Method 1: Determine the parking space area information composed of the corner information of the free parking space in the parking space image; merge the area information of the free parking space in the parking space image and the parking space area information composed of the corner information, based on the fusion The area information determines the detection result of the free parking space in the parking space image. For example, the parking space area information surrounded by the corner point information of the free parking space is recorded as parking space area information 1, the area information of the free parking space in the parking space image is recorded as area information 2, and the parking space area information 1 and area The information 2 is fused to obtain a piece of area information 3. For example, the average value of the parking space area information 1 and the area information 2 is used as the area information 3, and the merged area information 3 is used as the detection result of the free parking space in the parking space image.

方式二，确定停车位图像中空闲停车位的区域信息的角点信息；将停车位图像中空闲停车位的角点信息和区域信息的角点信息进行融合，基于融合后的角点信息确定停车位图像中空闲停车位的检测结果。例如，停车位图像中空闲停车位的区域信息对应的角点分别记为角点a1、角点a2、角点a3和角点a4，停车位图像中空闲停车位的角点信息对应的角点分别记为角点b1、角点b2、角点b3和角点b4，其中，角点a1、角点a2、角点a3、角点a4分别与角点b1、角点b2、角点b3、角点b4一一对应，这样，可以将对应的两个角点融合为一个角点，例如，将角点a1与角点b1融合为一个角点ab1，这样可以获得新的角点信息，将这些新的角点信息作为停车位图像中空闲停车位的检测结果。Method 2: Determine the corner information of the free parking space in the parking space image; merge the corner information of the free parking space in the parking space image and the corner information of the area information, and determine the parking based on the fused corner information The detection result of the free parking space in the bit image. For example, the corner points corresponding to the area information of the free parking space in the parking space image are respectively marked as corner point a1, corner point a2, corner point a3, and corner point a4, and the corner points corresponding to the corner point information of the free parking space in the parking space image They are respectively denoted as corner point b1, corner point b2, corner point b3, and corner point b4, where corner point a1, corner point a2, corner point a3, and corner point a4 are connected to corner point b1, corner point b2, corner point b3, The corner point b4 has a one-to-one correspondence. In this way, the two corresponding corner points can be merged into one corner point. For example, the corner point a1 and the corner point b1 are merged into a corner point ab1, so that new corner point information can be obtained. The new corner point information is used as the detection result of the free parking space in the parking space image.

本申请实施例，通过将停车位图像中空闲停车位的区域信息和角点信息进行融合，确定停车位图像中空闲停车位的检测结果，可以提高空闲停车位的检测准确性。In the embodiment of the present application, the detection result of the free parking space in the parking space image can be determined by fusing the area information of the free parking space in the parking space image and the corner point information, which can improve the detection accuracy of the free parking space.

可选的，本申请实施例的方法可以在车辆寻找停车位时，才执行上述步骤。例如，智能驾驶***到达目的地，或者接收到外部发送的寻找停车位指令时，智能驾驶***控制电子设备工作。在上述场景下，若电子设备具有摄像头，则该电子设备中的处理器控制摄像头拍摄车辆周围的停车位图像，若电子设备没有摄像头，则电子设备向外部的摄像头发送拍照指令，以使摄像头将拍摄的车辆周围的停车位图像发送给电子设备。电子设备在获得停车位图像后，对该停车位图像进行处理，检测出停车位图像中空闲停车位的检测结果。例如，电子设备将获得的停车位图像输入到神经网络中，通过该神经网络的处理，输出停车位图像中空闲停车位的区域信息和/或角点信息，再基于停车位图像中空闲停车位的区域信息和/或角点信息确定停车位图像中空闲停车位的检测结果，进而实现对空闲停车位的准确检测。Optionally, the method of the embodiment of the present application may execute the foregoing steps only when the vehicle is looking for a parking space. For example, when the smart driving system arrives at the destination, or when it receives an instruction to find a parking space sent from the outside, the smart driving system controls the electronic equipment to work. In the above scenario, if the electronic device has a camera, the processor in the electronic device controls the camera to capture images of parking spaces around the vehicle. If the electronic device does not have a camera, the electronic device sends a photo command to the external camera so that the camera will The captured image of the parking space around the vehicle is sent to the electronic device. After obtaining the parking space image, the electronic device processes the parking space image to detect the detection result of the free parking space in the parking space image. For example, the electronic device inputs the obtained parking space image into the neural network, and through the processing of the neural network, outputs the area information and/or corner point information of the free parking space in the parking space image, and then based on the free parking space in the parking space image The area information and/or corner point information of the image determine the detection result of the free parking space in the parking space image, and then realize the accurate detection of the free parking space.

可选的，该电子设备还与智能驾驶***连接，可以将空闲停车位的检测结果发送给智能驾驶***，智能驾驶***根据该空闲停车位的检测结果，控制车辆停放在该空闲停车位上。Optionally, the electronic device is also connected to the intelligent driving system, and can send the detection result of the idle parking space to the intelligent driving system, and the intelligent driving system controls the vehicle to park in the idle parking space according to the detection result of the idle parking space.

本申请实施例提供的停车位检测方法，通过获取停车位图像，将停车位图像输入到神经网络中，获得所述停车位图像中空闲停车位的区域信息和/或角点信息；基于所述停车位图像中空闲停车位的区域信息和/或角点信息，确定所述停车位图像中空闲停车位的检测结果。即本申请实施例的检测方法，只需要将获得的停车位图像输入到神经网络中，即可得到准确的空闲停车位的区域信息和/或角点信息，不需要前期的图像处理，整个检测过程简单，耗时短，且基于停车位图像中空闲停车位的区域信息和/或角点信息，来确定出停车位图像中空闲停车位的检测结果，有效提高了空闲停车位的检测准确性。In the parking space detection method provided by the embodiments of the present application, the parking space image is obtained by inputting the parking space image into the neural network to obtain the area information and/or corner point information of the free parking space in the parking space image; The area information and/or corner point information of the free parking space in the parking space image determines the detection result of the free parking space in the parking space image. That is to say, the detection method of the embodiment of the present application only needs to input the obtained parking space image into the neural network to obtain the accurate area information and/or corner point information of the free parking space, without the need for pre-image processing, the entire detection The process is simple and time-consuming, and based on the area information and/or corner information of the free parking space in the parking space image, the detection result of the free parking space in the parking space image is determined, which effectively improves the detection accuracy of the free parking space .

在一些实施例中，为了检测出停车位图像中不完整的空闲停车位，则本申请实施例的方法，在S102将所述停车位图像输入神经网络中，获得所述停车位图像中空闲停车位的区域信息和/或角点信息之前，还包括：In some embodiments, in order to detect an incomplete free parking space in a parking space image, the method of the embodiment of the present application inputs the parking space image into the neural network in S102 to obtain the free parking space in the parking space image Before the location information and/or corner point information, it also includes:

S102a、在所述停车位图像的四周边缘向外扩充预设值。S102a: Extend a preset value outward at the peripheral edges of the parking space image.

可选的，所述预设值小于或等于停车位长度的一半。Optionally, the preset value is less than or equal to half the length of the parking space.

示例性的，可参照图4a和图4b所示，假设图4a为电子设备获取的停车位图像，该停车位图像中包括两个空闲停车位，空闲停车位1和空闲停车位2，其中，停车位图像中仅包括空闲停车位2的局部。为了检测到空闲停车位2，则将图4a所示的停车位图像的四周边缘向外扩充预设值，如图4b中的黑色边框所示，获得图4b所示的结果。这样，可以增大停车位图像的视角范围，可以检测出局部位于停车位图像外的空闲停车位，进一步增大了停车位检测的准确性。Exemplarily, referring to Figures 4a and 4b, suppose that Figure 4a is a parking space image acquired by an electronic device. The parking space image includes two free parking spaces, free parking space 1 and free parking space 2, where, Only a part of the free parking space 2 is included in the parking space image. In order to detect the free parking space 2, the peripheral edges of the parking space image shown in FIG. 4a are expanded outward by a preset value, as shown by the black border in FIG. 4b, and the result shown in FIG. 4b is obtained. In this way, the viewing angle range of the parking space image can be increased, and free parking spaces partially located outside the parking space image can be detected, which further increases the accuracy of parking space detection.

此时，上述S103将停车位图像输入神经网络中，获得所述停车位图像中空闲停车位的区域信息和/或角点信息，可以被S103a替换：At this time, the above S103 inputs the parking space image into the neural network to obtain the area information and/or corner point information of the free parking space in the parking space image, which can be replaced by S103a:

S103a、将扩充后的所述停车位图像输入所述神经网络中，获得所述停车位图像中空闲停车位的区域信息和/或角点信息。S103a: Input the expanded parking space image into the neural network, and obtain area information and/or corner point information of free parking spaces in the parking space image.

例如，将图4b输入至神经网络，可以检测出图4b中空闲停车位1和空闲停车位2的区域信息，和/或空闲停车位1和空闲停车位2的角点信息。For example, inputting Fig. 4b to the neural network can detect the area information of free parking space 1 and free parking space 2 in Fig. 4b, and/or the corner point information of free parking space 1 and free parking space 2 in Fig. 4b.

本申请实施例的方法，在检测停车位图像中空闲停车位的区域信息和/或角点信息之前，在停车位图像的四周边缘向外扩充预设值，再将扩充后的所述停车位图像输入神经网络中，这样可以检测出局部位于停车位图像外的空闲停车位，进一步提高了停车位检测的准确性和实用性。In the method of the embodiment of the present application, before detecting the area information and/or corner point information of the free parking space in the parking space image, the preset value is expanded outward on the periphery of the parking space image, and then the expanded parking space The image is input into the neural network, so that it can detect the free parking space that is partially outside the parking space image, which further improves the accuracy and practicability of parking space detection.

图3为本申请实施例提供的停车位检测方法的流程示意图二，在上述实施例的基础上，本申请实施例的方法还包括对神经网络进行训练的过程，如图3所示，训练过程包括：FIG. 3 is a schematic diagram of the second flow of the parking space detection method provided by the embodiment of the application. On the basis of the above-mentioned embodiment, the method of the embodiment of this application also includes a process of training a neural network. As shown in FIG. 3, the training process include:

S201、获取多个停车位训练图像。S201. Acquire multiple parking space training images.

该多个停车位训练图像可以是电子设备从数据库中获得，也可以为电子设备以往拍摄的，本申请实施例对电子设备获取多个停车位训练图像的具体过程不做限制。The multiple parking space training images may be obtained by the electronic device from the database, or may be taken by the electronic device in the past. The embodiment of the present application does not limit the specific process of obtaining multiple parking space training images by the electronic device.

其中，每个停车位图像包括一个或多个空闲停车位，例如图4b为停车位训练图像，其中包括空闲停车位1和空闲停车位2。Wherein, each parking space image includes one or more free parking spaces. For example, FIG. 4b is a parking space training image, which includes free parking space 1 and free parking space 2.

可选的，上述停车位训练图像可以为使用广角摄像头采集的图像，该图像具有一定的扭曲度。使用该广角摄像头拍摄的停车位训练图像来训练神经网络时，可以使得训练后神经网络对不同视角拍摄的停车位图像进行预测，进而在保证预测准确性的提前下，降低了对停车位图像的拍摄要求。Optionally, the aforementioned parking space training image may be an image collected using a wide-angle camera, and the image has a certain degree of distortion. When using the parking space training images taken by the wide-angle camera to train the neural network, the neural network can predict parking space images taken from different perspectives after training, and then reduce the cost of parking space images while ensuring the accuracy of the prediction. Shooting requirements.

空闲停车位的关键点可以包括停车位边线上的点、空闲停车位的角点，或者是空闲停车位两对角线的交点，根据这些关键点，可以准确获得空闲停车位的区域以及位置。The key points of the free parking space may include points on the edge of the parking space, the corner points of the free parking space, or the intersection of two diagonals of the free parking space. According to these key points, the area and location of the free parking space can be accurately obtained.

上述停车位训练图像包括有对空闲停车位的关键点信息的标注信息。例如图4b所示，标注有空闲停车位4的关键点为：关键点1、关键点2、关键点3和关键点4。需要说明的是，图4b为空闲停车位1的关键点的一种标注方式，空闲停车位1的关键点包括但不限于上述4个关键点，空闲停车位1的关键点的具体数量和选取方式根据实际需要确定，本申请实施例对此不做限制。The above-mentioned parking space training image includes tagging information of key point information of the free parking space. For example, as shown in Figure 4b, the key points marked with free parking space 4 are: key point 1, key point 2, key point 3, and key point 4. It should be noted that Figure 4b is a way of marking the key points of free parking space 1. The key points of free parking space 1 include but are not limited to the above 4 key points. The specific number and selection of key points of free parking space 1 The manner is determined according to actual needs, which is not limited in the embodiment of the present application.

S202、使用所述多个停车位训练图像，训练所述神经网络。S202. Use the multiple parking space training images to train the neural network.

将包括有对空闲停车位的关键点信息的标注信息的多个停车位训练图像输入神经网络中，以基于神经网络输入的检测结果和对空闲停车位的关键点信息的标注信息之间的差异调整该神经网络的网络参数，完成该神经网络的训练。Input a plurality of parking space training images including the annotation information of the key point information of the free parking space into the neural network, based on the difference between the detection result of the neural network input and the annotation information of the key point information of the free parking space Adjust the network parameters of the neural network to complete the training of the neural network.

在一种可能的实现方式中，为了实现神经网络对停车位图像中不完整的空闲停车位的检测，在神经网络的训练过程中，所使用的停车位训练图像的四周边缘向外扩充预设值。可选的，上述预设值小于或等于停车位长度的一半。In a possible implementation, in order to realize the detection of incomplete free parking spaces in the parking space image by the neural network, in the training process of the neural network, the peripheral edges of the parking space training image used are expanded outward. value. Optionally, the aforementioned preset value is less than or equal to half the length of the parking space.

继续参照图4b所示，将图4b所示的停车位训练图像的四周边缘向外扩充预设值，所述预设值小于或等于停车位长度的一半，如图4b中的黑色边框所示。这样，可以增大停车位训练图像的视角范围，目的是为了使训练后的神经网络在后续停车位检测时，可以检测出局部位于拍摄的停车位图像外的空闲停车位。Continuing to refer to Figure 4b, the peripheral edges of the parking space training image shown in Figure 4b are expanded outward by a preset value, which is less than or equal to half the length of the parking space, as shown by the black border in Figure 4b . In this way, the viewing angle range of the parking space training image can be increased, and the purpose is to enable the trained neural network to detect the free parking space partially outside the captured parking space image during subsequent parking space detection.

将图4b所示的停车位训练图像的四周边缘向外扩充预设值后，图4b中的空闲停车位2可以完整显示，这样可以对空闲停车位2的关键点进行标注，例如，标注有空闲停车位2的关键点为：关键点11、关键点12、关键点13、关键点14、关键点15和关键点16，其中关键点14位于扩充区域内。需要说明的是，图4b仅为空闲停车位2的关键点信息的一种可能标注方式，空闲停车位2的关键点包括但不限于上述6个关键点，空闲停车位2的关键点的具体数量和选取方式根据实际需要确定，本申请实施例对此不做限制。After expanding the peripheral edges of the parking space training image shown in Figure 4b outward by the preset value, the free parking space 2 in Figure 4b can be displayed completely, so that the key points of the free parking space 2 can be marked, for example, marked with The key points of the free parking space 2 are: key point 11, key point 12, key point 13, key point 14, key point 15 and key point 16, among which key point 14 is located in the expansion area. It should be noted that Figure 4b is only a possible way of marking the key point information of the free parking space 2. The key points of the free parking space 2 include but are not limited to the above 6 key points. The specific key points of the free parking space 2 The quantity and selection method are determined according to actual needs, which are not limited in the embodiment of the present application.

可选的，上述空闲停车位1和空闲停车位2的关键点的数量和选取方式可以相同，也可以不同，只要保证空闲停车位1的各关键点依次连接包围的面积为空闲停车位1的面积，空闲停车位2的各关键点依次连接包围的面积为空闲停车位2的面积即可。Optionally, the number and selection method of the key points of free parking space 1 and free parking space 2 can be the same or different, as long as it is ensured that the key points of free parking space 1 are connected to the area surrounded by free parking space 1 in turn. Area, the key points of the free parking space 2 are connected to the area enclosed by the free parking space 2 in turn.

图4b中，空闲停车位1的关键点3和4与空闲停车位2的关键点13和11重合。In Figure 4b, the key points 3 and 4 of the free parking space 1 coincide with the key points 13 and 11 of the free parking space 2.

可选的，停车位训练图像中空闲停车位的关键点信息包括所述空闲停车位的至少一个角点信息，其中，各角点信息对应的角点为空闲停车位的两条边线的交点。Optionally, the key point information of the free parking space in the parking space training image includes at least one corner point information of the free parking space, wherein the corner point corresponding to each corner point information is the intersection of the two sidelines of the free parking space.

本申请实施例，停车位训练图像的四周边缘向外扩充预设值，补充不完整的空闲停车位，使得停车位训练图像中包括不完整空闲停车位的关键点信息的标准信息，使用这样的停车位训练图像来训练神经网络，可以使训练后的神经网络预测出停车位图像中没有拍摄完整的空闲停车位，提高了停车位检测全面性和准确性。In the embodiment of the application, the peripheral edges of the parking space training image are expanded outward by preset values to supplement incomplete free parking spaces, so that the parking space training image includes standard information of key point information of incomplete free parking spaces. The parking space training image is used to train the neural network, which can make the trained neural network predict that there is no complete free parking space in the parking space image, which improves the comprehensiveness and accuracy of parking space detection.

在上述神经网络的训练实施例的基础上，在一种可以的实现方式中，如图5所示，上述S202使用所述多个停车位训练图像，训练所述神经网络，具体可以包括：Based on the foregoing neural network training embodiment, in a possible implementation, as shown in FIG. 5, the foregoing S202 uses the multiple parking space training images to train the neural network, which may specifically include:

S301、获得所述停车位训练图像中空闲停车位的角点信息和所述停车位训练图像中空闲停车位的关键点信息构成的区域信息。S301: Obtain area information composed of corner point information of the free parking space in the parking space training image and key point information of the free parking space in the parking space training image.

示例性，以图4b中的空闲停车位2为例，空闲停车位2的6个关键点中包括4个角点，例如，四个角点的标注记为1，其他的关键点记为0，这样，空闲停车位2的6个关键点信息假设为：{"kpts":[[1346.2850971922246，517.6241900647948，1.0]，[1225.010799136069，591.1447084233262,1.0]，[1280.6479481641468，666.6522678185745，0.0]，[1300.5183585313175，728.2505399568034，1.0]，[1339.2656587473002，707.3866090712743，0.0]，[1431.6630669546437，630.8855291576674，1.0]]}。For example, taking the free parking space 2 in Figure 4b as an example, the 6 key points of the free parking space 2 include 4 corner points, for example, the four corner points are marked as 1, and the other key points are marked as 0 , In this way, the six key points of free parking space 2 are assumed to be: {"kpts":[[1346.2850971922246,517.6241900647948,1.0],[1225.010799136069,591.1447084233262,1.0],[1280.6479481641468,666.6522678185745,0.0],[1300.5183585313175,728.2505399568034 , 1.0], [1339.2656587473002, 707.3866090712743, 0.0], [1431.6630669546437, 630.8855291576674, 1.0]]}.

获得上述空闲停车位2的6个关键点信息构成的区域信息，例如，将空闲停车位2的6个关键点按照顺时针或逆时针排序，然后求其包围的面积，将包围的面积作为空闲停车位2的区域信息。该空闲停车位2的区域信息可以作为停车位的检测过程中区域信息的真值。Obtain the area information composed of the 6 key points of the free parking space 2. For example, sort the 6 key points of the free parking space 2 clockwise or counterclockwise, and then find the enclosed area, and use the enclosed area as free Area information for parking space 2. The area information of the free parking space 2 can be used as the true value of the area information in the process of detecting the parking space.

参照上述方法，获得多个停车位训练图像中的每一张停车位训练图像中空闲停车位的关键点信息构成的区域信息，形成空闲停车位的区域真值(Groundtruth)。With reference to the above method, the area information composed of the key point information of the free parking space in each of the parking space training images in the multiple parking space training images is obtained, and the ground truth of the free parking space is formed.

另外，继续以图4b中的空闲停车位2为例，从空闲停车位2的6个关键点信息中获得空闲停车位2的4个角点信息，假设为：{"角点":[[1346.2850971922246，517.6241900647948，1.0]，[1225.010799136069，591.1447084233262,1.0]，[1300.5183585313175，728.2505399568034，1.0]，[1431.6630669546437，630.8855291576674，1.0]]}。这4个角点信息构成空闲停车位2的角点真值。In addition, continue to take the free parking space 2 in Fig. 4b as an example, obtain the 4 corner information of the free parking space 2 from the 6 key point information of the free parking space 2, suppose it is: {"corner":[[ 1346.2850971922246, 517.6241900647948, 1.0], [1225.010799136069, 591.1447084233262, 1.0], [1300.5183585313175, 728.2505399568034, 1.0], [1431.6630669546437, 630.8855291576674, 1.0]]}. The four corner information constitutes the corner truth value of the free parking space 2.

参照上述方法，从多个停车位训练图像中的每一张停车位训练图像中空闲停车位的角点信息，形成空闲停车位的角点真值。With reference to the above method, the corner point information of the free parking space in each of the multiple parking space training images is formed to form the corner point truth value of the free parking space.

S302、使用所述停车位训练图像，以及所述停车位训练图像中空闲停车位的角点信息和区域信息，训练所述神经网络。S302. Use the parking space training image, and the corner point information and area information of the free parking space in the parking space training image to train the neural network.

参照图6所示，图6为本申请实施例涉及的神经网络的一种结构示意图，本申请实施例涉及的神经网络包括但不限于图6所示的神经网络。Referring to FIG. 6, FIG. 6 is a schematic diagram of a structure of a neural network involved in an embodiment of this application. The neural network involved in an embodiment of this application includes but is not limited to the neural network shown in FIG. 6.

如图6所述，该神经网络可包括实例分割层，该实例分割层用于获得空闲停车位的区域信息。As shown in FIG. 6, the neural network may include an instance segmentation layer, and the instance segmentation layer is used to obtain area information of free parking spaces.

可选的，本申请实施例的神经网络为mask-RCNN结构的神经网络，如图6所示，该神经网络还包括：特征金字塔网络(Feature Pyramid Networks，FPN)检测底部和区域卷积神经网络(Region CNN，RCNN)位置回归层，其中，FPN检测底部的输出端与RCNN位置回归层的输入端连接，RCNN位置回归层的输出端与实例分割层的输入端连接。这样FPN检测底部用于从所述停车位训练图像中检测出空闲停车位的检测框，如图6所示的矩形框。接着，将检测出的空闲停车位的检测框输入RCNN位置回归层，RCNN位置回归层对FPN检测底部检测出的空闲停车位的检测框进行微调。RCNN位置回归层将微调后的空闲停车位的检测框输入到实例分割层中，该实例分割层分割出空闲停车位的区域信息，例如图7中白色区域所示。Optionally, the neural network in this embodiment of the application is a neural network with a mask-RCNN structure, as shown in FIG. 6, the neural network also includes: Feature Pyramid Networks (FPN) detection bottom and regional convolutional neural network (Region CNN, RCNN) position regression layer, where the output terminal at the bottom of the FPN detection is connected with the input terminal of the RCNN position regression layer, and the output terminal of the RCNN position regression layer is connected with the input terminal of the instance segmentation layer. In this way, the FPN detection bottom is used to detect the detection frame of the free parking space from the parking space training image, such as the rectangular frame shown in FIG. 6. Then, the detection frame of the detected free parking space is input to the RCNN position regression layer, and the RCNN position regression layer fine-tunes the detection frame of the free parking space detected at the bottom of the FPN detection. The RCNN position regression layer inputs the fine-tuned detection frame of the free parking space into the instance segmentation layer, and the instance segmentation layer segments the area information of the free parking space, for example, as shown in the white area in FIG. 7.

可选的，上述实例分割层由一系列卷积层或池化层按照预设顺序堆叠而成。Optionally, the foregoing example segmentation layer is formed by stacking a series of convolutional layers or pooling layers in a preset order.

继续参照图6所示，该神经网络还可包括关键点检测层，该关键点检测层用于获得空闲停车位的角点信息。Continuing to refer to Fig. 6, the neural network may also include a key point detection layer, which is used to obtain corner point information of free parking spaces.

可选的，该关键点检测层的输入端与RCNN位置回归层的输出端连接，RCNN位置回归层将微调后的空闲停车位的检测框输入到关键点检测层中，该关键点检测层输出空闲停车位的角点信息，例如图7中黑色角点所示。需要说明的是，图7中两个相邻的空闲停车位的一条边线重合，进而使得另两个空闲停车位中两个角点重合。Optionally, the input of the key point detection layer is connected to the output of the RCNN position regression layer. The RCNN position regression layer inputs the fine-tuned free parking space detection frame into the key point detection layer, and the key point detection layer outputs The corner point information of the free parking space, for example, is shown in the black corner point in Figure 7. It should be noted that one edge of two adjacent free parking spaces in FIG. 7 overlaps, so that two corner points of the other two free parking spaces overlap.

根据上述方法，通过神经网络可以预测出的空闲停车位的区域信息和角点信息，再将预测的空闲停车位的区域信息与上述步骤获得的停车位训练图像中空闲停车位的关键点信息构成的区域信息进行比较，将预测的空闲停车位的角点信息与上述步骤获得的停车位训练图像中空闲停车位的角点信息进行比较，调整神经网络的参数。重复上述步骤，直到神经网络的训练次数达到预设次数，或者，神经网络的预测误差达到预设误差值为止。According to the above method, the area information and corner point information of the free parking space that can be predicted through the neural network, and then the area information of the predicted free parking space and the key point information of the free parking space in the parking space training image obtained in the above steps are formed The area information is compared, the predicted corner information of the free parking space is compared with the corner information of the free parking space in the parking space training image obtained in the above steps, and the parameters of the neural network are adjusted. Repeat the above steps until the number of training times of the neural network reaches the preset number, or the prediction error of the neural network reaches the preset error value.

本申请实施例的方法，通过获得所述停车位训练图像中空闲停车位的角点信息和所述停车位训练图像中空闲停车位的关键点信息构成的区域信息；使用所述停车位训练图像，以及所述停车位训练图像中空闲停车位的角点信息和区域信息，训练所述神经网络，使得训练后的神经网络可以准确预测出空闲停车位的区域信息和/或角点信息。The method of the embodiment of the present application obtains the area information formed by the corner point information of the free parking space in the parking space training image and the key point information of the free parking space in the parking space training image; using the parking space training image , And the corner point information and area information of the free parking space in the parking space training image, training the neural network so that the trained neural network can accurately predict the area information and/or corner point information of the free parking space.

本申请实施例提供的任一种停车位的检测方法可以由任意适当的具有数据处理能力的设备执行，包括但不限于：终端设备或服务器等。或者，本申请实施例提供的任一种停车位的检测方法可以由处理器执行，如处理器通过调用存储器存储的相应指令来执行本申请实施例提及的任一种停车位检测方法。下文不再赘述。Any parking space detection method provided in the embodiments of the present application can be executed by any suitable device with data processing capabilities, including but not limited to: terminal devices or servers. Alternatively, any parking space detection method provided in the embodiment of the present application may be executed by a processor. For example, the processor executes any parking space detection method mentioned in the embodiment of the present application by calling a corresponding instruction stored in a memory. I won't repeat it below.

本领域普通技术人员可以理解：实现上述方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成，前述的程序可以存储于一计算机可读取存储介质中，该程序在执行时，执行包括上述方法实施例的步骤；而前述的存储介质包括：ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。A person of ordinary skill in the art can understand that all or part of the steps in the above method embodiments can be implemented by a program instructing relevant hardware. The foregoing program can be stored in a computer readable storage medium. When the program is executed, it is executed. Including the steps of the foregoing method embodiment; and the foregoing storage medium includes: ROM, RAM, magnetic disk, or optical disk and other media that can store program codes.

图8为本申请实施例提供的停车位的检测装置的结构示意图一。如图8所示，本实施例的停车位的检测装置100可以包括：FIG. 8 is a first structural diagram of a parking space detection device provided by an embodiment of the application. As shown in FIG. 8, the parking space detection device 100 of this embodiment may include:

第一获取模块110，用于获取停车位图像；The first acquisition module 110 is used to acquire parking space images;

处理模块120，用于将所述停车位图像输入神经网络中，获得所述停车位图像中空闲停车位的区域信息和/或角点信息；The processing module 120 is configured to input the parking space image into a neural network to obtain area information and/or corner point information of an idle parking space in the parking space image;

确定模块130，用于基于所述停车位图像中空闲停车位的区域信息和/或角点信息，确定所述停车位图像中空闲停车位的检测结果。The determining module 130 is configured to determine the detection result of the free parking space in the parking space image based on the area information and/or corner point information of the free parking space in the parking space image.

本申请实施例的停车位的检测装置，可以用于执行上述所示方法实施例的技术方案，其实现原理和技术效果类似，此处不再赘述。The parking space detection device of the embodiment of the present application can be used to implement the technical solutions of the method embodiment shown above, and its implementation principles and technical effects are similar, and will not be repeated here.

在一种可能的实现方式下，所述确定模块130，用于将所述停车位图像中空闲停车位的区域信息和角点信息进行融合，确定所述停车位图像中空闲停车位的检测结果。In a possible implementation manner, the determining module 130 is configured to merge the area information and corner point information of the free parking space in the parking space image to determine the detection result of the free parking space in the parking space image .

在另一种可能的实现方式下，所述确定模块130，用于所述停车位图像中空闲停车位的角点信息构成的停车位区域信息；将所述停车位图像中空闲停车位的区域信息和所述角点信息构成的停车位区域信息进行融合，确定所述停车位图像中空闲停车位的检测结果。In another possible implementation manner, the determining module 130 is used for parking space area information formed by corner point information of free parking spaces in the parking space image; and calculating the free parking space area in the parking space image The information and the parking space area information formed by the corner point information are fused to determine the detection result of the free parking space in the parking space image.

图9为本申请实施例提供的停车位的检测装置的结构示意图，所述停车位的检测装置100还包括：扩充模块140，FIG. 9 is a schematic structural diagram of a parking space detection device provided by an embodiment of the application. The parking space detection device 100 further includes an expansion module 140,

扩充模块140，用于在所述停车位图像的四周边缘向外扩充预设值，所述预设值小于或等于停车位长度的一半；The expansion module 140 is configured to expand a preset value outward on the peripheral edges of the parking space image, and the preset value is less than or equal to half the length of the parking space;

所述处理模块120，用于将扩充后的所述停车位图像输入所述神经网络中，获得所述停车位图像中空闲停车位的区域信息和/或角点信息。The processing module 120 is configured to input the expanded parking space image into the neural network to obtain area information and/or corner point information of free parking spaces in the parking space image.

图10为本申请实施例提供的停车位的检测装置的结构示意图三，所述停车位的检测装置100包括：FIG. 10 is a third structural diagram of a parking space detection device provided by an embodiment of the application, and the parking space detection device 100 includes:

第二获取模块150，用于获取多个停车位训练图像；The second acquisition module 150 is used to acquire multiple parking space training images;

训练模块160，用于使用所述多个停车位训练图像，训练所述神经网络，其中，所述停车位训练图像包括有对空闲停车位的关键点信息的标注信息。The training module 160 is configured to use the multiple parking space training images to train the neural network, wherein the parking space training image includes annotation information for key point information of the free parking space.

在一种可能的实现方式中，所述停车位训练图像的四周边缘向外扩充预设值，所述预设值小于或等于停车位长度的一半。In a possible implementation manner, the peripheral edges of the parking space training image are expanded outward by a preset value, and the preset value is less than or equal to half the length of the parking space.

在另一种可能的实现方式中，所述停车位训练图像中空闲停车位的关键点信息包括所述空闲停车位的至少一个角点信息。In another possible implementation manner, the key point information of the free parking space in the parking space training image includes at least one corner point information of the free parking space.

在另一种可能的实现方式中，所述训练模块160，用于获得所述停车位训练图像中空闲停车位的角点信息和所述停车位训练图像中空闲停车位的关键点信息构成的区域信息；使用所述停车位训练图像，以及所述停车位训练图像中空闲停车位的角点信息和区域信息，训练所述神经网络。In another possible implementation manner, the training module 160 is configured to obtain corner point information of the free parking space in the parking space training image and key point information of the free parking space in the parking space training image Area information; use the parking space training image, and the corner information and area information of the free parking space in the parking space training image to train the neural network.

可选的，所述停车位训练图像为广角摄像头拍摄的图像。Optionally, the parking space training image is an image taken by a wide-angle camera.

图11为本申请实施例提供的电子设备的结构示意图，如图11所示，本实施例的电子设备30包括：FIG. 11 is a schematic structural diagram of an electronic device provided by an embodiment of this application. As shown in FIG. 11, the electronic device 30 of this embodiment includes:

存储器310，用于存储计算机程序；The memory 310 is used to store computer programs;

处理器320，用于执行所述计算机程序，以实现上述的停车位的检测方法，其实现原理和技术效果类似，此处不再赘述。The processor 320 is configured to execute the computer program to implement the above-mentioned parking space detection method. The implementation principles and technical effects are similar, and will not be repeated here.

进一步的，当本申请实施例中停车位的检测方法的至少一部分功能通过软件实现时，本申请实施例还提供一种计算机存储介质，为易失性或非易失性计算机存储介质，计算机存储介质用于储存为上述对停车位的检测的计算机软件指令，当其在计算机上运行时，使得计算机可以执行上述方法实施例中各种可能的停车位的检测方法。在计算机上加载和执行所述计算机执行指令时，可全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机指令可以存储在计算机存储介质中，或者从一个计算机存储介质向另一个计算机存储介质传输，所述传输可以通过无线(例如蜂窝通信、红外、短距离无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，数字通用光盘(Digital Versatile Disc，DVD))、或者半导体介质(例如固态驱动器(Solid State Disk，SSD))等。Further, when at least a part of the functions of the parking space detection method in the embodiment of the present application is implemented by software, the embodiment of the present application also provides a computer storage medium, which is a volatile or non-volatile computer storage medium. The medium is used to store the above-mentioned computer software instructions for detecting parking spaces, and when running on a computer, the computer can execute various possible parking space detecting methods in the above method embodiments. When the computer-executable instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application can be generated in whole or in part. The computer instructions can be stored in a computer storage medium, or transmitted from one computer storage medium to another computer storage medium, and the transmission can be transmitted to another by wireless (such as cellular communication, infrared, short-range wireless, microwave, etc.) Website site, computer, server or data center for transmission. The computer storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center integrated with one or more available media. The usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, a Digital Versatile Disc (DVD)), or a semiconductor medium (for example, a solid state drive (Solid State Disk, SSD)) )Wait.

在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(Digital Subscriber Line，DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如SSD)等。In the above-mentioned embodiments, it may be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in the form of a computer program product in whole or in part. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website, computer, server, or data center. Transmission to another website, computer, server, or data center via wired (for example, coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (for example, infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center integrated with one or more available media. The usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, an SSD).

在本申请实施例中，“至少一个”是指一个或者多个，“多个”是指两个或两个以上。“和/或”，描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B的情况，其中A，B可以是单数或者复数。字符“/”一般表示前后关联对象是一种“或”的关系；在公式中，字符“/”，表示前后关联对象是一种“相除”的关系。“以下至少一项(个)”或其类似表达，是指的这些项中的任意组合，包括单项(个)或复数项(个)的任意组合。例如，a，b，或c中的至少一项(个)，可以表示：a，b，c，a-b，a-c，b-c，或a-b-c，其中，a，b，c可以是单个，也可以是多个。In the embodiments of the present application, "at least one" refers to one or more, and "multiple" refers to two or more. "And/or" describes the association relationship of the associated objects, indicating that there can be three relationships, for example, A and/or B, which can mean: A alone exists, both A and B exist, and B exists alone, where A, B can be singular or plural. The character "/" generally indicates that the associated objects before and after are in an "or" relationship; in the formula, the character "/" indicates that the associated objects before and after are in a "division" relationship. "The following at least one item (a)" or similar expressions refers to any combination of these items, including any combination of a single item (a) or plural items (a). For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple One.

可以理解的是，在本申请实施例中涉及的各种数字编号仅为描述方便进行的区分，并不用来限制本申请实施例的范围。It can be understood that the various numerical numbers involved in the embodiments of the present application are only for easy distinction for description, and are not used to limit the scope of the embodiments of the present application.

可以理解的是，在本申请的实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本公开实施例的实施过程构成任何限定。It can be understood that, in the embodiments of the present application, the size of the sequence numbers of the aforementioned processes does not mean the order of execution. The execution order of the processes should be determined by their functions and internal logic, and should not be used in the embodiments of the present disclosure. The implementation process constitutes any limitation.

最后应说明的是：以上各实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述各实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的范围。Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the application, not to limit them; although the application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand: It is still possible to modify the technical solutions described in the foregoing embodiments, or equivalently replace some or all of the technical features; these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the application range.

Claims

一种停车位的检测方法，所述方法包括：A method for detecting parking spaces, the method comprising:

获取停车位图像；Obtain parking space images;

将所述停车位图像输入神经网络中，获得所述停车位图像中空闲停车位的区域信息和/或角点信息；Inputting the parking space image into a neural network to obtain area information and/or corner point information of free parking spaces in the parking space image;

基于所述停车位图像中空闲停车位的区域信息和/或角点信息，确定所述停车位图像中空闲停车位的检测结果。Based on the area information and/or corner point information of the free parking space in the parking space image, the detection result of the free parking space in the parking space image is determined.
根据权利要求1所述的方法，其中，所述基于所述停车位图像中空闲停车位的区域信息和角点信息，确定所述停车位图像中空闲停车位的检测结果，包括：The method according to claim 1, wherein the determining the detection result of the free parking space in the parking space image based on the area information and corner point information of the free parking space in the parking space image comprises:

将所述停车位图像中空闲停车位的区域信息和角点信息进行融合，确定所述停车位图像中空闲停车位的检测结果。The area information and corner point information of the free parking space in the parking space image are fused to determine the detection result of the free parking space in the parking space image.
根据权利要求2所述的方法，其中，所述将所述停车位图像中空闲停车位的区域信息和角点信息进行融合，确定所述空闲停车位的检测结果，包括：The method according to claim 2, wherein said fusing the area information and corner point information of the free parking space in the parking space image to determine the detection result of the free parking space comprises:

确定所述停车位图像中空闲停车位的角点信息构成的停车位区域信息；Determining parking space area information formed by corner point information of free parking spaces in the parking space image;

将所述停车位图像中空闲停车位的区域信息和所述角点信息构成的停车位区域信息进行融合，基于融合后的区域信息确定所述停车位图像中空闲停车位的检测结果。The area information of the free parking space in the parking space image and the parking space area information formed by the corner point information are merged, and the detection result of the free parking space in the parking space image is determined based on the merged area information.
根据权利要求1-3任一项所述的方法，其中，所述将所述停车位图像输入神经网络中，获得所述停车位图像中空闲停车位的区域信息和/或角点信息之前，所述方法还包括：The method according to any one of claims 1 to 3, wherein before the input of the parking space image into a neural network, the area information and/or corner point information of the free parking space in the parking space image is obtained, The method also includes:

在所述停车位图像的四周边缘向外扩充预设值，所述预设值小于或等于停车位长度的一半；Extend a preset value outward on the peripheral edges of the parking space image, and the preset value is less than or equal to half the length of the parking space;

所述将所述停车位图像输入神经网络中，获得所述停车位图像中空闲停车位的区域信息和/或角点信息，包括：The inputting the parking space image into a neural network to obtain the area information and/or corner point information of the free parking space in the parking space image includes:

将扩充后的所述停车位图像输入所述神经网络中，获得所述停车位图像中空闲停车位的区域信息和/或角点信息。The expanded parking space image is input into the neural network to obtain the area information and/or corner point information of the free parking space in the parking space image.
根据权利要求1-4任一项所述的方法，其中，所述方法还包括：The method according to any one of claims 1-4, wherein the method further comprises:

获取多个停车位训练图像；Acquire training images of multiple parking spaces;

使用所述多个停车位训练图像，训练所述神经网络，其中，所述停车位训练图像包括有对空闲停车位的关键点信息的标注信息。The neural network is trained using the plurality of parking space training images, wherein the parking space training image includes annotation information of key point information of the free parking space.
根据权利要求5所述的方法，其中，所述停车位训练图像的四周边缘向外扩充预设值，所述预设值小于或等于停车位长度的一半。The method according to claim 5, wherein the peripheral edges of the parking space training image are expanded outward by a preset value, and the preset value is less than or equal to half the length of the parking space.
根据权利要求5或6所述的方法，其中，所述停车位训练图像中空闲停车位的关键点信息包括所述空闲停车位的至少一个角点信息。The method according to claim 5 or 6, wherein the key point information of the free parking space in the parking space training image includes at least one corner point information of the free parking space.
根据权利要求7所述的方法，其中，所述使用所述多个停车位训练图像，训练所述神经网络，包括：The method according to claim 7, wherein said training said neural network using said plurality of parking space training images comprises:

获得所述停车位训练图像中空闲停车位的角点信息和所述停车位训练图像中空闲停车位的关键点信息构成的区域信息；Obtaining area information composed of corner point information of the free parking space in the parking space training image and key point information of the free parking space in the parking space training image;

使用所述停车位训练图像，以及所述停车位训练图像中空闲停车位的角点信息和区域信息，训练所述神经网络。Training the neural network using the parking space training image and the corner point information and area information of the free parking space in the parking space training image.
根据权利要求5-8任一项所述的方法，其中，所述停车位训练图像为广角摄像头拍摄的图像。The method according to any one of claims 5-8, wherein the parking space training image is an image taken by a wide-angle camera.
一种停车位的检测装置，包括：A detection device for parking spaces, including:

第一获取模块，用于获取停车位图像；The first acquisition module is used to acquire parking space images;

处理模块，用于将所述停车位图像输入神经网络中，获得所述停车位图像中空闲停车位的区域信息和/或角点信息；A processing module, configured to input the parking space image into a neural network to obtain area information and/or corner point information of free parking spaces in the parking space image;

确定模块，用于基于所述停车位图像中空闲停车位的区域信息和/或角点信息，确定所述停车位图像中空闲停车位的检测结果。The determining module is configured to determine the detection result of the free parking space in the parking space image based on the area information and/or corner point information of the free parking space in the parking space image.
根据权利要求10所述的装置，其中，The device of claim 10, wherein:

所述确定模块，用于将所述停车位图像中空闲停车位的区域信息和角点信息进行融合，确定所述停车位图像中空闲停车位的检测结果。The determining module is used to merge the area information and corner point information of the free parking space in the parking space image to determine the detection result of the free parking space in the parking space image.
根据权利要求11所述的装置，其中，The device according to claim 11, wherein:

所述确定模块，用于所述停车位图像中空闲停车位的角点信息构成的停车位区域信息；将所述停车位图像中空闲停车位的区域信息和所述角点信息构成的停车位区域信息进行融合，基于融合后的区域信息确定所述停车位图像中空闲停车位的检测结果。The determining module is used for parking space area information composed of corner point information of free parking spaces in the parking space image; parking space composed of the area information of free parking spaces in the parking space image and the corner point information The area information is fused, and the detection result of the free parking space in the parking space image is determined based on the fused area information.
根据权利要求10-12任一项所述的装置，其中，所述装置还包括：The device according to any one of claims 10-12, wherein the device further comprises:

扩充模块，用于在所述停车位图像的四周边缘向外扩充预设值，所述预设值小于或等于停车位长度的一半；An expansion module, configured to expand a preset value outward on the peripheral edges of the parking space image, and the preset value is less than or equal to half the length of the parking space;

所述处理模块，用于将扩充后的所述停车位图像输入所述神经网络中，获得所述停车位图像中空闲停车位的区域信息和/或角点信息。The processing module is configured to input the expanded parking space image into the neural network to obtain area information and/or corner point information of the free parking space in the parking space image.
根据权利要求10-13任一项所述的装置，其中，所述装置还包括：The device according to any one of claims 10-13, wherein the device further comprises:

第二获取模块，用于获取多个停车位训练图像；The second acquisition module is used to acquire multiple parking space training images;

训练模块，用于使用所述多个停车位训练图像，训练所述神经网络，其中，所述停车位训练图像包括有对空闲停车位的关键点信息的标注信息。The training module is used to train the neural network using the multiple parking space training images, wherein the parking space training image includes annotation information for key point information of the free parking space.
根据权利要求14所述的装置，其中，所述停车位训练图像的四周边缘向外扩充预设值，所述预设值小于或等于停车位长度的一半。14. The device of claim 14, wherein the peripheral edges of the parking space training image are expanded outward by a preset value, and the preset value is less than or equal to half the length of the parking space.
根据权利要求14或15所述的装置，其中，所述停车位训练图像中空闲停车位的关键点信息包括所述空闲停车位的至少一个角点信息。The device according to claim 14 or 15, wherein the key point information of the free parking space in the parking space training image includes at least one corner point information of the free parking space.
根据权利要求16所述的装置，其中，The device according to claim 16, wherein:

所述训练模块，用于获得所述停车位训练图像中空闲停车位的角点信息和所述停车位训练图像中空闲停车位的关键点信息构成的区域信息；使用所述停车位训练图像，以及所述停车位训练图像中空闲停车位的角点信息和区域信息，训练所述神经网络。The training module is used to obtain the area information formed by the corner point information of the free parking space in the parking space training image and the key point information of the free parking space in the parking space training image; using the parking space training image, And the corner point information and area information of the free parking space in the parking space training image to train the neural network.
根据权利要求14-17任一项所述的装置，其中，所述停车位训练图像为广角摄像头拍摄的图像。The device according to any one of claims 14-17, wherein the parking space training image is an image taken by a wide-angle camera.
一种电子设备，包括：An electronic device including:

存储器，用于存储计算机程序；Memory, used to store computer programs;

处理器，用于执行所述计算机程序，以实现如权利要求1-9任一项所述的停车位的检测方法。The processor is configured to execute the computer program to implement the parking space detection method according to any one of claims 1-9.
一种计算机存储介质，所述存储介质中存储计算机程序，所述计算机程序在执行时实现如权利要求1-9任一项所述的停车位的检测方法。A computer storage medium in which a computer program is stored, and the computer program, when executed, implements the parking space detection method according to any one of claims 1-9.