WO2023045836A1

WO2023045836A1 - Luminaire detection method and apparatus, device, medium, chip, product, and program

Info

Publication number: WO2023045836A1
Application number: PCT/CN2022/119250
Authority: WO
Inventors: 李七星; 甘伟豪; 武伟
Original assignee: 上海商汤智能科技有限公司
Priority date: 2021-09-24
Filing date: 2022-09-16
Publication date: 2023-03-30
Also published as: CN113808117A

Abstract

The present disclosure relates to a luminaire detection method and apparatus, a device, a medium, a chip, a product, and a program. The luminaire detection method comprises: obtaining an image requiring detection, wherein a scene corresponding to the image requiring detection comprises a luminaire; binarizing the image requiring detection to obtain a binarized image, and determining a candidate luminaire frame on the basis of the binarized image; and identifying a candidate region corresponding to the candidate luminaire frame in the image requiring detection, and determining a target luminaire frame in the image requiring detection, wherein the target luminaire frame corresponds to the location of the luminaire. Because the candidate luminaire frame is selected by means of binarization, targeted identification needs to be performed only on the candidate region corresponding to the candidate luminaire frame, such that the identification range is reduced, identification accuracy is improved, and the accuracy and robustness of landscape light detection are improved.

Description

灯具检测方法、装置、设备、介质、芯片、产品及程序Lamp detection method, device, equipment, medium, chip, product and program

相关申请的交叉引用Cross References to Related Applications

本专利申请要求2021年09月24日提交的中国专利申请号为202111122367.7，申请名称为“灯具检测方法、装置、设备及存储介质”的优先权，该公开的全文以引用的方式并入本公开中。This patent application claims the priority of the Chinese patent application number 202111122367.7 submitted on September 24, 2021, and the application name is "lamp detection method, device, equipment and storage medium". The entire disclosure is incorporated by reference into this disclosure middle.

技术领域technical field

本公开涉及灯具检测技术领域，尤其涉及一种灯具检测方法、装置、设备、介质、芯片、产品及程序。The present disclosure relates to the technical field of lamp detection, and in particular to a lamp detection method, device, equipment, medium, chip, product and program.

背景技术Background technique

以计算机视觉为代表的人工智能技术正在被大量用于智慧城市建设当中，其中城市管理是一大应用领域，其特点是需求广泛、多样、零散，比如违规经营占道检测、垃圾检测、烟火检测、景观灯检测等等。相关技术中，使用深度学习算法对景观灯进行检测，但是楼宇景观灯形态非常多，无法枚举，高空相机较少，数据采集困难，因此完全采用深度学习进行景观灯检测的效果不佳。Artificial intelligence technology represented by computer vision is being widely used in the construction of smart cities. Among them, urban management is a major application field, which is characterized by extensive, diverse, and fragmented needs, such as detection of illegal operation, garbage detection, and firework detection. , landscape light detection and so on. In related technologies, deep learning algorithms are used to detect landscape lights. However, there are so many forms of building landscape lights that it is impossible to enumerate them. There are few high-altitude cameras and data collection is difficult. Therefore, the effect of completely using deep learning to detect landscape lights is not good.

发明内容Contents of the invention

本公开提供一种灯具检测方法、装置、设备、介质、芯片、产品及程序。The disclosure provides a lamp detection method, device, equipment, medium, chip, product and program.

根据本公开实施例的第一方面，提供一种灯具检测方法，包括：According to a first aspect of an embodiment of the present disclosure, a lamp detection method is provided, including:

获取待检测图像，其中，所述待检测图像对应的场景内包括灯具；Acquiring an image to be detected, wherein the scene corresponding to the image to be detected includes lamps;

将所述待检测图像进行二值化处理，得到二值化图像，并基于所述二值化图像确定灯具候选框；performing binarization processing on the image to be detected to obtain a binarized image, and determining a lamp candidate frame based on the binarized image;

对所述待检测图像中对应所述灯具候选框的候选区域进行识别，确定所述待检测图像的灯具目标框，其中，所述灯具目标框对应所述灯具所在的位置。Identifying a candidate area corresponding to the lamp candidate frame in the image to be detected, and determining a lamp target frame in the image to be detected, wherein the lamp target frame corresponds to a position of the lamp.

在一个示例中，所述待检测图像为待检测视频的一帧图像，所述待检测视频对应的场景内包括灯具；In an example, the image to be detected is a frame image of a video to be detected, and the scene corresponding to the video to be detected includes lamps;

所述方法还包括：The method also includes:

根据所述待检测图像的灯具目标框和参考图像的灯具目标框，确定所述参考图像的灯具目标框对应的灯具的明暗状态，其中，所述参考图像为所述待检测视频中，所述待检测图像之前的一帧图像。According to the lamp target frame of the image to be detected and the lamp target frame of the reference image, determine the light and dark state of the lamp corresponding to the lamp target frame of the reference image, wherein the reference image is in the video to be detected, the A frame of image before the image to be detected.

这样，根据待检测图像的灯具目标框和参考图像的灯具目标框，确定参考图像的灯具目标框对应的灯具的明暗状态，从而能够获取每个参考图像的灯具目标框中的灯具在两帧图像中的明暗变化，进而能够确定参考图像中每个灯具的明暗变化情况。In this way, according to the luminaire target frame of the image to be detected and the luminaire target frame of the reference image, the light and dark state of the luminaire corresponding to the luminaire target frame of the reference image can be determined, so that the two-frame image of the luminaire in the luminaire target frame of each reference image can be obtained In this way, the light and shade changes of each lamp in the reference image can be determined.

在一个示例中，所述根据所述待检测图像的灯具目标框和参考图像的灯具目标框，确定所述参考图像的灯具目标框对应的灯具的明暗状态，包括：In an example, the determining the light and dark state of the lamp corresponding to the lamp target frame of the reference image according to the lamp target frame of the image to be detected and the reference image includes:

确定所述参考图像的每个灯具目标框和所述待检测图像的每个灯具目标框的交并比；determining the intersection ratio of each lamp target frame of the reference image and each lamp target frame of the image to be detected;

在所述参考图像的灯具目标框与所述待检测图像的每个灯具目标框的交并比，均小于预设的第一比例阈值，确定所述参考图像的灯具目标框对应的灯具的明暗状态为黑暗；When the intersection ratio between the lamp target frame of the reference image and each lamp target frame of the image to be detected is less than a preset first ratio threshold, determine the lightness and darkness of the lamp corresponding to the lamp target frame of the reference image state is dark;

在所述参考图像的灯具目标框与所述待检测图像的任一灯具目标框的交并比，大于或等于所述第一比例阈值的情况下，确定所述参考图像的灯具目标框对应的灯具的明暗状态为明亮。When the intersection ratio between the lamp target frame of the reference image and any lamp target frame of the image to be detected is greater than or equal to the first ratio threshold, determine the corresponding The light and dark state of the lamp is bright.

这样，通过灯具目标框的交并比确定灯具的明暗状态，由于灯具目标框对应灯具所在的位置，因此这种方式能够简单且准确地确定灯具明暗状态。In this way, the light and shade state of the light fixture is determined by the intersection ratio of the light fixture target frame. Since the light fixture target frame corresponds to the position of the light fixture, this method can simply and accurately determine the light and shade state of the light fixture.

确定所述参考图像的所述灯具目标框中的全部像素的像素值平均值为第一平均值，并确定所述待检测图像中对应的所述灯具目标框中的全部像素的像素值平均值为第二平均值；Determining the average value of the pixel values of all the pixels in the lamp target frame of the reference image as the first average value, and determining the average pixel value of all the pixels in the corresponding lamp target frame in the image to be detected is the second average value;

在所述第一平均值与所述第二平均值的差值大于预设的第一像素阈值的情况下，确定所述灯具目标框对应的灯具的明暗状态为黑暗；When the difference between the first average value and the second average value is greater than a preset first pixel threshold, determine that the light and shade state of the lamp corresponding to the target frame of the lamp is dark;

在所述第一平均值与所述第二平均值的差值小于或等于预设的第一像素阈值的情况下，确定所述灯具目标框对应的灯具的明暗状态为明亮。In a case where the difference between the first average value and the second average value is less than or equal to a preset first pixel threshold, it is determined that the light and dark state of the lamp corresponding to the lamp target frame is bright.

这样，通过对应目标框的平均像素差，精确地确定灯具是否熄灭，从而提高了灯具明暗状态的识别精度。In this way, through the average pixel difference corresponding to the target frame, it is accurately determined whether the lamp is off, thereby improving the recognition accuracy of the light and dark state of the lamp.

在一个示例中，所述将所述待检测图像进行二值化处理，包括：In an example, the binarization of the image to be detected includes:

确定所述待检测图像和所述参考图像间的每个像素对的像素差，其中，所述像素对中的一个像素点在所述待检测图像中的位置，和另一个像素点在所述参考图像中的位置相同；determining the pixel difference of each pixel pair between the image to be detected and the reference image, wherein the position of one pixel in the pixel pair in the image to be detected is the same as the position of another pixel in the image to be detected same position in the reference image;

在全部的像素对中，确定像素差大于预设的第二像素阈值的像素对的第一数量；Among all the pixel pairs, determine a first number of pixel pairs whose pixel difference is greater than a preset second pixel threshold;

在所述第一数量与全部的像素对的数量的比例大于预设的第二比例阈值的情况下，将所述待检测图像进行二值化处理。In a case where the ratio of the first number to the number of all pixel pairs is greater than a preset second ratio threshold, the image to be detected is binarized.

这样，每次获取待检测图像后，确定第一数量与全部的像素对的数量的比例是否大于预设的第二比例阈值，从而确定待检测图像与参考图像对应的场景是否相同，即确定高空广角相机是否发生移动，若场景不同则舍弃，若场景相同则进一步确定灯具的明暗状态，从而避免了高空广角相机移动造成检测结果不准确的问题，提高了灯具目标框的检测精度以及灯具明暗状态的确定精度。In this way, after each acquisition of the image to be detected, it is determined whether the ratio of the first number to the number of all pixel pairs is greater than the preset second ratio threshold, thereby determining whether the scene corresponding to the image to be detected and the reference image is the same, that is, to determine whether the high altitude Whether the wide-angle camera moves, discard it if the scene is different, and further determine the light and shade state of the lamp if the scene is the same, thereby avoiding the problem of inaccurate detection results caused by the movement of the high-altitude wide-angle camera, and improving the detection accuracy of the target frame of the lamp and the light and dark state of the lamp the determination accuracy.

将所述待检测图像转换为灰度图像；Converting the image to be detected into a grayscale image;

将所述灰度图像进行二值化处理。Binarize the grayscale image.

这样，通过将所述待检测图像转换为灰度图像，将所述灰度图像进行二值化处理，从而能够通过灰度图像准确地对待检测图像进行二值化处理，并准确地得到待检测图像中的明亮像素和黑暗像素。In this way, by converting the image to be detected into a grayscale image and performing binarization processing on the grayscale image, the image to be detected can be accurately binarized through the grayscale image, and the image to be detected can be accurately obtained. Bright and dark pixels in an image.

在一个示例中，所述基于所述二值化图像确定灯具候选框，包括：In an example, the determining the candidate frame of the lamp based on the binarized image includes:

对所述二值化图像进行图像腐蚀处理，得到腐蚀图像；performing image erosion processing on the binarized image to obtain an etched image;

基于所述腐蚀图像确定所述灯具候选框。The lamp candidate frame is determined based on the corrosion image.

这样，通过图像腐蚀处理，可以消除二值化图像中的噪声点，进而使二值化图像中的连通区域的边缘平滑。In this way, through image erosion processing, the noise points in the binarized image can be eliminated, and then the edges of the connected regions in the binarized image can be smoothed.

基于所述二值化图像中由明亮像素组成的连通区域，确定所述灯具候选框。The lamp candidate frame is determined based on a connected region composed of bright pixels in the binarized image.

这样，基于所述二值化图像中由明亮像素组成的连通区域，确定所述灯具候选框，从而能够使得灯具候选框能够准确地对应到待检测图像中的灯具。In this way, the lamp candidate frame is determined based on the connected region composed of bright pixels in the binarized image, so that the lamp candidate frame can accurately correspond to the lamp in the image to be detected.

在一个示例中，所述基于所述二值化图像中由明亮像素组成的连通区域，确定所述灯具候选框，包括：基于所述二值化图像中由明亮像素组成的面积大于或等于预设的面积阈值的连通区域，确定所述灯具候选框。In an example, the determining the lamp candidate frame based on the connected area composed of bright pixels in the binarized image includes: based on the area composed of bright pixels in the binarized image being greater than or equal to a predetermined Set the connected area of the area threshold to determine the luminaire candidate box.

这样，通过去除面积较小的连通区域，可以减少噪声点，使后续针对候选区域的检测更具有针对性，降低负荷，提高效率，提高精度。In this way, by removing the connected regions with smaller areas, noise points can be reduced, making the subsequent detection of candidate regions more targeted, reducing load, improving efficiency, and improving accuracy.

在一个示例中，所述对所述待检测图像中对应所述灯具候选框的候选区域进行识别，确定所述待检测图像的灯具目标框，包括：In an example, the identifying the candidate area corresponding to the lamp candidate frame in the image to be detected, and determining the lamp target frame of the image to be detected includes:

将每个所述候选区域输入至预先完成训练的分类神经网络，通过所述分类神经网络输出每个所述候选区域的分类结果，其中，所述分类结果包括灯具或非灯具；Each of the candidate areas is input into a pre-trained classification neural network, and a classification result of each of the candidate areas is output through the classification neural network, wherein the classification results include lamps or non-lamps;

将分类结果为灯具的候选区域确定为所述灯具目标框。The classification result is determined as the candidate area of the luminaire as the target frame of the luminaire.

这样，通过预先完成训练的分类神经网络对每个候选区域进行分类，从而能够准确地确定每个候选区域是否包括灯具，提高了确定每个候选区域中是否有灯具的准确性。In this way, each candidate area is classified through the pre-trained classification neural network, so that it can be accurately determined whether each candidate area includes lamps, and the accuracy of determining whether each candidate area includes lamps is improved.

在一个示例中，所述获取待检测图像，包括：In an example, the acquisition of the image to be detected includes:

将从所述待检测视频中获取的与所述参考图像的时间间隔为预设时长的一帧图像，确定为所述待检测图像。Determining as the image to be detected a frame of image acquired from the video to be detected and whose time interval with the reference image is a preset duration.

这样，由于待检测图像是与所述参考图像的时间间隔为预设时长的一帧图像，从而能够确定预设时长前后的两帧图像中灯具明暗状态的变化，减少了灯具检测时的计算量。In this way, since the time interval between the image to be detected and the reference image is a frame image with a preset duration, the change of the light and shade state of the lamp in the two frames of images before and after the preset duration can be determined, reducing the amount of calculation when detecting the lamp .

在一个示例中，所述方法还包括：In one example, the method also includes:

删除所述参考图像，将所述待检测图像保存为参考图像，并保存所述待检测图像的灯具目标框。The reference image is deleted, the image to be detected is saved as a reference image, and the lamp target frame of the image to be detected is saved.

这样，通过随时更新参考图像，从而减少内存占用，且增加灯具明暗状态的确定精度。In this way, by updating the reference image at any time, the memory usage is reduced, and the determination accuracy of the light and dark state of the lamp is increased.

根据本公开实施例的第二方面，提供一种灯具检测装置，包括：According to a second aspect of an embodiment of the present disclosure, a lamp detection device is provided, including:

获取部分，配置为获取待检测图像，其中，所述待检测图像对应的场景内包括灯具；The acquisition part is configured to acquire the image to be detected, wherein the scene corresponding to the image to be detected includes lamps;

处理部分，配置为将所述待检测图像进行二值化处理，得到二值化图像，并基于所述二值化图像确定灯具候选框；The processing part is configured to perform binarization processing on the image to be detected to obtain a binarized image, and determine a lamp candidate frame based on the binarized image;

确定部分，配置为对所述待检测图像中对应所述灯具候选框的候选区域进行识别，确定所述待检测图像的灯具目标框，其中，所述灯具目标框对应所述灯具所在的位置。The determining part is configured to identify a candidate area corresponding to the lamp candidate frame in the image to be detected, and determine a lamp target frame in the image to be detected, wherein the lamp target frame corresponds to the location of the lamp.

所述装置还包括判断部分；所述判断部分，配置为：The device also includes a judging part; the judging part is configured to:

在一个示例中，所述判断部分还配置为：In an example, the judgment part is further configured as:

在一个示例中，所述处理部分还配置为：In one example, the processing section is further configured to:

将所述灰度图像进行二值化处理。Binarize the grayscale image.

基于所述二值化图像中由明亮像素组成的面积大于或等于预设的面积阈值的连通区域，确定所述灯具候选框。The lamp candidate frame is determined based on the connected regions in the binarized image that are composed of bright pixels and whose area is greater than or equal to a preset area threshold.

在一个示例中，所述确定部分还配置为：In one example, the determining part is further configured to:

在一个示例中，所述获取部分还配置为：In one example, the acquisition section is also configured to:

在一个示例中，灯具检测装置还包括缓存部分；所述缓存部分，配置为：In an example, the lamp detection device further includes a cache part; the cache part is configured as:

根据本公开实施例的第三方面，提供一种电子设备，所述设备包括存储器、处理器，所述存储器用于存储可在处理器上运行的计算机指令，所述处理器用于在执行所述计算机指令时实现第一方面所述的方法。According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, the device includes a memory and a processor, the memory is used to store computer instructions that can be run on the processor, and the processor is used to execute the The computer instructions implement the method described in the first aspect.

根据本公开实施例的第四方面，提供一种计算机可读存储介质，其上存储有计算机程序，所述程序被处理器执行时实现第一方面所述的方法。According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the method described in the first aspect is implemented.

根据本公开实施例的第五方面，提供一种芯片，包括：处理器，用于从存储器中调用并运行计算机程序，使得安装有所述芯片的设备执行如第一方面所述的方法。According to a fifth aspect of the embodiments of the present disclosure, there is provided a chip, including: a processor, configured to invoke and run a computer program from a memory, so that a device installed with the chip executes the method described in the first aspect.

根据本公开实施例的第六方面，提供一种计算机程序产品，所述计算机程序产品承载有程序代码，所述程序代码包括的指令可配置为执行如第一方面所述的方法。According to a sixth aspect of the embodiments of the present disclosure, a computer program product is provided, the computer program product carries program code, and the program code includes instructions that can be configured to execute the method as described in the first aspect.

根据本公开实施例的第七方面，提供一种计算机程序，包括计算机可读代码，在所述计算机可读代码在电子设备中运行的情况下，所述电子设备中的处理器执行如第一方面所述的方法。According to a seventh aspect of the embodiments of the present disclosure, there is provided a computer program, including computer readable codes. When the computer readable codes run in an electronic device, a processor in the electronic device executes the first method described in the aspect.

根据上述实施例可知，通过获取待检测图像，并对待检测图像进行二值化处理，从而得到二值化图像，然后基于二值化图像确定为灯具候选框，再对待检测图像中对应上述灯具候选框的候选区域进行识别，以确定每个候选区域内是否为灯具，即确定待检测图像中的灯具目标框，灯具目标框为灯具所在的位置。由于通过二值化处理筛选出了灯具候选框，则仅需要对灯具候选框对应的候选区域进行针对性识别，缩小了识别范围，提高了识别精度，增加了灯具检测的精度和鲁棒性。According to the above-mentioned embodiment, it can be seen that by acquiring the image to be detected and performing binarization processing on the image to be detected, a binarized image is obtained, and then determined as a candidate frame of a lamp based on the binarized image, and then the candidate frame corresponding to the above lamp in the image to be detected Identify the candidate areas of the frame to determine whether each candidate area is a lamp, that is, determine the target frame of the lamp in the image to be detected, and the target frame of the lamp is the position of the lamp. Since the candidate frame of the lamp is screened out through the binarization process, only the candidate area corresponding to the candidate frame of the lamp needs to be identified, which reduces the recognition range, improves the recognition accuracy, and increases the accuracy and robustness of the lamp detection.

为使本公开的上述目的、特征和优点能更明显易懂，下文特举较佳实施例，并配合所附附图，作详细说明如下。In order to make the above-mentioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments will be described in detail below together with the accompanying drawings.

附图说明Description of drawings

为了更清楚地说明本公开实施例的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，此处的附图被并入说明书中并构成本说明书中的一部分，这些附图示出了符合本公开的实施例，并与说明书一起用于说明本公开的技术方案。In order to illustrate the technical solutions of the embodiments of the present disclosure more clearly, the following will briefly introduce the accompanying drawings used in the embodiments. The accompanying drawings here are incorporated into the specification and constitute a part of the specification. The drawings show the embodiments consistent with the present disclosure, and are used together with the description to explain the technical solution of the present disclosure.

图1是本公开一实施例示出的灯具检测方法的流程图；Fig. 1 is a flowchart of a lamp detection method shown in an embodiment of the present disclosure;

图2是本公开另一实施例示出的灯具检测方法的流程图Fig. 2 is a flow chart of a lamp detection method shown in another embodiment of the present disclosure

图3是本公开一实施例示出的确定灯具明暗状态的方法的流程图；Fig. 3 is a flow chart of a method for determining the light and dark state of a lamp according to an embodiment of the present disclosure;

图4是本公开另实施例示出的确定灯具明暗状态的方法的流程图；Fig. 4 is a flow chart of a method for determining the light and dark state of a lamp according to another embodiment of the present disclosure;

图5是本公开实施例示出的灯具检测装置的结构示意图；5 is a schematic structural diagram of a lamp detection device shown in an embodiment of the present disclosure;

图6是本公开实施例示出的电子设备的结构示意图；FIG. 6 is a schematic structural diagram of an electronic device shown in an embodiment of the present disclosure;

图7为本公开实施例的芯片的示意性结构图。FIG. 7 is a schematic structural diagram of a chip according to an embodiment of the present disclosure.

具体实施方式Detailed ways

这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本公开相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本公开的一些方面相一致的装置和方法的例子。Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present disclosure as recited in the appended claims.

在本公开使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本公开。在本公开和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。还应当理解，本文中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。The terminology used in the present disclosure is for the purpose of describing particular embodiments only, and is not intended to limit the present disclosure. As used in this disclosure and the appended claims, the singular forms "a", "the", and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It should also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

应当理解，尽管在本公开可能采用术语第一、第二、第三等来描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本公开范围的情况下，第一信息也可以被称为第二信息，类似地，第二信息也可以被称为第一信息。取决于语境，如在此所使用的词语“如果”可以被解释成为“在……时”或“当……时”或“响应于确定”。It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of the present disclosure, first information may also be called second information, and similarly, second information may also be called first information. Depending on the context, the word "if" as used herein may be interpreted as "at" or "when" or "in response to a determination."

另外，本公开实施例所记载的技术方案之间，在不冲突的情况下，可以任意组合。在本公开的描述中，“多个”的含义是两个或两个以上，“多帧”的含义是两帧或两个以帧，除非另有明确具体的限定。In addition, the technical solutions described in the embodiments of the present disclosure may be combined arbitrarily if there is no conflict. In the description of the present disclosure, "multiple" means two or more, and "multiple frames" means two or more frames, unless otherwise specifically defined.

本公开至少一个实施例提供了一种灯具检测方法，请参照附图1，其示出了该方法的流程，该方法可以应用于灯具检测装置或电子设备，包括步骤S101至步骤S103。At least one embodiment of the present disclosure provides a lamp detection method. Please refer to FIG. 1 , which shows the flow of the method. The method can be applied to a lamp detection device or electronic equipment, including steps S101 to S103.

其中，该方法可以用于对楼宇的景观灯进行检测，也可以对其他类型的灯具进行检测，下面主要以景观灯为例对本实施例提供的检测方法进行介绍，但是可以理解的是，这并非对本公开提供的检测方法所针对的灯具类型的限制。即本公开实施例中以灯具为景观灯进行举例，在另一些实施例中，灯具还可以为其它类型的灯，例如，其它类型的灯可以为台灯、家庭照明灯或路灯等。Among them, this method can be used to detect landscape lights of buildings, and can also detect other types of lamps. The following mainly uses landscape lights as an example to introduce the detection method provided in this embodiment, but it can be understood that this is not Restrictions on the types of luminaires targeted by the detection methods provided by this disclosure. That is to say, in the embodiments of the present disclosure, the lamps are taken as landscape lamps for example. In other embodiments, the lamps may also be other types of lamps. For example, other types of lamps may be desk lamps, home lighting lamps, or street lamps.

景观灯一般在较高的位置，因此该检测方法所针对的对象，可以为架设在高楼楼顶的高空广角相机拍摄的图像或视频，因为高空广角相机的视野开阔，可以拍摄到大范围的城市高楼大厦。高空广角相机所拍摄的相机或视频，可以在夜间拍摄，从而使得到的图像或视频中，景观灯较为明显，进而，该方法所检测到的景观灯为被点亮的景光灯，或者说处于明亮状态的景光灯。Landscape lights are generally at a higher position, so the object of this detection method can be an image or video taken by a high-altitude wide-angle camera installed on the roof of a high-rise building, because the high-altitude wide-angle camera has a wide field of vision and can capture a wide range of cities. tall buildings. The camera or video shot by the high-altitude wide-angle camera can be shot at night, so that in the obtained image or video, the landscape lights are more obvious, and then the landscape lights detected by this method are illuminated landscape lights, or Landscape light in bright state.

另外，该方法可以由终端设备或服务器等电子设备执行，终端设备可以为用户设备(User Equipment，UE)、移动设备、用户终端、终端、蜂窝电话、无绳电话、个人数字处理(Personal Digital Assistant，PDA)手持设备、计算设备、车载设备、可穿戴设备等，该方法可以通过处理器调用存储器中存储的计算机可读指令的方式来实现。或者，可以通过服务器执行该方法，服务器可以为本地服务器、云端服务器等。In addition, the method can be executed by electronic equipment such as a terminal device or a server, and the terminal device can be user equipment (User Equipment, UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (Personal Digital Assistant, PDA) handheld devices, computing devices, vehicle-mounted devices, wearable devices, etc., the method can be implemented by calling the computer-readable instructions stored in the memory by the processor. Alternatively, the method may be executed by a server, and the server may be a local server, a cloud server, or the like.

在步骤S101中，获取待检测图像，其中，所述待检测图像对应的场景内包括灯具。In step S101, an image to be detected is acquired, wherein a scene corresponding to the image to be detected includes lamps.

其中，待检测图像可以为上述高空广角相机拍摄的图像，或者高空广角相机录制的视频中的一帧图像。由于高空广角相机的拍摄视角中的场景包括景观灯，因此所得到的待检测图像中包括景观灯的图像。高空广角相机可以按照一定的时间间隔拍摄图像，则获取每次拍摄的图像作为待检测图像；高空广角相机可以持续拍摄视频，则可以按照一定的时间间隔从上述视频中抽取视频帧，并将每次抽取的视频帧作为待检测图像。Wherein, the image to be detected may be an image taken by the above-mentioned high-altitude wide-angle camera, or a frame image in a video recorded by the high-altitude wide-angle camera. Since the scene in the shooting angle of view of the high-altitude wide-angle camera includes landscape lights, the obtained images to be detected include images of landscape lights. The high-altitude wide-angle camera can take images according to a certain time interval, and each captured image is obtained as the image to be detected; the high-altitude wide-angle camera can continuously shoot videos, and the video frames can be extracted from the above-mentioned videos according to a certain time interval, and each The second extracted video frame is used as the image to be detected.

在步骤S102中，将所述待检测图像进行二值化处理，得到二值化图像，并基于所述二值化图像确定灯具候选框。In step S102, the image to be detected is subjected to binarization processing to obtain a binarized image, and a candidate frame of a lamp is determined based on the binarized image.

在一些实施例中，所述将所述待检测图像进行二值化处理，可以包括：将所述待检测图像转换为灰度图像；将所述灰度图像进行二值化处理。In some embodiments, the performing binarization processing on the image to be detected may include: converting the image to be detected into a grayscale image; and performing binarization processing on the grayscale image.

在一些实施例中，如果待检测图像为灰度图像，则直接对待检测图像进行二值化处理。如果是待检测图像为彩色图像，则可以先将待检测图像转换为灰度图像，即可以根据每个像素的红(R)、黄(G)、蓝(B)像素值确定对应的亮度值，从而基于亮度值确定彩色图像对应的灰度图像。In some embodiments, if the image to be detected is a grayscale image, the image to be detected is directly binarized. If the image to be detected is a color image, the image to be detected can be converted into a grayscale image first, that is, the corresponding brightness value can be determined according to the red (R), yellow (G), and blue (B) pixel values of each pixel , so as to determine the grayscale image corresponding to the color image based on the brightness value.

二值化处理时，可以设置一个亮度阈值(1至254之间的任一个值，例如230、200、254等等)，将大于或等于该亮度阈值的像素的亮度调整为255，将小于该亮度阈值的像素的亮度调整为0，从而使每个像素的亮度均为255(明亮像素)和0(黑暗像素)中的一个，相邻的多个明亮像素组成可以组成连通区域，相邻的多个黑暗像素也可以组成连通区域，进而整个二值化图像包括明亮像素组成的连通区域和黑暗像素组成的连通区域。由于待检测图像是在夜间拍摄的，因此(处于明亮状态的)景观灯的亮度是高于场景内的其他物体的，因此明亮像素组成的连通区域为景观灯的概率较大，因此可以确定所述二值化图像中，由明亮像素组成的连通区域为所述灯具候选框。During binarization, you can set a brightness threshold (any value between 1 and 254, such as 230, 200, 254, etc.), adjust the brightness of pixels greater than or equal to the brightness threshold to 255, and adjust the brightness of pixels smaller than the brightness threshold to 255. The brightness of the pixel of the brightness threshold is adjusted to 0, so that the brightness of each pixel is one of 255 (bright pixel) and 0 (dark pixel), and multiple adjacent bright pixels can form a connected area, adjacent A plurality of dark pixels can also form a connected region, and then the entire binary image includes a connected region composed of bright pixels and a connected region composed of dark pixels. Since the image to be detected is taken at night, the brightness of the landscape lights (in a bright state) is higher than that of other objects in the scene, so the connected area composed of bright pixels has a higher probability of being landscape lights, so it can be determined that all In the binarized image, the connected region composed of bright pixels is the candidate frame of the lamp.

在一些实施例中，所述基于所述二值化图像确定灯具候选框，可以包括：对所述二值化图像进行图像腐蚀处理，得到腐蚀图像；基于所述腐蚀图像确定所述灯具候选框。In some embodiments, the determining the candidate frame of the lamp based on the binarized image may include: performing image erosion processing on the binarized image to obtain a corroded image; determining the candidate frame of the lamp based on the corroded image .

二值化处理后，还可以对二值化图像进行图像腐蚀处理，即将周围像素(周围的范围根据预先设置的维度确定，例如2*2的维度)均为明亮像素的明亮像素保持高亮(即将其亮度保持在255)，将周围包括黑暗像素的明亮像素调整为黑暗像素(就将其亮度调整为0)。After the binarization process, the image erosion process can also be performed on the binarized image, that is, the surrounding pixels (the surrounding range is determined according to the preset dimension, for example, the dimension of 2*2) are bright pixels that are all bright pixels, and the bright pixels are kept highlighted ( That is, keep its brightness at 255), and adjust the surrounding bright pixels including dark pixels to dark pixels (just adjust its brightness to 0).

在一些实施例中，所述基于所述二值化图像确定灯具候选框，包括：基于所述二值化图像中由明亮像素组成的连通区域，确定所述灯具候选框。在一些实施方式中，可以将由明亮像素组成的连通区域确定为灯具候选框。In some embodiments, the determining the lamp candidate frame based on the binarized image includes: determining the lamp candidate frame based on a connected region composed of bright pixels in the binarized image. In some implementations, a connected region composed of bright pixels may be determined as a candidate frame of a lamp.

在一些实施例中，所述基于所述二值化图像中由明亮像素组成的连通区域，确定所述灯具候选框，可以包括：基于所述二值化图像中由明亮像素组成的面积大于或等于预设的面积阈值的连通区域，确定所述灯具候选框。In some embodiments, the determining the lamp candidate frame based on the connected area composed of bright pixels in the binarized image may include: based on the area composed of bright pixels in the binarized image is greater than or The connected regions equal to the preset area threshold are used to determine the candidate box of the lamp.

二值化处理后，还可以去除二值化图像中，面积小于预设的面积阈值的连通区域，因为这些小于面积阈值的连通区域可能为噪声点。因为连通区域是被认定为景观灯所在的区域的，因此面积阈值可以根据景观灯所在区域的面积下限确定。After the binarization process, the connected regions whose area is smaller than the preset area threshold in the binarized image can also be removed, because these connected regions smaller than the area threshold may be noise points. Because the connected area is identified as the area where the landscape light is located, the area threshold can be determined according to the lower limit of the area of the area where the landscape light is located.

可以理解的是，上述两种去除噪声的处理(即图像腐蚀处理以及去除面积较小的噪声点的处理)，可以择一使用，也可以都使用。当使用两种处理时，可以先进行图像腐蚀处理，再去除面积较小的噪声点；或者，可以先去除面积较小的噪声点，再进行图像腐蚀处理。It can be understood that one or both of the above two noise removal processes (that is, the image erosion process and the process of removing noise points with a smaller area) can be used. When two kinds of processing are used, the image erosion processing can be performed first, and then the noise points with a smaller area can be removed; or, the noise points with a smaller area can be removed first, and then the image erosion processing can be performed.

在步骤S103中，对所述待检测图像中对应所述灯具候选框的候选区域进行识别，确定所述待检测图像的灯具目标框，其中，所述灯具目标框对应所述灯具所在的位置。In step S103, the candidate area corresponding to the lamp candidate frame in the image to be detected is identified, and the lamp target frame of the image to be detected is determined, wherein the lamp target frame corresponds to the location of the lamp.

其中，可以将每个所述候选区域输入至预先完成训练的分类神经网络，通过所述分类神经网络输出每个所述候选区域的分类结果，其中，所述分类结果包括灯具或非灯具；再将分类结果为灯具的候选区域确定为所述灯具目标框。Wherein, each of the candidate areas may be input into a pre-trained classification neural network, and a classification result of each of the candidate areas is output through the classification neural network, wherein the classification results include lamps or non-lamps; The classification result is determined as the candidate area of the luminaire as the target frame of the luminaire.

在一些实施例中，可以预先准备训练集，训练集中的样本图像可以选出多个局部区域，并将这些局部区域中为灯具的像的区域标注为灯具目标框，将剩余的局部区域标注为对照框；然后将样本图像中输入至分类神经网络中，输出对各个局部区域的预测结果，根据预测结果与对应的标注结果确定网络损失值，并进一步根据网络损失值调整分类神经网络的网络参数，直至该分类神经网络收敛。In some embodiments, the training set can be prepared in advance, and a plurality of local regions can be selected from the sample images in the training set, and the region that is the image of the lamp in these local regions is marked as the target frame of the lamp, and the remaining local regions are marked as Control frame; then input the sample image into the classification neural network, output the prediction results for each local area, determine the network loss value according to the prediction results and corresponding labeling results, and further adjust the network parameters of the classification neural network according to the network loss value , until the classification neural network converges.

灯具目标框可以为能够框住灯具的像的最小矩形框。每个灯具目标框内具有一个灯具，且灯具的状态为明亮状态的景观灯，换句话说，本步骤中所确定的待检测图像的灯具目标框，为待检测图像中处于明亮状态的灯具的像。The target frame of the luminaire may be the smallest rectangular frame capable of enclosing the image of the luminaire. There is a lamp in each lamp target frame, and the state of the lamp is a landscape lamp in a bright state. In other words, the lamp target frame of the image to be detected determined in this step is the brightness picture.

根据上述实施例可知，通过获取待检测图像，并对待检测图像进行二值化处理，从而得到二值化图像，然后基于二值化图像确定灯具候选框，再对待检测图像中对应上述灯具候选框的候选区域进行识别，以确定每个候选区域内是否为灯具，即确定待检测图像中的灯具目标框，灯具目标框为灯具所在的位置。由于通过二值化处理筛选出了灯具候选框，则仅需要对灯具候选框对应的候选区域进行针对性识别，缩小了识别范围，提高了识别精度，增加了灯具检测的精度和鲁棒性。According to the above-mentioned embodiment, it can be seen that by acquiring the image to be detected and performing binarization processing on the image to be detected, a binarized image is obtained, and then the candidate frame of the lamp is determined based on the binarized image, and then the candidate frame of the lamp corresponding to the above-mentioned lamp in the image to be detected Identify the candidate areas to determine whether each candidate area is a lamp, that is, determine the target frame of the lamp in the image to be detected, and the target frame of the lamp is the position of the lamp. Since the candidate frame of the lamp is screened out through the binarization process, only the candidate area corresponding to the candidate frame of the lamp needs to be identified, which reduces the recognition range, improves the recognition accuracy, and increases the accuracy and robustness of the lamp detection.

在本公开的一些实施例中，所述待检测图像为待检测视频的一帧图像，所述待检测视频对应的场景内包括灯具；In some embodiments of the present disclosure, the image to be detected is a frame image of a video to be detected, and the scene corresponding to the video to be detected includes lamps;

基于此，请参照附图2，所述灯具检测方法在步骤S101之后，灯具检测方法还包括步骤S104：根据所述待检测图像的灯具目标框和参考图像的灯具目标框，确定所述参考图像的灯具目标框对应的灯具的明暗状态，其中，所述参考图像为所述待检测视频中，所述待检测视频之前的一帧图像。Based on this, please refer to the accompanying drawing 2, after step S101, the lamp detection method further includes step S104: according to the lamp target frame of the image to be detected and the lamp target frame of the reference image, determine the reference image The light and dark states of the lamps corresponding to the target frame of the lamps, wherein the reference image is an image of a frame before the video to be detected in the video to be detected.

其中，本实施例提供的检测方法是针对每一次获取的待检测图像重复循环运行的，也就是说，每次从待检测视频中获取到待检测图像后均按照步骤S101至步骤S103确定出灯具目标框。由于上一次获取到的待检测图像也确定出了灯具目标框，因此可以通过比较本次待检测图像的目标框和上一次待检测图像的目标框，确定上一次待检测图像中每个目标框对应的灯具的明暗状态，例如，上一次待检测图像中的某个目标框在本次待检测图像中的对应区域也具有目标框，则认为该目标框对应的灯具为明亮状态，即在两次获取待检测图像之间该灯具未熄灭；上一次待检测图像中的某个目标框在本次待检测图像中的对应区域不具有目标框，则认为该目标框对应的灯具为黑暗状态，即在两次获取待检测图像之间该灯具已熄灭。Among them, the detection method provided by this embodiment is to repeatedly run in a loop for each acquired image to be detected, that is to say, each time the image to be detected is obtained from the video to be detected, the lamp is determined according to steps S101 to S103. target box. Since the last image to be detected has also determined the target frame of the lamp, it is possible to determine each target frame in the last image to be detected by comparing the target frame of the image to be detected this time with the target frame of the last image to be detected The light and dark state of the corresponding lamp, for example, if a certain target frame in the image to be detected last time also has a target frame in the corresponding area of the image to be detected this time, then the lamp corresponding to the target frame is considered to be in a bright state, that is, in the two The lamp is not extinguished between the acquisition of the image to be detected for the second time; if a target frame in the image to be detected last time does not have a target frame in the corresponding area of the image to be detected this time, then the lamp corresponding to the target frame is considered to be in a dark state. That is, the lamp has been turned off between two acquisitions of images to be detected.

需要注意的是，为了区分两次获取的待检测图像，可以将上一次获取的待检测图像称之为参考图像，将本次获取的待检测图像继续称之为待检测图像。因此待检测图像和参考图像为待检测视频中的两帧不同时刻的图像，参考图像在前，待检测图像在后。It should be noted that, in order to distinguish the images to be detected acquired twice, the image to be detected to be acquired last time may be called a reference image, and the image to be detected to be detected acquired this time may continue to be called an image to be detected to be detected. Therefore, the image to be detected and the reference image are images of two frames at different times in the video to be detected, the reference image is in front, and the image to be detected is in the back.

由于每次获取的待检测图像，在确定出灯具目标框之后，需要在下一次获取的待检测图像确定出灯具目标框后，作为参考图像使用，因此每一次获取的待检测图像在确定出灯具目标框后均保存待用。另外，由于每一次获取的待检测图像只被作为参考图像使用一次，因此在步骤S104确定出参考图像的每个灯具目标框对应的灯具的明暗状态后，可以删除所述参考图像，将所述待检测图像保存为参考图像，并保存所述待检测图像的灯具目标框。Since the image to be detected acquired each time needs to be used as a reference image after the target frame of the lamp is determined in the image to be detected to be acquired next time, the image to be detected each time is determined after the target of the lamp is determined. All frames are saved for later use. In addition, since the image to be detected acquired each time is only used once as a reference image, after the light and dark state of the lamp corresponding to each lamp target frame of the reference image is determined in step S104, the reference image can be deleted, and the The image to be detected is saved as a reference image, and the lamp target frame of the image to be detected is saved.

可以理解的是，当第一次从待检测视频中获取待检测图像后，在确定出该待检测图像的灯具目标框后，不执行步骤S104，而是直接将该待检测图像保存为参考图像，且保存该待检测图像的灯具目标框。It can be understood that when the image to be detected is acquired from the video to be detected for the first time, after the target frame of the lamp of the image to be detected is determined, step S104 is not executed, but the image to be detected is directly saved as a reference image , and save the lamp target frame of the image to be detected.

在一个示例中，可以按照一定的时间间隔获取待检测图像，即获取待检测图像时，是从待检测视频中，获取的与所述参考图像的时间间隔为预设时长的一帧图像，确定为所述待检测图像。预设时长可以为3s，按照该预设时长可以重复的获取待检测视频。In an example, the image to be detected can be acquired at a certain time interval, that is, when the image to be detected is acquired, a frame of image with a time interval of a preset duration from the reference image is obtained from the video to be detected, and determined is the image to be detected. The preset duration may be 3s, and the video to be detected may be acquired repeatedly according to the preset duration.

在一个示例中，可以按照如图3所示的方式，根据所述待检测图像的灯具目标框和参考图像的灯具目标框，确定所述参考图像的灯具目标框对应的灯具的明暗状态，包括步骤S301至步骤S303。In an example, as shown in FIG. 3 , according to the lamp target frame of the image to be detected and the lamp target frame of the reference image, the light and dark state of the lamp corresponding to the lamp target frame of the reference image is determined, including Step S301 to step S303.

在步骤S301中，确定所述参考图像的每个灯具目标框和所述待检测图像的每个灯具目标框的交并比。In step S301, an intersection ratio between each lamp object frame of the reference image and each lamp object frame of the image to be detected is determined.

按照顺序依次取参考图像中的每个灯具目标框，并在每次取得灯具目标框后，依次确定该灯具目标框与待检测图像的每个灯具目标框的交并比。Each lamp target frame in the reference image is sequentially obtained, and after each lamp target frame is obtained, the intersection ratio between the lamp target frame and each lamp target frame of the image to be detected is sequentially determined.

在步骤S302中，在所述参考图像的灯具目标框与所述待检测图像的每个灯具目标框的交并比，均小于预设的第一比例阈值的情况下，确定所述参考图像的灯具目标框对应的灯具的明暗状态为黑暗。In step S302, when the intersection ratios of the lamp object frame of the reference image and each lamp object frame of the image to be detected are smaller than a preset first ratio threshold, determine the The light and shade state of the light fixture corresponding to the light fixture target frame is dark.

在步骤S303中，在所述参考图像的灯具目标框与所述待检测图像的任一灯具目标框的交并比的情况下，大于或等于所述第一比例阈值的情况下，确定所述参考图像的灯具目标框对应的灯具的明暗状态为明亮。In step S303, in the case that the intersection ratio between the lamp target frame of the reference image and any lamp target frame of the image to be detected is greater than or equal to the first ratio threshold, determine the The light and dark state of the light fixture corresponding to the light fixture target frame of the reference image is bright.

参考图像的灯具目标框与待检测图像的灯具目标框的交并比，大于预设的第一比例阈值，则表征这两个灯具目标框对应的场景相同，或者说对应着场景内同一个的灯具，即参考图像中的灯具目标框在待检测图像中存在匹配的灯具目标框，则可以初步确定该灯具目标框对应的灯具依然处于明亮状态，未熄灭，因此将该灯具目标框对应的灯具的明暗状态为明亮。If the intersection ratio between the lamp target frame of the reference image and the lamp target frame of the image to be detected is greater than the preset first ratio threshold, it means that the scenes corresponding to the two lamp target frames are the same, or correspond to the same scene in the scene. lamps, that is, the lamp target frame in the reference image has a matching lamp target frame in the image to be detected, then it can be preliminarily determined that the lamp corresponding to the lamp target frame is still bright and not extinguished, so the lamp corresponding to the lamp target frame The light and dark state of is bright.

相对应的，参考图像的某灯具目标框与待检测图像的任一灯具目标框的交并比，小于预设的第一比例阈值，则表征待检测图像中无与该灯具目标框匹配的灯具目标框。因此，可以确定所述参考图像的灯具目标框对应的灯具的明暗状态为黑暗。Correspondingly, if the intersection ratio between a certain lamp target frame in the reference image and any lamp target frame in the image to be detected is less than the preset first ratio threshold, it means that there is no lamp in the image to be detected that matches the target frame of the lamp target box. Therefore, it may be determined that the light and dark state of the light fixture corresponding to the light fixture target frame of the reference image is dark.

本示例中，通过灯具目标框的交并比确定灯具的明暗状态，由于灯具目标框对应灯具所在的位置，因此这种方式能够简单且准确地确定灯具明暗状态。In this example, the light and shade state of the light fixture is determined by the intersection ratio of the target frame of the light fixture. Since the target frame of the light fixture corresponds to the position of the light fixture, this method can simply and accurately determine the light and shade state of the light fixture.

在另一个示例中，可以按照如图4所示的方式，根据所述待检测图像的灯具目标框和参考图像的灯具目标框，确定所述参考图像的灯具目标框对应的灯具的明暗状态，包括步骤S401至步骤S403。In another example, according to the method shown in FIG. 4 , according to the target frame of the target image of the image to be detected and the target frame of the target image of the lamp, the light and dark state of the lamp corresponding to the target frame of the lamp of the reference image is determined, It includes step S401 to step S403.

在步骤S401中，确定所述参考图像的所述灯具目标框中的全部像素的像素值平均值为第一平均值，并确定所述待检测图像中对应的所述灯具目标框中的全部像素的像素值平均值为第二平均值。In step S401, it is determined that the average value of the pixel values of all pixels in the lamp target frame of the reference image is the first average value, and all pixels in the corresponding lamp target frame in the image to be detected are determined The pixel value average of is the second average.

在步骤S402中，在所述第一平均值与所述第二平均值的差值大于预设的第一像素阈值的情况下，确定所述灯具目标框对应的灯具的明暗状态为黑暗。In step S402, if the difference between the first average value and the second average value is greater than a preset first pixel threshold, it is determined that the light and shade state of the lamp corresponding to the lamp target frame is dark.

在步骤S403中，在所述第一平均值与所述第二平均值的差值小于或等于预设的第一像素阈值的情况下，确定所述灯具待定框对应的灯具的明暗状态为明亮。In step S403, when the difference between the first average value and the second average value is less than or equal to the preset first pixel threshold, it is determined that the light and dark state of the light fixture corresponding to the light fixture pending frame is bright .

其中，第一像素阈值可以预设为100。处于黑暗状态，表征该灯具在参考图像对应的时刻之后出现了熄灭。Wherein, the first pixel threshold may be preset as 100. In the dark state, it means that the lamp has been extinguished after the corresponding moment of the reference image.

本实施例中，通过对应目标框的平均像素差，精确地确定灯具是否熄灭，从而提高了灯具明暗状态的识别精度。In this embodiment, whether the lamp is extinguished is accurately determined through the average pixel difference corresponding to the target frame, thereby improving the recognition accuracy of the light and dark state of the lamp.

可以理解的是，上述如图3所示的示例中的确定灯具的明暗状态的方式，和如图4所示的示例中的确定灯具的明暗状态的方式，可以择一使用，也可以结合使用。在结合两种方式确定灯具的明暗状态时，可以先利用如图3所示的方式，所不同的是，在执行步骤S302时，在所述参考图像的灯具目标框与所述待检测图像的每个灯具目标框的交并比，均小于预设的第一比例阈值的情况下，确定所述参考图像的灯具目标框为候选目标框；然后再执行如图4所示的方式，所不同的是，步骤S401中仅确定候选目标框中的全部像素的像素值平均值为第一平均值，仅确定待检测图像中与候选目标框对应的灯具目标框中的全部像素的像素值平均值为第二平均值，步骤S402和步骤S403中仅利用第一平均值和第二平均值确定灯具候选框对应的灯具的明暗状态，即在在所述第一平均值与所述第二平均值的差值大于预设的第一像素阈值，确定所述灯具候选框对应的灯具的明暗状态为黑暗，在所述第一平均值与所述第二平均值的差值小于或等于预设的第一像素阈值，确定所述灯具候选框对应的灯具的明暗状态为明亮。It can be understood that, the method of determining the light and dark state of the lamp in the example shown in FIG. 3 and the method of determining the light and dark state of the lamp in the example shown in FIG. 4 can be used alternatively, or can be used in combination. . When combining the two methods to determine the light and dark state of the lamp, the method shown in Figure 3 can be used first. When the intersection and union ratio of each lamp target frame is less than the preset first ratio threshold, determine that the lamp target frame of the reference image is a candidate target frame; then execute the method as shown in Figure 4, the difference Notably, in step S401, only the average value of the pixel values of all pixels in the candidate target frame is determined as the first average value, and only the average pixel value of all pixels in the lamp target frame corresponding to the candidate target frame in the image to be detected is determined For the second average value, in steps S402 and S403, only the first average value and the second average value are used to determine the light and dark state of the lamp corresponding to the lamp candidate frame, that is, between the first average value and the second average value The difference between the first average value and the second average value is less than or equal to the preset The first pixel threshold determines that the light and dark state of the lamp corresponding to the lamp candidate frame is bright.

结合两种方式判断灯具目标框对应的灯具的明暗状态，首先利用目标框之间的交并比筛选出可能出现熄灭的目标框，即候选框，然后重点对候选框进行像素对比来确定灯具的明暗状态。既提高了灯具明暗状态判断的准确性，又避免了过多提高运算负荷。Combining the two methods to judge the light and dark state of the lamp corresponding to the target frame of the lamp, first use the intersection ratio between the target frames to filter out the target frame that may appear to be extinguished, that is, the candidate frame, and then focus on the pixel comparison of the candidate frame to determine the brightness of the lamp. light and dark state. It not only improves the accuracy of judging the light and dark state of the lamp, but also avoids excessively increasing the calculation load.

本公开的一些实施例中，可以按照下述方式将所述待检测图像进行二值化处理：首先，确定所述待检测图像和所述参考图像间的每个像素对的像素差，其中，所述像素对中的一个像素点在所述待检测图像中的位置，和另一个像素点在所述参考图像中的位置相同；接下来，在全部的像素对中，确定像素差大于预设的第二像素阈值的像素对的第一数量；最后，在所述第一数量与全部的像素对的数量的比例大于预设的第二比例阈值的情况下，将所述待检测图像进行二值化处理。In some embodiments of the present disclosure, the image to be detected may be binarized in the following manner: first, the pixel difference of each pixel pair between the image to be detected and the reference image is determined, wherein, The position of one pixel in the pixel pair in the image to be detected is the same as the position of another pixel in the reference image; next, in all the pixel pairs, it is determined that the pixel difference is greater than the preset The first number of pixel pairs of the second pixel threshold; finally, in the case that the ratio of the first number to the number of all pixel pairs is greater than the preset second ratio threshold, the image to be detected is subjected to two value processing.

其中，像素差可以通过对应的两个像素相减得到的差，并对该差取绝对值得到。像素差的大小，可以用来表征待检测图像对应的场景与参考图像对应的场景的匹配程度，而小于或等于第二像素阈值的像素差，表征在两张图像中的该像素对应的场景相同，大于第二像素阈值的像素差，表征在两张图像中的该像素对应的场景不同。进一步的，第一数量与全部的像素对的数量的比例大于预设的第二比例阈值，表征两张图像对应的场景相同，即高空广角相机在获取这两张图像时的角度相同，在获取参考图像后未发生移动，因此这种情况下可以对待检测图像进行二值化处理，并进一步按照后续步骤确定参考图像的目标框对应的灯具的明暗状态。Wherein, the pixel difference can be obtained by subtracting the difference between two corresponding pixels, and taking the absolute value of the difference. The size of the pixel difference can be used to represent the matching degree of the scene corresponding to the image to be detected and the scene corresponding to the reference image, and the pixel difference less than or equal to the second pixel threshold indicates that the scene corresponding to the pixel in the two images is the same , the pixel difference greater than the second pixel threshold indicates that the scenes corresponding to the pixel in the two images are different. Further, the ratio of the first number to the number of all pixel pairs is greater than the preset second ratio threshold, indicating that the scenes corresponding to the two images are the same, that is, the high-altitude wide-angle camera has the same angle when acquiring the two images. There is no movement after the reference image, so in this case, the image to be detected can be binarized, and the light and dark state of the lamp corresponding to the target frame of the reference image can be further determined according to the subsequent steps.

相对应的，在所述第一数量与全部的像素对的数量的比例小于或等于预设的第二比例阈值的，则表征两张图像对应的场景不同，即高空广角相机在获取这两张图像时的角度不同，在获取参考图像后发生了移动。因此这种情况下，不对待检测图像进行二值化处理，即舍弃该待检测图像，进一步的可以再重新获取待检测图像。Correspondingly, when the ratio of the first number to the number of all pixel pairs is less than or equal to the preset second ratio threshold, it indicates that the scenes corresponding to the two images are different, that is, the high-altitude wide-angle camera captures the two images The angles of the images are different, and the movement occurred after the reference image was acquired. Therefore, in this case, the image to be detected is not binarized, that is, the image to be detected is discarded, and the image to be detected can be acquired again.

本实施例中，每次获取待检测图像后，确定第一数量与全部的像素对的数量的比例是否大于预设的第二比例阈值，从而确定待检测图像与参考图像对应的场景是否相同，即确定高空广角相机是否发生移动，若场景不同则舍弃，若场景相同则进一步确定灯具的明暗状态，从而避免了高空广角相机移动造成检测结果不准确的问题，提高了灯具目标框的检测精度以及灯具明暗状态的确定精度。In this embodiment, after each acquisition of the image to be detected, it is determined whether the ratio of the first number to the number of all pixel pairs is greater than a preset second ratio threshold, so as to determine whether the scene corresponding to the image to be detected and the reference image are the same, That is to determine whether the high-altitude wide-angle camera moves, discard it if the scene is different, and further determine the light and dark state of the lamp if the scene is the same, thereby avoiding the problem of inaccurate detection results caused by the movement of the high-altitude wide-angle camera, improving the detection accuracy of the target frame of the lamp and The determination accuracy of the light and dark state of the lamp.

本申请实施例中获取待检测图像，其中，所述待检测图像对应的场景内包括灯具；将所述待检测图像进行二值化处理，得到二值化图像，并基于所述二值化图像确定灯具候选框；对所述待检测图像中对应所述灯具候选框的候选区域进行识别，确定所述待检测图像的灯具目标框，其中，所述灯具目标框对应所述灯具所在的位置。In the embodiment of the present application, the image to be detected is acquired, wherein the scene corresponding to the image to be detected includes lamps; the image to be detected is subjected to binarization processing to obtain a binarized image, and based on the binarized image Determine a lamp candidate frame; identify a candidate area corresponding to the lamp candidate frame in the image to be detected, and determine a lamp target frame in the image to be detected, wherein the lamp target frame corresponds to the location of the lamp.

本公开实施例中的灯具检测方法可以包括以下三种方案：相机摄像头转动判断，景观灯区域确定以及熄灭状态判断。The lamp detection method in the embodiment of the present disclosure may include the following three schemes: camera head rotation judgment, landscape lamp area determination and extinguished state judgment.

对于相机摄像头转动判断，电子设备的输入为接入的相机视频流，输出为摄像头是否存在转动。对相机拍摄的相机视频流，可以间隔T帧(如75帧，一般3秒)取一帧，并缓存，将该帧作为参考帧(即上述的参考图像)，从而有缓存的第N-T帧(即上述的参考图像)和当前的第N帧(即上述的待检测图像)，对这两帧做图像差的运算(对应上述步骤：确定所述待检测图像和所述参考图像间的每个像素对的像素差)。示例性地，方法包括以下三步：For judging the rotation of the camera, the input of the electronic device is the video stream of the connected camera, and the output is whether there is rotation of the camera. For the camera video stream captured by the camera, one frame can be taken at an interval of T frames (such as 75 frames, generally 3 seconds), and cached, and the frame is used as a reference frame (ie, the above-mentioned reference image), so that there is a cached N-T frame ( That is, the above-mentioned reference image) and the current Nth frame (that is, the above-mentioned image to be detected), the calculation of the image difference is performed on these two frames (corresponding to the above steps: determine each frame between the image to be detected and the reference image Pixel difference of a pixel pair). Exemplarily, the method includes the following three steps:

(1)确定两图像对应的像素值相减后的绝对值，当绝对值大于设定的阈值(即上述的第二像素阈值)时，计超出阈值的数量加1，最终得到的相差比较大的像素个数总数diff_count(数值与上述的第一数量相同)；(1) Determine the absolute value after the subtraction of the pixel values corresponding to the two images. When the absolute value is greater than the set threshold (that is, the second pixel threshold above), add 1 to the number exceeding the threshold, and the final difference is relatively large. The total number of pixels diff_count (the value is the same as the first number above);

(2)计算差异率，差异率＝diff_count/像素总数；其中，像素总数的数值与上述的全部的像素对的数量的数值相同，差异率即是第一数量与全部的像素对的数量的比例；(2) Calculate the difference rate, difference rate=diff_count/total number of pixels; wherein, the value of the total number of pixels is the same as the value of the number of all pixel pairs mentioned above, and the difference rate is the ratio of the first number to the number of all pixel pairs ;

(3)如果差异率>设定的阈值(即上述的第二比例阈值)，则认为摄像头已经转动，本次不进行后面的景观灯熄灭状态判断。(3) If the difference rate > the set threshold (that is, the above-mentioned second ratio threshold), it is considered that the camera has rotated, and this time the subsequent landscape light off state judgment is not performed.

对于景观灯区域确定，电子设备输入的是第N帧的图像，输出的是第N帧的景观灯目标框。对当前帧图像(即上述的待检测图像)，如果不是灰度图，先转为灰度图；根据阈值(如230)对灰度图的图像像素值进行二值化处理，即像素值>＝230的设置为255，像素值<230的设置为0；然后进行图像腐蚀，以设置的维度进行(如2×2)，图像腐蚀的目的是为了消除噪声点，平滑边缘。在图像腐蚀之后，根据像素值255查找连通域，基于一个连通域确定一个矩形框。查找连通域相当于查找高亮的区域。在一些实施例中，可以根据设定的面积阈值，过滤掉一些较小的连通域，较小的连通域有可能是噪声点。将找到的候选的高亮区域对应的矩形框对应的区域(即上述的灯具候选框的候选区域)，送入深度学习的景观灯二分类网络(即上述的预先完成训练的分类神经网络)，确认矩形框对应的区域内是否存在景观灯，得到确认的景观灯目标框(即上述的灯具目标框)。For the determination of the landscape light area, the input of the electronic device is the image of the Nth frame, and the output is the target frame of the Nth frame of the landscape light. For the current frame image (that is, the above-mentioned image to be detected), if it is not a grayscale image, first convert it to a grayscale image; perform binarization on the image pixel value of the grayscale image according to the threshold (such as 230), that is, the pixel value> =230 is set to 255, and pixel value <230 is set to 0; then image erosion is performed in the set dimension (such as 2×2). The purpose of image erosion is to eliminate noise points and smooth edges. After the image is corroded, the connected domain is searched according to the pixel value 255, and a rectangular frame is determined based on a connected domain. Finding connected domains is equivalent to finding highlighted regions. In some embodiments, some smaller connected domains may be filtered out according to the set area threshold, and the smaller connected domains may be noise points. The region corresponding to the rectangular frame corresponding to the candidate highlighted region found (that is, the candidate region of the above-mentioned lamp candidate frame) is sent to the landscape lamp binary classification network of deep learning (that is, the above-mentioned pre-trained classification neural network), Confirm whether there is a landscape light in the area corresponding to the rectangular frame, and obtain the confirmed target frame of the landscape light (that is, the target frame of the above-mentioned lamp).

对于熄灭状态判断，电子设备输入的是第N-T帧和第N帧的景观灯目标框，输出的是第N帧的景观灯是否熄灭。第N-T帧的景观灯目标框，和第N帧的所有景观灯目标框进行相交判断，最大的交并比小于设定阈值的，可以认为这个景观灯在第N-T帧存在，在第N帧里不存在，所有这样的景观灯作为熄灭的景观灯候选框(即灯具候选框)。对于景观灯候选框的在第N-T帧和N帧的对应区域的像素值，分别计算均值，两个均值相减的绝对值大于设定阈值(即上述的第一像素阈值，例如100)，则认为前后帧相差比较大，确认为景观灯熄灭。For the judgment of the extinguished state, the input of the electronic device is the N-T frame and the landscape light target frame of the Nth frame, and the output is whether the landscape light of the Nth frame is extinguished. The target frame of the landscape light in the N-T frame is intersected with all the target frames of the N-th frame. If the maximum intersection ratio is less than the set threshold, it can be considered that the landscape light exists in the N-T frame. In the N frame does not exist, all such landscape lights are regarded as extinguished landscape light candidate frames (ie lamp candidate frames). For the pixel values of the corresponding regions of the N-T frame and the N frame of the landscape light candidate frame, calculate the mean value respectively, and the absolute value of the subtraction of the two mean values is greater than the set threshold (ie the above-mentioned first pixel threshold value, such as 100), then It is considered that the difference between the front and rear frames is relatively large, and it is confirmed that the landscape lights are off.

在一些实施例中，可以将缓存的第N-T帧及对应的景观灯目标框(即上述的灯具目标框)删除，更新为当前第N帧及对应的景观灯目标框，待下次第N+T帧时处理判断。In some embodiments, the cached N-T frame and the corresponding landscape lighting target frame (that is, the above-mentioned lighting target frame) can be deleted, updated to the current Nth frame and the corresponding landscape lighting target frame, and the N+T next time Frame time processing judgment.

在本公开实施例中，对接入的相机视频，间隔一定时间取帧(如3秒)，对前一帧和后一帧的图像像素进行相减，如果图像差异率超出设定的阈值(即上述的第二比例阈值)，就认为相机转动了，不再进行本次的景观灯熄灭判断。利用图像处理的方法，将待检测图像先转为灰度图，然后进行图像二值化和图像腐蚀处理，查找连通域以找到高亮区域，将找到的候选的高亮区域对应的矩形框对应的区域，送入分类网络，以确定该区域是否为景观灯或是否包括景观灯。根据前后帧确定的景观灯目标框进行框的交并比计算，确定后一帧无框，前一帧有框的景观灯，确认为该景观灯熄灭。In the embodiment of the present disclosure, for the connected camera video, frames are taken at intervals (such as 3 seconds), and the image pixels of the previous frame and the next frame are subtracted. If the image difference rate exceeds the set threshold ( That is, the above-mentioned second ratio threshold), it is considered that the camera has rotated, and the judgment of turning off the landscape light is no longer performed this time. Using the method of image processing, the image to be detected is first converted into a grayscale image, and then image binarization and image erosion processing are performed, the connected domain is searched to find the highlighted area, and the rectangular frame corresponding to the found candidate highlighted area is corresponding to The area of the area is sent to the classification network to determine whether the area is a landscape light or whether it includes a landscape light. According to the target frame of the landscape light determined in the front and rear frames, the intersection and union ratio of the frame is calculated. If the landscape light is determined to have no frame in the next frame and has a frame in the previous frame, it is confirmed that the landscape light is off.

在本公开实施例中，利用图像差异率进行摄像头转动判断，消除由于摄像头转动而导致前后帧画面变化，从而带来景观灯熄灭判断的不准确。利用图像处理方法确定景观灯区域(即上述的灯具候选框的候选区域)，然后进行是否存在景观灯的识别，能够提高景观灯的识别精度。利用前后帧的对比的算法逻辑，来确定景观灯是否熄灭，能够提高景观灯是否熄灭的准确性。In the embodiment of the present disclosure, the image difference ratio is used to judge the rotation of the camera, eliminating the inaccurate judgment of turning off the landscape light caused by the change of the front and rear frames due to the rotation of the camera. Using an image processing method to determine the landscape light area (ie, the candidate area of the above-mentioned lamp candidate frame), and then identify whether there is a landscape light, can improve the recognition accuracy of the landscape light. Using the algorithm logic of comparing the front and back frames to determine whether the landscape lights are off can improve the accuracy of whether the landscape lights are off.

本公开实施例还提供一种灯具检测装置，请参照附图5，其示出了灯具检测装置500的结构，包括：The embodiment of the present disclosure also provides a lamp detection device, please refer to accompanying drawing 5, which shows the structure of a lamp detection device 500, including:

获取部分501，配置为获取待检测图像，其中，所述待检测图像对应的场景内包括灯具；The acquiring part 501 is configured to acquire the image to be detected, wherein the scene corresponding to the image to be detected includes lamps;

处理部分502，配置为将所述待检测图像进行二值化处理，得到二值化图像，并基于所述二值化图像确定灯具候选框；The processing part 502 is configured to perform binarization processing on the image to be detected to obtain a binarized image, and determine a lamp candidate frame based on the binarized image;

确定部分503，配置为对所述待检测图像中对应所述灯具候选框的候选区域进行识别，确定所述待检测图像的灯具目标框，其中，所述灯具目标框对应所述灯具所在的位置。The determination part 503 is configured to identify the candidate area corresponding to the lamp candidate frame in the image to be detected, and determine the target frame of the lamp in the image to be detected, wherein the target frame of the lamp corresponds to the location of the lamp .

所述装置还包括判断部分，配置为：The device also includes a judging part configured to:

在一个示例中，所述处理部分还配置为：将所述待检测图像转换为灰度图像；将所述灰度图像进行二值化处理。In an example, the processing part is further configured to: convert the image to be detected into a grayscale image; and perform binarization on the grayscale image.

在一个示例中，所述处理部分还配置为：对所述二值化图像进行图像腐蚀处理，得到腐蚀图像；基于所述腐蚀图像确定所述灯具候选框。In an example, the processing part is further configured to: perform image erosion processing on the binarized image to obtain an erosion image; determine the candidate frame of the lamp based on the erosion image.

在一个示例中，所述处理部分还配置为：基于所述二值化图像中由明亮像素组成的连通区域，确定所述灯具候选框。In an example, the processing part is further configured to: determine the lamp candidate frame based on a connected region composed of bright pixels in the binarized image.

在一个示例中，所述处理部分还配置为：基于所述二值化图像中由明亮像素组成的面积大于或等于预设的面积阈值的连通区域，确定所述灯具候选框。In an example, the processing part is further configured to: determine the lamp candidate frame based on connected regions in the binarized image whose area composed of bright pixels is greater than or equal to a preset area threshold.

在一个示例中，所述确定部分还配置为：将每个所述候选区域输入至预先完成训练的分类神经网络，通过所述分类神经网络输出每个所述候选区域的分类结果，其中，所述分类结果包括灯具或非灯具；将分类结果为灯具的候选区域确定为所述灯具目标框。In an example, the determining part is further configured to: input each of the candidate regions into a pre-trained classification neural network, and output a classification result of each of the candidate regions through the classification neural network, wherein the The classification result includes lamps or non-lamps; and the candidate area where the classification result is a lamp is determined as the target frame of the lamp.

在一个示例中，所述获取部分还配置为：将从所述待检测视频中获取的与所述参考图像的时间间隔为预设时长的一帧图像，确定为所述待检测图像。In an example, the obtaining part is further configured to: determine a frame of image acquired from the video to be detected with a time interval of a preset duration from the reference image as the image to be detected.

在一个示例中，灯具检测装置还包括缓存部分，配置为：删除所述参考图像，将所述待检测图像保存为参考图像，并保存所述待检测图像的灯具目标框。In one example, the lamp detection device further includes a cache part configured to: delete the reference image, save the image to be detected as a reference image, and save the target frame of the lamp in the image to be detected.

关于上述实施例中的装置，其中各个部分执行操作的实施方式已经在上述方法实施例中进行了详细描述，此处将不做详细阐述说明。With regard to the apparatus in the above embodiments, the implementation manner in which each part performs operations has been described in detail in the above method embodiments, and will not be described in detail here.

以上装置实施例的描述，与上述方法实施例的描述是类似的，具有同方法实施例相似的有益效果。对于本公开装置实施例中未披露的技术细节，请参照本公开方法实施例的描述而理解。The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiments of the present disclosure, please refer to the description of the method embodiments of the present disclosure for understanding.

本公开实施例还提供了一种电子设备，请参照附图6，其示出了该电子设备600的结构，所述电子设备600包括存储器601和处理器602，所述存储器601用于存储可在处理器602上运行的计算机指令，所述处理器602用于在执行所述计算机指令时实现上述任一实施例中的方法。An embodiment of the present disclosure also provides an electronic device. Please refer to FIG. 6, which shows the structure of the electronic device 600. The electronic device 600 includes a memory 601 and a processor 602. Computer instructions running on the processor 602, the processor 602 is configured to implement the method in any of the foregoing embodiments when executing the computer instructions.

本公开实施例还提供了一种计算机可读存储介质，其上存储有计算机程序，所述程序被处理器执行时实现上述任一实施例中的方法。An embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the method in any of the above-mentioned embodiments is implemented.

本公开实施例还提供了一种芯片，请参照附图7，图7所示的芯片700包括处理器710，用于从存储器中调用并运行计算机程序，使得安装有所述芯片的设备执行上述任一实施例中的方法。The embodiment of the present disclosure also provides a chip, please refer to accompanying drawing 7, the chip 700 shown in Fig. 7 includes processor 710, is used for calling and running computer program from memory, makes the device that described chip is installed execute above-mentioned The method in any of the examples.

在一些实施例中，如图7所示，芯片700还可以包括存储器720。其中，处理器710可以从存储器720中调用并运行计算机程序，以实现上述任一实施例中的方法。In some embodiments, as shown in FIG. 7 , the chip 700 may further include a memory 720 . Wherein, the processor 710 can invoke and run a computer program from the memory 720, so as to implement the method in any of the foregoing embodiments.

其中，存储器720可以是独立于处理器710的一个单独的器件，也可以集成在处理器710中。Wherein, the memory 720 may be an independent device independent of the processor 710 , or may be integrated in the processor 710 .

在一些实施例中，该芯片700还可以包括输入接口730。其中，处理器710可以控制该输入接口730与其他设备或芯片进行通信，例如，可以获取其他设备或芯片发送的信息或数据。In some embodiments, the chip 700 may further include an input interface 730 . Wherein, the processor 710 can control the input interface 730 to communicate with other devices or chips, for example, can obtain information or data sent by other devices or chips.

在一些实施例中，该芯片700还可以包括输出接口740。其中，处理器710可以控制该输出接口740与其他设备或芯片进行通信，例如，可以向其他设备或芯片输出信息或数据。In some embodiments, the chip 700 may further include an output interface 740 . Wherein, the processor 710 can control the output interface 740 to communicate with other devices or chips, for example, can output information or data to other devices or chips.

在一些实施例中，该芯片可应用于本公开实施例中的控制网元或执行网元，并且该芯片可以实现本公开实施例的各个方法中由控制网元或执行网元实现的相应流程，为了简洁，在此不再赘述。应理解，本公开实施例提到的芯片还可以称为***级芯片，***芯片，芯片***或片上***芯片等。In some embodiments, the chip can be applied to the control network element or the execution network element in the embodiments of the present disclosure, and the chip can implement the corresponding processes implemented by the control network element or the execution network element in the various methods of the embodiments of the present disclosure , for the sake of brevity, it is not repeated here. It should be understood that the chips mentioned in the embodiments of the present disclosure may also be referred to as system-on-chip, system-on-chip, system-on-a-chip, or system-on-chip.

本公开实施例还提供了一种计算机程序产品，所述计算机程序产品承载有程序代码，所述程序代码包括的指令可配置为执行上述任一实施例中的方法。An embodiment of the present disclosure further provides a computer program product, where the computer program product carries a program code, and instructions included in the program code can be configured to execute the method in any of the foregoing embodiments.

本公开实施例还提供了一种计算机程序，包括计算机可读代码，在所述计算机可读代码在电子设备中运行的情况下，所述电子设备中的处理器执行上述任一实施例中的方法。An embodiment of the present disclosure also provides a computer program, including computer-readable codes. When the computer-readable codes run in an electronic device, a processor in the electronic device executes the program in any of the above-mentioned embodiments. method.

这里需要指出的是：以上灯具检测装置、电子设备、计算机存储介质、芯片、计算机程序产品实施例的描述，与上述方法实施例的描述是类似的，具有同方法实施例相似的有益效果。对于本公开灯具检测装置、电子设备、计算机存储介质、芯片、计算机程序产品实施例中未披露的技术细节，请参照本公开方法实施例的描述而理解。It should be pointed out here that: the above descriptions of the lamp detection device, electronic equipment, computer storage medium, chip, and computer program product embodiments are similar to the descriptions of the above method embodiments, and have similar beneficial effects as the method embodiments. For the technical details not disclosed in the embodiment of the lamp detection device, electronic equipment, computer storage medium, chip, and computer program product of the present disclosure, please refer to the description of the method embodiment of the present disclosure for understanding.

上述灯具检测装置、芯片或处理器可以包括以下任一个或多个的集成：特定用途集成电路(Application Specific Integrated Circuit，ASIC)、数字信号处理器(Digital Signal Processor，DSP)、数字信号处理装置(Digital Signal Processing Device，DSPD)、可编程逻辑装置(Programmable Logic Device，PLD)、现场可编程门阵列(Field Programmable Gate Array，FPGA)、中央处理器(Central Processing Unit，CPU)、图形处理器(Graphics Processing Unit，GPU)、嵌入式神经网络处理器(neural-network processing units，NPU)、控制器、微控制器、微处理器。可以理解地，实现上述处理器功能的电子器件还可以为其它，本公开实施例不作具体限定。The above-mentioned lamp detection device, chip or processor may include the integration of any one or more of the following: application specific integrated circuit (Application Specific Integrated Circuit, ASIC), digital signal processor (Digital Signal Processor, DSP), digital signal processing device ( Digital Signal Processing Device, DSPD), Programmable Logic Device (Programmable Logic Device, PLD), Field Programmable Gate Array (Field Programmable Gate Array, FPGA), Central Processing Unit (Central Processing Unit, CPU), Graphics Processor (Graphics Processing Unit, GPU), embedded neural network processors (neural-network processing units, NPU), controller, microcontroller, microprocessor. Understandably, the electronic device that implements the above processor function may also be other, which is not specifically limited in this embodiment of the present disclosure.

上述计算机存储介质/存储器可以是只读存储器(Read Only Memory，ROM)、可编程只读存储器(Programmable Read-Only Memory，PROM)、可擦除可编程只读存储器(Erasable Programmable Read-Only Memory，EPROM)、电可擦除可编程只读存储器(Electrically Erasable Programmable Read-Only Memory，EEPROM)、磁性随机存取存储器(Ferromagnetic Random Access Memory，FRAM)、快闪存储器(Flash Memory)、磁表面存储器、光盘、或只读光盘(Compact Disc Read-Only Memory，CD-ROM)等存储器；也可以是包括上述存储器之一或任意组合的各种终端，如移动电话、计算机、平板设备、个人数字助理等。The above-mentioned computer storage medium/memory can be read-only memory (Read Only Memory, ROM), programmable read-only memory (Programmable Read-Only Memory, PROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, EPROM), Electrically Erasable Programmable Read-Only Memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), Magnetic Random Access Memory (Ferromagnetic Random Access Memory, FRAM), Flash Memory (Flash Memory), Magnetic Surface Memory, CD-ROM, or CD-ROM (Compact Disc Read-Only Memory, CD-ROM) and other memories; it can also be various terminals including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc. .

应理解，说明书通篇中提到的“一个实施例”或“一实施例”或“本公开实施例”或“前述实施例”或“一些实施方式”或“一些实施例”意味着与实施例有关的特定特征、结构或特性包括在本公开的至少一个实施例中。因此，在整个说明书各处出现的“在一个实施例中”或“在一实施例中”或“本公开实施例”或“前述实施例”或“一些实施方式”或“一些实施例”未必一定指相同的实施例。此外，这些特定的特征、结构或特性可以任意适合的方式结合在一个或多个实施例中。应理解，在本公开的各种实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本公开实施例的实施过程构成任何限定。上述本公开实施例序号仅仅为了描述，不代表实施例的优劣。It should be understood that references throughout the specification to "one embodiment" or "an embodiment" or "an embodiment of the present disclosure" or "the foregoing embodiments" or "some implementations" or "some embodiments" mean the same as implementing A specific feature, structure, or characteristic related to an example is included in at least one embodiment of the present disclosure. Thus, appearances of "in one embodiment" or "in an embodiment" or "embodiments of the present disclosure" or "the foregoing embodiments" or "some implementations" or "some embodiments" throughout the specification do not necessarily mean Must refer to the same embodiment. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that in various embodiments of the present disclosure, the sequence numbers of the above-mentioned processes do not mean the order of execution, and the execution order of the processes should be determined by their functions and internal logic, rather than by the embodiments of the present disclosure. The implementation process constitutes any limitation. The serial numbers of the above-mentioned embodiments of the present disclosure are for description only, and do not represent the advantages and disadvantages of the embodiments.

在未做特殊说明的情况下，灯具检测装置执行本公开实施例中的任一步骤，可以是灯具检测装置的处理器执行该步骤。除非特殊说明，本公开实施例并不限定灯具检测装置执行下述步骤的先后顺序。另外，不同实施例中对数据进行处理所采用的方式可以是相同的方法或不同的方法。还需说明的是，本公开实施例中的任一步骤是灯具检测可以独立执行的，即灯具检测装置执行上述实施例中的任一步骤时，可以不依赖于其它步骤的执行。Unless otherwise specified, the lamp detection device executes any step in the embodiments of the present disclosure, and may be a processor of the lamp detection device executes the step. Unless otherwise specified, the embodiments of the present disclosure do not limit the order in which the lamp detection device executes the following steps. In addition, the methods for processing data in different embodiments may be the same method or different methods. It should also be noted that any step in the embodiments of the present disclosure can be independently executed for lamp detection, that is, when the lamp detection device executes any step in the foregoing embodiments, it may not depend on the execution of other steps.

在本公开所提供的几个实施例中，应该理解到，所揭露的设备和方法，可以通过其它的方式实现。以上所描述的设备实施例仅仅是示意性的，例如，所述模块的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，如：多个模块或组件可以结合，或可以集成到另一个***，或一些特征可以忽略，或不执行。另外，所显示或讨论的各组成部分相互之间的耦合、或直接耦合、或通信连接可以是通过一些接口，设备或模块的间接耦合或通信连接，可以是电性的、机械的或其它形式的。In the several embodiments provided in the present disclosure, it should be understood that the disclosed devices and methods may be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or May be integrated into another system, or some features may be ignored, or not implemented. In addition, the mutual coupling, or direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or modules may be in electrical, mechanical or other forms of.

上述作为分离部件说明的模块可以是、或也可以不是物理上分开的，作为模块显示的部件可以是、或也可以不是物理模块；既可以位于一个地方，也可以分布到多个网络模块上；可以根据实际的需要选择其中的部分或全部模块来实现本实施例方案的目的。The modules described above as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules; they may be located in one place or distributed to multiple network modules; Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

另外，在本公开各实施例中的各功能模块可以全部集成在一个处理模块中，也可以是各模块分别单独作为一个模块，也可以两个或两个以上模块集成在一个模块中；上述集成的模块既可以采用硬件的形式实现，也可以采用硬件加软件功能模块的形式实现。In addition, each functional module in each embodiment of the present disclosure can be integrated into one processing module, or each module can be used as a single module, or two or more modules can be integrated into one module; the above-mentioned integration The modules can be implemented in the form of hardware, or in the form of hardware plus software function modules.

本公开所提供的几个方法实施例中所揭露的方法，在不冲突的情况下可以任意组合，得到新的方法实施例。本公开所提供的几个产品实施例中所揭露的特征，在不冲突的情况下可以任意组合，得到新的产品实施例。The methods disclosed in the several method embodiments provided in the present disclosure can be combined arbitrarily to obtain new method embodiments if there is no conflict. The features disclosed in several product embodiments provided in the present disclosure can be combined arbitrarily without conflict to obtain new product embodiments.

本公开所提供的几个方法或设备实施例中所揭露的特征，在不冲突的情况下可以任意组合，得到新的方法实施例或设备实施例。The features disclosed in several method or device embodiments provided in the present disclosure may be combined arbitrarily without conflict to obtain new method embodiments or device embodiments.

本领域普通技术人员可以理解：实现上述方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成，前述的程序可以存储于计算机存储介质中，该程序在执行时，执行包括上述方法实施例的步骤；而前述的存储介质包括：移动存储设备、只读存储器(Read Only Memory，ROM)、磁碟或者光盘等各种可以存储程序代码的介质。Those of ordinary skill in the art can understand that all or part of the steps for realizing the above-mentioned method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer storage medium. and the aforementioned storage media include: various media that can store program codes such as removable storage devices, read-only memory (Read Only Memory, ROM), magnetic disks or optical disks.

或者，本公开上述集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用时，也可以存储在一个计算机存储介质中。基于这样的理解，本公开实施例的技术方案本质上或者说对相关技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机、服务器、或者网络设备等)执行本公开各个实施例所述方法的全部或部分。而前述的存储介质包括：移动存储设备、ROM、磁碟或者光盘等各种可以存储程序代码的介质。Alternatively, if the above-mentioned integrated modules of the present disclosure are realized in the form of software function modules and sold or used as independent products, they may also be stored in a computer storage medium. Based on this understanding, the essence of the technical solutions of the embodiments of the present disclosure or the part that contributes to the related technologies can be embodied in the form of software products, the computer software products are stored in a storage medium, and include several instructions to make A computer device (which may be a personal computer, a server, or a network device, etc.) executes all or part of the methods described in various embodiments of the present disclosure. The aforementioned storage medium includes various media capable of storing program codes such as removable storage devices, ROMs, magnetic disks or optical disks.

在本公开实施例中，不同实施例中相同步骤和相同内容的说明，可以互相参照。在本公开实施例中，术语“并”不对步骤的先后顺序造成影响，例如，电子设备执行A，并执行B，可以是电子设备先执行A，再执行B，或者是电子设备先执行B，再执行A，或者是电子设备执行A的同时执行B。In the embodiments of the present disclosure, descriptions of the same steps and the same content in different embodiments may refer to each other. In the embodiments of the present disclosure, the term "and" does not affect the order of the steps. For example, the electronic device executes A and then executes B. It may be that the electronic device executes A first and then B, or the electronic device executes B first. Execute A again, or execute B while the electronic device executes A.

在本公开实施例和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。As used in the examples of this disclosure and the appended claims, the singular forms "a", "said" and "the" are also intended to include the plural forms unless the context clearly dictates otherwise.

应当理解，本文中使用的术语“和/或”仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，一般表示前后关联对象是一种“或”的关系。It should be understood that the term "and/or" used herein is only an association relationship describing associated objects, which means that there may be three relationships, for example, A and/or B, which may mean that A exists alone, and A and B exist simultaneously. B, there are three situations of B alone. In addition, the character "/" in this article generally indicates that the contextual objects are an "or" relationship.

需要说明的是，本公开所涉及的各个实施例中，可以执行全部的步骤或者可以执行部分的步骤，只要能够形成一个完整的技术方案即可。It should be noted that, in each embodiment involved in the present disclosure, all steps may be performed or part of steps may be performed, as long as a complete technical solution can be formed.

在本公开中，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性。术语“多个”指两个或两个以上，除非另有明确的限定。In the present disclosure, the terms "first" and "second" are used for descriptive purposes only, and should not be understood as indicating or implying relative importance. The term "plurality" means two or more, unless otherwise clearly defined.

本领域技术人员在考虑说明书及实践这里公开的公开后，将容易想到本公开的其它实施方案。本公开旨在涵盖本公开的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本公开的一般性原理并包括本公开未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本公开的真正范围和精神由下面的权利要求指出。Other embodiments of the disclosure will be readily apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. The present disclosure is intended to cover any modification, use or adaptation of the present disclosure. These modifications, uses or adaptations follow the general principles of the present disclosure and include common knowledge or conventional technical means in the technical field not disclosed in the present disclosure. . The specification and examples are to be considered exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

应当理解的是，本公开并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本公开的范围仅由所附的权利要求来限制。It should be understood that the present disclosure is not limited to the precise constructions which have been described above and shown in the drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

工业实用性Industrial Applicability

本公开提供一种灯具检测方法、装置、设备、介质、芯片、产品及程序，其中，通过获取待检测图像，并对待检测图像进行二值化处理，从而得到二值化图像，然后基于二值化图像确定为灯具候选框，再对待检测图像中对应上述灯具候选框的候选区域进行识别，以确定每个候选区域内是否为灯具，即确定待检测图像中的灯具目标框，灯具目标框为灯具所在的位置。由于通过二值化处理筛选出了灯具候选框，则仅需要对灯具候选框对应的候选区域进行针对性识别，缩小了识别范围，提高了识别精度，增加了灯具检测的精度和鲁棒性。The disclosure provides a lamp detection method, device, equipment, medium, chip, product, and program, wherein, by acquiring an image to be detected and performing binarization processing on the image to be detected, a binarized image is obtained, and then based on the binary The image is determined as a lamp candidate frame, and then the candidate area corresponding to the above lamp candidate frame in the image to be detected is identified to determine whether each candidate area is a lamp, that is, the lamp target frame in the image to be detected is determined. The lamp target frame is The location of the light fixture. Since the candidate frame of the lamp is screened out through the binarization process, only the candidate area corresponding to the candidate frame of the lamp needs to be identified, which reduces the recognition range, improves the recognition accuracy, and increases the accuracy and robustness of the lamp detection.

Claims

一种灯具检测方法，包括：A lamp detection method, comprising:

获取待检测图像，其中，所述待检测图像对应的场景内包括灯具；Acquiring an image to be detected, wherein the scene corresponding to the image to be detected includes lamps;

将所述待检测图像进行二值化处理，得到二值化图像，并基于所述二值化图像确定灯具候选框；performing binarization processing on the image to be detected to obtain a binarized image, and determining a lamp candidate frame based on the binarized image;

对所述待检测图像中对应所述灯具候选框的候选区域进行识别，确定所述待检测图像的灯具目标框，其中，所述灯具目标框对应所述灯具所在的位置。Identifying a candidate area corresponding to the lamp candidate frame in the image to be detected, and determining a lamp target frame in the image to be detected, wherein the lamp target frame corresponds to a position of the lamp.
根据权利要求1所述的灯具检测方法，其中，所述待检测图像为待检测视频的一帧图像，所述待检测视频对应的场景内包括灯具；The lamp detection method according to claim 1, wherein the image to be detected is a frame image of a video to be detected, and the scene corresponding to the video to be detected includes a lamp;

所述方法还包括：The method also includes:

根据所述待检测图像的灯具目标框和参考图像的灯具目标框，确定所述参考图像的灯具目标框对应的灯具的明暗状态，其中，所述参考图像为所述待检测视频中，所述待检测图像之前的一帧图像。According to the lamp target frame of the image to be detected and the lamp target frame of the reference image, determine the light and dark state of the lamp corresponding to the lamp target frame of the reference image, wherein the reference image is in the video to be detected, the A frame of image before the image to be detected.
根据权利要求2所述的灯具检测方法，其中，所述根据所述待检测图像的灯具目标框和参考图像的灯具目标框，确定所述参考图像的灯具目标框对应的灯具的明暗状态，包括：The lamp detection method according to claim 2, wherein, according to the lamp target frame of the image to be detected and the lamp target frame of the reference image, determining the light and dark state of the lamp corresponding to the lamp target frame of the reference image includes :

确定所述参考图像的每个灯具目标框和所述待检测图像的每个灯具目标框的交并比；determining the intersection ratio of each lamp target frame of the reference image and each lamp target frame of the image to be detected;

在所述参考图像的灯具目标框与所述待检测图像的每个灯具目标框的交并比，均小于预设的第一比例阈值，确定所述参考图像的灯具目标框对应的灯具的明暗状态为黑暗；When the intersection ratio between the lamp target frame of the reference image and each lamp target frame of the image to be detected is less than a preset first ratio threshold, determine the lightness and darkness of the lamp corresponding to the lamp target frame of the reference image state is dark;

在所述参考图像的灯具目标框与所述待检测图像的任一灯具目标框的交并比，大于或等于所述第一比例阈值的情况下，确定所述参考图像的灯具目标框对应的灯具的明暗状态为明亮。When the intersection ratio between the lamp target frame of the reference image and any lamp target frame of the image to be detected is greater than or equal to the first ratio threshold, determine the corresponding The light and dark state of the lamp is bright.
根据权利要求2或3所述的灯具检测方法，其中，所述根据所述待检测图像的灯具目标框和参考图像的灯具目标框，确定所述参考图像的灯具目标框对应的灯具的明暗状态，包括：The lamp detection method according to claim 2 or 3, wherein, according to the lamp target frame of the image to be detected and the lamp target frame of the reference image, the light and dark state of the lamp corresponding to the lamp target frame of the reference image is determined ,include:

确定所述参考图像的所述灯具目标框中的全部像素的像素值平均值为第一平均值，并确定所述待检测图像中对应的所述灯具目标框中的全部像素的像素值平均值为第二平均值；Determining the average value of the pixel values of all the pixels in the lamp target frame of the reference image as the first average value, and determining the average pixel value of all the pixels in the corresponding lamp target frame in the image to be detected is the second average value;

在所述第一平均值与所述第二平均值的差值大于预设的第一像素阈值的情况下，确定所述灯具目标框对应的灯具的明暗状态为黑暗；When the difference between the first average value and the second average value is greater than a preset first pixel threshold, determine that the light and shade state of the lamp corresponding to the target frame of the lamp is dark;

在所述第一平均值与所述第二平均值的差值小于或等于预设的第一像素阈值的情况下，确定所述灯具目标框对应的灯具的明暗状态为明亮。In a case where the difference between the first average value and the second average value is less than or equal to a preset first pixel threshold, it is determined that the light and dark state of the lamp corresponding to the lamp target frame is bright.
根据权利要求2至4任一项所述的灯具检测方法，其中，所述将所述待检测图像进行二值化处理，包括：The lamp detection method according to any one of claims 2 to 4, wherein said binarizing the image to be detected includes:

确定所述待检测图像和所述参考图像间的每个像素对的像素差，其中，所述像素对中的一个像素点在所述待检测图像中的位置，和另一个像素点在所述参考图像中的位置相同；determining the pixel difference of each pixel pair between the image to be detected and the reference image, wherein the position of one pixel in the pixel pair in the image to be detected is the same as the position of another pixel in the image to be detected same position in the reference image;

在全部的像素对中，确定像素差大于预设的第二像素阈值的像素对的第一数量；Among all the pixel pairs, determine a first number of pixel pairs whose pixel difference is greater than a preset second pixel threshold;

在所述第一数量与全部的像素对的数量的比例大于预设的第二比例阈值的情况下，将所述待检测图像进行二值化处理。In a case where the ratio of the first number to the number of all pixel pairs is greater than a preset second ratio threshold, the image to be detected is binarized.
根据权利要求1至5任一项所述的灯具检测方法，其中，所述将所述待检测图像进行二值化处理，包括：The lamp detection method according to any one of claims 1 to 5, wherein said binarizing the image to be detected includes:

将所述待检测图像转换为灰度图像；Converting the image to be detected into a grayscale image;

将所述灰度图像进行二值化处理。Binarize the grayscale image.
根据权利要求1至6任一项所述的灯具检测方法，其中，所述基于所述二值化图像确定灯具候选框，包括：The lamp detection method according to any one of claims 1 to 6, wherein said determining a lamp candidate frame based on the binarized image comprises:

对所述二值化图像进行图像腐蚀处理，得到腐蚀图像；performing image erosion processing on the binarized image to obtain an etched image;

基于所述腐蚀图像确定所述灯具候选框。The lamp candidate frame is determined based on the corrosion image.
根据权利要求1至7任一项所述的灯具检测方法，其中，所述基于所述二值化图像确定灯具候选框，包括：The lamp detection method according to any one of claims 1 to 7, wherein said determining a lamp candidate frame based on the binarized image comprises:

基于所述二值化图像中由明亮像素组成的连通区域，确定所述灯具候选框。The lamp candidate frame is determined based on a connected region composed of bright pixels in the binarized image.
根据权利要求8所述的灯具检测方法，其中，所述基于所述二值化图像中由明亮像素组成的连通区域，确定所述灯具候选框，包括：The lamp detection method according to claim 8, wherein the determination of the lamp candidate frame based on the connected region composed of bright pixels in the binarized image comprises:

基于所述二值化图像中由明亮像素组成的面积大于或等于预设的面积阈值的连通区域，确定所述灯具候选框。The lamp candidate frame is determined based on the connected regions in the binarized image that are composed of bright pixels and whose area is greater than or equal to a preset area threshold.
根据权利要求1至9任一项所述的灯具检测方法，其中，所述对所述待检测图像中对应所述灯具候选框的候选区域进行识别，确定所述待检测图像的灯具目标框，包括：The lamp detection method according to any one of claims 1 to 9, wherein the identifying the candidate area corresponding to the lamp candidate frame in the image to be detected determines the lamp target frame of the image to be detected, include:

将每个所述候选区域输入至预先完成训练的分类神经网络，通过所述分类神经网络输出每个所述候选区域的分类结果，其中，所述分类结果包括灯具或非灯具；Each of the candidate areas is input into a pre-trained classification neural network, and a classification result of each of the candidate areas is output through the classification neural network, wherein the classification results include lamps or non-lamps;

将分类结果为灯具的候选区域确定为所述灯具目标框。The classification result is determined as the candidate area of the luminaire as the target frame of the luminaire.
根据权利要求2至5任一项所述的灯具状态方法，其中，所述获取待检测图像，包括：The lamp state method according to any one of claims 2 to 5, wherein said acquiring the image to be detected comprises:

将从所述待检测视频中获取的与所述参考图像的时间间隔为预设时长的一帧图像，确定为所述待检测图像。Determining as the image to be detected a frame of image acquired from the video to be detected and whose time interval with the reference image is a preset duration.
根据权利要求2至5、11任一项所述的灯具检测方法，其中，所述方法还包括：The lamp detection method according to any one of claims 2 to 5, 11, wherein the method further comprises:

删除所述参考图像，将所述待检测图像保存为参考图像，并保存所述待检测图像的灯具目标框。The reference image is deleted, the image to be detected is saved as a reference image, and the lamp target frame of the image to be detected is saved.
一种灯具检测装置，包括：A lamp detection device, comprising:

获取部分，配置为获取待检测图像，其中，所述待检测图像对应的场景内包括灯具；The acquisition part is configured to acquire the image to be detected, wherein the scene corresponding to the image to be detected includes lamps;

处理部分，配置为将所述待检测图像进行二值化处理，得到二值化图像，并基于所述二值化图像确定灯具候选框；The processing part is configured to perform binarization processing on the image to be detected to obtain a binarized image, and determine a lamp candidate frame based on the binarized image;

确定部分，配置为对所述待检测图像中对应所述灯具候选框的候选区域进行识别，确定所述待检测图像的灯具目标框，其中，所述灯具目标框对应所述灯具所在的位置。The determining part is configured to identify a candidate area corresponding to the lamp candidate frame in the image to be detected, and determine a lamp target frame in the image to be detected, wherein the lamp target frame corresponds to the location of the lamp.
根据权利要求13所述的灯具检测装置，其中，所述待检测图像为待检测视频的一帧图像，所述待检测视频对应的场景内包括灯具；The lamp detection device according to claim 13, wherein the image to be detected is a frame image of a video to be detected, and the scene corresponding to the video to be detected includes lamps;

所述灯具检测装置还包括判断部分；所述判断部分，配置为根据所述待检测图像的灯具目标框和参考图像的灯具目标框，确定所述参考图像的灯具目标框对应的灯具的明暗状态，其中，所述参考图像为所述待检测视频中，所述待检测图像之前的一帧图像。The lamp detection device further includes a judgment part; the judgment part is configured to determine the light and dark state of the lamp corresponding to the lamp target frame of the reference image according to the lamp target frame of the image to be detected and the lamp target frame of the reference image , wherein the reference image is a frame of image before the image to be detected in the video to be detected.
根据权利要求14所述的灯具检测装置，其中，所述判断部分，还配置为：The lamp detection device according to claim 14, wherein the judging part is further configured to:

确定所述参考图像的每个灯具目标框和所述待检测图像的每个灯具目标框的交并比；determining the intersection ratio of each lamp target frame of the reference image and each lamp target frame of the image to be detected;

在所述参考图像的灯具目标框与所述待检测图像的每个灯具目标框的交并比，均小于预设的第一比例阈值，确定所述参考图像的灯具目标框对应的灯具的明暗状态为黑暗；When the intersection ratio between the lamp target frame of the reference image and each lamp target frame of the image to be detected is less than a preset first ratio threshold, determine the lightness and darkness of the lamp corresponding to the lamp target frame of the reference image state is dark;

在所述参考图像的灯具目标框与所述待检测图像的任一灯具目标框的交并比，大于或等于所述第一比例阈值的情况下，确定所述参考图像的灯具目标框对应的灯具的明暗状态为明亮。When the intersection ratio between the lamp target frame of the reference image and any lamp target frame of the image to be detected is greater than or equal to the first ratio threshold, determine the corresponding The light and dark state of the lamp is bright.
根据权利要求14或15所述的灯具检测装置，其中，所述判断部分，还配置为：The lamp detection device according to claim 14 or 15, wherein the judging part is further configured to:

确定所述参考图像的所述灯具目标框中的全部像素的像素值平均值为第一平均值，并确定所述待检测图像中对应的所述灯具目标框中的全部像素的像素值平均值为第二平均值；Determining the average value of the pixel values of all the pixels in the lamp target frame of the reference image as the first average value, and determining the average pixel value of all the pixels in the corresponding lamp target frame in the image to be detected is the second average value;

在所述第一平均值与所述第二平均值的差值大于预设的第一像素阈值的情况下，确定所述灯具目标框对应的灯具的明暗状态为黑暗；When the difference between the first average value and the second average value is greater than a preset first pixel threshold, determine that the light and shade state of the lamp corresponding to the target frame of the lamp is dark;

在所述第一平均值与所述第二平均值的差值小于或等于预设的第一像素阈值的情况下，确定所述灯具目标框对应的灯具的明暗状态为明亮。In a case where the difference between the first average value and the second average value is less than or equal to a preset first pixel threshold, it is determined that the light and dark state of the lamp corresponding to the lamp target frame is bright.
根据权利要求14至16任一项所述的灯具检测装置，其中，所述处理部分，还配置为：The lamp detection device according to any one of claims 14 to 16, wherein the processing part is further configured to:

确定所述待检测图像和所述参考图像间的每个像素对的像素差，其中，所述像素对中的一个像素点在所述待检测图像中的位置，和另一个像素点在所述参考图像中的位置相同；determining the pixel difference of each pixel pair between the image to be detected and the reference image, wherein the position of one pixel in the pixel pair in the image to be detected is the same as the position of another pixel in the image to be detected same position in the reference image;

在全部的像素对中，确定像素差大于预设的第二像素阈值的像素对的第一数量；Among all the pixel pairs, determine a first number of pixel pairs whose pixel difference is greater than a preset second pixel threshold;

在所述第一数量与全部的像素对的数量的比例大于预设的第二比例阈值的情况下，将所述待检测图像进行二值化处理。In a case where the ratio of the first number to the number of all pixel pairs is greater than a preset second ratio threshold, the image to be detected is binarized.
根据权利要求13至17任一项所述的灯具检测装置，其中，所述处理部分，还配置为：将所述待检测图像转换为灰度图像；将所述灰度图像进行二值化处理。The lamp detection device according to any one of claims 13 to 17, wherein the processing part is further configured to: convert the image to be detected into a grayscale image; perform binarization processing on the grayscale image .
根据权利要求13至18任一项所述的灯具检测装置，其中，所述处理部分，还配置为：对所述二值化图像进行图像腐蚀处理，得到腐蚀图像；基于所述腐蚀图像确定所述灯具候选框。The lamp detection device according to any one of claims 13 to 18, wherein the processing part is further configured to: perform image corrosion processing on the binarized image to obtain a corrosion image; determine the corrosion image based on the corrosion image Candidate boxes for lamps.
根据权利要求13至19任一项所述的灯具检测装置，其中，所述处理部分，还配置为基于所述二值化图像中由明亮像素组成的连通区域，确定所述灯具候选框。The lamp detection device according to any one of claims 13 to 19, wherein the processing part is further configured to determine the lamp candidate frame based on connected regions composed of bright pixels in the binarized image.
根据权利要求20所述的灯具检测装置，其中，所述处理部分，还配置为基于所述二值化图像中由明亮像素组成的面积大于或等于预设的面积阈值的连通区域，确定所述灯具候选框。The lamp detection device according to claim 20, wherein the processing part is further configured to determine the Luminaire candidate box.
根据权利要求13至21任一项所述的灯具检测装置，其中，所述确定部分，还配置为：The lamp detection device according to any one of claims 13 to 21, wherein the determination part is further configured to:

将每个所述候选区域输入至预先完成训练的分类神经网络，通过所述分类神经网络输出每个所述候选区域的分类结果，其中，所述分类结果包括灯具或非灯具；Each of the candidate areas is input into a pre-trained classification neural network, and a classification result of each of the candidate areas is output through the classification neural network, wherein the classification results include lamps or non-lamps;

将分类结果为灯具的候选区域确定为所述灯具目标框。The classification result is determined as the candidate area of the luminaire as the target frame of the luminaire.
根据权利要求14至17任一项所述的灯具检测装置，其中，所述获取部分，还配置为将从所述待检测视频中获取的与所述参考图像的时间间隔为预设时长的一帧图像，确定为所述待检测图像。The lamp detection device according to any one of claims 14 to 17, wherein the acquisition part is further configured to set the time interval between the video to be detected and the reference image to a preset duration A frame image is determined as the image to be detected.
根据权利要求14至17、23任一项所述的灯具检测装置，其中，所述灯具检测装置还包括缓存部分；所述缓存部分，配置为删除所述参考图像，将所述待检测图像保存为参考图像，并保存所述待检测图像的灯具目标框。The lamp detection device according to any one of claims 14 to 17, 23, wherein, the lamp detection device further comprises a cache part; the cache part is configured to delete the reference image and save the image to be detected as a reference image, and save the lamp target frame of the image to be detected.
一种电子设备，所述电子设备包括存储器和处理器，所述存储器用于存储可在所述处理器上运行的计算机指令，所述处理器用于在执行所述计算机指令时实现权利要求1至12任一项所述的方法。An electronic device comprising a memory and a processor, the memory is used to store computer instructions executable on the processor, the processor is used to implement claims 1 to 1 when executing the computer instructions 12. The method described in any one.
一种计算机可读存储介质，其上存储有计算机程序，所述程序被处理器执行时实现权利要求1至12任一所述的方法。A computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.
一种芯片，包括：处理器，用于从存储器中调用并运行计算机程序，使得安装有所述芯片的设备执行如权利要求1至12任一项所述的方法。A chip, comprising: a processor, configured to invoke and run a computer program from a memory, so that a device equipped with the chip executes the method according to any one of claims 1 to 12.
一种计算机程序产品，所述计算机程序产品承载有程序代码，所述程序代码包括的指令可配置为执行如权利要求1至12任一项所述的方法。A computer program product, the computer program product carries a program code, and the program code includes instructions configured to execute the method according to any one of claims 1 to 12.
一种计算机程序，包括计算机可读代码，在所述计算机可读代码在电子设备中运行的情况下，所述电子设备中的处理器执行如权利要求1至12任一项所述的方法。A computer program, comprising computer-readable codes, when the computer-readable codes run in an electronic device, a processor in the electronic device executes the method according to any one of claims 1 to 12.