WO2021139197A1

WO2021139197A1 - Image processing method and apparatus

Info

Publication number: WO2021139197A1
Application number: PCT/CN2020/114089
Authority: WO
Inventors: 杨洁; 何东杰
Original assignee: ***股份有限公司
Priority date: 2020-01-08
Filing date: 2020-09-08
Publication date: 2021-07-15
Also published as: CN111192377A; CN111192377B

Abstract

An image processing method and apparatus. The method comprises: a mobile apparatus photographing each signal lamp when traveling on a tour inspection route, so as to obtain a tour inspection image (S301); according to a distribution feature, in a machine room, of each signal lamp, extracting one or more areas to be inspected from the tour inspection image, wherein each area to be inspected includes at least one signal lamp (S302); and inputting each area to be inspected into a neural network model to identify the state of the signal lamp in each area to be inspected. In the image processing method, a distribution feature, in a machine room, of a signal lamp is firstly used to perform coarse identification on a tour inspection image, so as to obtain each area to be inspected, and a neural network model is then used to perform fine identification on each area to be inspected, so as to determine the state of the signal lamp, such that the identification precision of the state of the signal lamp can be improved by means of two identification processes, i.e. coarse identification and fine identification, and the processing data volume of the neural network model can also be reduced, thereby improving the efficiency of image identification.

Description

一种图像处理方法及装置Image processing method and device

相关申请的交叉引用Cross-references to related applications

本申请要求在2020年01月08日提交中国专利局、申请号为202010016502.9、申请名称为“一种图像处理方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims the priority of a Chinese patent application filed with the Chinese Patent Office, the application number is 202010016502.9, and the application name is "an image processing method and device" on January 8, 2020, the entire content of which is incorporated into this application by reference .

技术领域Technical field

本发明涉及图像处理技术领域，尤其涉及一种图像处理方法及装置。The present invention relates to the technical field of image processing, in particular to an image processing method and device.

背景技术Background technique

互联网数据中心(Internet Data Center，IDC)机房是在互联网通信线路和带宽资源的基础上建立的标准化的机房环境，IDC机房可以容纳多种类型的设备，比如服务器、监控设备、管理设备、安全设备等，每个设备上可以设置有一个或多个信号灯，信号灯的状态用于标识设备或设备上的部件是否处于正常运行状态。在实际操作中，由于IDC机房中放置的设备及设备上的部件的数量和种类均较多，导致IDC机房中存在大量的信号灯，因此，在对IDC机房中的设备进行巡检时，如何快速有效地对巡检图像中的信号灯进行识别，对于监控设备及部件的运行状态、及时排查故障是非常重要的。The Internet Data Center (IDC) computer room is a standardized computer room environment established on the basis of Internet communication lines and bandwidth resources. The IDC computer room can accommodate various types of equipment, such as servers, monitoring equipment, management equipment, and security equipment. Etc., each device can be provided with one or more signal lights, and the status of the signal lights is used to identify whether the device or the components on the device are in a normal operating state. In actual operation, due to the large number and types of equipment placed in the IDC computer room and the components on the equipment, there are a large number of signal lights in the IDC computer room. Therefore, how to quickly inspect the equipment in the IDC computer room? Effectively identifying the signal lights in the inspection image is very important for monitoring the operating status of equipment and components and troubleshooting in time.

特征匹配是一种常用的图像处理方式，具体实施时，针对于IDC机房中采集到的每张巡检图像，可以基于信号灯的标志性特征(比如亮度)从每张巡检图像上提取出与信号灯的标志性特征的匹配程度较高的目标像素点，并组合目标像素点得到巡检图像中的信号灯。然而，该种方式仅能基于较少的特征来匹配得到目标像素点，从而会导致信号灯的识别精度较低。Feature matching is a commonly used image processing method. In specific implementation, for each inspection image collected in the IDC computer room, it can be extracted from each inspection image based on the iconic characteristics of the signal light (such as brightness). The target pixels with a higher degree of matching of the iconic features of the signal lights are combined to obtain the signal lights in the inspection image. However, this method can only match to obtain target pixels based on fewer features, which will result in lower recognition accuracy of the signal light.

综上，目前亟需一种图像处理方法，用以解决现有技术采用特征匹配的方式识别图像中的目标对象(即信号灯)所导致的识别精度较低的技术问题。In summary, there is an urgent need for an image processing method to solve the technical problem of low recognition accuracy caused by the prior art using feature matching to recognize target objects (ie, signal lights) in images.

发明内容Summary of the invention

本发明实施例提供一种图像处理方法及装置，用以解决现有技术采用特征匹配的方式识别图像中的目标对象(即信号灯)所导致的识别精度较低的技术问题。The embodiments of the present invention provide an image processing method and device to solve the technical problem of low recognition accuracy caused by the prior art using feature matching to recognize target objects (ie, signal lights) in an image.

第一方面，本发明实施例提供的一种图像处理方法，所述方法应用于移动装置，所述移动装置按照巡检路线对机房内的各个信号灯进行巡检；所述方法包括：In a first aspect, an image processing method provided by an embodiment of the present invention is applied to a mobile device, and the mobile device performs inspection on each signal lamp in a computer room according to an inspection route; the method includes:

所述移动装置在所述巡检路线上行进时对所述各个信号灯进行拍摄，得到巡检图像，根据所述各个信号灯在所述机房中的分布特征，从所述巡检图像中提取得到一个或多个待检测区域，每个待检测区域上包含至少一个信号灯，将每个待检测区域输入神经网络模型，以识别出每个待检测区域中的信号灯状态；所述神经网络模型使用已标记信号灯状态的图像训练得到。When the mobile device is traveling on the patrol route, the respective signal lights are photographed to obtain a patrol image, and one is extracted from the patrol image according to the distribution characteristics of the respective signal lights in the computer room. Or multiple areas to be inspected, each area to be inspected contains at least one signal light, and each area to be inspected is input into the neural network model to identify the signal light state in each area to be inspected; the neural network model uses the marked The image training of the signal lamp state is obtained.

本发明实施例中，通过先使用信号灯在机房中的分布特征对巡检图像进行粗识别以得到各个待检测区域，再使用神经网络模型对各个待检测区域进行精识别以确定信号灯状态，能够通过粗识别和精识别的两次识别过程来提高信号灯状态的识别精度，还可以降低神经网络模型的处理数据量，提高图像识别的效率。In the embodiment of the present invention, the patrol image is roughly identified by using the distribution characteristics of the signal lights in the computer room to obtain each area to be inspected, and then the neural network model is used to perform fine identification of each area to be inspected to determine the status of the signal light. The two recognition processes of rough recognition and fine recognition can improve the recognition accuracy of the signal lamp state, and can also reduce the amount of processing data of the neural network model and improve the efficiency of image recognition.

在一种可能的实现方式中，所述移动装置根据所述各个信号灯在所述机房中的分布特征，从所述巡检图像中提取得到一个或多个待检测区域之前，还确定所述巡检图像中满足信号灯的颜色特征的像素点，根据满足信号灯的颜色特征的像素点，得到至少一个信号灯区域，根据所述至少一个信号灯区域确定所述各个信号灯在所述巡检图像上的分布特征，并基于所述各个信号灯在所述巡检图像上的分布特征和所述各个信号灯在所述机房中的分布特征的偏差，对所述巡检图像进行校正。In a possible implementation manner, the mobile device also determines the patrol before extracting one or more areas to be detected from the patrol image according to the distribution characteristics of the respective signal lights in the computer room. The pixel points in the inspection image that meet the color characteristics of the signal light are obtained, at least one signal light area is obtained according to the pixel points that meet the color characteristics of the signal light, and the distribution characteristics of the respective signal lights on the inspection image are determined according to the at least one signal light area , And correcting the inspection image based on the deviation between the distribution characteristics of the respective signal lights on the inspection image and the distribution characteristics of the respective signal lights in the computer room.

在上述实现方式中，通过在识别巡检图像中的信号灯之前对巡检图像进行校正，可以弥补拍摄角度失误或拍摄环境干扰所导致的巡检图像失真的技术问题，提高后续识别的准确性，并降低后续识别所需的工作量，提高识别的效率。In the above implementation, by correcting the inspection image before identifying the signal light in the inspection image, the technical problem of the inspection image distortion caused by the shooting angle error or the interference of the shooting environment can be compensated, and the accuracy of subsequent recognition can be improved. And reduce the workload required for subsequent identification and improve the efficiency of identification.

在一种可能的实现方式中，所述各个信号灯在所述机房中呈阵列分布；所述移动装置根据所述至少一个信号灯区域确定所述各个信号灯在所述巡检图像上的分布特征，并基于所述各个信号灯在所述巡检图像上的分布特征和所述各个信号灯在所述机房中的分布特征的偏差，对所述巡检图像进行校正，包括：所述移动装置确定出每个信号灯区域中的信号灯的中心点，在所述巡检图像上设置与所述阵列对应的各个轴，针对于所述阵列对应的任一轴，确定各个信号灯区域中的信号灯的中心点在所述轴上的坐标值，并对在所述轴上的坐标值的差值小于第一预设阈值的多个中心点进行线性拟合，得到一条或多条拟合线，根据所述一条或多条拟合线与所述轴的角度关系，对所述巡检图像进行校正。In a possible implementation, the signal lights are distributed in an array in the computer room; the mobile device determines the distribution characteristics of the signal lights on the inspection image according to the at least one signal light area, and Based on the deviation of the distribution characteristics of the respective signal lights on the inspection image and the distribution characteristics of the respective signal lights in the computer room, correcting the inspection image includes: the mobile device determines each For the center point of the signal light in the signal light area, each axis corresponding to the array is set on the inspection image, and for any axis corresponding to the array, it is determined that the center point of the signal light in each signal light area is in the The coordinate values on the axis are linearly fitted to multiple center points where the difference of the coordinate values on the axis is less than the first preset threshold to obtain one or more fitting lines, according to the one or more The angle relationship between the fitting line and the axis is used to correct the inspection image.

在上述实现方式中，通过使用拟合线与阵列对应的轴的角度关系对倾斜畸变的巡检图像进行校正，能够将巡检图像从倾斜状态准确地恢复到标准状态，如此，基于标准状态的巡检图像进行神经网络模型处理，能够提高神经网络模型输出结果的准确性，提高信号灯的识别精度。In the above-mentioned implementation manner, by using the angular relationship between the fitted line and the axis corresponding to the array to correct the oblique and distorted inspection image, the inspection image can be accurately restored from the oblique state to the standard state. In this way, based on the standard state The inspection image is processed by the neural network model, which can improve the accuracy of the output result of the neural network model and the recognition accuracy of the signal light.

在一种可能的实现方式中，所述各个信号灯在所述机房中呈阵列分布；所述移动装置根据所述各个信号灯在所述机房中的分布特征，从所述巡检图像中提取得到一个或多个待检测区域，包括：所述移动装置确定所述巡检图像中满足信号灯的颜色特征的像素点，根据满足信号灯的颜色特征的像素点，得到至少一个信号灯区域，并确定出每个信号灯区域中的信号灯的中心点；在所述巡检图像上设置与所述阵列对应的任一轴，确定各个信号灯区域中的信号灯的中心点在所述轴上的坐标值，并对在所述轴上的坐标值的差值小于第一预设阈值的多个中心点进行线性拟合，得到一条或多条拟合线，在所述巡检图像中对每条拟合线扩展设定像素值，以确定出每条拟合线对应的待检测区域。In a possible implementation, the signal lights are distributed in an array in the computer room; the mobile device extracts one from the inspection image according to the distribution characteristics of the signal lights in the computer room. Or multiple areas to be detected, including: the mobile device determines the pixel points that meet the color characteristics of the signal light in the patrol image, obtains at least one signal light area according to the pixel points that meet the color characteristics of the signal light, and determines each The center point of the signal light in the signal light area; any axis corresponding to the array is set on the inspection image, the coordinate value of the center point of the signal light in each signal light area on the axis is determined, and the coordinate value of the center point of the signal light in each signal light area is determined. A plurality of center points whose coordinate values on the axis have a difference less than a first preset threshold are linearly fitted to obtain one or more fitted lines, and each fitted line is expanded and set in the inspection image Pixel value to determine the area to be detected corresponding to each fitted line.

在上述实现方式中，通过从巡检图像上提取得到每条拟合线对应的待检测区域，能够准确地将巡检图像中信号灯较为集中的区域提取出来，如此，基于较为集中的信号灯区域进行神经网络模型处理，能够在避免漏掉待识别的信号灯的同时，更有针对性的进行信号灯的识别，还可以降低过拟合的概率，提高识别的精度和效率。In the above implementation, by extracting the area to be detected corresponding to each fitting line from the inspection image, it is possible to accurately extract the area where the signal lights are concentrated in the inspection image. In this way, it is based on the more concentrated signal light area. The neural network model processing can avoid missing the signal lights to be identified, and at the same time, the signal lights can be identified more specifically, and the probability of over-fitting can also be reduced, and the accuracy and efficiency of identification can be improved.

在一种可能的实现方式中，所述各个信号灯在所述机房中呈阵列分布；所述移动装置根据所述各个信号灯在所述机房中的分布特征，从所述巡检图像中提取得到一个或多个待检测区域，包括：针对于每列信号灯，所述移动装置根据该列信号灯的阵列分布情况，从所述巡检图像中确定出初始信号灯的识别范围，并基于所述初始信号灯的识别范围和设定间隔范围从所述巡检图像中确定出其它信号灯的识别范围，根据各个信号灯的识别范围，从所述巡检图像中提取得到所述各个信号灯对应的待检测区域。In a possible implementation, the signal lights are distributed in an array in the computer room; the mobile device extracts one from the inspection image according to the distribution characteristics of the signal lights in the computer room. Or multiple areas to be detected, including: for each column of signal lights, the mobile device determines the identification range of the initial signal light from the inspection image according to the array distribution of the signal light in the column, and based on the initial signal light The recognition range and the set interval range determine the recognition range of other signal lights from the inspection image, and extract the area to be detected corresponding to each signal light from the inspection image according to the recognition range of each signal light.

在上述实现方式中，通过预先根据各个信号灯的实际分布情况预测出每个信号灯在巡检图像上的识别范围，可以在拍摄得到巡检图像后直接根据识别范围提取得到每个信号灯对应的待检测区域，该种方式无需基于巡检图像做额外处理，从而可以较好地提高识别的效率。In the above implementation, by predicting the recognition range of each signal light on the inspection image according to the actual distribution of each signal light in advance, the inspection image can be directly extracted according to the recognition range to obtain the to-be-detected corresponding to each signal light after the inspection image is captured. Area, this method does not require additional processing based on the inspection image, which can better improve the efficiency of recognition.

第二方面，本发明实施例提供的一种图像处理装置，所述装置按照巡检路线对机房内的各个信号灯进行巡检；所述装置包括：In a second aspect, an image processing device provided by an embodiment of the present invention, the device patrols each signal lamp in the machine room according to a patrol route; the device includes:

拍摄模块，用于在所述巡检路线上行进时对所述各个信号灯进行拍摄，得到巡检图像；A photographing module, configured to photograph each of the signal lights while traveling on the patrol route to obtain a patrol image;

处理模块，用于根据所述各个信号灯在所述机房中的分布特征，从所述巡检图像中提取得到一个或多个待检测区域，每个待检测区域上包含至少一个信号灯；A processing module, configured to extract one or more areas to be inspected from the inspection image according to the distribution characteristics of the signal lights in the computer room, and each area to be inspected contains at least one signal light;

识别模块，用于将每个待检测区域输入神经网络模型，以识别出每个待检测区域中的信号灯状态；所述神经网络模型使用已标记信号灯状态的图像训练得到。The recognition module is used to input each to-be-detected area into a neural network model to identify the signal light state in each to-be-detected area; the neural network model is obtained by training using the image of the marked signal light state.

在一种可能的实现方式中，所述处理模块根据所述各个信号灯在所述机房中的分布特征，从所述巡检图像中提取得到一个或多个待检测区域之前，还确定所述巡检图像中满足信号灯的颜色特征的像素点，根据满足信号灯的颜色特征的像素点，得到至少一个信号灯区域，根据所述至少一个信号灯区域确定所述各个信号灯在所述巡检图像上的分布特征，并基于所述各个信号灯在所述巡检图像上的分布特征和所述各个信号灯在所述机房中的分布特征的偏差，对所述巡检图像进行校正。In a possible implementation, the processing module also determines the patrol before extracting one or more areas to be detected from the patrol image according to the distribution characteristics of the respective signal lights in the computer room. The pixel points in the inspection image that meet the color characteristics of the signal light are obtained, at least one signal light area is obtained according to the pixel points that meet the color characteristics of the signal light, and the distribution characteristics of the respective signal lights on the inspection image are determined according to the at least one signal light area , And correcting the inspection image based on the deviation between the distribution characteristics of the respective signal lights on the inspection image and the distribution characteristics of the respective signal lights in the computer room.

在一种可能的实现方式中，所述各个信号灯在所述机房中呈阵列分布；所述处理模块具体用于：确定出每个信号灯区域中的信号灯的中心点，在所述巡检图像上设置与所述阵列对应的各个轴，针对于所述阵列对应的任一轴，确定各个信号灯区域中的信号灯的中心点在所述轴上的坐标值，并对在所述轴上的坐标值的差值小于第一预设阈值的多个中心点进行线性拟合，得到一条或多条拟合线，根据所述一条或多条拟合线与所述轴的角度关系，对所述巡检图像进行校正。In a possible implementation manner, the signal lights are distributed in an array in the computer room; the processing module is specifically configured to: determine the center point of the signal light in each signal light area, and display it on the inspection image Set each axis corresponding to the array, for any axis corresponding to the array, determine the coordinate value of the center point of the signal light in each signal light area on the axis, and compare the coordinate value on the axis Perform linear fitting on multiple center points whose difference value is less than the first preset threshold to obtain one or more fitting lines. According to the angular relationship between the one or more fitting lines and the axis, the patrol is Check the image for correction.

在一种可能的实现方式中，所述各个信号灯在所述机房中呈阵列分布；所述处理模块具体用于：确定所述巡检图像中满足信号灯的颜色特征的像素点，根据满足信号灯的颜色特征的像素点，得到至少一个信号灯区域，并确定出每个信号灯区域中的信号灯的中心点；进一步地，在所述巡检图像上设置与所述阵列对应的任一轴，确定各个信号灯区域中的信号灯的中心点在所述轴上的坐标值，并对在所述轴上的坐标值的差值小于第一预设阈值的多个中心点进行线性拟合，得到一条或多条拟合线，在所述巡检图像中对每条拟合线扩展设定像素值，以确定出每条拟合线对应的待检测区域。In a possible implementation manner, the signal lights are distributed in an array in the computer room; the processing module is specifically configured to: determine the pixel points in the inspection image that meet the color characteristics of the signal lights, and according to satisfy the signal lights The pixel points of the color feature obtain at least one signal light area, and determine the center point of the signal light in each signal light area; further, any axis corresponding to the array is set on the inspection image to determine each signal light The coordinate value of the center point of the signal light in the area on the axis, and linear fitting is performed on multiple center points whose coordinate value difference on the axis is less than the first preset threshold to obtain one or more Fitting line, expanding and setting the pixel value for each fitting line in the inspection image to determine the area to be detected corresponding to each fitting line.

在一种可能的实现方式中，所述各个信号灯在所述机房中呈阵列分布；所述处理模块具体用于：针对于每列信号灯，根据该列信号灯的阵列分布情况，从所述巡检图像中确定出初始信号灯的识别范围，并基于所述初始信号灯的识别范围和设定间隔范围从所述巡检图像中确定出其它信号灯的识别范围，根据各个信号灯的识别范围，从所述巡检图像中提取得到所述各个信号灯对应的待检测区域。In a possible implementation, the signal lights are distributed in an array in the computer room; the processing module is specifically configured to: for each column of signal lights, according to the array distribution of the column of signal lights, from the inspection The recognition range of the initial signal light is determined in the image, and the recognition range of other signal lights is determined from the inspection image based on the recognition range of the initial signal light and the set interval range. The area to be detected corresponding to each signal lamp is extracted from the inspection image.

第三方面，本发明实施例提供的一种计算设备，包括至少一个处理器以及至少一个存储器，其中，所述存储器存储有计算机程序，当所述程序被所述处理器执行时，使得所述处理器执行上述第一方面任意所述的图像处理方法。In a third aspect, a computing device provided by an embodiment of the present invention includes at least one processor and at least one memory, wherein the memory stores a computer program, and when the program is executed by the processor, the The processor executes the image processing method described in any of the foregoing first aspect.

第四方面，本发明实施例提供的一种计算机可读存储介质，其存储有可由计算设备执行的计算机程序，当所述程序在所述计算设备上运行时，使得所述计算设备执行上述第一方面任意所述的图像处理方法。In a fourth aspect, a computer-readable storage medium provided by an embodiment of the present invention stores a computer program that can be executed by a computing device. When the program runs on the computing device, the computing device executes the above-mentioned first On the one hand, any of the image processing methods described above.

本发明的这些方面或其他方面在以下实施例的描述中会更加简明易懂。These and other aspects of the present invention will be more concise and understandable in the description of the following embodiments.

附图说明Description of the drawings

为了更清楚地说明本发明实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简要介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域的普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。In order to explain the technical solutions in the embodiments of the present invention more clearly, the following will briefly introduce the drawings needed in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained from these drawings without creative labor.

图1为本发明实施例提供的一种IDC机房的***架构示意图；FIG. 1 is a schematic diagram of a system architecture of an IDC computer room provided by an embodiment of the present invention;

图2为本发明实施例提供的一种巡检路线的示意图；Figure 2 is a schematic diagram of a patrol route provided by an embodiment of the present invention;

图3为本发明实施例提供的一种图像处理方法对应的流程示意图；FIG. 3 is a schematic flowchart of an image processing method provided by an embodiment of the present invention;

图4为本发明实施例提供的一种机房中各个信号灯的阵列分布示意图；4 is a schematic diagram of the array distribution of each signal lamp in a computer room provided by an embodiment of the present invention;

图5为本发明实施例提供的一种纵向拟合线的示意图；FIG. 5 is a schematic diagram of a longitudinal fitting line provided by an embodiment of the present invention;

图6为提取方式一对应的一种待检测区域的示意图；FIG. 6 is a schematic diagram of a region to be detected corresponding to extraction method one;

图7为提取方式二对应的一种待检测区域的示意图；FIG. 7 is a schematic diagram of a to-be-detected area corresponding to the second extraction method;

图8为本发明实施例提供的一种图像处理装置的结构示意图；FIG. 8 is a schematic structural diagram of an image processing apparatus provided by an embodiment of the present invention;

图9为本发明实施例提供的一种计算设备的结构示意图。FIG. 9 is a schematic structural diagram of a computing device provided by an embodiment of the present invention.

具体实施方式Detailed ways

为了使本发明的目的、技术方案和优点更加清楚，下面将结合附图对本发明作进一步地详细描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其它实施例，都属于本发明保护的范围。In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

图1为本发明实施例提供的一种IDC机房的结构示意图，如图1所示，IDC机房中可以设置有至少一排机柜，比如机柜101～机柜106。其中，机柜101～机柜104可以并列设置，机柜105和机柜106可以并列设置，每排机柜上可以设置有多台设备，比如服务器、数据采集设备、监控设备、温控设备等。其中，机柜可以为单层结构，多台设备并列放置在单层结构上，或者，机柜也可以为多层结构，多台设备分别放置在多层结构上，每层结构上可以并列放置一台或多台设备，具体不作限定。FIG. 1 is a schematic structural diagram of an IDC computer room provided by an embodiment of the present invention. As shown in FIG. 1, at least one row of cabinets, such as cabinet 101 to cabinet 106, may be provided in the IDC computer room. Among them, the cabinets 101 to 104 can be arranged in parallel, the cabinet 105 and the cabinet 106 can be arranged in parallel, and each row of cabinets can be equipped with multiple devices, such as servers, data acquisition equipment, monitoring equipment, temperature control equipment, and so on. Among them, the cabinet can be a single-layer structure, and multiple devices can be placed side by side on a single-layer structure, or the cabinet can also be a multi-layer structure, and multiple devices can be placed on a multi-layer structure, and each layer can be placed in parallel. Or multiple devices, the specific is not limited.

本发明实施例中，每个设备上可以设置有一个或多个信号灯，为了便于设备的部署和走线，一般可以将设备的全部信号灯部署在设备的一个侧面上，如此，在对IDC机房中的设备进行巡检时，移动装置可以仅对设备上设置有信号灯的一面进行巡检。若巡检确定某一部件对应的信号灯为绿色，说明该部件处于正常运行状态；若信号灯为橘色，说明该部件处于告警状态；若信号灯为红色，说明该部件处于故障状态。In the embodiment of the present invention, each device can be provided with one or more signal lights. In order to facilitate the deployment and wiring of the device, generally all signal lights of the device can be deployed on one side of the device. In this way, in the IDC room When patrolling the equipment of the mobile device, the mobile device can only patrol the side where the signal lamp is set on the equipment. If the signal light corresponding to a certain component is green, it means that the component is in normal operation; if the signal light is orange, it means that the component is in the alarm state; if the signal light is red, it means that the component is in a fault state.

为了便于描述，将设备上部署有信号灯的一面称为正面，将设备上未部署有信号灯的一面称为背面。For ease of description, the side where the signal light is deployed on the device is called the front side, and the side where the signal light is not deployed on the device is called the back side.

相应地，为了便于设备的巡检，机房中任意两排相邻的机柜上放置的设备可以正面对正面、背面对背面。举例来说，如图1所示，机柜101上放置有设备W ₁，机柜102上放置有设备W ₂，机柜103上放置有设备W ₃，若设备W ₁与设备W ₂相对的一面为设备W ₁的背面，则设备W ₂与设备W ₁相对的一面可以为设备W ₂的背面，设备W ₂与设备W ₃相对的一面可以为设备W ₂的正面，且设备W ₃与设备W ₂相对的一面可以为设备W ₃的正面。相应地，在对机柜101～机柜103上的设备进行巡检时，移动装置可以对机柜101背离机柜102的一面(如图1所示意的T ₁面)上的设备进行巡检、对机柜102相对于机柜103的一面(如图1所示意的T ₂面)上的设备以及机柜103相对于机柜102的一面(如图1所示意的T ₃面)上的设备进行巡检。 Correspondingly, in order to facilitate the inspection of the equipment, the equipment placed on any two adjacent rows of cabinets in the computer room can be front to front and back to back. For example, as shown in Figure 1, a device W ₁ _{is placed on the cabinet 101, a device W 2} is placed on the cabinet 102, and _{a device W 3} is placed on the cabinet 103. If the opposite side of the device _{W 1} and the device W _{2 is the device} the back surface _{W. 1,} the apparatus W ₂ device _{W. 1} opposite side of the device a W rear surface, the device W ₂ with the device W ₂ ₃ opposite side may face of the device W _2, and the device W ₃ and the device W ₂ opposite side of the device can be positive W _3. Accordingly, when the device 101 to the cabinet on the cabinet 103 inspection, the mobile device 101 can be away from the side of the cabinet of the cabinet 102 (FIG. 1 schematically T ₁ of the surface) is performed on the inspection apparatus of the cabinet 102 with respect to one side of the cabinet 103 is performed on the inspection device (FIG. 1 schematically T ₂ of surface) 103 on the cabinet and the equipment with respect to one side of the cabinet 102 (FIG. 1 schematically T ₃ of the surface).

图2为采用该种实现方式得到的一种巡检路线的示意图，如图2所示，巡检过程可以为：移动装置从起始位置A点出发，分别经由路径AB-路径BE-路径EP-路径PE-路径EM-路径ME-路径EG-路径GI对机柜101～机柜106包括的设备的信号灯进行巡检。Figure 2 is a schematic diagram of a patrol route obtained by adopting this implementation method. As shown in Figure 2, the patrol process can be as follows: the mobile device starts from the starting position A and goes through the path AB-path BE-path EP -Path PE-path EM-path ME-path EG-path GI patrols the signal lights of the equipment included in the cabinet 101 to the cabinet 106.

基于图1所示意的IDC机房和图2所示意的巡检路径，图3为本发明实施例提供的一种图像处理方法对应的流程示意图，该方法包括：Based on the IDC computer room shown in FIG. 1 and the inspection path shown in FIG. 2, FIG. 3 is a schematic flowchart of an image processing method according to an embodiment of the present invention, and the method includes:

步骤301，移动装置在巡检路线上行进时对各个信号灯进行拍摄，得到巡检图像。In step 301, the mobile device photographs each signal lamp while traveling on the patrol route to obtain a patrol image.

本发明实施例中，移动装置上相对于设备正面的一侧可以设置有摄像装置，移动装置在巡检路线上运动时，摄像装置可以拍摄机柜上的设备正面，得到巡检图像；其中，拍摄得到巡检图像的方式可以有多种，比如摄像装置可以按照设定周期直接拍摄巡检图像，或者也可以先录制巡检视频，再从巡检视频中截取得到巡检图像，具体不作限定。In the embodiment of the present invention, a camera device may be provided on the side of the mobile device opposite to the front of the equipment. When the mobile device moves on the inspection route, the camera device can photograph the front of the equipment on the cabinet to obtain inspection images; There are many ways to obtain the inspection image. For example, the camera device can directly shoot the inspection image according to the set period, or it can also record the inspection video first, and then intercept the inspection image from the inspection video, and the specifics are not limited.

步骤302，移动装置根据各个信号灯在机房中的分布特征，从所述巡检图像中提取得到一个或多个待检测区域，每个待检测区域上包含至少一个信号灯。Step 302: The mobile device extracts one or more areas to be detected from the inspection image according to the distribution characteristics of each signal light in the computer room, and each area to be detected includes at least one signal light.

本发明实施例中，移动装置可以采用同步方式进行图像处理，或者也可以采用异步方式进行图像处理，若采用同步方式进行图像处理，则移动装置可以在巡检完成后，再针对于巡检过程中所拍摄的全部巡检图像进行一一识别，若采用异步方式进行图像处理，则移动装置可以在每拍摄得到一张巡检图像后，创建一个并行线程，使用并行线程对拍摄的巡检图形进行图像识别，并使用原线程继续拍摄其它的巡检图像。In the embodiment of the present invention, the mobile device can perform image processing in a synchronous manner, or can also perform image processing in an asynchronous manner. If the image processing is performed in a synchronous manner, the mobile device can focus on the inspection process after the inspection is completed. All the patrol images captured in the image are identified one by one. If the image processing is performed in an asynchronous manner, the mobile device can create a parallel thread after each patrol image is captured, and use the parallel thread to perform the image processing on the patrol image. Perform image recognition, and use the original thread to continue shooting other inspection images.

在一种可能的实现方式中，移动装置在对巡检图像进行图像识别之前，还可以对巡检图像进行校正，校正的过程可以包括如下步骤a～步骤c：In a possible implementation manner, before the mobile device performs image recognition on the patrol image, the patrol image may also be corrected, and the correction process may include the following steps a to c:

步骤a，移动装置从巡检图像中确定出满足信号灯的颜色特征的像素点。Step a: The mobile device determines the pixel points that meet the color characteristics of the signal light from the inspection image.

其中，颜色特征是指像素点的三元色(Red Green Blue，RGB)特征，可以包括色调、色饱和度和亮度中的任意一项或任意多项。Among them, the color feature refers to the three-primary color (Red Green Blue, RGB) feature of the pixel, which can include any one or more of hue, color saturation, and brightness.

在一个示例中，移动装置可以先确定巡检图像中每个像素点的颜色特征值，若某一像素点的颜色特征值满足信号灯的颜色特征值范围，则可以确定该像素点属于信号灯，若某一像素点的颜色特征值不满足信号灯的颜色特征范围，则可以确定该像素点不属于信号灯。其中，信号灯的颜色特征值范围是统计已确定的信号灯图像上每个像素点的颜色特征值得到的，信号灯的颜色特征值范围包括像素点的最低颜色特征值和像素点的最高颜色特征值。In an example, the mobile device can first determine the color feature value of each pixel in the inspection image. If the color feature value of a certain pixel meets the color feature value range of the signal light, it can determine that the pixel belongs to the signal light. If the color characteristic value of a certain pixel does not meet the color characteristic range of the signal light, it can be determined that the pixel does not belong to the signal light. The color feature value range of the signal light is obtained by counting the color feature value of each pixel on the signal light image that has been determined. The color feature value range of the signal light includes the lowest color feature value of the pixel and the highest color feature value of the pixel.

在一个示例中，移动装置在确定出巡检图像上的每个像素点的类型(属于信号灯或不属于信号灯)后，还可以根据各个像素点的类型对巡检图像进行二值化处理，从而将巡检图像转化为灰白图像；比如，移动装置可以将巡检图像中属于信号灯的像素点替换为黑色或灰色，将不属于信号灯的像素点替换为白色。如此，通过对巡检图像进行二值化处理，能够剔除巡检图像上与校正无关的信息，而仅保留与校正有关的信息，从而可以提高图像校正的效率。In an example, after determining the type of each pixel on the inspection image (belonging to a signal lamp or not belonging to a signal light), the mobile device can also perform binarization processing on the inspection image according to the type of each pixel, thereby The inspection image is converted into a gray image; for example, the mobile device can replace the pixels belonging to the signal light in the inspection image with black or gray, and replace the pixels not belonging to the signal light with white. In this way, by performing binarization processing on the patrol image, the information irrelevant to the correction can be eliminated from the patrol image, and only the information related to the correction is retained, so that the efficiency of image correction can be improved.

步骤b，移动装置根据满足颜色特征的像素点，得到一个或多个信号灯区域。Step b: The mobile device obtains one or more signal light areas according to the pixel points that meet the color characteristics.

具体实施中，移动装置在确定出巡检图像上的每个像素点的类型后，可以根据属于信号灯的各个临近的像素点，组合得到一个或多个像素区域(即信号灯区域)，每个像素区域用于标识一个或多个信号灯；或者，在对巡检图像进行二值化处理后，可以连线巡检图像上黑色或灰色的临近像素点，得到一个或多个像素区域。In specific implementation, after the mobile device determines the type of each pixel on the patrol image, it can combine to obtain one or more pixel areas (that is, the signal light area) according to the neighboring pixels belonging to the signal light, and each pixel The area is used to identify one or more signal lights; or, after the inspection image is binarized, the black or gray adjacent pixels on the inspection image can be connected to obtain one or more pixel areas.

作为一个示例，移动装置在得到一个或多个像素区域后，还可以将距离一个或多个像素区域较远的单一像素点删除，比如将单一像素点替换为白色，从而去除椒盐噪声，降低噪声对后续图像校正的影响。As an example, after obtaining one or more pixel regions, the mobile device can also delete a single pixel that is far away from one or more pixel regions, such as replacing a single pixel with white, thereby removing salt and pepper noise and reducing noise Impact on subsequent image correction.

步骤c，移动装置根据各个信号灯区域确定各个信号灯在巡检图像上的分布特征，并基于各个信号灯在巡检图像上的分布特征和各个信号灯在机房中的分布特征的偏差，对巡检图像进行校正。Step c: The mobile device determines the distribution characteristics of each signal light on the inspection image according to each signal light area, and performs the inspection image based on the distribution characteristics of each signal light on the inspection image and the deviation of the distribution characteristics of each signal light in the computer room. Correction.

举例来说，针对于任一信号灯区域，若巡检图像上的该信号灯区域包含的各个信号灯满足椭圆形分布，而机房中的各个信号灯实际满足圆形分布，则巡检装置可以对椭圆形分布与圆形分布的偏差对巡检图像进行拉伸校正；或者，若巡检图像上的该信号灯区域包含的各个信号灯满足倾斜的正多边形分布，机房中的各个信号灯实际满足正多边形分布，则巡检装置可以对正多边形分布与正多边形分布的偏差对巡检图像进行旋转校正，等等。For example, for any signal light area, if each signal light contained in the signal light area on the inspection image satisfies an elliptical distribution, and each signal light in the equipment room actually satisfies a circular distribution, the inspection device can perform an elliptical distribution. The deviation from the circular distribution is used to stretch and correct the inspection image; or, if the signal lights contained in the signal light area on the inspection image meet the inclined regular polygonal distribution, and each signal light in the computer room actually meets the regular polygonal distribution, then the inspection The inspection device can perform rotation correction on the inspection image for the deviation between the regular polygon distribution and the regular polygon distribution, and so on.

本发明实施例中，通过在识别巡检图像中的信号灯之前对巡检图像进行校正，可以弥补拍摄角度失误或拍摄环境干扰所导致的巡检图像失真的技术问题，提高后续识别的准确性，并降低后续识别所需的工作量，提高识别的效率。In the embodiment of the present invention, by correcting the inspection image before identifying the signal light in the inspection image, the technical problem of the inspection image distortion caused by the shooting angle error or the interference of the shooting environment can be compensated, and the accuracy of subsequent recognition can be improved. And reduce the workload required for subsequent identification and improve the efficiency of identification.

图4为本发明实施例提供的一种机房中各个信号灯的阵列分布示意图，如图4所示，机房中的各个信号灯可以满足M行N列的阵列分布，其中，M为4，N为4。需要说明的是，图4仅是一种示例性的信号灯分布示意图，并不构成对本方案的限定，具体实施中，M和N可以由本领域技术人员根据实际场景进行设置，不作限定。4 is a schematic diagram of the array distribution of signal lights in a computer room provided by an embodiment of the present invention. As shown in FIG. 4, each signal light in the computer room can satisfy an array distribution of M rows and N columns, where M is 4 and N is 4 . It should be noted that FIG. 4 is only an exemplary signal light distribution diagram, and does not constitute a limitation to the solution. In specific implementation, M and N can be set by those skilled in the art according to actual scenarios, and are not limited.

基于图4所示意的信号灯，在一种可能的实现方式中，移动装置可以按照如下步骤对巡检图像进行校正：Based on the signal light shown in Figure 4, in a possible implementation manner, the mobile device can correct the inspection image according to the following steps:

步骤一，移动装置确定出每个信号灯区域中的信号灯的中心点。Step 1: The mobile device determines the center point of the signal light in each signal light area.

具体实施中，针对于巡检图像上的每个像素区域，移动装置可以根据信号灯的形状特征确定出每个像素区域中的信号灯的中心点；其中，确定信号灯的中心点的方式可以有多种，比如可以将每个像素区域中全部像素点的平均坐标作为信号灯的中心点，或者也可以将每个像素区域输入机器学习模型，使用机器学习模型确定出每个像素区域的中心点，机器学习模型是使用已标注信号灯的中心点的图像训练得到的，机器学习模型能够准确识别图像中每个信号灯的中心点。In specific implementation, for each pixel area on the patrol image, the mobile device can determine the center point of the signal light in each pixel area according to the shape characteristics of the signal light; among them, there are many ways to determine the center point of the signal light. For example, the average coordinates of all pixels in each pixel area can be used as the center point of the signal light, or each pixel area can be input into a machine learning model, and the machine learning model can be used to determine the center point of each pixel area. Machine learning The model is trained using the image of the center point of the marked signal light, and the machine learning model can accurately identify the center point of each signal light in the image.

步骤二，在巡检图像上设置阵列对应的各个轴。Step two, set each axis corresponding to the array on the inspection image.

以M行N列的阵列为例，具体实施中，可以先在巡检图像上设置二维坐标系，二维坐标系的第一坐标轴可以对应巡检图像的第一边，二维坐标系的第二坐标轴可以对应巡检图像的第二边，第一边与第二边为巡检图像上相互垂直的边；由于巡检图像的第一边和第二边分别与M行N列的阵列的行和列对应，因此第一坐标轴和第二坐标轴可以分别与M行N列的阵列的行和列对应。Taking an array of M rows and N columns as an example, in specific implementation, a two-dimensional coordinate system can be set on the inspection image first, and the first axis of the two-dimensional coordinate system can correspond to the first side of the inspection image, and the two-dimensional coordinate system The second coordinate axis of can correspond to the second side of the inspection image, the first side and the second side are mutually perpendicular sides on the inspection image; because the first side and the second side of the inspection image are respectively M rows and N columns The rows and columns of the array correspond to each other, so the first coordinate axis and the second coordinate axis can correspond to the rows and columns of the array of M rows and N columns, respectively.

需要说明的是，阵列对应的各个轴基于阵列的类型进行设置，比如若阵列为三维阵列，则可以在巡检图像上设置三维坐标系，三维坐标系的三个坐标轴分别与三维阵列的长、宽和高对应，或者，若阵列为斜角阵列，则可以设置坐标系的各个轴分别与斜角阵列的行列对应，且各个轴的夹角与斜角阵列的夹角相同。It should be noted that each axis corresponding to the array is set based on the type of the array. For example, if the array is a three-dimensional array, a three-dimensional coordinate system can be set on the inspection image. The three coordinate axes of the three-dimensional coordinate system are respectively related to the length of the three-dimensional array. , Width and height correspond to each other, or if the array is an oblique array, each axis of the coordinate system can be set to correspond to the rows and columns of the oblique array, and the included angle of each axis is the same as the angle of the oblique array.

为了便于描述，下面将第一坐标轴称横轴，将第二坐标轴称为纵轴。For ease of description, the first coordinate axis is referred to as the horizontal axis, and the second coordinate axis is referred to as the vertical axis.

步骤三，针对于任一轴，根据各个信号灯区域中的信号灯的中心点在所述轴上的坐标值对各个信号灯的中心点进行拟合，得到一条或多条拟合线。Step 3: For any axis, fit the center point of each signal light according to the coordinate value of the center point of the signal light in each signal light area on the axis to obtain one or more fitting lines.

具体实施中，移动装置可以先确定各个信号灯区域中的信号灯的中心点在横轴上的坐标值和在纵轴上的坐标值，再按照坐标值对各个中心点进行聚类，将坐标值相近的多个中心点聚为一类，然后对同一类的中心点进行拟合，得到该类中心点对应的拟合线。比如，移动装置可以将在横轴上的坐标值的差值小于第一预设阈值的多个中心点聚为一类，并对多个中心点进行线性拟合，得到对应的纵向拟合线，以及可以将在纵轴上的坐标值的差值小于第二预设阈值的多个中心点聚为一类，并对多个中心点进行线性拟合，得到对应的横向拟合线。In specific implementation, the mobile device may first determine the coordinate value on the horizontal axis and the coordinate value on the vertical axis of the center point of the signal light in each signal light area, and then cluster each center point according to the coordinate value, and the coordinate value is similar The multiple center points of are gathered into one category, and then the center points of the same category are fitted to obtain the fitting line corresponding to the center points of this category. For example, the mobile device may group multiple center points whose coordinate values on the horizontal axis have a difference less than a first preset threshold into one category, and perform linear fitting on the multiple center points to obtain the corresponding longitudinal fitting line , And a plurality of center points whose coordinate value difference on the vertical axis is less than the second preset threshold can be grouped into one type, and linear fitting may be performed on the plurality of center points to obtain the corresponding horizontal fitting line.

举例来说，基于图4所示意的信号灯分布，图5为本发明实施例提供的一种纵向拟合线的示意图，如图5所示，每条纵向拟合线可以包含横坐标相近的尽可能多的信号灯区域的信号灯的中心点。For example, based on the signal light distribution shown in FIG. 4, FIG. 5 is a schematic diagram of a longitudinal fitting line provided by an embodiment of the present invention. As shown in FIG. The center point of the signal light in the possibly more signal light area.

步骤四，针对于任一轴，根据所述轴上聚类得到的各条拟合线与所述轴的角度关系，对巡检图像进行校正。Step 4: Regarding any axis, the inspection image is corrected according to the angular relationship between each fitting line obtained by clustering on the axis and the axis.

其中，校正的方式可以包括横向旋转校正、纵向旋转校正、拉伸校正中的任意一项或任意多项，也可以包括其它校正，不作限定。The correction method may include any one or more of horizontal rotation correction, vertical rotation correction, and stretch correction, and may also include other corrections, which are not limited.

在一个示例中，可以先求出各条纵向拟合线的斜率，然后将各条纵向拟合线的平均斜率或者加权平均斜率作为纵向旋转校正值，相应地，也可以求出各条横向拟合线的斜率，然后将各条横向拟合线的平均斜率或者加权平均斜率作为横向旋转校正值，进而使用纵向旋转校正值和横向旋转校正值分别对巡检图像进行校正。在旋转校正完成后，还可以对巡检图像进行拉伸校正，比如若某一条拟合线上存在多个信号灯的中心点位于拟合线的一侧，且多个信号灯的中心点满足设定的变形规律，则可以使用设定的变形规律对应的拉伸校正值对巡检图像进行拉伸校正，以将多个信号灯的中心点恢复到拟合线上。In an example, the slope of each longitudinal fitting line can be obtained first, and then the average slope or weighted average slope of each longitudinal fitting line can be used as the longitudinal rotation correction value. Correspondingly, each horizontal fitting line can also be obtained. Combine the slope of the line, and then use the average slope or weighted average slope of each horizontal fitting line as the horizontal rotation correction value, and then use the vertical rotation correction value and the horizontal rotation correction value to respectively correct the inspection image. After the rotation correction is completed, the inspection image can also be stretched and corrected. For example, if there are multiple signal lights on a fitting line whose center points are located on one side of the fitting line, and the center points of multiple signal lights meet the setting In order to restore the center points of multiple signal lights to the fitting line, the patrol image can be stretched and corrected using the stretch correction value corresponding to the set deformation law.

需要说明的是，本发明实施例不限定各类校正的顺序，比如可以先进行横线旋转校正，再进行纵向旋转校正，最后进行拉伸校正，或者也可以先进行拉伸校正，再进行横向旋转校正，最后进行纵向旋转校正，等等；可以理解的，校正的顺序也可以根据实际业务需要进行设置，不作限定。It should be noted that the embodiments of the present invention do not limit the order of various corrections. For example, horizontal line rotation correction can be performed first, then longitudinal rotation correction can be performed, and finally stretch correction can be performed, or stretching correction can be performed first, and then horizontal rotation correction can be performed. Rotation correction, and finally longitudinal rotation correction, etc.; it is understandable that the order of correction can also be set according to actual business needs, and is not limited.

本发明实施例中，通过使用拟合线的斜率对倾斜畸变的巡检图像进行校正，能够将巡检图像从倾斜状态准确地恢复到标准状态，如此，基于标准状态的巡检图像进行神经网络模型处理，能够提高神经网络模型输出结果的准确性，提高信号灯的识别精度。In the embodiment of the present invention, by using the slope of the fitted line to correct the oblique and distorted inspection image, the inspection image can be accurately restored from the inclined state to the standard state. In this way, a neural network is performed based on the inspection image in the standard state. Model processing can improve the accuracy of the output results of the neural network model and improve the recognition accuracy of the signal lights.

需要说明的是，上述步骤a和步骤b是基于巡检图像的复制图像进行图像处理，而步骤c是基于巡检图像进行图像处理，也就是说，根据步骤a和b确定校正标准后，基于步骤c对巡检图像进行校准。It should be noted that the above steps a and b are image processing based on the copied image of the patrol image, and step c is image processing based on the patrol image, that is to say, after the correction standard is determined according to steps a and b, it is based on Step c calibrate the inspection image.

本发明实施例中，根据信号灯在机房中的分布特征从巡检图像中提取得到一个或多个待检测区域的方式可以有多种，下面描述两种可能的提取方式：In the embodiment of the present invention, there may be multiple ways to extract one or more areas to be inspected from the inspection image according to the distribution characteristics of the signal lights in the computer room. Two possible extraction methods are described below:

提取方式一Extraction method one

具体实施中，在得到矫正后的巡检图像后，移动装置可以再重新按照上述步骤a和步骤b对校正后的巡检图像进行分析，得到巡检图像上各个信号灯区域的信号灯的中心点在阵列对应的某一轴上的拟合线，比如纵向拟合线，或者横向拟合线，由于此处对校正后的巡检图像进行分析，因此得到的各个拟合线与对应的轴垂直，如图6所示。相应地，以纵向拟合线为例，移动装置还可以将巡检图像上各个信号灯区域的中心点中距离纵向拟合线较远的中心点删除，比如中心点A和中心点B，从而避免脉冲噪声的干扰，提高信号灯识别的精确性。In specific implementation, after obtaining the corrected inspection image, the mobile device can analyze the corrected inspection image again according to the above steps a and b, and obtain that the center point of the signal light in each signal light area on the inspection image is The fitting line on a certain axis corresponding to the array, such as the vertical fitting line or the horizontal fitting line. Since the corrected inspection image is analyzed here, each fitting line obtained is perpendicular to the corresponding axis. As shown in Figure 6. Correspondingly, taking the longitudinal fitting line as an example, the mobile device can also delete the central points farther from the longitudinal fitting line among the central points of each signal light area on the inspection image, such as central point A and central point B, thereby avoiding The interference of impulse noise improves the accuracy of signal light recognition.

在一个示例中，针对于每条纵向拟合线，移动装置可以先在巡检图像上标记出该条纵向拟合线与巡检图像的边缘的交点(比如图6所示意的交点C ₁和交点C ₂)，然后针对于任一交点，以该交点为中心沿巡检图像的边缘方向扩展设定像素范围，得到第一像素点和第二像素点，如此，移动装置可以将该条纵向拟合线扩展得到的各个像素点连线，将连线围成的矩形条区域作为该条纵向拟合线对应的待检测区域。其中，设定像素范围可以基于信号灯的尺寸进行设置，比如可以略大于信号灯的半径。在上述示例中，通过以略大于信号灯的半径的尺寸确定待检测区域，使得待检测区域中能够包含该条纵向拟合线上的全部完整的信号灯，且不遗漏信号灯的信息，从而可以提高后续识别的准确性。 In an example, for each longitudinal fitting line, the mobile device may first mark the intersection point between the longitudinal fitting line and the edge of the inspection image on the inspection image (for example, the intersection points C ₁ and 1 shown in Figure 6). The intersection point C ₂ ), and then for any intersection point, the set pixel range is extended along the edge direction of the inspection image with the intersection point as the center, and the first pixel point and the second pixel point are obtained. The connection of each pixel point obtained by the expansion of the fitting line is taken, and the rectangular area enclosed by the connection is used as the to-be-detected area corresponding to the longitudinal fitting line. Among them, the set pixel range can be set based on the size of the signal light, for example, it can be slightly larger than the radius of the signal light. In the above example, the area to be detected is determined by a size slightly larger than the radius of the signal light, so that the area to be detected can contain all the complete signal lights on the longitudinal fitting line, and the information of the signal light is not missed, thereby improving the follow-up Accuracy of recognition.

在上述提取方式中，通过从巡检图像上提取得到每条拟合线对应的待检测区域，能够准确地将巡检图像中信号灯较为集中的区域提取出来，如此，基于较为集中的信号灯区域进行神经网络模型处理，能够在避免漏掉待识别的信号灯的同时，更有针对性的进行信号灯的识别，还可以降低过拟合的概率，提高识别的精度和效率。In the above extraction method, by extracting the area to be detected corresponding to each fitting line from the inspection image, it is possible to accurately extract the area where the signal lights are concentrated in the inspection image. In this way, the area is based on the more concentrated signal lights. The neural network model processing can avoid missing the signal lights to be identified, and at the same time, the signal lights can be identified more specifically, and the probability of over-fitting can also be reduced, and the accuracy and efficiency of identification can be improved.

提取方式二Extraction method two

具体实施中，得到矫正后的巡检图像后，针对于巡检图像上的每一列信号灯，移动装置可以根据该列信号灯在机柜中的实际分布情况，确定出该列信号灯中的初始信号灯在巡检图像中的识别范围以及任意两个信号灯的间隔范围，然后根据初始信号灯的识别范围和设定间隔范围分别从巡检图像中确定出其它信号灯的识别范围，如此，移动装置可以根据各列信号灯中每个信号灯的识别范围，从巡检图像中提取得到每个信号灯对应的待检测区域。其中，初始信号灯可以为位于最下方的信号灯，也可以为位于最上方的信号灯，还可以为位于中间的信号灯，不作限定。In specific implementation, after the corrected inspection image is obtained, for each column of signal lights on the inspection image, the mobile device can determine that the initial signal light in the column of signal lights is patrolling according to the actual distribution of the column of signal lights in the cabinet. Check the recognition range of the image and the interval range of any two signal lights, and then determine the recognition range of other signal lights from the inspection image according to the recognition range of the initial signal light and the set interval range. In this way, the mobile device can determine the recognition range of other signal lights according to each column of signal lights. The identification range of each signal light in the patrol image is extracted to obtain the area to be inspected corresponding to each signal light. The initial signal light may be the signal light located at the bottom, the signal light located at the top, or the signal light located in the middle, which is not limited.

图7为提取方式二对应的一种待检测区域的示意图，如图7所示，若各个信号灯在机房满足M行N列的阵列分布，则每一列信号灯中的任意两个相邻的信号灯的设定间隔范围可以相同。以第一列信号灯为例，若任意两个相邻的信号灯在纵坐标上的设定间隔范围为(h ₁，h ₂)，初始信号灯X ₁的识别范围为实线框所示，则初始信号灯X ₁在纵坐标上的识别范围为(d ₁，d ₂)，如此，与初始信号灯X ₁相邻的信号灯X ₂在纵坐标上的识别范围为(d ₁+h ₁，d ₂+h ₂)，信号灯X ₃在纵坐标上的识别范围为(d ₁+2h ₁，d ₂+2h ₂)，信号灯X ₄在纵坐标上的识别范围为(d ₁+3h ₁，d ₂+3h ₂)，且，信号灯X ₂、信号灯X ₃和信号灯X ₄在横坐标上的识别范围与初始信号灯X ₁在横坐标上的识别范围相同。当确定出每个信号灯在横坐标上的识别范围和纵坐标上的识别范围后，可以将以两个识别范围所框出的区域作为每个信号灯对应的待检测区域。 Figure 7 is a schematic diagram of a to-be-detected area corresponding to extraction method 2. As shown in Figure 7, if each signal lamp in the machine room meets the array distribution of M rows and N columns, the number of any two adjacent signal lamps in each column of signal lamps The setting interval range can be the same. Taking the first column of signal lights as an example, if the set interval range of any two adjacent signal lights on the ordinate is (h ₁ , h ₂ ), the recognition range of the initial signal light X ₁ is shown by the solid line frame, then the initial The recognition range of the signal light X ₁ on the ordinate is (d ₁ , d ₂ ), so the recognition range of the signal light X ₂ _{adjacent to the initial signal X 1} on the ordinate is (d ₁ +h ₁ , d ₂ + h ₂ ), the recognition range of the signal light X ₃ on the ordinate is (d ₁ +2h ₁ , d ₂ +2h ₂ ), and the recognition range of the signal light X ₄ _{on the ordinate is (d 1} +3h ₁ , d ₂ + 3h ₂ ), and _{the recognition range of the signal lamp X 2} , the signal lamp X ₃ and the signal lamp X ₄ on the abscissa is the same as the recognition range of the initial signal lamp X ₁ on the abscissa. After determining the recognition range of each signal lamp on the abscissa and the recognition range on the ordinate, the area framed by the two recognition ranges can be used as the area to be detected corresponding to each signal lamp.

在上述提取方式中，通过预先根据各个信号灯的实际分布情况预测出每个信号灯在巡检图像上的识别范围，可以在拍摄得到巡检图像后直接根据识别范围提取得到每个信号灯对应的待检测区域，该种方式无需基于巡检图像做额外处理，从而可以较好地提高识别的效率，由于机房中各个信号灯的间隔范围基本相同，因此基于间隔范围确定待检测区域的方式可以更具有规律性，操作更简便。In the above extraction method, by predicting the recognition range of each signal light on the inspection image according to the actual distribution of each signal light in advance, the inspection image can be directly extracted according to the recognition range to obtain the to-be-detected corresponding to each signal light after the inspection image is captured. Area, this method does not require additional processing based on the inspection image, which can better improve the efficiency of recognition. Since the interval range of each signal lamp in the computer room is basically the same, the method of determining the area to be detected based on the interval range can be more regular , The operation is more convenient.

本发明实施例中，通过使用信号灯的分布特征从巡检图像上确定出待检测区域，相对于直接将巡检图像输入神经网络模型确定待检测区域的方式来说，处理的数据量更少，且针对性更强，从而能够在提高图像处理效率的同时，提高图像处理的精确性。In the embodiment of the present invention, the area to be detected is determined from the inspection image by using the distribution characteristics of the signal light. Compared with the method of directly inputting the inspection image into the neural network model to determine the area to be inspected, the amount of processed data is less. And it is more targeted, which can improve the accuracy of image processing while improving the efficiency of image processing.

步骤303，所述移动装置将每个待检测区域输入神经网络模型，以识别出每个待检测区域中的信号灯状态。Step 303: The mobile device inputs each area to be detected into the neural network model to identify the status of the signal light in each area to be detected.

其中，信号灯状态可以包括信号灯位置和颜色，神经网络模型为使用已标注信号灯状态的图像训练得到的，神经网络模型能够识别出图像中的信号灯在机房中的实际位置和颜色。Wherein, the status of the signal light can include the position and color of the signal light. The neural network model is obtained by training using an image with the status of the signal light marked. The neural network model can identify the actual position and color of the signal light in the image in the computer room.

本发明实施例中，移动装置可以直接将每个待检测区域输入神经网络模型，从而确定出每个待检测区域中包含的全部信号灯的状态，然而，若待检测区域的范围较大，则待检测区域中会包含除信号灯以外的较多其它图像信息(比如机柜信息)，比如上述提取方式一所提取的待检测区域，因此，若采用直接将待检测区域输入神经网络模型的方式进行信号灯识别，则会加重神经网络模型的工作量，降低信号灯识别的效率，且由于识别的针对性较弱，还可能会使得神经网络模型过拟合，从而降低信号灯识别的效果。In the embodiment of the present invention, the mobile device can directly input each area to be detected into the neural network model, thereby determining the state of all the signal lights contained in each area to be detected. However, if the area to be detected is larger, the The detection area will contain more image information (such as cabinet information) other than the signal light, such as the area to be detected extracted by the above extraction method 1. Therefore, if the area to be detected is directly input into the neural network model for signal light recognition , It will increase the workload of the neural network model and reduce the efficiency of signal light recognition, and because the recognition is weak, it may also overfit the neural network model, thereby reducing the effect of signal light recognition.

基于此，在一种可能的实现方式中，移动装置可以采用滑窗方式确定每个待检测区域中的信号灯状态，具体地说，可以先设定一个滑动窗口，并以滑动窗口为基准从待检测区域中截取出多个识别窗口，多个识别窗口中可以具有部分相同的像素点或像素区域；进一步地，再将各个识别窗口输入神经网络模型进行识别，针对于任一识别窗口，若该识别窗口中包含信号灯，则神经网络模型可以输出该信号灯在机房中的位置和颜色(红色、黄的、绿色等)，若该识别窗口中不包含信号灯，则神经网络模型可以继续识别下一个识别窗口，直至识别完全部的识别窗口。Based on this, in a possible implementation manner, the mobile device can use a sliding window method to determine the status of the signal light in each area to be detected. Specifically, a sliding window can be set first, and the sliding window can be used as a reference from the waiting area. Multiple recognition windows are intercepted from the detection area, and the multiple recognition windows may have part of the same pixel or pixel area; further, each recognition window is input into the neural network model for recognition, for any recognition window, if the If the recognition window contains a signal light, the neural network model can output the position and color of the signal light in the computer room (red, yellow, green, etc.). If the recognition window does not contain a signal light, the neural network model can continue to recognize the next recognition Window until the entire recognition window is recognized.

举例来说，若某一个待检测区域为100像素点*10像素点的区域，滑动窗口设置为10像素点*10像素点，则可以基于滑动窗口从该待检测区域中分别截取出第1至第91识别窗口，第1识别窗口为(第1-第10像素点)*10像素点的区域，第1识别窗口为(第2-第11像素点)*10像素点的区域，第3识别窗口为(第3-第12像素点)*10像素点的区域，……，第91识别窗口为(第91-第100像素点)*10像素点的区域。For example, if a certain area to be detected is an area of 100 pixels * 10 pixels, and the sliding window is set to 10 pixels * 10 pixels, the first to The 91st recognition window, the first recognition window is the area of (1st to 10th pixel)*10 pixels, the first recognition window is the area of (2nd to 11th pixel)*10 pixels, the third recognition The window is an area of (3rd to 12th pixel)*10 pixels,..., the 91st recognition window is an area of (91st to 100th pixel)*10 pixels.

需要说明的是，上述仅是一种示例，并不构成对本方案的限定，具体实施中，也可以采用其它滑窗方式得到识别窗口，比如每隔几个像素点确定一个识别窗口。It should be noted that the above is only an example and does not constitute a limitation to this solution. In specific implementation, other sliding window methods may also be used to obtain the identification window, for example, an identification window is determined every few pixels.

在一个示例中，若神经网络模型对多个识别窗口的识别结果中包括位置间隔小于第三预设阈值的信号灯，则神经网络模型可以选取其中任意一个信号灯作为多个识别窗口的联合识别结果，从而避免识别出重复的信号灯，提高识别的准确性。In an example, if the recognition result of the neural network model for multiple recognition windows includes the signal lights whose position interval is less than the third preset threshold, the neural network model can select any one of the signal lights as the joint recognition result of the multiple recognition windows, This avoids recognizing repeated signal lights and improves the accuracy of recognition.

作为一个示例，移动装置在识别出机房中的各个信号灯的状态后，可以直接将全部信号灯的状态发送给运维人员，也可以先对各个信号灯的状态进行安全分析，若确定某一信号灯的状态满足告警规则，则可以使用该信号灯的状态生成告警信息，并将告警信息推送给运维人员，比如通过邮件、短信、即时通信等推送给运维人员，具体不作限定。As an example, after the mobile device recognizes the status of each signal light in the computer room, it can directly send the status of all the signal lights to the operation and maintenance personnel, or first perform a safety analysis on the status of each signal light. If the status of a certain signal light is determined If the alarm rules are met, the status of the signal light can be used to generate alarm information and push the alarm information to the operation and maintenance personnel, such as by email, SMS, instant messaging, etc., and the specifics are not limited.

本发明的上述实施例中，所述移动装置在所述巡检路线上行进时对所述各个信号灯进行拍摄，得到巡检图像，根据所述各个信号灯在所述机房中的分布特征，从所述巡检图像中提取得到一个或多个待检测区域，每个待检测区域上包含至少一个信号灯，将每个待检测区域输入神经网络模型，以识别出每个待检测区域中的信号灯状态；所述神经网络模型使用已标记信号灯状态的图像训练得到。本发明实施例中，通过先使用信号灯在机房中的分布特征对巡检图像进行粗识别以得到各个待检测区域，再使用神经网络模型对各个待检测区域进行精识别以确定信号灯状态，能够通过粗识别和精识别的两次识别过程来提高信号灯状态的识别精度，还可以降低神经网络模型的处理数据量，提高图像识别的效率。In the above-mentioned embodiment of the present invention, the mobile device photographs the respective signal lights while traveling on the patrol route to obtain a patrol image. According to the distribution characteristics of the respective signal lights in the computer room, One or more areas to be inspected are extracted from the inspection image, and each area to be inspected contains at least one signal light, and each area to be inspected is input into the neural network model to identify the signal light state in each area to be inspected; The neural network model is obtained by training using images of marked signal lamp states. In the embodiment of the present invention, the patrol image is roughly identified by using the distribution characteristics of the signal lights in the computer room to obtain each area to be inspected, and then the neural network model is used to perform fine identification of each area to be inspected to determine the status of the signal light. The two recognition processes of rough recognition and fine recognition can improve the recognition accuracy of the signal lamp state, and can also reduce the amount of processing data of the neural network model and improve the efficiency of image recognition.

针对上述方法流程，本发明实施例还提供一种图像处理装置，该装置的具体内容可以参照上述方法实施。In view of the foregoing method flow, an embodiment of the present invention also provides an image processing device, and the specific content of the device can be implemented with reference to the foregoing method.

图8为本发明实施例提供的一种图像处理装置的结构示意图，所述装置按照巡检路线对机房内的各个信号灯进行巡检；如图8所示，该装置包括：Fig. 8 is a schematic structural diagram of an image processing device provided by an embodiment of the present invention. The device patrols each signal lamp in the computer room according to a patrol route; as shown in Fig. 8, the device includes:

拍摄模块801，用于在所述巡检路线上行进时对所述各个信号灯进行拍摄，得到巡检图像；The photographing module 801 is configured to photograph the signal lights while traveling on the patrol route to obtain a patrol image;

处理模块802，用于根据所述各个信号灯在所述机房中的分布特征，从所述巡检图像中提取得到一个或多个待检测区域，每个待检测区域上包含至少一个信号灯；The processing module 802 is configured to extract one or more areas to be inspected from the inspection image according to the distribution characteristics of the signal lights in the computer room, and each area to be inspected includes at least one signal light;

识别模块803，用于将每个待检测区域输入神经网络模型，以识别出每个待检测区域中的信号灯状态；所述神经网络模型使用已标记信号灯状态的图像训练得到。The recognition module 803 is configured to input each area to be detected into a neural network model to identify the state of the signal light in each area to be detected; the neural network model is obtained by training using the image of the marked signal light state.

可选地，所述处理模块802根据所述各个信号灯在所述机房中的分布特征，从所述巡检图像中提取得到一个或多个待检测区域之前，还用于：Optionally, the processing module 802 is further configured to: before extracting one or more areas to be detected from the inspection image according to the distribution characteristics of the signal lights in the computer room:

确定所述巡检图像中满足信号灯的颜色特征的像素点；Determine the pixel points in the inspection image that meet the color characteristics of the signal light;

根据满足信号灯的颜色特征的像素点，得到至少一个信号灯区域；Obtain at least one signal light area according to the pixel points that meet the color characteristics of the signal light;

根据所述至少一个信号灯区域确定所述各个信号灯在所述巡检图像上的分布特征，并基于所述各个信号灯在所述巡检图像上的分布特征和所述各个信号灯在所述机房中的分布特征的偏差，对所述巡检图像进行校正。Determine the distribution characteristics of the respective signal lights on the inspection image according to the at least one signal light area, and based on the distribution characteristics of the respective signal lights on the inspection image and the distribution characteristics of the respective signal lights in the computer room The deviation of the distribution characteristics is corrected for the inspection image.

可选地，所述各个信号灯在所述机房中呈阵列分布；Optionally, the signal lights are distributed in an array in the computer room;

所述处理模块802具体用于：The processing module 802 is specifically configured to:

确定出每个信号灯区域中的信号灯的中心点；Determine the center point of the signal light in each signal light area;

在所述巡检图像上设置与所述阵列对应的各个轴；Setting each axis corresponding to the array on the inspection image;

针对于所述阵列对应的任一轴，确定各个信号灯区域中的信号灯的中心点在所述轴上的坐标值，并对在所述轴上的坐标值的差值小于第一预设阈值的多个中心点进行线性拟合，得到一条或多条拟合线，根据所述一条或多条拟合线与所述轴的角度关系，对所述巡检图像进行校正。For any axis corresponding to the array, determine the coordinate value of the center point of the signal light in each signal light area on the axis, and determine if the difference between the coordinate values on the axis is less than the first preset threshold Linear fitting is performed on a plurality of center points to obtain one or more fitting lines, and the inspection image is corrected according to the angular relationship between the one or more fitting lines and the axis.

确定所述巡检图像中满足信号灯的颜色特征的像素点，根据满足信号灯的颜色特征的像素点，得到至少一个信号灯区域，并确定出每个信号灯区域中信号灯的中心点；在所述巡检图像上设置所述阵列对应的任一轴，确定各个信号灯区域中的信号灯的中心点在所述轴上的坐标值，并对在所述轴上的坐标值的差值小于第一预设阈值的多个中心点进行线性拟合，得到一条或多条拟合线，在所述巡检图像中对每条拟合线扩展设定像素值，以确定出每条拟合线对应的待检测区域。Determine the pixels that meet the color characteristics of the signal lights in the inspection image, obtain at least one signal light area according to the pixels that meet the color characteristics of the signal light, and determine the center point of the signal light in each signal light area; in the inspection Set any axis corresponding to the array on the image, determine the coordinate value of the center point of the signal light in each signal light area on the axis, and check that the difference between the coordinate values on the axis is less than a first preset threshold Perform linear fitting on multiple center points of, to obtain one or more fitting lines, and expand and set the pixel value for each fitting line in the inspection image to determine the to-be-detected corresponding to each fitting line area.

针对于每列信号灯，根据该列信号灯的阵列分布情况，从所述巡检图像中确定出初始信号灯的识别范围，并基于所述初始信号灯的识别范围和设定间隔范围从所述巡检图像中确定出其它信号灯的识别范围；For each column of signal lights, according to the array distribution of the column of signal lights, the identification range of the initial signal light is determined from the inspection image, and based on the identification range of the initial signal light and the set interval range from the inspection image Determine the recognition range of other signal lights;

根据各个信号灯的识别范围，从所述巡检图像中提取得到所述各个信号灯对应的待检测区域。According to the identification range of each signal lamp, the area to be detected corresponding to each signal lamp is extracted from the inspection image.

从上述内容可以看出：本发明的上述实施例中，所述移动装置在所述巡检路线上行进时对所述各个信号灯进行拍摄，得到巡检图像，根据所述各个信号灯在所述机房中的分布特征，从所述巡检图像中提取得到一个或多个待检测区域，每个待检测区域上包含至少一个信号灯，将每个待检测区域输入神经网络模型，以识别出每个待检测区域中的信号灯状态；所述神经网络模型使用已标记信号灯状态的图像训练得到。本发明实施例中，通过先使用信号灯在机房中的分布特征对巡检图像进行粗识别以得到各个待检测区域，再使用神经网络模型对各个待检测区域进行精识别以确定信号灯状态，能够通过粗识别和精识别的两次识别过程来提高信号灯状态的识别精度，还可以降低神经网络模型的处理数据量，提高图像识别的效率。It can be seen from the above content that in the above-mentioned embodiment of the present invention, the mobile device photographs the respective signal lights while traveling on the patrol route to obtain a patrol image, and according to the respective signal lights in the computer room One or more areas to be inspected are extracted from the inspection image, each area to be inspected contains at least one signal light, and each area to be inspected is input into the neural network model to identify each area to be inspected. The state of the signal light in the detection area; the neural network model is obtained by training using the image of the marked signal light state. In the embodiment of the present invention, the patrol image is roughly identified by using the distribution characteristics of the signal lights in the computer room to obtain each area to be inspected, and then the neural network model is used to finely identify each area to be inspected to determine the status of the signal light. The two recognition processes of rough recognition and fine recognition can improve the recognition accuracy of the signal status, and can also reduce the amount of processing data of the neural network model and improve the efficiency of image recognition.

基于同一发明构思，本发明实施例还提供了一种计算设备，如图9所示，包括至少一个处理器901，以及与至少一个处理器连接的存储器902，本发明实施例中不限定处理器901与存储器902之间的具体连接介质，图9中处理器901和存储器902之间通过总线连接为例。总线可以分为地址总线、数据总线、控制总线等。Based on the same inventive concept, an embodiment of the present invention also provides a computing device. As shown in FIG. 9, it includes at least one processor 901 and a memory 902 connected to the at least one processor. The embodiment of the present invention does not limit the processor. The specific connection medium between the 901 and the memory 902 is the connection between the processor 901 and the memory 902 through a bus in FIG. 9 as an example. The bus can be divided into address bus, data bus, control bus and so on.

在本发明实施例中，存储器902存储有可被至少一个处理器901执行的指令，至少一个处理器901通过执行存储器902存储的指令，可以执行前述的图像处理方法中所包括的步骤。In the embodiment of the present invention, the memory 902 stores instructions that can be executed by at least one processor 901, and the at least one processor 901 can execute the steps included in the aforementioned image processing method by executing the instructions stored in the memory 902.

其中，处理器901是计算设备的控制中心，可以利用各种接口和线路连接计算设备的各个部分，通过运行或执行存储在存储器902内的指令以及调用存储在存储器902内的数据，从而实现数据处理。可选的，处理器901可包括一个或多个处理单元，处理器901可集成应用处理器和调制解调处理器，其中，应用处理器主要处理操作***、用户界面和应用程序等，调制解调处理器主要处理下发指令。可以理解的是，上述调制解调处理器也可以不集成到处理器901中。在一些实施例中，处理器901和存储器902可以在同一芯片上实现，在一些实施例中，它们也可以在独立的芯片上分别实现。Among them, the processor 901 is the control center of the computing device, which can use various interfaces and lines to connect various parts of the computing device, and realize data by running or executing instructions stored in the memory 902 and calling data stored in the memory 902. deal with. Optionally, the processor 901 may include one or more processing units, and the processor 901 may integrate an application processor and a modem processor. The application processor mainly processes the operating system, user interface, and application programs. The adjustment processor mainly handles issuing instructions. It can be understood that the foregoing modem processor may not be integrated into the processor 901. In some embodiments, the processor 901 and the memory 902 may be implemented on the same chip, and in some embodiments, they may also be implemented on separate chips.

处理器901可以是通用处理器，例如中央处理器(CPU)、数字信号处理器、专用集成电路(Application Specific Integrated Circuit，ASIC)、现场可编程门阵列或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件，可以实现或者执行本发明实施例中公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者任何常规的处理器等。结合图像处理实施例所公开的方法的步骤可以直接体现为硬件处理器执行完成，或者用处理器中的硬件及软件模块组合执行完成。The processor 901 may be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application specific integrated circuit (ASIC), a field programmable gate array or other programmable logic devices, discrete gates or transistors Logic devices and discrete hardware components can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor or the like. The steps of the method disclosed in combination with the image processing embodiment may be directly embodied as executed and completed by a hardware processor, or executed and completed by a combination of hardware and software modules in the processor.

存储器902作为一种非易失性计算机可读存储介质，可用于存储非易失性软件程序、非易失性计算机可执行程序以及模块。存储器902可以包括至少一种类型的存储介质，例如可以包括闪存、硬盘、多媒体卡、卡型存储器、随机访问存储器(Random Access Memory，RAM)、静态随机访问存储器(Static Random Access Memory，SRAM)、可编程只读存储器(Programmable Read Only Memory，PROM)、只读存储器(Read Only Memory，ROM)、带电可擦除可编程只读存储器(Electrically Erasable Programmable Read-Only Memory，EEPROM)、磁性存储器、磁盘、光盘等等。存储器902是能够用于携带或存储具有指令或数据结构形式的期望的程序代码并能够由计算机存取的任何其他介质，但不限于此。本发明实施例中的存储器902还可以是电路或者其它任意能够实现存储功能的装置，用于存储程序指令和/或数据。The memory 902, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The memory 902 may include at least one type of storage medium, for example, may include flash memory, hard disk, multimedia card, card-type memory, random access memory (Random Access Memory, RAM), static random access memory (Static Random Access Memory, SRAM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), magnetic memory, disk , CD, etc. The memory 902 is any other medium that can be used to carry or store desired program codes in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto. The memory 902 in the embodiment of the present invention may also be a circuit or any other device capable of realizing a storage function for storing program instructions and/or data.

基于同一发明构思，本发明实施例还提供了一种计算机可读存储介质，其存储有可由计算设备执行的计算机程序，当所述程序在所述计算设备上运行时，使得所述计算设备执行上述图3任意所述的图像处理方法。Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium that stores a computer program executable by a computing device, and when the program runs on the computing device, the computing device executes The image processing method arbitrarily described in FIG. 3 above.

本领域内的技术人员应明白，本发明的实施例可提供为方法、或计算机程序产品。因此，本发明可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本发明可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、 CD-ROM、光学存储器等)上实施的计算机程序产品的形式。Those skilled in the art should understand that the embodiments of the present invention can be provided as a method or a computer program product. Therefore, the present invention may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

本发明是参照根据本发明实施例的方法、设备(***)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。The present invention is described with reference to flowcharts and/or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and/or block in the flowchart and/or block diagram, and the combination of processes and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing equipment to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing equipment are generated It is a device that realizes the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device. The device implements the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to produce computer-implemented processing, so as to execute on the computer or other programmable equipment. The instructions provide steps for implementing the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

尽管已描述了本发明的优选实施例，但本领域内的技术人员一旦得知了基本创造性概念，则可对这些实施例作出另外的变更和修改。所以，所附权利要求意欲解释为包括优选实施例以及落入本发明范围的所有变更和修改。Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications to these embodiments once they learn the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present invention.

显然，本领域的技术人员可以对本发明进行各种改动和变型而不脱离本发明的精神和范围。这样，倘若本发明的这些修改和变型属于本发明权利要求及其等同技术的范围之内，则本发明也意图包含这些改动和变型在内。Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. In this way, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims

一种图像处理方法，其特征在于，所述方法应用于移动装置，所述移动装置按照巡检路线对机房内的各个信号灯进行巡检；所述方法包括：An image processing method, characterized in that the method is applied to a mobile device, and the mobile device patrols each signal lamp in a computer room according to a patrol route; the method includes:

所述移动装置在所述巡检路线上行进时对所述各个信号灯进行拍摄，得到巡检图像；When the mobile device is traveling on the patrol route, photographing the respective signal lights to obtain a patrol image;

所述移动装置根据所述各个信号灯在所述机房中的分布特征，从所述巡检图像中提取得到一个或多个待检测区域，每个待检测区域上包含至少一个信号灯；The mobile device extracts one or more areas to be inspected from the inspection image according to the distribution characteristics of the signal lights in the computer room, and each area to be inspected includes at least one signal light;

所述移动装置将每个待检测区域输入神经网络模型，以识别出每个待检测区域中的信号灯状态；所述神经网络模型使用已标记信号灯状态的图像训练得到。The mobile device inputs each to-be-detected area into a neural network model to identify the signal light state in each to-be-detected area; the neural network model is obtained by training using the image of the marked signal light state.
根据权利要求1所述的方法，其特征在于，所述移动装置根据所述各个信号灯在所述机房中的分布特征，从所述巡检图像中提取得到一个或多个待检测区域之前，还包括：The method according to claim 1, characterized in that, before the mobile device extracts one or more areas to be detected from the inspection image according to the distribution characteristics of the signal lights in the computer room, further include:

所述移动装置从所述巡检图像中确定出满足信号灯的颜色特征的像素点；The mobile device determines, from the inspection image, the pixel that meets the color characteristics of the signal light;

所述移动装置根据满足信号灯的颜色特征的像素点，得到至少一个信号灯区域；The mobile device obtains at least one signal light area according to the pixel points satisfying the color characteristics of the signal light;

所述移动装置根据所述至少一个信号灯区域确定所述各个信号灯在所述巡检图像上的分布特征，并基于所述各个信号灯在所述巡检图像上的分布特征和所述各个信号灯在所述机房中的分布特征的偏差，对所述巡检图像进行校正。The mobile device determines the distribution characteristics of each signal light on the inspection image according to the at least one signal light area, and based on the distribution characteristics of each signal light on the inspection image and the location of each signal light The deviation of the distribution feature in the computer room is corrected, and the inspection image is corrected.
根据权利要求2所述的方法，其特征在于，所述各个信号灯在所述机房中呈阵列分布；The method according to claim 2, wherein the signal lights are distributed in an array in the computer room;

所述移动装置根据所述至少一个信号灯区域确定所述各个信号灯在所述巡检图像上的分布特征，并基于所述各个信号灯在所述巡检图像上的分布特征和所述各个信号灯在所述机房中的分布特征的偏差，对所述巡检图像进行校正，包括：The mobile device determines the distribution feature of each signal light on the inspection image according to the at least one signal light area, and based on the distribution feature of each signal light on the inspection image and the location of each signal light The deviation of the distribution feature in the computer room and the correction of the inspection image include:

所述移动装置确定出每个信号灯区域中的信号灯的中心点；The mobile device determines the center point of the signal light in each signal light area;

所述移动装置在所述巡检图像上设置与所述阵列对应的各个轴；The mobile device sets each axis corresponding to the array on the patrol image;

针对于所述阵列对应的任一轴，所述移动装置确定各个信号灯区域中的信号灯的中心点在所述轴上的坐标值，并对在所述轴上的坐标值的差值小于第一预设阈值的多个中心点进行线性拟合，得到一条或多条拟合线；根据所述一条或多条拟合线与所述轴的角度关系，对所述巡检图像进行校正。For any axis corresponding to the array, the mobile device determines the coordinate value of the center point of the signal light in each signal light area on the axis, and the difference of the coordinate value on the axis is smaller than the first Performing linear fitting on a plurality of center points of a preset threshold to obtain one or more fitting lines; and correcting the inspection image according to the angular relationship between the one or more fitting lines and the axis.
根据权利要求1至3中任一项所述的方法，其特征在于，所述各个信号灯在所述机房中呈阵列分布；The method according to any one of claims 1 to 3, wherein the signal lights are distributed in an array in the computer room;

所述移动装置根据所述各个信号灯在所述机房中的分布特征，从所述巡检图像中提取得到一个或多个待检测区域，包括：The mobile device extracts one or more areas to be detected from the inspection image according to the distribution characteristics of the respective signal lights in the computer room, including:

所述移动装置从所述巡检图像中确定出满足信号灯的颜色特征的像素点，根据满足信号灯的颜色特征的像素点，得到至少一个信号灯区域，并确定每个信号灯区域中的信号灯的中心点；The mobile device determines the pixel points that meet the color characteristics of the signal light from the inspection image, obtains at least one signal light area according to the pixel points that meet the color characteristics of the signal light, and determines the center point of the signal light in each signal light area ；

所述移动装置在所述巡检图像上设置与所述阵列对应的任一轴，确定各个信号灯区域中的信号灯的中心点在所述轴上的坐标值，并对在所述轴上的坐标值的差值小于第一预设阈值的多个中心点进行线性拟合，得到一条或多条拟合线；The mobile device sets any axis corresponding to the array on the inspection image, determines the coordinate value of the center point of the signal light in each signal light area on the axis, and compares the coordinates on the axis Performing linear fitting on multiple center points whose value difference is less than the first preset threshold to obtain one or more fitting lines;

所述移动装置在所述巡检图像中对每条拟合线扩展设定像素值，以确定出每条拟合线对应的待检测区域。The mobile device expands and sets the pixel value for each fitted line in the inspection image to determine the area to be detected corresponding to each fitted line.
根据权利要求1至3中任一项所述的方法，其特征在于，所述各个信号灯在所述机房中呈阵列分布；The method according to any one of claims 1 to 3, wherein the signal lights are distributed in an array in the computer room;

所述移动装置根据所述各个信号灯在所述机房中的分布特征，从所述巡检图像中提取得到一个或多个待检测区域，包括：The mobile device extracts one or more areas to be detected from the inspection image according to the distribution characteristics of the respective signal lights in the computer room, including:

针对于每列信号灯，所述移动装置根据该列信号灯的阵列分布情况，从所述巡检图像中确定出初始信号灯的识别范围，并基于所述初始信号灯的识别范围和设定间隔范围从所述巡检图像中确定出其它信号灯的识别范围；For each column of signal lights, the mobile device determines the identification range of the initial signal light from the inspection image according to the array distribution of the signal light in the column, and determines the identification range of the initial signal light based on the identification range and the set interval range of the initial signal light. The identification range of other signal lights is determined in the inspection image;

所述移动装置根据各个信号灯的识别范围，从所述巡检图像中提取得到所述各个信号灯对应的待检测区域。The mobile device extracts the area to be detected corresponding to each signal lamp from the inspection image according to the recognition range of each signal lamp.
一种图像处理装置，其特征在于，所述装置按照巡检路线对机房内的各个信号灯进行巡检；所述装置包括：An image processing device, characterized in that the device patrols each signal lamp in the machine room according to a patrol route; the device includes:

拍摄模块，用于在所述巡检路线上行进时对所述各个信号灯进行拍摄，得到巡检图像；A photographing module, configured to photograph each of the signal lights while traveling on the patrol route to obtain a patrol image;

处理模块，用于根据所述各个信号灯在所述机房中的分布特征，从所述巡检图像中提取得到一个或多个待检测区域，每个待检测区域上包含至少一个信号灯；A processing module, configured to extract one or more areas to be inspected from the inspection image according to the distribution characteristics of the signal lights in the computer room, and each area to be inspected contains at least one signal light;

识别模块，用于将每个待检测区域输入神经网络模型，以识别出每个待检测区域中的信号灯状态；所述神经网络模型使用已标记信号灯状态的图像训练得到。The recognition module is used to input each to-be-detected area into a neural network model to identify the signal light state in each to-be-detected area; the neural network model is obtained by training using the image of the marked signal light state.
根据权利要求6所述的装置，其特征在于，所述处理模块根据所述各个信号灯在所述机房中的分布特征，从所述巡检图像中提取得到一个或多个待检测区域之前，还用于：The device according to claim 6, wherein the processing module further extracts one or more areas to be detected from the inspection image according to the distribution characteristics of the signal lights in the computer room. Used for:

确定出所述巡检图像中满足信号灯的颜色特征的像素点；Determine the pixel points in the inspection image that meet the color characteristics of the signal light;

根据满足信号灯的颜色特征的像素点，得到至少一个信号灯区域；Obtain at least one signal light area according to the pixel points that meet the color characteristics of the signal light;

根据所述至少一个信号灯区域确定所述各个信号灯在所述巡检图像上的分布特征，并基于所述各个信号灯在所述巡检图像上的分布特征和所述各个信号灯在所述机房中的分布特征的偏差，对所述巡检图像进行校正。Determine the distribution characteristics of the respective signal lights on the inspection image according to the at least one signal light area, and based on the distribution characteristics of the respective signal lights on the inspection image and the distribution characteristics of the respective signal lights in the computer room The deviation of the distribution characteristics is corrected for the inspection image.
根据权利要求6所述的装置，其特征在于，所述各个信号灯在所述机房中呈阵列分布；The device according to claim 6, wherein the signal lights are distributed in an array in the computer room;

所述处理模块具体用于：The processing module is specifically used for:

确定出每个信号灯区域中的信号灯的中心点；Determine the center point of the signal light in each signal light area;

在所述巡检图像上设置与所述阵列对应的各个轴；Setting each axis corresponding to the array on the inspection image;

针对于所述阵列对应的任一轴，确定各个信号灯区域中的信号灯的中心点在所述轴上的坐标值，并对在所述轴上的坐标值的差值小于第一预设阈值的多个中心点进行线性拟合，得到一条或多条拟合线；根据所述一条或多条拟合线与所述轴的角度关系，对所述巡检图像进行校正。For any axis corresponding to the array, determine the coordinate value of the center point of the signal light in each signal light area on the axis, and determine if the difference between the coordinate values on the axis is less than the first preset threshold Linear fitting is performed on a plurality of center points to obtain one or more fitting lines; and the inspection image is corrected according to the angular relationship between the one or more fitting lines and the axis.
根据权利要求6至8中任一项所述的装置，其特征在于，所述各个信号灯在所述机房中呈阵列分布；The device according to any one of claims 6 to 8, wherein the signal lights are distributed in an array in the computer room;

所述处理模块具体用于：The processing module is specifically used for:

确定所述巡检图像中满足信号灯的颜色特征的像素点，根据满足信号灯的颜色特征的像素点，得到至少一个信号灯区域，并确定出每个信号灯区域中的信号灯的中心点；Determine the pixels in the inspection image that meet the color characteristics of the signal lights, obtain at least one signal light area according to the pixels that meet the color characteristics of the signal light, and determine the center point of the signal light in each signal light area;

在所述巡检图像上设置与所述阵列对应的任一轴，确定各个信号灯区域中的信号灯的中心点在所述轴上的坐标值，并对在所述轴上的坐标值的差值小于第一预设阈值的多个中心点进行线性拟合，得到一条或多条拟合线；Set any axis corresponding to the array on the inspection image, determine the coordinate value of the center point of the signal light in each signal light area on the axis, and calculate the difference between the coordinate values on the axis Perform linear fitting on multiple center points less than the first preset threshold to obtain one or more fitting lines;

在所述巡检图像中对每条拟合线扩展设定像素值，以确定出每条拟合线对应的待检测区域。In the inspection image, each fitted line is expanded to set the pixel value to determine the area to be detected corresponding to each fitted line.
根据权利要求6至8中任一项所述的装置，其特征在于，所述各个信号灯在所述机房中呈阵列分布；The device according to any one of claims 6 to 8, wherein the signal lights are distributed in an array in the computer room;

所述处理模块具体用于：The processing module is specifically used for:

针对于每列信号灯，根据该列信号灯的阵列分布情况，从所述巡检图像中确定出初始信号灯的识别范围，并基于所述初始信号灯的识别范围和设定间隔范围从所述巡检图像中确定出其它信号灯的识别范围；For each column of signal lights, according to the array distribution of the column of signal lights, the identification range of the initial signal light is determined from the inspection image, and based on the identification range of the initial signal light and the set interval range from the inspection image Determine the recognition range of other signal lights;

根据各个信号灯的识别范围，从所述巡检图像中提取得到所述各个信号灯对应的待检测区域。According to the identification range of each signal lamp, the area to be detected corresponding to each signal lamp is extracted from the inspection image.
一种计算设备，其特征在于，包括至少一个处理器以及至少一个存储器，其中，所述存储器存储有计算机程序，当所述程序被所述处理器执行时，使得所述处理器执行权利要求1～5任一权利要求所述的方法。A computing device, characterized by comprising at least one processor and at least one memory, wherein the memory stores a computer program, and when the program is executed by the processor, the processor executes claim 1 ~5 The method of any one of claims.
一种计算机可读存储介质，其特征在于，其存储有可由计算设备执行的计算机程序，当所述程序在所述计算设备上运行时，使得所述计算设备执行权利要求1～5任一权利要求所述的方法。A computer-readable storage medium, characterized in that it stores a computer program executable by a computing device, and when the program runs on the computing device, the computing device executes any one of claims 1 to 5 Require the described method.