WO2023125671A1

WO2023125671A1 - Image processing method, electronic device, storage medium and program product

Info

Publication number: WO2023125671A1
Application number: PCT/CN2022/142861
Authority: WO
Inventors: 葛成伟
Original assignee: 中兴通讯股份有限公司
Priority date: 2021-12-31
Filing date: 2022-12-28
Publication date: 2023-07-06
Also published as: CN116433561A

Abstract

The present application provides an image processing method, an electronic device, a computer readable storage medium and a computer program product. The image processing method comprises: acquiring an image to be processed (S100); acquiring feature vectors of pixels in the image to be processed (S200); acquiring a pre-trained uniform pixel feature vector set (S300); obtaining vector uniformity of the pixels according to the uniform pixel feature vector set and the feature vectors of the pixels (S400); and obtaining, according to the vector uniformity of the pixels, a uniformity heatmap of the image to be processed (S500).

Description

图像处理方法、电子设备、存储介质及程序产品Image processing method, electronic device, storage medium and program product

相关申请的交叉引用Cross References to Related Applications

本申请基于申请号为202111670966.2、申请日为2021年12月31日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本申请作为参考。This application is based on a Chinese patent application with application number 202111670966.2 and a filing date of December 31, 2021, and claims the priority of this Chinese patent application. The entire content of this Chinese patent application is hereby incorporated by reference into this application.

技术领域technical field

本申请涉及人工智能领域，尤其涉及一种图像处理方法、电子设备、计算机可读存储介质及计算机程序产品。The present application relates to the field of artificial intelligence, and in particular to an image processing method, electronic equipment, computer readable storage medium and computer program product.

背景技术Background technique

计算机视觉是人工智能领域的一个重要分支，是指计算机能够代替人眼和大脑对特定目标进行视觉认识，例如在工业上通过计算机对产品的颜色均匀度、亮度均匀度或纹理均匀度的识别，来进行产品优劣或瑕疵的检测。Computer vision is an important branch of the field of artificial intelligence, which means that computers can replace human eyes and brains for visual recognition of specific targets, such as the identification of color uniformity, brightness uniformity or texture uniformity of products through computers in industry, To detect product pros and cons or defects.

然而，在目前的相关技术中，计算机在进行视觉识别和判断时，需要依赖人为预先设计好的视觉特征进行比对，因此导致在不同的应用场景的普适性不强，同时计算机在识别对比后，只能输出单一的判断结果，例如产品是否有瑕疵，而无法对产品瑕疵的具***置进行显示，导致用户难以对产品瑕疵进行优化，用户使用感受不好。However, in the current related technologies, when the computer performs visual recognition and judgment, it needs to rely on artificially pre-designed visual features for comparison, which leads to poor universality in different application scenarios. Finally, only a single judgment result can be output, such as whether the product is defective, but the specific location of the product defect cannot be displayed, which makes it difficult for users to optimize the product defect, and the user experience is not good.

发明内容Contents of the invention

以下是对本文详细描述的主题的概述。本概述并非是为了限制权利要求的保护范围。The following is an overview of the topics described in detail in this article. This summary is not intended to limit the scope of the claims.

本申请实施例提供了一种图像处理法、电子设备以及计算机可读存储介质及计算机程序产品。Embodiments of the present application provide an image processing method, electronic equipment, a computer-readable storage medium, and a computer program product.

第一方面，本申请实施例提供了一种图像处理方法，包括：获取待处理图像；获取所述待处理图像中像素的特征向量；获取预先训练得到的均匀像素特征向量集合；根据所述均匀像素特征向量集合与所述像素的特征向量，得到所述像素的向量均匀度；根据所述像素的向量均匀度，得到所述待处理图像的均匀度热力图。In the first aspect, the embodiment of the present application provides an image processing method, including: acquiring an image to be processed; acquiring feature vectors of pixels in the image to be processed; acquiring a set of uniform pixel feature vectors obtained by pre-training; The pixel feature vector set and the feature vector of the pixel are used to obtain the vector uniformity of the pixel; according to the vector uniformity of the pixel, a heat map of the uniformity of the image to be processed is obtained.

第二方面，本申请实施例还提供了一种电子设备，包括：存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现如上所述的图像处理方法。In the second aspect, the embodiment of the present application also provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and operable on the processor, and the processor implements the above when executing the computer program. The image processing method described.

第三方面，本申请实施例还提供了一种计算机可读存储介质，存储有计算机可执行指令，所述计算机可执行指令用于执行如上所述的图像处理方法。In a third aspect, the embodiment of the present application further provides a computer-readable storage medium storing computer-executable instructions, and the computer-executable instructions are used to execute the image processing method as described above.

第四方面，本申请实施例还提供了一种计算机程序产品，包括计算机程序或计算机指令，所述计算机程序或所述计算机指令存储在计算机可读存储介质中，计算机设备的处理器从所述计算机可读存储介质读取所述计算机程序或所述计算机指令，所述处理器执行所述计算机程序或所述计算机指令，使得所述计算机设备执行如上所述的图像处理方法。In a fourth aspect, the embodiment of the present application further provides a computer program product, including a computer program or a computer instruction, the computer program or the computer instruction is stored in a computer-readable storage medium, and the processor of the computer device reads from the The computer-readable storage medium reads the computer program or the computer instruction, and the processor executes the computer program or the computer instruction, so that the computer device executes the above-mentioned image processing method.

本申请的其它特征和优点将在随后的说明书中阐述，并且，部分地从说明书中变得显而易见，或者通过实施本申请而了解。本申请的目的和其他优点可通过在说明书、权利要求书以及附图中所特别指出的结构来实现和获得。Other features and advantages of the application will be set forth in the description which follows, and, in part, will be obvious from the description, or can be learned by practice of the application. The objectives and other advantages of the application will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

附图说明Description of drawings

附图用来提供对本申请技术方案的进一步理解，并且构成说明书的一部分，与本申请的实施例一起用于解释本申请的技术方案，并不构成对本申请技术方案的限制。The accompanying drawings are used to provide a further understanding of the technical solution of the present application, and constitute a part of the specification, and are used together with the embodiments of the present application to explain the technical solution of the present application, and do not constitute a limitation to the technical solution of the present application.

图1是本申请一个实施例提供的图像处理方法的流程图；Fig. 1 is a flowchart of an image processing method provided by an embodiment of the present application;

图2是图1中步骤S400的一个具体方法的流程图；Fig. 2 is a flowchart of a specific method of step S400 in Fig. 1;

图3是本申请一个实施例提供的图像处理方法的流程图；FIG. 3 is a flowchart of an image processing method provided by an embodiment of the present application;

图4是本申请一个实施例提供的图像特征提取模型训练过程的流程图；Fig. 4 is the flow chart of the image feature extraction model training process provided by one embodiment of the present application;

图5是本申请一个实施例提供的图像特征提取模型特征提取网络结构示意图；FIG. 5 is a schematic diagram of a feature extraction network structure of an image feature extraction model provided by an embodiment of the present application;

图6是本申请一个实施例提供的目标区域示意图；Fig. 6 is a schematic diagram of a target area provided by an embodiment of the present application;

图7是本申请一个实施例提供的图像处理方法的流程图；FIG. 7 is a flowchart of an image processing method provided by an embodiment of the present application;

图8是本申请一个实施例提供的图像处理方法的效果示意图；Fig. 8 is a schematic diagram of the effect of an image processing method provided by an embodiment of the present application;

图9是本申请一个实施例提供的电子设备的示意图。Fig. 9 is a schematic diagram of an electronic device provided by an embodiment of the present application.

具体实施方式Detailed ways

为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处所描述的具体实施例仅用以解释本申请，并不用于限定本申请。In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, not to limit the present application.

需要说明的是，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于流程图中的顺序执行所示出或描述的步骤。说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than in the flowchart. The terms "first", "second" and the like in the specification and claims and the above drawings are used to distinguish similar objects, and not necessarily used to describe a specific sequence or sequence.

图像的识别与处理在工业机器视觉中具有重要的现实意义，例如在烟草行业，根据烟叶的颜色均匀度对烟叶进行等级判别，即颜色均匀度识别；在汽车、电子零部件质检过程中，根据图像的亮度均匀度进行产品瑕疵检测，即亮度均匀度识别；在纺织行业，根据图像的纹理均匀度判别潜在的布料瑕疵区域，即纹理均匀度识别。Image recognition and processing has important practical significance in industrial machine vision. For example, in the tobacco industry, tobacco leaves are graded according to their color uniformity, that is, color uniformity recognition; in the quality inspection process of automobiles and electronic parts, Product defect detection according to the brightness uniformity of the image, that is, brightness uniformity recognition; in the textile industry, potential fabric defect areas are identified according to the texture uniformity of the image, that is, texture uniformity recognition.

然而，相关技术中，对图像中特定区域的识别的需要依据特定的视觉特征，如颜色、亮度或者纹理等，且这些视觉特征大部分需要人为提前设计，如基于Lab、HSV等颜色模型的像素统计特性判断颜色均匀度；使用灰度共生矩阵、LBP等纹理特征描述子进行纹理特征提取，再根据特征统计特性进行纹理均匀度判别。上述通过人为预先设计视觉特征进行非均匀区域的图像识别方法在进行产品优劣或瑕疵检测时普适性不高，且无法在像素级别对图像中的非均匀之处进行显示，导致一些细小的瑕疵可能难以被肉眼观测，给用户的实际使用带来不便。However, in related technologies, the recognition of a specific region in an image needs to be based on specific visual features, such as color, brightness, or texture, and most of these visual features need to be designed in advance, such as pixels based on color models such as Lab and HSV. The color uniformity is judged by statistical characteristics; texture feature extraction is carried out by using texture feature descriptors such as gray level co-occurrence matrix and LBP, and then the texture uniformity is judged according to the characteristic statistical characteristics. The above-mentioned image recognition method for non-uniform areas by artificially pre-designing visual features is not universally applicable when detecting product quality or defects, and cannot display the non-uniform parts of the image at the pixel level, resulting in some small Defects may be difficult to observe with the naked eye, causing inconvenience to users in actual use.

本申请提供了一种图像处理方法、电子设备、计算机可读存储介质及计算机程序产品，获取待处理图像与待处理图像中像素的特征向量，获取预先训练得到的均匀像素特征向量集合，然后根据均匀像素特征向量集合与像素的特征向量，得到像素的向量均匀度，最后根据像素的向量均匀度，得到待处理图像的均匀度热力图。根据本申请实施例的方案，通过提取待处理图像中像素的特征向量，并基于预先训练得到的均匀像素特征向量集合进行向量均匀度的度量，输出能够精确反映图像中像素均匀度的均匀度热力图，方便用户对非均匀区域的识别与优化，且无需人工提前设置视觉特征，使图像处理方法具有更加广泛的应用场景。The application provides an image processing method, an electronic device, a computer-readable storage medium, and a computer program product. The image to be processed and the feature vector of the pixels in the image to be processed are obtained, and a set of uniform pixel feature vectors obtained by pre-training is obtained, and then according to The set of uniform pixel feature vectors and the feature vector of the pixel are used to obtain the vector uniformity of the pixel, and finally the uniformity heat map of the image to be processed is obtained according to the vector uniformity of the pixel. According to the scheme of the embodiment of the present application, by extracting the feature vectors of the pixels in the image to be processed, and measuring the vector uniformity based on the set of uniform pixel feature vectors obtained through pre-training, the uniformity heat that can accurately reflect the uniformity of the pixels in the image is output It is convenient for users to identify and optimize non-uniform areas, and it does not need to manually set visual features in advance, so that the image processing method has a wider range of application scenarios.

下面结合附图，对本申请实施例作进一步阐述。The embodiments of the present application will be further described below in conjunction with the accompanying drawings.

如图1所示，图1是本申请一个实施例提供的图像处理方法的流程图，该图像处理方法可以应用在终端，例如手机、计算机、摄像头等电子设备上。在图1的示例中，该图像处理方法可以包括但不限于步骤S100至步骤S500。As shown in FIG. 1 , FIG. 1 is a flow chart of an image processing method provided by an embodiment of the present application. The image processing method can be applied to terminals, such as electronic devices such as mobile phones, computers, and cameras. In the example of FIG. 1, the image processing method may include but not limited to step S100 to step S500.

步骤S100：获取待处理图像。Step S100: Acquiring images to be processed.

需要说明的是，对任一待处理图像，空间分辨率没有限制，即无论待处理图像是高分辨率的高清图像或是低分辨率的模糊图像，对本申请实施例中图像处理方法的实施没有影响。It should be noted that for any image to be processed, the spatial resolution is not limited, that is, no matter whether the image to be processed is a high-resolution high-definition image or a low-resolution blurred image, there is no limitation on the implementation of the image processing method in the embodiment of the present application. Influence.

步骤S200：获取待处理图像中像素的特征向量。Step S200: Obtain feature vectors of pixels in the image to be processed.

需要说明的是，为了获取待处理图像中像素的特征向量，可以将待处理图像输入到图像特征提取模型，模型的输出就是待处理图像中像素的特征向量。基于不同的场景或需求，可以设置对完整的待处理图像进行特征提取，也可以对局部的待处理图像进行特征提取，局部的图像特征提取可以减少特征提取的时间，进而提高特征提取的效率。It should be noted that, in order to obtain the feature vector of the pixel in the image to be processed, the image to be processed can be input to the image feature extraction model, and the output of the model is the feature vector of the pixel in the image to be processed. Based on different scenarios or requirements, feature extraction can be performed on complete images to be processed, or feature extraction on partial images to be processed. Local image feature extraction can reduce the time of feature extraction and improve the efficiency of feature extraction.

还需要说明的是，在一些应用场景中，可以通过机器学习模型或者人工加工的方式，预先获得各种典型的非均匀图像以及其对应的特征向量，因此，在获得待处理图像时，可以通过人工或模型对待处理图像进行分类或判断，获得其对应的特征向量。这种特征向量的获取方式更加简单快速，尤其适用于不支持随时运行图像特征提取模型的应用场景。It should also be noted that in some application scenarios, various typical non-uniform images and their corresponding feature vectors can be obtained in advance through machine learning models or manual processing. Therefore, when obtaining images to be processed, you can use The image to be processed is classified or judged manually or by the model, and its corresponding feature vector is obtained. This method of obtaining feature vectors is simpler and faster, and is especially suitable for application scenarios that do not support running image feature extraction models at any time.

步骤S300：获取预先训练得到的均匀像素特征向量集合。Step S300: Obtain a set of uniform pixel feature vectors obtained in pre-training.

需要说明的是，均匀像素特征向量集合是对机器学习中图像特征提取模型进行训练后模型输出的像素特征向量集合，是可以通过预先训练得到的。用于训练模型的训练样本可以是完整图像或局部图像，训练样本的分辨率可以相同也可以不相同。It should be noted that the set of uniform pixel feature vectors is the set of pixel feature vectors output by the model after training the image feature extraction model in machine learning, and can be obtained through pre-training. The training samples used to train the model can be complete images or partial images, and the resolutions of the training samples can be the same or different.

需要说明的是，图像的均匀度可体现在多种方面，例如图像中的颜色均匀度、亮度均匀度、纹理均匀度等，本申请实施例提出的图像处理方法，能够实现多种情况、多种类型下的图像均匀度识别，下文中采用三种示例作进一步说明。It should be noted that the uniformity of an image can be reflected in various aspects, such as color uniformity, brightness uniformity, texture uniformity, etc. in the image. The image processing method proposed in the embodiment of this application can realize various The recognition of image uniformity under different types, three examples are used for further explanation below.

示例一：Example one:

为了得到能够对图像中的颜色特征提取的图像特征提取模型，首先采用多张具有均匀颜色的完整图像或完整图像的局部图像块作为训练样本，对ImageNet大规模数据集上预训练的ResNet50卷积神经网络模型进行训练，此时，ResNet50是图像特征提取模型，通过此模型能够对具有均匀颜色的像素特征进行特征提取，进而通过均匀像素特征向量集合进行表示。In order to obtain an image feature extraction model that can extract color features in an image, firstly multiple complete images with uniform colors or local image blocks of complete images are used as training samples, and the ResNet50 pre-trained on the ImageNet large-scale dataset is convoluted The neural network model is trained. At this time, ResNet50 is an image feature extraction model. Through this model, the feature extraction of pixel features with uniform color can be performed, and then represented by a set of uniform pixel feature vectors.

示例二：Example two:

为了得到能够对图像中的纹理特征提取的图像特征提取模型，采用多张具有均匀纹理的完整图像或完整图像的局部图像块作为训练样本，对经过预训练的DenseNet卷积神经网络模型进行训练，此时，DenseNet是图像特征提取模型，通过此模型能够对具有均匀纹理的像素特征进行特征提取，进而通过均匀像素特征向量集合进行表示。In order to obtain an image feature extraction model capable of extracting texture features in images, multiple complete images with uniform texture or local image blocks of complete images are used as training samples to train the pre-trained DenseNet convolutional neural network model. At this time, DenseNet is an image feature extraction model, through which the feature extraction of pixel features with uniform texture can be performed, and then represented by a set of uniform pixel feature vectors.

示例三：Example three:

为了得到能够对图像中的亮度特征提取的图像特征提取模型，采用多张具有均匀亮度的完整图像或完整图像的局部图像块作为训练样本，对经过预训练的其他卷积神经网络模型进行深度学习，通过此模型能够对具有亮度纹理的像素特征进行特征提取，进而通过均匀像素特征向量集合进行表示。In order to obtain an image feature extraction model that can extract the brightness features in the image, multiple complete images with uniform brightness or partial image blocks of the complete image are used as training samples to conduct deep learning on other pre-trained convolutional neural network models , through this model, the feature extraction of pixel features with brightness texture can be performed, and then represented by a set of uniform pixel feature vectors.

本领域技术人员可以理解的是，上述三个示例采用深度卷积神经网络模型作为图像特征提取模型是因为其在图像识别方面具有良好的表现，但是其他机器学习模型也可以应用到本申请实施例提供的图像处理方法中。Those skilled in the art can understand that the above three examples use the deep convolutional neural network model as the image feature extraction model because it has good performance in image recognition, but other machine learning models can also be applied to the embodiment of the present application In the image processing method provided.

需要说明的是，均匀像素特征向量集合可以是通过模型训练后直接提取的图像像素特征向量的集合，也可以是对模型训练后提取的图像像素特征聚类后得到的数量更少的图像像素特征集合。It should be noted that the set of uniform pixel feature vectors can be a set of image pixel feature vectors directly extracted after model training, or a smaller number of image pixel features obtained by clustering image pixel features extracted after model training gather.

在一实施例中，为了提高均匀像素特征向量的表征能力，对图像特征提取模型的输出层的输出特征进行特征聚合。In one embodiment, in order to improve the representation capability of the uniform pixel feature vector, feature aggregation is performed on the output features of the output layer of the image feature extraction model.

步骤S400：根据均匀像素特征向量集合与像素的特征向量，得到像素的向量均匀度。Step S400: According to the uniform pixel feature vector set and the pixel feature vector, obtain the vector uniformity of the pixel.

需要说明的是，特征之间相似度的度量方法具有多种，其主要取决于特征向量之间的计算关系。对于特征相似度的度量，特征相似度越小，说明两个特征向量之间差距越大，图像像素的差距就越大，反之，特征相似度越大，说明两个特征向量之间差距越小，图像像素的差距就越小。例如，在对图像上的颜色、亮度、纹理等进行是否均匀的识别与处理时，是基于对待处理图像上的像素点的特征向量与均匀像素特征向量集合中的特征向量进行相似度比较，当相似度高时，说明待处理图像上的像素点与均匀的像素点在特征上是相似的，因此可能识别为颜色、亮度、纹理均匀的像素点。It should be noted that there are many methods for measuring the similarity between features, which mainly depend on the calculation relationship between feature vectors. For the measurement of feature similarity, the smaller the feature similarity, the larger the gap between the two feature vectors, and the larger the gap between image pixels. Conversely, the larger the feature similarity, the smaller the gap between the two feature vectors. , the smaller the difference in image pixels. For example, when identifying and processing whether the color, brightness, texture, etc. on the image are uniform, it is based on the similarity comparison between the feature vectors of the pixels on the image to be processed and the feature vectors in the set of uniform pixel feature vectors. When the similarity is high, it means that the pixels on the image to be processed are similar in characteristics to uniform pixels, so it may be recognized as pixels with uniform color, brightness, and texture.

在一实施例中，图像处理方法提供了一种通过计算余弦均匀度，进而获得像素均匀度的度量方法，如图2所示，包括以下步骤：In one embodiment, the image processing method provides a measurement method for obtaining pixel uniformity by calculating cosine uniformity, as shown in FIG. 2 , including the following steps:

参照图2所示，在一实施例中，对步骤S400进行说明，步骤S400可以包括但不限于以下步骤：Referring to FIG. 2, in an embodiment, step S400 is described, and step S400 may include but not limited to the following steps:

步骤S411：根据像素的特征向量与均匀像素特征向量集合中的各个均匀像素特征向量，计算得到像素的向量均匀度集合，向量均匀度集合包括向量均匀度元素。Step S411: According to the feature vector of the pixel and each uniform pixel feature vector in the uniform pixel feature vector set, calculate the vector uniformity set of the pixel, and the vector uniformity set includes the vector uniformity element.

步骤S412：将像素的向量均匀度集合中数值最大的向量均匀度元素，确定为像素的向量均匀度。Step S412: Determine the vector uniformity element with the largest value in the vector uniformity set of the pixel as the vector uniformity of the pixel.

在一实施例中，像素的向量均匀度可以为余弦均匀度，假设待处理图像中的像素p的特征向量为v _p，根据训练得到的均匀像素特征向量集合为{v ₀,v ₁,L,v _N}，则可以根据以下公式计算得到像素p的余弦均匀度： In one embodiment, the vector uniformity of a pixel may be cosine uniformity. Assume that the feature vector of pixel p in the image to be processed is v _p , and the set of uniform pixel feature vectors obtained according to training is {v ₀ , v ₁ , L ,v _N }, then the cosine uniformity of pixel p can be calculated according to the following formula:

上述公式(1)中，s _p表示余弦均匀度，v _q表示均匀像素特征向量集合中的均匀像素特征向量，N表示均匀像素特征向量集合中向量的数量。 In the above formula (1), _sp represents the cosine uniformity, v _q represents the uniform pixel feature vector in the uniform pixel feature vector set, and N represents the number of vectors in the uniform pixel feature vector set.

值得注意的是，上述计算过程仅是对待处理图像中一个像素进行的余弦均匀度计算，在图像处理方法的实际应用中，会对待处理图像中多个像素甚至是全部像素均执行上述计算。其中，计算像素p与均匀像素特征向量集合中每个特征向量之间的余弦值，并选取最大的余弦相似度作为像素p的余弦相似度，能够得到最能够代表像素p的特征向量。可以理解的是，余弦均匀度仅是两个向量之间距离度量的一种表示，在实际应用中，可能需要对余弦均匀度进行进一步计算得到均匀度热力图中像素的实际均匀度值。It is worth noting that the above calculation process is only the cosine uniformity calculation of one pixel in the image to be processed. In the actual application of the image processing method, the above calculation will be performed on multiple pixels or even all pixels in the image to be processed. Among them, the cosine value between the pixel p and each feature vector in the uniform pixel feature vector set is calculated, and the largest cosine similarity is selected as the cosine similarity of the pixel p, so that the feature vector that can best represent the pixel p can be obtained. It can be understood that the cosine uniformity is only a representation of the distance measure between two vectors. In practical applications, it may be necessary to further calculate the cosine uniformity to obtain the actual uniformity value of the pixel in the uniformity heat map.

步骤S500：根据像素的向量均匀度，得到待处理图像的均匀度热力图。Step S500: According to the vector uniformity of the pixels, obtain the uniformity heat map of the image to be processed.

需要说明的是，均匀度热力图是以像素为单位的能够反映图像中像素均匀度的图，图中每个像素的颜色深浅能够反映与之对应的待处理图像中的像素的均匀度情况。采用余弦均匀度计算像素的均匀度情况，可以在获得像素均匀度后，依据实际需求，人工设置热力图的显示方式。例如，如希望用深颜色表示非均匀区域，浅颜色表示均匀区域，对于余弦均匀度度量方法，则需要对余弦均匀度值小的像素采用深色表示，对余弦均匀度值大的像素采用浅色表示。像素深浅是依据用户需求设定的，而其中像素深浅的区别在于使用不同均匀度方法度量后得到的均匀度值。It should be noted that the uniformity heat map is a map that can reflect the uniformity of pixels in the image in units of pixels, and the color depth of each pixel in the graph can reflect the uniformity of the corresponding pixels in the image to be processed. Using the cosine uniformity to calculate the uniformity of the pixels, you can manually set the display mode of the heat map according to the actual needs after obtaining the uniformity of the pixels. For example, if you want to use dark colors to represent non-uniform areas and light colors to represent uniform areas, for the cosine uniformity measurement method, you need to use dark colors for pixels with small cosine uniformity values, and use light colors for pixels with large cosine uniformity values. color representation. The pixel depth is set according to the user's needs, and the difference between the pixel depth is the uniformity value measured by different uniformity methods.

需要说明的是，像素均匀度热力图可以为待处理图像的全局像素热力图，也可以为局部像素均匀度热力图。输出像素均匀度热力图能够帮助用户更好的观察到目标像素的分布情况。It should be noted that the pixel uniformity heat map may be a global pixel heat map of the image to be processed, or a local pixel uniformity heat map. The output pixel uniformity heat map can help users better observe the distribution of target pixels.

还需要说明的是，像素的均匀度热力图是一种表示图像均匀度的热力图的总称，在实际应用中，也可以采用非均匀度热力图进行图像均匀度的表示，其生成方法与原理与均匀度热力图相同。It should also be noted that the uniformity heat map of pixels is a general term for a heat map representing image uniformity. In practical applications, non-uniformity heat maps can also be used to represent image uniformity. Its generation method and principle Same as Uniformity Heatmap.

在后文的实施例中，对均匀度热力图进行进一步标识处理，使非均匀像素更加突出的显示出来。如图3所示，至少包括以下步骤：In the following embodiments, further identification processing is performed on the uniformity heat map, so that the non-uniform pixels are displayed more prominently. As shown in Figure 3, at least the following steps are included:

步骤S410：根据均匀像素特征向量集合与像素的特征向量，得到像素的余弦均匀度。Step S410: Obtain the cosine uniformity of the pixel according to the uniform pixel feature vector set and the pixel feature vector.

在本申请实施例中，采用余弦均匀度度量像素的均匀度。In the embodiment of the present application, the cosine uniformity is used to measure the uniformity of pixels.

步骤S510：根据像素的余弦均匀度，得到待处理图像的均匀度热力图。Step S510: According to the cosine uniformity of the pixels, the uniformity heat map of the image to be processed is obtained.

步骤S600：根据均匀度热力图和预设的均匀度判断条件，在像素中确定非均匀像素。在本申请实施例中，使用余弦相似度进行均匀度度量时，非均匀度判断条件为，像素对应的余弦均匀度是否小于第一预设像素均匀度阈值，在一示例中通过将计算得到的多个像素的余弦均匀度与第一预设像素均匀度阈值进行比较，对于各个像素，如果其余弦均匀度大于第一预设像素均匀度阈值，则此像素为均匀像素；如果其余弦均匀度小于第一预设像素均匀度阈值，则此像素为非均匀像素。Step S600: Determine non-uniform pixels among pixels according to the uniformity heat map and preset uniformity judgment conditions. In the embodiment of the present application, when using the cosine similarity to measure the uniformity, the non-uniformity judgment condition is whether the cosine uniformity corresponding to the pixel is less than the first preset pixel uniformity threshold, in an example, the calculated The cosine uniformity of multiple pixels is compared with the first preset pixel uniformity threshold. For each pixel, if the cosine uniformity is greater than the first preset pixel uniformity threshold, the pixel is a uniform pixel; if the cosine uniformity If it is smaller than the first preset pixel uniformity threshold, the pixel is a non-uniform pixel.

需要说明的是，在本申请实施例中，是通过判断各个像素的均匀度是否小于预设像素均匀度阈值，来识别出异常的像素，即识别出不均匀的像素。It should be noted that, in the embodiment of the present application, abnormal pixels, that is, non-uniform pixels are identified by judging whether the uniformity of each pixel is smaller than a preset pixel uniformity threshold.

步骤S700：对非均匀像素进行标识处理。需要说明的是，对非均匀像素的标识处理有多种方法，只要是能够将均匀像素与非均匀像素进行区分的方法均属于对非均匀像素进行了标识处理。Step S700: Perform identification processing on non-uniform pixels. It should be noted that there are many methods for identifying non-uniform pixels, and any method capable of distinguishing uniform pixels from non-uniform pixels belongs to the identification process for non-uniform pixels.

可以理解的是，非均匀像素的标识为在待处理图像上突出显示非均匀像素，例如把非均匀像素的轮廓圈出或者高亮显示。需要说明的是，像素均匀度热力图可以为待处理图像的全局像素热力图，也可以为局部像素热力图。输出像素热力图能够帮助用户更好的观察到目标像素的分布情况。It can be understood that the identification of the non-uniform pixels is to highlight the non-uniform pixels on the image to be processed, for example, circle or highlight the contours of the non-uniform pixels. It should be noted that the pixel uniformity heat map may be a global pixel heat map of the image to be processed, or a local pixel heat map. The output pixel heat map can help users better observe the distribution of target pixels.

本申请实施例提供的图像处理方法，通过将待处理的图像的图像特征的提取，通过与预先训练得到的均匀像素特征向量进行对比，能够对待处理图像中不符合特征要求的非均匀像素进行识别，无需人工提前设置视觉特征，使图像处理方法具有更加广泛的应用场景；同时还能对识别出的非均匀像素进行标识，方便用户对这些非均匀像素进行识别与优化，综合提升了图像处理能力。The image processing method provided in the embodiment of the present application can identify non-uniform pixels in the image to be processed that do not meet the feature requirements by extracting the image features of the image to be processed and comparing it with the uniform pixel feature vector obtained through pre-training , no need to manually set visual features in advance, so that the image processing method has a wider range of application scenarios; at the same time, it can also identify the identified non-uniform pixels, which is convenient for users to identify and optimize these non-uniform pixels, and comprehensively improves image processing capabilities .

如图4所示，图4是本申请一个实施例提供的图像特征提取模型训练过程的流程图，至少包括如下步骤：As shown in Figure 4, Figure 4 is a flow chart of the image feature extraction model training process provided by one embodiment of the present application, at least including the following steps:

步骤S310：获取训练样本，训练样本为均匀图像。Step S310: Obtain a training sample, which is a uniform image.

需要说明的是，训练样本，可以是一些完整图像，也可以是图像的一个局部图像块，同时，不同训练样本图像的空间分辨率可以不一致。在模型训练的过程中，可以采用同一批样本反复进行输入，也可以将同一批样本分批，每一次迭代，输入不同批次的样本。It should be noted that the training samples may be some complete images or a partial image block of the image, and meanwhile, the spatial resolutions of different training sample images may be inconsistent. In the process of model training, the same batch of samples can be used for repeated input, or the same batch of samples can be divided into batches, and different batches of samples can be input for each iteration.

步骤S320：将训练样本输入至图像特征提取模型，得到样本均匀像素特征向量集合；其中，图像特征提取模型为卷积神经网络模型。Step S320: Input the training samples into the image feature extraction model to obtain a sample uniform pixel feature vector set; wherein, the image feature extraction model is a convolutional neural network model.

在一实施例中，采用卷积神经网络作为图像特征提取模型，其相对于其他机器学习模型，具有较强的特征提取能力。使用在ImageNet、COCO等大规模数据集上预训练的卷积神经网络模型提取图像分布像素的特征。而采用无监督学习的特征提取方法可以避免人为对数据加标签，不仅能够大大减少人为标注的工作量，还能使任何大规模数据集上的预训练模型均可用于本申请实施例提供的图像均匀度识别方案。In one embodiment, a convolutional neural network is used as an image feature extraction model, which has stronger feature extraction capabilities than other machine learning models. Use the convolutional neural network model pre-trained on large-scale data sets such as ImageNet and COCO to extract the features of image distribution pixels. The feature extraction method using unsupervised learning can avoid artificially labeling the data, not only can greatly reduce the workload of human labeling, but also make the pre-trained model on any large-scale data set available for the image provided by the embodiment of this application Uniformity identification scheme.

值得说明的是，在图像特征提取过程中，特征维度非常高，为了提高深度特征的表征能力，需要增加特征感受野，对不同卷积层的输出特征进行聚合，采用特征聚合的卷积神经网络拓扑图如图5所示。It is worth noting that in the process of image feature extraction, the feature dimension is very high. In order to improve the representation ability of deep features, it is necessary to increase the feature receptive field, aggregate the output features of different convolutional layers, and use the convolutional neural network of feature aggregation. The topology diagram is shown in Figure 5.

在本申请实施例中，将待处理图像输入至输入层101，输出层102至105在卷积神经网络中的作用是特征提取，采用卷积神经网络的中间三个输出层进行特征聚合，其中，假设输出层3 104是第j层输出层，那么输出层2 103是j-1层，输出层4 105是j+1层。In the embodiment of the present application, the image to be processed is input to the input layer 101, and the function of the output layers 102 to 105 in the convolutional neural network is feature extraction, and the middle three output layers of the convolutional neural network are used for feature aggregation, wherein , assuming that the output layer 3 104 is the jth output layer, then the output layer 2 103 is the j-1 layer, and the output layer 4 105 is the j+1 layer.

样本x _i经过预训练卷积神经网络模型推理后第j层输出的特征图表示为： The feature map output by the jth layer of the sample x _i after pre-trained convolutional neural network model reasoning is expressed as:

上述公式(2)中，C _j、H _j、W _j分别表示特征图的通道数、高度及宽度，φ _ij表示样本i在第j层的输出。 In the above formula (2), C _j , H _j , and W _j represent the number of channels, height, and width of the feature map, respectively, and φ _ij represents the output of sample i at layer j.

在本申请实施例中，使用预训练模型的中间三层输出进行特征聚合，聚合特征表示为：In the embodiment of this application, the output of the middle three layers of the pre-trained model is used for feature aggregation, and the aggregated feature is expressed as:

F(x _i)＝f _agg({φ _ij-1,φ _ij,φ _ij+1}),j∈{2} (3) F(x _i )＝f _agg ({φ _ij-1 ,φ _ij ,φ _ij+1 }),j∈{2} (3)

上述公式(3)中，f _agg(·)为聚合操作算子，首先对输出层3 104的输出特征φ _ij进行2x上采样，对输出层4 105的输出特征φ _ij+1进行4x上采样，然后将插值后的特征图与输出层2103的特征图φ _ij-1进行通道合并，以此完成三个不同层的特征聚合；最后对聚合特征进行上采样，使输出空间分辨率与输入图像分辨率保存一致。 In the above formula (3), f _agg ( ) is an aggregation operator, firstly perform 2x upsampling on the output feature φ _ij of the output layer 3 104, and perform 4x upsampling on the output feature φ _ij+1 of the output layer 4 105 , and then merge the interpolated feature map with the feature map φ _ij-1 of the output layer 2103 to complete the feature aggregation of three different layers; finally, upsample the aggregated features to make the output spatial resolution the same as the input image The resolution is kept consistent.

假定训练样本x _i的空间分辨率为H _i×W _i，通过预训练卷积神经网络提取到的深度特征表示为： Assuming that the spatial resolution of the training sample _xi is H _i ×W _i , the depth features extracted by the pre-trained convolutional neural network are expressed as:

F _i＝f(x _i) (4) F _i ＝f(x _i ) (4)

上述公式(4)中，F _i表示训练样本通过预训练卷积神经网络提取到的深度特征表示，f(·)表示特征提取算子，特征提取算子f(·)的输出通道数为C _f，空间维度为H _i×W _i，特征空间维度与输入图像空间分辨率保存一致，即每一个像素可以用一个C _f维度的特征向量来表示，经过特征提取后，样本x _i可以得到H _i×W _i个均匀像素的C _f维度深度特征向量，每一个像素的特征向量表示为： In the above formula (4), F _i represents the deep feature representation extracted by the training sample through the pre-trained convolutional neural network, f( ) represents the feature extraction operator, and the number of output channels of the feature extraction operator f( ) is C _f , the spatial dimension is H _i ×W _i , and the feature space dimension is consistent with the spatial resolution of the input image, that is, each pixel can be represented by a feature vector of C _f dimension. After feature extraction, the sample _xi can get H The C _f -dimensional depth feature vector of _i ×W _i uniform pixels, the feature vector of each pixel is expressed as:

上述公式(5)中，v _k表示像素的深度特征向量，

表示像素的深度特征向量的维度。 In the above formula (5), v _k represents the depth feature vector of the pixel,

Denotes the dimensionality of the depth feature vector for a pixel.

通过上述特征提取，可以获取大量像素的特征向量，假定像素的个数为M，即预设的目标聚类特征向量维度为M，则样本均匀像素特征向量集合可表示为：Through the above feature extraction, the feature vectors of a large number of pixels can be obtained. Assuming that the number of pixels is M, that is, the preset target clustering feature vector dimension is M, then the sample uniform pixel feature vector set can be expressed as:

Ω＝{v ₀,v ₁,Λ,v _M} (6) Ω＝{v ₀ ,v ₁ ,Λ,v _M } (6)

上述公式(6)中，Ω表示样本均匀像素特征向量集合，v _M表示样本均匀像素特征向量。 In the above formula (6), Ω represents the sample uniform pixel feature vector set, and v _M represents the sample uniform pixel feature vector.

步骤S330：对样本均匀像素特征向量集合中的元素进行聚类处理，得到均匀像素特征向量集合。Step S330: Perform clustering processing on the elements in the set of sample uniform pixel feature vectors to obtain a set of uniform pixel feature vectors.

通常情况下，由于训练样本较多，训练样本图像分辨率较大，涉及像素个数众多，为了便于后续像素级均匀度度量，需对特征集合进行聚类，减少特征向量像素数量级。Usually, due to the large number of training samples, the resolution of the training sample image is large, and the number of pixels involved is large. In order to facilitate the subsequent pixel-level uniformity measurement, the feature set needs to be clustered to reduce the order of magnitude of the feature vector pixels.

在本申请实施例中，k-means++聚类算法进行特征集合聚类，假定最终聚类的均匀像素个数为N(N<<M)，最终得到的像素特征向量集合可表示为：In the embodiment of the present application, the k-means++ clustering algorithm performs feature set clustering, assuming that the uniform number of pixels in the final cluster is N (N<<M), and the final pixel feature vector set can be expressed as:

Ω′＝{v ₀,v ₁,Λ,v _N} (7) Ω′＝{v ₀ ,v ₁ ,Λ,v _N } (7)

上述公式(7)中，Ω'表示聚类后的均匀像素特征向量集合。In the above formula (7), Ω' represents the set of uniform pixel feature vectors after clustering.

如图6所示，提供了目标区域示意图，每个正方形网格代表一个像素，在本申请实施例中，是对均匀纹理像素进行识别与标识。可以看出，图中像素A、B、C、D、E、F均为纹理不均匀像素，其中像素B、C、D、E、F相邻，构成了像素区域。在一些应用场景中，用户对一些面积较小的非均匀像素区域可能不是很关心，因此，可以通过提前对像素区域的面积进行设置，只对那些面积超过预设面积的像素区域进行标识，而这些满足标识条件的像素区域，被定义为目标区域。在本申请实施例中，采用轮廓线对目标区域进行标识，使其突显出来，方便用户观察。As shown in FIG. 6 , a schematic diagram of the target area is provided, and each square grid represents a pixel. In the embodiment of the present application, uniform texture pixels are identified and marked. It can be seen that the pixels A, B, C, D, E, and F in the figure are all pixels with uneven texture, and the pixels B, C, D, E, and F are adjacent to each other to form a pixel area. In some application scenarios, users may not be very concerned about some non-uniform pixel areas with small areas. Therefore, by setting the area of the pixel area in advance, only those pixel areas whose area exceeds the preset area can be identified. These pixel areas satisfying the identification conditions are defined as target areas. In the embodiment of the present application, the contour line is used to mark the target area to make it stand out, which is convenient for the user to observe.

在另一个实施例中，还可以对目标区域的数量及每个目标区域包含的像素数量进行输出显示，能够更加精确地让用户了解到图像情况，方便用户后续对图像甚至产品的优化。In another embodiment, the number of target areas and the number of pixels included in each target area can also be output and displayed, which can allow users to understand the image situation more accurately, and facilitate subsequent optimization of images and even products for users.

由于本申请提供的图像处理方法可以预先构造的图像处理模型实现，因此，在下面以一个实施例中，说明采用余弦构造的图像处理模型来实现本申请提供的图像处理方法的具体流程，如图7所示，包括以下步骤：Since the image processing method provided by the present application can be realized by a pre-configured image processing model, in the following an embodiment, the specific process of implementing the image processing method provided by the present application by using the cosine-structured image processing model is described, as shown in Fig. 7, including the following steps:

步骤S810：加载图像处理模型，图像处理模型中包含训练好的图像特征提取模型以及均匀像素的特征向量集合。Step S810: Load an image processing model, which includes a trained image feature extraction model and a set of feature vectors of uniform pixels.

可以理解的是，均匀像素的特征向量集合一般通过磁盘存储，形成模型文件，在需要使用时加载；通过预先输入多张颜色均匀、纹理均匀或者亮度均匀的样本图像对图像特征提取进行训练，而训练好的图像特征提取模型能够对输入的待处理图像进行特征的提取。图像特征提取模型可以作为图像处理模型中的一个功能模块存在。It can be understood that the feature vector set of uniform pixels is generally stored on disk to form a model file and loaded when needed; the image feature extraction is trained by inputting multiple sample images with uniform color, uniform texture or uniform brightness in advance, and The trained image feature extraction model can extract features from the input image to be processed. The image feature extraction model can exist as a functional module in the image processing model.

在本申请实施例中，选择4张均匀图像作为训练样本，图像分辨率为1080x1920，值得注意的是，本申请实施例的均匀样本也可以是均匀图像的一个局部图像块，不同训练图像的尺寸可以不一致。In the embodiment of this application, 4 uniform images are selected as training samples, and the image resolution is 1080x1920. It is worth noting that the uniform sample in the embodiment of this application can also be a local image block of a uniform image. The size of different training images Can be inconsistent.

步骤S820：将视频帧输入至图像处理模型。Step S820: Input the video frame into the image processing model.

在本申请实施例中，从磁盘加载离线视频并读取视频帧作为输入源，即待处理图像。In the embodiment of the present application, the offline video is loaded from the disk and the video frame is read as the input source, that is, the image to be processed.

但是值得说明的是，也可以通过rtmp或rtsp读取网络视频流作为输入源。But it is worth noting that the network video stream can also be read as an input source through rtmp or rtsp.

步骤S830：图像特征提取模型对输入的视频帧进行特征向量提取，得到待处理图像中像素的特征向量。Step S830: The image feature extraction model extracts feature vectors from the input video frame to obtain feature vectors of pixels in the image to be processed.

在本申请实施例中，任何大规模数据集上的预训练模型均可用于本申请的均匀像素特征提取，本申请实施例选择在ImageNet数据集上预训练的ResNet50卷积神经网络模型提取均匀分布像素的特征，获得像素的特征向量。In the embodiment of this application, any pre-trained model on a large-scale data set can be used for the uniform pixel feature extraction of this application, and the embodiment of this application chooses the ResNet50 convolutional neural network model pre-trained on the ImageNet data set to extract uniform distribution Features of the pixel, get the feature vector of the pixel.

本申请实施例中样本x _i的空间分辨率为1080x1920，通过预训练卷积神经网络提取到的深度特征表示为： In the embodiment of the present application, the spatial resolution of the sample x _i is 1080x1920, and the depth features extracted by the pre-trained convolutional neural network are expressed as:

F _i＝f(x _i) F _i =f(x _i )

深度特征提取算子f(·)的输出通道数为3584，空间维度为1080x1920，特征空间维度与输入图像空间分辨率保存一致，即每一个像素可以用一个3584维度的特征向量来表示。4张1080x1920的均匀样本图像可以获取M＝8294400个均匀像素的特征向量，每个特征向量的维度为3584，样本均匀像素特征向量集合可表示为：The number of output channels of the deep feature extraction operator f( ) is 3584, and the spatial dimension is 1080x1920. The feature space dimension is consistent with the spatial resolution of the input image, that is, each pixel can be represented by a 3584-dimensional feature vector. Four uniform sample images of 1080x1920 can obtain feature vectors of M=8294400 uniform pixels, and the dimension of each feature vector is 3584. The sample uniform pixel feature vector set can be expressed as:

Ω＝{v ₀,v ₁,Λ,v _M} Ω＝{v ₀ ,v ₁ ,Λ,v _M }

在本申请实施例中，为了降低后续特征均匀度的计算量，对特征集合进行聚类，降低特征向量像素数量级。使用k-means++聚类算法进行特征向量集合聚类，最终聚类的均匀像素个数为N＝100，最终得到的聚类后的样本均匀像素特征向量集合可表示为：In the embodiment of the present application, in order to reduce the calculation amount of subsequent feature uniformity, the feature set is clustered to reduce the order of magnitude of the feature vector pixels. Use the k-means++ clustering algorithm to cluster the eigenvector set, the number of uniform pixels in the final cluster is N=100, and the finally obtained clustered sample uniform pixel eigenvector set can be expressed as:

Ω′＝{v ₀,v ₁,Λ,v _N} Ω′＝{v ₀ ,v ₁ ,Λ,v _N }

与均匀样本学习过程类似，本申请实施例中假定视频帧y的空间分辨率为1080x1920，经过预训练模型提取到的深度特征F _y表示为： Similar to the uniform sample learning process, in the embodiment of the present application, it is assumed that the spatial resolution of the video frame y is 1080x1920, and the depth feature F _y extracted by the pre-trained model is expressed as:

F _y(i,j)＝f(y),(1≤i≤1080,1≤j≤1920) F _y (i, j) = f (y), (1≤i≤1080,1≤j≤1920)

其中，F _y的维度为3584×1080×1920。 Wherein, the dimension of F _y is 3584×1080×1920.

步骤S840：对图像处理模型将视频帧中每个像素的特征向量进行均匀度计算。Step S840: Calculate the uniformity of the feature vector of each pixel in the video frame by the image processing model.

根据已经训练到每个均匀像素的样本均匀像素特征向量集合Ω′，对待处理图像的深度特征图F _y进行特征均匀度的度量，也即均匀度的度量，得到每个像素位置的均匀度热力图： According to the sample uniform pixel feature vector set Ω′ that has been trained to each uniform pixel, the depth feature map F _y of the image to be processed is measured for feature uniformity, that is, the measurement of uniformity, and the uniformity thermal power of each pixel position is obtained picture:

S _y(i,j),(1≤i≤1080,1≤j≤1920) S _y (i,j),(1≤i≤1080,1≤j≤1920)

S _y的取值范围为[0,1]，0表明均匀度最差，1表明均匀度最好。 The value range of _Sy is [0,1], 0 indicates the worst uniformity, and 1 indicates the best uniformity.

本申请实施例中，特征均匀度度量采用向量余弦均匀度度量方法，对待度量像素p及其深度特征向量v _p，根据训练得到的样本均匀像素特征向量集合{v ₀,v ₁,L,v _N}，像素p的均匀度取值可以表示为： In the embodiment of the present application, the vector cosine uniformity measurement method is used to measure the feature uniformity. The pixel p to be measured and its depth feature vector v _p are obtained according to the sample uniform pixel feature vector set {v ₀ , v ₁ , L, v _N }, the uniformity value of pixel p can be expressed as:

本申请实施例中N＝100。In the examples of this application, N=100.

步骤S850：显示视频帧对应的像素均匀度热力图。Step S850: Display the heat map of pixel uniformity corresponding to the video frame.

步骤S860：提取出均匀度不达标的非均匀像素，圈出非均匀像素的轮廓。Step S860: Extract non-uniform pixels whose uniformity is not up to standard, and circle the contours of the non-uniform pixels.

如图8所示，提供了根据上述实施例图像处理方法对图像进行处理的效果示意图，可以看出，输出了待处理图像对应的非均匀度热力图，在非均匀度热力图中，1表明均匀度最差，0表明均匀度最好，也即颜色越深，说明越不均匀，颜色越浅，越均匀，用户可以很容易地看出图像中不均匀的像素点或区域，直观有效。As shown in Figure 8, a schematic diagram of the effect of image processing according to the image processing method of the above-mentioned embodiment is provided. It can be seen that the non-uniformity heat map corresponding to the image to be processed is output. In the non-uniformity heat map, 1 indicates The uniformity is the worst, and 0 indicates the best uniformity, that is, the darker the color, the more uneven it is, and the lighter the color, the more uniform it is. Users can easily see the uneven pixels or areas in the image, which is intuitive and effective.

在另一些实施例中，为了更加明显的标识，可以在均匀度热力图中叠加显示非均匀区域轮廓，并显示非均匀区域个数。In some other embodiments, for more obvious identification, the outline of the non-uniform region may be superimposed on the uniformity heat map, and the number of non-uniform regions may be displayed.

另外，本申请的一个实施例还提供了一种电子设备110，该电子设备110包括：处理器111、存储器112及存储在存储器上并可在处理器上运行的计算机程序，处理器能够执行计算机程序实现如上所述的图像处理方法。In addition, an embodiment of the present application also provides an electronic device 110, the electronic device 110 includes: a processor 111, a memory 112, and a computer program stored in the memory and operable on the processor, and the processor can execute computer The program realizes the image processing method as described above.

存储器作为一种非暂态计算机可读存储介质，可用于存储非暂态软件程序以及非暂态性计算机可执行程序。此外，存储器可以包括高速随机存取存储器，还可以包括非暂态存储器，例如至少一个磁盘存储器件、闪存器件、或其他非暂态固态存储器件。在一些实施方式中，存储器可包括相对于处理器远程设置的存储器，这些远程存储器可以通过网络连接至该处理器。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。As a non-transitory computer-readable storage medium, memory can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory may include memory located remotely from the processor, which remote memory may be connected to the processor via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

实现上述实施例的图像处理方法所需的非暂态软件程序以及指令存储在存储器中，当被处理器执行时，执行上述实施例中的图像处理方法，例如，执行以上描述的图1中的方法步骤S100至S500、图2中的方法步骤S411至S412、图3中的方法步骤S100至S700、图4中的方法步骤S310至S330、图7中的方法步骤S810至S860。The non-transitory software programs and instructions required to realize the image processing method of the above-mentioned embodiment are stored in the memory, and when executed by the processor, the image processing method in the above-mentioned embodiment is executed, for example, the above-described image processing method in FIG. 1 is executed. Method steps S100 to S500, method steps S411 to S412 in FIG. 2 , method steps S100 to S700 in FIG. 3 , method steps S310 to S330 in FIG. 4 , method steps S810 to S860 in FIG. 7 .

以上所描述的网元实施例仅仅是示意性的，其中作为分离部件说明的单元可以是或者也可以不是物理上分开的，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本申请实施例方案的目的。The network element embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the embodiment of the present application.

此外，本申请的一个实施例还提供了一种计算机可读存储介质，该计算机可读存储介质存储有计算机可执行指令，该计算机可执行指令被一个处理器或控制器执行，例如，被上述网元实施例中的一个处理器执行，可使得上述处理器执行上述实施例中的图像处理方法，例如，执行以上描述的图1中的方法步骤S100至S500、图2中的方法步骤S411至S412、图3中的方法步骤S100至S700、图4中的方法步骤S310至S330、图7中的方法步骤S810至S860。In addition, an embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by a processor or a controller, for example, by the above-mentioned Execution by a processor in the network element embodiment may cause the processor to execute the image processing method in the above embodiment, for example, execute the method steps S100 to S500 in FIG. 1 and the method steps S411 to S411 in FIG. 2 described above. S412, method steps S100 to S700 in FIG. 3 , method steps S310 to S330 in FIG. 4 , method steps S810 to S860 in FIG. 7 .

此外，本申请实施例还提供了一种计算机程序产品，包括计算机程序或计算机指令，所述计算机程序或所述计算机指令存储在计算机可读存储介质中，计算机设备的处理器从所述计算机可读存储介质读取所述计算机程序或所述计算机指令，所述处理器执行所述计算机程序或所述计算机指令，使得所述计算机设备执行如上所述的图像处理方法，例如，执行以上描述的图1中的方法步骤S100至S500、图2中的方法步骤S411至S412、图3中的方法步骤S100至S700、图4中的方法步骤S310至S330、图7中的方法步骤S810至S860。In addition, an embodiment of the present application also provides a computer program product, including a computer program or a computer instruction, the computer program or the computer instruction is stored in a computer-readable storage medium, and the processor of the computer device can read from the computer Reading the storage medium to read the computer program or the computer instruction, the processor executes the computer program or the computer instruction, so that the computer device executes the above-mentioned image processing method, for example, executes the above-described Method steps S100 to S500 in FIG. 1 , method steps S411 to S412 in FIG. 2 , method steps S100 to S700 in FIG. 3 , method steps S310 to S330 in FIG. 4 , and method steps S810 to S860 in FIG. 7 .

本申请实施例包括：获取待处理图像与待处理图像中像素的特征向量，获取预先训练得到的均匀像素特征向量集合，然后根据均匀像素特征向量集合与像素的特征向量，得到像素的向量均匀度，最后根据像素的向量均匀度，得到待处理图像的均匀度热力图。根据本申请实施例的方案，通过提取待处理图像中像素的特征向量，并基于预先训练得到的均匀像素特征向量集合进行向量均匀度的度量，输出能够精确反映图像中像素均匀度的均匀度热力图，方便用户对非均匀区域的识别与优化，且无需人工提前设置视觉特征，使图像处理方法具有更加广泛的应用场景。The embodiment of the present application includes: obtaining the image to be processed and the feature vector of the pixel in the image to be processed, obtaining the uniform pixel feature vector set obtained in advance training, and then obtaining the vector uniformity of the pixel according to the uniform pixel feature vector set and the pixel feature vector , and finally according to the vector uniformity of the pixel, the uniformity heat map of the image to be processed is obtained. According to the scheme of the embodiment of the present application, by extracting the feature vectors of the pixels in the image to be processed, and measuring the vector uniformity based on the set of uniform pixel feature vectors obtained through pre-training, the uniformity heat that can accurately reflect the uniformity of the pixels in the image is output It is convenient for users to identify and optimize non-uniform areas, and it does not need to manually set visual features in advance, so that the image processing method has a wider range of application scenarios.

本领域普通技术人员可以理解，上文中所公开方法中的全部或某些步骤、***可以被实施为软件、固件、硬件及其适当的组合。某些物理组件或所有物理组件可以被实施为由处理器，如中央处理器、数字信号处理器或微处理器执行的软件，或者被实施为硬件，或者被实施为集成电路，如专用集成电路。这样的软件可以分布在计算机可读介质上，计算机可读介质可以包括计算机存储介质(或非暂时性介质)和通信介质(或暂时性介质)。如本领域普通技术人员公知的，术语计算机存储介质包括在用于存储信息(诸如计算机可读指令、数据结构、程序模块或其他数据)的任何方法或技术中实施的易失性和非易失性、可移除和不可移除介质。计算机存储介质包括但不限于RAM、ROM、EEPROM、闪存或其他存储器技术、CD-ROM、数字多功能盘(DVD)或其他光盘存储、磁盒、磁带、磁盘存储或其他磁存储装置、或者可以用于存储期望的信息并且可以被计算机访问的任何其他的介质。此外，本领域普通技术人员公知的是，通信介质通常包含计算机可读指令、数据结构、程序模块或者诸如载波或其他传输机制之类的调制数据信号中的其他数据，并且可包括任何信息递送介质。Those skilled in the art can understand that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware and an appropriate combination thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit . Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). As known to those of ordinary skill in the art, the term computer storage media includes both volatile and nonvolatile media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. permanent, removable and non-removable media. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cartridges, tape, magnetic disk storage or other magnetic storage devices, or can Any other medium used to store desired information and which can be accessed by a computer. In addition, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media .

以上是对本申请的若干实施进行了具体说明，但本申请并不局限于上述实施方式，熟悉本领域的技术人员在不违背本申请本质的前提下还可作出种种的等同变形或替换，这些等同的变形或替换均包含在本申请权利要求所限定的范围内。The above is a specific description of several implementations of the present application, but the present application is not limited to the above-mentioned embodiments. Those skilled in the art can also make various equivalent deformations or replacements without violating the essence of the present application. Any modification or substitution is included within the scope defined by the claims of the present application.

Claims

一种图像处理方法，包括：An image processing method, comprising:

获取待处理图像；Get the image to be processed;

获取所述待处理图像中像素的特征向量；Acquiring feature vectors of pixels in the image to be processed;

获取预先训练得到的均匀像素特征向量集合；Obtain a set of uniform pixel feature vectors obtained from pre-training;

根据所述均匀像素特征向量集合与所述像素的特征向量，得到所述像素的向量均匀度；Obtaining the vector uniformity of the pixel according to the set of uniform pixel feature vectors and the feature vector of the pixel;

根据所述像素的向量均匀度，得到所述待处理图像的均匀度热力图。According to the vector uniformity of the pixels, a uniformity heat map of the image to be processed is obtained.
根据权利要求1所述的方法，还包括：The method according to claim 1, further comprising:

根据所述均匀度热力图和预设的非均匀度判断条件，在所述像素中确定非均匀像素；Determine non-uniform pixels among the pixels according to the uniformity heat map and preset non-uniformity judgment conditions;

对所述非均匀像素进行标识处理。Perform identification processing on the non-uniform pixels.
根据权利要求1所述的方法，其中，所述根据所述均匀像素特征向量集合与所述像素的特征向量，得到所述像素的向量均匀度，包括：The method according to claim 1, wherein the obtaining the vector uniformity of the pixel according to the set of uniform pixel feature vectors and the feature vector of the pixel comprises:

根据所述像素的特征向量与所述均匀像素特征向量集合中的各个均匀像素特征向量，计算得到所述像素的向量均匀度集合，所述向量均匀度集合包括向量均匀度元素；Calculate and obtain a vector uniformity set of the pixel according to the feature vector of the pixel and each uniform pixel feature vector in the uniform pixel feature vector set, and the vector uniformity set includes vector uniformity elements;

将所述像素的所述向量均匀度集合中数值最大的向量均匀度元素，确定为所述像素的向量均匀度。Determine the vector uniformity element with the largest numerical value in the vector uniformity set of the pixel as the vector uniformity of the pixel.
根据权利要求2所述的方法，其中，所述非均匀度判断条件包括：非均匀的像素对应的向量均匀度小于预设像素均匀度阈值；The method according to claim 2, wherein the non-uniformity judgment condition includes: the vector uniformity corresponding to the non-uniform pixel is less than a preset pixel uniformity threshold;

所述根据所述均匀度热力图和预设的非均匀度判断条件，在所述像素中确定非均匀像素，包括：The determination of non-uniform pixels in the pixels according to the uniformity heat map and the preset non-uniformity judgment conditions includes:

将所述像素的所述向量均匀度与所述预设像素均匀度阈值进行比较；comparing the vector uniformity of the pixel with the preset pixel uniformity threshold;

将数值小于所述预设像素均匀度阈值的向量均匀度确定为目标向量均匀度；Determining a vector uniformity whose value is smaller than the preset pixel uniformity threshold as the target vector uniformity;

将所述目标向量均匀度对应的像素确定为非均匀像素。Determine the pixels corresponding to the uniformity of the target vector as non-uniform pixels.
根据权利要求2所述的方法，其中，所述对所述非均匀像素进行标识处理，包括：The method according to claim 2, wherein said identifying said non-uniform pixels comprises:

判断是否存在非均匀区域，其中，所述非均匀区域包括数量大于预设像素数量阈值的所述非均匀像素；judging whether there is a non-uniform area, wherein the non-uniform area includes the non-uniform pixels whose number is greater than a preset pixel number threshold;

当判断存在所述非均匀区域，对所述非均匀区域进行标识处理。When it is determined that the non-uniform area exists, the non-uniform area is identified.
根据权利要求5所述的方法，其中，所述对所述非均匀区域进行标识处理，包括：The method according to claim 5, wherein the identifying the non-uniform region comprises:

将所述非均匀像素进行合并处理，得到非均匀区域；Merging the non-uniform pixels to obtain a non-uniform area;

在所述待处理图像上突出显示所述非均匀区域的轮廓。The outline of the non-uniform region is highlighted on the image to be processed.
根据权利要求2所述的方法，其中，所述对所述非均匀像素进行标识处理，包括：The method according to claim 2, wherein said identifying said non-uniform pixels comprises:

在所述待处理图像上突出显示所述非均匀像素。The non-uniform pixels are highlighted on the image to be processed.
根据权利要求1所述的方法，其中，所述均匀像素特征向量集合由以下步骤得到：The method according to claim 1, wherein, the set of uniform pixel feature vectors is obtained by the following steps:

获取训练样本，所述训练样本为均匀图像样本；Obtain a training sample, the training sample is a uniform image sample;

将所述训练样本输入至所述图像特征提取模型，得到样本均匀像素特征向量集合；Inputting the training samples into the image feature extraction model to obtain a set of sample uniform pixel feature vectors;

对所述样本均匀像素特征向量集合中的元素进行聚类处理，得到所述均匀像素特征向量集合。Clustering is performed on the elements in the set of sample uniform pixel feature vectors to obtain the set of uniform pixel feature vectors.
根据权利要求8所述的方法，其中，所述图像特征提取模型包括第一输出层、第二输出层和第三输出层；The method according to claim 8, wherein said image feature extraction model comprises a first output layer, a second output layer and a third output layer;

所述将所述训练样本输入至所述图像特征提取模型，得到样本均匀像素特征向量集合，包括：Said inputting said training sample into said image feature extraction model to obtain sample uniform pixel feature vector set, including:

将所述训练样本输入至所述图像特征提取模型，得到所述第一输出层输出的第一特征图、所述第二输出层输出的第二特征图以及所述第三输出层输出的第三特征图；The training sample is input to the image feature extraction model to obtain the first feature map output by the first output layer, the second feature map output by the second output layer, and the first feature map output by the third output layer. Three feature maps;

对所述第二特征图进行上采样，得到第二上采样特征图；Upsampling the second feature map to obtain a second upsampling feature map;

对所述第三特征图进行上采样，得到第三上采样特征图；Upsampling the third feature map to obtain a third upsampling feature map;

将所述第一特征图与所述第二上采样特征图、第三上采样特征图进行通道合并，得到聚合特征图；channel-merging the first feature map, the second upsampled feature map, and the third upsampled feature map to obtain an aggregated feature map;

根据所述训练样本的图像分辨率，对所述聚合特征图进行上采样，得到聚合后的特征向量集合；Upsampling the aggregated feature map according to the image resolution of the training sample to obtain an aggregated feature vector set;

将所述聚合后的特征向量集合确定为所述样本均匀像素特征向量集合。The aggregated feature vector set is determined as the sample uniform pixel feature vector set.
根据权利要求8所述的方法，其中，所述对所述样本均匀像素特征向量集合中的元素进行聚类处理，得到所述均匀像素特征向量集合，包括：The method according to claim 8, wherein said clustering the elements in the set of uniform pixel feature vectors of the samples to obtain the set of uniform pixel feature vectors comprises:

根据预设的聚类特征向量维度，对所述样本均匀像素特征向量集合中的元素进行特征聚类，得到聚类后的特征向量集合；According to the preset clustering feature vector dimension, perform feature clustering on the elements in the sample uniform pixel feature vector set to obtain the clustered feature vector set;

将所述聚类后的特征向量集合确定为所述样本均匀像素特征向量集合。The clustered feature vector set is determined as the sample uniform pixel feature vector set.
一种电子设备，包括：存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，其中，所述处理器执行所述计算机程序时实现如权利要求1至10任一项所述的图像处理方法。An electronic device, comprising: a memory, a processor, and a computer program stored on the memory and operable on the processor, wherein, when the processor executes the computer program, the computer program described in any one of claims 1 to 10 is implemented. The image processing method described above.
一种计算机可读存储介质，存储有计算机可执行指令，所述计算机可执行指令用于执行权利要求1至10任一项所述的图像处理方法。A computer-readable storage medium storing computer-executable instructions for executing the image processing method according to any one of claims 1 to 10.
一种计算机程序产品，包括计算机程序或计算机指令，其中，所述计算机程序或所述计算机指令存储在计算机可读存储介质中，计算机设备的处理器从所述计算机可读存储介质读取所述计算机程序或所述计算机指令，所述处理器执行所述计算机程序或所述计算机指令，使得所述计算机设备执行如权利要求1至10任一项所述的图像处理方法。A computer program product comprising a computer program or computer instructions, wherein the computer program or the computer instructions are stored in a computer-readable storage medium, and a processor of a computer device reads the computer-readable storage medium from the computer-readable storage medium A computer program or the computer instruction, the processor executes the computer program or the computer instruction, so that the computer device executes the image processing method according to any one of claims 1 to 10.