CN112733895A

CN112733895A - Method and device for determining image similarity and computer storage medium

Info

Publication number: CN112733895A
Application number: CN202011605383.7A
Authority: CN
Inventors: 薛卓阳
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2020-12-30
Filing date: 2020-12-30
Publication date: 2021-04-30
Anticipated expiration: 2040-12-30
Also published as: CN112733895B

Abstract

The embodiment of the application discloses a method and a device for determining image similarity and a computer storage medium, and belongs to the technical field of image processing. The method comprises the following steps: and determining the similarity between the first image and the second image according to the obtained gradient feature vector of the first image and the obtained gradient feature vector of the second image. The gradient feature vector of the first image and the gradient feature vector of the second image both indicate the pixel gradient distribution condition of pixel points in the corresponding image on a first-order gradient, or indicate the pixel gradient distribution condition of pixel points in the corresponding image on each step from the first-order gradient to a k-order gradient. According to the method and the device, the similarity between the first image and the second image is determined through the gradient feature vector of the first image and the gradient feature vector of the second image, and the stability and the timeliness of the computer in determining the similarity are guaranteed.

Description

Method and device for determining image similarity and computer storage medium

Technical Field

The embodiment of the application relates to the technical field of image processing, in particular to a method and a device for determining image similarity and a computer storage medium.

Background

The image processing technology is a technology that performs operations such as statistics, analysis, and processing on an image, thereby realizing functions such as transferring an image and searching for an image in a computer environment. The image processing technology can realize the function of searching images by images, and the searching images by images are widely applied to aspects of big data image retrieval, video inspection, shopping search and the like. To realize the function of searching images by using the image processing technology of the computer, the similarity between the known image and the image to be searched needs to be determined.

In the related art, when a computer measures similarity between a known image and an image to be searched, first, the computer needs to perform feature extraction on the known image and the image to be searched. And extracting the color, texture, shape and other characteristics of the known image and the image to be searched by the computer to obtain the characteristic information of the known image and the image to be searched. After the computer extracts the features of the known image and the image to be searched, the computer determines the similarity of the known image and the image to be searched by using the extracted feature information. Determining a cosine value of an included angle between the feature vector in the feature information of the known image and the feature vector in the feature information of the image to be searched, and judging the similarity of the known image and the image to be searched according to the size of the cosine value of the included angle between the feature vector in the feature information of the known image and the feature vector in the feature information of the image to be searched.

In the above technology, when extracting features of a known image and an image to be searched, for an image with features such as obvious color contrast, prominent texture feature, and complex shape, the extraction process of the features by a computer is complex. And when the similarity between the known image and the image to be searched is determined based on the method, the requirement on the performance of the computer is high. Therefore, when the computer searches the map, the stability and timeliness of the searching process are often difficult to guarantee.

Disclosure of Invention

The embodiment of the application provides a method and a device for determining image similarity and a computer storage medium, which can ensure the stability and effectiveness of a computer in determining the similarity. The technical scheme is as follows:

in a first aspect, a method for determining image similarity is provided, the method including:

acquiring a gradient feature vector of a first image and a gradient feature vector of a second image, wherein the gradient feature vector of the first image and the gradient feature vector of the second image both indicate the pixel gradient distribution condition of pixel points in corresponding images on a first-order gradient or indicate the pixel gradient distribution condition of pixel points in corresponding images on each step from the first-order gradient to a k-order gradient;

determining a similarity between the first image and the second image based on the gradient feature vector of the first image and the gradient feature vector of the second image.

Optionally, in a case that the gradient feature vector of the first image and the gradient feature vector of the second image both indicate pixel gradient distribution conditions of pixel points in corresponding images on a first-order gradient, the gradient feature vector of the first image includes M-dimensional feature values corresponding to M first-order gradient direction intervals, and N-dimensional feature values corresponding to N first-order gradient magnitude intervals, the gradient feature vector of the second image includes M-dimensional feature values corresponding to M first-order gradient direction intervals, and N-dimensional feature values corresponding to N first-order gradient magnitude intervals, where M and N are positive integers greater than 1;

the gradient feature vector of the first image comprises any one of M-dimensional feature values indicating the number of pixel points in the first image, wherein the first-order gradient direction of the pixel points is located in a corresponding gradient direction interval, and the gradient feature vector of the first image comprises any one of N-dimensional feature values indicating the number of pixel points in the first image, wherein the first-order gradient amplitude of the pixel points is located in a corresponding gradient amplitude interval;

any one of the M-dimensional feature values included in the gradient feature vector of the second image indicates the number of pixel points in the second image in which the first-order gradient direction is located in the corresponding gradient direction interval, and any one of the N-dimensional feature values included in the gradient feature vector of the second image indicates the number of pixel points in the second image in which the first-order gradient amplitude is located in the corresponding gradient amplitude interval.

Optionally, in a case that the gradient feature vector of the first image and the gradient feature vector of the second image both indicate pixel gradient distribution conditions of pixel points in corresponding images at each step from a first-order gradient to a k-order gradient, the gradient feature vector of the first image includes P-dimensional feature values corresponding to P i-step direction intervals, and Q-dimensional feature values corresponding to Q i-step amplitude intervals, the gradient feature vector of the second image includes P-dimensional feature values corresponding to P i-step direction intervals, and Q-dimensional feature values corresponding to Q i-step amplitude intervals, P and Q are positive integers greater than 1, and i is a positive integer greater than or equal to 1 and less than or equal to k;

any one of P-dimensional characteristic values included in the gradient characteristic vector of the first image indicates the number of pixel points in the first image, wherein the i-step gradient direction of the first image is located in a corresponding gradient direction interval, and any one of Q-dimensional characteristic values included in the gradient characteristic vector of the first image indicates the number of pixel points in the first image, wherein the i-step gradient amplitude of the first image is located in a corresponding gradient amplitude interval;

any one of the P-dimensional feature values included in the gradient feature vector of the second image indicates the number of pixel points in the second image, wherein the i-step gradient direction of the pixel points is located in the corresponding gradient direction interval, and any one of the Q-dimensional feature values included in the gradient feature vector of the second image indicates the number of pixel points in the second image, wherein the i-step gradient amplitude of the pixel points is located in the corresponding gradient amplitude interval.

Optionally, the first image includes a plurality of sub-images, and the gradient feature vector of the first image is obtained by superimposing the gradient feature vectors of each of the plurality of sub-images in the first image;

the second image comprises a plurality of sub-images, and the gradient feature vector of the second image is obtained by superposing the gradient feature vector of each sub-image in the plurality of sub-images in the second image;

the number of the sub-images included in the first image is the same as that of the sub-images included in the second image, the size of the sub-images included in the first image is the same as that of the sub-images included in the second image, and the gradient feature vector of each sub-image in the plurality of sub-images in the first image and the gradient feature vector of each sub-image in the plurality of sub-images in the second image indicate the pixel gradient distribution condition of the pixel points in the corresponding sub-image on the first-order gradient or indicate the pixel gradient distribution condition of the pixel points in the corresponding sub-image on each step from the first-order gradient to the k-order gradient.

Optionally, before the obtaining the gradient feature vector of the first image and the gradient feature vector of the second image, the method further includes:

compressing the first picture into a picture of a reference size, the reference size being the size of the second picture;

and acquiring a gradient feature vector of the image with the reference size, and taking the gradient feature vector of the image with the reference size as the gradient feature vector of the first image.

Optionally, the method further comprises:

in response to the detected search instruction, acquiring a second image with the largest similarity with the first image or acquiring a second image with the similarity with the first image exceeding a similarity threshold from the plurality of second images, and taking the acquired second image as a search result for the search instruction;

wherein the search instruction instructs to query the image associated with the first image, the similarity between the first image and each second image is determined based on the gradient feature vector of the first image and the gradient feature vector of each second image, and the gradient feature vector of the first image and the gradient feature vector of each second image both indicate the pixel gradient distribution condition of the pixel points in the corresponding image on the first order gradient or both indicate the pixel gradient distribution condition of the pixel points in the corresponding image on each step from the first order gradient to the k order gradient.

In a second aspect, an apparatus for determining image similarity is provided, the apparatus comprising:

the acquisition module is used for acquiring a gradient feature vector of a first image and a gradient feature vector of a second image, wherein the gradient feature vector of the first image and the gradient feature vector of the second image both indicate the pixel gradient distribution condition of pixel points in corresponding images on a first-order gradient or indicate the pixel gradient distribution condition of pixel points in corresponding images on each step from the first-order gradient to a k-order gradient;

a determining module for determining a similarity between the first image and the second image based on the gradient feature vector of the first image and the gradient feature vector of the second image.

Optionally, the gradient feature vector of the first image and the gradient feature vector of the second image both indicate pixel gradient distribution conditions of pixel points in the corresponding images on a first-order gradient;

the gradient feature vector of the first image comprises M-dimensional feature values corresponding to M first-order gradient direction intervals respectively and N-dimensional feature values corresponding to N first-order gradient amplitude intervals respectively, the gradient feature vector of the second image comprises M-dimensional feature values corresponding to M first-order gradient direction intervals respectively and N-dimensional feature values corresponding to N first-order gradient amplitude intervals respectively, and M and N are positive integers greater than 1;

the gradient feature vector of the first image comprises any one of M-dimensional feature values indicating the number of pixels in the first image in which a first-order gradient direction is located in a corresponding gradient direction interval, the gradient feature vector of the first image comprises any one of N-dimensional feature values indicating the number of pixels in the first image in which a first-order gradient amplitude is located in a corresponding gradient amplitude interval, the gradient feature vector of the second image comprises any one of M-dimensional feature values indicating the number of pixels in the second image in which a first-order gradient direction is located in a corresponding gradient direction interval, and the gradient feature vector of the second image comprises any one of N-dimensional feature values indicating the number of pixels in the second image in which a first-order gradient amplitude is located in a corresponding gradient amplitude interval.

Optionally, the apparatus further comprises a compression module for compressing the first picture into a picture of a reference size, the reference size being the size of the second picture;

the obtaining module is further configured to obtain a gradient feature vector of the image with the reference size, and use the gradient feature vector of the image with the reference size as the gradient feature vector of the first image.

Optionally, the apparatus further comprises:

the obtaining module is used for responding to the detected search instruction, obtaining a second image with the maximum similarity with the first image from the plurality of second images, or obtaining a second image with the similarity with the first image exceeding a similarity threshold value, and taking the obtained second image as a search result aiming at the search instruction;

In a third aspect, there is provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the method for determining image similarity as described in the first aspect above.

In a fourth aspect, an apparatus for determining image similarity is provided, the apparatus comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to perform a method of determining image similarity as described in the first aspect above.

In a fifth aspect, a computer-readable storage medium is provided, the computer-readable storage medium having stored thereon instructions, which when executed by a processor, implement a method for determining image similarity according to the first aspect.

The beneficial effects brought by the technical scheme provided by the embodiment of the application at least comprise:

and determining the similarity between the first image and the second image through the acquired gradient feature vector of the first image and the acquired gradient feature vector of the second image. The gradient feature vector is a distribution of pixel gradients of the images, and therefore the gradient feature vector of the first image and the gradient feature vector of the second image respectively represent feature information of the first image and the second image. Because the complexity of the determination process and the acquisition process for determining the distribution condition of the pixel gradient of the image is small, the complexity of the computer for determining the similarity between the first image and the second image is relatively small, and the requirement on the performance of the computer is low. In addition, since the complexity of the computer for determining the similarity between the first image and the second image is small, the stability and the effectiveness of the computer for determining the similarity between the first image and the second image are ensured.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic architecture diagram of an embedded system according to an embodiment of the present application.

Fig. 2 is a flowchart of a method for determining image similarity according to an embodiment of the present disclosure.

Fig. 3 is a schematic structural diagram of an apparatus for determining image similarity according to an embodiment of the present application.

Fig. 4 is a block diagram of a terminal 400 according to an embodiment of the present disclosure.

Fig. 5 is a schematic structural diagram of a server according to an embodiment of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the embodiments of the present application more clear, the embodiments of the present application will be further described in detail with reference to the accompanying drawings.

For convenience of description, an application scenario of the embodiment of the present application is described first.

The image searching method comprises the steps of inputting a first image into a computer, determining the similarity of the first image and a known image in the computer to obtain a second image with high similarity to the first image, and accordingly achieving the purpose of searching images through images.

The method provided by the embodiment of the application is applied to the scene of searching the images by the images. Optionally, the method provided by the embodiment of the present application is also used in any other scene in which the similarity between two images needs to be determined. Such as face recognition, are not illustrated here.

In order to implement a method for determining image similarity, an embodiment of the present application provides an embedded system. For the convenience of the following description, the embedded system will be explained in detail.

Fig. 1 is a schematic architecture diagram of an embedded system according to an embodiment of the present application. As shown in fig. 1, the embedded system 100 includes an image acquisition platform 101, an image transmission module 102, and an image analysis processing platform 103.

The image acquisition platform is used for acquiring images or videos and preprocessing the images or videos. That is, the image capture platform is used for image or video entry and pre-processing. The image transmission module is used for converting the image or video which is preprocessed by the image acquisition platform into an image signal and transmitting the image signal to the image analysis processing platform. The image analysis processing platform is used for counting, analyzing and storing the image signals.

In a possible implementation manner, the image capturing platform captures an image or a video by using a camera, and performs preprocessing such as sharpening and denoising on the captured image or video to obtain a preprocessed image or video. After the image acquisition platform finishes preprocessing the image or the video, the preprocessed image or the preprocessed video is sent to the image transmission module. After receiving the image or the video processed by the image acquisition platform, the image transmission module transmits the image to the image analysis processing platform or extracts a certain frame in the video, and then transmits the image corresponding to the extracted certain frame to the image analysis processing platform. After receiving the image transmitted by the image processing module, the image analysis processing platform carries out structuring processing on the image, analyzes each characteristic of the image, stores each characteristic of the image and the image, and further can carry out similarity comparison by utilizing the stored image and the known image, and carries out similarity comparison between each characteristic of the stored image and each characteristic of the known image, thereby realizing the function of searching the image.

The embedded system shown in fig. 1 is a software system, and each module in the embedded system can be deployed in any hardware device. For example, the embedded system may be deployed on a terminal, and at this time, the terminal implements the method for determining image similarity provided in the embodiment of the present application. Alternatively, the embedded system may also be deployed on a server, and at this time, the server implements the method for determining image similarity provided in the embodiment of the present application.

The embedded system is deployed in an embedded device, and when the embedded device is a portable small device such as a mobile phone, the embedded system is also called a miniaturized embedded system. At present, miniaturized embedded devices are more and more widely used, and the advantages of portability and miniaturization are favored in various fields. Miniaturized embedded devices have also found wide application in the field of image or video acquisition and analysis.

The method provided by the embodiment of the present application is further explained below based on the embedded system shown in fig. 1. Fig. 2 is a flowchart of a method for determining image similarity according to an embodiment of the present disclosure, where the method for determining image similarity may include the following steps.

Step 201: the embedded system obtains a gradient feature vector of the first image and a gradient feature vector of the second image.

In order to acquire the gradient feature vector of the first image and the gradient feature vector of the second image, the first image and the second image need to be acquired first. In an application scenario of searching the image, the first image is represented as an image to be searched, and the second image is represented as a known image. Optionally, the first image is represented as a known image, and the second image is represented as an image to be searched. The following embodiments are described by taking the first image as the image to be searched and the second image as the known image when the application scene of searching the image is referred to.

The embedded system obtains the first image in the following way: and the user uploads the first image to the embedded system, and the embedded system acquires the first image uploaded by the user.

The embedded system obtains the second image in the following way: in one possible implementation, a search control is displayed on a hardware device on which the embedded system is deployed. A user triggers a search control displayed on hardware equipment through preset operation, and the hardware equipment detects a search instruction and sends the search instruction to an embedded system. The embedded system receives the search instruction and acquires a plurality of second images in response to the detected search instruction. Wherein the search instruction indicates to query for images associated with the first image.

The plurality of acquired second images can be acquired based on the internet or acquired based on a second image library stored in the embedded system.

The image associated with the first image may be an image of the same category as the object appearing in the first image, for example, if the object appearing in the first image is a cat, the image associated with the first image is an image of the cat. The image having the same source as the first image may be the image, and how to determine the image associated with the first image is not limited herein, and the image associated with the first image may be specified by the user.

The image searching control displayed on the hardware equipment with the embedded system may be displayed on the hardware equipment with the embedded system after the user uploads the first image, or displayed on the hardware equipment with the embedded system before the user uploads the first image.

The embedded system obtains the second image in the following way: in another possible implementation manner, after receiving a first image uploaded by a user, the embedded system automatically starts a picture searching function, and responds to a detected picture searching instruction to acquire a plurality of second images.

In the embodiment of the application, for more stable and rapid determination of the image similarity, the gradient feature vector is adopted as a feature vector required for subsequent determination of the image similarity. Because the image is composed of a plurality of pixel points, and each pixel point corresponds to the gradient of the pixel, when the gradient feature vector of the image is obtained based on the pixel point of the image, compared with the feature vector extracted by the features of obvious color contrast, prominent texture feature, complex shape and the like in the related technology, the complexity of the process is low, and the image similarity can be stably and quickly determined.

Thus, the embedded system obtains the gradient feature vector of the first image and the gradient feature vector of the second image. And for each second image in the plurality of second images, respectively executing the operation of acquiring the gradient feature vector of the first image and the gradient feature vector of the second image. The gradient feature vector of the first image and the gradient feature vector of the second image both indicate the pixel gradient distribution condition of pixel points in the corresponding image on a first-order gradient, or indicate the pixel gradient distribution condition of pixel points in the corresponding image on each step from the first-order gradient to a k-order gradient. The gradient feature vector of the first image and the gradient feature vector of the second image both indicate the pixel gradient distribution condition of pixel points in the corresponding image on each step from the first-order gradient to the k-order gradient, and represent that the gradient feature vector of the first image and the gradient feature vector of the second image are both multi-order gradient feature vectors formed by connecting feature vectors of each step from the first-order gradient to the k-order gradient in series. The concatenated multi-order gradients have k gradient eigenvectors. k is a positive integer greater than 1.

It should be noted that the number and accuracy of the features of the image described by the first-order gradient feature vector are smaller than those of the image described by the multi-order gradient feature vector, but the acquisition process of the first-order gradient feature vector is more and the acquisition process of the gradient feature vector is simple. Whether the obtained gradient feature vector is a first-order gradient feature vector or a multi-order gradient feature vector needs to be determined according to actual conditions.

The gradient feature vector is determined based on the gradient of the image pixel points, the gradient of the image pixel points comprises a gradient direction and a gradient amplitude, the gradient direction indicates the direction in which the pixel value of each pixel point in the image changes most quickly, and the gradient amplitude indicates the intensity of the pixel value of each pixel point in the image. The pixel value change of each pixel point in the image reflects the texture and shape characteristics of the image, and the intensity of the pixel value of each pixel point in the image reflects the color characteristics of the image. Based on the gradient of the image pixel points, the characteristics of the image such as color, texture and shape can be obtained, and compared with the method for directly obtaining the characteristics of the image such as color, texture and shape, the method is more accurate and the obtained characteristics are more accurate.

The pixel gradient distribution condition is expressed as the distribution condition of the gradient direction and the gradient amplitude of all pixel points in the corresponding image.

In an implementation manner, in a case that the gradient feature vector of the first image and the gradient feature vector of the second image indicate pixel gradient distribution conditions of pixel points in the corresponding images on a first-order gradient, the gradient feature vector of the first image includes M-dimensional feature values corresponding to M first-order gradient direction intervals, and N-dimensional feature values corresponding to N first-order gradient amplitude intervals, the gradient feature vector of the second image includes M-dimensional feature values corresponding to M first-order gradient direction intervals, and N-dimensional feature values corresponding to N first-order gradient amplitude intervals, where M and N are positive integers greater than 1.

Any one of the M-dimensional characteristic values included in the gradient characteristic vector of the first image indicates the number of pixel points in the first-order gradient direction of the first image, which are located in the corresponding gradient direction interval. Any one of the N-dimensional feature values included in the gradient feature vector of the first image indicates the number of pixel points in the first image in which the first-order gradient magnitude is located in the corresponding gradient magnitude interval. Any one of the M-dimensional feature values included in the gradient feature vector of the second image indicates the number of pixel points in the second image, in which the first-order gradient direction is located in the corresponding gradient direction interval, and any one of the N-dimensional feature values included in the gradient feature vector of the second image indicates the number of pixel points in the second image, in which the first-order gradient amplitude is located in the corresponding gradient amplitude interval.

Since the method for obtaining the gradient feature vector of the first image is the same as that of the gradient feature vector of the second image under the condition that the gradient feature vector of the first image and the gradient feature vector of the second image indicate the pixel gradient distribution condition of the pixel points in the corresponding images on the first-order gradient, taking the first image as an example, how to obtain the gradient feature vector of the first image is explained, and how to obtain the gradient feature vector of the second image will not be explained subsequently.

In the first image, in order to facilitate statistics of the number of gradient directions of the pixel points, the first-order gradient direction in the first image is divided into M first-order gradient direction intervals in advance. In this case, in a possible implementation manner, the obtaining process of the M-dimensional eigenvalues corresponding to the M first-order gradient direction intervals is as follows: counting the number of the first-order gradient directions of all pixel points in the first image in each of M first-order gradient direction intervals, taking the number of the first-order gradient directions in each first-order gradient direction interval as a characteristic value, and obtaining an M-dimensional characteristic value in the M first-order gradient direction intervals.

In addition, the dividing of the gradient direction in the first image into M first-order gradient direction intervals may be dividing all possible gradient directions into M equal parts, so as to obtain M first-order gradient direction intervals.

For example, the first-order gradient direction of the first image is divided into 6 first-order gradient direction intervals in the 360 ° direction, the first interval is 0 ° to 60 °, the second interval is 60 ° to 120 °, the second interval is 120 ° to 180 °, the second interval is 180 ° to 240 °, the second interval is 240 ° to 300 °, and the second interval is 300 ° to 360 °. Then, the number of the first-order gradient directions of the pixel points of the first image appearing in the first-order gradient direction interval in each first-order gradient direction interval in the 6 first-order gradient direction intervals is counted, and then the characteristic value corresponding to the first-order gradient direction interval is obtained.

It should be noted that, in practical applications, it is best to divide the first-order gradient direction into 16 equal parts, but the embodiment of the present application does not limit how many equal parts the first-order gradient direction is divided into.

In the first image, in order to facilitate statistics of the number of gradient amplitudes of the pixel points, the first-order gradient amplitudes in the first image are divided into N first-order gradient amplitude intervals with a step length of C in advance. In this case, in a possible implementation manner, the implementation manner of the N-dimensional eigenvalues corresponding to the N first-order gradient amplitude intervals includes: counting the number of the first-order gradient amplitudes of all pixel points in the first image in each of N first-order gradient amplitude intervals, taking the number of the first-order gradient amplitudes in each first-order gradient amplitude interval as a characteristic value, and obtaining N-dimensional characteristic values in the N first-order gradient amplitude intervals.

For example, the first order gradient magnitude of the first image is divided into [0, C ], [ C, 2C ], [2C, 3C ], [3C, 4C) … (N × C, + ∞) intervals. Then, the number of the first-order gradient amplitudes of the pixel points of the first image appearing in each first-order gradient amplitude interval in the N first-order gradient amplitude intervals is counted, and the characteristic value corresponding to the first-order gradient amplitude interval is obtained.

After obtaining the M-dimensional feature value of the first image in the first-order gradient direction and the N-dimensional feature value of the first-order gradient magnitude, the computer takes the M-dimensional feature value of the first-order gradient direction of the first image as an M-dimensional feature vector of the first-order gradient direction of the first image, and the M-dimensional feature vector of the first-order gradient direction is denoted as R1. The N-dimensional feature value of the first order gradient magnitude of the first image is taken as the N-dimensional feature vector of the first order gradient magnitude of the first image, and the N-dimensional feature vector of the first order gradient direction is denoted as R2.

In order to facilitate subsequent similarity determination, normalization processing may be performed on the M-dimensional feature vector R1 in the first-order gradient direction of the first image and the N-dimensional feature vector R2 in the first-order gradient amplitude of the first image, so as to obtain a first-order gradient feature vector R of the first image with dimension M + N. The specific calculation formula is as follows:

alternatively, the M-dimensional feature vector R1 in the first-order gradient direction of the first image and the N-dimensional feature vector R2 in the first-order gradient magnitude of the first image may be directly merged without other processing, and the merged vector may be referred to as the first-order gradient feature vector R of the first image, where the first-order gradient feature vector R of the first image may be represented as R ═ R1, R2.

In addition, in a possible implementation manner, the first-order gradient direction and the first-order gradient magnitude of the first image are implemented by: and determining the first order gradient direction and the first order gradient magnitude of the first image by using a Sobel operator.

Illustratively, the pixel value of any pixel point in the first image is expressed as a function f (x, y), (x, y) representing the position coordinate of the pixel point, and f (x, y) representing the pixel value at the pixel point. Therefore, the position coordinates of any pixel (x, y) in the first image and the surrounding eight pixels can be shown in table 1.

TABLE 1

(x-1,y-1)	(x,y-1)	(x+1,y-1)
			(x-1,y)	(x，y)	(x+1,y)
(x-1,y+1)	(x,y+1)	(x+1,y+1)

The Sobel operator includes two sets of 3 × 3 matrices, and the matrix of the Sobel operator in the x direction is Gx1, as shown in table 2.

TABLE 2

-1	0	+1
			-2	0	+2
-1	0	+1

The matrix of the Sobel operator in the y direction is Gy1, as shown in table 3.

TABLE 3

-1	+2	+2
			0	0	0
-1	-2	-1

And multiplying f (x, y) by Gx1 and Gy1 respectively to obtain Gx and Gy, wherein Gx represents the pixel value difference value of the pixel point (x, y) in the transverse direction, and Gy represents the pixel value difference value of the first image in the longitudinal direction. Therefore, the first-order gradient direction of the pixel point (x, y) is θ ═ arctan (Gx/Gy), and the first-order gradient amplitude is

Optionally, the first-order gradient direction and the first-order gradient magnitude of the first image may also be implemented by using a Scharr (sarr) operator, a Laplacian operator, and the like, and the embodiment of the present application is not limited to how to determine the first-order gradient direction and the first-order gradient magnitude of the first image.

In another possible implementation manner, in a case that the gradient feature vector of the first image and the gradient feature vector of the second image indicate a pixel gradient distribution condition of a pixel point in a corresponding image at each step from a first-order gradient to a k-order gradient, the gradient feature vector of the first image includes P-dimensional feature values corresponding to P i-step direction intervals and Q-dimensional feature values corresponding to Q i-step amplitude intervals, the gradient feature vector of the second image includes P-dimensional feature values corresponding to P i-step direction intervals and Q-dimensional feature values corresponding to Q i-step amplitude intervals, P and Q are positive integers greater than 1, and i is a positive integer greater than or equal to 1 and less than or equal to k.

Any one of the P-dimensional characteristic values included in the gradient characteristic vector of the first image indicates the number of pixel points in the first image, wherein the i-step gradient direction of the pixel points is located in the corresponding gradient direction interval. Any one of the Q-dimensional feature values included in the gradient feature vector of the first image indicates the number of pixel points in the first image in which the i-step gradient magnitude is located in the corresponding gradient magnitude interval. Any one of P-dimensional characteristic values included in the gradient characteristic vector of the second image indicates the number of pixel points of the second image, wherein the i-step gradient direction of the second image is located in the corresponding gradient direction interval, and any one of Q-dimensional characteristic values included in the gradient characteristic vector of the second image indicates the number of pixel points of the second image, wherein the i-step gradient amplitude of the second image is located in the corresponding gradient amplitude interval.

Since the method for obtaining the gradient feature vector of the first image is the same as that of the gradient feature vector of the second image under the condition that the gradient feature vector of the first image and the gradient feature vector of the second image indicate the pixel gradient distribution condition of the pixel point in the corresponding image at each step from the first-order gradient to the k-order gradient, the method for obtaining the gradient feature vector of the first image is explained by taking the first image as an example, and the method for obtaining the gradient feature vector of the second image is not explained subsequently.

In the first image, in order to facilitate statistics of the number of gradient directions of the pixel points, i-step-degree directions in the first image are divided into P i-step-degree-direction intervals in advance. In this case, in a possible implementation manner, the process of obtaining the P-dimensional feature value corresponding to each of the P i-step direction intervals is as follows: counting the number of i-step-degree directions of all pixel points in the first image in each of P i-step-degree-direction intervals, taking the number of i-step-degree directions in each i-step-degree-direction interval as a characteristic value, and obtaining a P-dimensional characteristic value in each P i-step-degree-direction interval.

In addition, the above-mentioned dividing the gradient direction into P i gradient direction sections may be dividing all possible gradient directions equally into P equal parts, so as to obtain P i gradient direction sections.

It should be noted that, in practical applications, it is best to divide the i-step direction of the first image into 16 equal parts, but the embodiment of the present application does not limit how many equal parts the i-step direction of the first image is divided into.

In the first image, in order to facilitate statistics of the number of gradient amplitudes of the pixel points, i-step amplitude in the first image is divided into Q i-step amplitude intervals with a step length of T in advance. At this time, in a possible implementation manner, the process of obtaining the Q-dimensional characteristic values corresponding to the Q i-step amplitude intervals is as follows: counting the number of i-step amplitude values of all pixel points in the first image in each i-step amplitude value interval in Q i-step amplitude value intervals, taking the number of i-step amplitude values in each i-step amplitude value interval as a characteristic value, and obtaining a Q-dimensional characteristic value in each Q i-step amplitude value interval.

For example, the i-step amplitude of the image is divided into [0, T ], [ T, 2T ], [2T, 3T ], [3T, 4T) … (Q x T, + ∞) intervals. Then, the number of the first-order gradient amplitudes of the pixel points of the first image appearing in each first-order gradient amplitude interval in the Q first-order gradient amplitude intervals is counted, and the characteristic value corresponding to the first-order gradient amplitude interval is obtained.

After obtaining the P-dimensional feature value of the first image in the i-step degree direction and the Q-dimensional feature value of the i-step degree amplitude from the above-mentioned images, the computer takes the P-dimensional feature value of the i-step degree direction of the first image as the P-dimensional feature vector of the i-step degree direction of the first image, and the P-dimensional feature vector of the i-step degree direction is denoted as C1. The Q-dimensional feature value of the i-step gradient of the first image is taken as the Q-dimensional feature vector of the i-step gradient of the first image, and the Q-dimensional feature vector of the i-step gradient direction is denoted as C2.

In order to facilitate subsequent similarity determination, normalization processing is further performed on the i-dimensional feature vector C1 of the first image in the i-step degree direction and the Q-dimensional feature vector C2 of the i-step degree amplitude of the first image, so as to obtain a gradient feature vector C of the P + Q-dimensional first image. The specific calculation formula is as follows:

alternatively, the i-dimensional feature vector C1 in the i-step degree direction of the first image and the i-dimensional feature vector C2 in the i-step degree amplitude of the first image may be directly merged without other processing, and the merged vector may be used as the i-step degree feature vector C of the first image, where the i-step degree feature vector C of the first image may be represented as C ═ C1, C2.

In addition, the implementation manner of the i-step gradient direction and the i-step gradient amplitude of the first image may refer to the implementation manner of determining the first-order gradient direction and the first-order gradient amplitude based on the Sobel operator, and is not described herein again.

Optionally, the implementation manner of the i-step gradient direction and the i-step gradient amplitude of the first image may also be a Scharr operator, a Laplacian operator, and the like, and the embodiment of the present application is not limited to how to determine the i-step gradient direction and the i-step gradient amplitude of the first image.

In addition, in the first-order gradient eigenvector, the first-order gradient direction is divided into M first-order gradient direction intervals, and the first-order gradient amplitude is divided into N first-order gradient amplitude intervals. In the gradient feature vector from the first-order gradient to the k-order gradient, i-step gradient direction is divided into P i-step direction intervals, and i-step gradient amplitude is divided into Q i-step gradient amplitude intervals. In one possible implementation, the number of first-order gradient-direction intervals divided in the first-order gradient direction is the same as the number of i-step-degree-direction intervals divided in the i-step-degree direction, that is, M is equal to P. The number of i-step amplitude direction intervals divided by the i-step amplitude is the same as the number of i-step amplitude intervals divided by the i-step amplitude, that is, N is equal to Q. In another possible implementation manner, the number of first-order gradient direction intervals divided in the first-order gradient direction is different from the number of i-step gradient direction intervals divided in the i-step gradient direction, that is, M is not equal to P. The number of i-step amplitude intervals divided by the i-step amplitude is different from the number of i-step amplitude intervals divided by the i-step amplitude, that is, N is not equal to Q.

In addition, in a possible implementation manner, the first image comprises a plurality of sub-images, and the gradient feature vector of the first image is obtained by superposing the gradient feature vector of each sub-image in the plurality of sub-images in the first image. The second image comprises a plurality of sub-images, and the gradient feature vector of the second image is obtained by superposing the gradient feature vectors of each sub-image in the plurality of sub-images in the second image. The number of the sub-images included in the first image is the same as that of the sub-images included in the second image, the size of the sub-images included in the first image is the same as that of the sub-images included in the second image, and the gradient feature vector of each sub-image in the plurality of sub-images in the first image and the gradient feature vector of each sub-image in the plurality of sub-images in the second image indicate the pixel gradient distribution condition of the pixel points in the corresponding sub-image on the first-order gradient or indicate the pixel gradient distribution condition of the pixel points in the corresponding sub-image on each step from the first-order gradient to the k-order gradient. Here, the explanation is also given taking the first image as an example.

In the first image, it should be noted that, when the computer performs gradient feature vector calculation on the first image, the computer may simultaneously determine the gradient feature vector of each sub-image, and merge the simultaneously determined gradient feature vectors of the sub-images of the image partition to obtain the gradient feature vector of the first image. That is, the first image may be divided into a plurality of sub-images in advance, then the plurality of sub-images are processed in parallel to obtain the gradient feature vector of each sub-image, and then the gradient feature vectors of all the sub-images are combined to obtain the gradient feature vector of the first image, so that the processing speed of the computer is increased, that is, the timeliness of the subsequent determination of the similarity is improved.

For example, the first image is divided into 8 × 12 sub-images in advance, and then the gradient feature vectors in the 8 × 12 sub-images may be determined in parallel, and then the gradient feature vectors of all the sub-images are combined to obtain the gradient feature vector of the first image.

For example, the first image includes 2 sub-images, a first sub-image and a second sub-image, a gradient feature vector of the first sub-image is in L dimension, and a gradient feature vector of the second sub-image is in U dimension, so that the gradient feature vector of the first image is in L + U dimension.

Before the gradient feature vector of the first image and the gradient feature vector of the second image are obtained, the first image may be compressed into an image of a reference size, the gradient feature vector of the image of the reference size may be obtained, and the gradient feature vector of the image of the reference size may be used as the gradient feature vector of the first image. Wherein the reference size is a size of the second image. That is, the first image is compressed to the same size as the second image.

The first image is compressed to be the same size as the second image, so that the computer can process all images according to a unified standard, the first image and the second image are subjected to accurate similarity detection, errors of detected similarities caused by different sizes can be avoided, and the data processing speed of the computer is improved.

The reference size may be W × H, and thus, when the image analysis processing platform receives an E × F size image, in one possible implementation, if E × F is greater than W × H, the E × F size image is compressed to W × H. In another possible implementation, if E × F is less than or equal to W × H, the E × F sized picture is not compressed.

It should be noted that the gradient feature vector of the second image in step 201 is consistent with the gradient feature vector algorithm of the first image, and is not described again.

Step 202: the embedded system determines a similarity between the first image and the second image based on the gradient feature vector of the first image and the gradient feature vector of the second image.

In a possible implementation manner, in a case that the gradient feature vector of the first image and the gradient feature vector of the second image both indicate a pixel gradient distribution condition of a pixel point in the corresponding image on a first-order gradient, the similarity between the first image and the second image is determined according to the first-order gradient feature vector of the first image and the first-order gradient feature vector of the second image.

The above implementation manner of determining the similarity between the first image and the second image according to the first-order gradient feature vector of the first image and the first-order gradient feature vector of the second image may be: the sum of squares of gradient feature vectors of all dimensions of M + N dimensions in the first order gradient feature vector of the first image is determined, the dimensions are represented by F, and a calculation formula is as follows.

X＝F1(1)²+F1(2)²+…+F1(M+N)²

And determining the sum of squares of gradient feature vectors of all the dimensions of M + N in the first-order gradient feature vector of the second image, wherein the calculation formula is as follows.

Y＝F2(1)²+F2(2)²+…+F2(M+N)²

The product Z of the gradient feature vector of all dimensions of the first image and each dimension feature in the gradient feature vector of all dimensions of the second image is determined.

Z＝F1(1)*F2(1)+F1(2)*F2(2)+…+F1(M+N)*F2(M+N)

Determining the similarity between the gradient feature vector of the first image and the gradient feature vector of the second image according to the determined Z, X and Y, wherein the calculation formula is as follows:

in another possible implementation manner, in a case that the gradient feature vector of the first image and the gradient feature vector of the second image indicate a pixel gradient distribution condition of a pixel point in a corresponding image at each step from the first-order gradient to the k-order gradient, the gradient feature vector after the first-order gradient to the k-order gradient of the first image and the gradient feature vector after the first-order gradient to the k-order gradient of the second image are connected in series determine a similarity between the first image and the second image.

The method for determining the similarity between the first image and the second image according to the gradient feature vector of the first order gradient of the first image connected in parallel to the k-step gradient and the gradient feature vector of the second image connected in parallel to the k-step gradient is the same as the method for determining the similarity between the first image and the second image according to the gradient feature vector of the first order gradient of the first image and the gradient feature vector of the first order gradient of the second image. This process is not described in detail herein.

Since in step 201, the similarity between the first image and each second image is determined based on the gradient feature vector of the first image and the gradient feature vector of each second image, the gradient feature vector of the first image and the gradient feature vector of each second image both indicate the pixel gradient distribution condition of the pixel points in the corresponding image on the first-order gradient, or both indicate the pixel gradient distribution condition of the pixel points in the corresponding image on each step from the first-order gradient to the k-order gradient. That is, for each of the plurality of second images, the operation of acquiring the gradient feature vector of the first image and the gradient feature vector of the second image is performed separately. Therefore, the similarity between one first image and each second image can be obtained based on the gradient feature vector of the first image and the gradient feature vector of each second image in the plurality of second images. At this time, in order to facilitate searching out an accurate image, a second image having the greatest similarity with the first image or a second image having a similarity with the first image exceeding a similarity threshold is acquired from the plurality of second images, and the acquired second image is used as a search result for the search instruction, that is, the second image is an image for which an accurate image is searched out. Wherein, the similarity threshold is set by the user.

In summary, in the embodiment of the present application, the similarity between the first image and the second image is determined by the obtained gradient feature vector of the first image and the gradient feature vector of the second image. The gradient feature vector is a distribution of pixel gradients of the image on the gradients, and therefore represents feature information of the image. Because the complexity of the determination process and the acquisition process for determining the distribution of the pixel gradient of the image on the gradient is small, the complexity of the computer for determining the similarity between the first image and the second image is relatively small, and the requirement on the performance of the computer is low. In addition, since the complexity of the computer for determining the similarity between the first image and the second image is small, the stability and the effectiveness of the computer for determining the similarity between the first image and the second image are ensured.

Fig. 3 is a schematic structural diagram of an apparatus for determining image similarity according to an embodiment of the present application, where the apparatus for determining image similarity may be implemented by software, hardware, or a combination of the two. The apparatus 300 for determining image similarity may include: a determining module 301 and an obtaining module 302.

The acquisition module is used for acquiring a gradient feature vector of a first image and a gradient feature vector of a second image, wherein the gradient feature vector of the first image and the gradient feature vector of the second image both indicate the pixel gradient distribution condition of pixel points in corresponding images on a first-order gradient or indicate the pixel gradient distribution condition of the pixel points in corresponding images on each step from the first-order gradient to a k-order gradient;

and the determining module is used for determining the similarity between the first image and the second image based on the gradient feature vector of the first image and the gradient feature vector of the second image.

Optionally, the gradient feature vector of the first image and the gradient feature vector of the second image both indicate pixel gradient distribution conditions of pixel points in the corresponding images on a first-order gradient, the gradient feature vector of the first image includes M-dimensional feature values corresponding to M first-order gradient direction intervals, and N-dimensional feature values corresponding to N first-order gradient amplitude intervals, the gradient feature vector of the second image includes M-dimensional feature values corresponding to M first-order gradient direction intervals, and N-dimensional feature values corresponding to N first-order gradient amplitude intervals, and M and N are positive integers greater than 1;

the gradient feature vector of the first image comprises any one of M-dimensional feature values indicating the number of pixel points of which the first-order gradient direction is located in a corresponding gradient direction interval in the first image, the gradient feature vector of the first image comprises any one of N-dimensional feature values indicating the number of pixel points of which the first-order gradient amplitude is located in a corresponding gradient amplitude interval in the first image, the gradient feature vector of the second image comprises any one of M-dimensional feature values indicating the number of pixel points of which the first-order gradient direction is located in a corresponding gradient direction interval in the second image, and the gradient feature vector of the second image comprises any one of N-dimensional feature values indicating the number of pixel points of which the first-order gradient amplitude is located in a corresponding gradient amplitude interval in the second image.

Optionally, the gradient feature vector of the first image and the gradient feature vector of the second image both indicate pixel gradient distribution conditions of pixel points in corresponding images at each step from the first-order gradient to the k-order gradient, the gradient feature vector of the first image includes P-dimensional feature values corresponding to P i-step-degree direction intervals and Q-dimensional feature values corresponding to Q i-step-degree amplitude intervals, the gradient feature vector of the second image includes P-dimensional feature values corresponding to P i-step-degree direction intervals and Q-dimensional feature values corresponding to Q i-step-degree amplitude intervals, P and Q are positive integers greater than 1, and i is a positive integer greater than or equal to 1 and less than or equal to k;

any characteristic value in the P-dimensional characteristic values included by the gradient characteristic vector of the first image indicates the number of pixel points of the first image, wherein the i-step gradient direction of the first image is located in the corresponding gradient direction interval, and any characteristic value in the Q-dimensional characteristic values included by the gradient characteristic vector of the first image indicates the number of pixel points of the first image, wherein the i-step gradient amplitude of the first image is located in the corresponding gradient amplitude interval;

any one of P-dimensional characteristic values included in the gradient characteristic vector of the second image indicates the number of pixel points of the second image, wherein the i-step gradient direction of the second image is located in the corresponding gradient direction interval, and any one of Q-dimensional characteristic values included in the gradient characteristic vector of the second image indicates the number of pixel points of the second image, wherein the i-step gradient amplitude of the second image is located in the corresponding gradient amplitude interval.

Optionally, the apparatus further comprises:

the acquisition module is used for responding to the detected search instruction, acquiring a second image with the maximum similarity with the first image from the plurality of second images or acquiring a second image with the similarity exceeding a similarity threshold value with the first image, and taking the acquired second image as a search result aiming at the search instruction;

the searching instruction indicates to query the image associated with the first image, the similarity between the first image and each second image is determined based on the gradient feature vector of the first image and the gradient feature vector of each second image, and the gradient feature vector of the first image and the gradient feature vector of each second image both indicate the pixel gradient distribution condition of the pixel points in the corresponding image on the first-order gradient or both indicate the pixel gradient distribution condition of the pixel points in the corresponding image on each step from the first-order gradient to the k-order gradient.

In summary, in the embodiment of the present application, the similarity between the first image and the second image is determined by the obtained gradient feature vector of the first image and the gradient feature vector of the second image. The gradient feature vector is a distribution condition of pixel gradients of the image on the gradients, and therefore the gradient feature vector of the first image and the gradient feature vector of the second image respectively represent feature information of the first image and the second image. Because the complexity of the determination process and the acquisition process for determining the distribution of the pixel gradient of the image on the gradient is small, the complexity of the computer for determining the similarity between the first image and the second image is relatively small, and the requirement on the performance of the computer is low. In addition, since the complexity of the computer for determining the similarity between the first image and the second image is small, the stability and the effectiveness of the computer for determining the similarity between the first image and the second image are ensured.

It should be noted that: in the apparatus for determining image similarity according to the foregoing embodiment, when determining image similarity, only the division of the functional modules is described as an example, and in practical applications, the function distribution may be completed by different functional modules according to needs, that is, the internal structure of the apparatus is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the apparatus for determining image similarity provided in the above embodiments and the method for determining image similarity belong to the same concept, and specific implementation processes thereof are described in the method embodiments in detail and are not described herein again.

Fig. 4 is a block diagram of a terminal 400 according to an embodiment of the present disclosure. The terminal 400 may be: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, motion video Experts compression standard Audio Layer 3), an MP4 player (Moving Picture Experts Group Audio Layer IV, motion video Experts compression standard Audio Layer 4), a notebook computer, or a desktop computer. The terminal 400 may also be referred to by other names such as user equipment, portable terminal, laptop terminal, desktop terminal, etc.

Generally, the terminal 400 includes: a processor 401 and a memory 402.

Processor 401 may include one or more processing cores, such as a 4-core processor, an 8-core processor, or the like. The processor 401 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 401 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 401 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content that the display screen needs to display. In some embodiments, the processor 401 may further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.

Memory 402 may include one or more computer-readable storage media, which may be non-transitory. Memory 402 may also include high speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 402 is used to store at least one instruction for execution by processor 401 to implement the method of determining image similarity provided by the method embodiments herein.

In some embodiments, the terminal 400 may further optionally include: a peripheral interface 403 and at least one peripheral. The processor 401, memory 402 and peripheral interface 403 may be connected by bus or signal lines. Each peripheral may be connected to the peripheral interface 403 via a bus, signal line, or circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 404, a display screen 405, a camera assembly 406, an audio circuit 407, a positioning assembly 408, and a power supply 409.

The peripheral interface 403 may be used to connect at least one peripheral related to I/O (Input/Output) to the processor 401 and the memory 402. In some embodiments, processor 401, memory 402, and peripheral interface 403 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 401, the memory 402 and the peripheral interface 403 may be implemented on a separate chip or circuit board, which is not limited by this embodiment.

The Radio Frequency circuit 404 is used for receiving and transmitting RF (Radio Frequency) signals, also called electromagnetic signals. The radio frequency circuitry 404 communicates with communication networks and other communication devices via electromagnetic signals. The rf circuit 404 converts an electrical signal into an electromagnetic signal to transmit, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 404 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and so forth. The radio frequency circuitry 404 may communicate with other terminals via at least one wireless communication protocol. The wireless communication protocols include, but are not limited to: metropolitan area networks, various generation mobile communication networks (2G, 3G, 4G, and 5G), Wireless local area networks, and/or WiFi (Wireless Fidelity) networks. In some embodiments, the rf circuit 404 may further include NFC (Near Field Communication) related circuits, which are not limited in this application.

The display screen 405 is used to display a UI (user interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display screen 405 is a touch display screen, the display screen 405 also has the ability to capture touch signals on or over the surface of the display screen 405. The touch signal may be input to the processor 401 as a control signal for processing. At this point, the display screen 405 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard. In some embodiments, the display screen 405 may be one, providing the front panel of the terminal 400; in other embodiments, the display screen 405 may be at least two, respectively disposed on different surfaces of the terminal 400 or in a folded design; in other embodiments, the display 405 may be a flexible display disposed on a curved surface or a folded surface of the terminal 400. Even further, the display screen 405 may be arranged in a non-rectangular irregular pattern, i.e. a shaped screen. The Display screen 405 may be made of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), and other materials.

The camera assembly 406 is used to capture images or video. Optionally, camera assembly 406 includes a front camera and a rear camera. Generally, a front camera is disposed at a front panel of the terminal, and a rear camera is disposed at a rear surface of the terminal. In some embodiments, the number of the rear cameras is at least two, and each of the rear cameras is any one of a main camera, a depth-of-field camera, a wide-angle camera and a telephoto camera, so that the main camera and the depth-of-field camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize panoramic shooting and VR (virtual reality) shooting functions or other fusion shooting functions. In some embodiments, camera assembly 406 may also include a flash. The flash lamp can be a monochrome temperature flash lamp or a bicolor temperature flash lamp. The double-color-temperature flash lamp is a combination of a warm-light flash lamp and a cold-light flash lamp, and can be used for light compensation at different color temperatures.

The audio circuit 407 may include a microphone and a speaker. The microphone is used for collecting sound waves of a user and the environment, converting the sound waves into electric signals, and inputting the electric signals to the processor 401 for processing, or inputting the electric signals to the radio frequency circuit 404 for realizing voice communication. For the purpose of stereo sound collection or noise reduction, a plurality of microphones may be provided at different portions of the terminal 400. The microphone may also be an array microphone or an omni-directional pick-up microphone. The speaker is used to convert electrical signals from the processor 401 or the radio frequency circuit 404 into sound waves. The loudspeaker can be a traditional film loudspeaker or a piezoelectric ceramic loudspeaker. When the speaker is a piezoelectric ceramic speaker, the speaker can be used for purposes such as converting an electric signal into a sound wave audible to a human being, or converting an electric signal into a sound wave inaudible to a human being to measure a distance. In some embodiments, audio circuitry 407 may also include a headphone jack.

The positioning component 408 is used to locate the current geographic position of the terminal 400 for navigation or LBS (Location Based Service). The Positioning component 408 may be a Positioning component based on the GPS (Global Positioning System) of the united states, the beidou System of china, the graves System of russia, or the galileo System of the european union.

The power supply 409 is used to supply power to the various components in the terminal 400. The power source 409 may be alternating current, direct current, disposable or rechargeable. When power source 409 comprises a rechargeable battery, the rechargeable battery may support wired or wireless charging. The rechargeable battery may also be used to support fast charge technology.

In some embodiments, the terminal 400 also includes one or more sensors 410. The one or more sensors 410 include, but are not limited to: acceleration sensor 411, gyro sensor 412, pressure sensor 413, fingerprint sensor 414, optical sensor 415, and proximity sensor 416.

The acceleration sensor 411 may detect the magnitude of acceleration in three coordinate axes of the coordinate system established with the terminal 400. For example, the acceleration sensor 411 may be used to detect components of the gravitational acceleration in three coordinate axes. The processor 401 may control the display screen 405 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 411. The acceleration sensor 411 may also be used for acquisition of motion data of a game or a user.

The gyro sensor 412 may detect a body direction and a rotation angle of the terminal 400, and the gyro sensor 412 may cooperate with the acceleration sensor 411 to acquire a 3D motion of the terminal 400 by the user. From the data collected by the gyro sensor 412, the processor 401 may implement the following functions: motion sensing (such as changing the UI according to a user's tilting operation), image stabilization at the time of photographing, game control, and inertial navigation.

The pressure sensor 413 may be disposed on a side bezel of the terminal 400 and/or on a lower layer of the display screen 405. When the pressure sensor 413 is disposed on the side frame of the terminal 400, a user's holding signal to the terminal 400 can be detected, and the processor 401 performs left-right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 413. When the pressure sensor 413 is disposed at the lower layer of the display screen 405, the processor 401 controls the operability control on the UI interface according to the pressure operation of the user on the display screen 405. The operability control comprises at least one of a button control, a scroll bar control, an icon control and a menu control.

The fingerprint sensor 414 is used for collecting a fingerprint of the user, and the processor 401 identifies the identity of the user according to the fingerprint collected by the fingerprint sensor 414, or the fingerprint sensor 414 identifies the identity of the user according to the collected fingerprint. Upon recognizing that the user's identity is a trusted identity, processor 401 authorizes the user to perform relevant sensitive operations including unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings, etc. The fingerprint sensor 414 may be disposed on the front, back, or side of the terminal 400. When a physical key or vendor Logo is provided on the terminal 400, the fingerprint sensor 414 may be integrated with the physical key or vendor Logo.

The optical sensor 415 is used to collect the ambient light intensity. In one embodiment, processor 401 may control the display brightness of display screen 405 based on the ambient light intensity collected by optical sensor 415. Specifically, when the ambient light intensity is high, the display brightness of the display screen 405 is increased; when the ambient light intensity is low, the display brightness of the display screen 405 is reduced. In another embodiment, the processor 401 may also dynamically adjust the shooting parameters of the camera assembly 406 according to the ambient light intensity collected by the optical sensor 415.

A proximity sensor 416, also known as a distance sensor, is typically disposed on the front panel of the terminal 400. The proximity sensor 416 is used to collect the distance between the user and the front surface of the terminal 400. In one embodiment, when the proximity sensor 416 detects that the distance between the user and the front surface of the terminal 400 gradually decreases, the processor 401 controls the display screen 405 to switch from the bright screen state to the dark screen state; when the proximity sensor 416 detects that the distance between the user and the front surface of the terminal 400 is gradually increased, the processor 401 controls the display screen 405 to switch from the breath-screen state to the bright-screen state.

Those skilled in the art will appreciate that the configuration shown in fig. 4 is not intended to be limiting of terminal 400 and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components may be used.

Embodiments of the present application further provide a non-transitory computer-readable storage medium, where instructions in the storage medium, when executed by a processor of a terminal, enable the terminal to perform the method for determining image similarity provided in the above embodiments.

Embodiments of the present application further provide a computer program product containing instructions, which when run on a terminal, cause the terminal to perform the method for determining image similarity provided in the foregoing embodiments.

Fig. 5 is a schematic structural diagram of a server according to an embodiment of the present application. The server may be a server in a cluster of background servers. Specifically, the method comprises the following steps:

the server 500 includes a Central Processing Unit (CPU)501, a system memory 504 including a Random Access Memory (RAM)502 and a Read Only Memory (ROM)503, and a system bus 505 connecting the system memory 504 and the central processing unit 501. The server 500 also includes a basic input/output system (I/O system) 506, which facilitates transfer of information between devices within the computer, and a mass storage device 507, which stores an operating system 513, application programs 514, and other program modules 515.

The basic input/output system 506 comprises a display 508 for displaying information and an input device 509, such as a mouse, keyboard, etc., for user input of information. Wherein a display 508 and an input device 509 are connected to the central processing unit 501 through an input output controller 510 connected to the system bus 505. The basic input/output system 506 may also include an input/output controller 510 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, input-output controller 510 also provides output to a display screen, a printer, or other type of output device.

The mass storage device 507 is connected to the central processing unit 501 through a mass storage controller (not shown) connected to the system bus 505. The mass storage device 507 and its associated computer-readable media provide non-volatile storage for the server 500. That is, the mass storage device 507 may include a computer readable medium (not shown) such as a hard disk or CD-ROM drive.

Without loss of generality, computer readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, EPROM, EEPROM, flash memory or other solid state memory technology, CD-ROM, DVD, or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that computer storage media is not limited to the foregoing. The system memory 504 and mass storage device 507 described above may be collectively referred to as memory.

According to various embodiments of the present application, server 500 may also operate as a remote computer connected to a network through a network, such as the Internet. That is, the server 500 may be connected to the network 512 through the network interface unit 511 connected to the system bus 505, or may be connected to other types of networks or remote computer systems (not shown) using the network interface unit 511.

The memory further includes one or more programs, and the one or more programs are stored in the memory and configured to be executed by the CPU. The one or more programs include instructions for performing the method for determining image similarity provided by the embodiments of the present application, including:

acquiring a gradient feature vector of a first image and a gradient feature vector of a second image, wherein the gradient feature vector of the first image and the gradient feature vector of the second image both indicate the pixel gradient distribution condition of pixel points in corresponding images on a first-order gradient or indicate the pixel gradient distribution condition of pixel points in corresponding images on each gradient from the first-order gradient to a k-order gradient;

Embodiments of the present application further provide a non-transitory computer-readable storage medium, where instructions in the storage medium, when executed by a processor of a server, enable the server to perform the method for determining image similarity provided in the foregoing embodiments.

Embodiments of the present application further provide a computer program product containing instructions, which when run on a server, cause the server to perform the method for determining image similarity provided by the foregoing embodiments.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The above description is only a preferred embodiment of the present application and should not be taken as limiting the present application, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for determining image similarity, the method comprising:

2. The method according to claim 1, wherein, in a case where the gradient feature vector of the first image and the gradient feature vector of the second image both indicate pixel gradient distribution conditions of pixel points in the respective images on first-order gradients, the gradient feature vector of the first image includes M-dimensional feature values corresponding to M first-order gradient direction intervals, respectively, and N-dimensional feature values corresponding to N first-order gradient magnitude intervals, the gradient feature vector of the second image includes M-dimensional feature values corresponding to M first-order gradient direction intervals, respectively, and N-dimensional feature values corresponding to N first-order gradient magnitude intervals, respectively, both M and N being positive integers greater than 1;

3. The method according to claim 1, wherein in a case where the gradient feature vector of the first image and the gradient feature vector of the second image both indicate pixel gradient distribution of pixel points in the respective images at each of first-order gradient to k-order gradient, the gradient feature vector of the first image includes P-dimensional feature values corresponding to P i-step degree direction intervals, respectively, and Q-dimensional feature values corresponding to Q i-step degree amplitude intervals, the gradient feature vector of the second image includes P-dimensional feature values corresponding to P i-step degree direction intervals, respectively, and Q-dimensional feature values corresponding to Q i-step degree amplitude intervals, respectively, the P and Q being positive integers greater than 1, the i being a positive integer greater than or equal to 1 and less than or equal to k;

4. The method of any of claims 1 to 3,

the first image comprises a plurality of sub-images, and the gradient feature vector of the first image is obtained by superposing the gradient feature vector of each sub-image in the plurality of sub-images in the first image;

5. The method of claim 1, wherein prior to obtaining the gradient feature vector of the first image and the gradient feature vector of the second image, the method further comprises:

6. The method of any of claims 1 to 5, further comprising:

7. An apparatus for determining image similarity, the apparatus comprising:

8. The apparatus according to claim 7, wherein in a case where the gradient feature vector of the first image and the gradient feature vector of the second image indicate pixel gradient distribution conditions of pixel points in the respective images at first-order gradients, the gradient feature vector of the first image includes M-dimensional feature values corresponding to M first-order gradient direction intervals, respectively, and N-dimensional feature values corresponding to N first-order gradient magnitude intervals, the gradient feature vector of the second image includes M-dimensional feature values corresponding to M first-order gradient direction intervals, respectively, and N-dimensional feature values corresponding to N first-order gradient magnitude intervals, respectively, M and N being positive integers greater than 1;

any one of the M-dimensional feature values included in the gradient feature vector of the second image indicates the number of pixel points in the second image, of which the first-order gradient direction is located in the corresponding gradient direction interval, and any one of the N-dimensional feature values included in the gradient feature vector of the second image indicates the number of pixel points in the second image, of which the first-order gradient amplitude is located in the corresponding gradient amplitude interval;

under the condition that the gradient feature vector of the first image and the gradient feature vector of the second image both indicate the pixel gradient distribution condition of pixel points in corresponding images on each step from a first-order gradient to a k-order gradient, the gradient feature vector of the first image comprises P-dimensional feature values respectively corresponding to P i-step direction intervals and Q-dimensional feature values respectively corresponding to Q i-step amplitude intervals, the gradient feature vector of the second image comprises P-dimensional feature values respectively corresponding to P i-step direction intervals and Q-dimensional feature values respectively corresponding to Q i-step amplitude intervals, P and Q are positive integers greater than 1, and i is a positive integer greater than or equal to 1 and less than or equal to k;

any one of P-dimensional feature values included in the gradient feature vector of the second image indicates the number of pixel points in the second image, wherein the i-step gradient direction of the pixel points is located in the corresponding gradient direction interval, and any one of Q-dimensional feature values included in the gradient feature vector of the second image indicates the number of pixel points in the second image, wherein the i-step gradient amplitude of the pixel points is located in the corresponding gradient amplitude interval;

the number of the sub-images included in the first image is the same as that of the sub-images included in the second image, the size of the sub-images included in the first image is the same as that of the sub-images included in the second image, and the gradient feature vector of each sub-image in the plurality of sub-images in the first image and the gradient feature vector of each sub-image in the plurality of sub-images in the second image both indicate the pixel gradient distribution condition of the pixel points in the corresponding sub-image on the first-order gradient or both indicate the pixel gradient distribution condition of the pixel points in the corresponding sub-image on each step from the first-order gradient to the k-order gradient;

wherein the apparatus further comprises a compression module for compressing the first picture into a picture of a reference size, the reference size being the size of the second picture;

the obtaining module is further configured to obtain a gradient feature vector of the image with the reference size, and use the gradient feature vector of the image with the reference size as the gradient feature vector of the first image;

the obtaining module is further configured to, in response to the detected search instruction, obtain a second image with the greatest similarity to the first image from the plurality of second images, or obtain a second image with a similarity to the first image exceeding a similarity threshold, and use the obtained second image as a search result for the search instruction;

9. An apparatus for determining image similarity, the apparatus comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to perform the steps of the method of any of the above claims 1 to 6.

10. A computer-readable storage medium having stored thereon instructions which, when executed by a processor, carry out the steps of the method of any of the preceding claims 1 to 6.