CN108230409B - Image similarity quantitative analysis method based on multi-factor synthesis of color and content - Google Patents

Image similarity quantitative analysis method based on multi-factor synthesis of color and content Download PDF

Info

Publication number
CN108230409B
CN108230409B CN201810263619.XA CN201810263619A CN108230409B CN 108230409 B CN108230409 B CN 108230409B CN 201810263619 A CN201810263619 A CN 201810263619A CN 108230409 B CN108230409 B CN 108230409B
Authority
CN
China
Prior art keywords
similarity
image
color
img1
images
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810263619.XA
Other languages
Chinese (zh)
Other versions
CN108230409A (en
Inventor
黄凯
郭延文
杜振龙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University
Original Assignee
Nanjing University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University filed Critical Nanjing University
Priority to CN201810263619.XA priority Critical patent/CN108230409B/en
Publication of CN108230409A publication Critical patent/CN108230409A/en
Application granted granted Critical
Publication of CN108230409B publication Critical patent/CN108230409B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a quantitative analysis method for image similarity based on multi-factor synthesis of color and content, which comprises the following steps: step 1, reading images img1 and img2, and preprocessing; step 2, analyzing color similarity; step 3, analyzing the content similarity; step 4, analyzing the comprehensive similarity of the color and the content; step 5, comprehensive operation: performing comprehensive operation on the results obtained in the steps 2, 3 and 4 to obtain a comprehensive similarity value similarity _ pri of the image; and 6, correcting the comprehensive similarity value similarity _ pri of the image: and (3) finely correcting the similarity _ pri according to the result obtained by the preprocessing in the step 1 to obtain a quantitative similarity value similarity _ val of the last two images.

Description

Image similarity quantitative analysis method based on multi-factor synthesis of color and content
Technical Field
The invention belongs to the fields of computer vision, digital image processing and the like, and particularly relates to a quantitative analysis method for image similarity based on multi-factor synthesis of color and content.
Background
The image similarity calculation mainly comprises the steps of carrying out similarity scoring on any two images and judging the similarity degree between the two images according to the degree of the score. The similarity calculation based on vision is mainly determined according to comprehensive factors such as color, content and scene correlation of images.
Image similarity calculation has become one of indispensable techniques in the field of image processing and computer vision. Industrial vision systems use similarity calculations for images in a variety of applications, such as defect detection, radar image recognition, traffic management, industrial monitoring, face recognition, medical diagnostics, and the like. In addition, many basic researches in the field of computer vision also require the calculation of image similarity, such as image matching, image retrieval, image classification, and the like. Even the similarity value between images can be used as one of the features of data in deep learning, so how to reasonably and accurately calculate the similarity value between images becomes important.
At present, the image similarity calculation based on vision mainly carries out simple histogram statistics on color information of an image, and judges the similarity of the image according to the similarity of the histogram. The SIFT feature points can keep good invariance to rotation, scale scaling and brightness change, and meanwhile, the image brightness change, affine change and noise also have good stability, so that the other similarity calculation for the image mainly utilizes the SIFT features to carry out SIFT matching on the image, and then judges the similarity of the image according to the matching result. Ssim (structural similarity index), the structural similarity index measures the similarity of an image through three comparison modules, namely brightness, contrast and structure, and is mainly used for judging the distortion degree of the image.
In a comprehensive view, the image similarity calculation method based on vision is simple at present, a systematic and reasonable method for quantitatively analyzing the image similarity based on vision does not exist, and a huge space for continuous research and perfection exists.
Disclosure of Invention
The purpose of the invention is as follows: the invention aims to solve the technical problem of reasonably scoring the similarity of any two images.
The technical scheme is as follows: the method is characterized in that similarity scoring is carried out on any two images according to information such as color, content, time, GPS and the like of the images, the final scoring result is a numerical value between 0 and 1, the larger the numerical value is, the more similar the two images are, and the method specifically comprises the following steps:
step 1, reading two images to be compared, img1 and img2, and preprocessing the images by taking img1 as reference images;
step 2, color similarity analysis: carrying out significance region detection on the images img1 and img2, distinguishing the foreground and the background of the images, carrying out interval division on RGB (Red, Green and Blue) color spaces, carrying out pixel-level interval clustering on the images to generate image foreground and background fingerprints, respectively carrying out correlation analysis on the image foreground and background fingerprints, and comprehensively generating a quantitative similarity value similarity _ color based on image color information;
and step 3, content similarity analysis: performing quantitative similarity analysis on the image according to the result obtained by preprocessing in the step 1, performing threshold segmentation on the Y component based on an OTSU method in a YUV color space to obtain a content map of the image, calculating a combined gradient domain combined Grad, and calculating a quantitative similarity value similarity _ contents of the image based on content information according to the content map of the image and the combined gradient domain combined Grad;
and 4, analyzing comprehensive similarity of colors and contents: respectively carrying out interval division on RGB color spaces of the images img1 and img2, carrying out pixel-level interval clustering, respectively generating an image pixel type label map labelMap1 of the image img1 and an image pixel type label map labelMap2 of the image img2, respectively carrying out granularity roughening operation on labelMap1 and labelMap2, and respectively recording the results as labelMap New1 and labelMap New 2; calculating quantitative similarity values of images based on color and content comprehensive information according to labelmapNew1 and labelmapNew 2;
step 5, comprehensive operation: carrying out comprehensive operation on the quantitative similarity value similarity _ color based on the image color information, the quantitative similarity value similarity _ contents based on the content information and the quantitative similarity value similarity _ color contents based on the color and content comprehensive information obtained in the steps 2, 3 and 4 to obtain a comprehensive similarity value similarity _ pri of the image;
and 6, correcting the comprehensive similarity value similarity _ pri of the image: and (3) finely correcting the comprehensive similarity value similarity _ pri of the images according to the result obtained by the preprocessing in the step (1) to obtain the quantitative similarity value similarity _ val of the last two images.
The method comprises the following steps of 1:
reading two images img1 and img2 to be compared in an RGB form, reading an image EXIF (Exchangeable image file, in which various information of the image such as an aperture, a shutter, white balance, ISO, a focal length, date and time, GPS and the like when the image is shot) information, obtaining image time and GPS information, and respectively performing brightness equalization processing on img1 and img2, specifically comprising the following steps:
step 1-1, converting an image RGB space into a YUV (one color space, the same as the RGB space, and belongs to one color space) space, and respectively obtaining the brightness components of two images, wherein the brightness component of img1 is Y1, and the brightness component of img2 is Y2;
step 1-2, respectively calculating the mean values of Y1 and Y2, and respectively recording the mean values as
Figure RE-GDA0001627599900000031
Calculating the mean difference
Figure RE-GDA0001627599900000032
Step 1-3, calculate new Y1, Y2 values:
if it is not
Figure RE-GDA0001627599900000033
Then
Figure RE-GDA0001627599900000034
If it is not
Figure RE-GDA0001627599900000035
Then
Figure RE-GDA0001627599900000036
Where (i, j) represents the position of the pixel unit in the original image, i represents the number of rows, j represents the number of columns, Y1(i, j) represents the pixel value at (i, j) corresponding to the luminance component Y1, and Y2(i, j) represents the pixel value at (i, j) corresponding to the luminance component Y2;
and 1-4, converting the YUV space into an RGB space, and respectively obtaining an image balanceImg1 corresponding to img1 and an image balanceImg2 corresponding to img 2.
The step 2 of the invention comprises the following steps:
step 2-1, downsampling img1 and img2 by using a salient object detection algorithm based on region comparison and carrying out salient region detection to obtain a foreground foregroudlmg 1 and a background backgroudlmg 1 of img1, a foreground foregroudlmg 2 and a background backgroudlmg 2 of img2 (specifically: Cheng, Ming-Ming, et al. "Salientobject detection and segmentation." IEEE Transactions on Pattern Analysis & Machine Analysis 1(2014): 1-1);
step 2-2, calculating fingerprints of foregroundImg1, backploulim 1, foregroundImg2 and backploulim 2, wherein the specific calculation steps are as follows: for the foregroudlimg 1, performing interval division on an RGB color space, averagely dividing each channel of three channels of RGB into 8 intervals, namely, the length of each interval is 32, forming 512 intervals by the three channels, initializing a 512-dimensional vector a, wherein each dimension of the vector a represents one interval; carrying out interval statistical clustering on each pixel of the foregroundImg1, counting the number of each interval pixel as a vector a, wherein the dimension value is obtained, and carrying out normalization processing to obtain the vector a as a foregroundImg1 fingerprint finrprintFore 1; the same calculation steps are carried out on backsgroundimg 1, forkroundimg 2 and backsgroundimg 2 to obtain fingerprint fingerprintBack1 of backsgroundimg 1, fingerprint fingerprintform 2 of forkroundimg 2 and fingerprint fingerprintBack2 of backsgroundimg 2 respectively;
step 2-3, calculating the correlation coefficient between finger printform 1 and finger printform 2 as coeff _ form; calculating the correlation coefficient between finger print Back1 and finger print Back2 as coeff _ back, wherein the correlation coefficient is rhoXYThe calculation formula is as follows:
Figure RE-GDA0001627599900000037
wherein Cov (X, Y) is the covariance of X and Y, and D (X), D (Y) are the variance of X and the variance of Y, respectively. And finally, based on the image color information similarity value similarity _ color, the similarity _ color is as follows:
similarity_color＝cFore*coeff_fore+cBack*coeff_back，
wherein the content of the first and second substances,
Figure RE-GDA0001627599900000041
cBack＝1-cFore，SforegroundImg1is the foreground area of Img1, Simg1Is img1 area.
Step 3 of the invention comprises the following steps:
step 3-1, down-sampling the balanceImg1 and the balanceImg2 obtained in the step 1, converting the down-sampled data into YUV color space, respectively obtaining luminance components YContents1 and 2 of the balanceImg2 of the balanceImg1, and carrying out Bilateral filtering processing on YContents1 and YContents2 (reference: https:// en. wikipedia. org/wiki/Bilateral _ filter);
step 3-2, threshold segmentation based on OTSU method (https:// en. wikipedia.org/wiki/Otsu% 27s _ method) is carried out on YConsets 1 and YConsets 2 to obtain content map of YConsets 1 as fullContents1 and content map of YConsets 2 as fullContents2, and content difference map contentsDiff is calculated by the following formula:
contentsDiff＝fullContents1⊙fullContents2；
step 3-3, calculating gradient domains of YContents1 and YContents2 to obtain gradient maps grad1 and grad2 of YContents1 and YContents2 respectively, and calculating a combined gradient domain combinegrade by using the following formula:
combineGrad＝grad1.*grad2，
denotes dot product operations between matrices;
normalized treatment of combineGrad (reference: https:// baike. ***. com/item/% E5% BD% 92% E4% B8% 80% E5% 8C% 96% E6% 96% B9% E6% B3% 95/10089118);
step 3-4, calculating similarity value similarity _ contents based on the image content information, wherein the specific formula is as follows:
Figure RE-GDA0001627599900000042
step 4 of the invention comprises the following steps:
step 4-1, downsampling balance img1 and balance img2 to the same size and performing pixel level clustering: the method comprises the steps of carrying out interval division on an RGB color space, averagely dividing each channel of three channels of RGB into 2 intervals, namely, the length of each interval is 128, forming 8 intervals by the three channels, and representing one label and 8 labels by each interval; label labeling is carried out on each pixel in the balance img1 and the balance img2 to form a label image, an image pixel class label image of the generated image img1 is labeled label map1, and an image pixel class label image of the image img2 is labeled label map 2;
step 4-2, fine to coarse grain size roughening operations were performed on labelMap1, labelMap2, respectively: calculating all communicated components of the label graph, finding out the label with the largest adjacent area of all the communicated components with the area smaller than one thousandth of the area of the label graph, marking the label as label Max, and marking all the labels of the communicated components as label Max; finally, the result of fine to coarse grain size roughening operation by labelMap1 is recorded as labelMap new1, and the result of fine to coarse grain size roughening operation by labelMap2 is recorded as labelMap new 2;
step 4-3, defining that if the label values of the corresponding pixel positions in labelMapNew1 and labelMapNew2 are equal, the pixels are overlapped, and the similarity value similarity _ colorContents based on the color and content comprehensive information is as follows:
Figure RE-GDA0001627599900000051
wherein S iscoverThe overlapping area of labelMapNew1 and labelMapNew2, SlabelMapNew1Is the area of labelMapNew 1.
The step 5 of the invention comprises the following steps:
calculating an image comprehensive similarity value similarity _ pri by the following formula:
Figure RE-GDA0001627599900000052
wherein, a is similarity _ color, b is similarity _ contents, c is similarity _ color contents, max () is maximum value function, min () is minimum value function, mid is median function, mean () is mean function.
Step 6 of the invention comprises the following steps:
step 6-1, calculating the time correlation coeff according to the image time information obtained in step 1timeThe calculation formula is as follows:
Figure RE-GDA0001627599900000053
Figure RE-GDA0001627599900000054
coefftime＝max(timeeffect*timecoeff+(1-timeeffect)*coeffpri,coeffpri)，
wherein, timec、timecoeffRepresenting an intermediate variable, detalT is the shooting time interval of two images, taking seconds as a unit, and e is the base number of a natural logarithm; timeconst＝86400，timeeffect＝0.2，coeffpri＝ similarity_pri；
Step 6-2, calculating the GPS correlation coeff according to the image GPS information obtained in the step 1gpsThe calculation formula is as follows:
Figure RE-GDA0001627599900000061
Figure RE-GDA0001627599900000062
coeffgps＝max(gpseffect*gpscoeff+(1-gpseffect)*coeffpri,coeffpri)，
wherein gpsc、gpscoeffRepresenting an intermediate variable, and the detalGPS is the shooting place distance of the two images and takes meters as a unit; gpsconst＝5000，gpseffect＝0.2,coeffpri＝similarity_pri；
And 6-3, calculating the final similarity value similarity _ val of the two images, wherein the calculation formula is as follows:
Figure RE-GDA0001627599900000063
the invention has the beneficial effects that:
(1) the invention relates to an image similarity calculation method, which can effectively and reasonably calculate the similarity value of any two images.
(2) The method greatly improves the operation speed through the downsampling strategy without influencing the final result, and is an efficient image similarity calculation method.
Drawings
The foregoing and other advantages of the invention will become more apparent from the following detailed description of the invention when taken in conjunction with the accompanying drawings.
FIG. 1 is a basic flow diagram of the process of the present invention.
Fig. 2a is a reference image of the first group of images fig. 2b and fig. 2 c.
Fig. 2b is a similarity comparison image having a similarity value of 0.83 with respect to fig. 2 a.
Fig. 2c is a similarity comparison image having a similarity value of 0.58 with respect to fig. 2 a.
Fig. 3a is a reference image of the second group of images, like fig. 3b, fig. 3 c.
Fig. 3b is a similarity comparison image having a similarity value of 0.97 with respect to fig. 3 a.
Fig. 3c is a similarity comparison image having a similarity value of 0.53 with respect to fig. 3 a.
Fig. 4a is a reference image of the third group of images fig. 4b, 4 c.
Fig. 4b is a similarity comparison image having a similarity value of 0.51 with respect to fig. 4 a.
Fig. 4c is a similarity comparison image having a similarity value of 0.76 with respect to fig. 4 a.
Fig. 5a shows a fourth set of reference images like fig. 5b and 5 c.
Fig. 5b is a similarity comparison image having a similarity value of 0.84 with respect to fig. 5 a.
Fig. 5c is a similarity comparison image having a similarity value of 0.74 with respect to fig. 5 a.
Detailed Description
The invention is further explained below with reference to the drawings and the embodiments.
The flow chart of the method is shown in figure 1, the method of the invention scores similarity of any two images mainly according to information such as color, content, time, GPS and the like of the images, the final scoring result is a numerical value between 0 and 1, the larger the numerical value is, the more similar the two images are, and the method specifically comprises the following steps:
step 1, reading two images to be compared, img1 and img2, and preprocessing the images by taking img1 as reference images;
step 2, analyzing color similarity: carrying out significance region detection on the images img1 and img2, distinguishing the foreground and the background of the images, carrying out interval division on RGB (Red, Green and Blue) color spaces, carrying out pixel-level interval clustering on the images to generate image foreground and background fingerprints, respectively carrying out correlation analysis on the image foreground and background fingerprints, and comprehensively generating a quantitative similarity value similarity _ color based on image color information;
and step 3, content similarity analysis: performing quantitative similarity analysis on the image according to the result obtained by preprocessing in the step 1, performing threshold segmentation on the Y component based on an OTSU method in a YUV color space to obtain a content map of the image, calculating a combined gradient domain combined Grad, and calculating a quantitative similarity value similarity _ contents of the image based on content information according to the content map of the image and the combined gradient domain combined Grad;
and 4, analyzing comprehensive similarity of colors and contents: respectively carrying out interval division on RGB color spaces of the images img1 and img2, carrying out pixel-level interval clustering, respectively generating an image pixel type label map labelMap1 of the image img1 and an image pixel type label map labelMap2 of the image img2, respectively carrying out granularity roughening operation on labelMap1 and labelMap2, and respectively recording the results as labelMap New1 and labelMap New 2; calculating quantitative similarity values of images based on color and content comprehensive information according to labelmapNew1 and labelmapNew 2;
step 5, comprehensive operation: carrying out comprehensive operation on the quantitative similarity value similarity _ color based on the image color information, the quantitative similarity value similarity _ contents based on the content information and the quantitative similarity value similarity _ color contents based on the color and content comprehensive information obtained in the steps 2, 3 and 4 to obtain a comprehensive similarity value similarity _ pri of the image;
and 6, correcting the comprehensive similarity value similarity _ pri of the image: and (3) finely correcting the comprehensive similarity value similarity _ pri of the images according to the result obtained by the preprocessing in the step (1) to obtain the quantitative similarity value similarity _ val of the last two images.
The method comprises the following steps of 1:
reading two images img1 and img2 to be compared in an RGB form, reading an image EXIF (Exchangeable image file, in which various information of the image such as an aperture, a shutter, white balance, ISO, a focal length, date and time, GPS and the like when the image is shot) information, obtaining image time and GPS information, and respectively performing brightness equalization processing on img1 and img2, specifically comprising the following steps:
step 1-1, converting an image RGB space into a YUV (one color space, the same as the RGB space, and belongs to one color space) space, and respectively obtaining the brightness components of two images, wherein the brightness component of img1 is Y1, and the brightness component of img2 is Y2;
step 1-2, respectively calculating the mean values of Y1 and Y2, and respectively recording the mean values as
Figure RE-GDA0001627599900000081
Calculating the mean difference
Figure RE-GDA0001627599900000082
Step 1-3, calculate new Y1, Y2 values:
if it is not
Figure RE-GDA0001627599900000083
Then
Figure RE-GDA0001627599900000084
If it is not
Figure RE-GDA0001627599900000085
Then
Figure RE-GDA0001627599900000086
Where (i, j) represents the position of the pixel unit in the original image, i represents the number of rows, j represents the number of columns, Y1(i, j) represents the pixel value at (i, j) corresponding to the luminance component Y1, and Y2(i, j) represents the pixel value at (i, j) corresponding to the luminance component Y2;
and 1-4, converting the YUV space into an RGB space, and respectively obtaining an image balanceImg1 corresponding to img1 and an image balanceImg2 corresponding to img 2.
The step 2 of the invention comprises the following steps:
step 2-1, downsampling img1 and img2 by using a salient object detection algorithm based on region comparison and carrying out salient region detection to obtain a foreground foregroudlmg 1 and a background backgroudlmg 1 of img1, a foreground foregroudlmg 2 and a background backgroudlmg 2 of img2 (specifically: Cheng, Ming-Ming, et al. "Salientobject detection and segmentation." IEEE Transactions on Pattern Analysis & Machine Analysis 1(2014): 1-1);
step 2-2, calculating fingerprints of foregroundImg1, backploulim 1, foregroundImg2 and backploulim 2, wherein the specific calculation steps are as follows: for the foregroudlimg 1, performing interval division on an RGB color space, averagely dividing each channel of three channels of RGB into 8 intervals, namely, the length of each interval is 32, forming 512 intervals by the three channels, initializing a 512-dimensional vector a, wherein each dimension of the vector a represents one interval; carrying out interval statistical clustering on each pixel of the foregroundImg1, counting the number of each interval pixel as a vector a, wherein the dimension value is obtained, and carrying out normalization processing to obtain the vector a as a foregroundImg1 fingerprint finrprintFore 1; the same calculation steps are carried out on backsgroundimg 1, forkroundimg 2 and backsgroundimg 2 to obtain fingerprint fingerprintBack1 of backsgroundimg 1, fingerprint fingerprintform 2 of forkroundimg 2 and fingerprint fingerprintBack2 of backsgroundimg 2 respectively;
step 2-3, calculating the correlation coefficient between finger printform 1 and finger printform 2 as coeff _ form; calculating the correlation coefficient between finger print Back1 and finger print Back2 as coeff _ back, wherein the correlation coefficient is rhoXYThe calculation formula is as follows:
Figure RE-GDA0001627599900000091
wherein Cov (X, Y) is the covariance of X and Y, and D (X), D (Y) are the variance of X and the variance of Y, respectively. And finally, based on the image color information similarity value similarity _ color, the similarity _ color is as follows:
similarity_color＝cFore*coeff_fore+cBack*coeff_back，
wherein the content of the first and second substances,
Figure RE-GDA0001627599900000092
cBack＝1-cFore，SforegroundImg1is the foreground area of Img1, Simg1Is img1 area.
Step 3 of the invention comprises the following steps:
step 3-1, down-sampling the balanceImg1 and the balanceImg2 obtained in the step 1, converting the down-sampled data into YUV color space, respectively obtaining luminance components YContents1 and 2 of the balanceImg2 of the balanceImg1, and carrying out Bilateral filtering processing on YContents1 and YContents2 (reference: https:// en. wikipedia. org/wiki/Bilateral _ filter);
step 3-2, threshold segmentation based on OTSU method (https:// en. wikipedia.org/wiki/Otsu% 27s _ method) is carried out on YConsets 1 and YConsets 2 to obtain content map of YConsets 1 as fullContents1 and content map of YConsets 2 as fullContents2, and content difference map contentsDiff is calculated by the following formula:
contentsDiff＝fullContents1⊙fullContents2；
step 3-3, calculating gradient domains of YContents1 and YContents2 to obtain gradient maps grad1 and grad2 of YContents1 and YContents2 respectively, and calculating a combined gradient domain combinegrade by using the following formula:
combineGrad＝grad1.*grad2，
normalized treatment of combineGrad (reference: https:// baike. ***. com/item/% E5% BD% 92% E4% B8% 80% E5% 8C% 96% E6% 96% B9% E6% B3% 95/10089118);
step 3-4, calculating similarity value similarity _ contents based on the image content information, wherein the specific formula is as follows:
Figure RE-GDA0001627599900000101
step 4 of the invention comprises the following steps:
step 4-1, downsampling balance img1 and balance img2 to the same size and performing pixel level clustering: the method comprises the steps of carrying out interval division on an RGB color space, averagely dividing each channel of three channels of RGB into 2 intervals, namely, the length of each interval is 128, forming 8 intervals by the three channels, and representing one label and 8 labels by each interval; label labeling is carried out on each pixel in the balance img1 and the balance img2 to form a label image, an image pixel class label image of the generated image img1 is labeled label map1, and an image pixel class label image of the image img2 is labeled label map 2;
step 4-2, fine to coarse grain size roughening operations were performed on labelMap1, labelMap2, respectively: calculating all communicated components of the label graph, finding out the label with the largest adjacent area of all the communicated components with the area smaller than one thousandth of the area of the label graph, marking the label as label Max, and marking all the labels of the communicated components as label Max; finally, the result of fine to coarse grain size roughening operation by labelMap1 is recorded as labelMap new1, and the result of fine to coarse grain size roughening operation by labelMap2 is recorded as labelMap new 2;
step 4-3, defining that if the label values of the corresponding pixel positions in labelMapNew1 and labelMapNew2 are equal, the pixels are overlapped, and the similarity value similarity _ colorContents based on the color and content comprehensive information is as follows:
Figure RE-GDA0001627599900000102
wherein S iscoverThe overlapping area of labelMapNew1 and labelMapNew2, SlabelMapNew1Is the area of labelMapNew 1.
The step 5 of the invention comprises the following steps:
calculating an image comprehensive similarity value similarity _ pri by the following formula:
Figure RE-GDA0001627599900000111
wherein, a is similarity _ color, b is similarity _ contents, c is similarity _ color contents, max () is maximum value function, min () is minimum value function, mid is median function, mean () is mean function.
Step 6 of the invention comprises the following steps:
step 6-1, calculating the time correlation coeff according to the image time information obtained in step 1timeThe calculation formula is as follows:
Figure RE-GDA0001627599900000112
Figure RE-GDA0001627599900000113
coefftime＝max(timeeffect*timecoeff+(1-timeeffect)*coeffpri,coeffpri)，
wherein, timec、timecoeffRepresenting an intermediate variable, detalT is the shooting time interval of two images, taking seconds as a unit, and e is the base number of a natural logarithm; timeconst＝86400，timeeffect＝0.2，coeffpri＝ similarity_pri；
Step 6-2, calculating the GPS correlation coeff according to the image GPS information obtained in the step 1gpsThe calculation formula is as follows:
Figure RE-GDA0001627599900000114
Figure RE-GDA0001627599900000115
coeffgps＝max(gpseffect*gpscoeff+(1-gpseffect)*coeffpri,coeffpri)，
wherein gpsc、gpscoeffRepresenting an intermediate variable, and the detalGPS is the shooting place distance of the two images and takes meters as a unit; gpsconst＝5000，gpseffect＝0.2,coeffpri＝similarity_pri；
And 6-3, calculating the final similarity value similarity _ val of the two images, wherein the calculation formula is as follows:
Figure RE-GDA0001627599900000116
the experimental development and operating environment of this example is: intel core i 5-45903.30 GHz quad CPU (processor), NVIDIA GeForce GTX 760GPU (display card), 8G memory, software environment Microsoft Visual Studio2013 (software development tool) and OpenCV 2.4.11 (computer vision open source library).
In this embodiment, four sets of images are outputted, where fig. 2a, 2b, and 2c are the first set, fig. 3a, 3b, and 3c are the second set, fig. 4a, 4b, and 4c are the third set, and fig. 5a, 5b, and 5c are the fourth set. Fig. 2a is a reference image of the first group of images fig. 2b and fig. 2 c. Fig. 2b is a similarity comparison image having a similarity value of 0.83 with respect to fig. 2 a. Fig. 2c is a similarity comparison image having a similarity value of 0.58 with respect to fig. 2 a.
Fig. 3a is a reference image of the second group of images, like fig. 3b, fig. 3 c. Fig. 3b is a similarity comparison image having a similarity value of 0.97 with respect to fig. 3 a. Fig. 3c is a similarity comparison image having a similarity value of 0.53 with respect to fig. 3 a.
Fig. 4a is a reference image of the third group of images fig. 4b, 4 c. Fig. 4b is a similarity comparison image having a similarity value of 0.51 with respect to fig. 4 a. Fig. 4c is a similarity comparison image having a similarity value of 0.76 with respect to fig. 4 a.
Fig. 5a shows a fourth set of reference images like fig. 5b and 5 c. Fig. 5b is a similarity comparison image having a similarity value of 0.84 with respect to fig. 5 a. Fig. 5c is a similarity comparison image having a similarity value of 0.74 with respect to fig. 5 a.
As can be seen from the figure, the invention can obtain better analysis results for any two images.
The present invention provides a quantitative analysis method for image similarity based on multi-factor synthesis of color and content, and a plurality of methods and ways for implementing the technical solution, and the above description is only a preferred embodiment of the present invention, it should be noted that, for those skilled in the art, a plurality of improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention. All the components not specified in the present embodiment can be realized by the prior art.

Claims (3)

1. The image similarity quantitative analysis method based on multi-factor synthesis of color and content is characterized by comprising the following steps of:
step 1, reading two images to be compared, img1 and img2, and preprocessing the images by taking img1 as reference images;
step 2, color similarity analysis: respectively carrying out significance region detection on the images img1 and img2, distinguishing image foreground and background, carrying out interval division on RGB color space, carrying out pixel-level interval clustering on the images to generate image foreground and background fingerprints, respectively carrying out correlation analysis on the image foreground and background fingerprints, and comprehensively generating a quantitative similarity value similarity _ color based on image color information;
and step 3, content similarity analysis: performing quantitative similarity analysis on the image according to the result obtained by preprocessing in the step 1, performing threshold segmentation on the Y component based on an OTSU method in a YUV color space to obtain a content map of the image, calculating a joint gradient domain combined Grad, and calculating a quantitative similarity value similarity _ contents of the image based on content information according to the content map of the image and the joint gradient domain combined Grad;
and 4, analyzing comprehensive similarity of colors and contents: respectively carrying out interval division on RGB color spaces of the images img1 and img2, carrying out pixel-level interval clustering, respectively generating an image pixel type label map labelMap1 of the image img1 and an image pixel type label map labelMap2 of the image img2, respectively carrying out granularity roughening operation on labelMap1 and labelMap2, and respectively recording the results as labelMap New1 and labelMap New 2; calculating quantitative similarity values of images based on color and content comprehensive information according to labelmapNew1 and labelmapNew 2;
step 5, comprehensive operation: carrying out comprehensive operation on the quantitative similarity value similarity _ color based on the image color information, the quantitative similarity value similarity _ contents based on the content information and the quantitative similarity value similarity _ color contents based on the color and content comprehensive information obtained in the steps 2, 3 and 4 to obtain a comprehensive similarity value similarity _ pri of the image;
and 6, correcting the comprehensive similarity value similarity _ pri of the image: according to the result obtained by the preprocessing in the step 1, finely correcting the comprehensive similarity value similarity _ pri of the images to obtain the quantitative similarity value similarity _ val of the last two images;
the step 1 comprises the following steps:
reading two images to be compared img1 and img2 in an RGB form, reading EXIF information of the images, obtaining image time and GPS information, and respectively carrying out brightness equalization processing on img1 and img2, wherein the method comprises the following specific steps:
step 1-1, converting an image RGB space into a YUV space, and respectively obtaining the brightness components of two images, wherein the brightness component of img1 is Y1, and the brightness component of img2 is Y2;
step 1-2, respectively calculating the mean values of Y1 and Y2, and respectively recording the mean values as
Figure FDA0002291113900000021
Calculating the mean difference
Figure FDA0002291113900000022
Step 1-3, calculate new Y1, Y2 values:
if it is not
Figure FDA0002291113900000023
Then
Figure FDA0002291113900000024
Y2＝Y2，
If it is not
Figure FDA0002291113900000025
Then
Figure FDA0002291113900000026
Y1＝Y1，
Where (i, j) represents the position of the pixel unit in the original image, i represents the number of rows, j represents the number of columns, Y1(i, j) represents the pixel value at (i, j) corresponding to the luminance component Y1, and Y2(i, j) represents the pixel value at (i, j) corresponding to the luminance component Y2;
step 1-4, converting the YUV space into an RGB space, and respectively obtaining an image balanceImg1 corresponding to img1 and an image balanceImg2 corresponding to img 2;
the step 2 comprises the following steps:
step 2-1, downsampling img1 and img2 by using a salient object detection algorithm based on region comparison and carrying out salient region detection to obtain a foreground forego dImg1 and a background Img1 of img1, a foreground forego dImg2 and a background Img2 of img 2;
step 2-2, calculating fingerprints of foregroundImg1, backploulim 1, foregroundImg2 and backploulim 2, wherein the specific calculation steps are as follows: for the forego undimg1, performing interval division on an RGB color space, averagely dividing each channel of three channels of RGB into 8 intervals, namely, each interval is 32 in length, forming 512 combinations by eight intervals of each channel of the three channels, covering a part of the color space by each combination, mutually excluding the color spaces, covering the whole RGB color space by a union set of the eight intervals, initializing a 512-dimensional vector a, wherein each dimension corresponds to or marks one of the combinations, and each dimension is one interval; carrying out interval statistical clustering on each pixel of the foregroundImg1, and counting the number of the pixels of the foregroundImg1 in each interval, wherein the numerical value of the pixels is used as the value of the corresponding dimension of the vector a; normalizing the vector a to obtain a vector a1 as a forego and dImg1 fingerprint finger printFore 1; the same calculation steps are carried out on backsgroundimg 1, forkroundimg 2 and backsgroundimg 2 to obtain fingerprint fingerprintBack1 of backsgroundimg 1, fingerprint fingerprintform 2 of forkroundimg 2 and fingerprint fingerprintBack2 of backsgroundimg 2 respectively;
step 2-3, calculating the correlation coefficient between finger printform 1 and finger printform 2 as coeff _ form; calculating a correlation coefficient between finger print back1 and finger print back2 as coeff _ back, and finally calculating similarity _ color based on image color information as follows:
similarity_color＝cFore*coeff_fore+cBack*coeff_back，
wherein the content of the first and second substances,
Figure FDA0002291113900000031
cBack＝1-cFore，SforegroundImg1is the foreground area of Img1, Simg1Img1 area;
the step 3 comprises the following steps:
step 3-1, down-sampling and converting the balance img1 and the balance img2 obtained in the step 1 into YUV color space, respectively obtaining a luminance component YContents1 of the balance img1 and a luminance component YContents2 of the balance img2, and carrying out bilateral filtering processing on the YContents1 and the YContents 2;
step 3-2, threshold segmentation based on OTSU method is carried out on YContents1 and YContents2 to obtain content graphs of YContents1 and YContents2 respectively as fullContents1 and fullContents2, and content difference graphs contentsDiff are calculated by using the following formula:
contentsDiff＝fullContents1⊙fullContents2；
step 3-3, calculating gradient domains of YContents1 and YContents2 to obtain gradient maps grad1 and grad2 of YContents1 and YContents2 respectively, and calculating a combined gradient domain combinegrade by using the following formula:
combineGrad＝grad1.*grad2，
carrying out normalization processing on the combineGrad;
step 3-4, calculating similarity value similarity _ contents based on the image content information, wherein the specific formula is as follows:
Figure FDA0002291113900000032
step 4 comprises the following steps:
step 4-1, downsampling balance img1 and balance img2 to the same size and performing pixel level clustering: the method comprises the following steps of (1) carrying out interval division on an RGB color space, averagely dividing each channel of three channels of RGB into 2 intervals, wherein the three channels have 8 combinations, and each combination is marked by one label and has 8 labels; label labeling is carried out on each pixel in the balance img1 and the balance img2 to form a label image, namely label map1 and label map2, an image pixel class label image of the generated image img1 is labeled as label map1, and an image pixel class label image of the image img2 is labeled as label map 2;
step 4-2, performing grain size roughening operation on labelMap1 and labelMap2 respectively: calculating all communicated components of the label graph, finding out the label with the largest adjacent area of all the communicated components with the area smaller than one thousandth of the area of the label graph, marking the label as label Max, and marking all the labels of the communicated components as label Max; finally, the results of the particle size roughening operations performed by labelMap1 were recorded as labelMap new1, and the results of the particle size roughening operations performed by labelMap2 were recorded as labelMap new 2;
step 4-3, defining that if the label values of the corresponding pixel positions in labelMapNew1 and labelMapNew2 are equal, the pixels are overlapped, and the similarity value similarity _ colorContents based on the color and content comprehensive information is as follows:
Figure FDA0002291113900000041
wherein S iscoverOverlapped by labelMapNew1 and labelMapNew2Area, SlabelMapNew1Is the area of labelMapNew 1.
2. The method of claim 1, wherein step 5 comprises the steps of:
calculating an image comprehensive similarity value similarity _ pri by the following formula:
Figure FDA0002291113900000042
wherein, a is similarity _ color, b is similarity _ contents, c is similarity _ color contents, max () is maximum value function, min () is minimum value function, mid is median function, mean () is mean function.
3. The method of claim 2, wherein step 6 comprises the steps of:
step 6-1, calculating the time correlation coeff according to the image time information obtained in step 1timeThe calculation formula is as follows:
Figure FDA0002291113900000043
Figure FDA0002291113900000044
coefftime＝max(timeeffect*timecoeff+(1-timeeffect)*coeffpri，coeffpri)，
the digital camera comprises a camera, a camera module and a camera module, wherein the camera module is used for shooting two images, and the camera module is used for shooting two images; timeconst＝86400，timeeffect＝0.2，coeffpri＝similarity_pri；
Step 6-2, calculating 6PS correlation coeff according to the image GPS information obtained in the step 1gpsThe calculation formula is as follows:
Figure FDA0002291113900000045
Figure FDA0002291113900000046
coeffgps＝max(gpseffect*gpscoeff+(1-gpseffect)*coeffpri，coeffpri)，
the digital GPS is used for acquiring the shooting distance of two images, wherein the digital GPS is used for acquiring the shooting distance of the two images in meters; gpsconst＝5000，gpseffect＝0.2，coeffpri＝similarity_pri；
And 6-3, calculating the final similarity value similarity _ val of the two images, wherein the calculation formula is as follows:
Figure FDA0002291113900000051
Figure FDA0002291113900000052
CN201810263619.XA 2018-03-28 2018-03-28 Image similarity quantitative analysis method based on multi-factor synthesis of color and content Active CN108230409B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810263619.XA CN108230409B (en) 2018-03-28 2018-03-28 Image similarity quantitative analysis method based on multi-factor synthesis of color and content

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810263619.XA CN108230409B (en) 2018-03-28 2018-03-28 Image similarity quantitative analysis method based on multi-factor synthesis of color and content

Publications (2)

Publication Number Publication Date
CN108230409A CN108230409A (en) 2018-06-29
CN108230409B true CN108230409B (en) 2020-04-17

Family

ID=62659086

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810263619.XA Active CN108230409B (en) 2018-03-28 2018-03-28 Image similarity quantitative analysis method based on multi-factor synthesis of color and content

Country Status (1)

Country Link
CN (1) CN108230409B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109598299A (en) * 2018-11-29 2019-04-09 微梦创科网络科技（中国）有限公司 A kind of image similarity determines method, apparatus and electronic equipment
CN112613521B (en) * 2020-12-28 2023-01-20 上海埃林哲软件***股份有限公司 Multilevel data analysis system and method based on data conversion

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20100071822A (en) * 2008-12-19 2010-06-29 주식회사 케이티 Apparatus and method for detecting clothes in image
CN102012939A (en) * 2010-12-13 2011-04-13 中国人民解放军国防科学技术大学 Method for automatically tagging animation scenes for matching through comprehensively utilizing overall color feature and local invariant features
CN102663391A (en) * 2012-02-27 2012-09-12 安科智慧城市技术（中国）有限公司 Image multifeature extraction and fusion method and system
CN104392233A (en) * 2014-11-21 2015-03-04 宁波大学 Image saliency map extracting method based on region
CN107256412A (en) * 2017-05-26 2017-10-17 东南大学 A kind of figure building method based on many human eye perceptual grouping characteristics

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20100071822A (en) * 2008-12-19 2010-06-29 주식회사 케이티 Apparatus and method for detecting clothes in image
CN102012939A (en) * 2010-12-13 2011-04-13 中国人民解放军国防科学技术大学 Method for automatically tagging animation scenes for matching through comprehensively utilizing overall color feature and local invariant features
CN102663391A (en) * 2012-02-27 2012-09-12 安科智慧城市技术（中国）有限公司 Image multifeature extraction and fusion method and system
CN104392233A (en) * 2014-11-21 2015-03-04 宁波大学 Image saliency map extracting method based on region
CN107256412A (en) * 2017-05-26 2017-10-17 东南大学 A kind of figure building method based on many human eye perceptual grouping characteristics

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Color and texture fusion-based method for content-based Image Retrieval;Abdolraheem Khader Alhassan et al.;《2017 International Conference on Communication, Control, Computing and Electronics Engineering》;20171231;第1-6页 *
Quality Assessment of Image Fusion Based on Image Content and Structural Similarity;Zhang Xiao-lin et al.;《2010 2nd International Conference on Information Engineering and Computer Science》;20101226;第1-4页 *

Also Published As

Publication number Publication date
CN108230409A (en) 2018-06-29

Similar Documents

Publication Publication Date Title
CN109154978B (en) System and method for detecting plant diseases
CN108492343B (en) Image synthesis method for training data for expanding target recognition
CN102426649B (en) Simple steel seal digital automatic identification method with high accuracy rate
Chung et al. Efficient shadow detection of color aerial images based on successive thresholding scheme
Ban et al. Face detection based on skin color likelihood
Wong et al. Saliency-enhanced image aesthetics class prediction
Stokman et al. Selection and fusion of color models for image feature detection
CN107844736B (en) Iris positioning method and device
CN109740572A (en) A kind of human face in-vivo detection method based on partial color textural characteristics
Buza et al. Skin detection based on image color segmentation with histogram and k-means clustering
Liang et al. Salient object detection using content-sensitive hypergraph representation and partitioning
CN112991238B (en) Food image segmentation method, system and medium based on texture and color mixing
CN107392105B (en) Expression recognition method based on reverse collaborative salient region features
CN104636754A (en) Intelligent image classifying method based on tongue body partition color feature
Trivedi et al. Automatic segmentation of plant leaves disease using min-max hue histogram and k-mean clustering
CN111259756A (en) Pedestrian re-identification method based on local high-frequency features and mixed metric learning
CN111161281A (en) Face region identification method and device and storage medium
CN111260645A (en) Method and system for detecting tampered image based on block classification deep learning
CN111209873A (en) High-precision face key point positioning method and system based on deep learning
CN108230409B (en) Image similarity quantitative analysis method based on multi-factor synthesis of color and content
Bai et al. Principal pixel analysis and SVM for automatic image segmentation
CN112070116B (en) Automatic artistic drawing classification system and method based on support vector machine
WO2016192213A1 (en) Image feature extraction method and device, and storage medium
Youlian et al. Face detection method using template feature and skin color feature in rgb color space
Tan et al. Gesture segmentation based on YCb'Cr'color space ellipse fitting skin color modeling

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant