CN110910347A - Image segmentation-based tone mapping image no-reference quality evaluation method - Google Patents

Image segmentation-based tone mapping image no-reference quality evaluation method Download PDF

Info

Publication number
CN110910347A
CN110910347A CN201910993399.0A CN201910993399A CN110910347A CN 110910347 A CN110910347 A CN 110910347A CN 201910993399 A CN201910993399 A CN 201910993399A CN 110910347 A CN110910347 A CN 110910347A
Authority
CN
China
Prior art keywords
image
feature
tone
area
region
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910993399.0A
Other languages
Chinese (zh)
Other versions
CN110910347B (en
Inventor
池碧蔚
郁梅
徐海勇
宋洋
蒋刚毅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ningbo University
Original Assignee
Ningbo University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ningbo University filed Critical Ningbo University
Priority to CN201910993399.0A priority Critical patent/CN110910347B/en
Publication of CN110910347A publication Critical patent/CN110910347A/en
Application granted granted Critical
Publication of CN110910347B publication Critical patent/CN110910347B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/40—Analysis of texture
    • G06T7/41—Analysis of texture based on statistical description of texture
    • G06T7/45—Analysis of texture based on statistical description of texture using co-occurrence matrix computation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30168—Image quality inspection
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Quality & Reliability (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a tone mapping image quality evaluation method based on image segmentation, aiming at the characteristic that main distortion types of different areas of a tone mapping image are not the same, the tone mapping image is divided into a complex area and a flat area, texture detail features are extracted from the complex area, chrominance features are extracted from the flat area, and the texture detail features and the chrominance features are also extracted from a global area. Aiming at the characteristic that details of a highlight area and a low-dark area of an image are distorted too much, the image is divided into the highlight area, the low-dark area and other areas, information entropy characteristics are extracted in different areas respectively to represent the distortion degree of the image, then the brightness distribution uniformity degree of the image is judged by taking a threshold value of the highlight area and the low-dark area as a characteristic, characteristic values with good effects when the threshold value is evaluated in different areas are reserved, characteristic values with poor effects are removed, and characteristic redundancy is reduced; the relevance between the obtained objective evaluation result and the subjective perception of human eyes is effectively improved.

Description

Image segmentation-based tone mapping image no-reference quality evaluation method
Technical Field
The invention relates to an image quality evaluation technology, in particular to a tone mapping image non-reference quality evaluation method based on image segmentation.
Background
The Dynamic Range of a High-Dynamic Range (HDR) image is very High, and compared with a common image, the HDR image can provide more image details and restore a real scene; in recent years, high-dynamic images and imaging technologies thereof have been widely applied in the fields of movie special effects, professional photography, virtual reality, image rendering and the like. However, since professional equipment for acquisition and display is expensive, the application of high-motion images is difficult to popularize in society. In order to enable HDR images to be displayed on conventional Dynamic Range (SDR) display devices, they can be mapped onto SDR, typically by a Tone-mapping (TM) algorithm; in the process, a corresponding image quality degradation may be introduced; an effective quality evaluation tool is essential for obtaining high-quality TM images.
According to the degree of dependence on a reference image, the objective quality evaluation of the image can be divided into 3 types of full reference, half reference and no reference, wherein the method for evaluating the quality without reference has the strongest practicability and has larger research difficulty; representative examples of the full reference methods include 1) Tone-mapped image quality index (TMQI): the method comprises the steps of obtaining structural fidelity on the basis of a Structural Similarity Index (SSIM), then establishing a non-reference naturalness evaluation model, and finally weighting and summing the structural fidelity and the naturalness of an image to obtain a quality score of a TM image; the representative method of the no-reference method is 2) the index quality assessment of tone-mapped images (BTMQI), which is researched from three aspects of information entropy, naturalness and structure, and the maximum characteristic of the HDR image compared with the LDR image is that more detail information is reserved in a highlight area and a low-brightness area, so that the brightness of the TM image is respectively enlarged and reduced by 4 scales, and the information amount reserved in the highlight area, the low-brightness area and the global area of the TM image is judged by using the information entropy of each scale.
However, the currently used tone mapping image quality evaluation method lacks deep research on local areas of an image, especially flat and complex areas of the image and high-brightness and low-dark areas of the image, and lacks an effective measurement method for feature importance to screen and optimize feature values.
Disclosure of Invention
The invention aims to provide a tone mapping image quality evaluation method based on image segmentation, which can effectively improve the correlation between objective evaluation results and subjective perception quality of human eyes.
The technical scheme adopted by the invention for solving the technical problems is as follows: a no-reference tone mapping image quality evaluation method based on image segmentation comprises the following steps:
① selecting 747 ESPL-LIVE image libraries to obtain any tone mapping image, and converting the tone mapping image into gray level image;
② texture-dividing the tone-mapped image, and recording the divided complex region image as GCompFlat area image denoted as Gflat；
③, performing brightness segmentation on the gray level image by using a maximum entropy threshold segmentation method, and marking the segmented middle brightness region as N, the low dark region as D and the high bright region as B;
④ tensor resolution is performed on the tone mapped image, and the first descendant of the core tensor is recorded as G1For the complex area image GCompThe tensor resolution is carried out and the first filial generation of the core tensor is marked as C1；
⑤ pairs G1Calculating to obtain gray level gradient co-occurrence matrix, and calculating G on the gray level gradient co-occurrence matrix1Corresponding 15 common features, respectively small gradient dominance f1Great gradient advantage f2Distribution of gray scaleUniformity f3Gradient distribution nonuniformity f4Energy f5Correlation f6Entropy of gray scale f7Gradient entropy f8Mixed entropy f9Difference moment f10Inverse differential moment f11Mean value of gray level f12Mean value of gradient f13Standard deviation of gray scale f14Gradient standard deviation f15The 15 features are used as feature vectors
Figure BDA0002239004100000021
It is shown that,
Figure BDA0002239004100000022
calculating C1Corresponding 15 common features, and using the feature vector
Figure BDA0002239004100000023
It is shown that,
Figure BDA0002239004100000024
⑥ calculate 9 features of the tone-mapped image related to the color moments, the first moment of the RGB three channels of the tone-mapped image, the second moment of the RGB three channels of the tone-mapped image, f37-f39Respectively representing the three moments of the RGB three channels of the tone-mapped image and using these 9 features
Figure BDA0002239004100000025
It is shown that,
Figure BDA0002239004100000026
wherein f is31-f33；
For flat area image GflatThese 9 features are also computed and the feature vector is used
Figure BDA0002239004100000031
It is shown that,
Figure BDA0002239004100000032
wherein f is40-f42Respectively representing a flat area image GflatFirst moment of the three channels of RGB of (1), f43-f45Respectively representing a flat area image GflatSecond moment of the three channels of RGB of (1), f46-f48Respectively representing a flat area image GflatThe third moment of the RGB three channels of (1);
⑦ the gray level median G of the gray level imageMIDThreshold T for dividing highlight and intermediate luminance regionsBAnd threshold T for low dark and medium brightness regionsDAs a characteristic, is described as
Figure BDA0002239004100000033
Figure BDA0002239004100000034
Respectively calculating the information entropies of the three areas B, D and N, and recording the information entropies as characteristics
Figure BDA0002239004100000035
Figure BDA0002239004100000036
⑧ will be characterized by
Figure BDA0002239004100000037
And
Figure BDA0002239004100000038
merge into a feature of 30 dimensions in length
Figure BDA0002239004100000039
Figure BDA00022390041000000310
And to
Figure BDA00022390041000000311
Quantifying the importance degree of the characteristic value by using a random forest model to obtain a characteristic importance vector [ w ] with the length of 30 dimensions1,..wi,..w30]Wherein w isiRepresents fiAt f1-f30Degree of importance in dimensional features, followed by a comparison of w1And w16,w2And w17,...,w15And w30Size; when w is1>w16When, let k 11, otherwise k1When w is 162>w17When, let k22, otherwise k2When w is 1715>w30When, let k1515, otherwise k1530; finally, let FW be1＝FeatG+C(k1),...FWx＝FeatG+C(kx),...FW15＝FeatG+C(k15) Wherein, FeatG+C(kx) Is represented in
Figure BDA00022390041000000312
30-dimensional feature of (2)xIs characterized in that x is more than or equal to 15 and more than or equal to 1; for G1And in C1The feature with higher importance is screened out from the same feature of the texture mentioned in the section, and the screened texture feature is recorded as
Figure BDA00022390041000000313
Figure BDA00022390041000000314
Will be characterized by
Figure BDA00022390041000000315
And
Figure BDA00022390041000000316
merge into one feature of 18 dimensions in length
Figure BDA00022390041000000317
And to
Figure BDA00022390041000000319
By performing feature screening in the same manner as described above, for the same chromaticity feature in the tone-mapped image and the flat-region image, the feature having higher importance is screened out, and the texture feature after screening is recorded as
Figure BDA00022390041000000320
Figure BDA00022390041000000321
Wherein, FSjJ is more than or equal to 9 and more than or equal to 1 for the screened chromaticity characteristics;
⑨ the contrast of the tone-mapped image is calculated and recorded as a feature
Figure BDA00022390041000000322
⑩ will be
Figure BDA00022390041000000323
Are combined into a feature vector
Figure BDA00022390041000000324
Figure BDA00022390041000000325
Randomly selecting 600 images from 747 tone-mapped images according to steps ① to steps
Figure BDA00022390041000000326
In the same manner, to obtain a feature vector for each tone-mapped image
Figure BDA00022390041000000327
As input to the training set, denoted as ITRAIN，
Figure BDA0002239004100000041
M is more than or equal to 600 and more than or equal to 1, and the corresponding subjective score MOS is also used as the input of the training set and is recorded as OTRAIN，OTRAIN＝[MOS1,MOS2,...,MOSm]Constructing a model;
Figure BDA0002239004100000042
the tone-mapped image to be evaluated is converted into a grayscale image and then steps ② through steps
Figure BDA0002239004100000043
Calculated feature vector
Figure BDA0002239004100000044
The output of the model, denoted as MOS, is obtained as the input to the modelTAnd define MOSTA higher value of (d) represents a better quality of the image to be evaluated.
The specific texture segmentation method in step ② is as follows:
② _1a, performing edge extraction on the gray level image by using a canny operator, and recording the image after the edge extraction as a;
② _1b, expanding a to make the image form a connected region as much as possible, recording the expanded image as b,
Figure BDA0002239004100000045
wherein S is a disc with the radius of 1 pixel, and Z is the displacement generated when the expansion element S is translated;
② _1c, edge-filling b with a line segment of 10 pixels in length, and recording the filled image as c;
② _1d, filling the c by using a hole filling algorithm, and marking the filled image as d;
② _1e, removing a region with the area less than 1500 pixels or the length and the width less than 10 in the image d by using a denoising algorithm, and recording the denoised image as e;
② _1f, traversing the pixel points in e, recording the position set of the point with the pixel value of 255, taking the point with the same position in the tone mapping image to form a complex area image, and forming the rest points in the tone mapping image into a flat area image.
The specific brightness dividing method in step ③ is as follows:
③ _1a, calculating the grayscale median of grayscale image by image grayscale histogram, and recording as GMIDMake the gray value greater than GMIDIs denoted as GBIs less than GMIDIs denoted as GDAnd the probability for each gray value is found and is denoted as pi，i＝1...255；
③ _1b, at GBCalculating the maximum entropy threshold of gray scale according to the maximum entropy division method, and recording the obtained threshold as TBMake the gray value greater than TBThe area (D) is marked as a highlight area B, and the gray value is smaller than TBIs denoted as a first intermediate luminance region N1；
③ _1c, at GDCalculating the maximum entropy threshold of gray level in the region, and recording the obtained threshold as TDMake the gray value greater than TDIs designated as a second intermediate luminance region N2Make the gray value less than TDIs marked as a low dark region D, and N is1And N2Adding the areas into an area N; in summary, the image is divided into a high brightness region B, a low dark region D, and a middle brightness region N.
The specific tensor decomposition method in step ④ is as follows:
④ _1a, first image ITMExpressed as third order tensor
Figure BDA0002239004100000051
l1And l2Width and height of the image, respectively; then, the third-order tensor is expanded according to the mode 1, the mode 2 and the mode 3 respectively to obtain corresponding matrixes
Figure BDA0002239004100000052
④ _1b, performing SVD on the matrix to obtain an orthogonal matrix U(1),U(2),U(3)Respectively having a size of l1×l1,l2×l23 × 3; the specific calculation mode is [ U ](i),S(i),V(i)]＝SVD(X(i)) (i ═ 1,2,3), where SVD (·) represents the singular value decomposition of the function;
④ _1c, calculating image ITMCore tensor
Figure BDA0002239004100000053
G＝A×1(U(1))T×2(U(2))T×3(U(3))T…×n(U(n))T
Since the sub-tensors of the core tensor G satisfy the ordering, i.e. G1>G2>G3Here, G will be1Is defined as child one, G2Is defined as the child two, G3Defining as the third generation; according to the order of the core tensor, G1Carries more information and energy, so it is ready for G1And calculating the texture features.
The specific manner of the feature importance measure in step ⑧ is as follows:
⑧ _1a, the Gini index of the node m in each decision tree is calculated as
Figure BDA0002239004100000054
Where K denotes a feature having K classes, pkRepresenting the proportion of the class k in the node m;
⑧ _1b, recalculating feature XjThe significance of the node m, i.e., the amount of change in Gini index before and after the node m branches, is
Figure BDA0002239004100000055
Wherein, GIlAnd GIrRespectively representing Gini indexes of two new nodes after branching;
⑧ _1c, then calculating the importance of the features in a decision tree, the decision tree i will have the feature XjIs denoted as M, then XjThe importance of the ith tree is:
Figure BDA0002239004100000056
⑧ _1d, finally calculating the feature importance of all decision trees, assuming n trees in random forest
Figure BDA0002239004100000057
And finally, performing normalization processing on all the obtained importance scores:
Figure BDA0002239004100000058
the calculation method of the contrast C in the step ⑨ is as follows:
Figure BDA0002239004100000059
where g (x, y) represents the pixel value of point (x, y), BRI represents the mean value of the luminance of the image, and M and N represent the length and width of the image, respectively.
Compared with the prior art, the invention has the advantages that: the method considers the difference of main distortion types of tone mapping images in a complex area and a flat area, divides the images into the complex area and the flat area, and extracts different characteristic values in different areas, so that the subsequent quality characteristic extraction is more targeted.
The method provides image characteristics such as gray gradient co-occurrence matrix, color moment and the like which are different from the traditional image quality evaluation, so that the extracted characteristics can reflect the quality degradation degree of the tone mapping image more accurately.
The method fully considers the redundancy among the features, so the random forest model is used for sequencing the importance of the features, and the feature with better effect is selected from the global and local texture and chromaticity features, so that the feature with poor effect is removed, and the feature redundancy is reduced.
The method comprehensively considers the difference between the tone mapping image and the traditional image, and innovations and improvements are carried out from the three angles of region segmentation, quality feature extraction and feature dimension reduction, so that the correlation between the objective evaluation result obtained by the method and the subjective perception of human eyes is effectively improved.
In order to verify the effectiveness of the method for evaluating the quality of the tone mapping image, three evaluation indexes are selected as measures of the quality of the method, wherein the evaluation indexes are respectively Pearson Linear Correlation Coefficient (PLCC), Spearman sequential correlation coefficient (SROCC) and Root Mean Square Error (RMSE) which respectively represent the correlation between the predicted fraction and the actual fraction. PLCC and SROCC have values between (0,1), and the closer to 1 the better, the smaller the RMSE the better.
Taking 147 residual color tone mapping images to be evaluated under ESPL-LIVE
Figure BDA0002239004100000061
Feature vector of
Figure BDA0002239004100000062
When the input is taken as the input of the test set, 747 is more than or equal to l and more than or equal to 601, the MOS value obtained by predicting each image is taken as the output and is recorded as PMOSlThe PMOS values of 147 pictures are taken as a set RF _ output, which is [ PMOS ═ PMOS1,PMOS2,...,PMOSl]Then 147 tone-mapped images to be evaluated
Figure BDA0002239004100000063
The MOS value of (1) is recorded as the set output _ test, which is [ MOS ] value1,MOS2,...,MOSl](ii) a The correlation coefficients PLCC, SROCC and RMSE are found using the IQA function, where f is IQA (RF _ output, output _ test). Wherein IQA (: indicates a fitting function, and f indicates PLCC, SROCC and RMSE correlation coefficients.
As can be seen from the data listed in Table 1, the objective quality evaluation predicted value of the tone mapping image calculated by the method of the present invention has good correlation with the average subjective score difference, wherein the PLCC correlation coefficient reaches 0.8252, the SROCC correlation coefficient reaches 0.7795, and the RMSE reaches 5.7833.
TABLE 1 Performance indicators for the correlation between the objective quality evaluation prediction value and the average subjective score difference for tone mapped images in a test image set calculated according to the method of the present invention
Type of index PLCC SROCC RMSE
End result 0.8252 0.7795 5.7833
Drawings
FIG. 1 is a block diagram of an overall implementation of the method of the present invention;
FIG. 2 is a gray scale gradient co-occurrence matrix feature table proposed by the method of the present invention.
Detailed Description
The invention is described in further detail below with reference to the embodiments of the drawings;
example (b): a no-reference tone mapping image quality evaluation method based on image segmentation comprises the following steps:
① selecting 747 ESPL-LIVE image libraries to obtain any tone mapping image marked as ITMTone mapping the image ITMConversion into a grayscale image, denoted as IG-TM；
② pairs tone-mapped image ITMPerforming texture segmentation, and recording the segmented complex region image as GCompFlat area image denoted as Gflat(ii) a The method specifically comprises the following steps:
② _1a, performing edge extraction on the gray level image by using a canny operator, and recording the image after the edge extraction as a;
② _1b, expanding a to make the image form a connected region as much as possible, recording the expanded image as b,
Figure BDA0002239004100000071
wherein S is a disc with the radius of 1 pixel, and Z is the displacement generated when the expansion element S is translated;
② _1c, edge-filling b with a line segment of 10 pixels in length, and recording the filled image as c;
② _1d, filling the c by using a hole filling algorithm, and marking the filled image as d;
② _1e, removing a region with the area less than 1500 pixels or the length and the width less than 10 in the image d by using a denoising algorithm, and recording the denoised image as e;
② _1f, traversing the pixel points in e, recording the position set of the point with the pixel value of 255, taking the point with the same position in the tone mapping image to form a complex area image, and forming the rest points in the tone mapping image into a flat area image;
③ pairs of grayscale images IG-TMPerforming brightness segmentation by using a maximum entropy threshold segmentation method, and recording a segmented middle brightness region as N, a low dark region as D and a high bright region as B; the method specifically comprises the following steps:
③ _1a, calculating the grayscale median of grayscale image by image grayscale histogram, and recording as GMIDMake the gray value greater than GMIDIs denoted as GBIs less than GMIDIs denoted as GDAnd the probability for each gray value is found and is denoted as pi，i＝1...255；
③ _1b, at GBRegion GBThe regions were determined according to the literature [ Kapur N, Sahoo P, Wong A. "A new method for gray-level picture thresholding using entropy of the histogram,"ComputerVision,Graphics,and Image Processing,vol.29,pp.273-285,1985]Calculating the maximum entropy threshold of gray scale by the maximum entropy division method, and recording the obtained threshold as TBMake the gray value greater than TBThe area (D) is marked as a highlight area B, and the gray value is smaller than TBIs denoted as a first intermediate luminance region N1；
③ _1c, at GDCalculating the maximum entropy threshold of gray level in the region, and recording the obtained threshold as TDMake the gray value greater than TDIs designated as a second intermediate luminance region N2Make the gray value less than TDIs marked as a low dark region D, and N is1And N2Adding the areas into an area N; in conclusion, the image is divided into a high brightness area B, a low dark area D and a middle brightness area N;
④ pairs tone-mapped image ITMCarrying out tensor decomposition, and recording the first filial generation of the core tensor as G1For the complex area image GCompThe tensor resolution is carried out and the first filial generation of the core tensor is marked as C1(ii) a The specific way of tensor decomposition is:
④ _1a, first image ITMExpressed as third order tensor
Figure BDA0002239004100000081
l1And l2Width and height of the image, respectively; then, the third-order tensor is expanded according to the mode 1, the mode 2 and the mode 3 respectively to obtain corresponding matrixes
Figure BDA0002239004100000082
④ _1b, performing SVD on the matrix to obtain an orthogonal matrix U(1),U(2),U(3)Respectively having a size of l1×l1,l2×l23 × 3; the specific calculation mode is [ U ](i),S(i),V(i)]＝SVD(X(i)) (i ═ 1,2,3), where SVD (·) represents the singular value decomposition of the function;
④_1c、computing an image ITMCore tensor
Figure BDA0002239004100000083
G＝A×1(U(1))T×2(U(2))T×3(U(3))T…×n(U(n))T
Since the sub-tensors of the core tensor G satisfy the ordering, i.e. G1>G2>G3Here, G will be1Is defined as child one, G2Is defined as the child two, G3Defining as the third generation; according to the order of the core tensor, G1Carries more information and energy, so it is ready for G1Calculating the texture features;
⑤ pairs G1Calculating to obtain gray level gradient co-occurrence matrix, and calculating G on the gray level gradient co-occurrence matrix1Corresponding 15 common features, respectively small gradient dominance f1Great gradient advantage f2Non-uniformity of gray distribution f3Gradient distribution nonuniformity f4Energy f5Correlation f6Entropy of gray scale f7Gradient entropy f8Mixed entropy f9Difference moment f10Inverse differential moment f11Mean value of gray level f12Mean value of gradient f13Standard deviation of gray scale f14Gradient standard deviation f15The specific calculation method is described in the literature [ Hongtong light, gray level-gradient co-occurrence matrix texture analysis method [ J]Journal of Automation, 1984(01):22-25.]The 15 features are used as feature vectors
Figure BDA0002239004100000091
It is shown that,
Figure BDA0002239004100000092
calculating C1Corresponding 15 common features, and using the feature vector
Figure BDA0002239004100000093
It is shown that,
Figure BDA0002239004100000094
⑥ calculating tone mapped image ITM9 features relating to the color moments, respectively tone-mapped image ITMRGB first moment, tone-mapped image I of three channelsTMSecond moment of the three channels of RGB of (1), f37-f39Respectively representing tone-mapped images ITMAnd use these 9 features as the third moment of the RGB three channels
Figure BDA0002239004100000095
It is shown that,
Figure BDA0002239004100000096
wherein f is31-f33；
For flat area image GflatThese 9 features are also computed and the feature vector is used
Figure BDA0002239004100000097
It is shown that,
Figure BDA0002239004100000098
wherein f is40-f42Respectively representing a flat area image GflatFirst moment of the three channels of RGB of (1), f43-f45Respectively representing a flat area image GflatSecond moment of the three channels of RGB of (1), f46-f48Respectively representing a flat area image GflatThe third moment of the RGB three channels of (1);
⑦ grayscale image IG-TMGray median value G ofMIDThreshold T for dividing highlight and intermediate luminance regionsBAnd threshold T for low dark and medium brightness regionsDAs a characteristic, is described as
Figure BDA0002239004100000099
Figure BDA00022390041000000910
Are respectively provided withCalculating the information entropy of the three areas B, D and N, and recording as the characteristics
Figure BDA00022390041000000911
Figure BDA00022390041000000912
⑧ will be characterized by
Figure BDA00022390041000000913
And
Figure BDA00022390041000000914
merge into a feature of 30 dimensions in length
Figure BDA00022390041000000915
Figure BDA00022390041000000916
And to
Figure BDA00022390041000000917
Using a random forest model to measure the feature importance to obtain a feature importance vector [ w ] with the length of 30 dimensions1,..wi,..w30]Wherein w isiRepresents fiAt f1-f30Degree of importance in dimensional features, followed by a comparison of w1And w16,w2And w17,...,w15And w30Size; when w is1>w16When, let k 11, otherwise k1When w is 162>w17When, let k22, otherwise k2When w is 1715>w30When, let k1515, otherwise k1530; finally, let FW be1＝FeatG+C(k1),...FWx＝FeatG+C(kx),...FW15＝FeatG+C(k15) Wherein, FeatG+C(kx) Is represented in
Figure BDA0002239004100000101
30-dimensional feature of (2)xIs characterized in that x is more than or equal to 15 and more than or equal to 1; for G1And in C1The feature with higher importance is screened out from the same feature of the texture mentioned in the section, and the screened texture feature is recorded as
Figure BDA0002239004100000102
Figure BDA0002239004100000103
Will be characterized by
Figure BDA0002239004100000104
And
Figure BDA0002239004100000105
merge into one feature of 18 dimensions in length
Figure BDA0002239004100000106
Figure BDA0002239004100000107
And to
Figure BDA0002239004100000108
Image I is tone-mapped by feature screening in the same manner as described aboveTMAnd in flat area image GflatThe feature with higher importance is screened out from the same chromaticity feature in the image, and the screened texture feature is recorded as the texture feature
Figure BDA0002239004100000109
Figure BDA00022390041000001010
Wherein, FSjJ is more than or equal to 9 and more than or equal to 1 for the screened chromaticity characteristics; the specific way of the feature importance measure is as follows:
⑧ _1a, the Gini index of the node m in each decision tree is calculated as
Figure BDA00022390041000001011
Where K denotes a feature having K classes, pkRepresenting the proportion of the class k in the node m;
⑧ _1b, recalculating feature XjThe significance of the node m, i.e., the amount of change in Gini index before and after the node m branches, is
Figure BDA00022390041000001012
Wherein, GIlAnd GIrRespectively representing Gini indexes of two new nodes after branching;
⑧ _1c, then calculating the importance of the features in a decision tree, the decision tree i will have the feature XjIs denoted as M, then XjThe importance of the ith tree is:
Figure BDA00022390041000001013
⑧ _1d, finally calculating the feature importance of all decision trees, assuming n trees in random forest
Figure BDA00022390041000001014
And finally, performing normalization processing on all the obtained importance scores:
Figure BDA00022390041000001015
⑨ calculating tone mapped image ITMContrast of (2), which is taken as a feature
Figure BDA00022390041000001016
The contrast C is calculated in the following manner:
Figure BDA00022390041000001017
wherein g (x, y) represents a pixel value of the point (x, y), BRI represents a luminance mean value of the image, and M and N represent a length and a width of the image, respectively;
⑩ will be
Figure BDA0002239004100000111
Are combined into a feature vector
Figure BDA0002239004100000112
Figure BDA0002239004100000113
Tone mapping image I at 747 sheets randomlyTMSelecting 600 images according to the steps ① to ①
Figure BDA0002239004100000114
In the same manner to obtain each tone-mapped image ITMFeature vector of
Figure BDA0002239004100000115
As input to the training set, denoted as ITRAIN，
Figure BDA0002239004100000116
M is more than or equal to 600 and more than or equal to 1, and the corresponding subjective score MOS is also used as the input of the training set and is recorded as OTRAIN，OTRAIN＝[MOS1,MOS2,...,MOSm]Constructing a model;
Figure BDA0002239004100000117
the tone-mapped image to be evaluated is converted into a grayscale image and then steps ② through steps
Figure BDA0002239004100000118
Calculated feature vector
Figure BDA0002239004100000119
The output of the model, denoted as MOS, is obtained as the input to the modelTAnd define MOSTA higher value of (d) represents a better quality of the image to be evaluated.

Claims (6)

1. A no-reference tone mapping image quality evaluation method based on image segmentation is characterized by comprising the following steps:
① selecting 747 ESPL-LIVE image libraries to obtain any tone mapping image, and converting the tone mapping image into gray level image;
② texture-dividing the tone-mapped image, and recording the divided complex region image as GCompFlat area image denoted as Gflat；
③, performing brightness segmentation on the gray level image by using a maximum entropy threshold segmentation method, and marking the segmented middle brightness region as N, the low dark region as D and the high bright region as B;
④ tensor resolution is performed on the tone mapped image, and the first descendant of the core tensor is recorded as G1For the complex area image GCompThe tensor resolution is carried out and the first filial generation of the core tensor is marked as C1；
⑤ pairs G1Calculating to obtain gray level gradient co-occurrence matrix, and calculating G on the gray level gradient co-occurrence matrix1Corresponding 15 common features, respectively small gradient dominance f1Great gradient advantage f2Non-uniformity of gray distribution f3Gradient distribution nonuniformity f4Energy f5Correlation f6Entropy of gray scale f7Gradient entropy f8Mixed entropy f9Difference moment f10Inverse differential moment f11Mean value of gray level f12Mean value of gradient f13Standard deviation of gray scale f14Gradient standard deviation f15The 15 features are used as feature vectors
Figure FDA0002239004090000011
It is shown that,
Figure FDA0002239004090000012
calculating C1Corresponding 15 common features, and using the feature vector
Figure FDA0002239004090000013
It is shown that,
Figure FDA0002239004090000014
⑥ calculate 9 features of the tone-mapped image related to the color moments, the first moment of the RGB three channels of the tone-mapped image, the second moment of the RGB three channels of the tone-mapped image, f37-f39Respectively representing the three moments of the RGB three channels of the tone-mapped image and using these 9 features
Figure FDA0002239004090000015
It is shown that,
Figure FDA0002239004090000016
wherein f is31-f33；
For flat area image GflatThese 9 features are also computed and the feature vector is used
Figure FDA0002239004090000017
It is shown that,
Figure FDA0002239004090000018
wherein f is40-f42Respectively representing a flat area image GflatFirst moment of the three channels of RGB of (1), f43-f45Respectively representing a flat area image GflatSecond moment of the three channels of RGB of (1), f46-f48Respectively representing a flat area image GflatThe third moment of the RGB three channels of (1);
⑦ the gray level median G of the gray level imageMIDThreshold T for dividing highlight and intermediate luminance regionsBAnd threshold T for low dark and medium brightness regionsDAs a characteristic, is described as
Figure FDA0002239004090000021
Figure FDA0002239004090000022
Respectively calculating the information entropies of the three areas B, D and N, and recording the information entropies as characteristics
Figure FDA0002239004090000023
Figure FDA0002239004090000024
⑧ will be characterized by
Figure FDA0002239004090000025
And
Figure FDA0002239004090000026
merge into a feature of 30 dimensions in length
Figure FDA0002239004090000027
Figure FDA0002239004090000028
And to
Figure FDA0002239004090000029
Quantifying the importance degree of the characteristic value by using a random forest model to obtain a characteristic importance vector [ w ] with the length of 30 dimensions1,..wi,..w30]Wherein w isiRepresents fiAt f1-f30Degree of importance in dimensional features, followed by a comparison of w1And w16,w2And w17,...,w15And w30Size; when w is1>w16When, let k11, otherwise k1When w is 162>w17When, let k22, otherwise k2When w is 1715>w30When, let k1515, otherwise k1530; finally, let FW be1＝FeatG+C(k1),...FWx＝FeatG+C(kx),...FW15＝FeatG+C(k15) Wherein, FeatG+C(kx) Is represented in
Figure FDA00022390040900000210
30-dimensional feature of (2)xIs characterized in that x is more than or equal to 15 and more than or equal to 1; for G1And in C1The feature with higher importance is screened out from the same feature of the texture mentioned in the section, and the screened texture feature is recorded as
Figure FDA00022390040900000211
Figure FDA00022390040900000212
Will be characterized by
Figure FDA00022390040900000213
And
Figure FDA00022390040900000214
merge into one feature of 18 dimensions in length
Figure FDA00022390040900000215
Figure FDA00022390040900000216
And to
Figure FDA00022390040900000217
By performing feature screening in the same manner as described above, for the same chromaticity feature in the tone-mapped image and the flat-region image, the feature having higher importance is screened out, and the texture feature after screening is recorded as
Figure FDA00022390040900000218
Figure FDA00022390040900000219
Wherein, FSjJ is more than or equal to 9 and more than or equal to 1 for the screened chromaticity characteristics;
⑨ the contrast of the tone-mapped image is calculated and recorded as a feature
Figure FDA00022390040900000220
⑩ will be
Figure FDA00022390040900000221
Are combined into a feature vector
Figure FDA00022390040900000222
Figure FDA00022390040900000225
Randomly selecting 600 images from 747 tone-mapped images according to steps ① to steps
Figure FDA00022390040900000226
In the same manner, to obtain a feature vector for each tone-mapped image
Figure FDA00022390040900000223
As input to the training set, denoted as ITRAIN，
Figure FDA00022390040900000224
M is more than or equal to 600 and more than or equal to 1, and the corresponding subjective score MOS is also used as the input of the training set and is recorded as OTRAIN，OTRAIN＝[MOS1,MOS2,...,MOSm]Constructing a model;
Figure FDA0002239004090000033
the tone-mapped image to be evaluated is converted into a grayscale image and then steps ② through steps
Figure FDA0002239004090000034
Calculated feature vector
Figure FDA0002239004090000031
The output of the model, denoted as MOS, is obtained as the input to the modelTAnd define MOSTA higher value of (d) represents a better quality of the image to be evaluated.
2. The method according to claim 1, wherein the texture segmentation method in step ② is as follows:
② _1a, performing edge extraction on the gray level image by using a canny operator, and recording the image after the edge extraction as a;
② _1b, expanding a to make the image form a connected region as much as possible, recording the expanded image as b,
Figure FDA0002239004090000032
wherein S is a disc with the radius of 1 pixel, and Z is the displacement generated when the expansion element S is translated;
② _1c, edge-filling b with a line segment of 10 pixels in length, and recording the filled image as c;
② _1d, filling the c by using a hole filling algorithm, and marking the filled image as d;
② _1e, removing a region with the area less than 1500 pixels or the length and the width less than 10 in the image d by using a denoising algorithm, and recording the denoised image as e;
② _1f, traversing the pixel points in e, recording the position set of the point with the pixel value of 255, taking the point with the same position in the tone mapping image to form a complex area image, and forming the rest points in the tone mapping image into a flat area image.
3. The method according to claim 1, wherein the specific brightness division manner in step ③ is as follows:
③ _1a, calculating the grayscale median of grayscale image by image grayscale histogram, and recording as GMIDMake the gray value greater than GMIDIs denoted as GBIs less than GMIDIs denoted as GDAnd the probability for each gray value is found and is denoted as pi，i＝1...255；
③ _1b, at GBCalculating the maximum entropy threshold of gray scale according to the maximum entropy division method, and recording the obtained threshold as TBMake the gray value greater than TBThe area (D) is marked as a highlight area B, and the gray value is smaller than TBIs denoted as a first intermediate luminance region N1；
③ _1c, at GDCalculating the maximum entropy threshold of gray level in the region, and recording the obtained threshold as TDMake the gray value greater than TDIs designated as a second intermediate luminance region N2Make the gray value less than TDIs marked as a low dark region D, and N is1And N2Adding the areas into an area N; in summary, the image is divided into a high brightness region B, a low dark region D, and a middle brightness region N.
4. The method for evaluating the quality of a tone-mapped image based on exposure analysis according to claim 1 or 2, wherein the specific tensor resolution in step ④ is as follows:
④ _1a, first image ITMExpressed as third order tensor
Figure FDA0002239004090000041
l1And l2Width and height of the image, respectively; then, the third-order tensor is expanded according to the mode 1, the mode 2 and the mode 3 respectively to obtain corresponding matrixes
Figure FDA0002239004090000042
④ _1b, performing SVD on the matrix to obtain an orthogonal matrix U(1),U(2),U(3)Size division ofIs other than1×l1,l2×l23 × 3; the specific calculation mode is [ U ](i),S(i),V(i)]＝SVD(X(i)) (i ═ 1,2,3), where SVD (·) represents the singular value decomposition of the function;
④ _1c, calculating image ITMCore tensor
Figure FDA0002239004090000043
G＝A×1(U(1))T×2(U(2))T×3(U(3))T…×n(U(n))T
Since the sub-tensors of the core tensor G satisfy the ordering, i.e. G1>G2>G3Here, G will be1Is defined as child one, G2Is defined as the child two, G3Defining as the third generation; according to the order of the core tensor, G1Carries more information and energy, so it is ready for G1And calculating the texture features.
5. The method of claim 1, wherein the measure of importance of the features in step ⑧ is determined by:
⑧ _1a, the Gini index of the node m in each decision tree is calculated as
Figure FDA0002239004090000044
Where K denotes a feature having K classes, pkRepresenting the proportion of the class k in the node m;
⑧ _1b, recalculating feature XjThe significance of the node m, i.e., the amount of change in Gini index before and after branching of the node m, is VIMjgmini＝GIm-GIl-GIrWherein, GIlAnd GIrRespectively representing Gini indexes of two new nodes after branching;
⑧_1cthen calculating the feature importance of a decision tree, and enabling the decision tree i to have the feature XjIs denoted as M, then XjThe importance of the ith tree is:
Figure FDA0002239004090000045
⑧ _1d, finally calculating the feature importance of all decision trees, assuming n trees in random forest
Figure FDA0002239004090000046
And finally, performing normalization processing on all the obtained importance scores:
Figure FDA0002239004090000047
6. the method according to claim 1, wherein the contrast C in step ⑨ is calculated by:
Figure FDA0002239004090000051
where g (x, y) represents the pixel value of point (x, y), BRI represents the mean value of the luminance of the image, and M and N represent the length and width of the image, respectively.
CN201910993399.0A 2019-10-18 2019-10-18 Tone mapping image non-reference quality evaluation method based on image segmentation Active CN110910347B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910993399.0A CN110910347B (en) 2019-10-18 2019-10-18 Tone mapping image non-reference quality evaluation method based on image segmentation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910993399.0A CN110910347B (en) 2019-10-18 2019-10-18 Tone mapping image non-reference quality evaluation method based on image segmentation

Publications (2)

Publication Number Publication Date
CN110910347A true CN110910347A (en) 2020-03-24
CN110910347B CN110910347B (en) 2023-06-06

Family

ID=69814601

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910993399.0A Active CN110910347B (en) 2019-10-18 2019-10-18 Tone mapping image non-reference quality evaluation method based on image segmentation

Country Status (1)

Country Link
CN (1) CN110910347B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113362354A (en) * 2021-05-07 2021-09-07 安徽国际商务职业学院 Method, system, terminal and storage medium for evaluating quality of tone mapping image
WO2021223540A1 (en) * 2020-05-08 2021-11-11 华为技术有限公司 Processing method, encoding device, and decoding device for high dynamic range (hdr) video
CN116894851A (en) * 2023-09-11 2023-10-17 吉林农业大学 Soil CT image pore extraction method and system

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120201456A1 (en) * 2009-10-08 2012-08-09 International Business Machines Corporation Transforming a digital image from a low dynamic range (ldr) image to a high dynamic range (hdr) image
CN103353982A (en) * 2013-05-15 2013-10-16 中山大学 Method for tone mapping based on histogram equalization
CN108010024A (en) * 2017-12-11 2018-05-08 宁波大学 It is a kind of blind with reference to tone mapping graph image quality evaluation method

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120201456A1 (en) * 2009-10-08 2012-08-09 International Business Machines Corporation Transforming a digital image from a low dynamic range (ldr) image to a high dynamic range (hdr) image
CN103353982A (en) * 2013-05-15 2013-10-16 中山大学 Method for tone mapping based on histogram equalization
CN108010024A (en) * 2017-12-11 2018-05-08 宁波大学 It is a kind of blind with reference to tone mapping graph image quality evaluation method

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
侯春萍;李浩;岳广辉: "局部和全局特征融合的色调映射图像质量评价" *
刘佳;郁梅;姜浩;马华林;: "结合美学特征的无参考色调映射图像质量评价" *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021223540A1 (en) * 2020-05-08 2021-11-11 华为技术有限公司 Processing method, encoding device, and decoding device for high dynamic range (hdr) video
CN113362354A (en) * 2021-05-07 2021-09-07 安徽国际商务职业学院 Method, system, terminal and storage medium for evaluating quality of tone mapping image
CN113362354B (en) * 2021-05-07 2024-04-30 安徽国际商务职业学院 Quality evaluation method, system, terminal and storage medium for tone mapping image
CN116894851A (en) * 2023-09-11 2023-10-17 吉林农业大学 Soil CT image pore extraction method and system
CN116894851B (en) * 2023-09-11 2023-11-14 吉林农业大学 Soil CT image pore extraction method and system

Also Published As

Publication number Publication date
CN110910347B (en) 2023-06-06

Similar Documents

Publication Publication Date Title
CN110046673B (en) No-reference tone mapping image quality evaluation method based on multi-feature fusion
Wang et al. Simple low-light image enhancement based on Weber–Fechner law in logarithmic space
CN101853504B (en) Image quality evaluating method based on visual character and structural similarity (SSIM)
CN109978854B (en) Screen content image quality evaluation method based on edge and structural features
CN112381897B (en) Low-illumination image enhancement method based on self-coding network structure
Tian et al. A multi-order derivative feature-based quality assessment model for light field image
CN110910347A (en) Image segmentation-based tone mapping image no-reference quality evaluation method
CN112950596B (en) Tone mapping omnidirectional image quality evaluation method based on multiple areas and multiple levels
CN113822830B (en) Multi-exposure image fusion method based on depth perception enhancement
CN105513033B (en) A kind of super resolution ratio reconstruction method that non local joint sparse indicates
CN109218716B (en) No-reference tone mapping image quality evaluation method based on color statistics and information entropy
CN110706196B (en) Clustering perception-based no-reference tone mapping image quality evaluation algorithm
Chen et al. Blind quality index for tone-mapped images based on luminance partition
Steffens et al. Cnn based image restoration: Adjusting ill-exposed srgb images in post-processing
Wang et al. Low-light image enhancement based on virtual exposure
CN107146220A (en) A kind of universal non-reference picture quality appraisement method
CN104346809A (en) Image quality evaluation method for image quality dataset adopting high dynamic range
Feng et al. Low-light image enhancement algorithm based on an atmospheric physical model
Xu et al. Remote-sensing image usability assessment based on ResNet by combining edge and texture maps
CN113409247B (en) Multi-exposure fusion image quality evaluation method
Zhao et al. A multi-scale U-shaped attention network-based GAN method for single image dehazing
CN111047618A (en) Multi-scale-based non-reference screen content image quality evaluation method
Dwivedi et al. Single image dehazing using extended local dark channel prior
CN112132774A (en) Quality evaluation method of tone mapping image
Zheng et al. Overwater image dehazing via cycle-consistent generative adversarial network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant