CN104023230A - Non-reference image quality evaluation method based on gradient relevance - Google Patents

Abstract

The invention provides a non-reference image quality evaluation method based on gradient relevance, and belongs to computer image analysis. The method comprises the following steps: firstly, determining three sub-characters, namely, an image gradient amplitude character, an image gradient direction change character and an image gradient amplitude change character, which are easily influenced by distortion in an image gradient; performing M*M image blocking on the three characters, and determining a statistic variance of the three characters in each image block, wherein the average number of the statistic variances of all the image blocks in the whole image is taken as an image characteristic; and finally, determining the image quality by combining a support vector with a two-step frame in image quality evaluation. The method provided by the invention has the advantages of being small in time complexity, high in subjective consistency, low in image feature dimension and good in universality; the method can be applied to applications related to small computer devices or image quality, and is high in practical value.

Description

A kind of non-reference picture quality appraisement method based on gradient relevance

Technical field

The present invention relates to a kind of image quality evaluating method, relate in particular to a kind of non-reference picture quality appraisement method based on gradient relevance, belong to image analysis technology field.

Background technology

Vision is the human cognitive world one of the most effective approach, and image is based upon on human vision basis.Image information can accurately directly help people to obtain this information environment to be expressed and the meaning, and this is that the information such as voice, word are incomparable.So art of image analysis has very consequence in computer research field.Recording image the state of a certain environment at a time, in the process of this recording status, there will be a little error, such as take pictures can be subject in process taking a picture impact, the X-ray image of hand tremor of camera can be subject to impact of X ray power and film etc. in imaging process.These impacts can be arrived the definition of imaging by direct interference, and then make the information reduction comprising in the middle of image and introduce distortion.

Image quality evaluating method is generally divided into subjective evaluation method and method for objectively evaluating.Usually, subjective evaluation method is individual to be evaluated to the quality score of this sub-picture to the visual experience of piece image based on self, and averages as the final score value of this sub-picture for the score value of same piece image many people.Although this evaluation method is the most accurately, the human resources of its consumption are large, and the time consuming time is long, are also easy to be subject to the impact of the factors such as knowledge, viewpoint between evaluation personnel simultaneously.Method for objectively evaluating is appliance computer evaluation map image quality, has got rid of the factor that personnel participate in, and assess effectiveness is increased greatly.

On the whole, method for evaluating objective quality is that Applied Computer Techniques substitutes human factor among subjective evaluation method, makes the result of result that method for evaluating objective quality evaluates and subjective evaluation method close.Finally complete the process of computer simulation human perception picture signal.Method for evaluating objective quality all has application in many aspects: 1. as image co-registration, cut apart the evaluation algorithms of the effect of scheduling algorithm; 2. as the preposition algorithm of image processing algorithm, for after algorithm initial value etc. is provided; 3. weigh the quality of communication channel; 4. be embedded into and apply in the middle of compact image collecting device etc.

Method for evaluating objective quality can be divided three classes again: full reference image quality appraisement method, partial reference image quality appraisement method and non-reference picture quality appraisement method.As its name suggests, full reference image quality appraisement method not only needs distorted image, also needs and the corresponding not distorted image of distorted image.Partial reference image quality appraisement method is the partial information of contrast distorted image and the corresponding not distorted image of distorted image, as feature of extracting etc., evaluates the quality of distorted image.Non-reference picture quality appraisement is only to need this kind of information of distorted image to evaluate the quality of distorted image.In the middle of practical application, with non-reference picture quality appraisement, there is the most Practical significance because in actual applications, with the corresponding original image of distorted image be very unobtainable.

In sum, carry out thering is theory significance and important using value widely for the research of method for evaluating objective quality.The people such as Moorthy have proposed two step frameworks of non-reference picture quality appraisement in document < < A two-step framework for constructing blind image quality indices > >, and the basic background technology relating to is mainly image gradient character and soble operators.

(1) two step frameworks of non-reference picture quality appraisement

The people such as Moorthy propose two step frameworks of non-reference picture quality appraisement.In this framework, first the distorted image of input is classified, at the mark that this distorted image is predicted in each class distortion, be weighted summation afterwards.

When providing an image training set with type of distortion in n, first need to set up the mapping between characteristics of image and distortion classification, in training pattern, be in model, to input correct distortion classification and characteristics of image, the training of distortion disaggregated model is completed, can obtain image fault by this mode input characteristics of image afterwards and classify.

Special needs to be pointed out is, in this disaggregated model, need to set up a hard grader, what need is a distortion that this image can be described shared probability in the middle of every kind of type of distortion, to obtain the vectorial p of a n dimension thus, the every dimension value in p has just represented that the distortion of input picture is at every kind of probability that definite type of distortion is shared.

Afterwards, training, for the regression model of every kind of type of distortion, is the mapping of setting up between characteristics of image and picture quality.When training, image training set is divided into n part, and every portion only comprises a kind of image of definite type of distortion, therefore, in the time of need to training distortion that n regression model predict to exist in input picture for certain particular type, the picture quality mark under this type.This can strengthen the accuracy of mapping greatly.

By in n regression model of tape test image input, will obtain n mass fraction, these mass fractions are become to the quality vector q of n dimension according to the corresponding order of category of model vector p.

Finally, use distortion class vector in image to be weighted summation these mass fractions, thereby obtain objective prediction mark

$Q=\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}{p}_i{q}_i---\left(1\right)$

Wherein, p _ithe i dimension component that represents vectorial p, q _ithe i dimension component that represents vectorial q, n represents the kind number of distortion.

(2) image gradient information

The gradient information of image has comprised great amount of images structural information, and the gradient of image generally represents that this gradation of image value has the place of acute variation, and these places are generally image border parts, and being both is also human visual system's sensitizing range.

For discrete digital picture, its gradient magnitude is commonly defined as:

$\mathrm{Gradient}(i,j)=\sqrt{{(\mathrm{Gradient}\_x(i,j))}^2+{(\mathrm{Gradient}\_y(i,j))}^2}$

Gradient_x (i, j) wherein, Gradient_y (i, j) be respectively use that various approximate Discrete Operators calculate at position i, the partial derivative of j point X and two orthogonal directions of Y, for example operator Sobel, Prewitt and Canny operator.

The direction of gradient is that gradient magnitude changes the fastest place, and the direction of image gradient is defined as:

$\mathrm{orientation}(i,j)=\mathrm{arc}\tan \left(\frac{\mathrm{Gradient}\_y(i,j)}{\mathrm{Gradient}\_x(i,j)}\right)$

In a word, while there is edge in image, necessarily there is larger Grad; And more level and smooth part in image, gray-value variation is less, generally has less gradient.Image in processing often the mould of gradient referred to as gradient, the image consisting of image gradient is called gradient image.

Summary of the invention

The object of the invention is that the time, the space complexity that in current non-reference picture quality appraisement method, exist are high in order to solve, the low inferior problem of performance, by set up one have high-performance, low complex degree, with subjective assessment result mutually a property high without with reference to natural image quality method.

The inventive method is achieved through the following technical solutions.

A non-reference picture quality appraisement method based on gradient relevance, comprises the following steps:

Step 1, to input distorted image carry out feature extraction.

First, every piece image is asked for to it three kinds aspect image gradient different sub-character, i.e. gradient magnitude character GM, gradient direction qualitative change CO and gradient magnitude qualitative change CM.Wherein, three kinds of gradient character are respectively by formula 1,2,3 definition:

$\mathrm{GM}(i,j)=\sqrt{{\left(\mathrm{Gx}(i,j)\right)}^2+{\left(\mathrm{Gy}(i,j)\right)}^2}---\left(1\right)$

CO(i,j)＝orientation(i,j)-orientation _avg(i,j) (2)

$\mathrm{CM}(i,j)=\sqrt{{(\mathrm{Gx}(i,j)-{\mathrm{Gx}}_{\mathrm{avg}}(i,j))}^2+{(\mathrm{Gy}(i,j)-{\mathrm{Gy}}_{\mathrm{avg}}(i,j))}^2}---\left(3\right)$

Wherein,

$\mathrm{orientation}(i,j)=\mathrm{arc}\tan \left(\frac{\mathrm{Gy}(i,j)}{\mathrm{Gx}(i,j)}\right)$

${\mathrm{orientation}}_{\mathrm{avg}}(i,j)=\mathrm{arc}\tan \left(\frac{{\mathrm{Gy}}_{\mathrm{avg}}(i,j)}{{\mathrm{Gx}}_{\mathrm{avg}}(i,j)}\right)$

${\mathrm{Gx}}_{\mathrm{avg}}=\frac{\underset{i}{\overset{M}{\mathrm{\&Sigma;}}}\underset{j}{\overset{N}{\mathrm{\&Sigma;}}}\mathrm{Gx}(i,j)}{M\&times;N}$

${\mathrm{Gy}}_{\mathrm{avg}}=\frac{\underset{i}{\overset{M}{\mathrm{\&Sigma;}}}\underset{j}{\overset{N}{\mathrm{\&Sigma;}}}\mathrm{Gy}(i,j)}{M\&times;N}$

Wherein, the direction of orientation (i, j) presentation video gradient, Gx and Gy be discrete digital picture at the derivative of X, two orthogonal directions of Y, M, N are the sizes of the window that sets of the variation in order to describe in region.And in asking for gradient G x and Gy, use sobel gradient calculation operator, and it is expanded to two groups by one group of orthogonal direction.Wherein comprise 0 degree and 90 one group of orthogonal direction of spending, and one group-45 is spent and 45 orthogonal directions of spending.

The sobel operator of orthogonal direction 0 degree and 90 degree is:

$S^y=\left[\begin{array}{ccccc}0& 0& 0& 0& 0\\ 0& 1& 0& -1& 0\\ 0& 2& 0& -2& 0\\ 0& 1& 0& -1& 0\\ 0& 0& 0& 0& 0\end{array}\right]$ With $S^x=\left[\begin{array}{ccccc}0& 0& 0& 0& 0\\ 0& 1& 2& 1& 0\\ 0& 0& 0& 0& 0\\ 0& -1& -2& -1& 0\\ 0& 0& 0& 0& 0\end{array}\right]$

The sobel operator of orthogonal direction-45 degree and 45 degree is:

$S^y=\left[\begin{array}{ccccc}0& 0& 0& 0& 0\\ 1& 0& -1& 0& 0\\ 0& 2& 0& 2& 0\\ 0& 0& 1& 0& -1\\ 0& 0& 0& 0& 0\end{array}\right]$ With $S^x=\left[\begin{array}{ccccc}0& 0& 0& 0& 0\\ 0& 0& 1& 0& -1\\ 0& 2& 0& -2& 0\\ 1& 0& -1& 0& 0\\ 0& 0& 0& 0& 0\end{array}\right]$

The operator that is designated as y is wherein for calculating the operator of Gy, and the operator that is designated as x on is wherein for calculating the operator of Gx.

Then, 6 above-mentioned tried to achieve width image gradient character images are carried out to piecemeal processing, in each piece gradient, ask in nature statistical variance, on this basis by this variance with log function standard after, to calculate statistical variance that the form of average asked for each piece gradient, be fused to the variance of general image gradient character.Specific as follows:

Each image gradient character is divided into the image gradient character image block of 128 * 128 sizes, and each image gradient character image block is asked for to its statistical variance, be i.e. the variance d of n piece image block _n.Computing formula is suc as formula 4:

d _n＝∑(h(x)-E(h(x))) ² (4)

Wherein,

h(x)＝pdf(θ)

Wherein, θ is every kind of parameter in character image, the Distribution Statistics that pdf is this parameter, and h (x) represents that the statistical probability that θ is carried out after quantitative statistics represents, E (h (x)) represents the expectation of h (x).

Then the variance d each being obtained _nwith the standardization of log function, and the statistical variance that the image block of the average method fused images gradient character of assembling of use is obtained is as final feature f, suc as formula 5.

$f=\frac{1}{N}\&times;\log \left(\underset{n=1}{\overset{N}{\mathrm{\&Pi;}}}{d}_n\right),n=\mathrm{1,2}.......N---\left(5\right)$

Finally, the distorted image of input is carried out down-sampled, become the image of the second yardstick, and repeat said process, finally obtain one group of 12 dimensional feature vector Feature.

Feature＝[f _GM,f _CO,f _CM×2orientation×2scale]

Step 2, the Feature Mapping of carrying out.Using 12 dimensional feature vectors that obtain through step 1 as final Image quality measures, set up accordingly the mapping relations between characteristics of image and image mark.

First, image library is divided into training set and test set.Described test set is used for setting up the mapping relations between characteristics of image and picture quality, and test set is used for testing the function of the mapping relations of setting up.Use the method for SVMs, the characteristics of image of image in test set is carried out to the training of the Environmental Evaluation Model of distortion disaggregated model and corresponding each distortion classification.

Then, flow process based on two step frameworks in non-reference picture quality appraisement, in test set, carry out the prediction test of mark, use distortion disaggregated model classified by the distortion of altimetric image, in the middle of distortion classification, service quality evaluation model is predicted tested picture quality again, thereby obtain tested picture quality mark, and then utilize existing algorithm performance standard to assess it.

Beneficial effect

The non-reference picture quality appraisement method based on image gradient relevance that the present invention proposes, compare with existing generic technology, have that subjective consistency is high, time and the little feature of space complexity, can be applied in the middle of mini-system, or be embedded in the algorithm and equipment that picture quality is relevant, there is very high using value.

Accompanying drawing explanation

Fig. 1 is the flow chart of the method for the invention.

Fig. 2 is the inventive method and several in addition full references, the box diagram that nothing is carried out subjective consistency comparison with reference to algorithm in the specific embodiment of the invention 1.

Embodiment

Below in conjunction with the drawings and specific embodiments, the inventive method is described in further details.

Embodiment 1:

As shown in Figure 1, a kind of non-reference picture quality appraisement method based on gradient relevance, comprises the following steps:

Step 1, input picture is carried out to feature extraction.

First, input picture is carried out the asking for of three character of the defined gradient of the present invention.By the image gradient of both direction being asked for to amplitude, direction variation and amplitude change information.

Then, by three character of both direction, six gradient character images carry out piecemeal processing altogether.Corresponding each piecemeal is asked for its corresponding statistical variance afterwards.

Afterwards, the statistical variance of the piecemeal of six kinds of gradient character images of gained is merged, use average mode of assembling, ask for the mean value of all block statistics variances in every kind of character, as the whole statistical variance of six kinds of final gradient character.

Finally, input picture is carried out down-sampled, is become the image of the second yardstick, repeat said process the most at last characteristics of image by 6 dimensions, be extended for 12 dimensions.

Step 2, the Feature Mapping of carrying out.Using 12 dimensional feature vectors that obtain through step 1 as final Image quality measures, set up accordingly the mapping relations between characteristics of image and image mark.

First use the method for SVMs (SVM), the characteristics of image of image in test set is carried out to the training of the Environmental Evaluation Model of distortion disaggregated model and corresponding each distortion classification.

Then the flow process based on two step frameworks in non-reference picture quality appraisement is carried out the prediction test of mark in test set.; first use distortion disaggregated model classified by the distortion of altimetric image; in the middle of distortion classification, service quality evaluation model is predicted tested picture quality again; thereby obtain tested picture quality mark, and then utilize existing algorithm performance index (SROCC) to assess the quality of algorithm.

In the present embodiment, by LIVE image data base, test efficiency of the present invention and performance.In order to make contrast, the present embodiment has been used some famous full reference image quality appraisement methods and without contrasting as of the present invention with reference to evaluation quality method.In test process, database is divided into test set and training set, according to five folding cross validation methods, test set and training set are set to respectively to 20% and 80%.Apply mechanically afterwards the mark that two step prediction framework of institute's foundation are herein predicted test set, and wherein needed classification and regression model characteristics of image in training set are trained.The final SROCC index of calculating prediction mark and true score is as evaluating foundation of the present invention.In order to reduce the impact of accidentalia, said process is repeated 1000 times, random division training set and test set, finally gets SROCC each time simultaneously, and the intermediate value of Spearman's correlation coefficient index is as final algorithm evaluation score (in Table 1).The value of SROCC has better correlation closer to 1 expression algorithm and human perception.In order to show more intuitively the good and bad relation of various algorithms, also drawn the box diagram of the SROCC value of various algorithms, as shown in Figure 2.

As can be seen from Table 1, performance of the present invention is best with reference to aggregate performance in the middle of algorithm (BIQI, DIIVINE, BLIINDS-II, BRISQUE and the present invention) five kinds of nothings.And all showing good subjective consistency for the image of each distortion classification, this just can prove that the present invention has good versatility, and in performance, has very large advantage.With respect to other three kinds of full reference image quality appraisement methods---Y-PSNR (PSNR), structural similarity algorithm (SSIM) and visual information fidelity algorithm (VIF), the performance than VIF algorithm of this algorithm is low.

In table 1 LIVE storehouse, each algorithm subjective consistency index (SROCC) relatively

	JP2K	JPEG	NOISE	BLUR	FF	ALL
							PSNR	0.8990	0.8484	0.9835	0.8076	0.8986	0.8293

SSIM	0.9510	0.9173	0.9697	0.9513	0.9555	0.8996
							VIF	0.9515	0.9104	0.9844	0.9722	0.9631	0.9521
BIQI	0.8551	0.7767	0.9764	0.9258	0.7695	0.7599
							DIIVINE	0.9352	0.8921	0.9828	0.9551	0.9096	0.9174
BLIINDS-II	0.9462	0.9350	0.9634	0.9336	0.8992	0.9329
							BRISQUE	0.9445	0.9221	0.9889	0.9578	0.9173	0.9432
Proposed	0.9531	0.9412	0.9858	0.9689	0.9079	0.9476

In order to prove that the present invention has good performance on time complexity, first by the present invention and three kinds famous in made relatively (DIIVINE, BLIINDS-II, the method proposing in BRISQUE and the present invention) with reference to the time complexity of evaluation algorithms.In table 2, listed these four kinds of algorithms and calculated required overall time and average every required time of width image of 982 width images in LIVE IQA database.Therefrom can know that the more other two kinds of nothings of time efficiency of the present invention exceed much with reference to algorithm DIIVINE and BLIINDS-II, and slightly more inferior than BRISQUE.

Table 2 is without the time complexity comparison of reference method

	982 width image times used	Average every width image time used
			DIIVINE	2.9519*10(4)s	30.5294
BLIINDS-II	1.3112*10(5)s	133.5213
			BRISQUE	109.3859s	0.111391s
The present invention	275.7670	0.280822s

Claims (1)

1. the non-reference picture quality appraisement method based on gradient relevance, is characterized in that comprising the following steps:

Step 1, to input distorted image carry out feature extraction;

First, every piece image is asked for to it three kinds aspect image gradient different sub-character, i.e. gradient magnitude character GM, gradient direction qualitative change CO and gradient magnitude qualitative change CM, wherein, three kinds of gradient character are defined as follows respectively:

$\mathrm{GM}(i,j)=\sqrt{{\left(\mathrm{Gx}(i,j)\right)}^2+{\left(\mathrm{Gy}(i,j)\right)}^2}---\left(1\right)$

CO(i,j)＝orientation(i,j)-orientation _avg(i,j) (2)

$\mathrm{CM}(i,j)=\sqrt{{(\mathrm{Gx}(i,j)-{\mathrm{Gx}}_{\mathrm{avg}}(i,j))}^2+{(\mathrm{Gy}(i,j)-{\mathrm{Gy}}_{\mathrm{avg}}(i,j))}^2}---\left(3\right)$

Wherein,

$\mathrm{orientation}(i,j)=\mathrm{arc}\tan \left(\frac{\mathrm{Gy}(i,j)}{\mathrm{Gx}(i,j)}\right)$

${\mathrm{orientation}}_{\mathrm{avg}}(i,j)=\mathrm{arc}\tan \left(\frac{{\mathrm{Gy}}_{\mathrm{avg}}(i,j)}{{\mathrm{Gx}}_{\mathrm{avg}}(i,j)}\right)$

${\mathrm{Gx}}_{\mathrm{avg}}=\frac{\underset{i}{\overset{M}{\mathrm{\&Sigma;}}}\underset{j}{\overset{N}{\mathrm{\&Sigma;}}}\mathrm{Gx}(i,j)}{M\&times;N}$

${\mathrm{Gy}}_{\mathrm{avg}}=\frac{\underset{i}{\overset{M}{\mathrm{\&Sigma;}}}\underset{j}{\overset{N}{\mathrm{\&Sigma;}}}\mathrm{Gy}(i,j)}{M\&times;N}$

Wherein, the direction of orientation (i, j) presentation video gradient, Gx and Gy be discrete digital picture at the derivative of X, two orthogonal directions of Y, M, N are the sizes of the window that sets of the variation in order to describe in region; And in asking for gradient G x and Gy, use sobel gradient calculation operator, and it is expanded to two groups by one group of orthogonal direction; Wherein comprise 0 degree and 90 one group of orthogonal direction of spending, and one group-45 is spent and 45 orthogonal directions of spending;

The sobel operator of orthogonal direction 0 degree and 90 degree is:

$S^y=\left[\begin{array}{ccccc}0& 0& 0& 0& 0\\ 0& 1& 0& -1& 0\\ 0& 2& 0& -2& 0\\ 0& 1& 0& -1& 0\\ 0& 0& 0& 0& 0\end{array}\right]$ With $S^x=\left[\begin{array}{ccccc}0& 0& 0& 0& 0\\ 0& 1& 2& 1& 0\\ 0& 0& 0& 0& 0\\ 0& -1& -2& -1& 0\\ 0& 0& 0& 0& 0\end{array}\right]$

The sobel operator of orthogonal direction-45 degree and 45 degree is:

$S^y=\left[\begin{array}{ccccc}0& 0& 0& 0& 0\\ 1& 0& -1& 0& 0\\ 0& 2& 0& 2& 0\\ 0& 0& 1& 0& -1\\ 0& 0& 0& 0& 0\end{array}\right]$ With $S^x=\left[\begin{array}{ccccc}0& 0& 0& 0& 0\\ 0& 0& 1& 0& -1\\ 0& 2& 0& -2& 0\\ 1& 0& -1& 0& 0\\ 0& 0& 0& 0& 0\end{array}\right]$

The operator that is designated as y is wherein for calculating the operator of Gy, and the operator that is designated as x on is wherein for calculating the operator of Gx;

Then, above-mentioned tried to achieve image gradient character image is carried out to piecemeal processing, in each piece gradient, ask in nature statistical variance, on this basis by this variance with log function standard after, the statistical variance of each piece gradient being asked for the form of calculating average is fused to the variance of general image gradient character, specific as follows:

Each image gradient character is divided into the image gradient character image block of 128 * 128 sizes, and each image gradient character image block is asked for to its statistical variance, be i.e. the variance d of n piece image block _n; Computing formula is suc as formula 4:

d _n＝∑(h(x)-E(h(x))) ² (4)

Wherein,

h(x)＝pdf(θ)

Wherein, θ is every kind of parameter in character image, the Distribution Statistics that pdf is this parameter, and h (x) represents that the statistical probability that θ is carried out after quantitative statistics represents, E (h (x)) represents the expectation of h (x);

Then the variance d each being obtained _nwith the standardization of log function, and the statistical variance that the image block of the average method fused images gradient character of assembling of use is obtained is as final feature f, suc as formula 5;

$f=\frac{1}{N}\&times;\log \left(\underset{n=1}{\overset{N}{\mathrm{\&Pi;}}}{d}_n\right),n=\mathrm{1,2}.......N---\left(5\right)$

Finally, the distorted image of input is carried out down-sampled, becomes the image of the second yardstick, and repeat said process, finally obtain one group of 12 dimensional feature vector Feature:

Feature＝[f _GM,f _CO,f _CM×2orientation×2scale] (6)

Step 2, the Feature Mapping of carrying out, using 12 dimensional feature vectors that obtain through step 1 as final Image quality measures, set up the mapping relations between characteristics of image and image mark accordingly;

First, image library is divided into training set and test set; Described test set is used for setting up the mapping relations between characteristics of image and picture quality, and test set is used for testing the function of the mapping relations of setting up; Use the method for SVMs, the characteristics of image of image in test set is carried out to the training of the Environmental Evaluation Model of distortion disaggregated model and corresponding each distortion classification;

Then, flow process based on two step frameworks in non-reference picture quality appraisement, in test set, carry out the prediction test of mark, use distortion disaggregated model classified by the distortion of altimetric image, in the middle of distortion classification, service quality evaluation model is predicted tested picture quality again, thereby obtain tested picture quality mark, and then can utilize existing algorithm performance standard to assess it.