CN111598826B - Picture objective quality evaluation method and system based on combined multi-scale picture characteristics - Google Patents

Abstract

The invention provides a method and a system for objectively evaluating picture quality based on joint multi-scale picture characteristics, wherein the method comprises the following steps: and (3) picture processing: processing the original image into image groups with different scales by using a Gaussian pyramid and a Laplacian pyramid, and respectively marking the image groups as y ₀ ⁽ⁿ⁾ And y is ₁ ⁽ⁿ⁾ From y ₁ ⁽ⁿ⁾ Extracting to obtain edge structure characteristics; and (3) edge salient feature extraction: using the luminance mask and contrast mask, a group of pictures y is processed from a Gaussian pyramid ₀ ⁽ⁿ⁾ Picture group y processed by Laplacian pyramid ₁ ⁽ⁿ⁾ And extracting to obtain the edge saliency characteristics. The invention has higher accuracy in evaluating the quality of the desktop picture, and the comprehensive performance is superior to the prior art and the distortion type of the desktop picture is found through the verification of the prior database: gaussian blur, motion blur and JPEG2000 compression distortion have outstanding superior performance.

Description

Picture objective quality evaluation method and system based on combined multi-scale picture characteristics

Technical Field

The invention relates to the field of image processing, in particular to a method and a system for evaluating objective quality of pictures based on joint multi-scale picture features.

Background

With the wide application of intelligent terminals, such as smartphones, tablets and notebook computers, desktop content pictures have become the most common and highest-consumption pictures in daily life of people instead of natural pictures. Desktop content pictures are computer-generated, a picture formed by combining graphics, texts and natural pictures, and are widely used in applications such as desktop games, desktop collaboration and remote education. For these applications, picture quality is particularly important. However, since the desktop content picture and the natural picture have different characteristics, the conventional quality evaluation method designed for the natural picture cannot well reflect the distortion condition of the desktop content picture: compared with the characteristics of rich colors and smooth edges of natural pictures, the desktop content pictures are often single in color, sharp in edges and rich in a large number of repeated graphics; and the distortion of the natural pictures is generally caused by limited physical sensor capability, but the distortion of the desktop content pictures is generally caused by the self reasons of the computer. Therefore, an accurate and efficient objective quality evaluation method for the picture of the inner surface content is strongly demanded.

Patent document CN108335289a (application number: 201810049789.8) discloses a full-reference fused image objective quality evaluation method, comprising: selecting a picture database as input of model training, grouping pictures according to distortion types, wherein pictures with different degrees of distortion under each distortion type respectively obtain file names and labels of each group of pictures; extracting features, namely selecting a plurality of full-reference metric algorithms, respectively scoring pictures in each distortion type, and calculating each group of pictures by using one full-reference metric algorithm to obtain a feature vector which forms a feature matrix; data preprocessing, namely normalizing the feature vector scores corresponding to the distorted image labels and the distorted types to be between (1, 100) and (0, 1), and performing transposition processing to meet the training requirement of the SVM; and (5) training the characteristics to obtain a quality evaluation model.

Disclosure of Invention

Aiming at the defects in the prior art, the invention aims to provide a method and a system for objectively evaluating the quality of a picture based on combined multi-scale picture characteristics.

The invention provides a picture objective quality evaluation method based on joint multi-scale picture characteristics, which comprises the following steps:

and (3) picture processing: processing the original image into image groups with different scales by using a Gaussian pyramid and a Laplacian pyramid, and respectively marking the image groups as y ₀ ⁽ⁿ⁾ And y is ₁ ⁽ⁿ⁾ From y ₁ ⁽ⁿ⁾ Extracting to obtain edge structure characteristics;

and (3) edge salient feature extraction: using the luminance mask and contrast mask, a group of pictures y is processed from a Gaussian pyramid ₀ ⁽ⁿ⁾ Picture group y processed by Laplacian pyramid ₁ ⁽ⁿ⁾ Extracting to obtain edge saliency characteristics;

and calculating the feature similarity: according to the obtained edge structure characteristics and edge saliency characteristics, calculating to obtain edge structure similarity and edge saliency similarity;

the characteristic combination step: according to the obtained edge structure similarity and edge saliency similarity, calculating to obtain a final local quality diagram;

feature pooling: and calculating to obtain a final objective evaluation score according to the obtained final local quality map.

Preferably, the picture processing step:

processing the original image into image groups with different scales by using a Gaussian pyramid and a Laplacian pyramid, and respectively marking the image groups as y ₀ ⁽ⁿ⁾ And y is ₁ ⁽ⁿ⁾ 。

Preferably, the edge salient feature extraction step:

using the luminance mask and contrast mask, a group of pictures y is processed from a Gaussian pyramid ₀ ⁽ⁿ⁾ Picture group y processed by Laplacian pyramid ₁ ⁽ⁿ⁾ The edge saliency characteristics are extracted and obtained, and the calculation formula is as follows:

wherein,,

C _LM ⁽ⁿ⁾ representing a brightness mask calculation result, which is based on image characteristics of the image group processed by the Gaussian pyramid and the Laplacian pyramid;

representing a y function;

y ₁ ⁽ⁿ⁾ representing the picture group processed by the Laplacian pyramid;

y ₀ ⁽ⁿ⁾ representing a group of pictures after gaussian pyramid processing;

n represents the number of layers of the layer;

layer y ₁ ⁽¹⁾ The displayed picture features are edge structure features;

representing y ₀ ⁽ⁿ⁺¹⁾ Carrying in the calculation result of the y function;

γ ₁ representing a brightness contrast threshold;

the absolute value is taken;

a ₁ a constant that represents a guarantee of equation stability;

C _LCM ⁽ⁿ⁾ representing the contrast mask calculation result, based on C _LM ⁽ⁿ⁾ Image characteristics of the processed image group;

n represents the number of layers of the layer;

C _LCM ⁽¹⁾ representing the picture feature represented when n=1 as an edge saliency feature;

a ₂ a constant that represents a guarantee of equation stability;

representing C _LM ⁽ⁿ⁺¹⁾ Carrying in the calculation result of the y function;

γ ₂ representing a contrast detectable threshold;

g (x, y; sigma) represents a Gaussian kernel function;

* Representing a convolution;

∈2 represents upsampling.

Preferably, the feature similarity calculation step:

according to the obtained edge structure characteristics and edge saliency characteristics, the edge structure similarity is obtained through calculation, and the calculation formula is as follows:

wherein,,

S ₁ (x, y) represents the structural similarity of the (x, y) sides of the points;

subscripts r and d represent that the feature is taken from a reference picture or a distorted picture, respectively;

y _1r ⁽¹⁾ (x, y) represents an edge structure feature of the reference picture when n=1 at the point (x, y);

y _1d ⁽²⁾ (x, y) represents the edge structure feature of the distorted picture when n=2 at point (x, y);

T ₁ the representation is a non-zero constant to ensure the stability of the equation;

according to the obtained edge structure characteristics and edge saliency characteristics, the edge saliency similarity is obtained through calculation, and the calculation formula is as follows:

S ₂ (x,y)＝MS ₁ (x,y) ^α ·MS ₂ (x,y)

wherein,,

S ₂ (x, y) represents the point (x, y) side saliency similarity;

alpha represents S ₂ M S in (x, y) ₁ (x, y) the weight;

MS ₁ (x, y) represents the edge structure similarity calculated by the similarity calculation function;

MS ₂ (x, y) represents the edge significant similarity calculated by the similarity calculation function;

w ₁ (x, y) represents a weight factor;

G _LMr ⁽¹⁾ (x, y) representsLM mask features when n=1 at point (x, y) of the reference picture;

G _LMd ⁽¹⁾ (x, y) represents LM mask characteristics when n=1 at point (x, y) of the reference picture;

T ₂ representing a non-zero constant to ensure the stability of the equation;

∑ _(x,y) w ₁ (x, y) represents w at all points on the picture ₁ Accumulation of (x, y);

C _LCMr ⁽¹⁾ (x, y) represents LCM mask features of the reference picture when n=1 at point (x, y);

C _LCMd ⁽¹⁾ (x, y) represents LCM mask features for a distorted picture when n=1 at point (x, y);

C _LCMd ⁽²⁾ (x, y) represents LCM mask features representing the distorted picture at point (x, y) where n=2;

T ₃ representing a non-zero constant to ensure the stability of the equation.

Preferably, the feature combining step:

according to the obtained edge structure similarity and edge saliency similarity, calculating to obtain local quality similarity, wherein the calculation formula is as follows:

S _QM (x,y)＝(S ₁ (x,y)) ^ξ ·(S ₂ (x,y)) ^ψ

＝(S ₁ (x,y)) ^ξ ·MS ₁ (x,y) ^μ ·MS ₂ (x,y) ^ψ

μ＝ψ·α

wherein,,

S _QM (x, y) represents local mass similarity at point (x, y);

xi represents S ₁ (x, y) at local mass S _QM (x, y) the weight;

psi represents M S ₁ (x, y) at local mass S _QM (x, y) the weight;

mu represents M S ₂ (x, y) at local mass S _QM (x, y) the weight;

alpha represents S ₂ M S in (x, y) ₁ (x, y) weight.

Preferably, the feature pooling step:

according to the obtained local quality graph similarity, calculating to obtain a final objective evaluation score, wherein the calculation formula is as follows:

w ₂ (x,y)＝max(y _1r ⁽²⁾ (x,y),y _1d ⁽²⁾ (x,y))

wherein,,

s represents the final objective evaluation score;

w ₂ (x, y) represents a weight parameter.

y _1r ⁽²⁾ (x, y) represents the edge structure feature of the reference picture when n=2 at the point (x, y).

The invention provides a desktop content picture objective quality evaluation system based on joint multi-scale picture characteristics, which comprises the following steps:

and a picture processing module: processing the original image into image groups with different scales by using a Gaussian pyramid and a Laplacian pyramid, and respectively marking the image groups as y ₀ ⁽ⁿ⁾ And y is ₁ ⁽ⁿ⁾ From y ₁ ⁽ⁿ⁾ Extracting to obtain edge structure characteristics;

and the edge salient feature extraction module is used for: using the luminance mask and contrast mask, a group of pictures y is processed from a Gaussian pyramid ₀ ⁽ⁿ⁾ Picture group y processed by Laplacian pyramid ₁ ⁽ⁿ⁾ Extracting to obtain edge saliency characteristics;

and the feature similarity calculation module is used for: according to the obtained edge structure characteristics and edge saliency characteristics, calculating to obtain edge structure similarity and edge saliency similarity;

and the characteristic combination module is used for: according to the obtained edge structure similarity and edge saliency similarity, calculating to obtain a final local quality diagram;

and (3) a characteristic pooling module: and calculating to obtain a final objective evaluation score according to the obtained final local quality map.

Preferably, the picture processing module:

processing the original image into image groups with different scales by using a Gaussian pyramid and a Laplacian pyramid, and respectively marking the image groups as y ₀ ⁽ⁿ⁾ And y is ₁ ⁽ⁿ⁾ ；

The edge salient feature extraction module is used for:

using the luminance mask and contrast mask, a group of pictures y is processed from a Gaussian pyramid ₀ ⁽ⁿ⁾ Picture group y processed by Laplacian pyramid ₁ ⁽ⁿ⁾ The edge saliency characteristics are extracted and obtained, and the calculation formula is as follows:

wherein,,

C _LM ⁽ⁿ⁾ representing a brightness mask calculation result, which is based on image characteristics of the image group processed by the Gaussian pyramid and the Laplacian pyramid;

representing a y function;

y ₁ ⁽ⁿ⁾ representing the picture group processed by the Laplacian pyramid;

y ₀ ⁽ⁿ⁾ representing a group of pictures after gaussian pyramid processing;

n represents the number of layers of the layer;

layer y ₁ ⁽¹⁾ The displayed picture features are edge structure features;

representing y ₀ ⁽ⁿ⁺¹⁾ Carrying in the calculation result of the y function;

γ ₁ representing a brightness contrast threshold;

the absolute value is taken;

a ₁ a constant that represents a guarantee of equation stability;

C _LCM ⁽ⁿ⁾ representing the contrast mask calculation result, based on C _LM ⁽ⁿ⁾ Image characteristics of the processed image group;

n represents the number of layers of the layer;

C _LCM ⁽¹⁾ representing the picture feature represented when n=1 as an edge saliency feature;

a ₂ a constant that represents a guarantee of equation stability;

representing C _LM ⁽ⁿ⁺¹⁾ Carrying in the calculation result of the y function;

γ ₂ representing a contrast detectable threshold;

g (x, y; sigma) represents a Gaussian kernel function;

* Representing a convolution;

∈2 represents upsampling;

the feature similarity calculation module is used for:

according to the obtained edge structure characteristics and edge saliency characteristics, the edge structure similarity is obtained through calculation, and the calculation formula is as follows:

wherein,,

S ₁ (x, y) represents the structural similarity of the (x, y) sides of the points;

subscripts r and d represent that the feature is taken from a reference picture or a distorted picture, respectively;

y _1r ⁽¹⁾ (x, y) represents an edge structure feature of the reference picture when n=1 at the point (x, y);

y _1d ⁽²⁾ (x, y) represents the edge structure feature of the distorted picture when n=2 at point (x, y);

T ₁ the representation is a non-zero constant to ensure the stability of the equation;

according to the obtained edge structure characteristics and edge saliency characteristics, the edge saliency similarity is obtained through calculation, and the calculation formula is as follows:

S ₂ (x,y)＝MS ₁ (x,y) ^α ·MS ₂ (x,y)

/>

wherein,,

S ₂ (x, y) represents the point (x, y) side saliency similarity;

alpha represents S ₂ M S in (x, y) ₁ (x, y) the weight;

MS ₁ (x, y) represents the edge structure similarity calculated by the similarity calculation function;

MS ₂ (x, y) represents the edge significant similarity calculated by the similarity calculation function;

w ₁ (x, y) represents a weight factor;

C _LMr ⁽¹⁾ (x, y) represents LM mask characteristics when n=1 at point (x, y) of the reference picture;

C _LMd ⁽¹⁾ (x, y) represents LM mask characteristics when n=1 at point (x, y) of the reference picture;

T ₂ representing a non-zero constant to ensure the stability of the equation;

∑ _(x,y) w ₁ (x, y) represents w at all points on the picture ₁ Accumulation of (x, y);

C _LCMr ⁽¹⁾ (x, y) represents LCM mask features of the reference picture when n=1 at point (x, y);

C _LCMd ⁽¹⁾ (x, y) represents LCM mask features for a distorted picture when n=1 at point (x, y);

C _LCMd ⁽²⁾ (x, y) represents LCM mask features representing the distorted picture at point (x, y) where n=2;

T ₃ representing a non-zero constant to ensure the stability of the equation.

Preferably, the feature combination module:

according to the obtained edge structure similarity and edge saliency similarity, calculating to obtain local quality similarity, wherein the calculation formula is as follows:

S _QM (x,y)＝(S ₁ (x,y)) ^ξ ·(S ₂ (x,y)) ^ψ

＝(S ₁ (x,y)) ^ξ ·MS ₁ (x,y) ^μ ·MS ₂ (x,y) ^ψ

μ＝ψ·α

wherein,,

S _QM (x, y) represents local mass similarity at point (x, y);

xi represents S ₁ (x, y) at local mass S _QM (x, y) the weight;

psi represents M S ₁ (x, y) at local mass S _QM (x, y) the weight;

mu represents M S ₂ (x, y) at local mass S _QM (x, y) the weight;

alpha represents S ₂ M S in (x, y) ₁ (x, y) weight.

The feature pooling module:

according to the obtained local quality graph similarity, calculating to obtain a final objective evaluation score, wherein the calculation formula is as follows:

w ₂ (x,y)＝max(y _1r ⁽²⁾ (x,y),y _1d ⁽²⁾ (x,y))

wherein,,

s represents the final objective evaluation score;

w ₂ (x, y) represents a weight parameter.

y _1r ⁽²⁾ (x, y) represents the edge structure feature of the reference picture when n=2 at the point (x, y).

According to the present invention, there is provided a computer-readable storage medium storing a computer program which, when executed by a processor, implements the steps of the method for objectively evaluating a picture based on joint multi-scale picture features as described in any one of the above.

Compared with the prior art, the invention has the following beneficial effects:

the invention has higher accuracy in evaluating the quality of the desktop picture, and the comprehensive performance is superior to the prior art and the distortion type of the desktop picture is found through the verification of the prior database: gaussian blur, motion blur and JPEG2000 compression distortion have outstanding superior performance.

Drawings

Other features, objects and advantages of the present invention will become more apparent upon reading of the detailed description of non-limiting embodiments, given with reference to the accompanying drawings in which:

fig. 1 is a schematic diagram of a process flow provided in the present invention.

Detailed Description

The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the present invention, but are not intended to limit the invention in any way. It should be noted that variations and modifications could be made by those skilled in the art without departing from the inventive concept. These are all within the scope of the present invention.

The invention provides a picture objective quality evaluation method based on joint multi-scale picture characteristics, which comprises the following steps:

and (3) picture processing: using Gaussian pyramids and pullsThe Laplacian pyramid processes the original picture into picture groups with different scales, which are respectively marked as y ₀ ⁽ⁿ⁾ And y is ₁ ⁽ⁿ⁾ From y ₁ ⁽ⁿ⁾ Extracting to obtain edge structure characteristics;

and (3) edge salient feature extraction: using the luminance mask and contrast mask, a group of pictures y is processed from a Gaussian pyramid ₀ ⁽ⁿ⁾ Picture group y processed by Laplacian pyramid ₁ ⁽ⁿ⁾ Extracting to obtain edge saliency characteristics;

and calculating the feature similarity: according to the obtained edge structure characteristics and edge saliency characteristics, calculating to obtain edge structure similarity and edge saliency similarity;

the characteristic combination step: according to the obtained edge structure similarity and edge saliency similarity, calculating to obtain a final local quality diagram;

feature pooling: and calculating to obtain a final objective evaluation score according to the obtained final local quality map.

Specifically, the picture processing step:

processing the original image into image groups with different scales by using a Gaussian pyramid and a Laplacian pyramid, and respectively marking the image groups as y ₀ ⁽ⁿ⁾ And y is ₁ ⁽ⁿ⁾ 。

Specifically, the edge salient feature extraction step:

using the luminance mask and contrast mask, a group of pictures y is processed from a Gaussian pyramid ₀ ⁽ⁿ⁾ Picture group y processed by Laplacian pyramid ₁ ⁽ⁿ⁾ The edge saliency characteristics are extracted and obtained, and the calculation formula is as follows:

wherein,,

C _LM ⁽ⁿ⁾ representing a brightness mask calculation result, which is based on image characteristics of the image group processed by the Gaussian pyramid and the Laplacian pyramid;

representing a y function;

y ₁ ⁽ⁿ⁾ representing the picture group processed by the Laplacian pyramid;

y ₀ ⁽ⁿ⁾ representing a group of pictures after gaussian pyramid processing;

n represents the number of layers of the layer;

layer y ₁ ⁽¹⁾ The displayed picture features are edge structure features;

representing y ₀ ⁽ⁿ⁺¹⁾ Carrying in the calculation result of the y function;

γ ₁ representing a brightness contrast threshold;

the absolute value is taken;

a ₁ a constant that represents a guarantee of equation stability;

C _LCM ⁽ⁿ⁾ representing the contrast mask calculation result, based on C _LM ⁽ⁿ⁾ Image characteristics of the processed image group;

n represents the number of layers of the layer;

C _LCM ⁽¹⁾ representing the picture feature represented when n=1 as an edge saliency feature;

a ₂ a constant that represents a guarantee of equation stability;

representing C _LM ⁽ⁿ⁺¹⁾ Carrying in the calculation result of the y function;

γ ₂ representing a contrast detectable threshold;

g (x, y; sigma) represents a Gaussian kernel function;

* Representing a convolution;

∈2 represents upsampling.

Specifically, the feature similarity calculation step:

according to the obtained edge structure characteristics and edge saliency characteristics, the edge structure similarity is obtained through calculation, and the calculation formula is as follows:

wherein,,

S ₁ (x, y) represents the structural similarity of the (x, y) sides of the points;

subscripts r and d represent that the feature is taken from a reference picture or a distorted picture, respectively;

y _1r ⁽¹⁾ (x, y) represents an edge structure feature of the reference picture when n=1 at the point (x, y);

y _1d ⁽²⁾ (x, y) represents the edge structure feature of the distorted picture when n=2 at point (x, y);

T ₁ the representation is a non-zero constant to ensure the stability of the equation;

according to the obtained edge structure characteristics and edge saliency characteristics, the edge saliency similarity is obtained through calculation, and the calculation formula is as follows:

S ₂ (x,y)＝MS ₁ (x,y) ^α ·MS ₂ (x,y)

/>

wherein,,

S ₂ (x, y) represents the point (x, y) side saliency similarity;

alpha represents S ₂ M S in (x, y) ₁ (x, y) the weight;

MS ₁ (x, y) represents the edge structure similarity calculated by the similarity calculation function;

MS ₂ (x, y) represents the edge significant similarity calculated by the similarity calculation function;

w ₁ (x, y) represents a weight factor;

C _LMr ⁽¹⁾ (x, y) represents LM mask characteristics when n=1 at point (x, y) of the reference picture;

C _LMd ⁽¹⁾ (x, y) represents LM mask characteristics when n=1 at point (x, y) of the reference picture;

T ₂ representing a non-zero constant to ensure the stability of the equation;

∑ _(x,y) w ₁ (x, y) represents w at all points on the picture ₁ Accumulation of (x, y);

C _LCMr ⁽¹⁾ (x, y) represents LCM mask features of the reference picture when n=1 at point (x, y);

C _LCMd ⁽¹⁾ (x, y) represents LCM mask features for a distorted picture when n=1 at point (x, y);

C _LCMd ⁽²⁾ (x, y) represents LCM mask features representing the distorted picture at point (x, y) where n=2;

T ₃ representing a non-zero constant to ensure the stability of the equation.

Specifically, the feature combination step:

according to the obtained edge structure similarity and edge saliency similarity, calculating to obtain local quality similarity, wherein the calculation formula is as follows:

S _QM (x,y)＝(S ₁ (x,y)) ^ξ ·(S ₂ (x,y)) ^ψ

＝(S ₁ (x,y)) ^ξ ·MS ₁ (x,y) ^μ ·MS ₂ (x,y) ^ψ

μ＝ψ·α

wherein,,

S _QM (x, y) represents local mass similarity at point (x, y);

xi represents S ₁ (x, y) at local mass S _QM (x, y) the weight;

psi represents M S ₁ (x, y) at local mass S _QM (x, y) the weight;

mu represents M S ₂ (x, y) at local mass S _QM (x, y) the weight;

alpha represents S ₂ M S in (x, y) ₁ (x, y) weight.

Specifically, the feature pooling step:

according to the obtained local quality graph similarity, calculating to obtain a final objective evaluation score, wherein the calculation formula is as follows:

w ₂ (x,y)＝max(y _1r ⁽²⁾ (x,y),y _1d ⁽²⁾ (x,y))

wherein,,

s represents the final objective evaluation score;

w ₂ (x, y) represents a weight parameter.

y _1r ⁽²⁾ (x, y) represents the edge structure feature of the reference picture when n=2 at the point (x, y).

The desktop content picture objective quality evaluation system based on the combined multi-scale picture features can be realized through the step flow of the picture objective quality evaluation method based on the combined multi-scale picture features. The person skilled in the art can understand the method for evaluating the objective quality of the picture based on the joint multi-scale picture features as a preferable example of the system for evaluating the objective quality of the desktop content picture based on the joint multi-scale picture features.

The invention provides a desktop content picture objective quality evaluation system based on joint multi-scale picture characteristics, which comprises the following steps:

and a picture processing module: processing the original image into image groups with different scales by using a Gaussian pyramid and a Laplacian pyramid, and respectively marking the image groups as y ₀ ⁽ⁿ⁾ And y is ₁ ⁽ⁿ⁾ From y ₁ ⁽ⁿ⁾ Extracting to obtain edge structure characteristics;

and the edge salient feature extraction module is used for: using the luminance mask and contrast mask, a group of pictures y is processed from a Gaussian pyramid ₀ ⁽ⁿ⁾ Picture group y processed by Laplacian pyramid ₁ ⁽ⁿ⁾ Extracting to obtain edge saliency characteristics;

and the feature similarity calculation module is used for: according to the obtained edge structure characteristics and edge saliency characteristics, calculating to obtain edge structure similarity and edge saliency similarity;

and the characteristic combination module is used for: according to the obtained edge structure similarity and edge saliency similarity, calculating to obtain a final local quality diagram;

and (3) a characteristic pooling module: and calculating to obtain a final objective evaluation score according to the obtained final local quality map.

Specifically, the picture processing module:

processing the original image into image groups with different scales by using a Gaussian pyramid and a Laplacian pyramid, and respectively marking the image groups as y ₀ ⁽ⁿ⁾ And y is ₁ ⁽ⁿ⁾ ；

The edge salient feature extraction module is used for:

using the luminance mask and contrast mask, a group of pictures y is processed from a Gaussian pyramid ₀ ⁽ⁿ⁾ Picture group y processed by Laplacian pyramid ₁ ⁽ⁿ⁾ The edge saliency characteristics are extracted and obtained, and the calculation formula is as follows:

wherein,,

C _LM ⁽ⁿ⁾ representing a brightness mask calculation result, which is based on image characteristics of the image group processed by the Gaussian pyramid and the Laplacian pyramid;

representing a y function;

y ₁ ⁽ⁿ⁾ representing the picture group processed by the Laplacian pyramid;

y ₀ ⁽ⁿ⁾ representing a group of pictures after gaussian pyramid processing;

n represents the number of layers of the layer;

layer y ₁ ⁽¹⁾ The displayed picture features are edge structure features;

representing y ₀ ⁽ⁿ⁺¹⁾ Carrying in the calculation result of the y function;

γ ₁ representing a brightness contrast threshold;

the absolute value is taken;

a ₁ a constant that represents a guarantee of equation stability;

C _LCM ⁽ⁿ⁾ representing the contrast mask calculation result, based on C _LM ⁽ⁿ⁾ Image characteristics of the processed image group;

n represents the number of layers of the layer;

C _LCM ⁽¹⁾ representing the picture feature represented when n=1 as an edge saliency feature;

a ₂ a constant that represents a guarantee of equation stability;

representing C _LM ⁽ⁿ⁺¹⁾ Carrying in the calculation result of the y function;

γ ₂ representing a contrast detectable threshold;

g (x, y; sigma) represents a Gaussian kernel function;

* Representing a convolution;

∈2 represents upsampling;

the feature similarity calculation module is used for:

according to the obtained edge structure characteristics and edge saliency characteristics, the edge structure similarity is obtained through calculation, and the calculation formula is as follows:

wherein,,

S ₁ (x, y) represents the structural similarity of the (x, y) sides of the points;

subscripts r and d represent that the feature is taken from a reference picture or a distorted picture, respectively;

y _1r ⁽¹⁾ (x, y) represents an edge structure feature of the reference picture when n=1 at the point (x, y);

y _1d ⁽²⁾ (x, y) represents the edge structure feature of the distorted picture when n=2 at point (x, y);

T ₁ the representation is a non-zero constant to ensure the stability of the equation;

according to the obtained edge structure characteristics and edge saliency characteristics, the edge saliency similarity is obtained through calculation, and the calculation formula is as follows:

S ₂ (x,y)＝MS ₁ (x,y) ^α ·MS ₂ (x,y)

wherein,,

S ₂ (x, y) represents the point (x, y) side saliency similarity;

alpha represents S ₂ M S in (x, y) ₁ (x, y) the weight;

MS ₁ (x, y) represents the edge structure similarity calculated by the similarity calculation function;

MS ₂ (x, y) represents the edge significant similarity calculated by the similarity calculation function;

w ₁ (x, y) represents a weight factor;

C _LMr ⁽¹⁾ (x, y) represents LM mask characteristics when n=1 at point (x, y) of the reference picture;

C _LMd ⁽¹⁾ (x, y) represents LM mask characteristics when n=1 at point (x, y) of the reference picture;

T ₂ representing a non-zero constant to ensure the stability of the equation;

∑ _(x,y) w ₁ (x, y) represents w at all points on the picture ₁ Accumulation of (x, y);

C _LCMr ⁽¹⁾ (x, y) represents LCM mask features of the reference picture when n=1 at point (x, y);

C _LCMd ⁽¹⁾ (x, y) represents LCM mask features for a distorted picture when n=1 at point (x, y);

C _LCMd ⁽²⁾ (x, y) represents LCM mask features representing the distorted picture at point (x, y) where n=2;

T ₃ representing a non-zero constant to ensure the stability of the equation.

Specifically, the feature combination module:

according to the obtained edge structure similarity and edge saliency similarity, calculating to obtain local quality similarity, wherein the calculation formula is as follows:

S _QM (x,y)＝(S ₁ (x,y)) ^ξ ·(S ₂ (x,y)) ^ψ

＝(S ₁ (x,y)) ^ξ ·MS ₁ (x,y) ^μ ·MS ₂ (x,y) ^ψ

μ＝ψ·α

wherein,,

S _QM (x, y) represents local mass similarity at point (x, y);

xi represents S ₁ (x, y) at local mass S _QM (x, y) the weight;

psi represents M S ₁ (x, y) at local mass S _QM (x, y) the weight;

mu represents M S ₂ (x, y) at local mass S _QM (x, y) the weight;

alpha represents S ₂ M S in (x, y) ₁ (x, y) weight.

The feature pooling module:

according to the obtained local quality graph similarity, calculating to obtain a final objective evaluation score, wherein the calculation formula is as follows:

w ₂ (x,y)＝max(y _1r ⁽²⁾ (x,y),y _1d ⁽²⁾ (x,y))

wherein,,

s represents the final objective evaluation score;

w ₂ (x, y) represents a weight parameter.

y _1r ⁽²⁾ (x, y) represents the edge structure feature of the reference picture when n=2 at the point (x, y).

According to the present invention, there is provided a computer-readable storage medium storing a computer program which, when executed by a processor, implements the steps of the method for objectively evaluating a picture based on joint multi-scale picture features as described in any one of the above.

The present invention will be described more specifically by way of preferred examples.

Preferred example 1:

the invention provides an objective quality evaluation method based on a combined multi-scale desktop content picture, which is used for extracting edge structural features and edge saliency features of the picture and evaluating distortion degrees. Specifically, in combination with the characteristics of the human eye vision system, a picture feature extraction mode is designed, and the extracted picture features comprise two parts:

1. the structural characteristics of the edges of the strip,

2. edge saliency features.

In order to achieve the above purpose, the present invention adopts the following technical scheme:

1. processing the original image into image groups with different scales by using a Gaussian pyramid and a Laplacian pyramid, and respectively marking the image groups as y ₀ ⁽ⁿ⁾ And y is ₁ ⁽ⁿ⁾ The method comprises the steps of carrying out a first treatment on the surface of the Let n=1 from y ₁ ⁽ⁿ⁾ Extracting edge structural characteristics;

2. the edge saliency features are extracted from the two pyramids using a luminance mask (Luminance Masking, LM) and a contrast mask (Luminance Contrast Masking, LCM), specifically calculated as follows:

wherein the method comprises the steps of

G (x, y; σ) represents a Gaussian kernel function, x represents convolution, and ε 2 represents upsampling.

Calculation result C of luminance mask _LM ⁽ⁿ⁾ Can detect the brightness change recognizable by human eyes and convert gamma according to the Buchsbaum curve ₁ Set to 1; contrast mask calculation result C _LCM ⁽ⁿ⁾ Can detect the change of the human eye identifiable contrast, and gamma is determined according to the contrast detectable threshold ₂ Set to 0.62.

Let n=1 slave C _LCM ⁽ⁿ⁾ And extracting edge saliency characteristics.

3. Feature similarity calculation

Edge structural similarity:

S ₂ (x,y)＝MS ₁ (x,y) ^α ·MS ₂ (x,y),

edge saliency similarity:

wherein subscripts r and d respectively denote that the feature is taken from a reference picture or a distorted picture, w ₁ (x,y)＝y _1r ⁽²⁾ ，T ₁ ,T ₂ ,T ₃ 0.07,1X 10-50 and 0.01 are taken respectively.

4. Feature combination

The final local mass map is:

S _QM (x,y)＝(S ₁ (x,y)) ^ξ ·(S ₂ (x,y)) ^ψ

＝(S ₁ (x,y)) ^ξ ·MS ₁ (x,y) ^μ ·MS ₂ (x,y) ^ψ ,

where μ=ψ·α,

1.8,0.02,0.9.

5. Feature pooling

Final objective evaluation score:

wherein w is ₂ ＝max(y _1r ⁽²⁾ (x,y),y _1d ⁽²⁾ (x,y))。

To illustrate the effectiveness of the above model, tests were performed on the desktop content picture authority database SIQAD. The SIQAD database includes 20 reference pictures, each picture corresponding to 7 orders of 7 distortion for a total of 980 distorted pictures. The 7 kinds of distortions include Gaussian Noise (GN), gaussian Blur (GB), motion Blur (MB), contrast variation (CC), JPEG compression (JPEG), JPEG2000 compression (J2K), layer efficient coding (LSC).

Three indices proposed by the VQEG expert group, which are specifically used to measure the consistency of subjective scores with objective evaluation scores, are used to determine the superiority of the model, and are pearson linear correlation coefficients (Pearson linear correlation coefficient, PLCC), root mean square error (Root mean squared error, RMSE) and Spearman rank-order correlation coefficients (Spearman rank-order correlation coefficient, SROCC), respectively, in the following manner:

wherein m and Q represent subjective score and objective score respectively,

represents the average value of the subjective score and the objective score, d _i Representing the difference between the subjective score ordering order and the objective score ordering order of the ith picture. Values of PLCC and SROCC lie between 0 and 1, with closer to 1 indicating better agreement between subjective and objective scores; the smaller the RMSE value, the smaller the difference between the subjective score and the objective score, the better the model performance.

Table 1 shows the test results on the database SIQAD, wherein PSNR, SSIM, MSSIM, IWSSIM, VIF, IFC, FSIM, SCQ is a quality evaluation method designed for natural pictures, SIQM, SQI, ESIM, MDOGS, GFM is an objective quality evaluation method designed for desktop content pictures in recent years, and it can be seen by comparing the data of each method:

aiming at the overall performance, the method has the advantages that the first name is listed in the bit sequence of the PLCC (PLCC) and RMSE (RMSE) evaluation indexes, and the second name is listed in the bit sequence of the SROCC;

for the independent distortion types, the method obtains 9 first names and 1 third name in total, is obviously superior to other methods, and has obvious superiority when evaluating the distortion types GB, MB and J2K.

Table one SIQAD database test results:

those skilled in the art will appreciate that the systems, apparatus, and their respective modules provided herein may be implemented entirely by logic programming of method steps such that the systems, apparatus, and their respective modules are implemented as logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc., in addition to the systems, apparatus, and their respective modules being implemented as pure computer readable program code. Therefore, the system, the apparatus, and the respective modules thereof provided by the present invention may be regarded as one hardware component, and the modules included therein for implementing various programs may also be regarded as structures within the hardware component; modules for implementing various functions may also be regarded as being either software programs for implementing the methods or structures within hardware components.

The foregoing describes specific embodiments of the present invention. It is to be understood that the invention is not limited to the particular embodiments described above, and that various changes or modifications may be made by those skilled in the art within the scope of the appended claims without affecting the spirit of the invention. The embodiments of the present application and features in the embodiments may be combined with each other arbitrarily without conflict.

Claims (7)

1. A picture objective quality evaluation method based on joint multi-scale picture features is characterized by comprising the following steps:

picture processingThe steps are as follows: processing the original image into image groups with different scales by using a Gaussian pyramid and a Laplacian pyramid, and respectively marking the image groups as y ₀ ⁽ⁿ⁾ And y is ₁ ⁽ⁿ⁾ From y ₁ ⁽ⁿ⁾ Extracting to obtain edge structure characteristics;

and (3) edge salient feature extraction: using the luminance mask and contrast mask, a group of pictures y is processed from a Gaussian pyramid ₀ ⁽ⁿ⁾ Picture group y processed by Laplacian pyramid ₁ ⁽ⁿ⁾ Extracting to obtain edge saliency characteristics;

and calculating the feature similarity: according to the obtained edge structure characteristics and edge saliency characteristics, calculating to obtain edge structure similarity and edge saliency similarity;

the characteristic combination step: according to the obtained edge structure similarity and edge saliency similarity, calculating to obtain a final local quality diagram;

feature pooling: calculating to obtain a final objective evaluation score according to the obtained final local quality map;

the edge salient feature extraction step comprises the following steps:

using the luminance mask and contrast mask, a group of pictures y is processed from a Gaussian pyramid ₀ ⁽ⁿ⁾ Picture group y processed by Laplacian pyramid ₁ ⁽ⁿ⁾ The edge saliency characteristics are extracted and obtained, and the calculation formula is as follows:

wherein,,

C _LM ⁽ⁿ⁾ representing the brightness mask calculation result based on Gaussian pyramid and Gaussian pyramidImage characteristics of the image group processed by the Laplacian pyramid;

representing a y function;

y ₁ ⁽ⁿ⁾ representing the picture group processed by the Laplacian pyramid;

y ₀ ⁽ⁿ⁾ representing a group of pictures after gaussian pyramid processing;

n represents the number of layers of the layer;

layer y ₁ ⁽¹⁾ The displayed picture features are edge structure features;

representing y ₀ ⁽ⁿ⁺¹⁾ Carrying in the calculation result of the y function;

γ ₁ representing a brightness contrast threshold;

the absolute value is taken;

a ₁ a constant that represents a guarantee of equation stability;

C _LCM ⁽ⁿ⁾ representing the contrast mask calculation result, based on C _LM ⁽ⁿ⁾ Image characteristics of the processed image group;

n represents the number of layers of the layer;

C _LCM ⁽¹⁾ representing the picture feature represented when n=1 as an edge saliency feature;

a ₂ a constant that represents a guarantee of equation stability;

representing C _LM ⁽ⁿ⁺¹⁾ Carrying in the calculation result of the y function;

γ ₂ representing a contrast detectable threshold;

g (x, y; sigma) represents a Gaussian kernel function;

* Representing a convolution;

∈2 represents upsampling.

2. The method for objectively evaluating a picture quality based on joint multi-scale picture features as claimed in claim 1, wherein the feature similarity calculation step:

according to the obtained edge structure characteristics and edge saliency characteristics, the edge structure similarity is obtained through calculation, and the calculation formula is as follows:

wherein,,

S ₁ (x, y) represents the structural similarity of the (x, y) sides of the points;

subscripts r and d represent that the feature is taken from a reference picture or a distorted picture, respectively;

y _1r ⁽¹⁾ (x, y) represents an edge structure feature of the reference picture when n=1 at the point (x, y);

y _1d ⁽²⁾ (x, y) represents the edge structure feature of the distorted picture when n=2 at point (x, y);

T ₁ the representation is a non-zero constant to ensure the stability of the equation;

according to the obtained edge structure characteristics and edge saliency characteristics, the edge saliency similarity is obtained through calculation, and the calculation formula is as follows:

S ₂ (x，y)＝MS ₁ (x，y) ^α ·MS ₂ (x，y)

wherein,,

S ₂ (x, y) represents the point (x, y) side saliency similarity;

alpha represents S ₂ M S in (x, y) ₁ (x, y) the weight;

MS ₁ (x, y) represents the edge structure similarity calculated by the similarity calculation function;

MS ₂ (x, y) represents the edge significant similarity calculated by the similarity calculation function;

w ₁ (x, y) represents a weight factor;

C _LMr ⁽¹⁾ (x, y) represents LM mask characteristics when n=1 at point (x, y) of the reference picture;

C _LMd ⁽¹⁾ (x, y) represents LM mask characteristics when n=1 at point (x, y) of the reference picture;

T ₂ representing a non-zero constant to ensure the stability of the equation;

∑ _(x，y) w ₁ (x, y) represents w at all points on the picture ₁ Accumulation of (x, y);

C _LCMr ⁽¹⁾ (x, y) represents LCM mask features of the reference picture when n=1 at point (x, y);

C _LCMd ⁽¹⁾ (x, y) represents LCM mask features for a distorted picture when n=1 at point (x, y);

C _LCMd ⁽²⁾ (x, y) represents LCM mask features representing the distorted picture at point (x, y) where n=2;

T ₃ representing a non-zero constant to ensure the stability of the equation.

3. The method for objectively evaluating a picture quality based on joint multi-scale picture features of claim 2, wherein the feature combining step:

according to the obtained edge structure similarity and edge saliency similarity, calculating to obtain local quality similarity, wherein the calculation formula is as follows:

S _QM (x，y)＝(S ₁ (x，y)) ^ξ ·(S ₂ (x，y)) ^ψ

＝(S ₁ (x，y)) ^ξ ·MS ₁ (x，y)μ·MS ₂ (x，y) ^ψ

μ＝ψ·α

wherein,,

S _QM (x, y) represents local mass similarity at point (x, y);

xi represents S ₁ (x, y) at local mass S _QM (x, y) the weight;

psi represents M S ₁ (x, y) at local mass S _QM (x, y) the weight;

mu represents M S ₂ (x, y) at local mass S _QM (x, y) the weight;

alpha represents S ₂ M S in (x, y) ₁ (x, y) weight.

4. A method of objective quality assessment of pictures based on joint multi-scale picture features according to claim 3, wherein the feature pooling step:

according to the obtained local quality graph similarity, calculating to obtain a final objective evaluation score, wherein the calculation formula is as follows:

w ₂ (x，y)＝max(y _1r ⁽²⁾ (x，y)，y _1d ⁽²⁾ (x，y))

wherein,,

s represents the final objective evaluation score;

w ₂ (x, y) represents a weight parameter;

y _1r ⁽²⁾ (x, y) represents the edge structure feature of the reference picture when n=2 at the point (x, y).

5. A desktop content picture objective quality evaluation system based on joint multi-scale picture characteristics is characterized by comprising:

and a picture processing module: processing the original image into image groups with different scales by using a Gaussian pyramid and a Laplacian pyramid, and respectively marking the image groups as y ₀ ⁽ⁿ⁾ And y is ₁ ⁽ⁿ⁾ From y ₁ ⁽ⁿ⁾ Extracting to obtain edge structure characteristics;

and the edge salient feature extraction module is used for: using the luminance mask and contrast mask, a group of pictures y is processed from a Gaussian pyramid ₀ ⁽ⁿ⁾ Picture group y processed by Laplacian pyramid ₁ ⁽ⁿ⁾ Extracting to obtain edge saliency characteristics;

and the feature similarity calculation module is used for: according to the obtained edge structure characteristics and edge saliency characteristics, calculating to obtain edge structure similarity and edge saliency similarity;

and the characteristic combination module is used for: according to the obtained edge structure similarity and edge saliency similarity, calculating to obtain a final local quality diagram;

and (3) a characteristic pooling module: calculating to obtain a final objective evaluation score according to the obtained final local quality map;

the picture processing module:

processing the original image into image groups with different scales by using a Gaussian pyramid and a Laplacian pyramid, and respectively marking the image groups as y ₀ ⁽ⁿ⁾ And y is ₁ ⁽ⁿ⁾ ；

The edge salient feature extraction module is used for:

using the luminance mask and contrast mask, a group of pictures y is processed from a Gaussian pyramid ₀ ⁽ⁿ⁾ Picture group y processed by Laplacian pyramid ₁ ⁽ⁿ⁾ The edge saliency characteristics are extracted and obtained, and the calculation formula is as follows:

/>

wherein,,

C _LM ⁽ⁿ⁾ representing a brightness mask calculation result, which is based on image characteristics of the image group processed by the Gaussian pyramid and the Laplacian pyramid;

representing a y function;

y ₁ ⁽ⁿ⁾ representing the picture group processed by the Laplacian pyramid;

y ₀ ⁽ⁿ⁾ representing a group of pictures after gaussian pyramid processing;

n represents the number of layers of the layer;

layer y ₁ ⁽¹⁾ The displayed picture features are edge structure features;

representing y ₀ ⁽ⁿ⁺¹⁾ Carrying in the calculation result of the y function;

γ ₁ representing a brightness contrast threshold;

the absolute value is taken;

a ₁ a constant that represents a guarantee of equation stability;

C _LCM ⁽ⁿ⁾ representing the contrast mask calculation result, based on C _LM ⁽ⁿ⁾ Image characteristics of the processed image group;

n represents the number of layers of the layer;

C _LCM ⁽¹⁾ representing the picture feature represented when n=1 as an edge saliency feature;

a ₂ a constant that represents a guarantee of equation stability;

representing C _LM ⁽ⁿ⁺¹⁾ Carrying in the calculation result of the y function;

γ ₂ representing a contrast detectable threshold;

g (x, y; sigma) represents a Gaussian kernel function;

* Representing a convolution;

∈2 represents upsampling;

the feature similarity calculation module is used for:

according to the obtained edge structure characteristics and edge saliency characteristics, the edge structure similarity is obtained through calculation, and the calculation formula is as follows:

wherein,,

S ₁ (x, y) represents the structural similarity of the (x, y) sides of the points;

subscripts r and d represent that the feature is taken from a reference picture or a distorted picture, respectively;

y _1r ⁽¹⁾ (x, y) represents an edge structure feature of the reference picture when n=1 at the point (x, y);

y _1d ⁽²⁾ (x, y) represents the edge structure feature of the distorted picture when n=2 at point (x, y);

T ₁ the representation is a non-zero constant to ensure the stability of the equation;

according to the obtained edge structure characteristics and edge saliency characteristics, the edge saliency similarity is obtained through calculation, and the calculation formula is as follows:

S ₂ (x，y)＝MS ₁ (x，y) ^α ·MS ₂ (x，y)

wherein,,

S ₂ (x, y) represents the point (x, y) side saliency similarity;

alpha represents S ₂ M S in (x, y) ₁ (x, y) the weight;

MS ₁ (x, y) represents calculation by similaritySimilarity of edge structures after function calculation;

MS ₂ (x, y) represents the edge significant similarity calculated by the similarity calculation function;

w ₁ (x, y) represents a weight factor;

C _LMr ⁽¹⁾ (x, y) represents LM mask characteristics when n=1 at point (x, y) of the reference picture;

C _LMd ⁽¹⁾ (x, y) represents LM mask characteristics when n=1 at point (x, y) of the reference picture;

T ₂ representing a non-zero constant to ensure the stability of the equation;

∑ _(x，y) w ₁ (x, y) represents w at all points on the picture ₁ Accumulation of (x, y);

C _LCMr ⁽¹⁾ (x, y) represents LCM mask features of the reference picture when n=1 at point (x, y);

C _LCMd ⁽¹⁾ (x, y) represents LCM mask features for a distorted picture when n=1 at point (x, y);

C _LCMd ⁽²⁾ (x, y) represents LCM mask features representing the distorted picture at point (x, y) where n=2;

T ₃ representing a non-zero constant to ensure the stability of the equation.

6. The desktop content picture objective quality assessment system based on joint multi-scale picture features of claim 5, wherein the feature combination module:

according to the obtained edge structure similarity and edge saliency similarity, calculating to obtain local quality similarity, wherein the calculation formula is as follows:

S _QM (x，y)＝(S ₁ (x，y)) ^ξ ·(S ₂ (x，y)) ^ψ

＝(S ₁ (x，y)) ^ξ ·MS ₁ (x，y) ^μ ·MS ₂ (x，y) ^ψ

μ＝ψ·α

wherein,,

S _QM (x, y) represents local mass similarity at point (x, y);

xi represents S ₁ (x, y) at local mass S _QM (x, y) the weight;

psi represents M S ₁ (x, y) at local mass S _QM (x, y) the weight;

mu represents M S ₂ (x, y) at local mass S _QM (x, y) the weight;

alpha represents S ₂ M S in (x, y) ₁ (x, y) the weight;

the feature pooling module:

according to the obtained local quality graph similarity, calculating to obtain a final objective evaluation score, wherein the calculation formula is as follows:

w ₂ (x，y)＝max(y _1r ⁽²⁾ (x，y)，y _1d ⁽²⁾ (x，y))

wherein,,

s represents the final objective evaluation score;

w ₂ (x, y) represents a weight parameter;

y _1r ⁽²⁾ (x, y) represents the edge structure feature of the reference picture when n=2 at the point (x, y).

7. A computer-readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the steps of the method for objective quality assessment of pictures based on joint multi-scale picture features of any one of claims 1 to 4.

Images