Summary of the invention
Technical problem to be solved by this invention is to provide a kind of objective evaluation method for quality of stereo images that can effectively improve the correlation of objective evaluation result and subjective perception.
The present invention solves the problems of the technologies described above the technical scheme that adopts: a kind of objective evaluation method for quality of stereo images is characterized in that comprising the following steps:
1. make S
orgUndistorted stereo-picture for original, make S
disFor the stereo-picture of distortion to be evaluated, with S
orgLeft visual point image be designated as L
org, with S
orgRight visual point image be designated as R
org, with S
disLeft visual point image be designated as L
dis, with S
disRight visual point image be designated as R
dis
2. to L
org, R
org, L
disAnd R
dis4 width images are implemented respectively singular value decomposition, obtain respectively L
org, R
org, L
disAnd R
disEach self-corresponding singular value vector of 4 width images, with L
orgThe singular value vector be designated as
With R
orgThe singular value vector be designated as
With L
disThe singular value vector be designated as
With R
disThe singular value vector be designated as
Wherein, the dimension of each singular value vector is m, and m=min (M, N), min () are for getting minimum value function, and the horizontal size of M presentation video is big or small, the vertical dimension size of N presentation video;
3. calculate L
orgThe singular value vector
With L
disThe singular value vector
The absolute difference vector, be designated as X
L,
With X
LAs L
disCharacteristic vector, calculate R
orgThe singular value vector
With R
disThe singular value vector
The absolute difference vector, be designated as X
R,
With X
RAs R
disCharacteristic vector, wherein, " || " is the symbol that takes absolute value;
4. to L
disCharacteristic vector X
LAnd R
disCharacteristic vector X
RCarry out linear weighted function, obtain S
disCharacteristic vector, be designated as X, X=w
LX
L+ w
RX
R, wherein, w
LExpression L
disWeights proportion, w
RExpression R
disWeights proportion, w
L+ w
R=1;
5. adopt n undistorted stereo-picture, set up its distortion stereo-picture set under the different distortion levels of different type of distortion, this distortion stereo-picture set comprises the stereo-picture of several distortions, utilizes the subjective quality assessment method to obtain respectively the average subjective scoring difference of the stereo-picture of every width distortion in the set of distortion stereo-picture, is designated as DMOS, DMOS=100-MOS, wherein, MOS represents the subjective scoring average, DMOS ∈ [0,100], n 〉=1;
6. adopt and calculate S
disThe identical method of characteristic vector X, the characteristic vector of the stereo-picture of every width distortion in the set of calculated distortion stereo-picture respectively, the characteristic vector for the stereo-picture of i width distortion in the set of distortion stereo-picture, be designated as X with it
i, wherein, 1≤i≤n ', the width number of the stereo-picture of the distortion that comprises in the set of n ' expression distortion stereo-picture;
7. adopt support vector regression to train the characteristic vector of the stereo-picture of all distortions of identical type of distortion in the set of distortion stereo-picture, and the support vector regression training pattern of utilizing training to obtain is tested the stereo-picture of every width distortion of same type of distortion, calculate the evaluating objective quality predicted value of the stereo-picture of every width distortion of identical type of distortion in the set of distortion stereo-picture, evaluating objective quality predicted value for the stereo-picture of i width distortion in the set of distortion stereo-picture, be designated as Q with it
i, Q
i=f (X
i), f () is the function representation form, Q
i=f (X
i) expression Q
iFor X
iFunction.
Described step detailed process 2. is:
2.-1, with size be the L of M * N
orgBe expressed as the two-dimensional matrix of M * N dimension, be designated as
By the two-dimensional matrix of singular value decomposition with M * N dimension
Be expressed as
Wherein,
The orthogonal matrix of expression M * M dimension,
The orthogonal matrix of expression N * N dimension,
Expression
Transposed matrix,
The diagonal matrix of expression M * N dimension;
2.-2, with the diagonal matrix of M * N dimension
Diagonal element as the two-dimensional matrix of M * N dimension
Singular value, from the two-dimensional matrix of M * N dimension
Singular value in take out the singular value formation L of m non-zero
orgThe singular value vector, be designated as
Wherein, m=min (M, N), min () is for getting minimum value function;
2.-3, to R
org, L
disAnd R
disAdopt the operation identical with step 2.-1 to 2.-2, obtain R
org, L
disAnd R
disThe singular value vector, be designated as respectively
With
Described step detailed process 7. is:
7.-1, the stereo-picture of all distortions of same type of distortion in the set of distortion stereo-picture is divided into mutually disjoint 5 groups of subsets, selects arbitrarily 4 groups of subset composing training sample datas set wherein, be designated as Ω
q, { X
k, DMOS
k∈ Ω
q, wherein, q represents training sample data set omega
qIn the width number of stereo-picture of the distortion that comprises, X
kExpression training sample data set omega
qIn the characteristic vector of stereo-picture of k width distortion, DMOS
kExpression training sample data set omega
qIn the average subjective scoring difference of stereo-picture of k width distortion, 1≤k≤q;
7.-2, structure X
kRegression function f (X
k),
Wherein, f () is the function representation form, and w is weight vector, w
TFor the transposed matrix of w, b is bias term,
Expression training sample data set omega
qIn the linear function of characteristic vector Xk of stereo-picture of k width distortion,
D(X
k, X
l) be the kernel function in support vector regression,
X
lFor training sample data set omega
qIn the characteristic vector of stereo-picture of l width distortion, γ is nuclear parameter, is used for the scope of reflection input sample value, the scope of sample value is larger, and the γ value is also just larger, the exponential function of exp () expression take e the end of as, e=2.71828183, " || || " for asking the Euclidean distance symbol;
7.-3, adopt support vector regression to training sample data set omega
qIn the characteristic vector of stereo-picture of all distortion train, make the regression function value and the error between average subjective scoring difference that obtain through training minimum, match obtains optimum weight vector w
optBias term b with optimum
opt, with the weight vector w of optimum
optBias term b with optimum
optCombination be designated as
The weight vector w of the optimum that utilization obtains
optBias term b with optimum
optStructure support vector regression training pattern, be designated as
Wherein, Ψ represents training sample data set omega
qIn the set of combination of the characteristic vector of stereo-picture of all distortion all weight vector of training and bias term,
Expression minimizes probability density function, X
inpExpress support for the input vector of vector regression training pattern, (w
opt)
TFor w
optTransposed matrix,
Express support for the input vector X of vector regression training pattern
inpLinear function;
7.-4, according to the support vector regression training pattern, the stereo-picture that remains the every width distortion in 1 group of subset is tested, prediction obtains the evaluating objective quality predicted value of the stereo-picture of every width distortion in this group subset, evaluating objective quality predicted value for the stereo-picture of j width distortion in this group subset, be designated as Q with it
j, Q
j=f (X
j),
Wherein, X
jThe characteristic vector that represents the stereo-picture of j width distortion in this group subset,
The linear function that represents the stereo-picture of j width distortion in this group subset;
7.-5, according to the process of step 7.-1 to 7.-4, respectively the stereo-picture of all distortions of different type of distortion in the set of distortion stereo-picture is trained, obtain the evaluating objective quality predicted value of the stereo-picture of every width distortion in the set of distortion stereo-picture.
Described step 6. in the characteristic vector process of the stereo-picture that calculates the JPEG compression artefacts, get w
L=0.50, w
R=0.50; In the characteristic vector process of the stereo-picture that calculates the JPEG2000 compression artefacts, get w
L=0.15, w
R=0.85; In the characteristic vector process of the stereo-picture that calculates the Gaussian Blur distortion, get w
L=0.10, w
R=0.90; In the characteristic vector process of the stereo-picture that calculates the white noise distortion, get w
L=0.20, w
R=0.80; In calculating the characteristic vector process of the stereo-picture of coding distortion H.264, get w
L=0.10, w
R=0.90.
Compared with prior art, the invention has the advantages that:
1) the inventive method is mapped to the characteristic vector of stereo-picture in a high-dimensional feature space by support vector regression, carry out again Linear Estimation in high-dimensional feature space, the characteristic vector of structure optimum regression function stereoscopic image is tested, avoided human visual system's correlation properties and the complicated simulation process of mechanism, and because training sample and test sample book are separate, can avoid test result to the depending on unduly of training data, thereby can effectively improve the correlation of objective evaluation result and subjective perception.
2) the inventive method adopts singular value decomposition method to obtain the left visual point image of stereo-picture and the characteristic vector of right visual point image, again according to the different type of distortion situations of stereo-picture, adopt different weights proportion to carry out linear weighted function to the characteristic vector of its left visual point image and right visual point image, obtain the characteristic vector information of stereo-picture, the characteristic vector information of the stereo-picture that obtains has stronger stability and can reflect preferably the mass change situation of stereo-picture, can reflect well the stereoscopic vision masking effect of human eye.
Description of drawings
Fig. 1 be the inventive method totally realize block diagram;
Fig. 2 a is the left visual point image of Akko (being of a size of 640 * 480) stereo-picture;
Fig. 2 b is the right visual point image of Akko (being of a size of 640 * 480) stereo-picture;
Fig. 3 a is the left visual point image of Altmoabit (being of a size of 1024 * 768) stereo-picture;
Fig. 3 b is the right visual point image of Altmoabit (being of a size of 1024 * 768) stereo-picture;
Fig. 4 a is the left visual point image of Balloons (being of a size of 1024 * 768) stereo-picture;
Fig. 4 b is the right visual point image of Balloons (being of a size of 1024 * 768) stereo-picture;
Fig. 5 a is the left visual point image of Doorflower (being of a size of 1024 * 768) stereo-picture;
Fig. 5 b is the right visual point image of Doorflower (being of a size of 1024 * 768) stereo-picture;
Fig. 6 a is the left visual point image of Kendo (being of a size of 1024 * 768) stereo-picture;
Fig. 6 b is the right visual point image of Kendo (being of a size of 1024 * 768) stereo-picture;
Fig. 7 a is the left visual point image of LeaveLaptop (being of a size of 1024 * 768) stereo-picture;
Fig. 7 b is the right visual point image of LeaveLaptop (being of a size of 1024 * 768) stereo-picture;
Fig. 8 a is the left visual point image of Lovebierd1 (being of a size of 024 * 768) stereo-picture;
Fig. 8 b is the right visual point image of Lovebier1 (being of a size of 1024 * 768) stereo-picture;
Fig. 9 a is the left visual point image of Newspaper (being of a size of 1024 * 768) stereo-picture;
Fig. 9 b is the right visual point image of Newspaper (being of a size of 1024 * 768) stereo-picture;
Figure 10 a is the left visual point image of Puppy (being of a size of 720 * 480) stereo-picture;
Figure 10 b is the right visual point image of Puppy (being of a size of 720 * 480) stereo-picture;
Figure 11 a is the left visual point image of Soccer2 (being of a size of 720 * 480) stereo-picture;
Figure 11 b is the right visual point image of Soccer2 (being of a size of 720 * 480) stereo-picture;
Figure 12 a is the left visual point image of Horse (being of a size of 720 * 480) stereo-picture;
Figure 12 b is the right visual point image of Horse (being of a size of 720 * 480) stereo-picture;
Figure 13 a is the left visual point image of Xmas (being of a size of 640 * 480) stereo-picture;
Figure 13 b is the right visual point image of Xmas (being of a size of 640 * 480) stereo-picture;
Figure 14 is the scatter diagram of objective image quality evaluation predicted value and average subjective scoring difference of the stereo-picture of each distortion in the set of distortion stereo-picture.
Embodiment
Embodiment is described in further detail the present invention below in conjunction with accompanying drawing.
A kind of objective evaluation method for quality of stereo images that the present invention proposes, it totally realizes block diagram as shown in Figure 1, it mainly comprises the following steps:
1. make S
orgUndistorted stereo-picture for original, make S
disFor the stereo-picture of distortion to be evaluated, with S
orgLeft visual point image be designated as L
org, with S
orgRight visual point image be designated as R
org, with S
disLeft visual point image be designated as L
dis, with S
disRight visual point image be designated as R
dis
2. to L
org, R
org, L
disAnd R
dis4 width images are implemented respectively singular value decomposition, obtain respectively L
org, R
org, L
disAnd R
disEach self-corresponding singular value vector of 4 width images, with L
orgThe singular value vector be designated as
With R
orgThe singular value vector be designated as
With L
disThe singular value vector be designated as
With R
disThe singular value vector be designated as
Wherein, the dimension of each singular value vector is m, and m=min (M, N), min () are for getting minimum value function, and the horizontal size of M presentation video is big or small, the vertical dimension size of N presentation video.
In the present embodiment, step detailed process 2. is:
2.-1, with size be the L of M * N
orgBe expressed as the two-dimensional matrix of M * N dimension, be designated as
By the two-dimensional matrix of singular value decomposition with M * N dimension
Be expressed as
Wherein,
The orthogonal matrix of expression M * M dimension,
The orthogonal matrix of expression N * N dimension,
Expression
Transposed matrix,
The diagonal matrix of expression M * N dimension;
2.-2, with the diagonal matrix of M * N dimension
Diagonal element as the two-dimensional matrix of M * N dimension
Singular value, from the two-dimensional matrix of M * N dimension
Singular value in take out the singular value formation L of m non-zero
orgThe singular value vector, be designated as
Wherein, m=min (M, N), min () is for getting minimum value function;
2.-3, to R
org, L
disAnd R
disAdopt the operation identical with step 2.-1 to 2.-2, obtain R
org, L
disAnd R
disThe singular value vector, be designated as respectively
With
3. calculate L
orgThe singular value vector
With L
disThe singular value vector
The absolute difference vector, be designated as X
L,
With X
LAs L
disCharacteristic vector, calculate R
orgThe singular value vector
With R
disThe singular value vector
The absolute difference vector, be designated as X
R,
With X
RAs R
disCharacteristic vector, wherein, " || " is the symbol that takes absolute value.
4. to L
disCharacteristic vector X
LAnd R
disCharacteristic vector X
RCarry out linear weighted function, obtain S
disCharacteristic vector, be designated as X, X=w
LX
L+ w
RX
R, wherein, w
LExpression L
disWeights proportion, w
RExpression R
disWeights proportion, w
L+ w
R=1.
5. adopt n undistorted stereo-picture, set up its distortion stereo-picture set under the different distortion levels of different type of distortion, this distortion stereo-picture set comprises the stereo-picture of several distortions, utilizes existing subjective quality assessment method to obtain respectively the average subjective scoring difference of the stereo-picture of every width distortion in the set of distortion stereo-picture, is designated as DMOS, DMOS=100-MOS, wherein, MOS represents the subjective scoring average, DMOS ∈ [0,100], n 〉=1.
in the present embodiment, utilize the stereo-picture as Fig. 2 a and Fig. 2 b formation, the stereo-picture that Fig. 3 a and Fig. 3 b form, the stereo-picture that Fig. 4 a and Fig. 4 b form, the stereo-picture that Fig. 5 a and Fig. 5 b form, the stereo-picture that Fig. 6 a and Fig. 6 b form, the stereo-picture that Fig. 7 a and Fig. 7 b form, the stereo-picture that Fig. 8 a and Fig. 8 b form, the stereo-picture that Fig. 9 a and Fig. 9 b form, the stereo-picture that Figure 10 a and Figure 10 b form, the stereo-picture that Figure 11 a and Figure 11 b form, the stereo-picture that Figure 12 a and Figure 12 b form, the stereo-picture that Figure 13 a and Figure 13 b the form undistorted stereo-picture of totally 12 width (n=12) has been set up its distortion stereo-picture set under the different distortion levels of different type of distortion, this distortion stereo-picture set comprises the stereo-picture of 312 width distortions of 5 kinds of type of distortion altogether, the stereo-picture of the distortion of JPEG compression totally 60 width wherein, the stereo-picture of the distortion of JPEG2000 compression is totally 60 width, the stereo-picture of the distortion of Gaussian Blur (Gaussian Blur) is totally 60 width, the stereo-picture of the distortion of white noise (White Noise) is totally 60 width, the stereo-picture of the distortion of H.264 encoding is totally 72 width.
6. adopt and calculate S
disThe identical method of characteristic vector X, the characteristic vector of the stereo-picture of every width distortion in the set of calculated distortion stereo-picture respectively, the characteristic vector for the stereo-picture of i width distortion in the set of distortion stereo-picture, be designated as X with it
i, wherein, 1≤i≤n ', the width number of the stereo-picture of the distortion that comprises in the set of n ' expression distortion stereo-picture.
In this specific embodiment, according to the stereoscopic vision masking effect inconsistent characteristic of human eye to different type of distortion, left visual point image and right visual point image to the stereo-picture of different type of distortion arrange different weights proportion, in the characteristic vector process of the stereo-picture that calculates the JPEG compression artefacts, get w
L=0.50, w
R=0.50; In the characteristic vector process of the stereo-picture that calculates the JPEG2000 compression artefacts, get w
L=0.15, w
R=0.85; In the characteristic vector process of the stereo-picture that calculates the Gaussian Blur distortion, get w
L=0.10, w
R=0.90; In the characteristic vector process of the stereo-picture that calculates the white noise distortion, get w
L=0.20, w
R=0.80; In calculating the characteristic vector process of the stereo-picture of coding distortion H.264, get w
L=0.10, w
R=0.90.
7. the characteristic vector due to the stereo-picture of distortion is the higher dimensional space vector, need to construct linear decision function and realize non-linear decision function in former space in higher dimensional space, support vector regression (Support Vector Regression, SVR) is a kind of reasonable method that realizes non-linear higher dimensional space conversion.Adopt support vector regression to train the characteristic vector of the stereo-picture of all distortions of identical type of distortion in the set of distortion stereo-picture, and the support vector regression training pattern of utilizing training to obtain is tested the stereo-picture of every width distortion of same type of distortion, calculate the evaluating objective quality predicted value of the stereo-picture of every width distortion of identical type of distortion in the set of distortion stereo-picture, evaluating objective quality predicted value for the stereo-picture of i width distortion in the set of distortion stereo-picture, be designated as Q with it
i, Q
i=f (X
i), f () is the function representation form, Q
i=f (X
i) expression Q
iFor X
iFunction.
In this specific embodiment, step detailed process 7. is:
7.-1, the stereo-picture of all distortions of same type of distortion in the set of distortion stereo-picture is divided into mutually disjoint 5 groups of subsets, selects arbitrarily 4 groups of subset composing training sample datas set wherein, be designated as Ω
q, { X
k, DMOS
k∈ Ω
q, wherein, q represents training sample data set omega
qIn the width number of stereo-picture of the distortion that comprises, X
kExpression training sample data set omega
qIn the characteristic vector of stereo-picture of k width distortion, DMOS
kExpression training sample data set omega
qIn the average subjective scoring difference of stereo-picture of k width distortion, 1≤k≤q;
7.-2, structure X
kRegression function f (X
k),
Wherein, f () is the function representation form, and w is weight vector, w
TFor the transposed matrix of w, b is bias term,
Expression training sample data set omega
qIn the characteristic vector X of stereo-picture of k width distortion
kLinear function,
D(X
k, X
l) be the kernel function in support vector regression,
X
lFor training sample data set omega
qIn the characteristic vector of stereo-picture of l width distortion, γ is nuclear parameter, is used for the scope of reflection input sample value, the scope of sample value is larger, and the γ value is also just larger, the exponential function of exp () expression take e the end of as, e=2.71828183, " || || " for asking the Euclidean distance symbol;
In the present embodiment, JPEG compression artefacts, JPEG 2000 compression artefacts, Gaussian Blur distortion, white noise distortion and H.264 the γ value of coding distortion get respectively 42,52,54,130 and 116.
7.-3, adopt support vector regression to training sample data set omega
qIn the characteristic vector of stereo-picture of all distortion train, make the regression function value and the error between average subjective scoring difference that obtain through training minimum, match obtains optimum weight vector w
optBias term b with optimum
opt, with the weight vector w of optimum
optBias term b with optimum
optCombination be designated as (w
opt, b
opt),
The weight vector w of the optimum that utilization obtains
optBias term b with optimum
optStructure support vector regression training pattern, be designated as
Wherein, Ψ represents training sample data set omega
qIn the set of combination of the characteristic vector of stereo-picture of all distortion all weight vector of training and bias term,
Expression minimizes probability density function, X
inpExpress support for the input vector of vector regression training pattern, (w
opt)
TFor w
optTransposed matrix,
Express support for the input vector X of vector regression training pattern
inpLinear function;
7.-4, according to the support vector regression training pattern, the stereo-picture that remains the every width distortion in 1 group of subset is tested, prediction obtains the evaluating objective quality predicted value of the stereo-picture of every width distortion in this group subset, evaluating objective quality predicted value for the stereo-picture of j width distortion in this group subset, be designated as Q with it
j, Q
j=f (X
j),
Wherein, X
jThe characteristic vector that represents the stereo-picture of j width distortion in this group subset,
The linear function that represents the stereo-picture of j width distortion in this group subset;
7.-5, according to the process of step 7.-1 to 7.-4, respectively the stereo-picture of all distortions of different type of distortion in the set of distortion stereo-picture is trained, obtain the evaluating objective quality predicted value of the stereo-picture of every width distortion in the set of distortion stereo-picture.
Adopt 12 undistorted stereo-pictures shown in Fig. 2 a to Figure 13 b to analyze objective image quality evaluation predicted value and the average correlation between the subjective scoring difference of the stereo-picture of the distortion that the present embodiment obtains at the stereo-picture of in various degree JPEG compression, JPEG2000 compression, Gaussian Blur, white noise and H.264 312 width distortions in the coding distortion situation.Here, utilize 2 objective parameters commonly used of evaluate image quality evaluating method as evaluation index, be Pearson correlation coefficient (the Correlation Coefficient under the nonlinear regression condition, CC), Spearman coefficient correlation (Rank-Order Correlation Coefficient, ROCC), the stereo-picture of CC reflection distortion is estimated the accuracy of objective models, and ROCC reflects its monotonicity.The objective image evaluation quality predicted value of the stereo-picture of the distortion that will calculate by the present embodiment is done four parameter L ogistic function nonlinear fittings, and the higher explanation method for objectively evaluating of CC and ROCC value is better with average subjective scoring difference correlation.CC, the ROCC coefficient of reflection three-dimensional image objective evaluation model performance are as shown in table 1, from the listed data of table 1 as can be known, correlation between the final objective image quality evaluation predicted value of the stereo-picture of the distortion that obtains by the present embodiment and average subjective scoring difference is very high, the result that shows objective evaluation result and human eye subjective perception is more consistent, is enough to illustrate the validity of the inventive method.
Figure 14 has provided the scatter diagram of objective image quality evaluation predicted value with the average subjective scoring difference of the stereo-picture of each distortion in the set of distortion stereo-picture, curve is obtained by four parameter L ogistic function nonlinear fittings, loose point is more concentrated, illustrates that the consistency of objective models and subjective perception is better.As can be seen from Figure 14, the scatter diagram that adopts the inventive method to obtain is more concentrated, and the goodness of fit between the subjective assessment data is higher.
The image quality evaluation predicted value of the stereo-picture of the distortion that table 1 the present embodiment obtains and the correlation between subjective scoring