CN104506852B - A kind of objective quality assessment method towards video conference coding - Google Patents

Abstract

The invention discloses a kind of objective quality assessment method towards video conference coding, including training and assessment two parts；Training department divides and includes step one: face and face area extract；Step 2: obtain the concerned degree of single pixel；Step 3: face area is calibrated and normalizes；Step 4: obtain gauss hybrid models；Evaluation part includes step one: for one group of video, automatically extract out number of pixels in background, face, left eye, right eye, mouth, nasal area；Step 2: face area is calibrated and normalizes；Step 3: obtain weight collection of illustrative plates；Step 4: calculate Y-PSNR based on gauss hybrid models, the picture quality after assessment video conferencing system coding.Present invention, avoiding conventional method and do not take into account the deficiency of video content, the precision of image quality measure can be promoted so that it is more reflect the result of subjective quality assessment by giving video image face more weight.

Description

A kind of objective quality assessment method towards video conference coding

Technical field

The present invention relates to a kind of objective quality assessment method towards video conference coding, belong to the sense of video conference coding Know visual quality assessment technology field.

Background technology

When assessing the efficiency of different Video coding modes, the index of visual quality is requisite.Perception video The visual quality of coding is assessed and can be divided into two classes: subjective evaluation and objective evaluation.Owing to the mankind are the most straight when watching video The recipient connect, subjective visual quality assessment is the most accurate in the method assessing Video coding, most reliable.But its inefficient and High cost promotes the development of the evaluation index of objective visual quality.The purpose of objective evaluation is to improve itself and subjective visual quality Correlation, to accurately measure visual quality.Most widely used objective indicator includes peak signal noise ratio (peak Signal-to-noise ratio, PSNR), structural similarity (structural similarity, SSIM), visual signal is made an uproar Acoustic ratio (visual signal-to-noise ratio, VSNR), visual quality metrics (video quality metrics, VQM) and based drive integrity of video evaluation (MOtion-based Video Integrity Evaluation, MOVIE)。

The video conference of perception Video coding has been widely studied, because face is one for video conference ROI (Region-of-Interest, area-of-interest).But, there are currently no the objective vision exclusively for video conference exploitation Method for evaluating quality.

Summary of the invention

The invention aims to solve the deficiency of the objective evaluation method of existing video quality, it is provided that Yi Zhongzhen Objective indicator to video conference coding, it is intended to improve the correlation between the subjective perceptual quality of beholder.

A kind of objective quality assessment method towards video conference coding, including training and assessment two parts；

Training department divides and includes following step:

Step one: face and face area extract；

Step 2: carry out eye tracker experiment, obtains the focus coordinate for each two field picture when tester watches video Position, obtains the concerned degree of single pixel；

Step 3: face area is calibrated and normalizes；

Step 4: obtain gauss hybrid models；

Evaluation part includes following step:

Step one: for one group of video, repetition training part steps one, automatically extract out background, face, left eye, right eye, Number of pixels in mouth, nasal area；

Step 2: the step 3 of repetition training process, calibrates face area and normalizes；

Step 3: obtain on the basis of gauss hybrid models in the training stage, calculate right eye, left eye, mouth, nose, face its Gaussian Profile weight around his region, the weight of background area and above each region, obtains weight collection of illustrative plates；

Step 4: on the basis of weight collection of illustrative plates, calculates Y-PSNR based on gauss hybrid models, assesses video conference Picture quality after system coding.

It is an advantage of the current invention that:

(1) the present invention is directed to the image quality measure method after video conferencing system coding, it is to avoid conventional method is not examined Consider the deficiency to video content, the precision of image quality measure can be promoted by giving video image face more weight, make It more reflects the result of subjective quality assessment；

(2) present invention is on the basis of each region of face (such as nose, face) extracts, and some key areas for face are composed Give bigger weight, thus meet current and future video conference system resolution ratio improves constantly, display size is increasing Development trend；

(3) present invention is by introducing the experimental data of eye tracker, in conjunction with the calculating instrument of statistical learning, can excavate at video The rule of people's visual attention during meeting, is applied to the image quality measure after video conferencing system coding, greatly further Width improves the degree of correlation of itself and subjective quality assessment.

Accompanying drawing explanation

Fig. 1 is the method flow diagram of the present invention；

Fig. 2 face feature automatic Calibration algorithm；

Automatically extracting of Fig. 3 face key area；

Fig. 4 calibration and normalized method；

The method for drafting of Fig. 5 weight collection of illustrative plates；

The calculating signal of Fig. 6 GMM-PSNR.

Detailed description of the invention

Below in conjunction with drawings and Examples, the present invention is described in further detail.

The method that present invention employs real time facial characteristic automatic Calibration, to follow the tracks of the key feature points of face.At face After detection, by combining this locality detection (texture information) and global optimization (face structure), the distributed mode of the point of key feature Type (PDM) generates in frame of video.In the present invention, face and the profile of face that 66 PDM extract are utilized.The PDM of 66 Can carry out the sampling of key point of face and facial characteristics well, and therefore these points can connect accurately to extract Face and the profile of face feature and region.Therefore, in the method that the PDM of 66 is used in, face and face's key with extraction are special Levy.Finally, face and face's key area are according to their contours extract.

In the video of conversational class scene, being found through experiments, in video, face content can attract the biggest portion of observer Divide concern power.Therefore, according to the difference of the concern power of observer, quantify the unequal importance of background, face and face further, Thus promote the objective quality assessment accuracy of video conference.In order to obtain the value of such unequal importance, in meeting The experiment of some eye tracker has been carried out on relevant video.

In an experiment, using eye tracker to have recorded observer when watching video, fall the eye gaze point in frame of video.Eye The focus of eyeball blinkpunkt representative observer, therefore, the result of eye tracking can be used to produce subjective concern model.Eye After dynamic instrument experiment, the number of the eye gaze point belonging to right eye, left eye, mouth, nose, other regions of face, and background is recorded. According to the number of eyeball fixes point fallen in zones of different, introduce a new concept, eye gaze point/pixel (EFP/P), with It is reflected in the pixel level of these region attention rates.Here, there is following EFP/P value.

After obtaining the result of above-mentioned eye tracker experiment, use it to train GMM, to produce the importance power of each frame of video Multigraph is composed.Therefore, GMM-PSNR can calculate by combining corresponding weight collection of illustrative plates.Before training GMM, pre-process The eye gaze point obtained at upper joint with calibration and normalization.Subsequently, in calibration and normalized eyeball fixes point, with expectation Maximize (EM) algorithm and train GMM.GMM can by run several times EM iteration until convergence and obtain.In view of the GMM obtained Parameter, weight collection of illustrative plates can be calculated, set up objective metric G MM-PSNR.

The present invention is a kind of objective quality assessment method towards video conference coding, and flow process is as it is shown in figure 1, include training With assessment two parts；

Training department divides and includes following step:

Step one: face and face area extract；

Face feature automatic Calibration algorithm is utilized to automatically extract out background, face, a left side in given video conference sequence Number of pixels in eye, right eye, mouth, nasal area.

Particularly as follows: first, obtain in video conference sequence in each two field picture by face feature automatic Calibration algorithm Face area key point, second, utilize mean value drift technology on the face area extracted, Local Search face area figure Left eye in Xiang, right eye, mouth, nasal area key point, and by these key points and the key point distributed model in database (PDM) mate, it is achieved left eye, right eye, mouth, nasal area key point optimize, the 3rd, optimized after each two field picture In face, left eye, right eye, mouth, nasal area key point, as in figure 2 it is shown, there are 66 key points, the 4th, respectively by face Portion, left eye, right eye, mouth, the key point of nasal area are connected, and obtain face, left eye, right eye, mouth, nose profile, such as Fig. 3 institute Show, the 5th, obtain number of pixels in face, left eye, right eye, mouth, nasal area respectively, image pixel number is deducted face's picture Element number, obtains background pixel number, finally realizes automatically extracting of face's key area.

Wherein, points distribution models is to use mean value drift technology, by the training to one group of standard testing image.

Wherein it is possible to the face extracted in different facial image, left eye, right eye, mouth, nasal area key point.

Step 2: carry out eye tracker experiment, obtains the focus coordinate for each two field picture when tester watches video Position, obtains the concerned degree of single pixel；

Pay close attention to if the concerned degree of single region (left eye, right eye, mouth, nose, other regions of face, background) is eyes Count out/this area pixel number (efp/p):

$\left\{\begin{array}{c}{c}_r=f_r/p_r\\ c_l=f_l/p_l\\ c_m=f_m/p_m\\ c_n=f_n/p_n\\ c_o=f_o/p_o\\ c_b=f_b/p_b\end{array}\right.$

Wherein: c_r、c_l、c_m、c_n、c_o、c_bRepresent right eye, left eye, mouth, nose, face other regions, background area respectively The degree of concern of single pixel, f_r、f_l、f_m、f_n、f_o、f_bBeing illustrated respectively in eye tracker experiment, tester falls on right eye, a left side Eye, mouth, nose, other regions of face, the eyes focus number of background area, p_r、p_l、p_m、p_n、p_o、p_bRepresent right eye, a left side respectively Pixel number in eye, mouth, nose, other regions of face, background area；

Step 3: face area is calibrated and normalizes；

Calibration can avoid the uncertainty that face diverse location in the picture is caused, and normalized method can make Obtain the present invention and be adapted to the situation that in video conference, human face region number of pixels does not waits.

Method particularly includes:

As shown in Fig. 4 (a), randomly select a two field picture, use leftmost side point in image face area key point, as school Quasi-original point B, obtains in other images point A in the leftmost side in face area key point, obtains coordinate transformation relation between A, B, will In other images, focus is changed according to coordinate transformation relation, completes calibration.

As shown in Fig. 4 (b), randomly selecting a two field picture, in employing image, the abscissa length of personage's right eye is (in 66 o'clock Right eye on the right side of point and the point on the left of right eye between distance) as normalization unit, by the focus root in other images It is normalized according to normalization unit.

Step 4: obtain gauss hybrid models；

Assume that eye gaze point obeys gauss hybrid models, on the basis of normalization with calibration eye tracker data, pass through The linear superposition that gauss hybrid models is write as Gaussian component is as follows:

$p\left(x^{*}\right)=\underset{k=1}{\overset{K}{\mathrm{\Σ}}}{\mathrm{\π}}_k{\ℵ}_k\left(x^{*}\right)$

${\ℵ}_k\left(x^{*}\right)=\frac{1}{2\mathrm{\π}}\·\frac{1}{{\left|{\mathrm{\Σ}}_k\right|}^{\frac{1}{2}}}\·\exp \{-\frac{1}{2}{(x^{*}-{\mathrm{\μ}}_k)}^T\·{\mathrm{\Σ}}_k^{-1}\·(x^{*}-{\mathrm{\μ}}_k)\}$

Wherein:Represent a Gaussian component, π_k,μ_kAnd Σ_kThe mixed coefficint of kth Gaussian component, average and Variance, and x^*Represent the eye gaze point after two-dimensional calibration and normalization.K represents the quantity of the Gaussian component of GMM.Due to nose The quantity of eye gaze point than eyes and the much less of mouth, here the number K of Gaussian component is set to 3, each of which Corresponding to right eye, left eye and mouth.Meanwhile, by μ_kIt is set to the normalization barycenter of each face feature.

Above-mentioned steps in off-line case, for one group of training video, is tested and data analysis by design eye tracker, Obtain the gauss hybrid models for assessing video conferencing system objective quality.

(2) evaluation part includes following step

Step one: the step one with training process is identical, automatically extracts out background, face, left eye, right eye, mouth, nose region Territory.

Step 2: the step 3 with training process is identical, calibrates and normalize the face area of video.Particular content is shown in figure 4。

Step 3: obtain on the basis of gauss hybrid models in the training stage, calculate right eye, left eye, mouth, nose, face its Gaussian Profile weight around his region, the weight of background area and above each region, particular content is shown in Fig. 5.

Fig. 5 is the method for drafting of weight collection of illustrative plates.In this example is implemented, weight collection of illustrative plates quantitation video conference system can be passed through Middle face and the importance of each pixel of background.The two field picture that input is video conference of this example.First, according to the side of Fig. 3 Method automatically extracts out face's key area with face.Secondly, the key point in video is calibrated according to the method for Fig. 4 And normalization.Finally, use Fig. 1 training department step by step two, four GMM train the parameter that obtains, according to each pixel affiliated area (mainly have powerful connections, face, left eye, right eye, nose, face), calculated the weight of each pixel by formula below, and export should The weight collection of illustrative plates of video conference image, sets the image each pixel importance when quality evaluation by weight size.

Wherein,

$g\left(x\right)=\frac{\underset{k}{\max }{\mathrm{\π}}_k{\ℵ}_k\left(x\right)}{{\mathrm{\Σ}}_{x\∈\mathrm{others}}\underset{k}{\max }{\mathrm{\π}}_k{\ℵ}_k\left(x\right)}\·p_o$

The present invention is not limited to adopt the weight size setting image pixel in this way.

Step 4: on the basis of weight collection of illustrative plates, calculates Y-PSNR (GMM-PSNR) based on gauss hybrid models, comments Estimate the picture quality after video conferencing system coding.Particular content is shown in Fig. 4.

Fig. 6 is the calculating signal of GMM-PSNR.In this example is implemented, the calculating exportable measurement video council of GMM-PSNR The GMM-PSNR of conference system encoded images quality.First, with tradition balancing method (such as PSNR), by calculating original regarding Frequently image and the root-mean-square error of video image to be assessed, obtain the residual error of image before and after encoding.Then, by root-mean-square error with The weighting of weight collection of illustrative plates is multiplied, and i.e. can get the value of GMM-MSE.Finally, by the method taken the logarithm, GMM-PSNR is calculated. Circular and computing formula thereof are in the explanation of Fig. 1.The present invention is not limited to the improvement to traditional PS NR.Also can be to it He is carried out by being multiplied with the weighting in weight collection of illustrative plates by balancing method (such as SSIM=Structural SIMilarity, SSIM) Improve.

Specific formula for calculation is as follows:

${\mathrm{MSE}}_{\mathrm{GMM}}=\frac{{\mathrm{\Σ}}_{i=1}^M{\mathrm{\Σ}}_{j=1}^N{({\mathrm{\ω}}_x\·(I_x^{\′}-I_x))}^2}{{\mathrm{\Σ}}_{i=1}^M{\mathrm{\Σ}}_{j=1}^N{\mathrm{\ω}}_x^2}$

${\mathrm{PSNR}}_{\mathrm{GMM}}=10\·\log \frac{{(2^n-1)}^2}{{\mathrm{MSE}}_{\mathrm{GMM}}}$

Wherein I '_xAnd I_xIt is to process the value of pixel x in video and original video frame respectively.M and N is along Vertical Square respectively To the pixel count with horizontal direction.N (=8) is bit depth.

Finally, Y-PSNR (GMM-based on gauss hybrid models after the present invention exportable video conferencing system coding PSNR), the reduction situation of picture quality before and after Video coding of weighing it is used for.Identical with conventional peak signal to noise ratio (PSNR), GMM- The unit of measurement of PSNR is dB.But, due to people when watching video for image in the attention rate in each region different, GMM- PSNR gives, for the face area that importance in video conferencing system does not waits, the weight that varies in size, thus be substantially improved its with The degree of correlation of subjective quality assessment.

The present invention can provide a kind of more efficiently evaluating method for the quality of transmission of video in video conference.Pass through Test, relative to traditional Objective Video appraisal procedure, for VQM, MOVIE, PSNR, GMM-PSNR significantly improves with main See testing standard, such as MOS, DMOS, between correlation, illustrate that GMM-PSNR can be as one more effectively towards video The objective metric of meeting coding.This is all highly beneficial for Video processing, compression and the video communication of video conference.It Can monitor the performance of video system, and provide regulation codec or the feedback of channel parameter, it is ensured that video quality is can In the range of acceptance.Video quality assessment standard can be used for the design to codec performance, evaluates and optimizes.It also may be used The Digital Video System of visual model is met for design and optimization.

The present invention relates to the objective quality assessment method of video sequence, the perception visual quality for video conference coding is commented Estimate.Present invention employs eye tracker experiment and face and the real-time technique of face feature extraction.In an experiment, background, face The attention rate of various piece is determined based on observer with the importance in face feature region.Utilize the eye that eye tracker collects Eyeball fixation point, and assume that it is distributed as gauss hybrid models, can generate a weights of importance collection of illustrative plates, thus observable person for The attention rate in each region in TV news.The weight collection of illustrative plates produced according to this, can distribute to each pixel in frame of video Different weights, thus improve existing Objective Quality Assessment method.More particularly it relates to an based on existing The perceived video quality assessment of the video conference coding of video quality evaluation method.

Claims (3)

1. towards an objective quality assessment method for video conference coding, including training and assessment two parts；

Training department divides and includes following step:

Step one: face and face area extract；

Face feature automatic Calibration algorithm is utilized to automatically extract out background, face, left eye, the right side in given video conference sequence Number of pixels in eye, mouth, nasal area；

Step 2: carry out eye tracker experiment, obtains the key point coordinate position for each two field picture when tester watches video, Obtain the concerned degree of single pixel；

If the concerned degree in single region is eyes key point number/this area pixel number efp/p, the most single region is Left eye, right eye, mouth, nose, other regions of face or background, then:

$\left\{\begin{array}{c}{c}_r=f_r/p_r\\ c_l=f_l/p_l\\ c_m=f_m/p_m\\ c_n=f_n/p_n\\ c_o=f_o/p_o\\ c_b=f_b/p_b\end{array}\right.$

Wherein: c_r、c_l、c_m、c_n、c_o、c_bRepresent respectively right eye, left eye, mouth, nose, other regions of face, background area single The degree of concern of pixel, f_r、f_l、f_m、f_n、f_o、f_bBe illustrated respectively in eye tracker experiment in, tester fall right eye, left eye, Mouth, nose, other regions of face, the eyes key point number of background area, p_r、p_l、p_m、p_n、p_o、p_bRepresent respectively right eye, left eye, Pixel number in mouth, nose, other regions of face, background area；

Step 3: face area is calibrated and normalizes；

Method particularly includes:

Randomly select a two field picture, use leftmost side point in image face area key point, as calibration original point B, obtain it Point A in the leftmost side in face area key point in his image, obtains coordinate transformation relation between A, B, by key point in other images Change according to coordinate transformation relation, complete calibration；

Randomly selecting a two field picture, in employing image, the abscissa length of personage's right eye is as normalization unit, by other images In key point be normalized according to normalization unit；

Step 4: obtain gauss hybrid models；

Assume that eye gaze point obeys gauss hybrid models, on the basis of normalization with calibration eye tracker data, pass through Gauss The linear superposition that mixed model is write as Gaussian component is as follows:

Wherein:Represent a Gaussian component, π_k,μ_kAnd Σ_kIt is the mixed coefficint of kth Gaussian component, average and variance, And x^*Represent the eye gaze point after two-dimensional calibration and normalization；K represents the quantity of the Gaussian component of GMM；

Above-mentioned steps 1-4 in off-line case, for one group of training video, it is thus achieved that be used for assessing video conferencing system objective quality Gauss hybrid models；

Evaluation part includes following step:

Step one: for one group of video, repetition training part steps one, automatically extract out background, face, left eye, right eye, mouth, Number of pixels in nasal area；

Step 2: the step 3 of repetition training process, calibrates face area and normalizes；

Step 3: on the basis of the training stage obtains gauss hybrid models, calculate right eye, left eye, mouth, nose, other districts of face Gaussian Profile weight around territory, the weight of background area and above each region, obtains weight collection of illustrative plates；

Step 4: on the basis of weight collection of illustrative plates, calculates Y-PSNR based on gauss hybrid models, assesses video conferencing system Picture quality after coding.

A kind of objective quality assessment method towards video conference coding the most according to claim 1, described training department Point step one particularly as follows:

The first, the face area obtained in video conference sequence in each two field picture by face feature automatic Calibration algorithm is crucial Point；

The second, utilize mean value drift technology on the face area extracted, left eye in Local Search face area image, Right eye, mouth, nasal area key point, and these key points are mated with the key point distributed model in database, it is achieved Left eye, right eye, mouth, nasal area key point optimize；

3rd, the face in each two field picture after being optimized, left eye, right eye, mouth, nasal area key point；

4th, respectively face, left eye, right eye, mouth, the key point of nasal area are connected, obtain face, left eye, right eye, mouth, Nose profile；

5th, obtain number of pixels in face, left eye, right eye, mouth, nasal area respectively, image pixel number is deducted face Number of pixels, obtains background pixel number, finally realizes automatically extracting of face's key area.

A kind of objective quality assessment method towards video conference coding the most according to claim 1, described training department In the step 4 divided, if K=3, correspond respectively to right eye, left eye and mouth；If μ_kNormalization barycenter for each face feature.