CN104065962B - The macroblock layer bit distribution optimization method that view-based access control model notes - Google Patents

Abstract

The invention provides the macroblock layer bit distribution optimization method that view-based access control model notes, which comprises the following steps：Step 1, judge that macro block belongs to foreground area or background area, detect moving region；Step 2, structural texture region is detected using gradient；Step 3, macroblock layer target bit allocation.On the basis of the present invention analyzes HVS vision attention characteristic, first by reference to the motion vector of macro block in frame, in conjunction with frame differential method and the position of current macro, extract the moving region for causing vision attention in image, then texture region is detected using average gradient, according to macro block region in image, optimize the target bit allocation of macro block.Set forth herein algorithm achieve the quality of coded picture suitable with JVT G012 algorithm, reduce the fluctuation of video sequence PSNR, make coded image integrally maintain stable objective quality, while also obtaining preferable coded image subjective quality.

Description

The macroblock layer bit distribution optimization method that view-based access control model notes

Technical field

The invention belongs to technical field of video processing, there is provided the macroblock layer bit distribution optimization side that view-based access control model notes Method.

Background technology

In H.264/AVC, the video matter after compression is generally evaluated with objective standard (such as Y-PSNR, PSNR) Amount, objective quality assessment method have the characteristics that speed is fast, easily carry out, but its result often less meets the subjective vision of human eye Impression, can only react video quality condition generally.Psychology of vision research shows that human visual system (HVS) is to image Affected by many factors such as brightness, contrast, color, texture, motion and positions with the subjective perception of video, and there is sky Between, the shielding effect of time and colour.In a complicated visual scene, the notice of human eye is rapidly by interested in scene Region attracted, and these information of priority treatment, this process are referred to as vision attention.Research table to vision noticing mechanism Bright, when video image is watched, visual attention is frequently not to be evenly distributed in entire image to human eye, but concentrates wherein Some parts, the image fault to this part is also more sensitive.Therefore in video coding process, can regarding in conjunction with HVS Feel attention mechanism, the more target bits of the region distribution concentrated by human eye notice, to obtain more preferable video quality, and right Human eye processing priority is relatively low, it is not easy to which less target bits are distributed in the region for arousing attention, and improves video coding efficiency.

Some video coding techniques based on HVS characteristic are had at present, and these coding methods are using psychology of vision and life The achievement that field of science obtains, by optimizing the target bits for attracting macro block in human eye notice distributed areas, improves coding Efficiency simultaneously achieves preferable subjective visual quality.Document H.Li, Z.B.Wang, H.J.Cui, K.Tang, An improved ROI-based rate controlalgorithm for H.264/AVC,IEEE ICSP 2(2006)16–20.；Y.Liu, Z.G.Li,Y.C.Soh,M.H.Loke,Conversational video communication ofH.264/AVC with Region of interest concern.IEEE ICIP (2006) 3,129 3132. is based on area-of-interest (region of Interest, ROI) Video coding is carried out, by carrying out detect and track to the colour of skin of object, face, hand in video image To extract ROI, as the ROI in these methods carries out extracting obtaining to specific video content, its application scenarios is subject to Limit.Document Chen ZZ, Han JW, Ngan KN (2006) Dynamic bit allocation for multiple video object coding.IEEETrans Multimed 8(6):1117–1124.Yang L,Zhang L,Ma S, Zhao D(2009)A ROI quality adjustable rate control scheme for low bitratevideo Coding.Picture Coding Symposium, Chicago, USA, May.06-08. are according to different right in video sequence As distributing target bits, method of this object-based coding method ratio based on ROI is more flexible, but is intended to efficiently differentiate And the different objects in video sequence are extracted, very high computation complexity is needed, is not suitable for Video Coding transmission.This chapter leads to The principal element of analyzing influence vision attention is crossed, the method for proposing quick detection vision attention region, by optimizing in not same district The target bit allocation of macro block in domain, improves the subjective and objective quality of coded image.

Content of the invention

It is an object of the invention to provide the macroblock layer bit that view-based access control model notes in a kind of H.264/AVC Video coding divides Join optimization method

The present invention is employed the following technical solutions to achieve these goals：

The macroblock layer bit distribution optimization method that view-based access control model notes, it is characterised in that comprise the following steps：

Step 1, judge that macro block belongs to foreground area or background area, detect moving region；

Step 2, structural texture region is detected using gradient；

Step 3, macroblock layer target bit allocation.

In technique scheme, step 1 is specially：

Will in one frame of video sequence include the N number of macro block of M, in t frame position coordinates for (i, j) macro block be expressed as MB (i, J), wherein i=0 ... M-1, j=0 ... N-1；The background motion caused by video camera action has very strong time continuity, because In this background area, in the motion of macro block and former frame, same position macro block is close to, and in consecutive frame in foreground area macro block fortune Dynamic significantly different.When the motion of macro block notices that intensity MAI exceedes vision attention intensity threshold THm, the macro block is just considered category In foreground area, otherwise then belong to background area；

I.e. moving region can be obtained by formula detection：

TH_mFor vision attention intensity threshold：

The motion of macro block notes intensity：

PI in above formula_t(i, j) is the position intensity of macro block,

In formula, (x_mc,y_mc) for current macro center pixel coordinate, (x_pc,y_pc) for present frame center pixel coordinate, Max is the ultimate range of image border to frame center；

MI_t(i, j) is the exercise intensity of macro block：

In formula, MVx_t-1(i,j),MVy_t-1(i, j) is respectively the level of the motion vector of (i, j) individual macro block in former frame And vertical component；

MI_aveMean motion intensity for macro block：

MSE_t(i, j) is the brightness mean square error of macro block：

In formula, I_t(m, n) and I_t-1(m, n) is respectively in present frame and former frame in same position macro block, coordinate for (m, N) brightness value of pixel；

MSE_aveBrightness mean difference for macro block：

In technique scheme, step 2 is specially：When macro block average level vertical gradient value Grad (i, j) is more than threshold value TH_rWhen, then using the macro block as structural texture region macro block, texture region obtained by formula：

Macro block average level vertical gradient is expressed as：

In formula, (i, j) is the position coordinates of macro block, I_(m,n)Represent that coordinate is the pixel brightness value of (m, n) in macro block；

In technique scheme, macroblock layer target bit allocation is improved described in step 3 is as follows：

In formula, T_adjust(i, j) is the target bit after current macro adjustment, N_mbFor the quantity of macro block in present frame, w_m For the vision attention weight factor of macro block,

The beneficial effect that the present invention possesses：

On the basis of the present invention has analysed HVS vision attention characteristic, first by reference to the motion vector of macro block in frame, in conjunction with Frame differential method and the position of current macro, extract the moving region for causing vision attention in image, then adopt average gradient Detection texture region, according to macro block region in image, optimizes the target bit allocation of macro block.Test result indicate that, herein The algorithm of proposition achieves the quality of coded picture suitable with JVT-G012 algorithm, reduces the fluctuation of video sequence PSNR, makes Coded image integrally maintains stable objective quality, while also obtaining preferable coded image subjective quality.

Description of the drawings

Fig. 1 is sequence Silent and Coastguard primitive frame；

The Acquiring motion area result of Fig. 2 frame differential method；

Moving region testing result of the Fig. 3 based on motion vector；

Fig. 4 moving region detection algorithm result；

Fig. 5 texture region testing result；

The subjective quality of the 183rd frame coded image of Fig. 6 Carphone compares.

Specific embodiment：

The present invention is described further below：

HVS vision attention characteristic：

Vision attention is a kind of unconscious process of HVS, while being also mankind's process of cognition most specifically, it is subject to two classes Factor affects：The factor of top-down (Concept-driven test) and the factor of bottom-up (stimulate and drive).The former is complicated from the mankind Mental process, affected by factors such as personal knowledge, hobbies, the pattern-recognition of such as knowledge based and experience, it makes one Eye directly pays close attention to the feature of some objects in scene.The latter refers to related to opthalmic optics' attribute and retina in video scene Factor, such as color, contrast, spatial concealment, temporal masking and object motion etc..Research in terms of vision attention, main at present If representing vision content by setting up bottom-up attention force analysis model, however analyze every kind of impact vision attention because The computation complexity of element is very high, is not appropriate for Video Coding application.Many experiments show that the object in video scene is transported Dynamic and texture region is the most important factor for affecting visual attention distribution, and therefore we mainly consider that these two aspects factor is excellent Change target bit allocation.

Moving region is detected：

One key property of video sequence is the presence of different types of motion in video scene, and such as camera translation draws The movement of background is played, object is with the motion of different tracks or the motion of simple randomization.In these scenes, beholder can note first Anticipating, those quickly move or the uncertain objects of movement locus, and this type games is referred to as " motion of attention ", otherwise is then referred to as " no The motion for noting ", the object of motion have attracted the most notice of beholder.Motion in video scene is divided into foreground moving With two kinds of background motion, background motion is mainly caused by the motion of video camera, such as scaling, translation and spinning movement, foreground moving Refer to the motion of object in video.Background motion is compared, foreground moving can more attract the attention of human eye, the vision attention of people is mainly received To the impact of foreground moving, the therefore extraction of moving region mainly considers the extraction to foreground moving region.

If in video scene, moving region and stagnant zone have the objective distortion of identical (MSE), but visually perceive The distortion of moving region can be some larger, is that human eye is more when the object for occurring moving in video the reason for cause this phenomenon How to focus on the object and have ignored static object, that is to say, that human eye is easier to discover the distortion of Moving Objects, therefore, It is necessary for moving region and distributes more target bits, obtains preferably subjective and objective coded video quality.

The algorithm of Acquiring motion area is broadly divided into two kinds at present：Frame differential method and the method based on motion vector. , using the luminance difference detection moving region of same position macro block in two continuous frames, this method is in static background for frame differential method Scene in can effectively detect moving region, in such as Fig. 2 (a), but in the scene of movement background, its Detection results is not Good, in such as Fig. 2 (b).Method based on motion vector detects moving region using the motion vector information of macro block, as shown in figure 3, This method can only detect region of the exercise intensity more than a certain threshold value, and the macro block of moving region is discontinuous, when to fortune During the target bits different with non-moving areas distribution of dynamic region, rough visual experience is will result in.Borrowed more than signing herein The exercise intensity of two methods computing macro block, judges that macro block belongs to foreground area or background area, while according in scene Middle region can more attract the characteristic of human eye this HVS of notice than other regions, propose motion region detection method.

Assume in one frame of video sequence, to include M × N number of macro block, in t frame, position coordinates is expressed as MB for the macro block of (i, j) (i, j), wherein i=0 ... M-1, j=0 ... N-1.

The exercise intensity of macro block is expressed as：

In formula, MVx_t-1(i,j),MVy_t-1(i, j) is respectively the level of the motion vector of (i, j) individual macro block in former frame And vertical component.

Mean motion intensity is：

The brightness mean square error (MSE) of macro block：

In formula, I_t(m, n) and I_t-1(m, n) is respectively in present frame and former frame in same position macro block, coordinate for (m, N) brightness value of pixel.

Mean difference is：

The position intensity of macro block is：

In formula, (x_mc,y_mc) for current macro center pixel coordinate, (x_pc,y_pc) for present frame center pixel coordinate, Max is the ultimate range of image border to center.

The motion of macro block notices that intensity is：

The background motion caused by video camera action have very strong time continuity, therefore in background area macro block fortune Dynamic close with same position macro block in former frame, and in foreground area, the motion of macro block is significantly different in consecutive frame.When macro block MAI exceedes threshold value TH_mWhen, the macro block is just considered to belong to foreground area, otherwise then belongs to background area.

Vision attention intensity threshold is defined as：

Moving region can be obtained by formula detection：

Moving region detection algorithm is verified by standard test sequences, experimental result is as shown in figure 4, by combining The change of motion vector, interframe luminance difference value and macro block position, the algorithm of this paper in the scene of static background and movement background all Moving region can preferably be extracted.

Texture region is detected

In video image, the Moving Objects of vision attention are caused only to account for the sub-fraction ratio of image, in background area Human eye preferentially notes texture region.Characteristic is selected according to vision, texture region can be divided into：Smooth region, single direction edge group The structural texture region for becoming and the random grain region without consistent edge direction, as HVS has video shielding effect, at random The distortion of texture region is difficult to be aware, and the distortion in structural texture region is easily noted, and compares texture area in coding Domain, the bit number that smooth region is consumed are less.Therefore distribute more bit to structural texture region, to improve image vision matter Amount.Efficiency high is had based on the edge detection method of gradient, the low feature of computation complexity, detects structure using gradient herein Texture region.

Macro block average level vertical gradient is expressed as：

In formula, (i, j) is the position coordinates of macro block, I_(m,n)Represent that coordinate is the pixel brightness value of (m, n) in macro block.When grand The gradient magnitude of block is more than threshold value TH_rWhen, then using the macro block as structural texture region macro block.

Texture region is obtained by formula：

The extraction of texture region is verified by standard sequence, experimental result is as shown.

The improvement of macroblock layer target bit allocation

As described in this paper chapter 2, the rate control techniques of layering used in JVT-G012 motion H.264/AVC, i.e., GOP layer, frame-layer and BU layer.If elementary cell is a frame, BU layer is identical with frame layer rate control, if elementary cell is grand Block, BU layer bit rate control target be for each macroblock allocation target bits so that the bit number of total Image Coding is close to which Desired value T (i, j), computing formula is：

Wherein, f_rbFor remaining bits number, N_ubFor uncoded macroblock number.

, comprising head bit number and texture bits number, head bit is by encoding motion vector and predictive mode for the target bits of macro block Required bit composition.The macro block of motion intense needs more head bit to represent motion vector, and complicated grand of texture Block is larger because of its residual energy, it is also desirable to more target bits.From formula, the remaining target bits in picture frame are averaged Each macro block in frame is distributed to, as the complicated macro block of motion intense and texture cannot obtain more target bits, is caused In image, in vision attention region, the coding quality of macro block does not meet human visual experience.The quality of video image be finally by seeing The person of seeing also is considered as the vision attention characteristic of HVS evaluated in rate control algorithm, has coded image preferably subjective Visual quality.

Herein in conjunction with the vision attention characteristic of HVS, while consideration is between zones of different during transition, the flatness of vision is right Macroblock layer target bit allocation is improved as follows：

In formula, T_adjust(i, j) is the target bit after current macro adjustment, N_mbFor the quantity of macro block in present frame, w_m For the vision attention weight factor of macro block, can be adjusted according to subjective vision, in testing herein, value is：

Experimental result and analysis

By set forth herein innovatory algorithm write H.264/AVC in reference software JM10.1, and by result and JM10.1 code Rate control algolithm is compared.Two kinds of algorithms all use a macro block as a BU layer, and prospect and the back of the body in scene is chosen in experiment The variant QCIF format standard sequence of scape motion and Texture complication is tested, and wherein Akiyo and Salesman sequence is Static background foreground moving intensity is relatively low, and other sequences are arranged such as comprising foreground and background motion and complex texture, experiment parameter Table 1.

1 experiment parameter of table is arranged

Frame per second	30f/s
		Coding mode	IPPP
Sequence length	100
		Reference frame	5
Rate-distortion optimization	ON
		Motion search range	16×16
Search-type	Full search
		Entropy code	CABAC

In order to compare the performance of rate control algorithm, the encoder bit rate error of sequence is：

In formula, R_actualFor actual coding code check, R_targetFor target bit rate.

PSNR variances sigma of the sequence per frame is defined as：

In formula, n is coding number of frames,For sequence average PSNR.

2 rate control accuracy of table compares

3 coded image objective quality of table compares

Knowable to the experimental result of table 2, this paper algorithm has higher rate control accuracy, code check precision model than G012 algorithm It is trapped among in 0.05-0.54%.Can be seen that in table 3, although the overall PSNR gain of two kinds of algorithms quite, but compares G012 calculation Method, the PSNR variance of all sequences have all been reduced, and PSNR less, the quality of coded picture of sequence of fluctuation of video sequence is described More stable.

This paper algorithm also obtain preferable subjective quality, be illustrated in figure 6 the 183rd frame code pattern of sequence C arphone Picture, due to being assigned with more target bits to moving region and texture region in this paper algorithm, (a) in figure face is clearly demarcated Aobvious more clear than (b) in figure.

Claims (3)

1. the macroblock layer bit distribution optimization method that view-based access control model notes, it is characterised in that comprise the following steps：

Step 1, judge that macro block belongs to foreground area or background area, detect moving region；

Step 2, structural texture region is detected using gradient；

Step 3, macroblock layer target bit allocation；

Step 1 is specially：

MxN macro block will be included in one frame of video sequence, position coordinates is expressed as MB (i, j) for the macro block of (i, j) in t frame, Wherein i=0 ... M-1, j=0 ... N-1；When the motion of macro block notices that intensity MAI exceedes vision attention intensity threshold THm, this is grand Block is just considered to belong to foreground area, otherwise then belongs to background area；

I.e. moving region can be obtained by formula detection：

$MR(i,j)=\left\{\begin{array}{cc}1,& if{\mathrm{MAI}}_t(i,j)>{\mathrm{TH}}_m\\ 0,& otherwise\end{array}\right.$

TH_mFor vision attention intensity threshold：

${\mathrm{TH}}_m=\frac{1}{M\×N}\underset{i=0}{\overset{M-1}{\Σ}}\underset{j=0}{\overset{N-1}{\Σ}}{\mathrm{MAI}}_t(i,j)$

The motion of macro block notes intensity：

${\mathrm{MAI}}_t(i,j)=(1-{\mathrm{PI}}_t(i,j\left)\right)\×(\frac{{\mathrm{MI}}_t(i,j)}{{\mathrm{MI}}_{ave}}+\frac{{\mathrm{MSE}}_t(i,j)}{{\mathrm{MSE}}_{ave}})$

PI in above formula_t(i, j) is the position intensity of macro block,

${\mathrm{PI}}_t(i,j)=\frac{\sqrt{{(x_{mc}-x_{pc})}^2+{(y_{mc}-y_{pc})}^2}}{\max }$

In formula, (x_mc,y_mc) for current macro center pixel coordinate, (x_pc,y_pc) for present frame center pixel coordinate, max is Image border is to the ultimate range of frame center；

MI_t(i, j) is the exercise intensity of macro block：

${\mathrm{MI}}_t(i,j)=\sqrt{{\mathrm{MVx}}_{t-1}{(i,j)}^2+{\mathrm{MVy}}_{t-1}{(i,j)}^2};$

In formula, MVx_t-1(i,j),MVy_t-1(i, j) be respectively former frame in (i, j) individual macro block motion vector level and hang down Straight component；

MI_aveMean motion intensity for macro block：

${\mathrm{MI}}_{ave}=\frac{1}{M\×N}\underset{i=0}{\overset{M-1}{\Σ}}\underset{j=0}{\overset{N-1}{\Σ}}{\mathrm{MI}}_t(i,j)$

MSE_t(i, j) is the brightness mean square error of macro block：

${\mathrm{MSE}}_t(i,j)=\frac{1}{16\×16}\underset{m=0}{\overset{15}{\Σ}}\underset{n=0}{\overset{15}{\Σ}}{\[I_t(m,n)-I_{t-1}(m,n)\]}^2$

In formula, I_t(m, n) and I_t-1(m, n) is respectively in present frame and former frame in same position macro block, and coordinate is (m, n) pixel Brightness value；

MSE_aveBrightness mean difference for macro block：

2. the macroblock layer bit distribution optimization method that view-based access control model according to claim 1 notes, it is characterised in that step 2 Specially：When macro block average level vertical gradient value Grad (i, j) is more than threshold value TH_rWhen, then using the macro block as structural texture The macro block in region, texture region are obtained by formula：

$TR(i,j)=\left\{\begin{array}{cc}1,& ifGrad(i,j)>{\mathrm{TH}}_r\\ 0,& otherwise\end{array}\right.$

Macro block average level vertical gradient is expressed as： In, (i, j) is the position coordinates of macro block, I_(m,n)Represent that coordinate is the pixel brightness value of (m, n) in macro block；

${\mathrm{TH}}_r=\frac{1}{M\×N}\underset{i=0}{\overset{M-1}{\Σ}}\underset{j=0}{\overset{N-1}{\Σ}}Grad(i,j).$

3. the macroblock layer bit distribution optimization method that view-based access control model according to claim 2 notes, it is characterised in that to grand Block layer target bit allocation is improved as follows：

$T_{adjust}(i,j)=T(i,j)\×\frac{w_c\×N_{mb}}{\underset{m=1}{\overset{N_{mb}}{\Σ}}{w}_m}$

In formula, T_adjust(i, j) is the target bit after current macro adjustment, N_mbFor the quantity of macro block in present frame, w_mFor grand The vision attention weight factor of block, T (i, j) are desired value；

$w_i=\left\{\begin{array}{ccc}1& if& MR(i,j)=1\\ 0.75,& if& MR(i,j)=0andTR(i,j)=1\\ 0.25,& if& MR(i,j)=0andTR(i,j)=0\end{array}\right..$