CN104539971B - It is a kind of to be applied to Bit distribution method between 2 viewpoints of viewpoint video coding - Google Patents

Abstract

It is a kind of to be applied to Bit distribution method between 2 viewpoints of viewpoint video coding, it is first determined the influence of coding distortion of the coding distortion of basic viewpoint to non-basic viewpoint, it is determined that the rate-distortion model of basic viewpoint and non-basic viewpoint；Then, the bit assignment problem between viewpoint is converted under the conditions of limited total overhead bit, minimizes the problem of the binary encoding distortion of basic viewpoint and non-basic viewpoint, and obtain optimal solution, i.e., bit allocation scheme between optimal viewpoint.The code efficiency of multi-view point video is the method increase so that under conditions of identical encoder bit rate, the multi-view point video quality that decoding end is rebuild is higher, the features such as with high-performance, highly versatile, in can be applied to the coding standard of multi-view point video.

Description

It is a kind of to be applied to Bit distribution method between 2 viewpoints of viewpoint video coding

Technical field

The present invention relates to Bit distribution method between a kind of viewpoint for improving multiple view video coding efficiency, belong to and regard Point video encoding mark technical field.

Background technology

Three-dimensional (3D) video, as following main video applications, refers to that user can by 3 D video display device To enjoy real 3 d video content.The correlation technique of 3 D video, such as, three-dimensional video acquisition, 3 d video encoding, The technologies such as the display of 3 D video are widely paid close attention to.In order to promote the standardization of 3 D video technology, fortune in 2002 Motion video expert group (Motion Picture Experts Group, MPEG) and Video Coding Experts Group (Video Coding Experts Group, VCEG) formulate based on multiple view video coding (Multi View Video H.264 Coding, MVC) standard (H.264/MVC).Current MPEG and VCEG is based on newest high-performance video coding standard (High Efficiency Video Coding, HEVC) formulates a new generation MVC standards, i.e. MV-HEVC.In multi-view point video In, user can from different angles watch the 3 D video of the angle so that user has the sense of reality incorporated in video scene Receive.Multi-view point video can be widely applied to the fields such as broadcast communication, amusement, education, medical treatment and video monitoring.It is more when needing During viewpoint, multi-view point video-depth representing 3D scenes can be used, and encoded using 3D-HEVC coding standards；User's end What end can be needed using View synthesis technology (Depth Image Based Rendering, the DIBR) synthesis based on depth Visual point image.Therefore, in existing MV-HEVC coding standards framework, 2 viewpoints are only defined and 3 videos of viewpoint is compiled Code structure.

Bit distribution method is the Bit distribution method of fixed proportion, fixed proportion bit point between existing multi-view point video The coding flow matched somebody with somebody is as shown in figure 1, it is different viewpoint distributing bits to use proportionality coefficient set in advance.For 2 viewpoints Video, the bits proportion between BV and NBV can be 2:8、4:6、6:4 etc.；For 3 non-bases of video BV and left side of viewpoint This viewpoint (NBVL), the bits proportion of the non-basic viewpoint (NBVR) in right side can be 4:3:3、6:2:2 etc..

The Bit distribution method of fixed proportion does not account for the influence of the coding distortion of BV to the coding distortion of NBV, only with Experience sets a fixed allocation proportion；Therefore do not possess versatility, the total quality of the multi-view point video of reconstruction is not also high. From theory analysis, the coding distortion that BV has considerable influence, BV to NBV can be propagate directly in NBV, therefore in multiple views Between bit allocation procedures in, it is contemplated that influences of the BV to NBV.

The content of the invention

Do not possess versatility for the Bit distribution method of fixed proportion, rebuild video total quality shortcoming not high, this Influence of the invention according to basic viewpoint (Base View, BV) to non-basic viewpoint (Non-Base View, NBV), proposes a kind of It is applied to Bit distribution method between 2 viewpoints of viewpoint video coding.Bit point between the viewpoint of the method concern multi-view point video Match somebody with somebody, in determining multi-view point video, the influence of the distortion of basic viewpoint to non-basic viewpoint designs optimal basic viewpoint and Fei Ji Bit distribution method between this viewpoint, improves the code efficiency of multi-view point video so that under conditions of identical encoder bit rate, The multi-view point video quality that decoding end is rebuild is higher.

It is of the invention to be applied to Bit distribution method between 2 viewpoints of viewpoint video coding, it is first determined basic viewpoint Influence of the coding distortion to the coding distortion of non-basic viewpoint, it is determined that the rate-distortion model of basic viewpoint and non-basic viewpoint；So Afterwards, the bit assignment problem between viewpoint is converted under the conditions of limited total overhead bit, minimizes basic viewpoint and non-basic The problem of the binary encoding distortion of viewpoint, and obtain optimal solution (bit allocation scheme between i.e. optimal viewpoint)；Specifically include with Lower step：

(1) Mathematical Modeling between the coding distortion and number of coded bits of the basic viewpoint of definition is：

D_bv=(a_bv·R_bv+b_bv)^-1,

Wherein, D_bvRepresent the coding distortion of basic viewpoint, R_bvRepresent the number of coded bits of basic viewpoint, a_bvAnd b_bvIt is mould Shape parameter, represents R_bvTo D_bvInfluence degree；The coding distortion D of non-basic viewpoint_nbvWith the number of coded bits of non-basic viewpoint R_nbvWith the number of coded bits R of basic viewpoint_bvBetween relation be：

D_nbv=η (a_bvR_bv+b_bv)^-1+(a_nbvR_nbv+b_nbv)^-1,

Wherein, η represents the influence degree of the coding distortion of the basic viewpoint of present frame to the coding distortion of NBV, a_nbvAnd b_nbv It is model parameter, represents R_nbvTo D_nbvInfluence degree；

η described in step (1) is calculated by following formula：

Wherein, Num_{PU_InterViewSKIPMode}In the non-basic viewpoint of expression present frame, using dancing mode between viewpoint The quantity of the predicting unit (Prediction Unit, PU) of (Inter View Skip Mode) coding, Num_PURepresent present frame Non- basic viewpoint in, the quantity of all predicting units.

(2) the preceding W frames for multi-view video sequences most being started, using rate control algorithm determine per frame basic viewpoint and The total bit number of non-basic viewpoint, and be different viewpoint distributing bits with ratio set in advance and encode, after coding The coding distortion and number of coded bits of basic viewpoint are respectively stored in array vectorWithIn, the coding distortion and number of coded bits of non-basic viewpoint are respectively stored in array vectorWithIn；

Ratio (initial bit allocation proportion) set in advance is in step (2)：

(3) according to the distortion of the basic viewpoint in the W frames before present frameWith bit numberAnd D in step (1)_bvAnd R_bvBetween Mathematical Modeling, equation is solved by linear regression Group：

Calculate the model parameter a of basic viewpoint_bvAnd b_bv；

(4) according to the distortion of the non-basic viewpoint in the W frames before present frameWith bit NumberThe bit number of BVAnd D in step (1)_nbvWith R_nbvWith R_bvBetween Mathematical Modeling, equation group is solved by linear regression：

Wherein η₀…η_W-1In the encoded W frames of expression, in 2 viewpoints per frame, the distortion of basic viewpoint is regarded substantially to non- The influence degree of the distortion of point, calculates the model parameter a of non-basic viewpoint_nbvAnd b_nbv；

(5) according to a obtained by step (3) and step (4)_bv、b_bv、a_nbvAnd b_nbv, and it is current using rate control algorithm The total bit number R of frame_c, by following formula：

It is the basic viewpoint and non-basic viewpoint istributes bit number of present frame, that is, calculates the R of present frame_bvAnd R_nbv；

(6) according to the principle of " first in first out ", by array vectorThe 0th data delete Remove, and the coding distortion and number of coded bits of present frame are added to the end of array vector, update array vector

(7) return to step (3), encode to next frame.

The main code efficiency for improving multi-view point video of the invention, determines the rate-distortion model of multi-view point video, and work as Influence of the basic viewpoint of previous frame (comprising multiple visual point images) to non-basic viewpoint；Then calculated using rate control techniques and worked as The usable total bit number of previous frame, and use optimal method, is the basic viewpoint and non-basic viewpoint distributing bit of every frame, and then Improve the code efficiency of multi-view point video.There is advantages below compared with prior art：

1) influence of the coding distortion according to basic viewpoint of the invention to the coding distortion of non-basic viewpoint, devises rate mistake Bit distribution method between the multiple views of true best performance, improves the code efficiency of multi-view point video；

2) the bit allocation proportion the present invention can adjust viewpoint to any video, with very strong adaptivity.

Brief description of the drawings

Fig. 1 is the coding flow chart of fixed proportion bit distribution.

Fig. 2 is the flow chart for being applied to Bit distribution method when 2 viewpoints are encoded of the invention.

Fig. 3 is the method that the method for the present invention and the distribution of fixed proportion bit is respectively adopted to three-dimensional video sequence The experimental result (rate distortion curve) of 2 viewpoints coding of Balloons compares figure.

Fig. 4 is the method that the method for the present invention and the distribution of fixed proportion bit is respectively adopted to three-dimensional video sequence The experimental result (rate distortion curve) of 2 viewpoints coding of BookArrival compares figure.

Specific embodiment

As Fig. 2, the present invention are applied to Bit distribution method between 2 viewpoints of viewpoint video coding, detailed process is as follows：

(1) Mathematical Modeling between the coding distortion and number of coded bits of the basic viewpoint (Base View, BV) of definition is

D_bv=(a_bv·R_bv+b_bv)^-1,

Wherein, D_bvRepresent the coding distortion of BV, R_bvRepresent the number of coded bits of BV, a_bvAnd b_bvIt is model parameter, represents R_bvTo D_bvInfluence degree；Coding distortion (the D of NBV_nbv) with the number of coded bits (R of NBV_nbv) and BV number of coded bits (R_bv) between relation be：

D_nbv=η (a_bvR_bv+b_bv)^-1+(a_nbvR_nbv+b_nbv)^-1,

Wherein, η represents the influence degree of the coding distortion of present frame BV to the coding distortion of NBV, a_nbvAnd b_nbvIt is model Parameter, represents R_nbvTo D_nbvInfluence degree；

Described η can be calculated by following formula

Wherein, Num_{PU_InterViewSKIPMode}In the NBV of expression present frame, using (the Inter View of dancing mode between viewpoint Skip Mode) coding predicting unit (Prediction Unit, PU) quantity, Num_PURepresent in the NBV of present frame, own The quantity of predicting unit.

(2) the preceding W frames for most starting to multi-view video sequences, determine the BV's and NBV per frame using rate control algorithm Total bit number, and be different viewpoint distributing bits with ratio set in advance and encode, by the coding distortion of the BV after coding Array vector is respectively stored in number of coded bitsWithIn, the volume of NBV Code distortion and number of coded bits are respectively stored in array vectorWith In；

The ratio set in advance (initial bit allocation proportion) is

(3) according to the distortion of the BV in the W frames before present frameWith bit numberAnd D in step (1)_bvAnd R_bvBetween Mathematical Modeling, equation group is solved by linear regression：

Calculate the model parameter a of BV_bvAnd b_bv；

(4) according to the distortion of the NBV in the W frames before present frameWith bit numberThe bit number of BVAnd D in step (1)_nbvWith R_nbvWith R_bvBetween Mathematical Modeling, equation group is solved by linear regression：

Wherein η₀…η_W-1In representing encoded W frames, in 2 viewpoints per frame, the shadow of the distortion of BV to the distortion of NBV The degree of sound, calculates the model parameter a of NBV_nbvAnd b_nbv；

(5) according to a obtained by step (3) and step (4)_bv、b_bv、a_nbv、b_nbv, and it is current using rate control algorithm The total bit number R of frame_c, by following formula：

It is BV the and NBV istributes bit numbers of present frame, that is, calculates the R of present frame_bvAnd R_nbv；

(6) according to the principle of " first in first out ", by array vectorThe 0th data delete Remove, and the coding distortion and number of coded bits of present frame are added to the end of array vector, update array vector

(7) return to step (3), encode to next frame.

Effect of the invention can be further illustrated by experiment.

Experiment test under the different bandwidth, encoder bit rate after being encoded to multi-view point video using the present invention and Rebuild the overall objective quality (Y-PSNR, PSNR) of multi-view point video.Fig. 3 and Fig. 4 are compared using the present invention and fixed The rate distortion curve of the Bit distribution method of ratio.Wherein Fig. 3 is the 2 viewpoints coding to three-dimensional video sequence Balloons Experimental result, Fig. 4 is the experimental result to 2 viewpoints coding of three-dimensional video sequence BookArrival.Can by Fig. 3 and Fig. 4 See, compared with the Bit distribution method of fixed proportion, after being encoded using the present invention, under identical encoder bit rate, rebuild many The total quality of viewpoint video is higher, illustrates that the present invention improves the coding efficiency of multi-view point video.

Claims (3)

1. it is a kind of to be applied to Bit distribution method between 2 viewpoints of viewpoint video coding, it is characterized in that, comprise the following steps：

(1) Mathematical Modeling between the coding distortion and number of coded bits of the basic viewpoint of definition is：

D_bv=(a_bv·R_bv+b_bv)^-1,

Wherein, D_bvRepresent the coding distortion of basic viewpoint, R_bvRepresent the number of coded bits of basic viewpoint, a_bvAnd b_bvFor model is joined Number, represents R_bvTo D_bvInfluence degree；The coding distortion D of non-basic viewpoint_nbvWith the number of coded bits R of non-basic viewpoint_nbvWith The number of coded bits R of basic viewpoint_bvBetween relation be：

D_nbv=η (a_bvR_bv+b_bv)^-1+(a_nbvR_nbv+b_nbv)^-1,

Wherein, η represents the influence degree of the coding distortion of the basic viewpoint of present frame to the coding distortion of NBV, a_nbvAnd b_nbvIt is mould Shape parameter, represents R_nbvTo D_nbvInfluence degree；

(2) the preceding W frames for most starting to multi-view video sequences, the basic viewpoint and Fei Ji per frame are determined using rate control algorithm The total bit number of this viewpoint, and be different viewpoint distributing bits with ratio set in advance and encode, by coding after it is basic The coding distortion and number of coded bits of viewpoint are respectively stored in array vectorWithIn, the coding distortion and number of coded bits of non-basic viewpoint are respectively stored in array vectorWithIn；

(3) according to the distortion of the basic viewpoint in the W frames before present frameWith bit numberAnd D in step (1)_bvAnd R_bvBetween Mathematical Modeling, equation group is solved by linear regression：

$\left[\begin{array}{c}{D}_{bv,0}^{-1}\\ .\\ .\\ .\\ .\\ .\\ .\\ D_{bv,W-1}^{-1}\end{array}\right]=a_{bv}\left[\begin{array}{c}{R}_{bv,0}\\ .\\ .\\ .\\ .\\ .\\ .\\ R_{bv,W-1}\end{array}\right]+b_{bv},$

Calculate the model parameter a of basic viewpoint_bvAnd b_bv；

(4) according to the distortion of the non-basic viewpoint in the W frames before present frameWith bit numberThe bit number of BVAnd D in step (1)_nbvWith R_nbvAnd R_bv Between Mathematical Modeling, equation group is solved by linear regression：

$\left[\begin{array}{c}{(D_{nbv,0}-{\mathrm{\η}}_0{D}_{bv,0})}^{-1}\\ .\\ .\\ .\\ .\\ .\\ .\\ {(D_{nbv,W-1}-{\mathrm{\η}}_{W-1}{D}_{bv,W-1})}^{-1}\end{array}\right]=a_{nbv}\left[\begin{array}{c}{R}_{nbv,0}\\ .\\ .\\ .\\ .\\ .\\ .\\ R_{nbv,W-1}\end{array}\right]+b_{nbv},$

Wherein η₀…η_W-1In the encoded W frames of expression, in 2 viewpoints per frame, the distortion of basic viewpoint is to non-basic viewpoint The influence degree of distortion, calculates the model parameter a of non-basic viewpoint_nbvAnd b_nbv；

(5) according to a obtained by step (3) and step (4)_bv、b_bv、a_nbvAnd b_nbv, and using working as that rate control algorithm determines The total bit number R of previous frame_c, by following formula：

$R_{bv}=\frac{R_c+\frac{b_{nbv}}{a_{nbv}}-\frac{b_{bv}}{\sqrt{a_{bv}{a}_{nbv}(1+\mathrm{\η})}}}{1+\sqrt{\frac{a_{bv}}{a_{nbv}(1+\mathrm{\η})}}},$

$R_{nbv}=\sqrt{\frac{a_{bv}}{a_{nbv}(1+\mathrm{\η})}}{R}_{bv}+\frac{b_{bv}}{\sqrt{a_{nbv}{a}_{bv}(1+\mathrm{\η})}}-\frac{b_{nbv}}{a_{nbv}},$

It is the basic viewpoint and non-basic viewpoint istributes bit number of present frame, that is, calculates the R of present frame_bvAnd R_nbv；

(6) according to the principle of " first in first out ", by array vectorThe 0th data delete, and will The coding distortion and number of coded bits of present frame are added to the end of array vector, update array vector

(7) return to step (3), encode to next frame.

2. it is according to claim 1 to be applied to Bit distribution method between 2 viewpoints of viewpoint video coding, it is characterized in that, η described in the step (1) is calculated by following formula：

$\mathrm{\η}=\frac{{\mathrm{Num}}_{PU\_InterViewSKIPMode}}{{\mathrm{Num}}_{PU}},$

Wherein, Num_{PU_InterViewSKIPMode}Represent present frame non-basic viewpoint in, using between viewpoint dancing mode encode it is pre- Survey the quantity of unit, Num_PUIn the non-basic viewpoint of expression present frame, the quantity of all predicting units.

3. it is according to claim 1 to be applied to Bit distribution method between 2 viewpoints of viewpoint video coding, it is characterized in that, Ratio set in advance is in the step (2)：