CN112422975A - Light field integrated picture coding optimization method based on two-dimensional hierarchical coding structure - Google Patents

Light field integrated picture coding optimization method based on two-dimensional hierarchical coding structure Download PDF

Info

Publication number
CN112422975A
CN112422975A CN202011267205.8A CN202011267205A CN112422975A CN 112422975 A CN112422975 A CN 112422975A CN 202011267205 A CN202011267205 A CN 202011267205A CN 112422975 A CN112422975 A CN 112422975A
Authority
CN
China
Prior art keywords
coding
coding unit
light field
frame
current
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011267205.8A
Other languages
Chinese (zh)
Inventor
高艳博
李帅
朱策
刘宇洋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Electronic Science and Technology of China
Original Assignee
University of Electronic Science and Technology of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Electronic Science and Technology of China filed Critical University of Electronic Science and Technology of China
Priority to CN202011267205.8A priority Critical patent/CN112422975A/en
Publication of CN112422975A publication Critical patent/CN112422975A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/177—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a group of pictures [GOP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
    • H04N19/19—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding using optimisation based on Lagrange multipliers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/567—Motion estimation based on rate distortion criteria
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/577—Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The invention discloses a light field integrated picture coding optimization method based on a two-dimensional hierarchical coding structure, which comprises the steps of firstly obtaining a light field data set and preprocessing the light field data set to obtain a light field pseudo video sequence, then coding the light field pseudo video sequence, and confirming a coding unit directly influenced by a current coding unit according to a direct dependency relationship between viewpoints under the two-dimensional hierarchical coding structure; calculating a bidirectional motion compensation error based on an original frame, and estimating a code rate increment of a coding unit directly influenced under bidirectional prediction; and finally, estimating the coding cost function of the current coding unit and updating the Lagrange multiplier. The inter-view dependent rate-distortion optimization technology provided by the invention captures the inter-view dependent relationship of dynamic change in different light field integrated pictures, and fully utilizes the inter-view dependent relationship in the rate-distortion optimization technology, thereby improving the coding gain and improving the coding efficiency of the light field integrated pictures.

Description

Light field integrated picture coding optimization method based on two-dimensional hierarchical coding structure
Technical Field
The invention relates to the technical field of picture/video coding, in particular to a light field integrated picture coding optimization method based on a two-dimensional hierarchical coding structure.
Background
Independent rate distortion optimization technology is adopted in the traditional picture/video coding method, however, independent optimal coding is not equal to overall global optimal.
(1) Two-dimensional hierarchical coding structure for light field integrated pictures
As an advanced technology in the Video Coding standard HEVC (High Efficiency Video Coding), a hierarchical Coding structure can greatly increase Coding gain, the structure allocates Video frames to different Coding layers, and Coding frames located in the same layer share the same Quantization Parameter QP (Quantization Parameter) and reference frame management manner. The coded frame at the lower layer has a lower QP and a better coding quality, and can provide a high-quality reference frame for other frames, so that the coded frame at the highest layer has the highest QP and is more frequently arranged as the reference frame of other frames.
The light field integrated pictures can be reorganized into a two-dimensional picture matrix, as shown in fig. 1, where the boxes represent the extracted different view pictures, which are very similar to each other although they represent different views. To remove inter-view picture redundancy, the two-dimensional picture matrix may be organized into a pseudo-multi-view video sequence, so that encoding is performed by extending the layer coding structure in HEVC to a two-dimensional layer coding structure. Specifically, the numbers in the box of fig. 1 represent the coding Order EOC (coding Order Count) of each picture, and it is noted that the view pictures at the four corners of the picture matrix are blurred and are not considered in the coding process herein.
In the two-dimensional hierarchical coding structure, only the view picture located at the central point is coded as an I-frame, the I-frame provides a high-quality prediction content for the surrounding view pictures, and the other view pictures are distributed into four quadrants and are all coded as B-frames. In each quadrant, the 6 view Pictures in the same row (column) are a Group of Pictures GOP (Group of Pictures), and the Pictures are allocated to different coding levels, that is, the view Pictures represented by the boxes with the same pattern in fig. 1 are allocated to the same coding layer, for example, a diagonal block is allocated to the first layer, the view Pictures represented by other view Pictures (including diagonal blocks) in the same quadrant are selected as reference frames, a view picture represented by a dot block is allocated to the second layer, the view picture represented by the same row is selected as a reference frame, a view picture represented by a horizontal block is allocated to the third layer, only the view picture represented by an immediately adjacent blank block is selected as a reference frame, and finally, all blank blocks are allocated to the fourth layer and are not used as reference frames.
Fig. 1 is a schematic diagram of a two-dimensional hierarchical coding structure for a light field picture, where a box represents a light field view picture, a number in the box represents a coding order of the view, different shades in the box represent hierarchies of the view, a grid shade represents an I frame, a slashed block is a first layer, a dotted block is a second layer, a horizontal line block is a third layer, a blank block is a fourth layer, and a frame located at a higher layer may select a frame located at a same layer or a lower layer as a reference value.
(2) Time domain dependent rate distortion optimization at high code rates
Currently, in video coding, it is still assumed that coding units are independent, and an independent rate-distortion optimization technique is adopted to respectively obtain the optimal coding parameters of each coding unit, for example, the independent rate-distortion optimization technique used in the existing coding standards h.264 and HEVC: given code rate RtThe distortion D is minimized, i.e., min { D } s.t.R ≦ RtWherein R represents the code rate of a current coding unit, D represents the distortion of the current coding unit, and the method is converted into an unconstrained form min { J }, where J is the lagrangian coding cost and λ is the lagrangian multiplier, using the lagrangian multiplier method.
The global rate-distortion optimization problem can be expressed as a given overall code rate RTBy minimizing the distortion of all coding units, i.e. by
Figure BDA0002776437480000021
Wherein M is the number of all coding units and can be converted into a code division multiple access coding unit by using a Lagrange multiplier method
Figure BDA0002776437480000022
Wherein λgIs a global lagrange multiplier. Under the assumption of high code rate, the coding cost for the current coding unit i in the P frame can be represented as J from the viewpoint of global rate distortion optimizationi＝Di+λg·Ri+λg·ΔRi+1(Di) Wherein Δ Ri+1(Di) The code rate increment of the directly affected coding unit i +1 introduced for the coding distortion of the current coding unit i. According to distortion DiHas an influence on whether or not it is affected, Δ Ri+1(Di) Can be defined as Δ Ri+1(Di)＝Ri+1(Di)-Ri(0) Under P-frame prediction, the directly affected unit is located in the frame next to the frame in which the current coding unit is located. Under the condition of high code rate, the code rate model can be expressed as
Figure BDA0002776437480000023
Wherein, delta2Is the source variance. Under the condition of high code rate, combining Taylor expansion, the code rate increment and D of the coding unit i +1 can be obtainediOf linear relationship, i.e. Δ Ri+1(Di)＝β·DiWhere β is related to the video content. Therefore, the coding cost of the current coding unit i can be converted into Ji＝Di+λg/(1+β)·RiI.e. adjusting the Lagrangian multiplier lambda by betagAnd (6) optimizing. Note that in the hierarchical coding structure, the initial Lagrangian multiplier for the non-reference frame may be the global Lagrangian multiplier λg。
However, the existing two-dimensional hierarchical coding structure adopts a fixed quantization parameter setting mode and a reference frame management mode, ignores the dynamic change characteristic among different light field integrated picture viewpoints, and cannot fully utilize the dependency relationship among the viewpoints, so that the optimal coding result cannot be obtained.
The time domain rate distortion optimization technology under the high code rate is suitable for P frame coding optimization under a hierarchical coding structure, and considering that the dependency relationship changes when the hierarchical coding structure is expanded into a two-dimensional hierarchical coding structure, the coding optimization method aiming at the hierarchical coding structure is not suitable any more; and the two-dimensional hierarchical coding structure is B frames, and the P frame and B frame coding difference is considered to be larger, so that the P frame optimization method is not applicable any more.
Disclosure of Invention
In view of the above problems, an object of the present invention is to provide a method for optimizing light field integrated picture coding based on a two-dimensional hierarchical coding structure, which captures inter-view dependency of dynamic changes in different light field integrated pictures, and fully utilizes the inter-view dependency in a rate-distortion optimization technique, thereby improving coding gain and light field integrated picture coding efficiency. The technical scheme is as follows:
a light field integrated picture coding optimization method based on a two-dimensional hierarchical coding structure comprises the following steps:
step 1: acquiring and pre-processing a light field dataset
Selecting a plurality of light field pictures from an image library, preprocessing the light field pictures by a Matlab light field tool kit to obtain a picture matrix, discarding fuzzy pictures positioned at the edge in the picture matrix, and organizing a light field pseudo video sequence according to a coding sequence;
step 2: encoding light field pseudo video sequences
Modifying coding parameters of reference software HM of HEVC, adopting a two-dimensional hierarchical coding structure, coding a view picture positioned at a central point into an I frame, distributing other view pictures into four quadrants, and coding all the views into B frames; in each quadrant, 6 view pictures positioned in the same row/column are taken as a picture group, and all the pictures are distributed into different coding layers;
and step 3: updating B frame Lagrange multiplier
Step 3.1: according to the direct dependency relationship between the viewpoints under the two-dimensional hierarchical coding structure, the current coding unit U is confirmediDirect-influencing coding unit Un；
Step 3.2: calculating the motion compensation error based on the original frame in the forward prediction and backward prediction, and estimating the coding unit U directly affected under the bidirectional predictionnThe code rate increment of (2);
step 3.3: estimating a current coding unit UiAnd updating the Lagrange multiplier according to the coding cost function.
Further, the Lagrange multiplier of the I frame is updated to
Figure BDA0002776437480000031
Figure BDA0002776437480000032
Is the original lagrangian multiplier of the I frame in the HEVC reference software HM.
Further, the step 3.1 specifically includes: the coding units in the same coding layer select two coding units in a lower layer or the same layer as a reference unit to carry out bidirectional prediction, and according to the criterion, the direct dependency relationship between the views under the two-dimensional layer coding structure is obtained by combining the reference relationship between the pictures in the group and between the pictures; according to the direct dependency relationship between the viewpoints, the current coding unit U is confirmediFrames with a direct impact relationship; dividing the current frame into 16-16 coding units, using the basic processing unit size, and obtaining the current coding unit U from the frame with direct influence relation by using the motion search technologyiIs the matched coding unit of (1), namely the unit is subjected to UiCoding units that are directly affected.
Further, the step 3.2 specifically includes: under bi-directional prediction, the affected coding unit UnDirectly subject to a forward coding unit Ui(i.e., current coding unit) and backward coding unit UkThe motion compensation errors are respectively as follows:
forward motion compensation error:
Figure BDA0002776437480000041
backward motion compensation error:
Figure BDA0002776437480000042
wherein F represents forward prediction, B represents backward prediction,
Figure BDA0002776437480000043
and
Figure BDA0002776437480000044
motion compensation error based on original frame in forward prediction and backward prediction, respectively, α ═ 1; diAnd DkRespectively representing the current coding unit UiAnd a backward coding unit UkThe distortion of (2);
its bi-directional motion compensation error is expressed as:
Figure BDA0002776437480000045
where bi represents bi-prediction, ω ═ 0.3;
according to a code rate model
Figure BDA0002776437480000046
Obtaining a current coding unit UiOf the coding distortion introduced affected coding unit UnCode rate increment Δ Rn(Di)：
Figure BDA0002776437480000047
Wherein, delta2Is the variance of the information source; rn(Di) Represents a subject of UiOf a direct influencing coding unit UnSpecifically the code rate is subjected to the coding unit UiDistortion D ofiInfluence of Rn(0) Indicates when the coding unit U isnIs not subjected to UiWhen there is an influence of (1), UnThe code rate of (2); d represents the general term of the distortion of the coding unit, and D < DOMCP，DOMCPThe motion compensation error based on the original frame is commonly referred to;
in fact, the fact thatSince D < DOMCPAnd D iskAnd DiAnd if not, then:
Figure BDA0002776437480000048
to pair
Figure BDA0002776437480000051
Performing Taylor expansion with
Figure BDA0002776437480000052
Instead of the former
Figure BDA0002776437480000053
Obtaining:
Figure BDA0002776437480000054
wherein the content of the first and second substances,
Figure BDA0002776437480000055
by obtaining the average motion compensation error of the current coding unit based on the original frame.
Further, the step 3.3 specifically includes:
estimating all current coding units UiThe coding rate variation of the coding unit directly influenced is the current coding unit UiIs coded at a cost JiComprises the following steps:
Figure BDA0002776437480000056
wherein R isiFor the current coding unit UiN represents a total of N views of the video sequence, and N-i affected coding units, wherein only the rate variation of the directly affected coding units, i.e., μ, is consideredn1 represents that the coding unit is a directly affected coding unit, and otherwise 0; lambda [ alpha ]gIs a global Lagrangian multiplier; represents the baseThe motion compensation error from the original frame is from forward prediction or backward prediction, and can be F or B;
by converting, the current coding unit UiThe cost of (c) is written as:
Figure BDA0002776437480000057
i.e. by updating the lagrangian multiplier.
The invention has the beneficial effects that:
1) the inter-view dependent rate-distortion optimization technology provided by the invention captures the inter-view dependent relationship of dynamic change in different light field integrated pictures, and fully utilizes the inter-view dependent relationship in the rate-distortion optimization technology, thereby improving the coding gain and improving the coding efficiency of the light field integrated pictures.
2) The invention re-inspects the inter-view dependency relationship under the two-dimensional hierarchical coding structure of the light field integrated picture and provides a dependency rate distortion optimization method aiming at the B frame, thereby better improving the performance of the optimization algorithm.
Drawings
Fig. 1 is a schematic diagram of a two-dimensional hierarchical coding structure of a light field picture.
Fig. 2 shows a reference relationship under a two-dimensional hierarchical coding structure, in which an arrow points to a reference frame. (a) Reference relations within a GOP; (b) reference relationships between different GOPs.
Fig. 3 is an illustration of view impact relationship (e.g. view with EOC ═ 2), where the arrow points to the impacted view.
Detailed Description
The invention is described in further detail below with reference to the figures and specific embodiments. The invention takes a two-dimensional hierarchical coding structure as a starting point, excavates the dependency relationship between the view points of the light field integrated picture under the structure, provides a global rate distortion optimization method aiming at the light field integrated picture, and fully considers the dependency relationship between the view points, thereby greatly improving the coding efficiency of the light field integrated picture.
(1) Direct dependency relationship between viewpoints under two-dimensional hierarchical coding structure
In the hierarchical coding structure, the coding unit selects the coding unit located at a lower layer (or the same layer) as the reference unit. In the two-dimensional hierarchical coding structure for light field integrated pictures, the coding units in the same layer select two coding units in lower layers (or the same layer) as reference units for bidirectional prediction, so that the reference relationship in a GOP is shown in fig. 2 (a). On the other hand, the slant-line view has high coding quality and can be used as a reference frame of other uncoded slant-line views (i.e., the same-layer view), so the reference relationship between different GOPs is connected by slant-line views, as shown in fig. 2(b), wherein the midpoint view selects a lower-layer slant-line view as a reference frame. By combining the reference relations in and among the GOPs, the inter-view dependency relation can be obtained, and then the coding units affected by the coded units are confirmed, namely the coding effect influence propagation path is described.
In summary, the dependency relationship of a certain view can be obtained according to the reference relationship between the views in and between the GOPs. Here, inter-view dependency of the diagonal view with EOC of 2 is given as shown in fig. 3 (only inter-view direct dependency is given).
(2) Inter-view dependent rate distortion optimization cost function under bidirectional prediction
The light field integrated picture pseudo video sequence is coded by using a two-dimensional hierarchical coding structure in HEVC, and inter-frame bidirectional predictive coding is included in the coding process, so that the coding cost of a current coding unit i is represented as follows:
Figure BDA0002776437480000061
wherein R isiFor the current coding unit UiCode rate of (D)iFor the current coding unit UiN represents a total of N views of the video sequence, Δ Rn(Di) From the current coding unit U for inter-frame predictioniOf the coding distortion introduced affected coding unit UnThe code rate increment of (2) has N-i affected coding units, wherein only the code rate variation of the directly affected coding units is considered,i.e. mun1 represents that the coding unit is a directly affected coding unit, and otherwise is 0. Lambda [ alpha ]gIs a global lagrange multiplier.
(3) Code rate delta estimation under bi-directional prediction
Obtaining coded unit U under high code rate conditioniDirect-influencing coding unit UnCode rate variation Δ R ofn(Di)。
3.1, according to the dependency relationship between the viewpoints under the two-dimensional hierarchical coding structure (mainly based on bidirectional prediction), the motion search technology is utilized to confirm the direct influence on the coding unit UnForward coding unit U ofiAnd a backward coding unit UkAnd respectively calculating corresponding unidirectional motion compensation errors, i.e. forward motion compensation errors
Figure BDA0002776437480000071
And backward motion compensation error
Figure BDA0002776437480000072
Where F represents forward prediction, B represents backward prediction,
Figure BDA0002776437480000073
and
Figure BDA0002776437480000074
which is the motion compensation error based on the original frame in forward prediction and backward prediction, respectively, alpha is an empirical value. For bi-directional prediction, its bi-directional motion compensation error can be expressed as
Figure BDA0002776437480000075
Where bi represents bi-prediction and ω is an empirical value. DiAnd DkRespectively representing the current coding unit UiAnd a backward coding unit UkOf (3) is detected.
3.2 in predictive coding, coding Unit UnIs subject to distortion Di(Current coding Unit U)iDistortion of) and its own influence, i.e. Rn(Di)＝Rn(0)+ΔRn(Di). According to a code rate model
Figure BDA0002776437480000076
The variation yields:
Figure BDA0002776437480000077
wherein D < DOMCP；DkAnd DiIrrespective of whether or not
Figure BDA0002776437480000078
3.3, for Δ Rn(Di) Taylor expansion is carried out to obtain:
Figure BDA0002776437480000079
for simplicity of calculation, use is made here of
Figure BDA00027764374800000710
Instead of the former
Figure BDA00027764374800000711
Then:
Figure BDA00027764374800000712
3.4, according to J2iAnd Δ R as described in 3.3n(Di) Current coding unit UiThe coding cost of (a) is updated as:
Figure BDA0002776437480000081
wherein, the motion compensation error based on the original frame can be predicted from the forward direction or the backward direction.
(4) I-frame parameter adjustment
Pulling of I-framesGrenarian multiplier
Figure BDA0002776437480000082
Wherein
Figure BDA0002776437480000083
Is the original lagrangian multiplier in the HEVC reference software HM.
(5) Detailed embodiments:
the embodiment is implemented based on reference software HM16.0 of HEVC, and the development environment is Visual Studio 2008.
Step 1: acquiring and pre-processing a light field dataset
A library of light field images provided by EPFL (freely downloaded on the internet) collected by a light field camera Lytro-Illum with a light field picture size of 5368 × 7728 was used. And 20 light field pictures including 10 scenes such as buildings, grids, natural scenery, characters and the like are selected from the image library, and each scene is selected with 2 pictures. Each lightfield picture is preprocessed by Matlab lightfield toolkit (downloaded freely on the internet) to obtain 434 × 625 size, 225 sub-picture matrixes (15 × 15), YUV4:2:0, 8 bits. Finally, the middle 165 sub-pictures of the picture matrix are organized into a pseudo video sequence in coding order (as numbered in fig. 1), where the blurred pictures at the edges in the 15 x 15 picture matrix are discarded.
Step 2: coding parameters of reference software HM of HEVC are modified, and a light field pseudo video sequence is coded by using a two-dimensional hierarchical coding structure. The first frame is encoded as an I-frame with the Lagrange multiplier updated to 4
Figure BDA0002776437480000084
Figure BDA0002776437480000085
Is the original lagrangian multiplier of the I frame in the HEVC reference software HM.
And step 3: updating B frame Lagrange multiplier
3.1 according to the direct dependency relationship between the viewpoints under the two-dimensional hierarchical coding structure, confirming the current coding unit UiCoding units that are directly affected.
And confirming the frame having direct influence relation with the current coding unit according to the direct dependency relation between the viewpoints under the two-dimensional hierarchical coding structure. Dividing the current frame into 16-by-16 coding units, and obtaining the matched coding unit of the current coding unit, such as U, in the frame with direct influence relation by using motion search technologynAs a directly affected unit, the motion compensation error (OMCP error) based on the original frame is calculated and divided by the number of pixels of the coding unit, i.e. 256, and is recorded as the average motion compensation error of the current coding unit.
3.2 estimating the coding Unit U directly affected under bidirectional predictionnCode rate increment of
Under bi-directional prediction, the coding unit UnDirectly subject to a forward coding unit UiAnd a backward coding unit UkCan be written as forward motion compensation errors, respectively
Figure BDA0002776437480000091
And backward motion compensation error
Figure BDA0002776437480000092
Where F represents forward prediction, B represents backward prediction,
Figure BDA0002776437480000093
and
Figure BDA0002776437480000094
the motion compensation error based on the original frame in forward prediction and backward prediction, respectively, α is 1. Its bi-directional motion compensation error can be expressed as
Figure BDA0002776437480000095
Where bi represents bi-prediction, ω ═ 0.3.
According to a code rate model
Figure BDA0002776437480000096
Obtaining:
Figure BDA0002776437480000097
wherein D < DOMCP，DkAnd DiIrrespective of whether or not
Figure BDA0002776437480000098
To pair
Figure BDA0002776437480000099
Taylor expansion is carried out to obtain:
Figure BDA00027764374800000910
wherein the content of the first and second substances,
Figure BDA00027764374800000911
by obtaining the average motion compensation error of the current coding unit based on the original frame.
3.3 estimating the current coding Unit UiIs encoded in the first frame
According to 3.2, it is possible to estimate all the current coding units UiThe coding rate variation of the coding unit directly influenced is the current coding unit UiThe coding cost of (a) is:
Figure BDA00027764374800000912
wherein R isiFor the code rate of the current coding unit i, DiFor the distortion of the current coding unit i, N represents a total of N views of the video sequence, Δ Rn(Di) For the code rate increment of the affected coding unit N introduced by the coding distortion of the current coding unit i in the interframe prediction, N-i affected coding units are totally used, wherein only the code rate variable quantity, namely mu, of the directly affected coding unit is consideredn1 means that the coding unit is a directly affected coding unit, otherwise 0, λgAs a global Lagrangian multiplier, λgThe value is the original Lagrange multiplier initial set value of the non-reference frame in the HEVC reference software HM.
By conversion, the current coding unit UiThe cost of (c) can be written as:
Figure BDA0002776437480000101
i.e. by updating the lagrangian multiplier.
And carrying out weighted average on the updated Lagrangian multipliers of the 16 × 16 coding units in each coding unit (CTU) to obtain the updated Lagrangian multipliers of each CTU, wherein the size of the CTU is 64 × 64.
And respectively coding each CTU according to the updated Lagrange factor of each CTU.
The experimental parameters are similar to the Random-access standard test parameter settings in HEVC, except that the level assignment and coding settings of each coded frame are set according to a two-dimensional level coding structure, as shown in fig. 1. Specifically, the central viewpoint is an I-frame, for which a test is performed with QP values (22,27,32, 37). The other frames are B frames, a GOP with the length of 6 is organized, a slash shaded block represents that the viewpoint is a first layer viewpoint, a dot shaded block represents a second layer viewpoint, a horizontal line shaded block represents a third layer viewpoint, a blank block represents a fourth layer viewpoint (non-reference frame layer), the same layer frame follows the same reference frame selection strategy, namely, a coding unit positioned in the same coding layer selects two coding units of a lower layer or the same layer as a reference unit for bidirectional prediction, and a QP value of each layer frame is obtained by adding a QP value of an I frame and a QP offset value, wherein the offset values of each layer are respectively (1,2,3 and 4). Compared with a two-dimensional hierarchical coding structure, the test result is shown in table 1, and compared with the two-dimensional hierarchical coding structure, the code rate of { 13.2%, 9.2%, 12.0% } can be saved on average on three components of { Y, U, V } by using the method.
TABLE 1 test results
Figure BDA0002776437480000102
Figure BDA0002776437480000111
Wherein, the 2D-HCS is a 2D Hierarchical Coding Structure, the two-dimensional Hierarchical Coding Structure and the BD-rate represent the code rate saving rate under the same reconstruction level.

Claims (5)

1. A light field integrated picture coding optimization method based on a two-dimensional hierarchical coding structure is characterized by comprising the following steps:
step 1: acquiring and pre-processing a light field dataset
Selecting a plurality of light field pictures from an image library, preprocessing the light field pictures by a Matlab light field tool kit to obtain a picture matrix, discarding fuzzy pictures positioned at the edge in the picture matrix, and organizing a light field pseudo video sequence according to a coding sequence;
step 2: encoding light field pseudo video sequences
Modifying coding parameters of reference software HM of HEVC, adopting a two-dimensional hierarchical coding structure, coding a view picture positioned at a central point into an I frame, distributing other view pictures into four quadrants, and coding all the views into B frames; in each quadrant, 6 view pictures positioned in the same row/column are taken as a picture group, and all the pictures are distributed into different coding layers;
and step 3: updating the remaining B frame Lagrangian multipliers
Step 3.1: according to the direct dependency relationship between the viewpoints under the two-dimensional hierarchical coding structure, the current coding unit U is confirmediDirect-influencing coding unit Un；
Step 3.2: calculating the motion compensation error based on the original frame in the forward prediction and backward prediction, and estimating the coding unit U directly affected under the bidirectional predictionnThe code rate increment of (2);
step 3.3: estimating a current coding unit UiAnd updating the Lagrange multiplier according to the coding cost function.
2. The light field integrated picture based on two-dimensional hierarchical coding structure according to claim 1The coding optimization method is characterized in that the Lagrangian multiplier of the I frame is updated to
Figure FDA0002776437470000011
Figure FDA0002776437470000012
Is the original lagrangian multiplier of the I frame in the HEVC reference software HM.
3. The light field integrated picture coding optimization method based on the two-dimensional hierarchical coding structure according to claim 1, wherein the step 3.1 specifically comprises: the coding units in the same coding layer select two coding units in a lower layer or the same layer as a reference unit to carry out bidirectional prediction, and according to the criterion, the direct dependency relationship between the views under the two-dimensional layer coding structure is obtained by combining the reference relationship between the pictures in the group and between the pictures; according to the direct dependency relationship between the viewpoints, the current coding unit U is confirmediFrames with a direct impact relationship; dividing the current frame into 16-16 coding units, using the basic processing unit size, and obtaining the current coding unit U from the frame with direct influence relation by using the motion search technologyiIs the matched coding unit of (1), namely the unit is subjected to UiCoding units that are directly affected.
4. The light field integrated picture coding optimization method based on the two-dimensional hierarchical coding structure according to claim 3, wherein the step 3.2 specifically comprises: under bi-directional prediction, the affected coding unit UnDirectly subject to a forward coding unit UiI.e. the current coding unit, and the backward coding unit UkThe motion compensation errors are respectively as follows:
forward motion compensation error:
Figure FDA0002776437470000021
backward motion compensation error:
Figure FDA0002776437470000022
wherein F represents forward prediction, B represents backward prediction,
Figure FDA0002776437470000023
and
Figure FDA0002776437470000024
respectively, motion compensation errors based on an original frame in forward prediction and backward prediction, wherein alpha is an empirical value; diAnd DkRespectively representing the current coding unit UiAnd a backward coding unit UkThe distortion of (2);
its bi-directional motion compensation error is expressed as:
Figure FDA0002776437470000025
wherein bi represents bidirectional prediction, and omega is an empirical value;
according to a code rate model
Figure FDA0002776437470000026
Obtaining a current coding unit UiOf the coding distortion introduced affected coding unit UnCode rate increment Δ Rn(Di)：
Figure FDA0002776437470000027
Wherein D is a general name of distortion of the coding unit, and D < DOMCP，DOMCPThe motion compensation error based on the original frame is commonly referred to; delta2Is the variance of the information source; rn(Di) The representation is subjected to the current coding unit UiDirect influence time coding unit UnThe code rate of (2); rn(0) Indicates that there is no current coding unit UiIs a coding unit UnThe code rate of (2);
since D < DOMCPAnd D iskAnd DiAnd if not, then:
Figure FDA0002776437470000028
to pair
Figure FDA0002776437470000029
Performing Taylor expansion with
Figure FDA00027764374700000210
Instead of the former
Figure FDA00027764374700000211
Obtaining:
Figure FDA00027764374700000212
wherein the content of the first and second substances,
Figure FDA0002776437470000031
by obtaining the average motion compensation error of the current coding unit based on the original frame.
5. The light field integrated picture coding optimization method based on the two-dimensional hierarchical coding structure according to claim 3, wherein the step 3.3 is specifically:
estimating all current coding units UiThe coding rate variation of the coding unit directly influenced is the current coding unit UiIs coded at a cost JiComprises the following steps:
Figure FDA0002776437470000032
wherein R isiFor the current coding unit UiCode rate of (1), N is represented byThe video sequence has N viewpoints and N-i affected coding units, wherein only the code rate variation of the directly affected coding units, namely mu, is consideredn1 represents that the coding unit is a directly affected coding unit, and otherwise 0; lambda [ alpha ]gIs a global Lagrangian multiplier; represents the motion compensation error based on the original frame, from forward prediction or backward prediction, as F or B;
by converting, the current coding unit UiThe cost of (c) is written as:
Figure FDA0002776437470000033
i.e. by updating the lagrangian multiplier.
CN202011267205.8A 2020-11-13 2020-11-13 Light field integrated picture coding optimization method based on two-dimensional hierarchical coding structure Pending CN112422975A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011267205.8A CN112422975A (en) 2020-11-13 2020-11-13 Light field integrated picture coding optimization method based on two-dimensional hierarchical coding structure

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011267205.8A CN112422975A (en) 2020-11-13 2020-11-13 Light field integrated picture coding optimization method based on two-dimensional hierarchical coding structure

Publications (1)

Publication Number Publication Date
CN112422975A true CN112422975A (en) 2021-02-26

Family

ID=74831215

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011267205.8A Pending CN112422975A (en) 2020-11-13 2020-11-13 Light field integrated picture coding optimization method based on two-dimensional hierarchical coding structure

Country Status (1)

Country Link
CN (1) CN112422975A (en)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107040782A (en) * 2017-04-21 2017-08-11 上海电力学院 The global Rate-distortion optimization method of Video coding based on Lagrangian method

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107040782A (en) * 2017-04-21 2017-08-11 上海电力学院 The global Rate-distortion optimization method of Video coding based on Lagrangian method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
高艳博: "基于率失真优化的视频编码方法研究", 《中国博士学位论文全文数据库 信息科技辑》 *

Similar Documents

Publication Publication Date Title
JP6814783B2 (en) Valid predictions using partition coding
JP6788699B2 (en) Effective partition coding with high partitioning degrees of freedom
JP4890201B2 (en) Method and system for managing multi-view video
JP5274766B2 (en) Method and system for predicting multi-view video
JP5106830B2 (en) Method and system for randomly accessing multiview video
US8098728B2 (en) Compressing and decompressing multiple, layered, video streams employing multi-directional spatial encoding
US8090025B2 (en) Moving-picture coding apparatus, method and program, and moving-picture decoding apparatus, method and program
US20100303148A1 (en) Macroblock-based dual-pass coding method
WO2007143876A1 (en) Method and apparatus for adaptively determining a bit budget for encoding video pictures
US20060204115A1 (en) Video encoding
KR20010021879A (en) Apparatus and method for macroblock based rate control in a coding system
CN108200431B (en) Bit allocation method for video coding code rate control frame layer
CN1258925C (en) Multiple visual-angle video coding-decoding prediction compensation method and apparatus
CN1695381A (en) Sharpness enhancement in post-processing of digital video signals using coding information and local spatial features
JP2008306705A (en) Bitrate control method and apparatus for video sequence encoding by using only intra picture
US20050053132A1 (en) Method and apparatus for 3-D subband video coding
JP3508916B2 (en) Moving image variable bit rate encoding method and apparatus
CN112422975A (en) Light field integrated picture coding optimization method based on two-dimensional hierarchical coding structure
Li et al. Rate-distortion-optimized content-adaptive coding for immersive networked experience of sports events
De Abreu et al. Fast MVC prediction structure selection for interactive multiview video streaming
Stefanoski et al. Image quality vs rate optimized coding of warps for view synthesis in 3D video applications
JP4522951B2 (en) Moving picture encoding method and apparatus, decoding method and apparatus, moving picture processing program, and computer-readable recording medium
Chiang et al. Enhanced Temporal Consistency for Global Patch Allocation in Video-based Point Cloud Compression
JPH0678287A (en) Coding method and device
KR100731981B1 (en) The method and system of multi-view video encoding based on grid type pyramid gop structure

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20210226

RJ01 Rejection of invention patent application after publication