WO2016049913A1

WO2016049913A1 - A simplified method for the depth modeling modes

Info

Publication number: WO2016049913A1
Application number: PCT/CN2014/088038
Authority: WO
Inventors: Xianguo Zhang; Kai Zhang; Jian-Liang Lin; Jicheng An; Yi-Wen Chen
Original assignee: Mediatek Singapore Pte. Ltd.
Priority date: 2014-09-30
Filing date: 2014-09-30
Publication date: 2016-04-07

Abstract

Methods of Depth Modeling Modes (DMM) for multi-view video coding and 3D video coding are disclosed. Several methods are proposed to reduce the complexity of DMMl including (1) Constraining the available wedgelet candidates' starting or ending points, or the available wedgelet directions; (2) Down scaling the starting and ending points of the wedgelets in the tables of larger PUs, and then utilizing the down-sampled wedgelet candidates for smaller intra PUs; (3) Constaining the available wedgelet number (or the table size) for each PU size.

Description

A SIMPLIFIED METHOD FOR THE DEPTH MODELING MODES

Field of the Invention

The invention relates generally to Multi-view video coding and Three-Dimensional (3D) video coding. In particular， the present invention relates to simplified methods for Depth Modeling modes (DMMs) in 3D video coding.

Description of the Related Art

3D video coding is developed for encoding or decoding video data of multiple views simultaneously captured by several cameras. Because coding the depth information consumes a lot of bits， kinds of coding tools are proposed to enhance the depth picture coding efficiency. Among these tools， the two Depth Modeling Modes (DMM1 and DMM4) are adopted to improve the intra prediction efficiency of depth pictures.

Generally speaking， DMM1 and DMM4 are separately based on Wedgelet and Contour partitioning. For a Wedgelet partition， the two regions are defined to be separated by a straight line， as illustrated in Figure 1， in which the two regions are labeled with P₁ and P₂.The separation line is determined by the start point S and the end point P， both located on different borders of the block. For the continuous signal space (see Figure 1， left) ， the separation line can be described by the equation of a straight line. The middle image of Figure 1 illustrates the partitioning for the discrete sample space. Here， the block consists of an array of samples with size u_B×v_B and the start and end points correspond to border samples. Although the separation line can be described by a line equation as well， the definition of regions P₁ and P₂ is different here， as only complete samples can be assigned as part of either of the two regions. For employing Wedgelet block partitions in the coding process， the partition information is stored in the form of partition patterns. Such a pattern consists of an array of size u_B×v_B and each element contains the binary information whether the corresponding sample belongs to region P₁ and P₂. The regions P₁ and P₂ are represented by black and white samples in Figure 1 (right) ， respectively.

Unlike for Wedgelets， the separation line between the two regions of a Contour partition of a block cannot be easily described by a geometrical function. As illustrated in Figure 2， the two regions P₁ and P₂ can be arbitrary shaped and even consist of multiple parts. Apart from that the properties of Contour and Wedgelet partitions are very similar. For employing Contour partitions in the coding process， the partition pattern (see example in Figure 2， right) is derived individually for each block from the signal of a reference block. Due to the lack of a functional description of the region separation line， no pattern lookup lists and consequently no search of the best matching partition are used for Contour partitions.

Above all， although DMM1 has the advantage of significant BDRate savings， the number of wedgelet patterns for DMM1 consumes a large table in both encoder and decoder to store the candidate patterns for intra prediction. Table 1 lists the size of each table for each Intra PU size in the JCT3V-I1001， 3D-HEVC Draft Text 5.

The wedgelet patterns of DMM1 can be classified into 6 categories， as Fig. 3 shows， including up-left corner， up-right corner， down-left corner， down-right corner， horizontal and vertical directions， numbered as 1～6^th direction.

Thereby， a method to reduce the wedgelet table size for DMM1 will be welcome， if the performance can be maintained.

Table 1. The wedgelet table size for each Intra PU size in the 3D-HEVC Draft Text 5

BRIEF SUMMARY OF THE INVENTION

In this invention， it is proposed to reduce the wedgelet table size for DMM1.

The methods include the following three kinds of simplifications： (1) Constraining the available wedgelet candidates’ starting or ending points； (2) Down scaling the starting and ending points of the wedgelets in the tables of larger PUs， and then utilizing the down-scaled wedgelet candidates for smaller intra PUs. (3) While adding wedgelet patterns into the wedgelet pattern list， constrain the total number of available wedgelets to a fixed number and do not add any pattern any more when the wedgelet list is full.

Other aspects and features of the invention will become apparent to those with ordinary skill in the art upon review of the following descriptions of specific embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention can be more fully understood by reading the subsequent detailed description and examples with references made to the accompanying drawings， wherein：

Fig. 1 is a diagram illustrating the wedgelet partition example of DMM1， continuous (left) and discrete signal space (middle) with corresponding partition pattern (right) .

Fig. 2 is a diagram illustrating the contour partition example of DMM4， continuous (left) and discrete signal space (middle) with corresponding partition pattern (right) .

Fig. 3 is a diagram illustrating the current available wedgelet candidates’ starting or ending points.

Fig. 4 is a diagram illustrating the simplification example of constraining the available wedgelet candidates’ starting or ending points.

DETAILED DESCRIPTION OF THE INVENTION

The following description is of the best-contemplated mode of carrying out the invention. This description is made for the purpose of illustrating the general principles of the invention and should not be taken in a limiting sense. The scope of the invention is best determined by reference to the appended claims.

There are three kinds of methods proposed in this invention to reduce the table size of DMM1. (1) Constraining the available wedgelet candidates’ starting or ending points， or the available wedgelet directions； (2) Down scaling the starting and ending points of the wedgelets in the tables of larger PUs， and then utilizing the down-scaled wedgelet candidates for smaller intra PUs. (3) While adding wedgelet pattern into the wedgelet pattern list， constrain the total number of available wedgelets to a fixed number and do not add any pattern any more when the wedgelet list is full.

A first embodiment of method 1 is to constrain the starting point position (x， y) by x％k＝＝m (m＜k) ， y％t＝＝n (n＜t) .

A second embodiment of method 1 is to constrain the ending point position (x， y) by x％k＝＝m (m＜k) ， y％t＝＝n (n＜t) .

Another embodiment of method 1 is to constrain the ending point position (x， y) to be from a limited set of values， such as x＜k or y＜t.

Another embodiment of method 1 is to constrain the starting point position (x， y) to be from a limited set of values， such as x＜k or y＜t.

Another embodiment of method 1 is to constrain the starting point or ending point only for a selected subset of PU sizes.

Another embodiment of method 1 is to constrain the starting point or ending point only for a selected subset of wedgelet directions， among all the 6 wedgelet directions. Let L denote size of A， L is smaller than 6. For example， Fig. 4 illustrates one example， which constrains the starting and ending points’ coordinates be even values.

Another embodiment of method 1 is to only utilize the wedgelet from a selected subset of all the wedgelet directions， A. Let L denote size of A， L is smaller than 6.

Any other embodiment which combines some of the above embodiments for method 1 can be also included.

A first embodiment of method 2 is to re-use the tables for n1xn1 size intra prediction for n2xn2 size intra prediction， when n2 is smaller than n1.

A second embodiment of method 2， in the re-using procedure， one temporal table can be generated by down scaling the table used for the larger intra PUs. And then， this generated table is utilized for the smaller intra PUs.

Another embodiment of method 2， in the re-using procedure， the smaller n×n PUs， when processing one pixel at position (x， y) ， the value at position (k×x， k×y) in the wedgelet table of m×m PU can be utilized， where k is equal to m/n.

Any other embodiment which combines some of the above embodiments for method 2 can be also included.

A first embodiment of method 3 is to constrain total available wedgelet number (or the table size) of the k×k intra PU to be the n to the power of 2， where n is larger for larger intra PUs.

A second embodiment of method 3， when adding one wedgelet pattern into the wedgelet list， the wedgelet is only added when the corresponding wedgelet list for the current intra PU is not full (or total wedgelet storage size is not larger than the table size) .

Another embodiment of method 3 is to only select limited wedgelet patterns for each wedgelet direction to make the total wedgelet storage size be smaller than the pre-defined table size.

Any other embodiment which combines some of the above embodiments for method 3 can be also included.

Any other embodiment which combines embodiments for method 1， method 2 or method 3 can be also included.

The proposed method described above can be used in a video encoder as well as in a video decoder. Embodiments of the method according to the present invention as described above may be implemented in various hardware， software codes， or a combination of both. For example， an embodiment of the present invention can be a circuit integrated into a video compression chip or program codes integrated into video compression software to perform the processing described herein. An embodiment of the present invention may also be program codes to be executed on a Digital Signal Processor (DSP) to perform the processing described herein. The invention may also involve a number of functions to be performed by a computer processor， a digital signal processor， a microprocessor， or field programmable gate array (FPGA) . These processors can be configured to perform particular tasks according to the invention， by executing machine-readable software code or firmware code that defines the particular methods embodied by the invention. The software code or firmware codes may be developed in different programming languages and different format or style. The software code may also be compiled for different target platform. However， different code formats， styles and languages of software codes and other means of configuring code to perform the tasks in accordance with the invention will not depart from the spirit and scope of the invention.

The invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples are to be considered in all respects only as illustrative and not restrictive. To the contrary， it is intended to cover various modifications and similar arrangements (as would be apparent to those skilled in the art) . Therefore， the scope of the appended claims should be accorded the broadest interpretation so as to encompass all such modifications and similar arrangements.

Claims

A method of depth modeling modes (DMM) for multi-view video coding or 3D video coding comprising， for DMM1 mode， there are limited stored wedgelet pattern candidates at each wedgelet direction for each intra PU size.
The method of as claimed in claim 1， wherein the available starting or ending points’ horizontal coordinates are constrained to a subset of {k|k＝0～s-1} for the s×s intra PU.
The method of as claimed in claim 1， wherein available wedgelet directions are constrained to a subset of all the six directions for each PU size.
The method of as claimed in claim 1， wherein table size for each intra PU size is pre-defined to a fixed value， so the wedgelet candidate number for each intra PU size cannot exceed the number of the wedgelets which the pre-defined table size can store.
The method of as claimed in claim 2， wherein the starting or ending points’ horizontal coordinates are limited to be values which are equal to t by taking them modulo k， where t is one positive value smaller than k， for example， t is equal to 0 and k is set 2.
The method of as claimed in claim 2， wherein the starting or ending points’ horizontal coordinates are constrained differently according to the PU size and the wedgelet direction itself.
The method of as claimed in claim 6， wherein the starting or ending points’ horizontal coordinates are constrained by the value k and t when the intra PU size is larger than m， for example， k， t and m are equal to 2， 0 and 8.
The method of as claimed in claim 6， wherein the starting or ending points’ horizontal coordinates are constrained by the value k_i and t_i when the intra PU size is larger than m_i for the wedgelet direction i.
The method of as claimed in claim 8， wherein typically for i＝0～3， k_i， t_i and m_i are equal to {2， 0， 8} ， {2， 0， 16} ， {2， 1， 8} ， {1， 0， 8} or {1， 0， 16} ； for i＝4～5， k_i， t_i and m_i are equal to {1， 0， 8} ， {2， 0， 8} ， {2， 1， 8} ， {2， 0， 16} ， {1， 0， 16} .
The method of as claimed in claim 9， wherein for the starting point and ending point of i＝0～3， k_i， t_i and m_i are equal to {2， 0， 8} ； for the starting point of i＝4～5， k_i， t_i and m_i are equal to {2， 0， 8} ； for the ending point of i＝4～5， k_i， t_i and m_i are equal to {1， 0， 8} .
The method of as claimed in claim 9， wherein for the starting point and ending point of i＝0～5， k_i， t_i and m_i are equal to {2， 0， 8} .
The method of as claimed in claim 9， wherein for the starting point and ending point of i＝0～3， k_i， t_i and m_i are equal to {2， 0， 8} ； for the starting point and ending point of i＝4～5， k_i， t_i and m_i are equal to {1， 0， 8} .
The method of as claimed in claim 3， wherein for PU size larger than s×s， only the horizontal and vertical wedgelet directions are included， for example， s is equal to 8.
The method of as claimed in claim 3， wherein for PU size not larger than s×s， only the corner directions are included， for example， s is equal to 8.
The method of as claimed in claim 3， wherein for PU size not larger than s×s， only the horizontal and vertical wedgelet directions are included， for example， s is equal to 8.
The method of as claimed in claim 3， wherein for PU size larger than s×s， only the horizontal and corner wedgelet directions are included， for example， s is equal to 8.
The method of as claimed in claim 3， wherein for PU size larger than s×s， only the vertical and corner wedgelet directions are included， for example， s is equal to 8.
The method of as claimed in claim 4， the table size t_s for the intra PU， whose size is s×s， is predefined to the n_s to the a power of 2， for example， n₁₆ is set 512 or 1024 and n₈ is set 512.
The method of as claimed in claim 4， while adding wedgelet into the wedgelet list for the s×s PU， any wedgelet can be added before the corresponding wedgelet table is full.
The method of as claimed in claim 4， while adding wedgelet into the wedgelet list for the s×s PU， there is one value， v_i， for the largest number of wedgelets for the wedgelet direction i (among the total six directions) ， in this case， the number of wedgelet candidates to be added into the wedgelet table for direction i cannot exceed v_i.
The method of as claimed in claim 1， wherein a smaller PU utilize the wedgelet table of a larger PU for DMM1 procedures， in this case， there is no wedgelet tables required to be stored for these smaller PUs.
The method of as claimed in claim 21， wherein while the smaller PUs utilize the wedgelet table of larger PUs for DMM1 procedures， temporal tables down-scaled from the larger PUs’ table are utilized for the DMM1 procedures.
The method of as claimed in claim 4， wherein the predefined table size is 2^n， where n is a positive integer.
The method of as claimed in claim 1， wherein the construction of the wedgelet tables are conducted in different ways for different PU size.
The method of as claimed in claim 1， wherein a lager PU utilize the wedgelet table of a smaller PU for DMM1 procedures， in this case， there is no wedgelet tables required to be stored for these larger PUs.
The method of as claimed in claim 1， wherein the construction of the wedgelet tables and/or the number of wedgelets in each table are signaled from the encoder to the decoder.
The method as claimed in claim 1， wherein a wedgelet table is down-sampled by restricting the starting point to odd (or even) positions only.
The method as claimed in claim 1， wherein a wedgelet table is down-sampled by restricting the ending point to odd (or even) positions only.
The method as claimed in claim 1， wherein a wedgelet table is down-sampled by restricting the starting point to odd (or even) positions only and restricting the ending point to odd (or even) positions only.
The method as claimed in claim 1， wherein a wedgelet table is down-sampled by restricting the starting point to odd (or even) positions only if the starting point is located in the left and right boundary， and/or in the above and bottom boundary.
The method as claimed in claim 1， wherein a wedgelet table is down-sampled by restricting the ending point to odd (or even) positions only if the ending point is located in the left and right boundary， and/or in the above and bottom boundary.