WO2015192314A1

WO2015192314A1 - A simplified method for depth based block partitioning

Info

Publication number: WO2015192314A1
Application number: PCT/CN2014/080062
Authority: WO
Inventors: Xianguo Zhang; Kai Zhang; Jicheng An; Han HUANG
Original assignee: Mediatek Singapore Pte. Ltd.
Priority date: 2014-06-17
Filing date: 2014-06-17
Publication date: 2015-12-23

Abstract

Methods of depth-based block partitioning (DBBP) for multi-view video coding and 3D video coding are disclosed. Several methods are proposed to reduce the calculation or accessing of the samples in the segment mask by performing the DBBP process sub-block by sub-block.

Description

A SIMPLIFIED METHOD FOR DEPTH BASED BLOCK

PARTITIONING

FIELD OF INVENTION

The invention relates generally to Multi-view video coding and Three-Dimensional (3D) video coding. In particular, the present invention relates to simplified methods for depth-based block partitioning in 3D video coding.

BACKGROUND OF THE INVENTION

3D video coding is developed for encoding or decoding video data of multiple views simultaneously captured by several cameras. Since all cameras capture the same scene for both the input texture videos and depth videos, the depth information can be utilized to improve the motion compensation efficiency of texture videos. Especially, the corresponding depth block of the texture block can represent the pixel level object segmentation, so it is reasonable to realize pixel-level segment based motion compensation by utilizing the depth information. Therefore, a depth-based block partitioning (DBBP) is adopted for texture video coding in the current 3D- HEVC.

The current depth-based block partitioning comprises steps of virtual depth derivation, block partitioning, block segmentation, predicted block combination and post-filtering. (1) Virtual depth derivation utilizes the disparity vector from neighboring blocks ( BDV) to address a virtual block of depth from the depth picture in the dependent depth view; (2) One partition among 2NxN, Nx2N, 2NxnU, 2NxnD, nLx2N and nRx2N is selected by the block partitioning to derive two predictive motion vectors (PMVs), which are utilized for the compensation procedure for the to-be-divided two segments; (3) Block segmentation process divides the current texture block into two segments according to segment mask generated from the virtual depth block; (4) Whereas the predicted block combination utilizes the PMVs and motion vector differences (MVDs) from block partitioning and the derived segment mask to fetch the predicted pixels, and then completes the combination by two-time 2Nx2N predictions with their own decoded motion vectors. Thus the combination is completed through two 2Nx2N compensations under the control of the calculated segment mask. (5) Finally, a post filtering process is utilized to smooth the internal pixel boundaries(where neighboring mask values are different) of the combined block.

From the description above, it can be found that segment mask generation, the predicted block combination and post filtering are all based on the mask, as shown in Fig. 1. Therefore, improved mask generation is the key point for DBBP.

In the process of mask generation, a mean value is firstly calculated by averaging each pixel of the virtual depth block. In the following, if the left-up corner pixel is larger than the mean value, all positions in the mask where the corresponding pixels in virtual depth are larger than the mean value are marked with 'Ο', others are marked with T; Otherwise, all positions in the mask where the corresponding pixels in virtual depth block are smaller or equal than the mean value are marked with 'Ο', others are marked with T.

In the mask based predicted block combination, the basic rule is, the 1^st 2Nx2N predicted block's pixels with "0" in the segment mask and 2^nd 2Nx2N predicted block's pixels with "1" in the segment mask are combined as the predicted pixels of the decoded CU. In addition, for border positions where 1 and 0 are neighboring in the segment mask, a blending operation in the final post-filtering step is utilized to average the two predicted values at the same position in the two predicted 2Nx2N blocks.

However, complexity problems exist in the following two aspects. (1) The mask generation process requires pixel-level mean-value calculation and pixel-by-pixel comparison.

(2) The combination of the two 2Nx2N predicted block must be operated pixel by pixel.

Thereby, a simplified method with computational complexity reduction for the mask based motion prediction will be welcome for DBBP process.

SUMMARY OF THE INVENTION

In this invention, it is proposed to reduce the calculation or access of the samples in the segment mask by performing the DBBP process of mask generation, block combination and post filtering sub-block by sub-block.

Other aspects and features of the invention will become apparent to those with ordinary skill in the art upon review of the following descriptions of specific embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention can be more fully understood by reading the subsequent detailed description and examples with references made to the accompanying drawings, wherein:

Fig. 1 is a diagram illustrating the mask based motion compensation steps of depth-based block partitioning (DBBP).

Fig. 2 is a diagram illustrating the modified mask based steps of depth-based block partitioning (DBBP). DETAILED DESCRIPTION

The following description is of the best-contemplated mode of carrying out the invention. This description is made for the purpose of illustrating the general principles of the invention and should not be taken in a limiting sense. The scope of the invention is best determined by reference to the appended claims.

There are many embodiments proposed in this invention to simplify depth-based block partitioning (DBBP).

In a first embodiment, the mean value for the virtual depth block is equal to the average results of all pixels at (x, y) positions of all the kxm-size sub blocks, where x and y are respectively smaller than k and m.

In a second embodiment, the size of the segment mask is unchanged, and the samples of each kxm-size sub-block in the segment mask are the same and equal to the comparison between one specified (x,y)-position pixel in the corresponding kxm-size sub-block in the virtual depth block and the mean value. An example is shown is Fig. 2, where k and m are equal to 2, x and y equal to 0.

In a third embodiment, the size of the segment mask is changed, and the samples at (x, y) position in the segment mask is equal to the comparison between one specified (kxx+i,mxy+j)- position pixel value in the virtual depth block and the mean value, where i and j should be respectively smaller than k and m. For example, k and m are equal to 2, i and j are equal to 0.

In a fourth embodiment, the size of the segment mask is unchanged, and the process of combing each kxm-size sub-block of two predicted 2Nx2N block is only according to the value at (x,y)-position in the corresponding kxm-size sub-block in the segment mask. For example, k and m are equal to 2, x and y are equal to 0.

In a fifth embodiment, the size of the segment mask is changed, and the process of combing each kxm-size sub-block at (x,y) position of two predicted 2Nx2N block is according to the pixel at (x/k, y/m) position of the segment mask. For example, k and m are equal to 2.

In a sixth embodiment, the process of filtering each pixel at (x,y) position of the kxm-size sub-block of combined block of the two predicted 2Nx2N block is related to the values x and y. For example, k and m are equal to 2.

In a seventh embodiment, any combinations of the first to sixth embodiments are included.

The proposed method described above can be used in a video encoder as well as in a video decoder. Embodiments of the method according to the present invention as described above may be implemented in various hardware, software codes, or a combination of both. For example, an embodiment of the present invention can be a circuit integrated into a video compression chip or program codes integrated into video compression software to perform the processing described herein. An embodiment of the present invention may also be program codes to be executed on a Digital Signal Processor (DSP) to perform the processing described herein. The invention may also involve a number of functions to be performed by a computer processor, a digital signal processor, a microprocessor, or field programmable gate array (FPGA). These processors can be configured to perform particular tasks according to the invention, by executing machine-readable software code or firmware code that defines the particular methods embodied by the invention. The software code or firmware codes may be developed in different programming languages and different format or style. The software code may also be compiled for different target platform. However, different code formats, styles and languages of software codes and other means of configuring code to perform the tasks in accordance with the invention will not depart from the spirit and scope of the invention.

The invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples are to be considered in all respects only as illustrative and not restrictive. To the contrary, it is intended to cover various modifications and similar arrangements (as would be apparent to those skilled in the art). Therefore, the scope of the appended claims should be accorded the broadest interpretation so as to encompass all such modifications and similar arrangements.

Claims

1. A method of depth-based block partition (DBBP) for multi-view video coding or 3D video coding comprising, a mean value calculation of a virtual depth, a mask generation, a predicted block combination or a post filtering process considers pixel values in each kxm sub- block of the virtual depth block are the same.

2. The method of as claimed in claim 1, wherein all pixels in each kxm sub-block of a current block utilize the same value in a segment mask for the predicted block combination.

3. The method of as claimed in claim 1, wherein the mean value for the virtual depth block is equal to average results of all pixels at (x, y) positions of all the kxm-size sub blocks, where x and y are respectively smaller than k and m.

4. The method of as claimed in claim 2, wherein the size of the segment mask is unchanged, and samples of each kxm-size sub-block in the segment mask are the same and equal to the comparison between one specified (x,y)-position pixel in the corresponding kxm- size sub-block in the virtual depth block and the mean value, x and y are must smaller than k and m, and typically equal to 0.

5. The method of as claimed in claim 4, wherein k and m are the same.

6. The method of as claimed in claim 2, wherein the size of the segment mask is changed, and samples at (x, y) position in the segment mask is equal to the comparison between one specified (kxx+i,mxy+j) position pixel value and the mean value, where i and j should be respectively smaller than k and m.

7. The method of as claimed in claim 2, wherein the size of the segment mask is unchanged, and the process of combing each kxm-size sub-block of two predicted 2Nx2N block is only according to the value at (x,y)-position in the corresponding kxm-size sub-block in the segment mask, wherein x and y are smaller than k and m.

8. The method of as claimed in claim 2, wherein the size of the segment mask is changed, and the process of combing each kxm-size sub-block at (x,y) position of two predicted 2Nx2N block is according to the pixel at (x/k, y/m) position of the segment mask.

9. The method of as claimed in claim 2, wherein the process of filtering each pixel at (x,y) position of the kxm-size sub-block of the combined block of the two predicted 2Nx2N block is related to the values x and y.

10. The method of as claimed in claim 1, wherein the mask generation, the predicted block combination or the post filtering process works sub-block by sub-block, wherein the sub-block sizes are the same or different with each other.