WO2020244296A1 - System and method for multi-layer representation of depth map during intra-frame coding - Google Patents
System and method for multi-layer representation of depth map during intra-frame coding Download PDFInfo
- Publication number
- WO2020244296A1 WO2020244296A1 PCT/CN2020/082464 CN2020082464W WO2020244296A1 WO 2020244296 A1 WO2020244296 A1 WO 2020244296A1 CN 2020082464 W CN2020082464 W CN 2020082464W WO 2020244296 A1 WO2020244296 A1 WO 2020244296A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- layer
- pixels
- depth
- encoder
- module
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 66
- 230000000750 progressive effect Effects 0.000 claims abstract description 27
- 238000005538 encapsulation Methods 0.000 claims abstract description 14
- 238000012545 processing Methods 0.000 claims abstract description 6
- 238000013507 mapping Methods 0.000 claims description 25
- 238000013139 quantization Methods 0.000 claims description 25
- 230000008569 process Effects 0.000 claims description 12
- 238000012804 iterative process Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 230000003044 adaptive effect Effects 0.000 claims description 3
- 230000000903 blocking effect Effects 0.000 claims 1
- 238000007906 compression Methods 0.000 abstract description 5
- 230000006835 compression Effects 0.000 abstract description 5
- 238000011002 quantification Methods 0.000 abstract description 3
- 238000006243 chemical reaction Methods 0.000 abstract description 2
- 238000004590 computer program Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000011218 segmentation Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 239000000969 carrier Substances 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/124—Quantisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
Definitions
- the present invention relates to a system and method for processing a depth map, and more particularly to a system and method for multi-layer representation of an intra-frame coding depth map.
- the depth map is an image or video that records the distance between the observable scene point and the camera light point. It provides additional information for related color pixels in color images or videos shot at the same location by specifying its depth in the scene. Therefore, the depth map is a key component of the 3D multimedia experience.
- the display device has 3D structure information from which to recover the scene depicted in the image or video with the depth map.
- the compression and encoding of the depth map is a process to reduce the amount of depth map data and provide standards for different terminal devices of the network to understand the transmitted data. Depth map coding can also be regarded as a part of the entire 3D video data compression process.
- the most common depth map coding or general video coding schemes follow the same "hybrid" video coding framework.
- the frame is divided into blocks, taking advantage of the spatial dependence between adjacent blocks and frames.
- the data is predicted and coded from previously coded blocks and frames.
- Intra coding is the basic step of depth map coding. It is part of the encoding process where data is only predicted from previous data from the same frame.
- the intra-frame coding method only performs operations with respect to the information included in the current frame, and does not perform operations with respect to the information included in any other frames in the video sequence.
- 3D-High Efficiency Video Coding The most advanced standard 3D high-efficiency video coding (“3D-High Efficiency Video Coding”) in the prior art adopts the following intra-frame coding methods, including: 1. Direct current (DC) and planar prediction; 2. Based on wedgelet (“wedgelet”) The depth modeling; 3. The depth modeling of the contour segment. Only smooth data can be processed by direction prediction. Wedge-based depth modeling and contour segment depth modeling can handle drastic changes, but there are only two layers in the blocks that the frame is divided into. In addition, the depth modeling of contour line segments often obtains sharp changes from the corresponding texture video. The limitation of these methods is that the segmentation is performed only once. Therefore, if the segmentation quality is poor, you must rely on other coding methods to improve the reconstruction quality.
- the method of the present invention to adaptively quantize the image and divide the image into constant-valued layers provides a system and method for compressing depth map data in a relatively complex scene.
- the present invention provides a system for multi-layer representation of an intra-frame coded depth map, including the following devices: a block module to block the depth map data; a progressive quantization module to set progressive quantization and stop conditions for the block depth map data.
- One aspect of the present invention also includes the following devices: a data encapsulation module, which performs data encapsulation on the multi-layer depth map that stops iteration; and a data output module, which outputs the encapsulated bit stream to the decoder end.
- the progressive quantization module for setting progressive quantization and its stopping conditions for the divided depth map data further includes: a layering module, which decomposes the depth block into multiple layers, and each layer contains the depth block A subset of pixels that are mutually exclusive with other layers; a multi-layer representation module that represents the multiple layers in a non-parametric way, one of which can represent any subset of all pixels in the depth block; an iterative module, which uses an iterative method to Encode multiple layers and continuously monitor reconstruction residuals and remaining areas with high priority.
- the progressive quantization module for setting progressive quantization and its stopping conditions for the divided depth map data includes: a new layer creation module, which initializes an empty layer list with an encoder, in the layer list Create a new layer, and then classify all pixels in the block as a unique layer; the encoder repeats the following modules until the end of the iteration process: calculation module, for each layer in the layer list, use the encryptor to calculate the graph The average value and variance of the depth value of all pixels in the layer, and each average value is attached to the corresponding layer; the depth value reconstruction module, the encoder finds the maximum value of all the calculated variances, and identifies the layer with the largest variance , Call it as the maximum variance layer; and create a reconstructed block by assigning a depth value to each pixel using the average value of the layer to which each pixel belongs; calculate the sum of squared errors between the reconstructed block and the original depth block; new layer The creation module uses the encoder to create a new layer at the end of the layer
- the predetermined threshold is the required reconstruction quality in terms of the sum of squared errors.
- the data encapsulation module further includes: an average value addition module, which uses an encoder to calculate the average value and variance of the depth values of all pixels in the layer for all layers in the layer list; Each average value is attached to the corresponding layer; the layer processing module, the encoder reorders the layers by sorting the area or the number of pixels of each layer in descending order, so that the layer with the most pixels is processed first; binary Mapping module, the encoder uses the layer that has not been encoded and has the largest number of pixels to form a binary map, mark all pixels in the layer as "1", mark other pixels as "0", and use context adaptive binary arithmetic coding Method to encode this binary mapping; after completing the mapping, the encoder continues to the next largest layer and repeats until one layer is left; the last layer does not require binary mapping because it will automatically fill in all remaining pixels.
- an average value addition module which uses an encoder to calculate the average value and variance of the depth values of all pixels in the layer for all layers in the layer list; Each average value is attached to the
- the final output data in the data encapsulation module is composed of the following content: an integer representing the number of layers in the depth block; a series of bits containing the binary mapping of all layers; a value representing the depth value of each layer Series integer.
- the present invention also provides a method for multi-layer representation of an intra-frame coding depth map, which includes the following steps: dividing the depth map data into blocks; setting the progressive quantization and stopping conditions for the divided depth map data.
- the present invention also provides an encoder for implementing the method of the present invention.
- the stepwise quantization and stopping conditions are set for the block depth map data, which includes: a new layer creation module, which uses the encoder to initialize an empty layer List, create a new layer in the layer list, and then classify all pixels in the block as a unique layer; the encoder repeats the following modules until the end of the iteration process: calculation module, for the layer list For each layer of, use the encryptor to calculate the mean value and variance of the depth value of all pixels in the layer, and attach each mean value to the corresponding layer; depth value reconstruction module, the encoder finds all the calculated variances The maximum value in, identify the layer with the largest variance, call it the largest variance layer; and create a reconstruction block by assigning a depth value to each pixel using the average value of the layer to which each pixel belongs; calculate the reconstruction block and the original depth block The sum of squared errors between; the new layer creation module uses the encoder to create a new layer at the end of the layer list
- the method described in the present invention processes depth map data in a manner that mimics the properties of depth data.
- Depth maps usually contain large and smooth areas with a clear boundary between the two.
- the present invention realizes the smoothness of the depth map by reducing the pixels in the same smooth area to simple representative values and at the same time investing resources in the recording pixel grouping.
- the quality target can make the reconstruction output of this method better serve the overall quality and bit rate control of the video encoder.
- One immediate application of the present invention is 3D video content compression for online video broadcasting. Another application is 3D video format conversion.
- Fig. 1 is a schematic diagram of a method for multi-layer representation of an intra-frame coded depth map according to the present invention.
- Figures 2a-2e are examples of the steps of setting progressive quantization and its stopping conditions in the method for multi-layer representation of intra-coded depth maps according to the present invention.
- Fig. 3 schematically shows a block diagram of a server for executing the method according to the present invention.
- Fig. 4 schematically shows a storage unit for holding or carrying program codes for implementing the method according to the present invention.
- the present invention aims to provide an effective method for processing complex depth map data.
- the depth map of the present invention is actually decomposed into several "shapes", which are combined into all the pixels in the depth block to be processed, and each of these shapes One is appended with a depth value for use as a decoding result.
- the method uses the standard derived based on variance to identify regions with poor reconstruction quality in an iterative manner to repeatedly improve the reconstruction quality.
- Fig. 1 is a schematic diagram of a method for multi-layer representation of an intra-frame coded depth map according to the present invention.
- the depth block is decomposed into multiple layers, and each layer contains a subset of pixels in the depth block that are mutually exclusive with other layers.
- step B Progressive quantization and its stopping conditions below for specific content.
- the multiple layers are represented in a non-parametric manner, and each layer can represent any subset of all pixels in the depth block.
- step B Progressive quantification and sub-step d. of its stopping conditions below. It adapts to complex environments and can assign depth values in the scene arbitrarily.
- the encoding process adopts an iterative method. See step B.
- Step A Divide the depth data into blocks
- the picture is divided into smaller units.
- the block width and height are usually powers of 2. For example, 2 ⁇ 2, 4 ⁇ 4, 8 ⁇ 8...
- Step B Set progressive quantization and its stopping conditions
- the step of setting the progressive quantization step and its stopping condition includes multiple stages. Among them, the encoder first initializes an empty "layer list", creates a new layer in the list, and then classifies all pixels in the block as a unique layer. After initialization, the encoder repeats the following process until the stop condition set during the process is met.
- the encryptor calculates the mean and variance of the depth values of all pixels in the layer. Each average value is attached to the corresponding layer.
- the encoder creates a "reconstruction block” by assigning a depth value to each pixel using the average value of the layer to which each pixel belongs. Calculate the sum of squared errors (SSE) between the reconstructed block and the original depth block. If the SSE is less than a preset threshold, for example, the predetermined threshold is the required reconstruction quality in terms of SSE, then this iterative process ends.
- SSE squared errors
- the encoder finds the maximum value among all the variances calculated in step a, identifies the layer with the largest variance, and calls it the maximum variance layer ("LLV").
- the encoder creates a new layer at the end of the layer list, selects all pixels in the maximum variance layer and whose depth value is greater than the average value of the maximum variance layer, deletes these pixels from the maximum variance layer and assigns them to the new layer . Go back to step a.
- Figures 2a-2e are an example of the step of setting progressive quantization and its stopping conditions in the method for multi-layer representation of an intra-coded depth map according to the present invention.
- Figure 2a-1 is an example of a depth block, which is colored with different shades of gray to indicate different values.
- the white part 201 represents the part with the depth value of "30"
- the second light gray part 202 represents the part with the depth value of "25”
- the darker gray part 203 represents the part with the depth value of "20" and "12”
- the dark gray portion 204 represents a portion with a depth value of "10".
- Figure 2a-2 shows that for the initial layer list, all pixels are located in the "0 layer”.
- Figure 2b-1 shows that after the first iteration, the entire block is divided into two layers.
- the dark gray part 205 in Figure 2b-1 is the part whose value is lower than 15 in the depth map of Figure 2a-1; the light gray part 206 in Figure 2b-1 is the value higher than 20 in the depth map of Figure 2a-1 part.
- Figure 2b-2 shows that according to Figure 2b-1, the block is divided into two layers, and a new layer “1 layer” is generated on the basis of "0 layer".
- Figure 2c-1 shows that the depth map is reconstructed from two average values. Among them, the dark gray part 205 in Fig. 2b-1 is taken as the depth map average value "10"; the light gray part 206 in Fig. 2b-2 is taken as the depth map average value "25".
- Figure 2c-2 shows that the level list after one iteration is divided into “level 1” and "level 0", where the part with the value “10” is regarded as “level 0"; the part with the value “25” is regarded as “level 1” .
- Figure 2d-1 shows that the second iteration is performed: the "layer 1" is split into two layers. Among them, the original depth value of the first layer is restored, and the original depth value of Figure 2a-1 is divided into two parts less than or equal to "25" in the "1 layer” part and two parts equal to "30". That is, the white part 207 and the darker gray part 208 in Fig. 2d-1.
- Figure 2d-2 shows that the part of the depth value equal to "30" is regarded as a new layer "2 layer”.
- Figure 2e-1 shows that the depth map is reconstructed from three average values.
- the dark gray part 205 in Figure 2b-1 the part with the average depth map "10" remains unchanged; the white part 207 in Figure 2d-1 with the depth value less than or equal to "25” is retaken
- the average value is "22"; the part with the depth value of "30" in the darker gray part 208 in Figure 2d-1 is kept unchanged; the average value is no longer taken.
- Figure 2e-2 shows the level list after the second iteration: divided into “2 levels", “1 level” and “0 level”, where the depth value is “10” as “0 level”; the depth value is "22”
- the part with “” is regarded as “1 layer”; the part with a depth value of "30” is regarded as “2 layer”.
- the encoder By completing the previous stage, the encoder now has a layer list of one or more layers, each layer has an average depth value and contains some pixels in the depth block. Further, the data encapsulation is completed through the following two sub-steps; in order to output the data to the decoder side:
- the encoder calculates the mean and variance of the depth values of all pixels in the layer for all layers in the layer list. Append each average to the corresponding layer. The encoder then reorders the layers by sorting the area or number of pixels of each layer in descending order, so that the layer with the most pixels is processed first.
- Binary mapping step the encoder uses the layer that has not been coded and has the largest number of pixels to form a binary mapping, marking all pixels in the layer as "1", marking other pixels as "0", and using context adaptation Binary arithmetic coding method to encode this binary map. After completing the mapping, the encoder continues to the next largest layer and repeats until one layer remains. The last layer does not require binary mapping because it will automatically fill in all remaining pixels.
- the decoder will receive the final output data.
- the final output data is composed of three parts: an integer representing the number of layers in the depth block in step B. Progressive quantization and its stopping conditions. A series of bits containing the binary mapping of all layers from step C. data encapsulation; and a series of integers corresponding to the depth value of each layer in step B. progressive quantization and its stopping conditions.
- the various component embodiments of the present invention may be implemented by hardware, or by software modules running on one or more processors, or by their combination.
- a microprocessor or a digital signal processor (DSP) can be used in practice to implement the method for improving video resolution and quality and the video encoder and the decoder of the display terminal according to the embodiments of the present invention.
- DSP digital signal processor
- the present invention can also be implemented as a device or device program (for example, a computer program and a computer program product) for executing part or all of the methods described herein.
- Such a program for realizing the present invention may be stored on a computer-readable medium, or may have the form of one or more signals. Such signals can be downloaded from Internet websites, or provided on carrier signals, or provided in any other form.
- Figure 3 shows a server, such as an application server, that can implement the invention.
- the server traditionally includes a processor 1010 and a computer program product in the form of a memory 1020 or a computer readable medium.
- the memory 1020 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM.
- the memory 1020 has a storage space 1030 for executing the program code 1031 of any method step in the above method.
- the storage space 1030 for program codes may include various program codes 1031 for implementing various steps in the above method. These program codes can be read from or written into one or more computer program products.
- These computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards or floppy disks. Such computer program products are usually portable or fixed storage units as described with reference to FIG. 4.
- the storage unit may have storage segments, storage spaces, etc. arranged similarly to the storage 1020 in the server of FIG. 3.
- the program code can be compressed in an appropriate form, for example.
- the storage unit includes computer-readable codes 1031', that is, codes that can be read by, for example, a processor such as 1010, which, when run by a server, causes the server to perform the steps in the method described above.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The present invention relates to a system and method for processing a depth map, and in particular, to a system and method for multi-layer representation of a depth map during intra-frame coding. In the present invention, depth map data is blocked, progressive quantification and stopping conditions are set for the blocked depth map data, data encapsulation is performed on a multi-layer depth map for which iteration is stopped, the bit stream obtained after the encapsulation is outputted to a decoder, the pixels in the same smooth region are reduced to simple representative values, and resources are invested in recorded pixel grouping to achieve the smoothness of the depth map. The quality target can make the reconstruction and output of the method better serve the overall quality and bit rate control of a video encoder. One direct application of the present invention is 3D video content compression for online video broadcasting. Another application is 3D video format conversion.
Description
本发明涉及对深度图进行处理的***和方法,特别是涉及用于帧内编码深度图多层表示的***和方法。The present invention relates to a system and method for processing a depth map, and more particularly to a system and method for multi-layer representation of an intra-frame coding depth map.
随着对图像传输要求的不断提高,为了获得图像和视频的高效存储和传输,需要对图像和视频的有效压缩方法。深度图是记录可观察场景点距摄像机光点的距离的图像或视频。它通过在场景中指定其深度,为彩色图像或在相同位置拍摄的视频中的相关颜色像素提供附加信息。因此,深度图是3D多媒体体验的关键组成部分。显示设备具有从中恢复具有深度图的图像或视频中描绘的场景的3D结构信息。深度图的压缩和编码是减少深度图数据量的过程,并为网络的不同终端的设备提供标准以理解传输的数据。深度图编码也可以被视为整个3D视频数据的压缩过程的一部分。With the continuous improvement of image transmission requirements, in order to obtain efficient storage and transmission of images and videos, effective compression methods for images and videos are required. The depth map is an image or video that records the distance between the observable scene point and the camera light point. It provides additional information for related color pixels in color images or videos shot at the same location by specifying its depth in the scene. Therefore, the depth map is a key component of the 3D multimedia experience. The display device has 3D structure information from which to recover the scene depicted in the image or video with the depth map. The compression and encoding of the depth map is a process to reduce the amount of depth map data and provide standards for different terminal devices of the network to understand the transmitted data. Depth map coding can also be regarded as a part of the entire 3D video data compression process.
最常见的深度图编码或一般视频编码方案遵循相同的“混合”视频编码框架。帧被分成块,利用相邻块和帧之间的空间依赖性。数据由先前编码的块和帧预测并编码。帧内编码是深度图编码的基础步骤。它是编码过程的一部分,其中数据仅由来自同一帧的先前数据预测。帧内编码方法仅相对于当前帧内包括的信息执行操作,而相对于视频序列中的任何其他帧中包括的信息不执行操作。The most common depth map coding or general video coding schemes follow the same "hybrid" video coding framework. The frame is divided into blocks, taking advantage of the spatial dependence between adjacent blocks and frames. The data is predicted and coded from previously coded blocks and frames. Intra coding is the basic step of depth map coding. It is part of the encoding process where data is only predicted from previous data from the same frame. The intra-frame coding method only performs operations with respect to the information included in the current frame, and does not perform operations with respect to the information included in any other frames in the video sequence.
现有技术的最先进标准3D高效率视频编码(“3D-High Efficiency Video Coding”)采用下列帧内编码方法,包括:1.直流(DC)和平面预测;2.基于楔形(“wedgelet”)的深度建模;3.轮廓段深度建模。通过方向预测仅能够处理平滑数据。基于楔形的深度建模和轮廓段深度建模可以处理急剧的变化,但是帧所分成的块中仅有两层。此外,轮廓线段深度建模经常从相应的纹理视频中获得急剧的变化。这些方法的局限性在于分割仅执行一次。因此,如果分割质量差,则必须依赖其他编码方法来提高重建质量。The most advanced standard 3D high-efficiency video coding ("3D-High Efficiency Video Coding") in the prior art adopts the following intra-frame coding methods, including: 1. Direct current (DC) and planar prediction; 2. Based on wedgelet ("wedgelet") The depth modeling; 3. The depth modeling of the contour segment. Only smooth data can be processed by direction prediction. Wedge-based depth modeling and contour segment depth modeling can handle drastic changes, but there are only two layers in the blocks that the frame is divided into. In addition, the depth modeling of contour line segments often obtains sharp changes from the corresponding texture video. The limitation of these methods is that the segmentation is performed only once. Therefore, if the segmentation quality is poor, you must rely on other coding methods to improve the reconstruction quality.
因此现有技术中需要提高压缩效率并更好地提高重建质量的压缩深度图的***和方法。Therefore, in the prior art, there is a need for a system and method for compressing a depth map that improves compression efficiency and better reconstruction quality.
发明内容Summary of the invention
本发明自适应地量化图像并将图像分成恒定值的层的方式,提供了一种在相对复杂的场景中压缩深度图数据的***和方法。The method of the present invention to adaptively quantize the image and divide the image into constant-valued layers provides a system and method for compressing depth map data in a relatively complex scene.
本发明提供一种帧内编码深度图多层表示的***,包括下列装置:分块模块,将深度图数据分块;渐进量化模块,对分块的深度图数据设置渐进量化及其停止条件。The present invention provides a system for multi-layer representation of an intra-frame coded depth map, including the following devices: a block module to block the depth map data; a progressive quantization module to set progressive quantization and stop conditions for the block depth map data.
本发明的一个方面,还包括如下装置:数据封装模块,对停止迭代的多层深度图进行数据封装;数据输出模块,输出封装后的比特流至解码器端。One aspect of the present invention also includes the following devices: a data encapsulation module, which performs data encapsulation on the multi-layer depth map that stops iteration; and a data output module, which outputs the encapsulated bit stream to the decoder end.
本发明的一个方面,其中:对分块的深度图数据设置渐进量化及其停止条件的渐进量化模块还包括:分层模块,将深度块被分解为多个层,每个层包含深度块中与其他层互斥的像素子集;多层表示模块,以 非参数方式表示所述多个层,其中一个层可以表示深度块中所有像素的任何子集;迭代模块,采用迭代方法对所述多个层进行编码,并持续监控重建残差和优先级高的剩余区域。In one aspect of the present invention, the progressive quantization module for setting progressive quantization and its stopping conditions for the divided depth map data further includes: a layering module, which decomposes the depth block into multiple layers, and each layer contains the depth block A subset of pixels that are mutually exclusive with other layers; a multi-layer representation module that represents the multiple layers in a non-parametric way, one of which can represent any subset of all pixels in the depth block; an iterative module, which uses an iterative method to Encode multiple layers and continuously monitor reconstruction residuals and remaining areas with high priority.
本发明的一个方面,对分块的深度图数据设置渐进量化及其停止条件的渐进量化模块包括:新图层创建模块,利用编码器初始化一个空的图层列表,在所述图层列表中创建一个新图层,然后将块中的所有像素分类为唯一的图层;编码器重复执行下列模块直到迭代过程结束:计算模块,对于图层列表中的每个图层,利用加密器计算图层内所有像素的深度值的均值和方差,并将每个平均值都附加到相应的图层;深度值重建模块,编码器找到计算出的所有方差中的最大值,识别具有最大方差的层,将其调用为最大方差层;并通过使用每个像素所属层的平均值为每个像素分配深度值来创建重建块;计算重建块与原始深度块之间的平方误差之和;新图层创建模块,利用编码器在图层列表的尾部创建一个新图层,选取最大方差层内并且深度值大于最大方差层平均值的所有像素,从最大方差层中删除这些像素并将它们分配给新图层;在所述平方误差之和小于预设阈值时,结束迭代过程。In one aspect of the present invention, the progressive quantization module for setting progressive quantization and its stopping conditions for the divided depth map data includes: a new layer creation module, which initializes an empty layer list with an encoder, in the layer list Create a new layer, and then classify all pixels in the block as a unique layer; the encoder repeats the following modules until the end of the iteration process: calculation module, for each layer in the layer list, use the encryptor to calculate the graph The average value and variance of the depth value of all pixels in the layer, and each average value is attached to the corresponding layer; the depth value reconstruction module, the encoder finds the maximum value of all the calculated variances, and identifies the layer with the largest variance , Call it as the maximum variance layer; and create a reconstructed block by assigning a depth value to each pixel using the average value of the layer to which each pixel belongs; calculate the sum of squared errors between the reconstructed block and the original depth block; new layer The creation module uses the encoder to create a new layer at the end of the layer list, selects all pixels in the maximum variance layer and whose depth value is greater than the average value of the maximum variance layer, deletes these pixels from the maximum variance layer and assigns them to the new Layer; when the sum of the square errors is less than the preset threshold, the iterative process ends.
本发明的另一个方面,其中,所述预定阈值为在平方误差之和方面所需的重建质量。In another aspect of the present invention, the predetermined threshold is the required reconstruction quality in terms of the sum of squared errors.
本发明的另一个方面,其中所述数据封装模块,还包括:平均值附加模块,利用编码器为其图层列表中的所有图层计算图层内所有像素的深度值的均值和方差;将每个平均值都附加到相应的图层;图层处理模块,编码器通过按降序对每个层的区域或像素数进行排序来重新排序图 层,从而首先处理具有最多像素的图层;二进制映射模块,编码器利用尚未编码且具有最大像素数的层,形成二进制映射,将该层中的所有像素标记为“1”,将其他像素标记为“0”,并使用上下文自适应二进制算术编码方法来编码这个二进制映射;完成该映射后,编码器继续进行下一个最大层并重复直到剩下一个层;最后一层不需要二进制映射,因为其会自动填充所有剩余像素。In another aspect of the present invention, the data encapsulation module further includes: an average value addition module, which uses an encoder to calculate the average value and variance of the depth values of all pixels in the layer for all layers in the layer list; Each average value is attached to the corresponding layer; the layer processing module, the encoder reorders the layers by sorting the area or the number of pixels of each layer in descending order, so that the layer with the most pixels is processed first; binary Mapping module, the encoder uses the layer that has not been encoded and has the largest number of pixels to form a binary map, mark all pixels in the layer as "1", mark other pixels as "0", and use context adaptive binary arithmetic coding Method to encode this binary mapping; after completing the mapping, the encoder continues to the next largest layer and repeats until one layer is left; the last layer does not require binary mapping because it will automatically fill in all remaining pixels.
本发明的另一个方面,其中数据封装模块中的最终输出数据由下列内容组成:表示深度块中的层数的整数;包含所有层的二进制映射的一系列位;体现每层的深度值的一系列整数。In another aspect of the present invention, the final output data in the data encapsulation module is composed of the following content: an integer representing the number of layers in the depth block; a series of bits containing the binary mapping of all layers; a value representing the depth value of each layer Series integer.
本发明还提供一种帧内编码深度图多层表示的方法,包括如下步骤:将深度图数据分块;对分块的深度图数据,设置渐进量化及其停止条件。The present invention also provides a method for multi-layer representation of an intra-frame coding depth map, which includes the following steps: dividing the depth map data into blocks; setting the progressive quantization and stopping conditions for the divided depth map data.
本发明还提供一种实现本发明所述方法的编码器,对分块的深度图数据设置渐进量化及其停止条件的,其包括:新图层创建模块,利用编码器初始化一个空的图层列表,在所述图层列表中创建一个新图层,然后将块中的所有像素分类为唯一的图层;所述编码器重复执行下列模块直到迭代过程结束:计算模块,对于图层列表中的每个图层,利用加密器计算图层内所有像素的深度值的均值和方差,并将每个平均值都附加到相应的图层;深度值重建模块,编码器找到计算出的所有方差中的最大值,识别具有最大方差的层,将其调用为最大方差层;并通过使用每个像素所属层的平均值为每个像素分配深度值来创建重建块;计算重建块与原始深度块之间的平方误差之和;新图层创建模块,利用编码器在 图层列表的尾部创建一个新图层,选取最大方差层内并且深度值大于最大方差层平均值的所有像素,从最大方差层中删除这些像素并将它们分配给新图层;在所述平方误差之和小于预设阈值时,结束迭代过程。The present invention also provides an encoder for implementing the method of the present invention. The stepwise quantization and stopping conditions are set for the block depth map data, which includes: a new layer creation module, which uses the encoder to initialize an empty layer List, create a new layer in the layer list, and then classify all pixels in the block as a unique layer; the encoder repeats the following modules until the end of the iteration process: calculation module, for the layer list For each layer of, use the encryptor to calculate the mean value and variance of the depth value of all pixels in the layer, and attach each mean value to the corresponding layer; depth value reconstruction module, the encoder finds all the calculated variances The maximum value in, identify the layer with the largest variance, call it the largest variance layer; and create a reconstruction block by assigning a depth value to each pixel using the average value of the layer to which each pixel belongs; calculate the reconstruction block and the original depth block The sum of squared errors between; the new layer creation module uses the encoder to create a new layer at the end of the layer list, selects all pixels in the maximum variance layer and whose depth value is greater than the average value of the maximum variance layer, starting from the maximum variance Delete these pixels from the layer and assign them to a new layer; when the sum of the square errors is less than a preset threshold, the iterative process ends.
本发明中描述的方法,以模仿深度数据的性质的方式处理深度图数据。深度图通常包含大而平滑的区域,两者之间有明显清晰的边界。本发明通过将相同平滑区域内的像素减少为简单的代表值,同时在记录像素分组中投入资源,来实现深度图的平滑度。质量目标可以使该方法的重建输出更好地服务于视频编码器的整体质量和比特率控制。本发明的一个直接应用是用于在线视频广播的3D视频内容压缩。另一个应用是3D视频格式转换。The method described in the present invention processes depth map data in a manner that mimics the properties of depth data. Depth maps usually contain large and smooth areas with a clear boundary between the two. The present invention realizes the smoothness of the depth map by reducing the pixels in the same smooth area to simple representative values and at the same time investing resources in the recording pixel grouping. The quality target can make the reconstruction output of this method better serve the overall quality and bit rate control of the video encoder. One immediate application of the present invention is 3D video content compression for online video broadcasting. Another application is 3D video format conversion.
为了更清楚地说明本发明实施例中的技术方案,下面将对实施例中所需要使用的附图作简单地介绍。显而易见地,下面描述中的附图仅仅是本发明的一些实例,对于本领域普通技术人员来讲,在不付出创新性劳动的前提下,还可以根据这些附图获得其他的附图。In order to more clearly describe the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings in the following description are only some examples of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.
图1为本发明所述的用于帧内编码深度图多层表示的方法示意图。Fig. 1 is a schematic diagram of a method for multi-layer representation of an intra-frame coded depth map according to the present invention.
图2a-图2e为本发明所述的用于帧内编码深度图多层表示的方法中设置渐进量化及其停止条件步骤的示例。Figures 2a-2e are examples of the steps of setting progressive quantization and its stopping conditions in the method for multi-layer representation of intra-coded depth maps according to the present invention.
图3示意性地示出了用于执行根据本发明的方法的服务器的框图;以及Fig. 3 schematically shows a block diagram of a server for executing the method according to the present invention; and
图4示意性地示出了用于保持或者携带实现根据本发明的方法的程序代码的存储单元。Fig. 4 schematically shows a storage unit for holding or carrying program codes for implementing the method according to the present invention.
以下阐述的是当前被认为是所要求保护的发明的优选实施例或最佳表示性示例的内容。仔细考虑了对实施例和优选实施例的将来和现在的表示或修改,在功能、目的、结构或结果方面作出实质性改变的任何变更或修改,都旨在被本专利的权利要求所涵盖。现在将参考附图仅以举例的方式描述本发明的优选实施例。What is set forth below is what is currently considered to be the preferred embodiment or best representative example of the claimed invention. After careful consideration of the future and present representations or modifications to the embodiments and preferred embodiments, any changes or modifications that make substantial changes in function, purpose, structure or results are intended to be covered by the claims of this patent. The preferred embodiments of the present invention will now be described by way of example only with reference to the accompanying drawings.
本发明旨在提供一种处理复杂深度图数据的有效方法。根据本发明的方法,本发明的深度图实际上被分解为几个“部分”(shape),将这几个部分结合起来为所要处理的深度图块中的所有像素,并且这些形状中的每一个附加有深度值以用作解码结果。进一步,该方法通过迭代的方式,通过使用基于方差导出的标准,来识别具有差的重建质量的区域从而重复地改进重建质量。The present invention aims to provide an effective method for processing complex depth map data. According to the method of the present invention, the depth map of the present invention is actually decomposed into several "shapes", which are combined into all the pixels in the depth block to be processed, and each of these shapes One is appended with a depth value for use as a decoding result. Further, the method uses the standard derived based on variance to identify regions with poor reconstruction quality in an iterative manner to repeatedly improve the reconstruction quality.
图1为本发明所述的用于帧内编码深度图多层表示的方法示意图。在步骤101,深度块被分解为多个层,每个层包含深度块中与其他层互斥的像素子集,参见下文步骤B.渐进量化及其停止条件的具体内容。在步骤102,所述多个层以非参数方式表示,每个层可以表示深度块中所有像素的任何子集。参见下文步骤B.渐进量化及其停止条件的子步骤d.的具体内容。它适应复杂的环境,可以任意分配场景中的深度值。在步骤103,编码过程采用迭代方法。参见下文步骤B.渐进量化及其停止条件的子步骤c。持续监控重建残差(即原始和解码深度图之间的差异的度量)和优先级高的剩余区域。该步骤103与现有技术中的轮廓线段深度建模有所不同,轮廓线段深度建模仅执行一次分割。Fig. 1 is a schematic diagram of a method for multi-layer representation of an intra-frame coded depth map according to the present invention. In step 101, the depth block is decomposed into multiple layers, and each layer contains a subset of pixels in the depth block that are mutually exclusive with other layers. Refer to step B. Progressive quantization and its stopping conditions below for specific content. In step 102, the multiple layers are represented in a non-parametric manner, and each layer can represent any subset of all pixels in the depth block. Refer to the detailed content of step B. Progressive quantification and sub-step d. of its stopping conditions below. It adapts to complex environments and can assign depth values in the scene arbitrarily. In step 103, the encoding process adopts an iterative method. See step B. Progressive quantification and sub-step c of its stopping conditions below. Continuously monitor the reconstruction residuals (a measure of the difference between the original and decoded depth maps) and the remaining areas with high priority. This step 103 is different from the contour line segment depth modeling in the prior art, and the contour line segment depth modeling only performs one segmentation.
进一步详细描述本发明的方法,所述方法包括如下步骤:The method of the present invention is further described in detail, and the method includes the following steps:
A.将深度图数据分块;A. Divide the depth map data into blocks;
B.对分块的深度图数据,设置渐进量化及其停止条件;B. Set the progressive quantization and its stopping conditions for the divided depth map data;
C.对停止迭代的多层深度图进行数据封装;C. Data encapsulation of the multi-layer depth map that stops iteration;
D.输出封装后的比特流。其中,D. Output the encapsulated bit stream. among them,
步骤A.将深度数据分块Step A. Divide the depth data into blocks
在该步骤中以深度图中的“块”开始,将图片划分为更小的单元。块宽和高度通常为2的幂。例如,2×2,4×4,8×8……In this step, starting with "blocks" in the depth map, the picture is divided into smaller units. The block width and height are usually powers of 2. For example, 2×2, 4×4, 8×8...
步骤B.设置渐进量化及其停止条件Step B. Set progressive quantization and its stopping conditions
设置渐进量化步骤及其停止条件的步骤包括多个阶段。其中,编码器首先初始化一个空的“图层列表”,在列表中创建一个新图层,然后将块中的所有像素分类为唯一的图层。在初始化之后,编码器重复以下过程,直到过程中设定的停止条件得到满足。The step of setting the progressive quantization step and its stopping condition includes multiple stages. Among them, the encoder first initializes an empty "layer list", creates a new layer in the list, and then classifies all pixels in the block as a unique layer. After initialization, the encoder repeats the following process until the stop condition set during the process is met.
a.对于图层列表中的每个图层,加密器计算图层内所有像素的深度值的均值和方差。每个平均值都附加到相应的图层。a. For each layer in the layer list, the encryptor calculates the mean and variance of the depth values of all pixels in the layer. Each average value is attached to the corresponding layer.
b.编码器通过使用每个像素所属层的平均值为每个像素分配深度值来创建“重建块”。计算重建块与原始深度块之间的平方误差之和(SSE)。如果该SSE小于预设阈值,例如所述预定阈值为在SSE方面所需的重建质量,那么这个迭代过程结束。b. The encoder creates a "reconstruction block" by assigning a depth value to each pixel using the average value of the layer to which each pixel belongs. Calculate the sum of squared errors (SSE) between the reconstructed block and the original depth block. If the SSE is less than a preset threshold, for example, the predetermined threshold is the required reconstruction quality in terms of SSE, then this iterative process ends.
c.编码器找到步骤a中计算的所有方差中的最大值,识别具有最大方差的层,将其调用为最大方差层(“LLV”)。c. The encoder finds the maximum value among all the variances calculated in step a, identifies the layer with the largest variance, and calls it the maximum variance layer ("LLV").
d.编码器在图层列表的尾部创建一个新图层,选取最大方差层内并且深度值大于最大方差层平均值的所有像素,从最大方差层中删除这些像素 并将它们分配给新图层。回到步骤a。d. The encoder creates a new layer at the end of the layer list, selects all pixels in the maximum variance layer and whose depth value is greater than the average value of the maximum variance layer, deletes these pixels from the maximum variance layer and assigns them to the new layer . Go back to step a.
图2a-图2e为本发明所述的用于帧内编码深度图多层表示的方法中设置渐进量化及其停止条件步骤的一个示例。Figures 2a-2e are an example of the step of setting progressive quantization and its stopping conditions in the method for multi-layer representation of an intra-coded depth map according to the present invention.
其中图2a-1为一个深度图块的示例,其用不同灰色阴影着色,表示不同的值。其中,白色部分201表示深度值为“30”的部分;次浅灰色部分202表示深度值为“25”的部分;较深灰色部分203表示深度值为“20”和“12”的部分;最深灰色部分204表示深度值为“10”的部分。Figure 2a-1 is an example of a depth block, which is colored with different shades of gray to indicate different values. Among them, the white part 201 represents the part with the depth value of "30"; the second light gray part 202 represents the part with the depth value of "25"; the darker gray part 203 represents the part with the depth value of "20" and "12"; The dark gray portion 204 represents a portion with a depth value of "10".
图2a-2表示,对于初始层列表,所有的像素都位于“0层”。Figure 2a-2 shows that for the initial layer list, all pixels are located in the "0 layer".
图2b-1表示,经过第一次迭代,将整个块分成两层。其中在图2b-1中的深灰色部分205为图2a-1深度图中值低于15的部分;在图2b-1中的浅灰色部分206为图2a-1深度图中值高于20的部分。Figure 2b-1 shows that after the first iteration, the entire block is divided into two layers. The dark gray part 205 in Figure 2b-1 is the part whose value is lower than 15 in the depth map of Figure 2a-1; the light gray part 206 in Figure 2b-1 is the value higher than 20 in the depth map of Figure 2a-1 part.
图2b-2表示,据图2b-1对块划分成两层,在“0层”的基础上产生新的层“1层”。Figure 2b-2 shows that according to Figure 2b-1, the block is divided into two layers, and a new layer "1 layer" is generated on the basis of "0 layer".
图2c-1表示,从两个平均值重建深度图。其中,将图2b-1中的深灰色部分205中取深度图平均值“10”;将图2b-2中的浅灰色部分206中取深度图平均值“25”。Figure 2c-1 shows that the depth map is reconstructed from two average values. Among them, the dark gray part 205 in Fig. 2b-1 is taken as the depth map average value "10"; the light gray part 206 in Fig. 2b-2 is taken as the depth map average value "25".
图2c-2表示,一次迭代后的层次列表,分成“1层”和“0层”,其中数值为“10”的部分作为“0层”;数值为“25”的部分作为“1层”。Figure 2c-2 shows that the level list after one iteration is divided into "level 1" and "level 0", where the part with the value "10" is regarded as "level 0"; the part with the value "25" is regarded as "level 1" .
图2d-1表示,进行第二次迭代:将“第1层”拆分为两层。其中,恢复第一层的原始深度值,将图2a-1的原始深度值,在“1层”的部分中分为小于或等于“25”的一部分与等于“30”的两部分。即图2d-1中的白色部分207和较深灰色部分208。Figure 2d-1 shows that the second iteration is performed: the "layer 1" is split into two layers. Among them, the original depth value of the first layer is restored, and the original depth value of Figure 2a-1 is divided into two parts less than or equal to "25" in the "1 layer" part and two parts equal to "30". That is, the white part 207 and the darker gray part 208 in Fig. 2d-1.
图2d-2表示,将深度值等于“30”的一部分作为新的一层“2层”。Figure 2d-2 shows that the part of the depth value equal to "30" is regarded as a new layer "2 layer".
图2e-1表示,从三个平均值重建深度图。将图2b-1中的深灰色部分205中取深度图平均值“10”的部分仍然保持不变;将图2d-1中的白色部分207中深度值小于或等于“25”的部分重新取平均值“22”;将图2d-1中的较深灰色部分208中深度值等于“30”的部分保持不变;不再取平均值。Figure 2e-1 shows that the depth map is reconstructed from three average values. In the dark gray part 205 in Figure 2b-1, the part with the average depth map "10" remains unchanged; the white part 207 in Figure 2d-1 with the depth value less than or equal to "25" is retaken The average value is "22"; the part with the depth value of "30" in the darker gray part 208 in Figure 2d-1 is kept unchanged; the average value is no longer taken.
图2e-2表示,二次迭代后的层次列表:分成“2层”、“1层”和“0层”,其中深度值为“10”的部分作为“0层”;深度值为“22”的部分作为“1层”;深度值为“30”的部分作为“2层”。Figure 2e-2 shows the level list after the second iteration: divided into "2 levels", "1 level" and "0 level", where the depth value is "10" as "0 level"; the depth value is "22" The part with "" is regarded as "1 layer"; the part with a depth value of "30" is regarded as "2 layer".
步骤C.数据封装Step C. Data encapsulation
通过完成前一阶段,编码器现在具有一个或多个层的层列表,每个层具有平均深度值并且包含深度块中的一些像素。进一步,通过下列两个子步骤,完成数据封装;以便将数据输出至解码器端:By completing the previous stage, the encoder now has a layer list of one or more layers, each layer has an average depth value and contains some pixels in the depth block. Further, the data encapsulation is completed through the following two sub-steps; in order to output the data to the decoder side:
a.计算步骤:编码器为其图层列表中的所有图层计算图层内所有像素的深度值的均值和方差。将每个平均值都附加到相应的图层。然后编码器通过按降序对每个层的区域或像素数进行排序来重新排序图层,从而首先处理具有最多像素的图层。a. Calculation steps: The encoder calculates the mean and variance of the depth values of all pixels in the layer for all layers in the layer list. Append each average to the corresponding layer. The encoder then reorders the layers by sorting the area or number of pixels of each layer in descending order, so that the layer with the most pixels is processed first.
b.二进制映射步骤:编码器利用尚未编码且具有最大像素数的层,形成二进制映射,将该层中的所有像素标记为“1”,将其他像素标记为“0”,并使用上下文自适应二进制算术编码方法来编码这个二进制映射。完成该映射后,编码器继续进行下一个最大层并重复直到剩下一个层。最后一层不需要二进制映射,因为它会自动填充所有剩余像素。b. Binary mapping step: the encoder uses the layer that has not been coded and has the largest number of pixels to form a binary mapping, marking all pixels in the layer as "1", marking other pixels as "0", and using context adaptation Binary arithmetic coding method to encode this binary map. After completing the mapping, the encoder continues to the next largest layer and repeats until one layer remains. The last layer does not require binary mapping because it will automatically fill in all remaining pixels.
步骤D.比特流输出Step D. Bitstream output
在数据封装步骤完成之后,解码器将会接收最终输出数据,最终输出数据是由三部分组成:整数,表示步骤B.渐进量化及其停止条件中深度块中的层数。一系列位,包含来自步骤C.数据封装的所有层的二进制映射;以及一系列整数,对应于步骤B.渐进量化及其停止条件中每层的深度值。After the data encapsulation step is completed, the decoder will receive the final output data. The final output data is composed of three parts: an integer representing the number of layers in the depth block in step B. Progressive quantization and its stopping conditions. A series of bits containing the binary mapping of all layers from step C. data encapsulation; and a series of integers corresponding to the depth value of each layer in step B. progressive quantization and its stopping conditions.
本发明的各个部件实施例可以以硬件实现,或者以在一个或者多个处理器上运行的软件模块实现,或者以它们的组合实现。本领域的技术人员应当理解,可以在实践中使用微处理器或者数字信号处理器(DSP)来实现根据本发明实施例的提升视频分辨率和质量的方法以及视频编码器和显示终端的解码器的一些或者全部部件的一些或者全部功能。本发明还可以实现为用于执行这里所描述的方法的一部分或者全部的设备或者装置程序(例如,计算机程序和计算机程序产品)。这样的实现本发明的程序可以存储在计算机可读介质上,或者可以具有一个或者多个信号的形式。这样的信号可以从因特网网站上下载得到,或者在载体信号上提供,或者以任何其他形式提供。The various component embodiments of the present invention may be implemented by hardware, or by software modules running on one or more processors, or by their combination. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) can be used in practice to implement the method for improving video resolution and quality and the video encoder and the decoder of the display terminal according to the embodiments of the present invention. Some or all of the functions of some or all of the components. The present invention can also be implemented as a device or device program (for example, a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for realizing the present invention may be stored on a computer-readable medium, or may have the form of one or more signals. Such signals can be downloaded from Internet websites, or provided on carrier signals, or provided in any other form.
例如,图3示出了可以实现根据本发明的服务器,例如应用服务器。该服务器传统上包括处理器1010和以存储器1020形式的计算机程序产品或者计算机可读介质。存储器1020可以是诸如闪存、EEPROM(电可擦除可编程只读存储器)、EPROM、硬盘或者ROM之类的电子存储器。存储器1020具有用于执行上述方法中的任何方法步骤的程序代码1031的存储空间1030。例如,用于程序代码的存储空间1030可以包括分别用于实现上面的方法中的各种步骤的各个程序代码1031。这些程序代码可以从一个或者多个计算机程序产品中读出或者写入到这一个或者多个计算机程序 产品中。这些计算机程序产品包括诸如硬盘,紧致盘(CD)、存储卡或者软盘之类的程序代码载体。这样的计算机程序产品通常为如参考图4所述的便携式或者固定存储单元。该存储单元可以具有与图3的服务器中的存储器1020类似布置的存储段、存储空间等。程序代码可以例如以适当形式进行压缩。通常,存储单元包括计算机可读代码1031’,即可以由例如诸如1010之类的处理器读取的代码,这些代码当由服务器运行时,导致该服务器执行上面所描述的方法中的各个步骤。For example, Figure 3 shows a server, such as an application server, that can implement the invention. The server traditionally includes a processor 1010 and a computer program product in the form of a memory 1020 or a computer readable medium. The memory 1020 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM. The memory 1020 has a storage space 1030 for executing the program code 1031 of any method step in the above method. For example, the storage space 1030 for program codes may include various program codes 1031 for implementing various steps in the above method. These program codes can be read from or written into one or more computer program products. These computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards or floppy disks. Such computer program products are usually portable or fixed storage units as described with reference to FIG. 4. The storage unit may have storage segments, storage spaces, etc. arranged similarly to the storage 1020 in the server of FIG. 3. The program code can be compressed in an appropriate form, for example. Generally, the storage unit includes computer-readable codes 1031', that is, codes that can be read by, for example, a processor such as 1010, which, when run by a server, causes the server to perform the steps in the method described above.
本文中所称的“一个实施例”、“实施例”或者“一个或者多个实施例”意味着,结合实施例描述的特定特征、结构或者特性包括在本发明的至少一个实施例中。此外,请注意,这里“在一个实施例中”的词语例子不一定全指同一个实施例。The term "one embodiment", "an embodiment" or "one or more embodiments" referred to herein means that a specific feature, structure or characteristic described in conjunction with the embodiment is included in at least one embodiment of the present invention. In addition, please note that the word examples "in one embodiment" herein do not necessarily all refer to the same embodiment.
以上描述并非旨在限制在限定本发明的以下权利要求书中使用的词语的含义或范围。而是提供了描述和说明以帮助理解各种实施例。预期未来在结构、功能或结果方面的修改将存在而并非实质性改变,并且权利要求书中的所有这些非实质性改变都旨在被权利要求所涵盖。因此,尽管已经说明和描述了本发明的优选实施例,但本领域技术人员将会理解,可以在不脱离要求保护的本发明的情况下做出许多改变和修改。另外,虽然术语“要求保护的发明”或“本发明”在本文中有时以单数形式使用,但将理解,存在如所描述和要求保护的多个发明。The above description is not intended to limit the meaning or scope of the words used in the following claims that define the present invention. Rather, descriptions and instructions are provided to help understand the various embodiments. It is expected that future changes in structure, function, or results will exist without substantial changes, and all these insubstantial changes in the claims are intended to be covered by the claims. Therefore, although the preferred embodiments of the present invention have been illustrated and described, those skilled in the art will understand that many changes and modifications can be made without departing from the claimed invention. In addition, although the term "claimed invention" or "present invention" is sometimes used herein in the singular, it will be understood that there are multiple inventions as described and claimed.
Claims (16)
- 一种帧内编码深度图多层表示的方法,包括如下步骤:A method for multi-layer representation of an intra-frame coded depth map includes the following steps:将深度图数据分块;Divide the depth map data into blocks;对分块的深度图数据,设置渐进量化及其停止条件。Set the progressive quantization and its stopping conditions for the divided depth map data.
- 如权利要求1所述的方法,还包括如下步骤:The method according to claim 1, further comprising the following steps:对停止迭代的多层深度图进行数据封装;Data encapsulation of multi-layer depth maps that stop iteration;输出封装后的比特流至解码器端。The encapsulated bit stream is output to the decoder side.
- 如权利要求1-2所述的方法,其中:The method according to claim 1-2, wherein:对分块的深度图数据,设置渐进量化及其停止条件的步骤包括:For the block depth map data, the steps to set the progressive quantization and its stopping conditions include:将深度块被分解为多个层,每个层包含深度块中与其他层互斥的像素子集;Decompose the depth block into multiple layers, and each layer contains a subset of pixels in the depth block that are mutually exclusive with other layers;以非参数方式表示所述多个层,其中一个层可以表示深度块中所有像素的任何子集;Representing the multiple layers in a non-parametric manner, one of which can represent any subset of all pixels in the depth block;采用迭代方法对所述多个层进行编码,并持续监控重建残差和优先级高的剩余区域。An iterative method is used to encode the multiple layers, and the reconstruction residuals and remaining areas with high priority are continuously monitored.
- 如权利要求1-2所述的方法,对分块的深度图数据,设置渐进量化及其停止条件的步骤包括:2. The method according to claim 1-2, the step of setting the progressive quantization and its stopping conditions for the divided depth map data includes:利用编码器初始化一个空的图层列表,在所述图层列表中创建一个新图层,然后将块中的所有像素分类为唯一的图层;Use the encoder to initialize an empty layer list, create a new layer in the layer list, and then classify all pixels in the block as a unique layer;编码器重复进行下列步骤直到迭代过程结束:The encoder repeats the following steps until the end of the iteration process:对于图层列表中的每个图层,利用加密器计算图层内所有像素的深度值的均值和方差,并将每个平均值都附加到相应的图层;For each layer in the layer list, use an encryptor to calculate the mean and variance of the depth values of all pixels in the layer, and attach each average to the corresponding layer;编码器找到计算出的所有方差中的最大值,识别具有最大方差的层,将其调用为最大方差层;并通过使用每个像素所属层的平均值为每个像素分配深度值来创建重建块;计算重建块与原始深度块之间的平方误差之和;The encoder finds the largest value among all the calculated variances, identifies the layer with the largest variance, and calls it as the largest variance layer; and creates a reconstruction block by assigning a depth value to each pixel using the average value of the layer to which each pixel belongs ; Calculate the sum of squared errors between the reconstructed block and the original depth block;利用编码器在图层列表的尾部创建一个新图层,选取最大方差层内并且深度值大于最大方差层平均值的所有像素,从最大方差层中删除这些像素并将它们分配给新图层;Use the encoder to create a new layer at the end of the layer list, select all pixels in the maximum variance layer and whose depth value is greater than the average value of the maximum variance layer, delete these pixels from the maximum variance layer and assign them to the new layer;在所述平方误差之和小于预设阈值时,结束迭代过程。When the sum of the square errors is less than the preset threshold, the iterative process ends.
- 如权利要求4所述的方法,其中,所述预定阈值为在平方误差之和方面所需的重建质量。The method of claim 4, wherein the predetermined threshold is the required reconstruction quality in terms of the sum of squared errors.
- 如权利要求2所述的方法,其中所述对停止迭代的多层深度图进行数据封装的步骤,包括如下步骤:3. The method of claim 2, wherein the step of performing data encapsulation on the multi-layer depth map with stopped iteration comprises the following steps:利用编码器为其图层列表中的所有图层计算图层内所有像素的深度值的均值和方差;将每个平均值都附加到相应的图层;Use the encoder to calculate the mean and variance of the depth values of all pixels in the layer for all the layers in the layer list; attach each average to the corresponding layer;编码器通过按降序对每个层的区域或像素数进行排序来重新排序图层,从而首先处理具有最多像素的图层;The encoder reorders the layers by sorting the area or the number of pixels of each layer in descending order, so that the layer with the most pixels is processed first;编码器利用尚未编码且具有最大像素数的层,形成二进制映射,将该层中的所有像素标记为“1”,将其他像素标记为“0”,并使用上下文自适应二进制算术编码方法来编码这个二进制映射;The encoder uses the layer that has not been encoded and has the largest number of pixels to form a binary map, marks all pixels in the layer as "1" and other pixels as "0", and uses the context-adaptive binary arithmetic coding method to encode This binary mapping;完成该映射后,编码器继续进行下一个最大层并重复直到剩下一个层;After completing the mapping, the encoder continues to the next largest layer and repeats until one layer remains;最后一层不需要二进制映射,因为其会自动填充所有剩余像素。The last layer does not require binary mapping because it will automatically fill in all remaining pixels.
- 如权利要求2或6中所述的方法,其中输出封装后的比特流的步骤中的最终输出数据由下列内容组成:The method according to claim 2 or 6, wherein the final output data in the step of outputting the encapsulated bit stream consists of the following content:表示深度块中的层数的整数;An integer representing the number of layers in the depth block;包含所有层的二进制映射的一系列位;A series of bits containing the binary mapping of all layers;体现每层的深度值的一系列整数。A series of integers representing the depth value of each layer.
- 一种帧内编码深度图多层表示的***,包括下列装置:A system for multi-layer representation of an intra-frame coded depth map includes the following devices:分块模块,将深度图数据分块;Blocking module to block the depth map data;渐进量化模块,对分块的深度图数据设置渐进量化及其停止条件。The progressive quantization module sets the progressive quantization and its stopping conditions for the block depth map data.
- 如权利要求1所述的***,还包括如下装置:The system according to claim 1, further comprising the following devices:数据封装模块,对停止迭代的多层深度图进行数据封装;Data encapsulation module, which encapsulates the multi-layer depth map that stops iteration;数据输出模块,输出封装后的比特流至解码器端。The data output module outputs the encapsulated bit stream to the decoder side.
- 如权利要求8-9所述的***,其中:The system of claims 8-9, wherein:对分块的深度图数据设置渐进量化及其停止条件的渐进量化模块还包括:The progressive quantization module for setting progressive quantization and its stopping conditions for the divided depth map data also includes:分层模块,将深度块被分解为多个层,每个层包含深度块中与其他层互斥的像素子集;The layering module decomposes the depth block into multiple layers, and each layer contains a subset of pixels in the depth block that are mutually exclusive with other layers;多层表示模块,以非参数方式表示所述多个层,其中一个层可以表示深度块中所有像素的任何子集;A multi-layer representation module, which represents the multiple layers in a non-parametric manner, one of which can represent any subset of all pixels in the depth block;迭代模块,采用迭代方法对所述多个层进行编码,并持续监控重建残差和优先级高的剩余区域。The iterative module uses an iterative method to encode the multiple layers and continuously monitors reconstruction residuals and remaining areas with high priority.
- 如权利要求8-9所述的***,对分块的深度图数据设置渐进量化及其停止条件的渐进量化模块包括:The system according to claims 8-9, the progressive quantization module for setting progressive quantization and its stopping conditions for the divided depth map data includes:新图层创建模块,利用编码器初始化一个空的图层列表,在所述图层列表中创建一个新图层,然后将块中的所有像素分类为唯一的图层;The new layer creation module uses the encoder to initialize an empty layer list, creates a new layer in the layer list, and then classifies all pixels in the block as a unique layer;编码器重复执行下列模块直到迭代过程结束:The encoder repeatedly executes the following modules until the end of the iteration process:计算模块,对于图层列表中的每个图层,利用加密器计算图层内所有像素的深度值的均值和方差,并将每个平均值都附加到相应的图层;Calculation module, for each layer in the layer list, use the encryptor to calculate the mean and variance of the depth values of all pixels in the layer, and attach each average to the corresponding layer;深度值重建模块,编码器找到计算出的所有方差中的最大值,识别具有最大方差的层,将其调用为最大方差层;并通过使用每个像素所属层的平均值为每个像素分配深度值来创建重建块;计算重建块与原始深度块之间的平方误差之和;Depth value reconstruction module, the encoder finds the maximum value of all the calculated variances, identifies the layer with the largest variance, calls it as the maximum variance layer; and assigns depth to each pixel by using the average value of the layer to which each pixel belongs Value to create a reconstructed block; calculate the sum of squared errors between the reconstructed block and the original depth block;新图层创建模块,利用编码器在图层列表的尾部创建一个新图层,选取最大方差层内并且深度值大于最大方差层平均值的所有像素,从最大方差层中删除这些像素并将它们分配给新图层;New layer creation module, use the encoder to create a new layer at the end of the layer list, select all pixels in the maximum variance layer and whose depth value is greater than the average value of the maximum variance layer, delete these pixels from the maximum variance layer and remove them Assign to the new layer;在所述平方误差之和小于预设阈值时,结束迭代过程。When the sum of the square errors is less than the preset threshold, the iterative process ends.
- 如权利要求11所述的***,其中,所述预定阈值为在平方误差之和方面所需的重建质量。The system of claim 11, wherein the predetermined threshold is the required reconstruction quality in terms of the sum of squared errors.
- 如权利要求9所述的***,其中所述数据封装模块,还包括:The system according to claim 9, wherein the data encapsulation module further comprises:平均值附加模块,利用编码器为其图层列表中的所有图层计算图层内所有像素的深度值的均值和方差;将每个平均值都附加到相应的图层;The average value appending module uses the encoder to calculate the average value and variance of the depth value of all pixels in the layer for all layers in the layer list; attach each average value to the corresponding layer;图层处理模块,编码器通过按降序对每个层的区域或像素数进行排序来重新排序图层,从而首先处理具有最多像素的图层;Layer processing module, the encoder reorders the layers by sorting the area or the number of pixels of each layer in descending order, so that the layer with the most pixels is processed first;二进制映射模块,编码器利用尚未编码且具有最大像素数的层,形成二进制映射,将该层中的所有像素标记为“1”,将其他像素标记为“0”,并使用上下文自适应二进制算术编码方法来编码这个二进制映射;完成该映射后,编码器继续进行下一个最大层并重复直到剩下一个层;最后一层不需要二进制映射,因为其会自动填充所有剩余像素。Binary mapping module, the encoder uses the layer that has not been coded and has the largest number of pixels to form a binary mapping, marking all pixels in the layer as "1" and other pixels as "0", and uses context adaptive binary arithmetic The encoding method is used to encode this binary mapping; after the mapping is completed, the encoder continues to the next largest layer and repeats until one layer is left; the last layer does not need a binary mapping because it will automatically fill all remaining pixels.
- 如权利要求9或13中所述的***,其中数据封装模块中的最终输出数据由下列内容组成:The system as claimed in claim 9 or 13, wherein the final output data in the data encapsulation module consists of the following content:表示深度块中的层数的整数;An integer representing the number of layers in the depth block;包含所有层的二进制映射的一系列位;A series of bits containing the binary mapping of all layers;体现每层的深度值的一系列整数。A series of integers representing the depth value of each layer.
- 一种实现如如权利要求1-7所述方法的编码器,对分块的深度图数据设置渐进量化及其停止条件的,其包括:An encoder for implementing the method as claimed in claims 1-7, which sets progressive quantization and its stopping conditions for the block depth map data, which includes:新图层创建模块,利用编码器初始化一个空的图层列表,在所述图层列表中创建一个新图层,然后将块中的所有像素分类为唯一的图层;The new layer creation module uses the encoder to initialize an empty layer list, creates a new layer in the layer list, and then classifies all pixels in the block as a unique layer;所述编码器重复执行下列模块直到迭代过程结束:The encoder repeatedly executes the following modules until the iteration process ends:计算模块,对于图层列表中的每个图层,利用加密器计算图层内所有像素的深度值的均值和方差,并将每个平均值都附加到相应的图层;Calculation module, for each layer in the layer list, use the encryptor to calculate the mean and variance of the depth values of all pixels in the layer, and attach each average to the corresponding layer;深度值重建模块,编码器找到计算出的所有方差中的最大值,识别具有最大方差的层,将其调用为最大方差层;并通过使用每个像素所属层的平均值为每个像素分配深度值来创建重建块;计算重建块与原始深度块之间的平方误差之和;Depth value reconstruction module, the encoder finds the maximum value of all the calculated variances, identifies the layer with the largest variance, calls it as the maximum variance layer; and assigns depth to each pixel by using the average value of the layer to which each pixel belongs Value to create a reconstructed block; calculate the sum of squared errors between the reconstructed block and the original depth block;新图层创建模块,利用编码器在图层列表的尾部创建一个新图层,选取最大方差层内并且深度值大于最大方差层平均值的所有像素,从最大方差层中删除这些像素并将它们分配给新图层;New layer creation module, use the encoder to create a new layer at the end of the layer list, select all pixels in the maximum variance layer and whose depth value is greater than the average value of the maximum variance layer, delete these pixels from the maximum variance layer and remove them Assign to the new layer;在所述平方误差之和小于预设阈值时,结束迭代过程。When the sum of the square errors is less than the preset threshold, the iterative process ends.
- 一种实现如如权利要求1-7所述方法的编码器,对停止迭代的多层深度图进行数据封装,其包括:An encoder for implementing the method according to claims 1-7, which performs data encapsulation on the multi-layer depth map that stops iteration, which comprises:平均值附加模块,利用编码器为其图层列表中的所有图层计算图层内所有像素的深度值的均值和方差;将每个平均值都附加到相应的图层;The average value appending module uses the encoder to calculate the average value and variance of the depth value of all pixels in the layer for all layers in the layer list; attach each average value to the corresponding layer;图层处理模块,编码器通过按降序对每个层的区域或像素数进行排序来重新排序图层,从而首先处理具有最多像素的图层;Layer processing module, the encoder reorders the layers by sorting the area or the number of pixels of each layer in descending order, so that the layer with the most pixels is processed first;二进制映射模块,编码器利用尚未编码且具有最大像素数的层,形成二进制映射,将该层中的所有像素标记为“1”,将其他像素标记为“0”,并使用上下文自适应二进制算术编码方法来编码这个二进制映射;完成该映射后,编码器继续进行下一个最大层并重复直到剩下一 个层;最后一层不需要二进制映射,因为其会自动填充所有剩余像素。Binary mapping module, the encoder uses the layer that has not been coded and has the largest number of pixels to form a binary mapping, marking all pixels in the layer as "1" and other pixels as "0", and uses context adaptive binary arithmetic The encoding method is used to encode this binary mapping; after the mapping is completed, the encoder continues to the next largest layer and repeats until one layer is left; the last layer does not need a binary mapping because it will automatically fill all remaining pixels.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910481532.4A CN112040245B (en) | 2019-06-04 | 2019-06-04 | System and method for intra-coded depth map multi-layer representation |
CN201910481532.4 | 2019-06-04 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2020244296A1 true WO2020244296A1 (en) | 2020-12-10 |
Family
ID=73575890
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/082464 WO2020244296A1 (en) | 2019-06-04 | 2020-03-31 | System and method for multi-layer representation of depth map during intra-frame coding |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN112040245B (en) |
WO (1) | WO2020244296A1 (en) |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060164268A1 (en) * | 2005-01-21 | 2006-07-27 | Samsung Electronics Co., Ltd. | Method, medium, and apparatus controlling bit rate in image data encoding |
CN101330631A (en) * | 2008-07-18 | 2008-12-24 | 浙江大学 | Method for encoding depth image of three-dimensional television system |
CN101835056A (en) * | 2010-04-29 | 2010-09-15 | 西安电子科技大学 | Allocation method for optimal code rates of texture video and depth map based on models |
CN104010196A (en) * | 2014-03-14 | 2014-08-27 | 北方工业大学 | 3D quality scalable video coding method based on HEVC |
CN106327458A (en) * | 2016-08-31 | 2017-01-11 | 上海交通大学 | Method for layered rendering based on image |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6348918B1 (en) * | 1998-03-20 | 2002-02-19 | Microsoft Corporation | Stereo reconstruction employing a layered approach |
AU2011241284B2 (en) * | 2010-04-13 | 2015-05-07 | Samsung Electronics Co., Ltd. | Video-encoding method and video-encoding apparatus using prediction units based on encoding units determined in accordance with a tree structure, and video-decoding method and video-decoding apparatus using prediction units based on encoding units determined in accordance with a tree structure |
CN110139108B (en) * | 2011-11-11 | 2023-07-18 | Ge视频压缩有限责任公司 | Apparatus and method for encoding multi-view signals into multi-view data streams |
US20140204088A1 (en) * | 2013-01-18 | 2014-07-24 | Microsoft Corporation | Surface codec using reprojection onto depth maps |
US9467681B2 (en) * | 2013-03-25 | 2016-10-11 | Microsoft Technology Licensing, Llc | Representation and compression of depth data |
CN103581647B (en) * | 2013-09-29 | 2017-01-04 | 北京航空航天大学 | A kind of depth map sequence fractal coding based on color video motion vector |
CN104202612B (en) * | 2014-04-15 | 2018-11-02 | 清华大学深圳研究生院 | The division methods and method for video coding of coding unit based on quaternary tree constraint |
CN105007494B (en) * | 2015-07-20 | 2018-11-13 | 南京理工大学 | Wedge-shaped Fractionation regimen selection method in a kind of frame of 3D video depths image |
CN106686383A (en) * | 2017-01-17 | 2017-05-17 | 湖南优象科技有限公司 | Depth map intra-frame coding method capable of preserving edge of depth map |
CN108734208B (en) * | 2018-05-15 | 2020-12-25 | 重庆大学 | Multi-source heterogeneous data fusion system based on multi-mode deep migration learning mechanism |
-
2019
- 2019-06-04 CN CN201910481532.4A patent/CN112040245B/en active Active
-
2020
- 2020-03-31 WO PCT/CN2020/082464 patent/WO2020244296A1/en active Application Filing
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060164268A1 (en) * | 2005-01-21 | 2006-07-27 | Samsung Electronics Co., Ltd. | Method, medium, and apparatus controlling bit rate in image data encoding |
CN101330631A (en) * | 2008-07-18 | 2008-12-24 | 浙江大学 | Method for encoding depth image of three-dimensional television system |
CN101835056A (en) * | 2010-04-29 | 2010-09-15 | 西安电子科技大学 | Allocation method for optimal code rates of texture video and depth map based on models |
CN104010196A (en) * | 2014-03-14 | 2014-08-27 | 北方工业大学 | 3D quality scalable video coding method based on HEVC |
CN106327458A (en) * | 2016-08-31 | 2017-01-11 | 上海交通大学 | Method for layered rendering based on image |
Also Published As
Publication number | Publication date |
---|---|
CN112040245B (en) | 2023-07-21 |
CN112040245A (en) | 2020-12-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11140401B2 (en) | Coded-block-flag coding and derivation | |
KR102499355B1 (en) | A shape-adaptive model-based codec for lossy and lossless image compression | |
KR102155184B1 (en) | Method for encoding a current block of a first image component relative to a reference block of at least one second image component, encoding device and corresponding computer program | |
US12041243B2 (en) | Systems and methods for compressing video | |
CN110896483A (en) | Method for compressing and decompressing image data | |
CN111316642A (en) | Method and apparatus for signaling image coding and decoding partition information | |
US20230370600A1 (en) | A method and apparatus for encoding and decoding one or more views of a scene | |
US20190347867A1 (en) | Method and apparatus for processing three dimensional object image using point cloud data | |
US11263786B2 (en) | Decoding data arrays | |
WO2020244296A1 (en) | System and method for multi-layer representation of depth map during intra-frame coding | |
Yoshida et al. | Two-layer lossless coding for high dynamic range images based on range compression and adaptive inverse tone-mapping | |
CN102685483B (en) | Decoding method | |
US12022078B2 (en) | Picture processing method and apparatus | |
US20160191937A1 (en) | Video data processing system | |
CN104168482B (en) | A kind of video coding-decoding method and device | |
CN115209147A (en) | Camera video transmission bandwidth optimization method, device, equipment and storage medium | |
CN115442617A (en) | Video processing method and device based on video coding | |
US10045022B2 (en) | Adaptive content dependent intra prediction mode coding | |
CN112509107A (en) | Point cloud attribute recoloring method, device and encoder | |
WO2021035717A1 (en) | Intra-frame chroma prediction method and apparatus, device, and video coding and decoding system | |
CN102685485B (en) | Coding method and device, and decoding method and device | |
CN115336264A (en) | Intra-frame prediction method, device, encoder, decoder, and storage medium | |
CN110969672A (en) | Image compression method and device | |
CN115988201B (en) | Method, apparatus, electronic device and storage medium for encoding film grain | |
US20240195990A1 (en) | Residual-free palatte mode coding |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20818683 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20818683 Country of ref document: EP Kind code of ref document: A1 |