WO2020006707A1 - Palette-based residual coding for video compression system - Google Patents

Palette-based residual coding for video compression system Download PDF

Info

Publication number
WO2020006707A1
WO2020006707A1 PCT/CN2018/094496 CN2018094496W WO2020006707A1 WO 2020006707 A1 WO2020006707 A1 WO 2020006707A1 CN 2018094496 W CN2018094496 W CN 2018094496W WO 2020006707 A1 WO2020006707 A1 WO 2020006707A1
Authority
WO
WIPO (PCT)
Prior art keywords
palette
prediction
block
pixels
pixel
Prior art date
Application number
PCT/CN2018/094496
Other languages
French (fr)
Inventor
Yu-chen SUN
Jicheng An
Jian Lou
Original Assignee
Alibaba Group Holding Limited
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Limited filed Critical Alibaba Group Holding Limited
Priority to CN201880094272.9A priority Critical patent/CN112219399A/en
Priority to PCT/CN2018/094496 priority patent/WO2020006707A1/en
Publication of WO2020006707A1 publication Critical patent/WO2020006707A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46—Embedding additional information in the video signal during the compression process
    • H04N19/463—Embedding additional information in the video signal during the compression process by compressing encoding parameters before transmission

Definitions

  • the state-of-the-art video coding standard such as H. 264/AVC (Advanced Video Coding) and H. 265/HEVC (High Efficiency Video Coding) standard, is based on a hybrid coding scheme using block-based prediction and transform coding.
  • a video compression system applies prediction methods, such as intra-prediction or inter-prediction, to derive a prediction frame.
  • the video compression system further encodes residual information to refine the prediction frame.
  • the residual information is usually compressed by transform and quantization techniques.
  • a decoder is able to reconstruct the video frame from the prediction frame and the residual information.
  • the decoder combines a prediction block and a residual block to generate a reconstructed block by simply adding prediction pixel values and residual pixel values to generate final reconstructed pixel values.
  • a new coding tool, palette mode is adopted to standardize Screen Content Coding Extensions of High Efficiency Video Coding (HEVC SCC) .
  • the palette mode can efficiently describe all pixels in a coding unit (CU) with few selected representative colors.
  • the palette mode is designed based on the observation that the pixel values in a screen content frame are usually concentrated on few color values.
  • An encoder analyzes the pixels in a CU and determines several representative colors to construct a palette table, i.e., a color mapping table between the representative color values and color indices.
  • the palette table is signaled in a bitstream. Pixels with pixel values close to palette colors are quantized to selected palette colors denoted by corresponding palette indices. The rest of the pixels are called escape pixels.
  • a special palette index value is reserved to represent the escape pixels, and pixel values of the escape pixels are signaled directly. All palette indices in the CU form a palette index map, which is transmitted to a decoder along with the escape pixel values.
  • a pixel or a palette index may represent a single color or three-color component values, for example, YCbCr or GBR.
  • FIG. 1 illustrates an example decoding process of a video compression system using prediction methods.
  • FIG. 2 illustrates an example prediction block, an example residual block, and an example reconstructed block derived by the video compression system of FIG. 1.
  • FIG. 3 illustrates an example diagram of the palette mode applied to an image.
  • FIG. 4 illustrates an example decoding process of the video compression system using HEVC SCC palette mode.
  • FIG. 5 illustrates an example palette index mapping using the HEVC SCC palette mode.
  • FIG. 6 illustrated an example flowchart for the palette-based residual coding.
  • FIG. 7 illustrates an example flowchart detailing one of blocks of FIG. 6.
  • FIG. 8 illustrates an example decoding process of the video compression system using palette-based residual coding mode.
  • FIG. 9 illustrates an example prediction block, an example palette-based residual block, and an example reconstructed block derived by the video compression system of FIG. 8 using the palette-based residual decoding.
  • FIG. 10 illustrates an example palette index map coding for the palette-based residual coding.
  • FIG. 11 illustrates another example palette index map coding for the palette-based residual coding.
  • FIG. 12 illustrates an example system for implementing the processes and methods for improving decoding performance by combining the palette mode and the prediction methods.
  • Systems and methods discussed herein are directed to improving video compression system performance in the HEVC palette mode, and more specifically to improving decoding performance by combining the palette mode and the prediction methods.
  • FIG. 1 illustrates an example decoding process 100 of a video compression system 102 using prediction methods.
  • the video compression system 102 first derives a prediction block 104 by applying a prediction method, such as intra-prediction 106 or inter-prediction 108, and then decodes a residual block 110 by applying a residual decoding method 112.
  • the video compression system 102 combines the prediction block 104 and the residual block 110 to generate a reconstructed block 114.
  • FIG. 2 illustrates example prediction block 104, residual block 110, and reconstructed block 114 derived by the video compression system 102 of FIG. 1.
  • the prediction block 104 and the residual block 110 are illustrated with the pixel values of coding units (CUs) . As discussed above with reference to FIG. 1, the corresponding pixel values of the prediction block 104 and the residual block 110 are simply added to generate corresponding pixel values of the reconstructed block 114.
  • CUs coding units
  • FIG. 3 illustrates an example diagram 300 of the palette mode applied to CU 302.
  • a pixel or a palette index is shown to correspond to only one value.
  • a pixel or a palette index may represent three color component values, such as YCbCr or GBR.
  • a flag is transmitted for each CU to indicate whether the palette mode is used for that CU, such as a CU 302. If the palette mode is used for the CU 302, the pixels, having pixel values close to palette colors, such as color A 304, color B 306, and color C 308, are represented by the palette color values 310 as shown in a color histogram 312. For example, the color A 304 is shown to be close to the palette color value of 50, the color B 306 is shown to be close to the palette color value of 80, and the color C 308 is shown to be close to the palette color value of 310.
  • the color A 304 is shown to be close to the palette color value of 50
  • the color B 306 is shown to be close to the palette color value of 80
  • the color C 308 is shown to be close to the palette color value of 310.
  • the palette color values 310 of the CU 302 are denoted by palette indices 314, where the palette color values 310 are representative colors of the CU 302 as shown in a palette table 316.
  • the remaining pixels are presented by a special palette index value, denoted as an escape index, and their pixel values are signaled directly.
  • a color is a 3-value, for example, YCbCr or GBR, vector.
  • Palette indices of all pixels in the CU 302 form a palette index map 318.
  • FIG. 4 illustrates an example decoding process 400 of the video compression system 402 using HEVC SCC palette mode.
  • a block, or a CU is encoded by the HEVC SCC palette mode 404, it is decoded independently to generate a reconstructed block 406 without using any prediction method or residual refinement.
  • a video decoder of the video compression system 402 decodes palette colors and indices. Palette colors are described by a palette table, such as the palette table 316, and encoded by palette table coding tools. Based on the palette table, a pixel of three color components is represented by a palette index, such as the palette index 314. Palette indices of all pixels in the CU form a palette index map, such as the palette index map 318, and are encoded by palette index map coding tools.
  • FIG. 5 illustrates an example the palette index map coding 500.
  • the video compression system 102 constructs a palette table predictor to predict a palette table, such as the palette table 316, of the current CU by applying predictive coding where the palette of the last palette coded CU is used as the predictor.
  • a palette table such as the palette table 316
  • the pixels in the current CU are represented by palette indices 502 corresponding to the colors in the palette as shown in a parameter table 504 along with other parameters of the palette index map coding.
  • the indices form a palette index map 506 which is divided into several runs and then encoded in horizontal or vertical traverse scan order. In this example, the horizontal scan order is assumed and the direction of the scan is shown as arrows in the palette index map 506.
  • 16 indices are encoded by run indices 508, three runs R0, R1, and R2, with horizontal traverse scan.
  • a run of pixels 512 will copy the palette indices from their above pixels.
  • a run, R2 is a copy above run with a run value of 10.
  • a palette_index_idc syntax element is first signaled, followed by a piece of PaletteIndexRun information.
  • a run of pixels 512 shares the same palette index 502, where the palette index 502 and the run value of the run of pixels 512 are derived from the palette_index_idc and PaletteIndexRun, respectively.
  • FIG. 6 illustrated an example flowchart 600 of the palette-based residual coding in a video compression system.
  • the video compression system may derive a prediction block, such as the prediction block 104, of a coding unit (CU) , which may be derived by using a conventional prediction method, such as an intra-prediction or inter-prediction method as discussed above with reference to FIG. 1.
  • the video compression system may construct a palette table predictor to predict the palette table of the current CU, and may utilize the palette of the last palette coded CU as the predictor.
  • the video compression system may use a palette of a non-adjacent coding unit relative to the coding unit as a palette predictor to predict a palette table of the coding unit, and apply the predicting method to the palette table to derive the prediction block.
  • the video compression system may decode palette-based residual information of the coding unit, and based on the decoded palette-based residual information, may derive a palette-based residual block of the coding unit at block 606.
  • the video compression system may combine the prediction block and the palette-based residual block to construct a reconstruction block.
  • the video compression system may refine the prediction block based on the decoded palette-based residual information and improve the coding and/or decoding performance by combining palette mode and the traditional prediction methods, thus improving the decoding performance of the video compression system.
  • FIG. 7 illustrates an example flowchart detailing block 608 of FIG. 6.
  • the video compression system may refine the prediction block 104 based on the decoded palette-based residual information by decoding the palette-based residual block at block 702, and classify pixels of the palette-based residual block into two groups, such as major color pixels and prediction pixels at block 704.
  • the major color pixels may have palette indices associated with major color pixel values of the decoded palette-based residual block similar to the association shown in the palette table 316.
  • the prediction pixels are assigned a predetermined color index, such as C0, C1, etc., and the video compression system may generate a palette index map based on the major color pixels and prediction pixels at block 708.
  • the video compression system may signal whether a pixel in the palette-based residual block is a major color pixel or a prediction pixel by decoding the palette index map and identifying an index having the predetermined color index as a prediction pixel at block 710 or by identifying a pixel encoded by a prediction run as a prediction pixel at block 712.
  • the video compression system may use the major color pixel values of the major color pixels and pixel values of the prediction block for the prediction pixels in the reconstruction block and transfer pixel values of the prediction block for the prediction pixels in the reconstruction block at block 714, thus combining the palette mode and the traditional prediction methods to improve the decoding performance of the video compression system.
  • FIG. 8 illustrates an example decoding process 800 of the video compression system 802 using the palette-based residual coding mode.
  • the video compression system 802 may include an option of combining the palette mode and the traditional prediction methods as described above with reference to FIGs. 6 and 7, in addition to the HEVC residual decoding as described above with reference to FIGs. 4 and 5.
  • the video compression system 802 may derive a palette-based residual block 804 by applying the palette-based residual decoding 806 to palette-based residual information.
  • the video compression system 802 may then construct a reconstruction block 808 by applying the palette-based prediction and residual block combination 810 to information from the prediction block 104 and the palette-based residual block 804.
  • FIG. 9 illustrates the prediction block 104, the palette-based residual block 804, and the reconstructed block 808 derived by the video compression system 802 of FIG. 8 using the palette-based residual decoding.
  • the video compression system 802 may derive the prediction block 104 as described above with reference to FIG. 1, and may then decode a palette-based residual block 804. Before reconstructing the reconstructed block 808, the video compression system 802 may derive and use the palette-based residual block 804 as described above with reference to FIG. 8 to refine the prediction block 104.
  • the video compression system 802 may classify pixels in the palette-based residual block 804 into two groups, major color pixels and prediction pixels.
  • the major color pixels may be pixels with pixel values, for example C0 and C1 as shown, where the pixel values are associated with major color values in a decoded palette table, such as the palette table 316 of FIG. 3.
  • the prediction pixels may be special pixels, shown as “P” in the palette-based residual block 804.
  • the video compression system 802 may directly use its corresponding prediction pixel value from the prediction block 104 as the corresponding reconstructed pixel value for the reconstructed block 808. If a pixel in the palette-based residual block 804 were a major color pixel having a corresponding major color value, such as C0 or C1, in the decoded palette table, the video compression system 802 would use the corresponding major color value to reconstruct the reconstructed block 808 as shown.
  • FIG. 10 illustrates an example palette index map coding 1000 for the palette-based residual coding.
  • the video compression system 802 may reserve a special color index, for example, 0, as shown under a palette index 1002 of a parameter table 1004, the maximal index value, or a pre-defined value.
  • the video compression system 802 may then use a conventional palette mode to decode a palette index map 1006 as described above with reference to FIG. 5.
  • the indices form the palette index map 1006 which is divided into several runs and then encoded in horizontal or vertical traverse scan order. In this example, the horizontal scan order is assumed and the direction of the scan is shown as arrows in the palette index map 1006.
  • 16 indices are encoded by run indices, four runs R0, R1, R2, and R3 shown under a run mode 1008, with a horizontal traverse scan.
  • a pixel in the palette index map 1006 were identified to have the reserved color index, it would be identified as a prediction pixel.
  • the bottom four pixels are identified as prediction pixels.
  • the video compression system 802 may reserve a special color index, “0” in this example, for these prediction pixels.
  • the video compression system 802 may use the copy index run, R3, for the four prediction pixels as indicated by a run of pixels 1012 to signal those four prediction pixels.
  • FIG. 11 illustrates another example palette index map coding 1100 for the palette-based residual coding.
  • the video compression system 802 may utilize a special run, such as a prediction run as shown “prediction” under a run mode 1102 of a parameter table 1104 instead of reserving a special color index under a palette index 1106.
  • the video compression system 802 may identify a prediction pixel of the palette-based residual block 804 as a pixel encoded by the prediction run. Similar to the palette index map coding of FIG. 10, the indices form the palette index map 1006 which is divided into four runs and then encoded in a horizontal scan order shown as arrows.
  • 16 indices are encoded by run indices 1108, four runs R0, R1, R2, and R3 with horizontal traverse scan. There are three, copy index mode, copy above mode, and prediction mode, under the run mode 1102, and for each starting position of a run, a flag is transmitted to indicate which run mode is used. If a pixel in the palette index map 1006 were identified to have been encoded by the prediction run, it would be identified as a prediction pixel. In FIG. 11, the video compression system 802 may use the prediction run, R3, for the four bottom pixels to encode as prediction pixels as indicated by a run of pixels 1110, and signal those four pixels as prediction pixels.
  • FIG. 12 illustrates an example system 1200 for implementing the processes and methods described above for improving decoding performance by combining the palette mode and the prediction methods.
  • the techniques and mechanisms described herein may be implemented by multiple instances of the system 1200 as well as by any other computing device, system, and/or environment.
  • the system 1200 shown in FIG. 12 is only one example of a system and is not intended to suggest any limitation as to the scope of use or functionality of any computing device utilized to perform the processes and/or procedures described above.
  • the system 1200 may include one or more processors 1202 and system memory 1204 communicatively coupled to the processor (s) 1202.
  • the processor (s) 1202 may execute one or more modules and/or processes to cause the processor (s) 1202 to perform a variety of functions.
  • the processor (s) 1202 may include a central processing unit (CPU) , a graphics processing unit (GPU) , both CPU and GPU, or other processing units or components known in the art. Additionally, each of the processor (s) 1202 may possess its own local memory, which also may store program modules, program data, and/or one or more operating systems.
  • system memory 1204 may be volatile, such as RAM, non-volatile, such as ROM, flash memory, miniature hard drive, memory card, and the like, or some combination thereof.
  • the system memory 1204 may include one or more computer-executable modules (modules) 1206 that are executable by the processor (s) 1202.
  • the modules 1206 may include, but are not limited to, a prediction module 1208, a residual module 1210, and a reconstruction module 1212.
  • the prediction 1208 may be configured to derive a prediction block of a coding unit as discussed above with reference to FIG. 1.
  • the prediction module may use a palette of a non-adjacent coding unit relative to the coding unit as a palette predictor to predict a palette table of the coding unit, and apply the predicting method, such as intra-prediction 102 or inter-prediction 108, to a palette table, such as the palette table 316 to derive a prediction block, such as the prediction block 104.
  • the residual module 1210 may be configured to decode palette-based residual information of the coding unit and to derive a palette-based residual block, such as the palette-based residual block 804, based on the decoded palette-based residual information.
  • the reconstruction module 1212 may be configured to combine the prediction block 104 and the palette-based residual block 804 to construct a reconstruction block, such as the reconstruction block 808, thus improving the decoding performance of the system 1200.
  • the residual module 1210 may be further configured to decode the palette-based residual block 804, and classify pixels of the palette-based residual block into major color pixels and prediction pixel.
  • the major color pixels may have palette indices associated with major color pixel values of the decoded palette-based residual block 804.
  • the residual module 1210 may be additionally configured to generate a palette index map, such as the palette index map 1006, based on the major color pixels and prediction pixels, and assign a predetermined color index, such as C0, C1, etc. to prediction pixels as shown in FIG. 9.
  • the reconstruction module 1212 may be further configured to signal whether a pixel in the palette-based residual block is a major color pixel or a prediction pixel by decoding the palette index map 1006 and identifying an index having the predetermined color index as a prediction pixel as discussed above with reference to FIG. 10, or by identifying a pixel encoded by a prediction run as a prediction pixel as discussed above with reference to FIG. 11.
  • the reconstruction module 1212 may be configured to use the major color pixel values of the major color pixels in the reconstruction block 808 and transfer pixel values of the prediction block for the prediction pixels in the reconstruction block 808, thus combining the palette mode and the traditional prediction methods to improve the decoding performance of the video compression system.
  • the system 1200 may additionally include an input/output (I/O) interface 1214 for receiving data associated with a coding unit, such as encoded data of the CU 302, to be processed, and for outputting the processed data of the coding unit, such as the reconstructed block 808.
  • the system 1200 may also include a communication module 1216 allowing the system 1200 to communicate with other devices (not shown) over a network (not shown) .
  • the network may include the Internet, wired media such as a wired network or direct-wired connections, and wireless media such as acoustic, radio frequency (RF) , infrared, and other wireless media.
  • RF radio frequency
  • Computer-readable instructions include routines, applications, application modules, program modules, programs, components, data structures, algorithms, and the like.
  • Computer-readable instructions can be implemented on various system configurations, including single-processor or multiprocessor systems, minicomputers, mainframe computers, personal computers, hand-held computing devices, microprocessor-based, programmable consumer electronics, combinations thereof, and the like.
  • the computer-readable storage media may include volatile memory (such as random access memory (RAM) ) and/or non-volatile memory (such as read-only memory (ROM) , flash memory, etc. ) .
  • volatile memory such as random access memory (RAM)
  • non-volatile memory such as read-only memory (ROM) , flash memory, etc.
  • the computer-readable storage media may also include additional removable storage and/or non-removable storage including, but not limited to, flash memory, magnetic storage, optical storage, and/or tape storage that may provide non-volatile storage of computer-readable instructions, data structures, program modules, and the like.
  • a non-transient computer-readable storage medium is an example of computer-readable media.
  • Computer-readable media includes at least two types of computer-readable media, namely computer-readable storage media and communications media.
  • Computer-readable storage media includes volatile and non-volatile, removable and non-removable media implemented in any process or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data.
  • Computer-readable storage media includes, but is not limited to, phase change memory (PRAM) , static random-access memory (SRAM) , dynamic random-access memory (DRAM) , other types of random-access memory (RAM) , read-only memory (ROM) , electrically erasable programmable read-only memory (EEPROM) , flash memory or other memory technology, compact disk read-only memory (CD-ROM) , digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by a computing device.
  • communication media may embody computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, or other transmission mechanism. As defined herein, computer-readable storage media do not include communication media.
  • the computer-readable instructions stored on one or more non-transitory computer-readable storage media that, when executed by one or more processors, may perform operations described above with reference to FIGs. 1-12.
  • computer-readable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular abstract data types.
  • the order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and/or in parallel to implement the processes.
  • a method comprising: deriving a prediction block of a coding unit; decoding palette-based residual information of the coding unit; deriving a palette-based residual block based on the decoded palette-based residual information; and combining the prediction block and the palette-based residual block to construct a reconstruction block.
  • deriving the prediction block of the coding unit includes: using a palette of a non-adjacent coding unit relative to the coding unit as a palette predictor to predict a palette table of the coding unit; and applying predicting coding to the palette table to derive the prediction block.
  • refining the prediction block based on the decoded palette-based residual information comprises: decoding the palette-based residual block; classifying pixels of the palette-based residual block into major color pixels and prediction pixels, the major color pixels having palette indices associated with major color pixel values of the decoded palette-based residual block; and generating a palette index map based on the major color pixels and prediction pixels.
  • the method as paragraph E recites, further comprising signaling whether a pixel in the palette-based residual block is a major color pixel or a prediction pixel includes: decoding the palette index map and identifying an index having the predetermined color index as a prediction pixel; or identifying a pixel encoded by a prediction run as a prediction pixel.
  • combining the prediction block and the palette-based residual block to construct the reconstruction block comprises: using the major color pixel values of the major color pixels in the reconstruction block; and transferring pixel values of the prediction block for the prediction pixels in the reconstruction block.
  • a computer-readable storage medium storing computer-readable instructions executable by one or more processors, that when executed by the one or more processors, cause the one or more processors to perform operations comprising: deriving a prediction block of a coding unit; decoding palette-based residual information of the coding unit; deriving a palette-based residual block based on the decoded palette-based residual information; and combining the prediction block and the palette-based residual block to construct a reconstruction block.
  • deriving the prediction block of the coding unit includes: using a palette of a non-adjacent coding unit relative to the coding unit as a palette predictor to predict a palette table of the coding unit; and applying predicting coding to the palette table to derive the prediction block.
  • refining the prediction block based on the decoded palette-based residual information comprises: decoding the palette-based residual block; classifying pixels of the palette-based residual block into major color pixels and prediction pixels, the major color pixels having palette indices associated with major color pixel values of the decoded palette-based residual block; and generating a palette index map based on the major color pixels and prediction pixels.
  • the computer-readable storage medium as paragraph L recites, further comprising signaling whether a pixel in the palette-based residual block is a major color pixel or a prediction pixel includes: decoding the palette index map and identifying an index having the predetermined color index as a prediction pixel; or identifying a pixel encoded by a prediction run as a prediction pixel.
  • combining the prediction block and the palette-based residual block to construct the reconstruction block comprises: using the major color pixel values of the major color pixels in the reconstruction block; and transferring pixel values of the prediction block for the prediction pixels in the reconstruction block.
  • a system comprising: one or more processors; and memory communicatively coupled to the one or more processors, the memory storing computer-executable modules executable by the one or more processors, the computer-executable modules including: a prediction module configured to derive a prediction block of a coding unit; a residual module configured to decode palette-based residual information of the coding unit and to derive a palette-based residual block based on the decoded palette-based residual information; and a reconstruction module configured to combine the prediction block and the palette-based residual block to construct a reconstruction block.
  • the prediction block is further configured to: use a palette of a non-adjacent coding unit relative to the coding unit as a palette predictor to predict a palette table of the coding unit; and apply predicting coding to the palette table to derive the prediction block.
  • the residual module is further configured: decode the palette-based residual block; classify pixels of the palette-based residual block into major color pixels and prediction pixels, the major color pixels having palette indices associated with major color pixel values of the decoded palette-based residual block; and generate a palette index map based on the major color pixels and prediction pixels.
  • the reconstruction module is further configured to signal whether a pixel in the palette-based residual block is a major color pixel or a prediction pixel by: decoding the palette index map and identifying an index having the predetermined color index as a prediction pixel; or identifying a pixel encoded by a prediction run as a prediction pixel.
  • the reconstruction module is further configured to: use the major color pixel values of the major color pixels in the reconstruction block; and transfer pixel values of the prediction block for the prediction pixels in the reconstruction block.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Systems and methods are provided for improving decoding performance of a video compression system by combining the palette mode and the traditional prediction methods comprising deriving a prediction block of a coding unit, decoding palette-based residual information of the coding unit, deriving a palette-based residual block based on the decoded palette-based residual information, and combining the prediction block and the palette-based residual block to construct a reconstruction block.

Description

PALETTE-BASED RESIDUAL CODING FOR VIDEO COMPRESSION SYSTEM BACKGROUND
The state-of-the-art video coding standard, such as H. 264/AVC (Advanced Video Coding) and H. 265/HEVC (High Efficiency Video Coding) standard, is based on a hybrid coding scheme using block-based prediction and transform coding. To compress a video frame, a video compression system applies prediction methods, such as intra-prediction or inter-prediction, to derive a prediction frame. Based on the prediction frame, the video compression system further encodes residual information to refine the prediction frame. The residual information is usually compressed by transform and quantization techniques. Based on those information, a decoder is able to reconstruct the video frame from the prediction frame and the residual information. The decoder combines a prediction block and a residual block to generate a reconstructed block by simply adding prediction pixel values and residual pixel values to generate final reconstructed pixel values.
A new coding tool, palette mode, is adopted to standardize Screen Content Coding Extensions of High Efficiency Video Coding (HEVC SCC) . The palette mode can efficiently describe all pixels in a coding unit (CU) with few selected representative colors. The palette mode is designed based on the observation that the pixel values in a screen content frame are usually concentrated on few color values. An encoder analyzes the pixels in a CU and determines several representative colors to construct a palette table, i.e., a color  mapping table between the representative color values and color indices. The palette table is signaled in a bitstream. Pixels with pixel values close to palette colors are quantized to selected palette colors denoted by corresponding palette indices. The rest of the pixels are called escape pixels. A special palette index value is reserved to represent the escape pixels, and pixel values of the escape pixels are signaled directly. All palette indices in the CU form a palette index map, which is transmitted to a decoder along with the escape pixel values. A pixel or a palette index may represent a single color or three-color component values, for example, YCbCr or GBR.
However, in the HEVC palette mode, if a block is encoded by the palette mode, it will be decoded independently without using any prediction method or residual refinement.
BRIEF DESCRIPTION OF THE DRAWINGS
The detailed description is set forth with reference to the accompanying figures. In the figures, the left-most digit (s) of a reference number identifies the figure in which the reference number first appears. The use of the same reference numbers in different figures indicates similar or identical items or features.
FIG. 1 illustrates an example decoding process of a video compression system using prediction methods.
FIG. 2 illustrates an example prediction block, an example residual block, and an example reconstructed block derived by the video compression system of FIG. 1.
FIG. 3 illustrates an example diagram of the palette mode applied to an image.
FIG. 4 illustrates an example decoding process of the video compression system using HEVC SCC palette mode.
FIG. 5 illustrates an example palette index mapping using the HEVC SCC palette mode.
FIG. 6 illustrated an example flowchart for the palette-based residual coding.
FIG. 7 illustrates an example flowchart detailing one of blocks of FIG. 6.
FIG. 8 illustrates an example decoding process of the video compression system using palette-based residual coding mode.
FIG. 9 illustrates an example prediction block, an example palette-based residual block, and an example reconstructed block derived by the video compression system of FIG. 8 using the palette-based residual decoding.
FIG. 10 illustrates an example palette index map coding for the palette-based residual coding.
FIG. 11 illustrates another example palette index map coding for the palette-based residual coding.
FIG. 12 illustrates an example system for implementing the processes and methods for improving decoding performance by combining the palette mode and the prediction methods.
DETAILED DESCRIPTION
Systems and methods discussed herein are directed to improving video compression system performance in the HEVC palette mode, and more specifically to improving decoding performance by combining the palette mode and the prediction methods.
FIG. 1 illustrates an example decoding process 100 of a video compression system 102 using prediction methods.
The video compression system 102 first derives a prediction block 104 by applying a prediction method, such as intra-prediction 106 or inter-prediction 108, and then decodes a residual block 110 by applying a residual decoding method 112. The video compression system 102 combines the prediction block 104 and the residual block 110 to generate a reconstructed block 114.
FIG. 2 illustrates example prediction block 104, residual block 110, and reconstructed block 114 derived by the video compression system 102 of FIG. 1.
The prediction block 104 and the residual block 110 are illustrated with the pixel values of coding units (CUs) . As discussed above with reference to FIG. 1, the corresponding pixel values of the prediction block 104 and the residual block 110 are simply added to generate corresponding pixel values of the reconstructed block 114.
FIG. 3 illustrates an example diagram 300 of the palette mode applied to CU 302. For simplicity, a pixel or a palette index is shown to correspond to only one value. However, in HEVC SCC, a pixel or a palette index may represent three color component values, such as YCbCr or GBR.
In the HEVC SCC palette mode, a flag is transmitted for each CU to indicate whether the palette mode is used for that CU, such as a CU 302. If the palette mode is used for the CU 302, the pixels, having pixel values close to palette colors, such as color A 304, color B 306, and color C 308, are represented by the palette color values 310 as shown in a color histogram 312. For example, the color A 304 is shown to be close to the palette color value of 50, the color B 306 is shown to be close to the palette color value of 80, and the color C 308 is shown to be close to the palette color value of 310. The palette color values 310 of the CU 302 are denoted by palette indices 314, where the palette color values 310 are representative colors of the CU 302 as shown in a palette table 316. The remaining pixels are presented by a special palette index value, denoted as an escape index, and their pixel values are signaled directly. Note that a color is a 3-value, for example, YCbCr or GBR, vector. Palette indices of all pixels in the CU 302 form a palette index map 318.
FIG. 4 illustrates an example decoding process 400 of the video compression system 402 using HEVC SCC palette mode.
Compared to the intra/inter-prediction methods described with reference to FIG. 1, if a block, or a CU, is encoded by the HEVC SCC palette mode 404, it is decoded independently to generate a reconstructed block 406  without using any prediction method or residual refinement. To decode a palette encoded block, a video decoder of the video compression system 402 decodes palette colors and indices. Palette colors are described by a palette table, such as the palette table 316, and encoded by palette table coding tools. Based on the palette table, a pixel of three color components is represented by a palette index, such as the palette index 314. Palette indices of all pixels in the CU form a palette index map, such as the palette index map 318, and are encoded by palette index map coding tools.
FIG. 5 illustrates an example the palette index map coding 500.
The video compression system 102 constructs a palette table predictor to predict a palette table, such as the palette table 316, of the current CU by applying predictive coding where the palette of the last palette coded CU is used as the predictor. After the palette table is encoded, the pixels in the current CU are represented by palette indices 502 corresponding to the colors in the palette as shown in a parameter table 504 along with other parameters of the palette index map coding. The indices form a palette index map 506 which is divided into several runs and then encoded in horizontal or vertical traverse scan order. In this example, the horizontal scan order is assumed and the direction of the scan is shown as arrows in the palette index map 506. In the palette index map 506, 16 indices are encoded by run indices 508, three runs R0, R1, and R2, with horizontal traverse scan. There are two run modes 510, copy index mode and copy above mode, and for each starting position of a run, a flag is transmitted to indicate which run mode is used.
If the copy above mode is used, a run of pixels 512 will copy the palette indices from their above pixels. In this example, a run, R2, is a copy above run with a run value of 10. If the copy index mode is used, a palette_index_idc syntax element is first signaled, followed by a piece of PaletteIndexRun information. The palette_index_idc syntax describes whether the run is copy index mode (palette_index_idc=0) or copy above mode (palette_index_idc=1) , and the PaletteIndexRun syntax describes the length of the run. A run of pixels 512 shares the same palette index 502, where the palette index 502 and the run value of the run of pixels 512 are derived from the palette_index_idc and PaletteIndexRun, respectively.
FIG. 6 illustrated an example flowchart 600 of the palette-based residual coding in a video compression system.
At block 602, the video compression system may derive a prediction block, such as the prediction block 104, of a coding unit (CU) , which may be derived by using a conventional prediction method, such as an intra-prediction or inter-prediction method as discussed above with reference to FIG. 1. The video compression system may construct a palette table predictor to predict the palette table of the current CU, and may utilize the palette of the last palette coded CU as the predictor. The video compression system may use a palette of a non-adjacent coding unit relative to the coding unit as a palette predictor to predict a palette table of the coding unit, and apply the predicting method to the palette table to derive the prediction block. At block 604, the video compression system may decode palette-based residual information of the coding unit, and  based on the decoded palette-based residual information, may derive a palette-based residual block of the coding unit at block 606. At 608, the video compression system may combine the prediction block and the palette-based residual block to construct a reconstruction block. The video compression system may refine the prediction block based on the decoded palette-based residual information and improve the coding and/or decoding performance by combining palette mode and the traditional prediction methods, thus improving the decoding performance of the video compression system.
FIG. 7 illustrates an example flowchart detailing block 608 of FIG. 6.
The video compression system may refine the prediction block 104 based on the decoded palette-based residual information by decoding the palette-based residual block at block 702, and classify pixels of the palette-based residual block into two groups, such as major color pixels and prediction pixels at block 704. The major color pixels may have palette indices associated with major color pixel values of the decoded palette-based residual block similar to the association shown in the palette table 316. At block 706, the prediction pixels are assigned a predetermined color index, such as C0, C1, etc., and the video compression system may generate a palette index map based on the major color pixels and prediction pixels at block 708.
The video compression system may signal whether a pixel in the palette-based residual block is a major color pixel or a prediction pixel by decoding the palette index map and identifying an index having the predetermined color index as a prediction pixel at block 710 or by identifying a  pixel encoded by a prediction run as a prediction pixel at block 712. The video compression system may use the major color pixel values of the major color pixels and pixel values of the prediction block for the prediction pixels in the reconstruction block and transfer pixel values of the prediction block for the prediction pixels in the reconstruction block at block 714, thus combining the palette mode and the traditional prediction methods to improve the decoding performance of the video compression system.
FIG. 8 illustrates an example decoding process 800 of the video compression system 802 using the palette-based residual coding mode.
The video compression system 802 may include an option of combining the palette mode and the traditional prediction methods as described above with reference to FIGs. 6 and 7, in addition to the HEVC residual decoding as described above with reference to FIGs. 4 and 5. The video compression system 802 may derive a palette-based residual block 804 by applying the palette-based residual decoding 806 to palette-based residual information. The video compression system 802 may then construct a reconstruction block 808 by applying the palette-based prediction and residual block combination 810 to information from the prediction block 104 and the palette-based residual block 804.
FIG. 9 illustrates the prediction block 104, the palette-based residual block 804, and the reconstructed block 808 derived by the video compression system 802 of FIG. 8 using the palette-based residual decoding.
The video compression system 802 may derive the prediction block 104 as described above with reference to FIG. 1, and may then decode a palette-based residual block 804. Before reconstructing the reconstructed block 808, the video compression system 802 may derive and use the palette-based residual block 804 as described above with reference to FIG. 8 to refine the prediction block 104. The video compression system 802 may classify pixels in the palette-based residual block 804 into two groups, major color pixels and prediction pixels. The major color pixels may be pixels with pixel values, for example C0 and C1 as shown, where the pixel values are associated with major color values in a decoded palette table, such as the palette table 316 of FIG. 3. The prediction pixels may be special pixels, shown as “P” in the palette-based residual block 804. If a pixel is encoded as “P” , the video compression system 802 may directly use its corresponding prediction pixel value from the prediction block 104 as the corresponding reconstructed pixel value for the reconstructed block 808. If a pixel in the palette-based residual block 804 were a major color pixel having a corresponding major color value, such as C0 or C1, in the decoded palette table, the video compression system 802 would use the corresponding major color value to reconstruct the reconstructed block 808 as shown.
FIG. 10 illustrates an example palette index map coding 1000 for the palette-based residual coding.
To signal whether a pixel in the palette-based residual block 804 is a major color pixel or a prediction pixel, the video compression system 802 may reserve a special color index, for example, 0, as shown under a palette index  1002 of a parameter table 1004, the maximal index value, or a pre-defined value. The video compression system 802 may then use a conventional palette mode to decode a palette index map 1006 as described above with reference to FIG. 5. The indices form the palette index map 1006 which is divided into several runs and then encoded in horizontal or vertical traverse scan order. In this example, the horizontal scan order is assumed and the direction of the scan is shown as arrows in the palette index map 1006. In the palette index map 1006, 16 indices are encoded by run indices, four runs R0, R1, R2, and R3 shown under a run mode 1008, with a horizontal traverse scan. There are two run modes, copy index mode and copy above mode, as shown under a run mode 1010, and for each starting position of a run, a flag is transmitted to indicate which run mode is used.
If a pixel in the palette index map 1006 were identified to have the reserved color index, it would be identified as a prediction pixel. In FIG. 10, the bottom four pixels are identified as prediction pixels. The video compression system 802 may reserve a special color index, “0” in this example, for these prediction pixels. The video compression system 802 may use the copy index run, R3, for the four prediction pixels as indicated by a run of pixels 1012 to signal those four prediction pixels.
FIG. 11 illustrates another example palette index map coding 1100 for the palette-based residual coding.
To signal whether a pixel in the palette-based residual block 804 is a major color pixel or a prediction pixel, the video compression system 802 may utilize a special run, such as a prediction run as shown “prediction” under a run  mode 1102 of a parameter table 1104 instead of reserving a special color index under a palette index 1106. The video compression system 802 may identify a prediction pixel of the palette-based residual block 804 as a pixel encoded by the prediction run. Similar to the palette index map coding of FIG. 10, the indices form the palette index map 1006 which is divided into four runs and then encoded in a horizontal scan order shown as arrows. In the palette index map 1006, 16 indices are encoded by run indices 1108, four runs R0, R1, R2, and R3 with horizontal traverse scan. There are three, copy index mode, copy above mode, and prediction mode, under the run mode 1102, and for each starting position of a run, a flag is transmitted to indicate which run mode is used. If a pixel in the palette index map 1006 were identified to have been encoded by the prediction run, it would be identified as a prediction pixel. In FIG. 11, the video compression system 802 may use the prediction run, R3, for the four bottom pixels to encode as prediction pixels as indicated by a run of pixels 1110, and signal those four pixels as prediction pixels.
FIG. 12 illustrates an example system 1200 for implementing the processes and methods described above for improving decoding performance by combining the palette mode and the prediction methods.
The techniques and mechanisms described herein may be implemented by multiple instances of the system 1200 as well as by any other computing device, system, and/or environment. The system 1200 shown in FIG. 12 is only one example of a system and is not intended to suggest any limitation as to the scope of use or functionality of any computing device utilized to  perform the processes and/or procedures described above. Other well-known computing devices, systems, environments and/or configurations that may be suitable for use with the embodiments include, but are not limited to, personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, game consoles, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, implementations using field programmable gate arrays ( “FPGAs” ) and application specific integrated circuits ( “ASICs” ) , and/or the like.
The system 1200 may include one or more processors 1202 and system memory 1204 communicatively coupled to the processor (s) 1202. The processor (s) 1202 may execute one or more modules and/or processes to cause the processor (s) 1202 to perform a variety of functions. In some embodiments, the processor (s) 1202 may include a central processing unit (CPU) , a graphics processing unit (GPU) , both CPU and GPU, or other processing units or components known in the art. Additionally, each of the processor (s) 1202 may possess its own local memory, which also may store program modules, program data, and/or one or more operating systems.
Depending on the exact configuration and type of the system 1200, the system memory 1204 may be volatile, such as RAM, non-volatile, such as ROM, flash memory, miniature hard drive, memory card, and the like, or some combination thereof. The system memory 1204 may include one or more  computer-executable modules (modules) 1206 that are executable by the processor (s) 1202.
The modules 1206 may include, but are not limited to, a prediction module 1208, a residual module 1210, and a reconstruction module 1212. The prediction 1208 may be configured to derive a prediction block of a coding unit as discussed above with reference to FIG. 1. The prediction module may use a palette of a non-adjacent coding unit relative to the coding unit as a palette predictor to predict a palette table of the coding unit, and apply the predicting method, such as intra-prediction 102 or inter-prediction 108, to a palette table, such as the palette table 316 to derive a prediction block, such as the prediction block 104. The residual module 1210 may be configured to decode palette-based residual information of the coding unit and to derive a palette-based residual block, such as the palette-based residual block 804, based on the decoded palette-based residual information. The reconstruction module 1212 may be configured to combine the prediction block 104 and the palette-based residual block 804 to construct a reconstruction block, such as the reconstruction block 808, thus improving the decoding performance of the system 1200.
The residual module 1210 may be further configured to decode the palette-based residual block 804, and classify pixels of the palette-based residual block into major color pixels and prediction pixel. The major color pixels may have palette indices associated with major color pixel values of the decoded palette-based residual block 804. The residual module 1210 may be additionally configured to generate a palette index map, such as the palette index map 1006,  based on the major color pixels and prediction pixels, and assign a predetermined color index, such as C0, C1, etc. to prediction pixels as shown in FIG. 9.
The reconstruction module 1212 may be further configured to signal whether a pixel in the palette-based residual block is a major color pixel or a prediction pixel by decoding the palette index map 1006 and identifying an index having the predetermined color index as a prediction pixel as discussed above with reference to FIG. 10, or by identifying a pixel encoded by a prediction run as a prediction pixel as discussed above with reference to FIG. 11. The reconstruction module 1212 may be configured to use the major color pixel values of the major color pixels in the reconstruction block 808 and transfer pixel values of the prediction block for the prediction pixels in the reconstruction block 808, thus combining the palette mode and the traditional prediction methods to improve the decoding performance of the video compression system.
The system 1200 may additionally include an input/output (I/O) interface 1214 for receiving data associated with a coding unit, such as encoded data of the CU 302, to be processed, and for outputting the processed data of the coding unit, such as the reconstructed block 808. The system 1200 may also include a communication module 1216 allowing the system 1200 to communicate with other devices (not shown) over a network (not shown) . The network may include the Internet, wired media such as a wired network or direct-wired connections, and wireless media such as acoustic, radio frequency (RF) , infrared, and other wireless media.
Some or all operations of the methods described above can be performed by execution of computer-readable instructions stored on a computer-readable storage medium, as defined below. The term “computer-readable instructions” as used in the description and claims, include routines, applications, application modules, program modules, programs, components, data structures, algorithms, and the like. Computer-readable instructions can be implemented on various system configurations, including single-processor or multiprocessor systems, minicomputers, mainframe computers, personal computers, hand-held computing devices, microprocessor-based, programmable consumer electronics, combinations thereof, and the like.
The computer-readable storage media may include volatile memory (such as random access memory (RAM) ) and/or non-volatile memory (such as read-only memory (ROM) , flash memory, etc. ) . The computer-readable storage media may also include additional removable storage and/or non-removable storage including, but not limited to, flash memory, magnetic storage, optical storage, and/or tape storage that may provide non-volatile storage of computer-readable instructions, data structures, program modules, and the like.
A non-transient computer-readable storage medium is an example of computer-readable media. Computer-readable media includes at least two types of computer-readable media, namely computer-readable storage media and communications media. Computer-readable storage media includes volatile and non-volatile, removable and non-removable media implemented in any process or technology for storage of information such as computer-readable instructions,  data structures, program modules, or other data. Computer-readable storage media includes, but is not limited to, phase change memory (PRAM) , static random-access memory (SRAM) , dynamic random-access memory (DRAM) , other types of random-access memory (RAM) , read-only memory (ROM) , electrically erasable programmable read-only memory (EEPROM) , flash memory or other memory technology, compact disk read-only memory (CD-ROM) , digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by a computing device. In contrast, communication media may embody computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, or other transmission mechanism. As defined herein, computer-readable storage media do not include communication media.
The computer-readable instructions stored on one or more non-transitory computer-readable storage media that, when executed by one or more processors, may perform operations described above with reference to FIGs. 1-12. Generally, computer-readable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and/or in parallel to implement the processes.
EXAMPLE CLAUSES
A. A method comprising: deriving a prediction block of a coding unit; decoding palette-based residual information of the coding unit; deriving a palette-based residual block based on the decoded palette-based residual information; and combining the prediction block and the palette-based residual block to construct a reconstruction block.
B. The method as paragraph A recites, wherein deriving the prediction block of the coding unit includes: using a palette of a non-adjacent coding unit relative to the coding unit as a palette predictor to predict a palette table of the coding unit; and applying predicting coding to the palette table to derive the prediction block.
C. The method as paragraph A recites, wherein combining the prediction block and the palette-based residual block to construct a reconstruction block includes refining the prediction block based on the decoded palette-based residual information.
D. The method as paragraph C recites, wherein refining the prediction block based on the decoded palette-based residual information comprises: decoding the palette-based residual block; classifying pixels of the palette-based residual block into major color pixels and prediction pixels, the major color pixels having palette indices associated with major color pixel values of the decoded palette-based residual block; and generating a palette index map based on the major color pixels and prediction pixels.
E. The method as paragraph D recites, wherein generating the palette index map based on the major color pixels and prediction pixels includes assigning to the prediction pixels.
F. The method as paragraph E recites, further comprising signaling whether a pixel in the palette-based residual block is a major color pixel or a prediction pixel includes: decoding the palette index map and identifying an index having the predetermined color index as a prediction pixel; or identifying a pixel encoded by a prediction run as a prediction pixel.
G. The method as paragraph F recites, wherein combining the prediction block and the palette-based residual block to construct the reconstruction block comprises: using the major color pixel values of the major color pixels in the reconstruction block; and transferring pixel values of the prediction block for the prediction pixels in the reconstruction block.
H. A computer-readable storage medium storing computer-readable instructions executable by one or more processors, that when executed by the one or more processors, cause the one or more processors to perform operations comprising: deriving a prediction block of a coding unit; decoding palette-based residual information of the coding unit; deriving a palette-based residual block based on the decoded palette-based residual information; and combining the prediction block and the palette-based residual block to construct a reconstruction block.
I. The computer-readable storage medium as paragraph H recites, wherein deriving the prediction block of the coding unit includes: using  a palette of a non-adjacent coding unit relative to the coding unit as a palette predictor to predict a palette table of the coding unit; and applying predicting coding to the palette table to derive the prediction block.
J. The computer-readable storage medium as paragraph H recites, wherein combining the prediction block and the palette-based residual block to construct a reconstruction block includes: refining the prediction block based on the decoded palette-based residual information.
K. The computer-readable storage medium as paragraph J recites, wherein refining the prediction block based on the decoded palette-based residual information comprises: decoding the palette-based residual block; classifying pixels of the palette-based residual block into major color pixels and prediction pixels, the major color pixels having palette indices associated with major color pixel values of the decoded palette-based residual block; and generating a palette index map based on the major color pixels and prediction pixels.
L. The computer-readable storage medium as paragraph K recites, wherein generating the palette index map based on the major color pixels and prediction pixels includes assigning a predetermined color index to prediction pixels.
M. The computer-readable storage medium as paragraph L recites, further comprising signaling whether a pixel in the palette-based residual block is a major color pixel or a prediction pixel includes: decoding the palette index map and identifying an index having the predetermined color index as a  prediction pixel; or identifying a pixel encoded by a prediction run as a prediction pixel.
N. The computer-readable storage medium as paragraph M recites, wherein combining the prediction block and the palette-based residual block to construct the reconstruction block comprises: using the major color pixel values of the major color pixels in the reconstruction block; and transferring pixel values of the prediction block for the prediction pixels in the reconstruction block.
O. A system comprising: one or more processors; and memory communicatively coupled to the one or more processors, the memory storing computer-executable modules executable by the one or more processors, the computer-executable modules including: a prediction module configured to derive a prediction block of a coding unit; a residual module configured to decode palette-based residual information of the coding unit and to derive a palette-based residual block based on the decoded palette-based residual information; and a reconstruction module configured to combine the prediction block and the palette-based residual block to construct a reconstruction block.
P. The system as paragraph O recites, wherein the prediction block is further configured to: use a palette of a non-adjacent coding unit relative to the coding unit as a palette predictor to predict a palette table of the coding unit; and apply predicting coding to the palette table to derive the prediction block.
Q. The system of claim O, wherein the residual module is further configured: decode the palette-based residual block; classify pixels of the palette-based residual block into major color pixels and prediction pixels, the major color pixels having palette indices associated with major color pixel values of the decoded palette-based residual block; and generate a palette index map based on the major color pixels and prediction pixels.
R. The system as paragraph Q recites, wherein the residual module is further configured to assign a predetermined color index to the prediction pixels.
S. The system as paragraph R recites, wherein the reconstruction module is further configured to signal whether a pixel in the palette-based residual block is a major color pixel or a prediction pixel by: decoding the palette index map and identifying an index having the predetermined color index as a prediction pixel; or identifying a pixel encoded by a prediction run as a prediction pixel.
T. The system as paragraph S recites, wherein the reconstruction module is further configured to: use the major color pixel values of the major color pixels in the reconstruction block; and transfer pixel values of the prediction block for the prediction pixels in the reconstruction block.
CONCLUSION
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the  subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementing the claims.

Claims (20)

  1. A method comprising:
    deriving a prediction block of a coding unit;
    decoding palette-based residual information of the coding unit;
    deriving a palette-based residual block based on the decoded palette-based residual information; and
    combining the prediction block and the palette-based residual block to construct a reconstruction block.
  2. The method of claim 1, wherein deriving the prediction block of the coding unit includes:
    using a palette of a non-adjacent coding unit relative to the coding unit as a palette predictor to predict a palette table of the coding unit; and
    applying predicting coding to the palette table to derive the prediction block.
  3. The method of claim 1, wherein combining the prediction block and the palette-based residual block to construct a reconstruction block includes:
    refining the prediction block based on the decoded palette-based residual information.
  4. The method of claim 3, wherein refining the prediction block based on the decoded palette-based residual information comprises:
    decoding the palette-based residual block;
    classifying pixels of the palette-based residual block into major color pixels and prediction pixels, the major color pixels having palette indices associated with major color pixel values of the decoded palette-based residual block; and
    generating a palette index map based on the major color pixels and prediction pixels.
  5. The method of claim 4, wherein generating the palette index map based on the major color pixels and prediction pixels includes assigning to the prediction pixels.
  6. The method of claim 5, further comprising signaling whether a pixel in the palette-based residual block is a major color pixel or a prediction pixel includes:
    decoding the palette index map and identifying an index having the predetermined color index as a prediction pixel; or
    identifying a pixel encoded by a prediction run as a prediction pixel.
  7. The method of claim 6, wherein combining the prediction block and the palette-based residual block to construct the reconstruction block comprises:
    using the major color pixel values of the major color pixels in the reconstruction block; and
    transferring pixel values of the prediction block for the prediction pixels in the reconstruction block.
  8. A computer-readable storage medium storing computer-readable instructions executable by one or more processors, that when executed by the one or more processors, cause the one or more processors to perform operations comprising:
    deriving a prediction block of a coding unit;
    decoding palette-based residual information of the coding unit;
    deriving a palette-based residual block based on the decoded palette-based residual information; and
    combining the prediction block and the palette-based residual block to construct a reconstruction block.
  9. The computer-readable storage medium of claim 8, wherein deriving the prediction block of the coding unit includes:
    using a palette of a non-adjacent coding unit relative to the coding unit as a palette predictor to predict a palette table of the coding unit; and
    applying predicting coding to the palette table to derive the prediction block.
  10. The computer-readable storage medium of claim 8, wherein combining the prediction block and the palette-based residual block to construct a reconstruction block includes:
    refining the prediction block based on the decoded palette-based residual information.
  11. The computer-readable storage medium of claim 10, wherein refining the prediction block based on the decoded palette-based residual information comprises:
    decoding the palette-based residual block;
    classifying pixels of the palette-based residual block into major color pixels and prediction pixels, the major color pixels having palette indices associated with major color pixel values of the decoded palette-based residual block; and
    generating a palette index map based on the major color pixels and prediction pixels.
  12. The computer-readable storage medium of claim 11, wherein generating the palette index map based on the major color pixels and prediction pixels includes assigning a predetermined color index to prediction pixels.
  13. The computer-readable storage medium of claim 12, further comprising signaling whether a pixel in the palette-based residual block is a major color pixel or a prediction pixel includes:
    decoding the palette index map and identifying an index having the predetermined color index as a prediction pixel; or
    identifying a pixel encoded by a prediction run as a prediction pixel.
  14. The computer-readable storage medium of claim 13, wherein combining the prediction block and the palette-based residual block to construct the reconstruction block comprises:
    using the major color pixel values of the major color pixels in the reconstruction block; and
    transferring pixel values of the prediction block for the prediction pixels in the reconstruction block.
  15. A system comprising:
    one or more processors; and
    memory communicatively coupled to the one or more processors, the memory storing computer-executable modules executable by the one or more processors, the computer-executable modules including:
    a prediction module configured to derive a prediction block of a coding unit;
    a residual module configured to decode palette-based residual information of the coding unit and to derive a palette-based residual block based on the decoded palette-based residual information; and
    a reconstruction module configured to combine the prediction block and the palette-based residual block to construct a reconstruction block.
  16. The system of claim 15, wherein the prediction block is further configured to:
    use a palette of a non-adjacent coding unit relative to the coding unit as a palette predictor to predict a palette table of the coding unit; and
    apply predicting coding to the palette table to derive the prediction block.
  17. The system of claim 15, wherein the residual module is further configured:
    decode the palette-based residual block;
    classify pixels of the palette-based residual block into major color pixels and prediction pixels, the major color pixels having palette indices associated with major color pixel values of the decoded palette-based residual block; and
    generate a palette index map based on the major color pixels and prediction pixels.
  18. The system of claim 17, wherein the residual module is further configured to assign a predetermined color index to the prediction pixels.
  19. The system of claim 18, wherein the reconstruction module is further configured to signal whether a pixel in the palette-based residual block is a major color pixel or a prediction pixel by:
    decoding the palette index map and identifying an index having the predetermined color index as a prediction pixel; or
    identifying a pixel encoded by a prediction run as a prediction pixel.
  20. The system of claim 19, wherein the reconstruction module is further configured to:
    use the major color pixel values of the major color pixels in the reconstruction block; and
    transfer pixel values of the prediction block for the prediction pixels in the reconstruction block.
PCT/CN2018/094496 2018-07-04 2018-07-04 Palette-based residual coding for video compression system WO2020006707A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201880094272.9A CN112219399A (en) 2018-07-04 2018-07-04 Palette-based residual coding in video compression systems
PCT/CN2018/094496 WO2020006707A1 (en) 2018-07-04 2018-07-04 Palette-based residual coding for video compression system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/094496 WO2020006707A1 (en) 2018-07-04 2018-07-04 Palette-based residual coding for video compression system

Publications (1)

Publication Number Publication Date
WO2020006707A1 true WO2020006707A1 (en) 2020-01-09

Family

ID=69060708

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/094496 WO2020006707A1 (en) 2018-07-04 2018-07-04 Palette-based residual coding for video compression system

Country Status (2)

Country Link
CN (1) CN112219399A (en)
WO (1) WO2020006707A1 (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1716998A (en) * 2004-06-14 2006-01-04 奥林巴斯株式会社 Picture compressor and picture reproducer
US20150341643A1 (en) * 2014-05-23 2015-11-26 Futurewei Technologies, Inc. Advanced screen content coding with improved palette table and index map coding methods
CN105392010A (en) * 2014-08-25 2016-03-09 联发科技股份有限公司 Method of palette index signaling for image and video coding
US20160100174A1 (en) * 2014-10-06 2016-04-07 Futurewei Technologies, Inc. Palette Mode In High Efficiency Video Coding (HEVC) Screen Content Coding (SCC)
CN106030651A (en) * 2014-03-21 2016-10-12 华为技术有限公司 Advanced screen content coding with improved color table and index map coding methods
CN106068648A (en) * 2014-01-02 2016-11-02 高通股份有限公司 The quantization of the effusion pixel of video block in palette decoding mode

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015100522A1 (en) * 2013-12-30 2015-07-09 Mediatek Singapore Pte. Ltd. Methods for inter-component residual prediction
CN107005717B (en) * 2014-11-12 2020-04-07 寰发股份有限公司 Skip pixel coding and decoding method in index mapping coding and decoding
US10904566B2 (en) * 2015-03-18 2021-01-26 Hfi Innovation Inc. Method and apparatus for index map coding in video and image compression

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1716998A (en) * 2004-06-14 2006-01-04 奥林巴斯株式会社 Picture compressor and picture reproducer
CN106068648A (en) * 2014-01-02 2016-11-02 高通股份有限公司 The quantization of the effusion pixel of video block in palette decoding mode
CN106030651A (en) * 2014-03-21 2016-10-12 华为技术有限公司 Advanced screen content coding with improved color table and index map coding methods
US20150341643A1 (en) * 2014-05-23 2015-11-26 Futurewei Technologies, Inc. Advanced screen content coding with improved palette table and index map coding methods
CN105392010A (en) * 2014-08-25 2016-03-09 联发科技股份有限公司 Method of palette index signaling for image and video coding
US20160100174A1 (en) * 2014-10-06 2016-04-07 Futurewei Technologies, Inc. Palette Mode In High Efficiency Video Coding (HEVC) Screen Content Coding (SCC)

Also Published As

Publication number Publication date
CN112219399A (en) 2021-01-12

Similar Documents

Publication Publication Date Title
US20230056351A1 (en) Coded-block-flag coding and derivation
US10652537B1 (en) Coding unit size adaptive palette mode for video compression system
US9788004B2 (en) Method of color index coding with palette stuffing
US10951895B2 (en) Context model selection based on coding unit characteristics
US11350106B2 (en) Method for encoding and decoding images, device for encoding and decoding images and corresponding computer programs
US11943470B2 (en) Method and apparatus for inter prediction in video coding system
WO2020177545A1 (en) Palette size constraint in palette mode for video compression system
WO2020056757A1 (en) Method, apparatus, and computer-readable storage medium for block partitioning tree separation under separation node
WO2020006707A1 (en) Palette-based residual coding for video compression system
US10708625B2 (en) Adaptive deblocking filter
WO2020243881A1 (en) Simplified intra block copy
CN116648915A (en) Point cloud encoding and decoding method, encoder, decoder and computer storage medium
WO2020177118A1 (en) Subsampled residual coding for video compression system
WO2020056760A1 (en) Transform selection for small size block
WO2020056759A1 (en) Method, apparatus, and computer-readable storage medium for small size block coding
US20180070078A1 (en) Image encoding and decoding method, encoding and decoding device and corresponding computer programs
US20240022718A1 (en) Content aware dataset generation and model selection for learned image compression
WO2020056763A1 (en) Adaptive coding of video coding units based on coding block type
WO2023249999A1 (en) System and method for geometry point cloud coding
WO2024010635A1 (en) System and method for multiple-hypothesis prediction for video coding
WO2022191947A1 (en) State based dependent quantization and residual coding in video coding
CN116055734A (en) Video coding method, device, equipment and storage medium
JP2018152851A (en) Encoding device, decoding device, and program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18925605

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18925605

Country of ref document: EP

Kind code of ref document: A1