US20210368205A1 - Video coding using intra sub-partition coding mode - Google Patents
Video coding using intra sub-partition coding mode Download PDFInfo
- Publication number
- US20210368205A1 US20210368205A1 US17/393,367 US202117393367A US2021368205A1 US 20210368205 A1 US20210368205 A1 US 20210368205A1 US 202117393367 A US202117393367 A US 202117393367A US 2021368205 A1 US2021368205 A1 US 2021368205A1
- Authority
- US
- United States
- Prior art keywords
- sub
- mode
- modes
- prediction
- partition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000005192 partition Methods 0.000 title claims abstract description 185
- 238000000034 method Methods 0.000 claims abstract description 118
- 230000008569 process Effects 0.000 claims description 53
- 241000023320 Luma <angiosperm> Species 0.000 claims description 50
- OSWPMRLSEDHDFF-UHFFFAOYSA-N methyl salicylate Chemical compound COC(=O)C1=CC=CC=C1O OSWPMRLSEDHDFF-UHFFFAOYSA-N 0.000 claims description 50
- 230000004044 response Effects 0.000 claims description 36
- 238000000638 solvent extraction Methods 0.000 claims description 24
- 230000011664 signaling Effects 0.000 claims description 13
- 235000007164 Oryza sativa Nutrition 0.000 claims description 10
- 235000009566 rice Nutrition 0.000 claims description 10
- 240000007594 Oryza sativa Species 0.000 claims 1
- 230000000875 corresponding effect Effects 0.000 description 36
- 239000000523 sample Substances 0.000 description 23
- 241000209094 Oryza Species 0.000 description 9
- 238000013461 design Methods 0.000 description 9
- 230000003044 adaptive effect Effects 0.000 description 8
- 230000002123 temporal effect Effects 0.000 description 8
- 239000013074 reference sample Substances 0.000 description 7
- 238000013139 quantization Methods 0.000 description 6
- 230000006835 compression Effects 0.000 description 5
- 238000007906 compression Methods 0.000 description 5
- 238000009795 derivation Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000003993 interaction Effects 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 5
- 238000012935 Averaging Methods 0.000 description 4
- 230000001419 dependent effect Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 238000013459 approach Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000001914 filtration Methods 0.000 description 3
- 238000004590 computer program Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010187 selection method Methods 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 244000233534 Eugenia apiculata Species 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/513—Processing of motion vectors
- H04N19/517—Processing of motion vectors by encoding
- H04N19/52—Processing of motion vectors by encoding by predictive encoding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/103—Selection of coding mode or of prediction mode
- H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/103—Selection of coding mode or of prediction mode
- H04N19/11—Selection of coding mode or of prediction mode among a plurality of spatial predictive coding modes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/119—Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
- H04N19/96—Tree coding, e.g. quad-tree coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
- H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/46—Embedding additional information in the video signal during the compression process
Definitions
- the present disclosure relates generally to video coding and compression. More specifically, this disclosure relates to systems and methods for performing video coding using intra sub-partition coding mode.
- Video coding can be performed according to one or more video coding standards.
- Some illustrative video coding standards include versatile video coding (VVC), joint exploration test model (JEM) coding, high-efficiency video coding (H.265/HEVC), advanced video coding (H.264/AVC), and moving picture experts group (MPEG) coding.
- Video coding generally utilizes predictive methods (e.g., inter-prediction, intra-prediction, or the like) that take advantage of redundancy inherent in video images or sequences.
- predictive methods e.g., inter-prediction, intra-prediction, or the like
- One goal of video coding techniques is to compress video data into a form that uses a lower bit rate, while avoiding or minimizing degradations to video quality.
- the first version of the HEVC standard offers an approximately 50% bit-rate saving or equivalent perceptual quality, compared to the prior-generation video coding standard (H.264/MPEG AVC).
- H.264/MPEG AVC prior-generation video coding standard
- VCEG Video Coding Experts Group
- MPEG Moving Picture Experts Group
- JVET Joint Video Exploration Team
- VVC Versatile Video Coding
- a video coding method is performed at a computing device having one or more processors and memory storing a plurality of programs to be executed by the one or more processors.
- the method includes independently generating a respective intra prediction for each of a plurality of corresponding sub-partitions, wherein each respective intra prediction is generated using a plurality of reference samples from a current coding block.
- a video coding method is performed at a computing device having one or more processors and memory storing a plurality of programs to be executed by the one or more processors.
- the method includes generating a respective intra prediction for each of a plurality of corresponding sub-partitions using only N modes out of M possible intra prediction modes for a luma component of an intra sub partition (ISP) coded block, wherein M and N are positive integers and N is less than M.
- ISP intra sub partition
- a video coding method is performed at a computing device having one or more processors and memory storing a plurality of programs to be executed by the one or more processors.
- the method includes generating an intra prediction using only N modes out of M possible intra prediction modes for a chroma component of an intra sub partition (ISP) coded block, wherein M and N are positive integers and N is less than M.
- ISP intra sub partition
- a video coding method is performed at a computing device having one or more processors and memory storing a plurality of programs to be executed by the one or more processors.
- the method includes generating a respective luma intra prediction for each of a plurality of corresponding sub-partitions of an entire intra sub partition (ISP) coded block for a luma component, and for a chroma component, generating a chroma intra prediction for the entire intra sub partition (ISP) coded block.
- ISP intra sub partition
- a video coding method is performed at a computing device having one or more processors and memory storing a plurality of programs to be executed by the one or more processors.
- the method includes generating a first prediction using an intra sub partition mode; generating a second prediction using an inter prediction mode; and combining the first and second predictions to generate a final prediction by applying a weighted averaging to the first prediction and the second prediction.
- FIG. 1 is a block diagram setting forth an illustrative encoder which may be used in conjunction with many video coding standards.
- FIG. 2 is a block diagram setting forth an illustrative decoder which may be used in conjunction with many video coding standards.
- FIG. 3 illustrates five types of exemplary block partitions for a multi-type tree structure.
- FIG. 4 illustrates a set of exemplary intra modes for use with the VVC standard.
- FIG. 5 illustrates a set of multiple reference lines for performing intra prediction.
- FIG. 6A illustrates a first set of reference samples and angular directions that are used for performing an intra prediction of a first rectangular block.
- FIG. 6B illustrates a second set of reference samples and angular directions that are used for performing an intra prediction of a second rectangular block.
- FIG. 6C illustrates a third set of reference samples and angular directions that are used for performing an intra prediction of a square block.
- FIG. 7 illustrates a set of exemplary locations for neighboring reconstructed samples that are used for a position-dependent intra prediction combination (PDPC) of one coding block.
- PDPC position-dependent intra prediction combination
- FIG. 8A illustrates an exemplary set of short-distance intra prediction (SDIP) partitions for an 8 ⁇ 4 block.
- SDIP short-distance intra prediction
- FIG. 8B illustrates an exemplary set of short-distance intra prediction (SDIP) partitions for a 4 ⁇ 8 block.
- SDIP short-distance intra prediction
- FIG. 8C illustrates an exemplary set of short-distance intra prediction (SDIP) partitions for an arbitrarily-sized block.
- SDIP short-distance intra prediction
- FIG. 9A is a plot of chroma values as a function of luma values, where the plot is used to derive a set of linear model parameters.
- FIG. 9B shows the locations of samples that are used for the derivation of the linear model parameters of FIG. 9A .
- FIG. 10 illustrates the generation of reference samples for intra prediction for all sub-partitions using only reference samples outside of a current coding block.
- FIG. 11 illustrates a combining of inter predictor samples and intra predictor samples for a first sub-partition of FIG. 10 .
- FIG. 12 illustrates a combining of inter predictor samples and intra predictions samples for a second sub-partition of FIG. 10 .
- first, second, third, etc. may be used herein to describe various information, the information should not be limited by these terms. These terms are only used to distinguish one category of information from another. For example, without departing from the scope of the present disclosure, first information may be termed as second information; and similarly, second information may also be termed as first information. As used herein, the term “if” may be understood to mean “when” or “upon” or “in response to,” depending on the context.
- video coding standards are similar, including those previously mentioned in the Background section.
- many video coding standards use block-based processing, and share similar video coding block diagrams to achieve video compression.
- the VVC standard is built upon the block-based hybrid video coding framework.
- FIG. 1 shows a block diagram of an illustrative encoder 100 which may be used in conjunction with many video coding standards.
- a video frame is partitioned into a plurality of video blocks for processing.
- a prediction is formed based on either an inter prediction approach or an intra prediction approach.
- inter prediction one or more predictors are formed through motion estimation and motion compensation, based on pixels from previously reconstructed frames.
- intra prediction predictors are formed based on reconstructed pixels in a current frame. Through mode decision, a best predictor may be chosen to predict a current block.
- a prediction residual representing the difference between a current video block and its predictor, is sent to a Transform 102 circuitry.
- Transform coefficients are then sent from the Transform 102 circuitry to a Quantization 104 circuitry for entropy reduction.
- Quantized coefficients are then fed to an Entropy Coding 106 circuitry to generate a compressed video bitstream.
- prediction-related information 110 from an inter prediction circuitry and/or an Intra Prediction 112 circuitry such as video block partition info, motion vectors, reference picture index, and intra prediction mode, are also fed through the Entropy Coding 106 circuitry and saved into a compressed video bitstream 114 .
- decoder-related circuitries are also needed in order to reconstruct pixels for the purpose of prediction.
- a prediction residual is reconstructed through an Inverse Quantization 116 circuitry and an Inverse Transform 118 circuitry. This reconstructed prediction residual is combined with a Block Predictor 120 to generate un-filtered reconstructed pixels for a current video block.
- an In-Loop Filter 115 is commonly used.
- a deblocking filter is available in AVC, HEVC, as well as the current version of VVC.
- SAO sample adaptive offset
- SAO sample adaptive offset
- ALF adaptive loop filter
- These in-loop filter operations are optional. Performing these operations helps to improve coding efficiency and visual quality. They may also be turned off as a decision rendered by the encoder 100 to save computational complexity.
- intra prediction is usually based on unfiltered reconstructed pixels, while inter prediction is based on filtered reconstructed pixels if these filter options are turned on by the encoder 100 .
- FIG. 2 is a block diagram setting forth an illustrative decoder 200 which may be used in conjunction with many video coding standards. This decoder 200 is similar to the reconstruction-related section residing in the encoder 100 of FIG. 1 .
- an incoming video bitstream 201 is first decoded through an Entropy Decoding 202 circuitry to derive quantized coefficient levels and prediction-related information.
- the quantized coefficient levels are processed through an Inverse Quantization 204 circuitry and an Inverse Transform 206 circuitry to obtain a reconstructed prediction residual.
- a block predictor mechanism implemented in an Intra/inter Mode Selector 212 , is configured to perform either an Intra Prediction 208 procedure, or a Motion Compensation 210 procedure, based on decoded prediction information.
- a set of unfiltered reconstructed pixels are obtained by summing up the reconstructed prediction residual from the Inverse Transform 206 circuitry and a predictive output generated by the block predictor mechanism, using a Summer 214 .
- the reconstructed block may further go through an In-Loop Filter 209 before it is stored in a Picture Buffer 213 which functions as a reference picture store.
- the reconstructed video in the Picture Buffer 213 can then be sent out to drive a display device, as well as used to predict future video blocks.
- a filtering operation is performed on these reconstructed pixels to derive a final reconstructed Video Output 222 .
- an input video signal to the encoder 100 is processed block by block.
- Each block is called a coding unit (CU)).
- a CU can be up to 128 ⁇ 128 pixels.
- HEVC High Efficiency Video Encoding
- JEM Joint Exploration Test Model
- VVC Versatile Video Encoding
- the basic unit for compression is termed a coding tree unit (CTU).
- CTU coding tree unit
- one coding tree unit (CTU) is split into CUs to adapt to varying local characteristics based on a quad/binary/ternary-tree structure.
- the concept of multiple partition unit type in the HEVC standard is not present in the VVC standard, i.e., the separation of CU, prediction unit (PU) and transform unit (TU) does not exist in the VVC standard; instead, each CU is always used as the basic unit for both prediction and transform without further partitions.
- the maximum CTU size for HEVC and JEM is defined as being up to 64 by 64 luma pixels, and two blocks of 32 by 32 chroma pixels, for a 4:2:0 chroma format.
- the maximum allowed size of the luma block in a CTU is specified to be 128 ⁇ 128 (although the maximum size of the luma transform block is 64 ⁇ 64).
- FIG. 3 illustrates five types of exemplary block partitions for a multi-type tree structure.
- the five types of exemplary block partitions include quaternary partitioning 301 , horizontal binary partitioning 302 , vertical binary partitioning 303 , horizontal ternary partitioning 304 , and vertical ternary partitioning 305 .
- one CTU is first partitioned by a quad-tree structure. Then, each quad-tree leaf node can be further partitioned by a binary and ternary tree structure.
- spatial prediction and/or temporal prediction may be performed using the configuration shown in FIG. 1 .
- Spatial prediction (or “intra prediction”) uses pixels from the samples of already-coded neighboring blocks (which are called reference samples) in the same video picture/slice to predict the current video block. Spatial prediction reduces spatial redundancy inherent in the video signal.
- Temporal prediction (also referred to as “inter prediction” or “motion compensated prediction”) uses reconstructed pixels from already-coded video pictures to predict the current video block. Temporal prediction reduces temporal redundancy inherent in the video signal. Temporal prediction signal for a given CU is usually signaled by one or more motion vectors (MVs) which indicate the amount and the direction of motion between the current CU and its temporal reference. Also, if multiple reference pictures are supported, one reference picture index is additionally sent, which is used to identify from which reference picture in the reference picture store the temporal prediction signal comes.
- MVs motion vectors
- an intra/inter mode decision 121 circuitry in the encoder 100 chooses the best prediction mode, for example based on the rate-distortion optimization method.
- the block predictor 120 is then subtracted from the current video block; and the resulting prediction residual is de-correlated using the transform 102 circuitry and the quantization 104 circuitry.
- the resulting quantized residual coefficients are inverse quantized by the inverse quantization 116 circuitry and inverse transformed by the inverse transform 118 circuitry 118 to form the reconstructed residual, which is then added back to the prediction block to form the reconstructed signal of the CU.
- in-loop filtering 115 such as a deblocking filter, a sample adaptive offset (SAO), and/or an adaptive in-loop filter (ALF) may be applied on the reconstructed CU before it is put in the reference picture store of the picture buffer 117 and used to code future video blocks.
- coding mode inter or intra
- prediction mode information motion information
- quantized residual coefficients are all sent to the entropy coding unit 106 to be further compressed and packed to form the bit-stream.
- the basic intra prediction scheme applied in the VVC standard is kept generally the same as that of the HEVC standard, except that several modules are further extended and/or improved in the VVC standard, such as intra sub-partition (ISP) coding mode, extended intra prediction with wide-angle intra directions, position-dependent intra prediction combination (PDPC) and 4-tap intra interpolation.
- ISP intra sub-partition
- PDPC position-dependent intra prediction combination
- 4-tap intra interpolation 4-tap intra interpolation.
- the VVC standard uses a set of previously-decoded samples that neighbor one current CU (i.e., above or left) to predict the samples of the CU.
- the amount of angular intra modes is extended from 33 modes in the HEVC standard to 93 modes in the VVC standard.
- both the HEVC standard and the VVC standard provide for a Planar mode (which assumes a gradual changing surface with a horizontal and a vertical slope derived from boundaries), and a DC mode (which assumes a flat surface).
- FIG. 4 illustrates a set of exemplary intra modes 400 for use with the VVC standard
- FIG. 5 illustrates a set of multiple reference lines for performing intra prediction.
- the set of exemplary intra modes 400 includes modes 0, 1, ⁇ 14, ⁇ 12, ⁇ 10, ⁇ 8, ⁇ 6, ⁇ 4, ⁇ 2, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46, 48, 50, 52, 54, 56, 58, 60, 62, 64, 66, 68, 70, 72, 74, 76, 78, and 80.
- Mode 0 corresponds to the Planar mode
- mode 1 corresponds to the DC mode.
- all of the defined intra modes i.e., planar, DC and angular directions
- the VVC standard utilize a set of neighboring reconstructed samples above and left to the predicted block as the reference for intra prediction.
- a multi reference line MRL is introduced in the VVC where two additional rows/columns (i.e., line 1 , 503 and line 3 , 505 of FIG. 5 ) are used for the intra prediction process.
- the index of the selected reference row/column is signaled from the encoder 100 ( FIG.
- FIG. 6A illustrates a first set of reference samples and angular directions 602 , 604 that are used for an intra prediction of a rectangular block (width W divided by height H equals 2).
- the first set of reference samples includes a first sample 601 and a second sample 603 .
- the second set of reference samples includes a third sample 605 and a fourth sample 607 .
- the third set of reference samples includes a fifth sample 609 and a sixth sample 610 .
- FIG. 6C illustrates the locations of the third reference samples that can be used in the VVC standard to derive the predicted samples of one intra block.
- FIG. 6C because the quad/binary/ternary tree partition structure is applied, in addition to the coding blocks of square shape, rectangular coding blocks also exist for the intra prediction procedure in the context of the VVC standard.
- the intra prediction samples are generated from either a non-filtered or a filtered set of neighboring reference samples, which may introduce discontinuities along the block boundaries between the current coding block and its neighbors.
- boundary filtering is applied in the HEVC standard by combing the first row/column of prediction samples of DC, horizontal (i.e., mode 18 of FIG. 4 ) and vertical (i.e., mode 50) prediction modes with the unfiltered reference samples, utilizing a 2-tap filter (for DC mode) or a gradient-based smoothing filter (for horizontal and vertical prediction modes).
- the position-dependent intra prediction combination (PDPC) tool in the VVC standard extends the foregoing concept by employing a weighted combination of intra prediction samples with unfiltered reference samples.
- the PDPC is enabled for the following intra modes without signaling: planar, DC, horizontal (i.e, mode 18), vertical (i.e., mode 50), angular directions close to the bottom-left diagonal directions (i.e., modes 2, 3, 4, . . . , 10) and angular directions close to the top-right diagonal directions (i.e., modes 58, 59, 60, . . . , 66).
- planar DC
- horizontal i.e, mode 18
- vertical i.e., mode 50
- angular directions close to the bottom-left diagonal directions i.e., modes 2, 3, 4, . . . , 10)
- angular directions close to the top-right diagonal directions i.e., modes 58, 59, 60, . . . , 66
- pred( x,y ) ( wL ⁇ R ⁇ 1,y +wT ⁇ R x, ⁇ 1 ⁇ wTL ⁇ R ⁇ 1, ⁇ 1 +(64 ⁇ wL ⁇ wT+wTL ) ⁇ pred( x,y )+32)>>6 (1)
- R x, ⁇ 1 , R ⁇ 1,y represent the reference samples located at the top and left of current sample (x, y), respectively, and R ⁇ 1, ⁇ 1 represents the reference sample located at the top-left corner of the current block.
- FIG. 7 illustrates a set of exemplary locations for neighboring reconstructed samples that are used for a position-dependent intra prediction combination (PDPC) of one coding block.
- a first reference sample 701 (R x, ⁇ 1 ), represents a reference sample located at the top of a current prediction sample (x, y).
- a second reference sample 703 (R ⁇ 1,y ) represents a reference sample located to the left of the current prediction sample (x, y).
- a third reference sample 705 (R ⁇ 1, ⁇ 1 ) represents the reference sample located at the top-left corner of the current prediction sample (x, y).
- Reference samples including the first, second, and third reference samples 701 , 703 and 705 are combined with the current prediction sample (x, y) during the PDPC process.
- the weights wL, wT and wTL in equation (1) are adaptively selected depending on prediction mode and sample position, as described as follows where the current coding block is assumed to be in the size of W ⁇ H:
- multiple transform selection (MTS) tool is enabled in the VVC standard by introducing additional core transforms of DCT-VIII and DST-VII.
- MTS multiple transform selection
- the adaptive selection of the transforms is enabled at the coding block level by signaling one MTS flag to a bitstream. Specifically, when the MTS flag is equal to 0 for one block, one pair of fixed transforms (e.g., DCT-II) are applied in the horizontal and vertical directions. Otherwise (when the MTS flag is equal to 1), two additional flags will be further signaled for the block to indicate the transform type (either DCT-VIII or DST-VII) for each direction.
- DCT-II fixed transforms
- one shape-adaptive transform selection method is applied to all intra-coded blocks in which the DCT-II and DST-VII transforms are implicitly enabled based on the width and height of the current block. More specifically, for each rectangular block, the method uses the DST-VII transform in the direction associated with shorter side of one block and DCT-II transform in the direction associated with longer side of the block. For each square block, the DST-VII is applied in both directions. Additionally, to avoid introducing new transforms in different block sizes, the DST-VII transform is only enabled when the shorter side of one intra-coded block is equal to or smaller than 16. Otherwise, the DCT-II transform is always applied.
- Table 2 illustrates the enabled horizontal and vertical transforms for intra-coded blocks based on the shape adaptive transform selection method in the VVC.
- an 8 ⁇ 8 block may be divided into four 2 ⁇ 8 or four 8 ⁇ 2 sub-blocks.
- One extreme case of such sub-block based intra prediction is so-called line-based prediction, wherein a block is divided into 1-D line/column for prediction.
- one W ⁇ H (width ⁇ height) block can be split either into H sub-blocks in size of W ⁇ 1 or into W sub-blocks in size of 1 ⁇ H for intra prediction.
- Each of the resulting lines/columns are coded in the same way of a normal 2-dimension (2-D) block (as shown in FIGS.
- FIG. 8A illustrates an exemplary set of short-distance intra prediction (SDIP) partitions for an 8 ⁇ 4 block 801
- FIG. 8B illustrates an exemplary set of short-distance intra prediction (SDIP) partitions for a 4 ⁇ 8 block 803
- FIG. 8C illustrates an exemplary set of short-distance intra prediction (SDIP) partitions for an arbitrarily-sized block 805 .
- a video coding tool called sub-partition prediction (ISP) was introduced into the VVC standard.
- ISP sub-partition prediction
- ISP is very similar to SDIP. Specifically, depending on the block size, the ISP divides the current coding block into 2 or 4 sub-blocks in either a horizontal or vertical direction, and each sub-block contains at least 16 samples.
- FIGS. 8A, 8B, and 8C illustrate all of the possible partition cases for different coding block sizes. Moreover, the following main aspects are also included in the current ISP design to handle its interaction with the other coding tools in the VVC standard:
- FIG. 9A is a plot of chroma values as a function of luma values, where the plot is used to derive a set of linear model parameters. More specifically, a straight-line 901 relationship between chroma values and luma values is used to derive a set of linear model parameters ⁇ and ⁇ as follows.
- a cross-component linear model (CCLM) prediction mode is used in VVC, for which the chroma samples are predicted based on the reconstructed luma samples of the same CU by using a linear model as follows:
- Linear model parameters ⁇ and ⁇ are derived from the straight-line 901 relationship between luma values and chroma values from two samples, which are minimum luma sample A (X A , Y A ) and maximum luma sample B (X B , Y B ) inside the set of neighboring luma samples, as exemplified in FIG. 9A .
- X A , Y A are the x-coordinate (i.e. luma value) and y-coordinate (i.e. chroma value) value for sample A
- X B , Y B are the x-coordinate and y-coordinate value for sample B.
- the linear model parameters ⁇ and ⁇ are obtained according to the following equations.
- Such a method is also called a min-Max method.
- the division in the equation above could be avoided and replaced by a multiplication and a shift.
- FIG. 9B shows the locations of the samples that are used for the derivation of the linear model parameters of FIG. 9A .
- the above two equations for the linear model parameters ⁇ and ⁇ are applied directly.
- the neighboring samples of the longer boundary are first subsampled to have the same number of samples as for the shorter boundary.
- FIG. 9B shows the locations of the left and above samples and the sample of the current block involved in the CCLM mode, including an N ⁇ N set of chroma samples 903 and a 2N ⁇ 2N set of luma samples 905 .
- these templates also can be used alternatively in the other 2 LM modes, called LM_A, and LM_L modes.
- LM_A mode only pixel samples in the above template are used to calculate the linear model coefficients. To get more samples, the above template is extended to (W+W). In LM_L mode, only pixel samples in the left template are used to calculate the linear model coefficients. To get more samples, the left template are extended to (H+H). Note that only one luma line (general line buffer in intra prediction) is used to make the down-sampled luma samples when the upper reference line is at the CTU boundary.
- Chroma mode signaling and derivation process are shown in Table 4.
- Chroma mode coding directly depends on the intra prediction mode of the corresponding luma block. Since separate block partitioning structure for luma and chroma components is enabled in I slices, one chroma block may correspond to multiple luma blocks. Therefore, for Chroma DM mode, the intra prediction mode of the corresponding luma block covering the center position of the current chroma block is directly inherited.
- VVC VVC
- ISP tool in VVC can enhance intra prediction efficiency, there is room to further improve the performance of VVC.
- some parts of the existing ISP would benefit from further simplification to provide for more efficient codec hardware implementations, and/or to provide improved coding efficiency.
- several methods are proposed to further improve the ISP coding efficiency, to simplify the existing ISP design, and/or to facilitate improved hardware implementations.
- FIG. 10 illustrates the generation of reference samples for intra prediction 1007 for all sub-partitions using only reference samples outside of a current coding block 1000 .
- the current coding block 1000 includes a first sub-partition 1 1001 , a second sub-partition 2 1002 , a third sub-partition 3 1003 , and a fourth sub-partition 4 1004 .
- it is proposed to generate an intra prediction for each sub-partition 1001 , 1002 , 1003 , and 1004 independently.
- all of the predictors for the sub-partitions 1001 , 1002 , 1003 , and 1004 can be generated in a parallel manner.
- the predictors for all the sub-partitions are generated using the same approaches as used in the conventional non-sub-partition intra modes. Specifically, no reconstructed samples of one sub-partition are used to generate the intra prediction samples for any other sub-partition in the same coding unit; all the predictors of each sub-partition 1001 , 1002 , 1003 , 1004 are generated using the reference samples of the current coding block 1000 as shown in FIG. 10 .
- the width of each sub-partition can be smaller than or equal to two.
- One detailed example is as follows. Pursuant to the ISP mode in the VVC standard, the dependence of 2 ⁇ N (width by height) subblock prediction on reconstructed values of previously decoded 2 ⁇ N subblocks of the coding block is not allowed so that the minimum width for prediction for subblocks becomes four samples. For example, an 8 ⁇ 8 coding block that is coded using ISP with vertical split is split into four prediction regions each of size 2 ⁇ 8, and the left two 2 ⁇ 8 prediction regions are merged into a first 4 ⁇ 8 prediction region to perform intra prediction. Transform 102 ( FIG. 1 ) circuitry is applied to each 2 ⁇ 8 partition.
- the right two 2 ⁇ 8 prediction regions are merged into a second 4 ⁇ 8 prediction region to perform intra prediction.
- Transform 102 circuitry is applied to each 2 ⁇ 8 partition. It is noted that the first 4 ⁇ 8 prediction region uses the neighboring pixels of the current coding block to generate the intra predictors, while the second 4 ⁇ 8 region uses the reconstructed pixels from the first 4 ⁇ 8 region (located to the left of the second 4 ⁇ 8 region) or the neighboring pixels from the current coding block (located to the top of the second 4 ⁇ 8).
- only the horizontal (HOR) prediction mode (shown as mode 18 in FIG. 4 ), as well as those prediction modes having mode indices smaller than 18 (as indicated in FIG. 4 ) may be used in forming the intra prediction of the horizontal sub-partitions; and only the vertical (VER) prediction mode (i.e. mode 50 in FIG. 4 ), and those prediction modes having mode indices larger than 50 (as indicated in FIG. 4 ) may be used in forming the intra prediction of the vertical sub-partitions.
- the HOR prediction mode shown as mode 18 in FIG. 4
- the intra prediction for each horizontal sub-partition can be performed independently and in parallel.
- the VER prediction mode mode 50 in FIG. 4
- all of the angular prediction modes with modes larger than 50 the intra prediction for each vertical sub-partition can be performed independently and in parallel.
- N being a positive integer.
- this single allowed mode may be Planar mode.
- this single allowed mode may be DC mode.
- this single allowed mode may be one of HOR prediction mode, VER prediction mode, or diagonal (DIA) mode (mode 34 in FIG. 4 ) intra prediction mode.
- only one mode is allowed for the luma component of an ISP coded block, and this mode may be different according to the sub-partition orientation, i.e. whether it is a horizontal or vertical sub-partition.
- this mode may be different according to the sub-partition orientation, i.e. whether it is a horizontal or vertical sub-partition.
- only the HOR prediction mode is allowed for the horizontal sub-partitioning, while only the VER prediction mode is allowed for the vertical sub-partitioning.
- only the VER prediction mode is allowed for the horizontal sub-partitioning, while only the HOR prediction mode is allowed for the vertical sub-partitioning.
- only two modes are allowed for the luma component of an ISP-coded block.
- Each of respective modes may be selected in response to a corresponding sub-partition orientation, i.e. whether the orientation of the sub-partition is horizontal or vertical.
- a corresponding sub-partition orientation i.e. whether the orientation of the sub-partition is horizontal or vertical.
- PLANAR and HOR prediction modes are allowed for the horizontal sub-partitioning
- PLANAR and VER prediction modes are allowed for the vertical sub-partitioning.
- the codewords may be generated using any of a variety of different processes, including a Truncated Binary (TB) binarization process, a Fixed-length binarization process, a Truncated Rice (TR) binarization process, a k-th order Exp-Golomb binarization process, a Limited EGk binarization process, etc. These binary codeword generation processes are well defined in the specification of HEVC.
- the truncated Rice with Rice parameter equal to zero is also known as truncated unary binarization.
- Table 5 One set of examples of codewords using different binarization methods is presented in Table 5.
- the MPM derivation process for the regular intra mode is directly re-used for ISP mode, and the signaling methods of the MPM flag and MPM index are kept the same as it is in the existing ISP design.
- N c is a positive integer.
- N c is a positive integer.
- this single allowed mode may be a direct mode (DM).
- this single allowed mode may be an LM mode.
- this single allowed mode may be one of the HOR prediction mode or the VER prediction mode.
- the direct mode (DM) is configured to apply the same intra prediction mode which is used by the corresponding luma block to the chroma blocks.
- only two modes are allowed for the chroma component of an ISP coded block.
- only DM and LM are allowed for the chroma components of an ISP coded block.
- only four modes are allowed for the chroma component of an ISP coded block.
- only DM, LM, LM_L and LM_A are allowed for the chroma components of an ISP coded block.
- the convention MPM mechanism is not utilized. Instead, fixed binary codewords are used to indicate the selected chroma modes in the bitstream.
- the chroma intra prediction modes for an ISP coded block can be signaled using the determined binary codewords.
- the codewords may be generated using different processes including the Truncated Binary (TB) binarization process, the Fixed-length binarization process, the truncated Rice (TR) binarization process, the k-th order Exp-Golomb binarization process, the Limited EGk binarization process, etc.
- FIG. 11 illustrates a combining of inter predictor samples and intra predictor samples for the first sub-partition 1001 of FIG. 10 .
- FIG. 12 illustrates a combining of inter predictor samples and intra predictions samples for the second sub-partition 1002 of FIG. 10 .
- a new prediction mode is provided where the prediction is generated as a weighted combination (e.g. weighted averaging) of the ISP mode and the inter prediction mode.
- the intra predictor generation for ISP mode is the same as illustrated previously in connection with FIG. 10 .
- the inter predictor may be generated through the process of merge mode or inter mode.
- the inter predictor samples 1101 for a current block is generated by performing motion compensation using the merge candidates indicated by the merge index.
- the intra predictor samples 1103 are generated by performing the intra prediction using the signaled intra mode. It is noted that this process may use the reconstructed samples of the previous sub-partition to generate the intra predictor samples for a non-first sub-partition (i.e., the second sub-partition 1002 ) as shown in FIG. 12 . After the inter and intra predictor samples are generated, they are weight-averaged to generate the final predictor samples for the sub-partition.
- the combined modes can be treated as an intra mode. Alternatively, the combined modes may be treated as an inter mode or a merge mode instead of intra mode.
- the value of CBF for the last sub-partition is always inferred as one.
- the value of CBF for the last sub-partition is always inferred as zero.
- a method of video coding comprises independently generating a respective intra prediction for each of a plurality of corresponding sub-partitions, wherein each respective intra prediction is generated using a plurality of reference samples from a current coding block.
- no reconstructed sample from a first sub-partition of the plurality of corresponding sub-partitions is used to generate a respective intra prediction for any other sub-partition of the plurality of corresponding sub-partitions.
- each of the plurality of corresponding sub-partitions has a width less than or equal to 2.
- the plurality of corresponding sub-partitions comprises a plurality of vertical sub-partitions and a plurality of horizontal sub-partitions
- the method further comprises using only a horizontal prediction mode to generate a first set of intra predictions for the plurality of horizontal subdivisions, and using only a vertical prediction mode to generate a second set of intra predictions for the plurality of vertical sub-partitions.
- the horizontal prediction mode is performed using a mode index smaller than 18.
- the vertical prediction mode is performed using a mode index larger than 50.
- the horizontal prediction mode is performed for each of the plurality of horizontal sub-partitions independently and in parallel.
- the vertical prediction mode is performed for each of the plurality of vertical sub-partitions independently and in parallel.
- the plurality of corresponding sub-partitions includes a last sub-partition
- the method further comprises signaling a coefficients block flag (CBF) value for the last sub-partition.
- CBF coefficients block flag
- the plurality of corresponding sub-partitions includes a last sub-partition
- the method further comprises inferring a coefficients block flag (CBF) value for the last sub-partition at a decoder.
- CBF coefficients block flag
- the coefficients block flag (CBF) value is always inferred as one.
- the coefficients block flag (CBF) value is always inferred as zero.
- a method of video coding comprises generating a respective intra prediction for each of a plurality of corresponding sub-partitions using only N modes out of M possible intra prediction modes for a luma component of an intra sub partition (ISP) coded block, wherein M and N are positive integers and N is less than M.
- ISP intra sub partition
- each of the plurality of corresponding sub-partitions has a width less than or equal to 2.
- N is equal to one, such that only a single mode is allowed for the luma component.
- the single mode is a planar mode.
- the single mode is a DC mode.
- the single mode is any one of following modes: a horizontal (HOR) prediction mode, a vertical (VER) prediction mode, or a diagonal (DIA) prediction mode.
- HOR horizontal
- VER vertical
- DIA diagonal
- the single mode is selected in response to a sub-partition orientation, wherein the horizontal (HOR) prediction mode is selected in response to the sub-partition orientation being horizontal, and wherein the vertical (VER) prediction mode is selected in response to the sub-partition orientation being vertical.
- the single mode is selected in response to a sub-partition orientation, wherein the horizontal (HOR) prediction mode is selected in response to the sub-partition orientation being vertical, and wherein the vertical (VER) prediction mode is selected in response to the sub-partition orientation being horizontal.
- N is equal to two, such that two modes are allowed for the luma component.
- a first set of two modes are selected in response to a first sub-partition orientation, and a second set of two modes are selected in response to a second sub-partition orientation.
- the first set of two modes comprises a planar mode and a horizontal (HOR) prediction mode
- the first sub-partition orientation comprises horizontal sub-partitioning
- the second set of two modes comprises a planar mode and a vertical (VER) prediction mode
- the second sub-partition orientation comprises vertical sub-partitioning
- each respective mode of the N modes is signaled using a corresponding binary codeword from a set of predetermined binary codewords.
- the set of predetermined binary codewords is generated using at least one of a truncated binary (TB) binarization process, a fixed-length binarization process, a truncated Rice (TR) binarization process, a truncated unary binarization process, a k-th order Exp-Golumb binarization process, or a limited EGk binarization process.
- TB truncated binary
- TR truncated Rice
- a method of video coding comprises generating an intra prediction using only N modes out of M possible intra prediction modes for a chroma component of an intra sub partition (ISP) coded block, wherein M and N are positive integers and N is less than M.
- ISP intra sub partition
- N is equal to one, such that only a single mode is allowed for the chroma component.
- the single mode is direct mode (DM), linear model (LM) mode, horizontal (HOR) prediction mode, or vertical (VER) prediction mode.
- N is equal to two, such that two modes are allowed for the chroma component.
- N is equal to four, such that four modes are allowed for the chroma component.
- each respective mode of the N modes is signaled using a corresponding binary codeword from a set of predetermined binary codewords.
- the set of predetermined binary codewords is generated using at least one of a truncated binary (TB) binarization process, a fixed-length binarization process, a truncated Rice (TR) binarization process, a truncated unary binarization process, a k-th order Exp-Golumb binarization process, or a limited EGk binarization process.
- TB truncated binary
- TR truncated Rice
- a method of video coding comprises generating a respective luma intra prediction for each of a plurality of corresponding sub-partitions of an entire intra sub partition (ISP) coded block for a luma component, and for a chroma component, generating a chroma intra prediction for the entire intra sub partition (ISP) coded block.
- ISP intra sub partition
- each of the plurality of corresponding sub-partitions has a width less than or equal to 2.
- a method of video coding comprises generating a first prediction using an intra sub partition mode; generating a second prediction using an inter prediction mode; and combining the first and second predictions to generate a final prediction by applying a weighted averaging to the first prediction and the second prediction.
- the second prediction is generated using at least one of a merge mode or an inter mode.
- a method of video decoding comprises independently performing an intra prediction for a plurality of corresponding sub-partitions, wherein the intra prediction is performed using a plurality of reference samples for a current coding block.
- no reconstructed sample from a first sub-partition of the plurality of corresponding sub-partitions is used to perform the intra prediction for any other sub-partition of the plurality of corresponding sub-partitions.
- each of the plurality of corresponding sub-partitions has a width less than or equal to 2.
- the plurality of corresponding sub-partitions are a plurality of vertical sub-partitions.
- the method of video decoding further comprises generating a set of prediction samples for the plurality of vertical sub-partitions using a vertical prediction mode with a mode index larger than 50.
- the method of video decoding further comprises performing the intra prediction for the plurality of vertical sub-partitions independently and in parallel.
- the plurality of corresponding sub-partitions includes a last sub-partition
- the method further comprises signaling a coefficients block flag (CBF) value for the last sub-partition.
- CBF coefficients block flag
- the plurality of corresponding sub-partitions includes a last sub-partition
- the method further comprises inferring a coefficients block flag (CBF) value for the last sub-partition at a decoder.
- CBF coefficients block flag
- the coefficients block flag (CBF) value is always inferred as one.
- the coefficients block flag (CBF) value is always inferred as zero.
- more than one prediction sample for the intra prediction are generated in parallel for the plurality of sub-partitions.
- a method of video decoding comprises performing an intra prediction for a plurality of corresponding sub-partitions using only N modes out of M intra prediction modes for a luma component of an intra sub partition (ISP) coded block, wherein M and N are positive integers and N is less than M.
- ISP intra sub partition
- each of the plurality of corresponding sub-partitions has a width less than or equal to 2.
- N is equal to one, such that only a single mode is allowed for the luma component.
- the single mode is a planar mode.
- the single mode is a DC mode.
- the single mode is any one of following modes: a horizontal (HOR) prediction mode, a vertical (VER) prediction mode, or a diagonal (DIA) prediction mode.
- HOR horizontal
- VER vertical
- DIA diagonal
- the method of video decoding further comprises selecting the single mode in response to a sub-partition orientation, wherein the horizontal (HOR) prediction mode is selected in response to the sub-partition orientation being horizontal, and wherein the vertical (VER) prediction mode is selected in response to the sub-partition orientation being vertical.
- the method of video decoding further comprises selecting the single mode in response to a sub-partition orientation, wherein the horizontal (HOR) prediction mode is selected in response to the sub-partition orientation being vertical, and wherein the vertical (VER) prediction mode is selected in response to the sub-partition orientation being horizontal.
- N is equal to two, such that two modes are allowed for the luma component.
- a first set of two modes are selected in response to a first sub-partition orientation, and a second set of two modes are selected in response to a second sub-partition orientation.
- the first set of two modes comprises a planar mode and a horizontal (HOR) prediction mode
- the first sub-partition orientation comprises horizontal sub-partitioning
- the second set of two modes comprises a planar mode and a vertical (VER) prediction mode
- the second sub-partition orientation comprises vertical sub-partitioning
- the method of video decoding further comprises signaling each mode of the N modes using a corresponding binary codeword from a set of predetermined binary codewords.
- the set of predetermined binary codewords is generated using at least one of a truncated binary (TB) binarization process, a fixed-length binarization process, a truncated Rice (TR) binarization process, a truncated unary binarization process, a k-th order Exp-Golumb binarization process, and a limited EGk binarization process.
- TB truncated binary
- TR truncated Rice
- a method of video decoding comprises performing an intra prediction using only N modes out of M intra prediction modes for a chroma component of an intra sub partition (ISP) coded block, wherein M and N are positive integers and N is less than M.
- ISP intra sub partition
- N is equal to one, such that only a single mode is allowed for the chroma component.
- the single mode is at least one of direct mode (DM), linear model (LM) mode, horizontal (HOR) prediction mode, and vertical (VER) prediction mode.
- DM direct mode
- LM linear model
- HOR horizontal
- VER vertical prediction mode
- N is equal to two, such that two modes are allowed for the chroma component.
- N is equal to four, such that four modes are allowed for the chroma component.
- the method of video decoding further comprises signaling each mode of the N modes using a corresponding binary codeword from a set of predetermined binary codewords.
- the set of predetermined binary codewords is generated using at least one of a truncated binary (TB) binarization process, a fixed-length binarization process, a truncated Rice (TR) binarization process, a truncated unary binarization process, a k-th order Exp-Golumb binarization process, and a limited EGk binarization process.
- TB truncated binary
- TR truncated Rice
- a method of video decoding comprises for a luma component, performing a luma intra prediction for a plurality of corresponding sub-partitions of an entire intra sub partition (ISP) coded block, and for a chroma component, performing a chroma intra prediction for the entire intra sub partition (ISP) coded block.
- ISP intra sub partition
- each of the plurality of corresponding sub-partitions has a width less than or equal to 2.
- a method of video decoding comprises performing a first prediction using an intra sub partition mode; performing a second prediction using an inter prediction mode; and combining the first and second predictions to generate a final prediction by applying a weighted averaging to the first prediction and the second prediction.
- the method of video decoding further comprises performing the second prediction using at least one of a merge mode and an inter mode.
- Computer-readable media may include computer-readable storage media, which corresponds to a tangible medium such as data storage media, or communication media including any medium that facilitates transfer of a computer program from one place to another, e.g., according to a communication protocol.
- Computer-readable media generally may correspond to (1) tangible computer-readable storage media which is non-transitory or (2) a communication medium such as a signal or carrier wave.
- Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and/or data structures for implementation of the implementations described in the present application.
- a computer program product may include a computer-readable medium.
- the above methods may be implemented using an apparatus that includes one or more circuitries, which include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic components.
- the apparatus may use the circuitries in combination with the other hardware or software components for performing the above described methods.
- Each module, sub-module, unit, or sub-unit disclosed above may be implemented at least partially using the one or more circuitries.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
Description
- This application is a continuation of PCT Application PCT/US2020/016888 filed on Feb. 5, 2020, which is based upon and claims the benefit of U.S. provisional patent application Ser. No. 62/801,214 filed on Feb. 5, 2019, the entire disclosures of which are incorporated herein by reference in their entireties for all purposes.
- The present disclosure relates generally to video coding and compression. More specifically, this disclosure relates to systems and methods for performing video coding using intra sub-partition coding mode.
- This section provides background information related to the present disclosure. The information contained within this section should not necessarily be construed as prior art.
- Any of various video coding techniques may be used to compress video data. Video coding can be performed according to one or more video coding standards. Some illustrative video coding standards include versatile video coding (VVC), joint exploration test model (JEM) coding, high-efficiency video coding (H.265/HEVC), advanced video coding (H.264/AVC), and moving picture experts group (MPEG) coding.
- Video coding generally utilizes predictive methods (e.g., inter-prediction, intra-prediction, or the like) that take advantage of redundancy inherent in video images or sequences. One goal of video coding techniques is to compress video data into a form that uses a lower bit rate, while avoiding or minimizing degradations to video quality.
- The first version of the HEVC standard, finalized in October of 2013, offers an approximately 50% bit-rate saving or equivalent perceptual quality, compared to the prior-generation video coding standard (H.264/MPEG AVC). Although the HEVC standard provides significant coding improvements compared to its predecessor, there is evidence that superior coding efficiency can be achieved over HEVC by using additional coding tools. Based on this evidence, both the Video Coding Experts Group (VCEG) and the Moving Picture Experts Group (MPEG) commenced exploratory work to develop new coding technologies for future video coding standardization. The Joint Video Exploration Team (JVET) was formed in October of 2015 by ITU-T VECG and ISO/IEC MPEG to begin significant study of advanced technologies that could enable substantial enhancement of coding efficiency. One reference software model, called the Joint Exploration Model (JEM), was maintained by the JVET by integrating several additional coding tools on top of the HEVC test model (HM).
- In October of 2017, the joint call for proposals (CfP) on video compression with capability beyond HEVC was issued by ITU-T and ISO/IEC. In April of 2018, 23 CfP responses were received and evaluated at the 10th JVET meeting. These responses demonstrated compression efficiency gain over the HEVC standard of approximately 40%. Based on such evaluation results, the JVET launched a new project to develop the next-generation video coding standard called Versatile Video Coding (VVC). Also during April of 2018, one reference software codebase, called the VVC test model (VTM), was established for demonstrating a reference implementation of the VVC standard.
- This section provides a general summary of the disclosure, and is not a comprehensive disclosure of its full scope or all of its features.
- According to a first aspect of the present disclosure, a video coding method is performed at a computing device having one or more processors and memory storing a plurality of programs to be executed by the one or more processors. The method includes independently generating a respective intra prediction for each of a plurality of corresponding sub-partitions, wherein each respective intra prediction is generated using a plurality of reference samples from a current coding block.
- According to a second aspect of the present disclosure, a video coding method is performed at a computing device having one or more processors and memory storing a plurality of programs to be executed by the one or more processors. The method includes generating a respective intra prediction for each of a plurality of corresponding sub-partitions using only N modes out of M possible intra prediction modes for a luma component of an intra sub partition (ISP) coded block, wherein M and N are positive integers and N is less than M.
- According to a third aspect of the present disclosure, a video coding method is performed at a computing device having one or more processors and memory storing a plurality of programs to be executed by the one or more processors. The method includes generating an intra prediction using only N modes out of M possible intra prediction modes for a chroma component of an intra sub partition (ISP) coded block, wherein M and N are positive integers and N is less than M.
- According to a fourth aspect of the present disclosure, a video coding method is performed at a computing device having one or more processors and memory storing a plurality of programs to be executed by the one or more processors. The method includes generating a respective luma intra prediction for each of a plurality of corresponding sub-partitions of an entire intra sub partition (ISP) coded block for a luma component, and for a chroma component, generating a chroma intra prediction for the entire intra sub partition (ISP) coded block.
- According to a fifth aspect of the present disclosure, a video coding method is performed at a computing device having one or more processors and memory storing a plurality of programs to be executed by the one or more processors. The method includes generating a first prediction using an intra sub partition mode; generating a second prediction using an inter prediction mode; and combining the first and second predictions to generate a final prediction by applying a weighted averaging to the first prediction and the second prediction.
- Hereinafter, sets of illustrative, non-limiting embodiments of the present disclosure will be described in conjunction with the accompanying drawings. Variations of structure, method, or functionality may be implemented by those of ordinary skill in the relevant art based on the examples presented herein, and such variations are all contained within the scope of the present disclosure. In cases where no conflict is present, the teachings of different embodiments may, but need not, be combined with one another.
-
FIG. 1 is a block diagram setting forth an illustrative encoder which may be used in conjunction with many video coding standards. -
FIG. 2 is a block diagram setting forth an illustrative decoder which may be used in conjunction with many video coding standards. -
FIG. 3 illustrates five types of exemplary block partitions for a multi-type tree structure. -
FIG. 4 illustrates a set of exemplary intra modes for use with the VVC standard. -
FIG. 5 illustrates a set of multiple reference lines for performing intra prediction. -
FIG. 6A illustrates a first set of reference samples and angular directions that are used for performing an intra prediction of a first rectangular block. -
FIG. 6B illustrates a second set of reference samples and angular directions that are used for performing an intra prediction of a second rectangular block. -
FIG. 6C illustrates a third set of reference samples and angular directions that are used for performing an intra prediction of a square block. -
FIG. 7 illustrates a set of exemplary locations for neighboring reconstructed samples that are used for a position-dependent intra prediction combination (PDPC) of one coding block. -
FIG. 8A illustrates an exemplary set of short-distance intra prediction (SDIP) partitions for an 8×4 block. -
FIG. 8B illustrates an exemplary set of short-distance intra prediction (SDIP) partitions for a 4×8 block. -
FIG. 8C illustrates an exemplary set of short-distance intra prediction (SDIP) partitions for an arbitrarily-sized block. -
FIG. 9A is a plot of chroma values as a function of luma values, where the plot is used to derive a set of linear model parameters. -
FIG. 9B shows the locations of samples that are used for the derivation of the linear model parameters ofFIG. 9A . -
FIG. 10 illustrates the generation of reference samples for intra prediction for all sub-partitions using only reference samples outside of a current coding block. -
FIG. 11 illustrates a combining of inter predictor samples and intra predictor samples for a first sub-partition ofFIG. 10 . -
FIG. 12 illustrates a combining of inter predictor samples and intra predictions samples for a second sub-partition ofFIG. 10 . - The terms used in the present disclosure are directed to illustrating particular examples, rather than to limit the present disclosure. The singular forms “a” “an” and “the” as used in the present disclosure as well as the appended claims also refer to plural forms unless other meanings are definitely contained in the context. It should be appreciated that the term “and/or” as used herein refers to any or all possible combinations of one or more associated listed items.
- It shall be understood that, although the terms “first,” “second,” “third,” etc. may be used herein to describe various information, the information should not be limited by these terms. These terms are only used to distinguish one category of information from another. For example, without departing from the scope of the present disclosure, first information may be termed as second information; and similarly, second information may also be termed as first information. As used herein, the term “if” may be understood to mean “when” or “upon” or “in response to,” depending on the context.
- Reference throughout this specification to “one embodiment,” “an embodiment,” “another embodiment,” or the like in the singular or plural means that one or more particular features, structures, or characteristics described in connection with an embodiment are included in at least one embodiment of the present disclosure. Thus, the appearances of the phrases “in one embodiment” or “in an embodiment,” “in another embodiment,” or the like in the singular or plural in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics in one or more embodiments may be combined in any suitable manner.
- Conceptually, many video coding standards are similar, including those previously mentioned in the Background section. For example, virtually all video coding standards use block-based processing, and share similar video coding block diagrams to achieve video compression. Like HEVC, the VVC standard is built upon the block-based hybrid video coding framework.
-
FIG. 1 shows a block diagram of anillustrative encoder 100 which may be used in conjunction with many video coding standards. In theencoder 100, a video frame is partitioned into a plurality of video blocks for processing. For each given video block, a prediction is formed based on either an inter prediction approach or an intra prediction approach. In inter prediction, one or more predictors are formed through motion estimation and motion compensation, based on pixels from previously reconstructed frames. In intra prediction, predictors are formed based on reconstructed pixels in a current frame. Through mode decision, a best predictor may be chosen to predict a current block. - A prediction residual, representing the difference between a current video block and its predictor, is sent to a
Transform 102 circuitry. Transform coefficients are then sent from theTransform 102 circuitry to aQuantization 104 circuitry for entropy reduction. Quantized coefficients are then fed to anEntropy Coding 106 circuitry to generate a compressed video bitstream. As shown inFIG. 1 , prediction-relatedinformation 110 from an inter prediction circuitry and/or anIntra Prediction 112 circuitry, such as video block partition info, motion vectors, reference picture index, and intra prediction mode, are also fed through theEntropy Coding 106 circuitry and saved into acompressed video bitstream 114. - In the
encoder 100, decoder-related circuitries are also needed in order to reconstruct pixels for the purpose of prediction. First, a prediction residual is reconstructed through anInverse Quantization 116 circuitry and anInverse Transform 118 circuitry. This reconstructed prediction residual is combined with a Block Predictor 120 to generate un-filtered reconstructed pixels for a current video block. - To improve coding efficiency and visual quality, an In-
Loop Filter 115 is commonly used. For example, a deblocking filter is available in AVC, HEVC, as well as the current version of VVC. In HEVC, an additional in-loop filter called SAO (sample adaptive offset) is defined to further improve coding efficiency. In the now-current version of the VVC standard, yet another in-loop filter called ALF (adaptive loop filter) is being actively investigated, and it has a good chance of being included in the final standard. - These in-loop filter operations are optional. Performing these operations helps to improve coding efficiency and visual quality. They may also be turned off as a decision rendered by the
encoder 100 to save computational complexity. - It should be noted that intra prediction is usually based on unfiltered reconstructed pixels, while inter prediction is based on filtered reconstructed pixels if these filter options are turned on by the
encoder 100. -
FIG. 2 is a block diagram setting forth anillustrative decoder 200 which may be used in conjunction with many video coding standards. Thisdecoder 200 is similar to the reconstruction-related section residing in theencoder 100 ofFIG. 1 . In the decoder 200 (FIG. 2 ), anincoming video bitstream 201 is first decoded through anEntropy Decoding 202 circuitry to derive quantized coefficient levels and prediction-related information. The quantized coefficient levels are processed through anInverse Quantization 204 circuitry and anInverse Transform 206 circuitry to obtain a reconstructed prediction residual. A block predictor mechanism, implemented in an Intra/inter Mode Selector 212, is configured to perform either anIntra Prediction 208 procedure, or aMotion Compensation 210 procedure, based on decoded prediction information. A set of unfiltered reconstructed pixels are obtained by summing up the reconstructed prediction residual from theInverse Transform 206 circuitry and a predictive output generated by the block predictor mechanism, using aSummer 214. The reconstructed block may further go through an In-Loop Filter 209 before it is stored in aPicture Buffer 213 which functions as a reference picture store. The reconstructed video in thePicture Buffer 213 can then be sent out to drive a display device, as well as used to predict future video blocks. In situations where the In-Loop Filter 209 is turned on, a filtering operation is performed on these reconstructed pixels to derive a finalreconstructed Video Output 222. - Returning now to
FIG. 1 , an input video signal to theencoder 100 is processed block by block. Each block is called a coding unit (CU)). In VTM-1.0, a CU can be up to 128×128 pixels. In High Efficiency Video Encoding (HEVC), Joint Exploration Test Model (JEM), and Versatile Video Encoding (VVC), the basic unit for compression is termed a coding tree unit (CTU). However, in contrast to the HEVC standard which partitions blocks only based on quad-trees, in the VVC standard, one coding tree unit (CTU) is split into CUs to adapt to varying local characteristics based on a quad/binary/ternary-tree structure. Additionally, the concept of multiple partition unit type in the HEVC standard is not present in the VVC standard, i.e., the separation of CU, prediction unit (PU) and transform unit (TU) does not exist in the VVC standard; instead, each CU is always used as the basic unit for both prediction and transform without further partitions. The maximum CTU size for HEVC and JEM is defined as being up to 64 by 64 luma pixels, and two blocks of 32 by 32 chroma pixels, for a 4:2:0 chroma format. The maximum allowed size of the luma block in a CTU is specified to be 128×128 (although the maximum size of the luma transform block is 64×64). -
FIG. 3 illustrates five types of exemplary block partitions for a multi-type tree structure. The five types of exemplary block partitions includequaternary partitioning 301, horizontalbinary partitioning 302, verticalbinary partitioning 303, horizontalternary partitioning 304, and verticalternary partitioning 305. In situations where a multi-type tree structure is utilized, one CTU is first partitioned by a quad-tree structure. Then, each quad-tree leaf node can be further partitioned by a binary and ternary tree structure. - Using one or more of the
exemplary block partitionings FIG. 3 , spatial prediction and/or temporal prediction may be performed using the configuration shown inFIG. 1 . Spatial prediction (or “intra prediction”) uses pixels from the samples of already-coded neighboring blocks (which are called reference samples) in the same video picture/slice to predict the current video block. Spatial prediction reduces spatial redundancy inherent in the video signal. - Temporal prediction (also referred to as “inter prediction” or “motion compensated prediction”) uses reconstructed pixels from already-coded video pictures to predict the current video block. Temporal prediction reduces temporal redundancy inherent in the video signal. Temporal prediction signal for a given CU is usually signaled by one or more motion vectors (MVs) which indicate the amount and the direction of motion between the current CU and its temporal reference. Also, if multiple reference pictures are supported, one reference picture index is additionally sent, which is used to identify from which reference picture in the reference picture store the temporal prediction signal comes.
- After spatial and/or temporal prediction is performed, an intra/
inter mode decision 121 circuitry in theencoder 100 chooses the best prediction mode, for example based on the rate-distortion optimization method. The block predictor 120 is then subtracted from the current video block; and the resulting prediction residual is de-correlated using thetransform 102 circuitry and thequantization 104 circuitry. The resulting quantized residual coefficients are inverse quantized by theinverse quantization 116 circuitry and inverse transformed by theinverse transform 118circuitry 118 to form the reconstructed residual, which is then added back to the prediction block to form the reconstructed signal of the CU. Further in-loop filtering 115, such as a deblocking filter, a sample adaptive offset (SAO), and/or an adaptive in-loop filter (ALF) may be applied on the reconstructed CU before it is put in the reference picture store of thepicture buffer 117 and used to code future video blocks. To form theoutput video bitstream 114, coding mode (inter or intra), prediction mode information, motion information, and quantized residual coefficients are all sent to theentropy coding unit 106 to be further compressed and packed to form the bit-stream. - The basic intra prediction scheme applied in the VVC standard is kept generally the same as that of the HEVC standard, except that several modules are further extended and/or improved in the VVC standard, such as intra sub-partition (ISP) coding mode, extended intra prediction with wide-angle intra directions, position-dependent intra prediction combination (PDPC) and 4-tap intra interpolation. One broad aspect of this disclosure is directed to improving the existing ISP design in the VVC standard. Moreover, other coding tools (e.g., the tools in the intra prediction and transform coding processes) that are included in the VVC standard and that are closely related to the proposed techniques in this disclosure are discussed in greater detail hereinafter.
- Intra Prediction Modes with Wide-Angle Intra Directions
- As in the case of the HEVC standard, the VVC standard uses a set of previously-decoded samples that neighbor one current CU (i.e., above or left) to predict the samples of the CU. However, to capture finer-edge directions that are present in natural video (especially for video content having a high resolution, e.g., 4K), the amount of angular intra modes is extended from 33 modes in the HEVC standard to 93 modes in the VVC standard. In addition to angular directions, both the HEVC standard and the VVC standard provide for a Planar mode (which assumes a gradual changing surface with a horizontal and a vertical slope derived from boundaries), and a DC mode (which assumes a flat surface).
-
FIG. 4 illustrates a set of exemplaryintra modes 400 for use with the VVC standard, andFIG. 5 illustrates a set of multiple reference lines for performing intra prediction. With reference toFIG. 4 , the set of exemplaryintra modes 400 includesmodes Mode 0 corresponds to the Planar mode, andmode 1 corresponds to the DC mode. Similar to the intra prediction process of the HEVC standard, all of the defined intra modes (i.e., planar, DC and angular directions) in the VVC standard utilize a set of neighboring reconstructed samples above and left to the predicted block as the reference for intra prediction. However, unlike the HEVC standard where only the nearest row/column (line FIG. 5 ) of reconstructed samples is used as reference, a multi reference line (MRL) is introduced in the VVC where two additional rows/columns (i.e.,line line FIG. 5 ) are used for the intra prediction process. The index of the selected reference row/column is signaled from the encoder 100 (FIG. 1 ) to the decoder 200 (FIG. 2 ). When a non-nearest row/column is selected inFIG. 5 , such as line 1503 or line 3505, the planar and DC modes ofFIG. 4 are excluded from the set of intra modes that can be used to predict the current block. -
FIG. 6A illustrates a first set of reference samples andangular directions first sample 601 and asecond sample 603.FIG. 6B illustrates a second set of reference samples andangular directions third sample 605 and afourth sample 607.FIG. 6C illustrates a third set of reference samples andangular directions fifth sample 609 and asixth sample 610. Assuming utilization of the nearest neighbors,FIG. 6C illustrates the locations of the third reference samples that can be used in the VVC standard to derive the predicted samples of one intra block. As shown inFIG. 6C , because the quad/binary/ternary tree partition structure is applied, in addition to the coding blocks of square shape, rectangular coding blocks also exist for the intra prediction procedure in the context of the VVC standard. - Due to the unequal width and height of one given block, various set of angular directions are selected for different block shapes, which is also called wide-angle intra prediction. Specifically, for both square and rectangular coding blocks, besides planar and DC modes, 65 out of 93 angular directions are also supported for each block shape, as shown in Table 1. Such a design can not only efficiently capture the directional structures that are typically present in video (by adaptively selecting angular directions based on block shapes) but also ensure that a total of 67 intra modes (i.e., planar, DC and 65 angular directions) are enabled for each coding block. This can achieve a good efficiency of signaling intra modes, while providing a consistent design across different block sizes.
-
TABLE 1 The selected angular directions for the intra prediction of different block shapes in the VVC Block shape Aspect ratio Selected angular directions Square, W = H W/H == 1 2~66 Flat rectangle, W > H W/H == 2 8~72 W/H == 4 12~76 W/H == 8 14~78 W/H == 16 16~80 W/H == 32 17~81 Tall rectangle, W < H W/H == 1/2 −4~60 W/H == 1/4 −8~56 W/H == 1/8 −10~54 W/H == 1/16 −12~52 W/H == 1/32 −13~51 - Position-Dependent Intra Prediction Combination
- As mentioned earlier, the intra prediction samples are generated from either a non-filtered or a filtered set of neighboring reference samples, which may introduce discontinuities along the block boundaries between the current coding block and its neighbors. To resolve such problems, boundary filtering is applied in the HEVC standard by combing the first row/column of prediction samples of DC, horizontal (i.e.,
mode 18 ofFIG. 4 ) and vertical (i.e., mode 50) prediction modes with the unfiltered reference samples, utilizing a 2-tap filter (for DC mode) or a gradient-based smoothing filter (for horizontal and vertical prediction modes). - The position-dependent intra prediction combination (PDPC) tool in the VVC standard extends the foregoing concept by employing a weighted combination of intra prediction samples with unfiltered reference samples. In the current VVC working draft, the PDPC is enabled for the following intra modes without signaling: planar, DC, horizontal (i.e, mode 18), vertical (i.e., mode 50), angular directions close to the bottom-left diagonal directions (i.e.,
modes modes 58, 59, 60, . . . , 66). Assuming the prediction sample located as coordinate (x, y) is pred(x,y), its corresponding value after the PDPC is calculated as: -
pred(x,y)=(wL×R −1,y +wT×R x,−1 −wTL×R −1,−1+(64−wL−wT+wTL)×pred(x,y)+32)>>6 (1) - where Rx,−1, R−1,y represent the reference samples located at the top and left of current sample (x, y), respectively, and R−1,−1 represents the reference sample located at the top-left corner of the current block.
-
FIG. 7 illustrates a set of exemplary locations for neighboring reconstructed samples that are used for a position-dependent intra prediction combination (PDPC) of one coding block. A first reference sample 701 (Rx,−1), represents a reference sample located at the top of a current prediction sample (x, y). A second reference sample 703 (R−1,y) represents a reference sample located to the left of the current prediction sample (x, y). A third reference sample 705 (R−1,−1) represents the reference sample located at the top-left corner of the current prediction sample (x, y). - Reference samples including the first, second, and
third reference samples -
- For the DC mode,
-
wT=32>>((y<<1)>>shift),wL=32>>((x<<1)>>shift),wTL=(wL>>4)+(wT>>4) (2) -
- For the planar mode,
-
wT=32>>((y<<1)>>shift),wL=32>>((x<<1)>>shift),wTL=0 (3) -
- For the horizontal mode:
-
wT=32>>((y<<1)>>shift),wL=32>>((x<<1)>>shift),wTL=wT (4) -
- For the vertical mode:
-
wT=32>>((y<<1)>>shift),wL=32>>((x<<1)>>shift),wTL=wL (5) -
- For bottom-left diagonal directions:
-
wT=16>>((y<<1)>>shift),wL=16>>((x<<1)>>shift),wTL=0 (6) -
- For top-right diagonal directions:
-
wT=16>>((y<<1)>>shift),wL=16>>((x<<1)>>shift),wTL=0 (7) - where shift=(log2(W)−2+log2(H)−2+2)>>2.
- Multiple Transform Selection and Shape Adaptive Transform Selection
- In addition to DCT-II transform that is used in the HEVC standard, multiple transform selection (MTS) tool is enabled in the VVC standard by introducing additional core transforms of DCT-VIII and DST-VII. In the VVC standard, the adaptive selection of the transforms is enabled at the coding block level by signaling one MTS flag to a bitstream. Specifically, when the MTS flag is equal to 0 for one block, one pair of fixed transforms (e.g., DCT-II) are applied in the horizontal and vertical directions. Otherwise (when the MTS flag is equal to 1), two additional flags will be further signaled for the block to indicate the transform type (either DCT-VIII or DST-VII) for each direction.
- On the other hand, due to the introduction of quad/binary/ternary-tree based block partitioning structure in the VVC standard, the distribution of the residuals of intra prediction is highly correlated with the block shape. Therefore, when the MTS is disabled (i.e., the MTS flag is equal to 0 for one coding block), one shape-adaptive transform selection method is applied to all intra-coded blocks in which the DCT-II and DST-VII transforms are implicitly enabled based on the width and height of the current block. More specifically, for each rectangular block, the method uses the DST-VII transform in the direction associated with shorter side of one block and DCT-II transform in the direction associated with longer side of the block. For each square block, the DST-VII is applied in both directions. Additionally, to avoid introducing new transforms in different block sizes, the DST-VII transform is only enabled when the shorter side of one intra-coded block is equal to or smaller than 16. Otherwise, the DCT-II transform is always applied.
- Table 2 illustrates the enabled horizontal and vertical transforms for intra-coded blocks based on the shape adaptive transform selection method in the VVC.
-
TABLE 2 Shape adaptive transform selection for intra blocks in the VVC Block size Horizontal trans. Vertical trans. min (W, H) > 16 DCT-II DCT-II min (W, H) ≤ 16 W = H DST-VII DST-VII W > H DCT-II DST-VII W < H DST-VII DCT-II - Intra Sub-Partition Coding Mode
- Conventional intra mode only utilizes the reconstructed samples that neighbor one coding block to generate the intra prediction samples of that block. Based on such a method, the spatial correlation between the predicted samples and the reference samples is roughly proportional to the distance between the predicted samples and the reference samples. Therefore, the samples at the inner portion (especially the samples located at the bottom-right corner of the block) usually have a lower prediction quality than the samples that are close to the block boundaries. To further improve the intra prediction efficiency, short-distance intra prediction (SDIP) was proposed and studied well. The method divides one intra coding block horizontally or vertically into multiple sub-blocks for prediction. Usually, a square block is divided into four sub-blocks. For example, an 8×8 block may be divided into four 2×8 or four 8×2 sub-blocks. One extreme case of such sub-block based intra prediction is so-called line-based prediction, wherein a block is divided into 1-D line/column for prediction. For example, one W×H (width×height) block can be split either into H sub-blocks in size of W×1 or into W sub-blocks in size of 1×H for intra prediction. Each of the resulting lines/columns are coded in the same way of a normal 2-dimension (2-D) block (as shown in
FIGS. 6A, 6B, 6C, and 7 ), i.e., it is predicted by one of available intra modes and the prediction error is decorrelated based on transform and quantization and sent to the decoder 200 (FIG. 2 ) for reconstruction. Consequently, the reconstructed samples in one sub-block (e.g. a line/column) can be used as references to predict the samples in the next sub-block. The above process is repeated until all of the sub-blocks within the current block are predicted and coded. Additionally, to reduce the signaling overhead, all the sub-blocks within one coding block shares the same intra mode. - With SDIP, different sub-block partitions may provide different coding efficiencies. In general, line-based prediction offers the best coding efficiency because it provides “the shortest prediction distance” among different partitions. On the other hand, it also has the poorest encoding/decoding throughput for codec hardware implementations. For example, considering a block with 4×4 sub-blocks versus the same block with 4×1 or 1×4 sub-blocks, the latter case is only one fourth of the throughput of the former case. In HEVC, the smallest intra prediction block size for luma is 4×4.
-
FIG. 8A illustrates an exemplary set of short-distance intra prediction (SDIP) partitions for an 8×4block 801,FIG. 8B illustrates an exemplary set of short-distance intra prediction (SDIP) partitions for a 4×8block 803, andFIG. 8C illustrates an exemplary set of short-distance intra prediction (SDIP) partitions for an arbitrarily-sized block 805. Recently, a video coding tool called sub-partition prediction (ISP) was introduced into the VVC standard. Conceptually, ISP is very similar to SDIP. Specifically, depending on the block size, the ISP divides the current coding block into 2 or 4 sub-blocks in either a horizontal or vertical direction, and each sub-block contains at least 16 samples. - Taken together,
FIGS. 8A, 8B, and 8C illustrate all of the possible partition cases for different coding block sizes. Moreover, the following main aspects are also included in the current ISP design to handle its interaction with the other coding tools in the VVC standard: -
- Interaction with wide-angle intra direction: the ISP is combined with the wide-angle intra direction. In the current design, the block size (i.e., the width/height ratio) that is used to determine whether a normal intra direction or its corresponding wide-angle intra direction should be applied is the one of the original coding block, i.e., the block before sub-block partitions.
- Interaction with multiple reference line: the ISP cannot be jointly enabled with the multiple reference line. Specifically, in the current VVC signaling design, the ISP enabling/disabling flag is signaled after the MRL index. When one intra block has one non-zero MRL index (i.e., referring to non-nearest neighboring samples), the ISP enabling/disabling flag is not signaled but inferred as 0, i.e. ISP is automatically disabled for the coding block in this case.
- Interaction with most probable mode: similar to normal intra mode, the intra mode that is used for one ISP block is signaled through the most probable mode (MPM) mechanism. However, compared to the normal intra mode, the following modifications are made to the MPM method for ISP: 1) each ISP block only enables the intra modes that are included in the MPM list and disables all the other intra modes that are not in the MPM list; 2) for each ISP block, its MPM list excludes the DC mode and prioritizes the horizontal intra modes for ISP horizontal partition and vertical modes for ISP vertical partition respectively.
- At least one non-zero coefficients block flag (CBF): In current VVC, a CBF flag is signaled for each transform unit (TU) to specify that the transform block contains one or more transform coefficient levels not equal to 0. Given a certain block using ISP, the decoder will assume that at least one of the sub-partitions has a non-zero CBF. For this reason, if n is the number of sub-partitions and the first n−1 sub-partitions have produced a zero CBF, then the CBF of the n-th sub-partition will be inferred to be 1. Therefore, it is not necessary to transmit and decode it.
- Interaction with multiple transform selection: the ISP is exclusively applied with the MTS, i.e., when one coding block uses ISP, its MTS flag is not signaled but always inferred as 0, i.e., disabled. However, instead of always using the DCT-II transform, a fixed set of core transforms (including DST-VII and DCT-II) are implicitly applied to ISP coded blocks based on the block sizes. Specifically, assuming W and H are the width and height of one ISP sub-partition, its horizontal and vertical transforms are selected according to the following rules as described in Table 3.
-
TABLE 3 The selected horizontal and vertical transforms for ISP blocks Horizontal Vertical Sub-block size Intra mode trans. trans. W = 2 or W > 32 All available intra DCT-II DCT-II modes H = 2 or H > 32 All available intra DCT-II DCT-II modes The other sub- Planar, DCT-II DCT- II block sizes 31, 32, 34, 36, 37 DC, 33, 35 DST-VII DCT- II 2, 4, 6. . . 28, 30, DST-VII DCT- II 39, 41, 43. . . 63, 65 - Cross-Component Linear Model Prediction
-
FIG. 9A is a plot of chroma values as a function of luma values, where the plot is used to derive a set of linear model parameters. More specifically, a straight-line 901 relationship between chroma values and luma values is used to derive a set of linear model parameters α and β as follows. To reduce the cross-component redundancy, a cross-component linear model (CCLM) prediction mode is used in VVC, for which the chroma samples are predicted based on the reconstructed luma samples of the same CU by using a linear model as follows: -
predC(i,j)=α·rec L′(i,j)+β (8) - where predC (i, j) represents the predicted chroma samples in a CU and recL′(i, j) represents the downsampled reconstructed luma samples of the same CU. Linear model parameters α and β are derived from the straight-
line 901 relationship between luma values and chroma values from two samples, which are minimum luma sample A (XA, YA) and maximum luma sample B (XB, YB) inside the set of neighboring luma samples, as exemplified inFIG. 9A . Here XA, YA are the x-coordinate (i.e. luma value) and y-coordinate (i.e. chroma value) value for sample A, and XB, YB are the x-coordinate and y-coordinate value for sample B. The linear model parameters α and β are obtained according to the following equations. -
- Such a method is also called a min-Max method. The division in the equation above could be avoided and replaced by a multiplication and a shift.
-
FIG. 9B shows the locations of the samples that are used for the derivation of the linear model parameters ofFIG. 9A . For a coding block with a square shape, the above two equations for the linear model parameters α and β are applied directly. For a non-square coding block, the neighboring samples of the longer boundary are first subsampled to have the same number of samples as for the shorter boundary.FIG. 9B shows the locations of the left and above samples and the sample of the current block involved in the CCLM mode, including an N×N set ofchroma samples 903 and a 2N×2N set ofluma samples 905. Besides using the above template and the left template to calculate the linear model coefficients together, these templates also can be used alternatively in the other 2 LM modes, called LM_A, and LM_L modes. - In LM_A mode, only pixel samples in the above template are used to calculate the linear model coefficients. To get more samples, the above template is extended to (W+W). In LM_L mode, only pixel samples in the left template are used to calculate the linear model coefficients. To get more samples, the left template are extended to (H+H). Note that only one luma line (general line buffer in intra prediction) is used to make the down-sampled luma samples when the upper reference line is at the CTU boundary.
- For chroma intra mode coding, a total of 8 intra modes are allowed for chroma intra mode coding. Those modes include five traditional intra modes and three cross-component linear model modes (CCLM, LM_A, and LM_L). Chroma mode signaling and derivation process are shown in Table 4. Chroma mode coding directly depends on the intra prediction mode of the corresponding luma block. Since separate block partitioning structure for luma and chroma components is enabled in I slices, one chroma block may correspond to multiple luma blocks. Therefore, for Chroma DM mode, the intra prediction mode of the corresponding luma block covering the center position of the current chroma block is directly inherited.
-
TABLE 4 Derivation of chroma prediction mode from luma mode when CCLM_is enabled Corresponding luma intra prediction mode Chroma X prediction mode 0 50 18 1 ( 0 <= X <= 66) 0 66 0 0 0 0 1 50 66 50 50 50 2 18 18 66 18 18 3 1 1 1 66 1 4 81 81 81 81 81 5 82 82 82 82 82 6 83 83 83 83 83 7 0 50 18 1 X - Although the ISP tool in VVC can enhance intra prediction efficiency, there is room to further improve the performance of VVC. Meanwhile, some parts of the existing ISP would benefit from further simplification to provide for more efficient codec hardware implementations, and/or to provide improved coding efficiency. In this disclosure, several methods are proposed to further improve the ISP coding efficiency, to simplify the existing ISP design, and/or to facilitate improved hardware implementations.
- Independent (or Parallel) Sub-Partition Predictor Generation for the ISP
-
FIG. 10 illustrates the generation of reference samples forintra prediction 1007 for all sub-partitions using only reference samples outside of acurrent coding block 1000. Thecurrent coding block 1000 includes afirst sub-partition 1 1001, asecond sub-partition 2 1002, athird sub-partition 3 1003, and afourth sub-partition 4 1004. In this disclosure, it is proposed to generate an intra prediction for each sub-partition 1001, 1002, 1003, and 1004 independently. In other words, all of the predictors for the sub-partitions 1001, 1002, 1003, and 1004 can be generated in a parallel manner. In one embodiment, the predictors for all the sub-partitions are generated using the same approaches as used in the conventional non-sub-partition intra modes. Specifically, no reconstructed samples of one sub-partition are used to generate the intra prediction samples for any other sub-partition in the same coding unit; all the predictors of each sub-partition 1001, 1002, 1003, 1004 are generated using the reference samples of thecurrent coding block 1000 as shown inFIG. 10 . - For the ISP mode in the VVC standard, the width of each sub-partition can be smaller than or equal to two. One detailed example is as follows. Pursuant to the ISP mode in the VVC standard, the dependence of 2×N (width by height) subblock prediction on reconstructed values of previously decoded 2×N subblocks of the coding block is not allowed so that the minimum width for prediction for subblocks becomes four samples. For example, an 8×8 coding block that is coded using ISP with vertical split is split into four prediction regions each of
size 2×8, and the left two 2×8 prediction regions are merged into a first 4×8 prediction region to perform intra prediction. Transform 102 (FIG. 1 ) circuitry is applied to each 2×8 partition. - Pursuant to this example, the right two 2×8 prediction regions are merged into a second 4×8 prediction region to perform intra prediction. Transform 102 circuitry is applied to each 2×8 partition. It is noted that the first 4×8 prediction region uses the neighboring pixels of the current coding block to generate the intra predictors, while the second 4×8 region uses the reconstructed pixels from the first 4×8 region (located to the left of the second 4×8 region) or the neighboring pixels from the current coding block (located to the top of the second 4×8).
- In another embodiment, only the horizontal (HOR) prediction mode (shown as
mode 18 inFIG. 4 ), as well as those prediction modes having mode indices smaller than 18 (as indicated inFIG. 4 ) may be used in forming the intra prediction of the horizontal sub-partitions; and only the vertical (VER) prediction mode (i.e.mode 50 inFIG. 4 ), and those prediction modes having mode indices larger than 50 (as indicated inFIG. 4 ) may be used in forming the intra prediction of the vertical sub-partitions. As a result, with the HOR prediction mode (shown asmode 18 inFIG. 4 ), and all of the angular prediction modes with modes smaller than 18, the intra prediction for each horizontal sub-partition can be performed independently and in parallel. Likewise, with the VER prediction mode (mode 50 inFIG. 4 ), and all of the angular prediction modes with modes larger than 50, the intra prediction for each vertical sub-partition can be performed independently and in parallel. - Intra Prediction Mode Coding for the Luma Component for ISP
- In this disclosure, it is proposed to allow only N modes out of all possible intra prediction modes for the luma component for an ISP-coded block (N being a positive integer). In one embodiment, only one mode is allowed for the luma component of an ISP-coded block. For example, this single allowed mode may be Planar mode. In another example, this single allowed mode may be DC mode. In a third example, this single allowed mode may be one of HOR prediction mode, VER prediction mode, or diagonal (DIA) mode (
mode 34 inFIG. 4 ) intra prediction mode. - In another embodiment, only one mode is allowed for the luma component of an ISP coded block, and this mode may be different according to the sub-partition orientation, i.e. whether it is a horizontal or vertical sub-partition. For example, only the HOR prediction mode is allowed for the horizontal sub-partitioning, while only the VER prediction mode is allowed for the vertical sub-partitioning. In yet another example, only the VER prediction mode is allowed for the horizontal sub-partitioning, while only the HOR prediction mode is allowed for the vertical sub-partitioning.
- In yet another embodiment, only two modes are allowed for the luma component of an ISP-coded block. Each of respective modes may be selected in response to a corresponding sub-partition orientation, i.e. whether the orientation of the sub-partition is horizontal or vertical. For example, only PLANAR and HOR prediction modes are allowed for the horizontal sub-partitioning, while only PLANAR and VER prediction modes are allowed for the vertical sub-partitioning.
- To signal the N modes allowed for the luma components of an ISP-coded block, the conventional most probable modes (MPM) mechanism is not utilized. Instead, we propose to signal the intra modes for the ISP coded block using the determined binary codewords for each intra mode. The codewords may be generated using any of a variety of different processes, including a Truncated Binary (TB) binarization process, a Fixed-length binarization process, a Truncated Rice (TR) binarization process, a k-th order Exp-Golomb binarization process, a Limited EGk binarization process, etc. These binary codeword generation processes are well defined in the specification of HEVC. The truncated Rice with Rice parameter equal to zero is also known as truncated unary binarization. One set of examples of codewords using different binarization methods is presented in Table 5.
-
TABLE 5 Binary Codewords Generated Using Different Binarization Methods Truncated Exp- Fixed- Truncated Rice (TRk) Golomb Length Unary Unary (TrU) k D 1;(EGk) (FL) N (U) cMaxD 7 cMax D7 k D 0 cMaxD 7 0 0 0 00 1 000 1 10 10 01 010 001 2 110 110 100 011 010 3 1110 1110 101 00100 011 4 11110 11110 1100 00101 100 5 111110 111110 1101 00110 101 6 1111110 1111110 1110 00111 110 7 11111110 1111111 1111 0001000 111 - Independent (or Parallel) Sub-Partition Predictor Generation for the ISP
- In yet another embodiment, the MPM derivation process for the regular intra mode is directly re-used for ISP mode, and the signaling methods of the MPM flag and MPM index are kept the same as it is in the existing ISP design.
- Intra Prediction Mode Coding for the Chroma Component for ISP
- In this disclosure, it is proposed to only allow Nc modes out of all possible chroma intra prediction modes for the chroma component for an ISP coded block (Nc is a positive integer). In one embodiment, only one mode is allowed for the chroma component of an ISP coded block. For example, this single allowed mode may be a direct mode (DM). In another example, this single allowed mode may be an LM mode. In a third example, this single allowed mode may be one of the HOR prediction mode or the VER prediction mode. The direct mode (DM) is configured to apply the same intra prediction mode which is used by the corresponding luma block to the chroma blocks.
- In another embodiment, only two modes are allowed for the chroma component of an ISP coded block. In one example, only DM and LM are allowed for the chroma components of an ISP coded block.
- In yet another embodiment, only four modes are allowed for the chroma component of an ISP coded block. In one example, only DM, LM, LM_L and LM_A are allowed for the chroma components of an ISP coded block.
- To signal the Nc modes for the chroma components of an ISP coded block, the convention MPM mechanism is not utilized. Instead, fixed binary codewords are used to indicate the selected chroma modes in the bitstream. For example, the chroma intra prediction modes for an ISP coded block can be signaled using the determined binary codewords. The codewords may be generated using different processes including the Truncated Binary (TB) binarization process, the Fixed-length binarization process, the truncated Rice (TR) binarization process, the k-th order Exp-Golomb binarization process, the Limited EGk binarization process, etc.
- Coding Block Size for the Chroma Components for ISP Coded Block
- In this disclosure, it is proposed to NOT allow sub-partition coding for chroma components for an ISP coded block. Instead, the normal whole block-based intra prediction is used for the chroma components of an ISP coded block. In other words, for an ISP coded block, sub-partitioning is performed only for its luma component with no sub-partitioning on its chroma components.
- Combination of ISP with Inter Prediction
-
FIG. 11 illustrates a combining of inter predictor samples and intra predictor samples for thefirst sub-partition 1001 ofFIG. 10 . Likewise,FIG. 12 illustrates a combining of inter predictor samples and intra predictions samples for thesecond sub-partition 1002 ofFIG. 10 . More specifically, to further improve coding efficiency, a new prediction mode is provided where the prediction is generated as a weighted combination (e.g. weighted averaging) of the ISP mode and the inter prediction mode. The intra predictor generation for ISP mode is the same as illustrated previously in connection withFIG. 10 . The inter predictor may be generated through the process of merge mode or inter mode. - In the illustrative example of
FIG. 11 , theinter predictor samples 1101 for a current block (including all sub-partitions) is generated by performing motion compensation using the merge candidates indicated by the merge index. Theintra predictor samples 1103 are generated by performing the intra prediction using the signaled intra mode. It is noted that this process may use the reconstructed samples of the previous sub-partition to generate the intra predictor samples for a non-first sub-partition (i.e., the second sub-partition 1002) as shown inFIG. 12 . After the inter and intra predictor samples are generated, they are weight-averaged to generate the final predictor samples for the sub-partition. The combined modes can be treated as an intra mode. Alternatively, the combined modes may be treated as an inter mode or a merge mode instead of intra mode. - CBF Signaling for ISP Coded Block
- To simplify the design of ISP, it is proposed in this disclosure to always signal CBF for the last sub-partition.
- In another embodiment of this disclosure, it is proposed to not signal the CBF of the last sub-partition, but to infer its value at decoder side. For example, the value of CBF for the last sub-partition is always inferred as one. In another example, the value of CBF for the last sub-partition is always inferred as zero.
- According to an embodiment of the present disclosure, a method of video coding comprises independently generating a respective intra prediction for each of a plurality of corresponding sub-partitions, wherein each respective intra prediction is generated using a plurality of reference samples from a current coding block.
- In some examples, no reconstructed sample from a first sub-partition of the plurality of corresponding sub-partitions is used to generate a respective intra prediction for any other sub-partition of the plurality of corresponding sub-partitions.
- In some examples, each of the plurality of corresponding sub-partitions has a width less than or equal to 2.
- In some examples, the plurality of corresponding sub-partitions comprises a plurality of vertical sub-partitions and a plurality of horizontal sub-partitions, and the method further comprises using only a horizontal prediction mode to generate a first set of intra predictions for the plurality of horizontal subdivisions, and using only a vertical prediction mode to generate a second set of intra predictions for the plurality of vertical sub-partitions.
- In some examples, the horizontal prediction mode is performed using a mode index smaller than 18.
- In some examples, the vertical prediction mode is performed using a mode index larger than 50.
- In some examples, the horizontal prediction mode is performed for each of the plurality of horizontal sub-partitions independently and in parallel.
- In some examples, the vertical prediction mode is performed for each of the plurality of vertical sub-partitions independently and in parallel.
- In some examples, the plurality of corresponding sub-partitions includes a last sub-partition, and the method further comprises signaling a coefficients block flag (CBF) value for the last sub-partition.
- In some examples, the plurality of corresponding sub-partitions includes a last sub-partition, and the method further comprises inferring a coefficients block flag (CBF) value for the last sub-partition at a decoder.
- In some examples, the coefficients block flag (CBF) value is always inferred as one.
- In some examples, the coefficients block flag (CBF) value is always inferred as zero.
- According to another embodiment of the present disclosure, a method of video coding comprises generating a respective intra prediction for each of a plurality of corresponding sub-partitions using only N modes out of M possible intra prediction modes for a luma component of an intra sub partition (ISP) coded block, wherein M and N are positive integers and N is less than M.
- In some examples, each of the plurality of corresponding sub-partitions has a width less than or equal to 2.
- In some examples, N is equal to one, such that only a single mode is allowed for the luma component.
- In some examples, the single mode is a planar mode.
- In some examples, the single mode is a DC mode.
- In some examples, the single mode is any one of following modes: a horizontal (HOR) prediction mode, a vertical (VER) prediction mode, or a diagonal (DIA) prediction mode.
- In some examples, the single mode is selected in response to a sub-partition orientation, wherein the horizontal (HOR) prediction mode is selected in response to the sub-partition orientation being horizontal, and wherein the vertical (VER) prediction mode is selected in response to the sub-partition orientation being vertical.
- In some examples, the single mode is selected in response to a sub-partition orientation, wherein the horizontal (HOR) prediction mode is selected in response to the sub-partition orientation being vertical, and wherein the vertical (VER) prediction mode is selected in response to the sub-partition orientation being horizontal.
- In some examples, N is equal to two, such that two modes are allowed for the luma component.
- In some examples, a first set of two modes are selected in response to a first sub-partition orientation, and a second set of two modes are selected in response to a second sub-partition orientation.
- In some examples, the first set of two modes comprises a planar mode and a horizontal (HOR) prediction mode, the first sub-partition orientation comprises horizontal sub-partitioning, the second set of two modes comprises a planar mode and a vertical (VER) prediction mode, and the second sub-partition orientation comprises vertical sub-partitioning.
- In some examples, each respective mode of the N modes is signaled using a corresponding binary codeword from a set of predetermined binary codewords.
- In some examples, the set of predetermined binary codewords is generated using at least one of a truncated binary (TB) binarization process, a fixed-length binarization process, a truncated Rice (TR) binarization process, a truncated unary binarization process, a k-th order Exp-Golumb binarization process, or a limited EGk binarization process.
- According to another embodiment of the present disclosure, a method of video coding comprises generating an intra prediction using only N modes out of M possible intra prediction modes for a chroma component of an intra sub partition (ISP) coded block, wherein M and N are positive integers and N is less than M.
- In some examples, N is equal to one, such that only a single mode is allowed for the chroma component.
- In some examples, the single mode is direct mode (DM), linear model (LM) mode, horizontal (HOR) prediction mode, or vertical (VER) prediction mode.
- In some examples, N is equal to two, such that two modes are allowed for the chroma component.
- In some examples, N is equal to four, such that four modes are allowed for the chroma component.
- In some examples, each respective mode of the N modes is signaled using a corresponding binary codeword from a set of predetermined binary codewords.
- In some examples, the set of predetermined binary codewords is generated using at least one of a truncated binary (TB) binarization process, a fixed-length binarization process, a truncated Rice (TR) binarization process, a truncated unary binarization process, a k-th order Exp-Golumb binarization process, or a limited EGk binarization process.
- According to another embodiment of the present disclosure, a method of video coding comprises generating a respective luma intra prediction for each of a plurality of corresponding sub-partitions of an entire intra sub partition (ISP) coded block for a luma component, and for a chroma component, generating a chroma intra prediction for the entire intra sub partition (ISP) coded block.
- In some examples, each of the plurality of corresponding sub-partitions has a width less than or equal to 2.
- According to another embodiment of the present disclosure, a method of video coding comprises generating a first prediction using an intra sub partition mode; generating a second prediction using an inter prediction mode; and combining the first and second predictions to generate a final prediction by applying a weighted averaging to the first prediction and the second prediction.
- In some examples, the second prediction is generated using at least one of a merge mode or an inter mode.
- According to an embodiment of the present disclosure, a method of video decoding comprises independently performing an intra prediction for a plurality of corresponding sub-partitions, wherein the intra prediction is performed using a plurality of reference samples for a current coding block.
- In some examples, no reconstructed sample from a first sub-partition of the plurality of corresponding sub-partitions is used to perform the intra prediction for any other sub-partition of the plurality of corresponding sub-partitions.
- In some examples, each of the plurality of corresponding sub-partitions has a width less than or equal to 2.
- In some examples, the plurality of corresponding sub-partitions are a plurality of vertical sub-partitions.
- In some examples, the method of video decoding further comprises generating a set of prediction samples for the plurality of vertical sub-partitions using a vertical prediction mode with a mode index larger than 50.
- In some examples, the method of video decoding further comprises performing the intra prediction for the plurality of vertical sub-partitions independently and in parallel.
- In some examples, the plurality of corresponding sub-partitions includes a last sub-partition, and the method further comprises signaling a coefficients block flag (CBF) value for the last sub-partition.
- In some examples, the plurality of corresponding sub-partitions includes a last sub-partition, and the method further comprises inferring a coefficients block flag (CBF) value for the last sub-partition at a decoder.
- In some examples, the coefficients block flag (CBF) value is always inferred as one.
- In some examples, the coefficients block flag (CBF) value is always inferred as zero.
- In some examples, more than one prediction sample for the intra prediction are generated in parallel for the plurality of sub-partitions.
- According to another embodiment of the present disclosure, a method of video decoding comprises performing an intra prediction for a plurality of corresponding sub-partitions using only N modes out of M intra prediction modes for a luma component of an intra sub partition (ISP) coded block, wherein M and N are positive integers and N is less than M.
- In some examples, each of the plurality of corresponding sub-partitions has a width less than or equal to 2.
- In some examples, N is equal to one, such that only a single mode is allowed for the luma component.
- In some examples, the single mode is a planar mode.
- In some examples, the single mode is a DC mode.
- In some examples, the single mode is any one of following modes: a horizontal (HOR) prediction mode, a vertical (VER) prediction mode, or a diagonal (DIA) prediction mode.
- In some examples, the method of video decoding further comprises selecting the single mode in response to a sub-partition orientation, wherein the horizontal (HOR) prediction mode is selected in response to the sub-partition orientation being horizontal, and wherein the vertical (VER) prediction mode is selected in response to the sub-partition orientation being vertical.
- In some examples, the method of video decoding further comprises selecting the single mode in response to a sub-partition orientation, wherein the horizontal (HOR) prediction mode is selected in response to the sub-partition orientation being vertical, and wherein the vertical (VER) prediction mode is selected in response to the sub-partition orientation being horizontal.
- In some examples, N is equal to two, such that two modes are allowed for the luma component.
- In some examples, a first set of two modes are selected in response to a first sub-partition orientation, and a second set of two modes are selected in response to a second sub-partition orientation.
- In some examples, the first set of two modes comprises a planar mode and a horizontal (HOR) prediction mode, the first sub-partition orientation comprises horizontal sub-partitioning, the second set of two modes comprises a planar mode and a vertical (VER) prediction mode, and the second sub-partition orientation comprises vertical sub-partitioning.
- In some examples, the method of video decoding further comprises signaling each mode of the N modes using a corresponding binary codeword from a set of predetermined binary codewords.
- In some examples, the set of predetermined binary codewords is generated using at least one of a truncated binary (TB) binarization process, a fixed-length binarization process, a truncated Rice (TR) binarization process, a truncated unary binarization process, a k-th order Exp-Golumb binarization process, and a limited EGk binarization process.
- According to another embodiment of the present disclosure, a method of video decoding comprises performing an intra prediction using only N modes out of M intra prediction modes for a chroma component of an intra sub partition (ISP) coded block, wherein M and N are positive integers and N is less than M.
- In some examples, N is equal to one, such that only a single mode is allowed for the chroma component.
- In some examples, the single mode is at least one of direct mode (DM), linear model (LM) mode, horizontal (HOR) prediction mode, and vertical (VER) prediction mode.
- In some examples, N is equal to two, such that two modes are allowed for the chroma component.
- In some examples, N is equal to four, such that four modes are allowed for the chroma component.
- In some examples, the method of video decoding further comprises signaling each mode of the N modes using a corresponding binary codeword from a set of predetermined binary codewords.
- In some examples, the set of predetermined binary codewords is generated using at least one of a truncated binary (TB) binarization process, a fixed-length binarization process, a truncated Rice (TR) binarization process, a truncated unary binarization process, a k-th order Exp-Golumb binarization process, and a limited EGk binarization process.
- According to another embodiment of the present disclosure, a method of video decoding comprises for a luma component, performing a luma intra prediction for a plurality of corresponding sub-partitions of an entire intra sub partition (ISP) coded block, and for a chroma component, performing a chroma intra prediction for the entire intra sub partition (ISP) coded block.
- In some examples, each of the plurality of corresponding sub-partitions has a width less than or equal to 2.
- According to another embodiment of the present disclosure, a method of video decoding comprises performing a first prediction using an intra sub partition mode; performing a second prediction using an inter prediction mode; and combining the first and second predictions to generate a final prediction by applying a weighted averaging to the first prediction and the second prediction.
- In some examples, the method of video decoding further comprises performing the second prediction using at least one of a merge mode and an inter mode.
- In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over, as one or more instructions or code, a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which corresponds to a tangible medium such as data storage media, or communication media including any medium that facilitates transfer of a computer program from one place to another, e.g., according to a communication protocol. In this manner, computer-readable media generally may correspond to (1) tangible computer-readable storage media which is non-transitory or (2) a communication medium such as a signal or carrier wave. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and/or data structures for implementation of the implementations described in the present application. A computer program product may include a computer-readable medium.
- Further, the above methods may be implemented using an apparatus that includes one or more circuitries, which include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic components. The apparatus may use the circuitries in combination with the other hardware or software components for performing the above described methods. Each module, sub-module, unit, or sub-unit disclosed above may be implemented at least partially using the one or more circuitries.
- Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed here. This application is intended to cover any variations, uses, or adaptations of the invention following the general principles thereof and including such departures from the present disclosure as come within known or customary practice in the art. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
- It will be appreciated that the present invention is not limited to the exact examples described above and illustrated in the accompanying drawings, and that various modifications and changes can be made without departing from the scope thereof. It is intended that the scope of the invention only be limited by the appended claims.
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US17/393,367 US20210368205A1 (en) | 2019-02-05 | 2021-08-03 | Video coding using intra sub-partition coding mode |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201962801214P | 2019-02-05 | 2019-02-05 | |
PCT/US2020/016888 WO2020163535A1 (en) | 2019-02-05 | 2020-02-05 | Video coding using intra sub-partition coding mode |
US17/393,367 US20210368205A1 (en) | 2019-02-05 | 2021-08-03 | Video coding using intra sub-partition coding mode |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2020/016888 Continuation WO2020163535A1 (en) | 2019-02-05 | 2020-02-05 | Video coding using intra sub-partition coding mode |
Publications (1)
Publication Number | Publication Date |
---|---|
US20210368205A1 true US20210368205A1 (en) | 2021-11-25 |
Family
ID=71947318
Family Applications (4)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/393,367 Pending US20210368205A1 (en) | 2019-02-05 | 2021-08-03 | Video coding using intra sub-partition coding mode |
US17/393,364 Active 2040-11-20 US11936890B2 (en) | 2019-02-05 | 2021-08-03 | Video coding using intra sub-partition coding mode |
US18/438,398 Pending US20240195998A1 (en) | 2019-02-05 | 2024-02-09 | Video Coding Using Intra Sub-Partition Coding Mode |
US18/438,402 Pending US20240187623A1 (en) | 2019-02-05 | 2024-02-10 | Video Coding Using Intra Sub-Partition Coding Mode |
Family Applications After (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/393,364 Active 2040-11-20 US11936890B2 (en) | 2019-02-05 | 2021-08-03 | Video coding using intra sub-partition coding mode |
US18/438,398 Pending US20240195998A1 (en) | 2019-02-05 | 2024-02-09 | Video Coding Using Intra Sub-Partition Coding Mode |
US18/438,402 Pending US20240187623A1 (en) | 2019-02-05 | 2024-02-10 | Video Coding Using Intra Sub-Partition Coding Mode |
Country Status (7)
Country | Link |
---|---|
US (4) | US20210368205A1 (en) |
EP (1) | EP3922029A4 (en) |
JP (2) | JP2022518612A (en) |
KR (3) | KR20230049758A (en) |
CN (2) | CN113630607A (en) |
MX (1) | MX2021009355A (en) |
WO (1) | WO2020163535A1 (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20210076028A1 (en) * | 2018-03-30 | 2021-03-11 | Lg Electronics Inc. | Image/video coding method based on intra prediction, and device therefor |
US20210368193A1 (en) * | 2019-02-05 | 2021-11-25 | Beijing Dajia Internet Information Technology Co., Ltd. | Video coding using intra sub-partition coding mode |
US20220166968A1 (en) * | 2019-03-22 | 2022-05-26 | Lg Electronics Inc. | Intra prediction method and apparatus based on multi-reference line in image coding system |
US11818344B2 (en) * | 2019-07-08 | 2023-11-14 | Lg Electronics Inc. | Adaptive loop filter-based video or image coding |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113875236A (en) * | 2019-03-11 | 2021-12-31 | Vid拓展公司 | Intra sub-partition in video coding |
WO2023132622A1 (en) * | 2022-01-04 | 2023-07-13 | 엘지전자 주식회사 | Dimd mode-based intra prediction method and device |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130136175A1 (en) * | 2011-09-12 | 2013-05-30 | Qualcomm Incorporated | Non-square transform units and prediction units in video coding |
US20170347103A1 (en) * | 2016-05-25 | 2017-11-30 | Arris Enterprises Llc | Weighted angular prediction coding for intra coding |
US20180302620A1 (en) * | 2015-10-15 | 2018-10-18 | Lg Electronics Inc. | METHOD FOR ENCODING AND DECODING VIDEO SIGNAL, AND APPARATUS THEREFOR (As Amended) |
US20190149828A1 (en) * | 2016-05-02 | 2019-05-16 | Industry-Universty Cooperation Foundation Hanyang University | Image encoding/decoding method and apparatus using intra-screen prediction |
US20200244980A1 (en) * | 2019-01-30 | 2020-07-30 | Tencent America LLC | Method and apparatus for improved sub-block partitioning intra sub-partitions coding mode |
Family Cites Families (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8976870B1 (en) * | 2006-08-30 | 2015-03-10 | Geo Semiconductor Inc. | Block and mode reordering to facilitate parallel intra prediction and motion vector prediction |
KR101432775B1 (en) * | 2008-09-08 | 2014-08-22 | 에스케이텔레콤 주식회사 | Video Encoding/Decoding Method and Apparatus Using Arbitrary Pixel in Subblock |
KR102344178B1 (en) | 2011-11-11 | 2021-12-29 | 지이 비디오 컴프레션, 엘엘씨 | Effective prediction using partition coding |
US9491457B2 (en) * | 2012-09-28 | 2016-11-08 | Qualcomm Incorporated | Signaling of regions of interest and gradual decoding refresh in video coding |
KR102445274B1 (en) | 2012-10-01 | 2022-09-20 | 지이 비디오 컴프레션, 엘엘씨 | Scalable video coding using subblock-based coding of transform coefficient blocks in the enhancement layer |
US20170339403A1 (en) | 2014-11-04 | 2017-11-23 | Samsung Electronics Co., Ltd. | Method and device for encoding/decoding video using intra prediction |
RU2018132742A (en) | 2016-02-16 | 2020-03-17 | Самсунг Электроникс Ко., Лтд. | METHOD OF INTERNAL PREDICTION FOR REDUCING ERRORS OF INTERNAL PREDICTION AND DEVICE FOR THIS |
CN110024410B (en) | 2016-11-25 | 2023-07-28 | 株式会社Kt | Method for encoding and decoding video |
CN117041559A (en) * | 2016-12-07 | 2023-11-10 | 株式会社Kt | Method for decoding or encoding video and apparatus for storing video data |
CN111971959B (en) * | 2018-02-09 | 2024-06-14 | 弗劳恩霍夫应用研究促进协会 | Partition-based intra coding concept |
KR20210027477A (en) * | 2018-07-13 | 2021-03-10 | 프라운호퍼 게젤샤프트 쭈르 푀르데룽 데어 안겐반텐 포르슝 에. 베. | Split intra coding concept |
CN113225559B (en) * | 2018-10-12 | 2023-05-16 | Oppo广东移动通信有限公司 | Video signal encoding method and decoding method and apparatus thereof |
US11570436B2 (en) * | 2019-01-28 | 2023-01-31 | Apple Inc. | Video signal encoding/decoding method and device therefor |
US20200252608A1 (en) * | 2019-02-05 | 2020-08-06 | Qualcomm Incorporated | Sub-partition intra prediction |
MX2021009355A (en) * | 2019-02-05 | 2021-09-14 | Beijing Dajia Internet Information Tech Co Ltd | Video coding using intra sub-partition coding mode. |
KR20200109276A (en) * | 2019-03-12 | 2020-09-22 | 주식회사 엑스리스 | Method for encoding/decoidng video signal and apparatus therefor |
WO2021006612A1 (en) * | 2019-07-08 | 2021-01-14 | 현대자동차주식회사 | Method and device for intra prediction coding of video data |
-
2020
- 2020-02-05 MX MX2021009355A patent/MX2021009355A/en unknown
- 2020-02-05 WO PCT/US2020/016888 patent/WO2020163535A1/en unknown
- 2020-02-05 CN CN202111101969.4A patent/CN113630607A/en active Pending
- 2020-02-05 KR KR1020237010889A patent/KR20230049758A/en active Application Filing
- 2020-02-05 JP JP2021545471A patent/JP2022518612A/en active Pending
- 2020-02-05 KR KR1020217024197A patent/KR102517389B1/en active IP Right Grant
- 2020-02-05 EP EP20753053.6A patent/EP3922029A4/en active Pending
- 2020-02-05 KR KR1020227003953A patent/KR20220021036A/en active Search and Examination
- 2020-02-05 CN CN202080010728.6A patent/CN113348671A/en active Pending
-
2021
- 2021-08-03 US US17/393,367 patent/US20210368205A1/en active Pending
- 2021-08-03 US US17/393,364 patent/US11936890B2/en active Active
-
2023
- 2023-05-10 JP JP2023077868A patent/JP2023090929A/en active Pending
-
2024
- 2024-02-09 US US18/438,398 patent/US20240195998A1/en active Pending
- 2024-02-10 US US18/438,402 patent/US20240187623A1/en active Pending
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130136175A1 (en) * | 2011-09-12 | 2013-05-30 | Qualcomm Incorporated | Non-square transform units and prediction units in video coding |
US20180302620A1 (en) * | 2015-10-15 | 2018-10-18 | Lg Electronics Inc. | METHOD FOR ENCODING AND DECODING VIDEO SIGNAL, AND APPARATUS THEREFOR (As Amended) |
US20190149828A1 (en) * | 2016-05-02 | 2019-05-16 | Industry-Universty Cooperation Foundation Hanyang University | Image encoding/decoding method and apparatus using intra-screen prediction |
US20170347103A1 (en) * | 2016-05-25 | 2017-11-30 | Arris Enterprises Llc | Weighted angular prediction coding for intra coding |
US20200244980A1 (en) * | 2019-01-30 | 2020-07-30 | Tencent America LLC | Method and apparatus for improved sub-block partitioning intra sub-partitions coding mode |
Non-Patent Citations (2)
Title |
---|
Hernández et. al. CE3: Line-based intra coding mode (Tests 2.1.1 and 2.1.2) Joint Video Experts Team (JVET) ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 12th Meeting: Macao, CN, 3–12 Oct. 2018 (Hernández) * |
SANTIAGO DE-LUXAN-HERNANDEZ et al., CE3: Intra Sub-Partitions Coding Mode (Tests 1.1.1 and 1.1.2), Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-M0102-v5, 13th Meeting: Marrakech, MA, 18 January 2019, (9p). (cited in ISR) * |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20210076028A1 (en) * | 2018-03-30 | 2021-03-11 | Lg Electronics Inc. | Image/video coding method based on intra prediction, and device therefor |
US11706405B2 (en) * | 2018-03-30 | 2023-07-18 | Lg Electronics Inc. | Image/video coding method based on intra prediction involving parsing of MPM index, and apparatus thereof |
US20230308637A1 (en) * | 2018-03-30 | 2023-09-28 | Lg Electronics Inc. | Image/video coding method based on intra prediction involving parsing of mpm index, and apparatus thereof |
US20210368193A1 (en) * | 2019-02-05 | 2021-11-25 | Beijing Dajia Internet Information Technology Co., Ltd. | Video coding using intra sub-partition coding mode |
US11936890B2 (en) * | 2019-02-05 | 2024-03-19 | Beijing Dajia Internet Information Technology Co., Ltd. | Video coding using intra sub-partition coding mode |
US20220166968A1 (en) * | 2019-03-22 | 2022-05-26 | Lg Electronics Inc. | Intra prediction method and apparatus based on multi-reference line in image coding system |
US11818344B2 (en) * | 2019-07-08 | 2023-11-14 | Lg Electronics Inc. | Adaptive loop filter-based video or image coding |
Also Published As
Publication number | Publication date |
---|---|
CN113630607A (en) | 2021-11-09 |
US20210368193A1 (en) | 2021-11-25 |
CN113348671A (en) | 2021-09-03 |
US20240187623A1 (en) | 2024-06-06 |
JP2023090929A (en) | 2023-06-29 |
JP2022518612A (en) | 2022-03-15 |
EP3922029A4 (en) | 2022-08-24 |
MX2021009355A (en) | 2021-09-14 |
KR20220021036A (en) | 2022-02-21 |
KR102517389B1 (en) | 2023-04-03 |
WO2020163535A1 (en) | 2020-08-13 |
EP3922029A1 (en) | 2021-12-15 |
KR20230049758A (en) | 2023-04-13 |
KR20210113259A (en) | 2021-09-15 |
US20240195998A1 (en) | 2024-06-13 |
US11936890B2 (en) | 2024-03-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US12034921B2 (en) | Apparatus and method for applying artificial neural network to image encoding or decoding | |
US11936890B2 (en) | Video coding using intra sub-partition coding mode | |
US20130272623A1 (en) | Intra prediction method and encoding apparatus and decoding apparatus using same | |
EP3522538A1 (en) | Image processing method and apparatus therefor | |
US20220014741A1 (en) | Methods and devices for intra sub-partition coding mode | |
US11991378B2 (en) | Method and device for video coding using various transform techniques | |
US12003737B2 (en) | Coding of transform coefficients in video coding | |
KR20230058033A (en) | Method and Apparatus for Video Encoding or Decoding | |
WO2024146511A1 (en) | Representative prediction mode of a block of pixels | |
WO2023198105A1 (en) | Region-based implicit intra mode derivation and prediction | |
WO2024022144A1 (en) | Intra prediction based on multiple reference lines | |
WO2024131778A1 (en) | Intra prediction with region-based derivation | |
WO2024016955A1 (en) | Out-of-boundary check in video coding | |
WO2023241347A1 (en) | Adaptive regions for decoder-side intra mode derivation and prediction | |
WO2023197998A1 (en) | Extended block partition types for video coding | |
WO2023217235A1 (en) | Prediction refinement with convolution model | |
WO2023217140A1 (en) | Threshold of similarity for candidate list | |
WO2023198187A1 (en) | Template-based intra mode derivation and prediction | |
WO2023208219A1 (en) | Cross-component sample adaptive offset | |
WO2023154574A1 (en) | Methods and devices for geometric partitioning mode with adaptive blending | |
WO2023158765A1 (en) | Methods and devices for geometric partitioning mode split modes reordering with pre-defined modes order | |
KR20240051257A (en) | Method and device for deriving intra mode on the decoder side | |
WO2023250047A1 (en) | Methods and devices for motion storage in geometric partitioning mode | |
WO2023081322A1 (en) | Intra prediction modes signaling | |
EP4049456A1 (en) | Methods and devices for lossless coding modes in video coding |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: BEIJING DAJIA INTERNET INFORMATION TECHNOLOGY CO., LTD., CHINA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:CHEN, YI-WEN;XIU, XIAOYU;WANG, XIANGLIN;AND OTHERS;SIGNING DATES FROM 20200128 TO 20200130;REEL/FRAME:057075/0780 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE AFTER FINAL ACTION FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: ADVISORY ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |