CN113812160A - Maximum possible mode list construction for screen content codec - Google Patents

Abstract

The construction of a maximum possible mode list for screen content codec is described. In an exemplary aspect, a method for video processing comprises: constructing two or more Maximum Possible Mode (MPM) lists of an intra-coding mode of a current block for a transition between the current block and a bitstream representation of the current block, wherein the two or more MPM lists include at least a first MPM list constructed using a first construction method and a second MPM list constructed using a second construction method different from the first construction method; and performing conversion using one MPM list selected from the constructed two or more MPM lists.

Description

Maximum possible mode list construction for screen content codec

Cross Reference to Related Applications

The present application claims timely priority and benefit from international patent application No. PCT/CN2019/086114 filed on 9/5/2019, in accordance with the regulations of the applicable patent laws and/or paris convention. The entire disclosure of international patent application No. PCT/CN2019/086114 is incorporated herein by reference as part of the disclosure of the present application.

Technical Field

This document relates to video encoding and decoding techniques, devices and systems.

Background

Despite advances in video compression, digital video still occupies the largest bandwidth usage on the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the demand for bandwidth for digital video usage is expected to continue to grow.

Disclosure of Invention

Apparatus, systems, and methods related to digital video coding and, more particularly, to adaptive loop filtering for video coding are described. The method may be applied to existing video codec standards, such as High Efficiency Video Codec (HEVC), as well as future video codec standards, such as multifunctional video codec (VVC), or codecs.

Video codec standards have been developed primarily through the development of the well-known ITU-T and ISO/IEC standards. ITU-T developed H.261 and H.263, ISO/IEC developed MPEG-1 and MPEG-4 visuals, and both organizations jointly developed the H.262/MPEG-2 video, H.264/MPEG-4 Advanced Video Codec (AVC), and H.265/HEVC standards. Since h.262, the video codec standard is based on a hybrid video codec structure, in which temporal prediction plus transform coding is employed. To explore future video codec technologies beyond HEVC, VCEG and MPEG have together established the joint video exploration team (jfet) in 2015. Since then, JFET has adopted many new approaches and applied them to a reference software named Joint Exploration Model (JEM). In month 4 of 2018, a joint video experts group (jfet) between VCEG (Q6/16) and ISO/IEC JTC1 SC29/WG11(MPEG) was created to work for the VVC standard with a goal of 50% bit rate reduction compared to HEVC.

In one representative aspect, the disclosed technology may be used to provide a method for video processing that includes constructing a first mode list of intra-coding modes based on intra-modes of neighboring blocks to a current video block; constructing a second mode list of intra-coding and decoding modes based on the first mode list; and performing intra-mode coding on the current video block using at least one of the first mode list or the second mode list.

In another representative aspect, the disclosed technology may be used to provide a method for video processing, the method comprising: constructing a mode list of an intra-coded mode to include the candidate mode based on intra-modes of neighboring blocks to the current video block, wherein constructing the mode list comprises locating the candidate mode at a position in the mode list based on a type of the candidate mode.

In one representative aspect, a method for video processing may be provided using the disclosed techniques. The method includes constructing two or more Maximum Possible Mode (MPM) lists for an intra codec mode of a current block for a transition between the current block and a bitstream representation of the current block, wherein the two or more MPM lists include at least a first MPM list constructed using a first construction method and a second MPM list constructed using a second construction method different from the first construction method; and performing conversion using one MPM list selected from the constructed two or more MPM lists.

In yet another representative aspect, the above-described method is implemented in the form of processor executable code and stored in a computer readable program medium.

In yet another representative aspect, an apparatus configured or operable to perform the above-described method is disclosed. The apparatus may include a processor programmed to implement such a method.

In yet another exemplary aspect, a video decoding apparatus implementing the video decoding method described herein is disclosed.

The above and other aspects and features of the disclosed technology are described in more detail in the accompanying drawings, the description and the claims.

Drawings

Fig. 1 shows an example of intra block copy.

Fig. 2 shows an example of a block coded in palette mode.

Fig. 3 shows an example of signaling palette entries using palette predictor signaling.

Fig. 4 shows an example of horizontal and vertical traversal scans for coding palette indices.

Fig. 5 shows an example of coding and decoding of palette indices.

FIG. 6 illustrates an example of a multi-type tree partitioning pattern.

Fig. 7 shows an example of the positions of samples used to derive α and β.

Fig. 8 shows an example of luminance mapping with a chroma scaling architecture.

Fig. 9 shows an example of 67 intra prediction modes.

Fig. 10 shows an example of left and upper neighboring blocks of a current block.

Fig. 11 shows an example of four reference lines adjacent to a prediction block.

Fig. 12a shows an example of sub-partitioning for 4x8 and 8x4 CUs, and fig. 12b shows an example of sub-partitioning for CUs other than 4x8, 8x4, and 4x 4.

13a and 13b illustrate a flow diagram of an example method for schema list construction in accordance with the disclosed technology.

Fig. 14 is a block diagram of an example of a hardware platform for implementing the visual media decoding or visual media encoding techniques described herein.

Fig. 15 is a flow diagram of an example method for video coding.

Detailed Description

Due to the increasing demand for higher resolution video, video encoding and decoding methods and techniques are ubiquitous in modern technology. Video codecs typically include electronic circuits or software that compress and decompress digital video and are continually being improved to provide higher codec efficiency. The video codec converts uncompressed video into a compressed format or vice versa. There is a complex relationship between video quality, the amount of data used to represent the video (as determined by the bit rate), the complexity of the encoding and decoding algorithms, the sensitivity to data loss and errors, ease of editing, random access, and end-to-end delay (lag). The compression format typically complies with a standard video compression specification, such as the High Efficiency Video Codec (HEVC) standard (also known as h.265 or MPEG-H section 2), the upcoming multi-function video codec (VVC) standard, or other current and/or future video codec standards.

Embodiments of the disclosed techniques may be applied to existing video codec standards (e.g., HEVC, h.265) and future standards to improve runtime performance. Section headings are used herein to enhance readability of the description, and do not in any way limit discussion or embodiments (and/or implementations) to only the respective sections.

1 Intra Block copy

Intra Block Copy (IBC), also known as current picture reference, has been used in HEVC screen content codec extensions (HEVC-SCC) and current VVC test model (VTM-4.0). IBC extends the concept of motion compensation from inter-coding to intra-coding. As shown in fig. 1, when IBC is applied, a current block is predicted by a reference block in the same picture. Samples in the reference block must have been reconstructed before the current block is encoded or decoded. Although IBC is not as efficient for most camera capture sequences, it exhibits significant codec gain for screen content. The reason is that there are a large number of repeating patterns, such as icons and text characters, in the screen content picture. IBC can effectively eliminate redundancy between these repeating patterns. In HEVC-SCC, if IBC selects the current picture as its reference picture, a Coding Unit (CU) of inter coding may apply IBC. In this case, the MV is renamed to a Block Vector (BV), which always has integer-pixel precision. To be compatible with the main grade HEVC, a current picture is marked as a "long-term" reference picture in a Decoded Picture Buffer (DPB). It should be noted that similarly, in many view/3D video coding standards, inter-view reference pictures are also labeled as "long-term" reference pictures.

Finding its reference blocks following BV, predictions can be generated by copying the reference blocks. The residual may be obtained by subtracting the reference pixel from the initial signal. The transform and quantization may then be applied as in the other codec modes.

However, some or all of the pixel values are not defined when the reference block is outside the picture, or overlaps the current block, or is outside the reconstruction region, or outside the active region limited by some constraints. Basically, there are two approaches to deal with such problems. One approach is to disallow such conditions, for example, in bitstream consistency. Another approach is to apply padding to those undefined pixel values. The following subsections describe the protocol in detail.

IBC in 1.2 HEVC screen content codec extensions

In the screen content codec extension of HEVC, when a block uses the current picture as a reference, it should be guaranteed that the entire reference block is in the available reconstruction region, as shown in the following specification text.

The variables offsetX and offsetY are derived as follows:

offsetX＝(ChromaArrayType＝＝0)？0:(mvCLX[0]&0x72:0)

[ equation 1]

offsetY＝(ChromaArrayType＝＝0)？0:(mvCLX[1]&0x72:0)

[ equation 2]

One requirement of bitstream conformance is that when the reference picture is in the current picture, the luma motion vector mvLX should obey the following constraints:

-upon invoking the derivation procedure of z-scan order block availability specified in clause 6.4.1, wherein the adjacent luminance positions (xNbY, yNbY) are taken as inputs, which are set equal to (xCurr, yCurr) (xCb, yCb), and which are set equal to (xPb + (mvLX [0] > >2) -offset x, yPb + (mvLX [1] > >2) -offset, then the output should be equal to true.

-upon invoking the derivation procedure of z-scan order block availability specified in clause 6.4.1, wherein as inputs (xCurr, yCurr) set equal to (xCb, yCb), and adjacent luminance positions (xNbY, yNbY) set equal to (xPb + (mvLX [0] > >2) + nPbW-1+ offset x, yPb + (mvLX [1] > >2) + nPbH-1+ offset y), the output should be equal to true.

One or both of the following conditions should be true:

(mvLX [0] > >2) + nPbW + xB1+ offset X has a value less than or equal to 0.

(mvLX [1] > >2) + nPbH + yB1+ offset Y has a value less than or equal to 0.

The following condition should be true:

(xpB + (mvLX [0] > >2) + nPbSw-1+ offset X)/CtbSizeY-xCyr/CtbSizeY < (yCurr/CtbSizeY- (yPb + (mvLX [1] > >2) + nPbSh-1+ offset Y)/CtbSizeY [ equation 3]

Thus, no situation occurs where the reference block overlaps the current block or the reference block is outside the picture. No padding of reference blocks or prediction blocks is required.

1.3 IBC in VCC test model

In the current VCC test model, i.e., VTM-4.0 design, the entire reference block should utilize the current Codec Tree Unit (CTU) and not overlap with the current block. Therefore, there is no need to pad the reference block or the prediction block. The IBC flag is coded as the prediction mode of the current CU. Thus, there are three prediction MODEs for each CU, MODE _ INTRA, MODE _ INTER, and MODE _ IBC.

1.3.1.IBC Merge mode

In IBC Merge mode, indices pointing to entries in the IBC Merge candidate list are parsed from the bitstream. The construction of the IBC merge list can be summarized according to the following sequence of steps:

step 1: derivation of spatial domain candidates

Step 2: insertion of HMVP candidates

And step 3: insertion of pairwise averaged candidates

In deriving spatial merge candidates, a maximum of four merge candidates are selected among candidates located at the positions shown by a1, B1, B0, a0, and B2. The order of derivation was a1, B1, B0, a0, and B2. Position B2 is only considered when any PU at position a1, B1, B0, a0 is unavailable (e.g., because it belongs to another stripe or slice) or is not coded with IBC mode. After adding the candidate at position a1, the insertion of the remaining candidates is subjected to a redundancy check, which ensures that candidates with the same motion information are excluded from the list, thereby improving the codec efficiency.

After inserting the spatial domain candidates, if the IBC merge list size is still smaller than the maximum IBC merge list size, the IBC candidates from the HMVP table may be inserted. Redundancy checking is performed when HMVP candidates are inserted.

Finally, the paired average candidates are inserted into the IBC merge list.

A merge candidate is referred to as an invalid merge candidate when the reference block identified by the merge candidate is outside the picture, or overlaps with the current block, or is outside the reconstruction region, or is outside the valid region bounded by some constraints.

Note that an invalid merge candidate may be inserted into the IBC merge list.

1.3.2 IBC AMVP mode

In the IBC AMVP mode, AMVP indices pointing to entries in the IBC AMVP list are parsed from the bitstream. The construction of the IBC AMVP list can be summarized according to the following sequence of steps:

step 1: derivation of spatial domain candidates

Check a0, a1 until a usable candidate is found.

B0, B1, B2 were checked until a usable candidate was found.

Step 2: insertion of HMVP candidates

And step 3: insertion of zero candidates

After inserting the spatial domain candidates, an IBC candidate from the HMVP table may be inserted if the IBC AMVP list size is still less than the maximum IBC AMVP list size.

Finally, zero candidates are inserted into the IBC AMVP list.

1.4 Adaptive Motion Vector Resolution (AMVR)

In HEVC, when use _ integer _ mv _ flag is equal to 0 in the slice header, a motion vector difference value (MVD) (between the motion vector of the CU and the predicted motion vector) is signaled in units of quarter luminance samples. In VVC, an Adaptive Motion Vector Resolution (AMVR) scheme at the CU level is introduced. AMVR allows MVD coding of CUs with different precisions. Depending on the mode of the current CU (normal AMVP mode or affine AVMP mode), the MVD of the current CU may be adaptively selected as follows:

normal AMVP mode: a quarter luminance sample, an integer luminance sample, or four luminance samples.

Affine AMVP mode: quarter luminance samples, integer luminance samples, or 1/16 luminance samples.

Conditionally signaling a CU level MVD resolution indication if the current CU has at least one non-zero MVD component. If all the MVD components (i.e., horizontal and vertical MVDs for reference list L0 and reference list L1) are zero, then a quarter luminance sample MVD resolution is inferred.

For a CU with at least one non-zero MVD component, a first flag is signaled to indicate whether quarter luma sample MVD precision is used for the CU. If the first flag is 0, no further signaling is needed and one-quarter luma sample MVD precision is used for the current CU. Otherwise, a second flag is signaled to indicate whether integer luma sample or four luma sample MVD precision is used for the normal AMVP CU. The same second flag is used to indicate whether integer or 1/16 luma samples MVD precision is used for affine AMVP CU. To ensure that the reconstructed MV has the desired precision (quarter luma samples, integer luma samples, or four luma samples), the motion vector predictor of the CU will be rounded to the same precision as the MVD before it was added together. Rounding the motion vector predictor towards zero (i.e. rounding to the positive infinity for the negative motion vector predictor and rounding to the negative infinity for the positive motion vector predictor).

The encoder determines the motion vector resolution for the current CU using RD checking. To avoid always performing three RD checks at the CU level for each MVD resolution, only the RF check of MVD precision beyond a quarter luminance sample is conditionally invoked. For normal AVMP mode, the RD cost is first calculated for the quarter luma sample MVD precision and the integer luma sample MV precision. Then, the RD cost of the integer-sample MVD precision is compared with the quarter-luminance-sample MVD precision to determine whether further checking of the RD cost of the four-luminance-sample MVD precision is required. And skipping RD checking of the four brightness sample points MVD precision when the RD cost of the one-fourth brightness sample point MVD precision is far less than the integral brightness sample point MVD precision. For affine AMVP mode, if the affine inter mode is not selected after checking the rate-distortion cost of affine merge/skip mode, quarter luma sample MVD precision normal AMVP mode, and quarter luma sample MVD precision affine AMVP mode, then the luma sample MV precision and 1-pixel MV precision affine inter mode are not checked 1/16. Further, the affine parameters obtained in the quarter-luminance-sample-MV-precision affine inter mode are used as starting search points in the 1/16 luminance samples and the quarter-luminance-sample-MV-precision affine inter mode.

Palette mode in 1.5 HEVC screen content codec extension (HEVC-SCC)

The basic idea behind palette mode is to represent samples in a CU by a small set of representative color values. This set is called a palette. Samples outside the palette may also be indicated by signaling escape symbols, followed by (possibly quantized) component values. This is illustrated in fig. 2.

1.5.1 palette entries codec

To codec a palette entry, a palette predictor is maintained. The palette and the maximum size of the palette predictor are signaled in the SPS. In HEVC-SCC, palette _ predictor _ initializer _ present _ flag is introduced in PPS. When this flag is 1, an entry for initializing the palette predictor is signaled in the bitstream. The palette predictor is initialized at the beginning of each CTU row, each stripe, and each slice. The palette predictor is reset to 0 according to the value of palette _ predictor _ initializer _ present _ flag, or initialized using a palette predictor initializer entry signaled in the PPS. In HEVC-SCC, a size 0 palette predictor initializer is enabled to allow explicit disabling of palette predictor initialization at PPS level.

For each entry in the palette predictor, a reuse flag is signaled to indicate whether it is part of the current palette. This is illustrated in fig. 3. The reuse flag is sent using a run length codec of zero. After this, the number of new palette entries is signaled using a0 th order exponential Golomb code. Finally, the component values for the new palette entries are signaled.

1.5.2 coding and decoding of palette indices

The palette indices are encoded and decoded using horizontal and vertical traversal scans as shown in fig. 4. The scanning order is explicitly signaled in the bitstream using palette _ transpose _ flag. For the remainder of the bar, the scan is assumed to be horizontal.

Two palette sampling modes are used: 'INDEX' and 'COPY _ ABOVE', encode and decode the palette INDEX. As previously described, the escape symbol is also signaled as 'INDEX' mode and assigned an INDEX equal to the maximum palette size. In addition to the top row, or when the previous pattern is 'COPY ABOVE', the pattern is signaled using a flag. In the 'COPY _ ABOVE' mode, the palette indices of the samples in the upper row are copied. In the 'INDEX' mode, the palette INDEX is explicitly signaled. For both 'INDEX' and 'COPY ABOVE' modes, a running value is signaled that specifies the number of subsequent samples that are also coded using the same mode. When an escape symbol is part of an operation in 'INDEX' or 'COPY _ ABOVE' mode, an escape component value is signaled for each escape symbol. The encoding and decoding of the palette indices is shown in fig. 5.

This syntax order is accomplished as follows. First, the number of index values of a CU is signaled. After this, the actual index value of the entire CU is signaled using truncated binary codec. Both the number of indices and the index value are coded in bypass mode. This groups together the index-dependent bypass bits. The palette sample pattern (if necessary) and operation are then signaled in an interleaved fashion. Finally, the component escape values corresponding to the escape samples for the entire CU are grouped together and coded in bypass mode.

After signaling the index value, an additional syntax element, last _ run _ type _ flag, is signaled. This syntax element, in combination with the number of indices, eliminates the need to signal the running value corresponding to the last run in the block.

In HEVC-SCC, the palette mode is also enabled for 4:2:2, 4:2:0, and monochrome chroma formats. The signaling of palette entries and palette indices is almost the same for all chroma formats. For non-monochrome formats, each palette entry consists of 3 components. For the monochrome format, each palette entry consists of a single component. For the under-sampled chroma direction, chroma samples are associated with a luma sample index that can be divided by 2. After the palette index is reconstructed for the CU, only the first component of the palette entry is used if the sample point has only a single component associated with it. Only the difference in signaling is for the escape component values. For each escape sample point, the number of signaled escape component values may be different depending on the number of components associated with the sample point.

1.6 coefficient coding in Skip mode

Several modifications are proposed to the coefficient codec in the transform skip (ts) mode to adapt the residual codec to the statistics and signal characteristics of the transform skip level.

The proposed modifications are listed below.

No last significant scan position:since the residual signal reflects the spatial residual after prediction and no energy compaction is performed by the transform for TS, no higher probability of trailing zero or insignificant level at the lower right corner of the transform block is given. The last active scan position signalling is then omitted in this case.

Sub-block CBF:leaving out the last valid scan position signaling requires modifying the sub-block CBF signaling with coded sub-block flag for TS as follows.

Due to quantization, the aforementioned invalid sequence may still occur inside the transform block. Then, the last valid scan position is eliminated as described above, and the coded _ sub _ block _ flag is coded and decoded for all the sub blocks, excluding the case where all CGs (except the last CG) have zero coefficients, so that the coded _ sub _ block _ flag does not need to be coded and decoded for the last CG.

A special case is given for the coded _ sub _ block _ flag of the sub-block covering the DC frequency position (upper left sub-block). In VVC draft 3, the coded _ sub _ block _ flag for that subblock is never signaled and is always inferred to be equal to 1. When the last active scan position is in another sub-block, this means that there is at least one active level outside the DC sub-block. Thus, a DC sub-block may contain only zero/non-active levels, although the coded _ sub _ block _ flag for that sub-block is inferred to be equal to 1. In the case where there is no last scanning position information in the TS, coded _ sub _ block _ flag for each subblock is signaled.

This also includes a coded sub block flag for the DC subblock, except when all other coded sub block flag syntax elements have been equal to 0. In this case, DC coded _ sub _ block _ flag is inferred to be equal to 1 (preferdcsbcbf ═ 1). Since there is at least one active level in this DC sub-block, the sig _ coeff _ flag syntax element for the first position at (0,0) is not signaled and derived equal to 1(infersbdcsigcoeff flag ═ 1), instead of if all other sig _ coeff _ flag syntax elements in this DC sub-block are equal to 0.

The context modeling for the coded _ sub _ block _ flag is changed. And calculating the context model index as the sum of a coded _ sub _ block _ flag at the left side of the current subblock and a coded _ sub _ block _ flag at the upper side of the current subblock, and logically separating the coded _ sub _ block _ flag from the coded _ sub _ block _ flag.

sig _ coeff _ flag context modeling:the local template in the sig _ coeff _ flag context modeling is modified to include only the neighbor to the left of the current scan position (NB0) and the neighbor above (NB 1). The context model offset is the significant proximity sig _ coeff _ flag NB0]+sig_coeff_flag[NB1]The number of the cells. Thus, the selection of a different context set depending on the diagonal d within the current transform block is eliminated. Three context models and a single set of context models are thus obtained for coding and decoding sig _ coeff _ flag.

abs _ level _ 1_ flag and par _ level _ flag context modeling: a single context model is employed for abs _ level _ gt1_ flag and par _ level _ flag.

abs _ remaining codec: although the empirical distribution of transform skip residual absolute levels typically still matches the laplacian or geometric distribution, there is a greater instability than the transform coefficient absolute levels. In particular, the variations within the successively implemented windows are higher for the residual absolute level. This facilitates the following modifications of abs _ remaining syntax binarization and context modeling:

higher truncation values are used in binarization, i.e. transition points from codec with sig _ coeff _ flag, abs _ level _ 1_ flag, par _ level _ flag and abs _ level _ 3_ flag to Rice code for abs _ remaining, and higher compression efficiency is obtained for the dedicated context model for each binary bit position. Raising the truncation value will result in more "greater than X" flags, e.g., introducing abs _ level _ gt5_ flag, abs _ level _ gt7_ flag, etc., until the truncation value is reached. The cutoff value itself is fixed to 5(numGtFlags ═ 5).

The template for rice parameter derivation is modified, i.e. only the neighbors to the left of the current scan position and the neighbors above are considered, similar to the local template modeled for the sig _ coeff _ flag context.

coeff _ sign _ flag context modeling:due to the non-stationarity inside the symbol sequence and the fact that the prediction residual is often biased, the symbols can be coded using a context model even when the global empirical distribution is almost uniformly distributed. The symbols are coded using a single dedicated context model and parsed after sig _ coeff _ flag to keep the bits of all context codings together.

1.7 quantized residual Block differential pulse code modulation (QR-BDPCM)

Quantized residual block differential pulse code modulation (QR-BDPCM) is proposed to efficiently codec screen content.

The prediction directions used in the QR-BDPCM may be a vertical prediction mode and a horizontal prediction mode. The entire block is intra-predicted by sample copy in a prediction direction (horizontal or vertical prediction) similar to the intra-prediction. The residual is quantized and the delta between the quantized residual and its predictor (horizontal or vertical) quantization is coded. This can be described as follows: for a block of size M (rows) N (columns), let r (i, j), 0 ≦ i ≦ M-1,0 ≦ j ≦ N-1 be the prediction residual after performing intra prediction either horizontally (copying left-side neighboring pixel values line-by-line within the prediction block) or vertically (copying the top neighboring line to each line in the prediction block) using unfiltered samples from the top or left block boundary samples. Let Q (r _ (i, j)), 0 ≦ i ≦ M-1,0 ≦ j ≦ N-1 denote a quantized version of the residual r _ (i, j), which is the difference between the initial block and the predicted block values. Then, block DPCM is applied to the quantized residual samples, resulting in a modified M N array R, whose elements are

Signaling vertical BDPCM:

for horizontal prediction, a similar rule applies, and residual quantized samples are obtained by the following equation:

quantizing the residual error to sample points

To the decoder.

At the decoder side, the above calculations are reversed to produce Q (r)_i,j) I is more than or equal to 0 and less than or equal to M-1, and j is more than or equal to 0 and less than or equal to N-1. For the case of a vertical prediction,

in the case of the horizontal case,

inverse quantized residual, Q^-1(Q(r_i,j) Added to the intra block predictor to produce reconstructed sample values.

The main benefit of this approach is that it works during coefficient parsing, simply by adding predictors when the coefficients are parsed, or it can do so after parsing.

Draft text changes for QR-BDPCM are shown below.

7.3.6.5 codec Unit syntax

Bdpcm _ flag [ x0] [ y0] equal to 1 specifies that there is a bdpcm _ dir _ flag in the codec unit comprising the luma codec block at position (x0, y0)

Bdpcm _ dir _ flag x0 y0 equal to 0 specifies that the prediction direction to be used in the bdpcm block is horizontal, otherwise vertical.

1.8 segmentation Structure

In HEVC, various local characteristics are accommodated by dividing the CTUs into CUs using a quadtree structure (denoted as a coding tree). At the leaf-CU level, it is decided whether to encode or decode a picture region using inter-picture (temporal) prediction or intra-picture (spatial) prediction. Each leaf-CU may be further divided into one, two, or four PUs, depending on the partition type of the PU. In one PU, the same prediction process is applied and the relevant information is transmitted to the decoder on a PU basis. After obtaining the residual block by applying the prediction process based on the PU partition type, the leaf-CU may be partitioned into Transform Units (TUs) according to another quadtree structure similar to a coding/decoding tree of the CU. An important feature of the HEVC structure is that it has multiple partitioning concepts, including CU, PU, and TU.

In VVC, a quadtree with nested multi-type trees using binary and ternary partition segmentation structures replaces the concept of multiple partition unit types, i.e., it eliminates the separation of CU, PU and TU concepts unless the CU needs to be oversized for the maximum transform length and supports greater flexibility of CU partition shapes. In the coding tree structure, a CU may be square or rectangular. A Codec Tree Unit (CTU) is first partitioned by a quadtree (also known as quadtree) structure. The quadtree leaf nodes may then be further partitioned by a multi-type tree structure. As shown in fig. 6, there are four division types in the multi-type tree structure, a vertical binary division (SPLIT _ BT _ VER), a horizontal binary division (SPLIT _ BT _ HOR), a vertical three-way division (SPLIT _ TT _ VER), and a horizontal three-way division (SPLIT _ TT _ HOR). The multi-type leaf nodes are called Codec Units (CUs), unless the CUs are too large for the maximum transform length, such partitions are used for prediction and transform processing without any further partitioning. This means that in most cases, in a quadtree with a nested multi-type tree codec block structure, a CU, a PU, and a TU have the same block size. An exception occurs when the maximum supported transform length is less than the width or height of the color component of the CU. In addition, the luminance and chrominance components have separate partitioning structures on the I-slice.

1.9 Cross-component Linear model prediction

To reduce cross-component redundancy, a cross-component linear model (CCLM) prediction mode is used in VTM4, for which chroma samples are predicted based on luma samples of the same CU reconstructed using a linear model as follows:

pred_C(i，j)＝α·rec_L' (i, j) + β [ equation 8)]

Where represents predicted chroma samples in a CU and represents downsampled reconstructed luma samples in the same CU. The linear model parameters α and β are derived from the relationship between the luminance and chrominance values from two samples, the two samples being the luminance sample having the smallest and largest sample values within the set of downsampled neighboring luminance samples, and its corresponding chrominance sample. The linear model parameters α and β are obtained according to the following equations.

β＝Y_b-α·X_b[ equation 10]

Wherein Y is_aAnd X_aRepresenting the luminance and chrominance values of the luminance sample having the largest luminance sample value. And X_bAnd Y_bRespectively, the luminance value and the chrominance value of the luminance sample having the smallest luminance sample value. Fig. 8 shows an example of left and upper samples and the position of the samples of the current block involved in CCLM mode.

2.10 luminance mapping with chroma scaling (LMCS)

In VTM4, a codec tool called luma map with chroma scaling (LMCS) is added as a new processing block before the loop filter. LMCS has two main components: 1) a loop map of the luminance component based on an adaptive piece-by-piece linear model; 2) for the chroma component, luma-dependent chroma residual scaling is applied. Figure 8 shows the LMCS architecture from the decoder perspective. The shaded blocks in FIG. 8 indicate where in the map domain the process is applied; these include inverse quantization, inverse transformation, luma intra prediction, and adding luma prediction along with luma residual. The unshaded blocks in FIG. 8 indicate where the process is applied in the initial (i.e., unmapped) domain; these include loop filters such as deblocking, ALF and SAO, motion compensated prediction, chroma intra prediction, adding chroma prediction along with chroma residual, and storing decoded pictures as reference pictures. The light yellow shaded block in fig. 8 is the new LMCS functional block, including the forward and inverse mapping of the luminance signal and the luminance-dependent chroma scaling process. Like most other tools in VVC, the LMCS may be enabled/disabled using the SPS flag at the sequence level.

2.11 Intra mode codec with 67 Intra prediction modes

To capture any edge direction present in natural video, the number of directional intra modes in VTM4 is extended from 33 (as used in HEVC) to 65. The new directional mode that is not in HEVC is depicted in fig. 9 as a red dashed arrow, the plane and the DC mode remain the same. These denser directional intra prediction modes are applicable to all block sizes as well as both luma and chroma intra prediction.

A unified 6-MPM list is proposed for intra blocks, regardless of whether MRL and ISP codec tools are applicable or not. Constructing MPM lists based on left intra modes

The syntax elements intra _ luma _ mpm _ flag [ x0] [ y0], intra _ luma _ not _ player _ flag [ x0] [ y0], intra _ luma _ mpm _ idx [ x0] [ y0] and intra _ luma _ mpm _ remaining [ x0] [ y0] specify the intra prediction modes for luma samples. The array indices x0, y0 specify the position of the top-left luma sample of the considered codec block relative to the top-left luma sample of the picture (x0, y 0). When intra _ luma _ mpm _ flag x0 y0 is equal to 1, the intra prediction mode is inferred from the codec units adjacent to the intra prediction according to clause 8.4.2.

In the absence of intra _ luma _ mpm _ flag x0 y0 (e.g., ISP enabled, or MRL enabled (reference index >0)), it is inferred to be equal to 1.

In the absence of intra _ luma _ not _ player _ flag x0 y0 (e.g., MRL enabled), it is inferred to be equal to 1.

8.4.2 derivation of luma Intra prediction modes

The inputs to this process are:

-a luminance position (xCb, yCb) specifying an upper left luma sample of the current luma codec block relative to an upper left luma sample of the current picture,

a variable cbWidth specifying the width of the current codec block in the luminance samples,

a variable cbHeight specifying the height of the current codec block in the luma samples.

In this process, the luma intraprediction mode IntraPredModey [ xCb ] [ yCb ] is derived.

Table 8-1 specifies the values and associated names of the IntraPredModey [ xCb ] [ yCb ] intra prediction modes.

TABLE 8-1-Specification of Intra prediction modes and associated names

NOTE-the INTRA prediction modes INTRA _ LT _ CCLM, INTRA _ L _ CCLM and INTRA _ T _ CCLM are applicable only to the chroma components.

IntraPredModey [ xCb ] [ yCb ] was derived as follows:

-if intra _ luma _ not _ player _ flag [ xCb ] [ yCb ] is equal to 1, the following sequential steps:

1. adjacent positions (xNbA, yNbA) and (xNbB, yNbB) are set equal to (xCb-1, yCb + cbHeight-1) and (xCb + cbWidth-1, yCb-1), respectively.

2. For the case where X is replaced by A or B, the variable candIntraPredModex is derived as follows:

-an availability derivation procedure for the blocks specified in clause 6.4.X [ Ed. (BB): a neighboring block availability check procedure tbd ] is invoked, wherein a position (xCurr, yCurr) equal to (xCb, yCb) and a neighboring position (xNbY, yNbY) equal to (xNbX, yNbX) are set as inputs, and the output is assigned to availableX.

-deriving the candidate intra prediction mode candlntrapredmodex as follows:

-candlntrapredmodex is set equal to INTRA _ plan if one or more of the following conditions is true.

The variable availableX is equal to false.

CuPredMode [ xNbX ] [ yNbX ] is not equal to MODE _ INTRA, ciip _ flag [ xNbX ] [ yNbX ] is not equal to 1.

-pcm _ flag [ xNbX ] [ yNbX ] equals 1.

X is equal to B, yCb-1 is less than ((yCb > > CtbLog2SizeY) < < CtbLog2 SizeY).

-otherwise candIntraPredModeX is set equal to IntraPredModeY [ xNbX ] [ yNbX ].

3. candModeList [ x ], where x ═ 0..4, is derived as follows.

If candrampredmodeb is equal to candrampredmodea and candrampredmodea is greater than INTRA _ DC, candModeList [ x ] is derived as follows, where x is 0.

candModelist [0] ═ candIntrarPredModea [ equation 11]

candModelist [1] ((candIntraPredModeA + 61)% 64) [ equation 12]

candModelist [2] ((candIntraPredModeA-1)% 64) [ equation 13]

candModelist [3] ═ INTRA _ DC [ equation 14]

candModelist [4] ((candIntraPredModeA + 60)% 64) [ equation 15]

Otherwise, if candirapredmodeb does not equal candirapredmodea, candirapredmodea or candirapredmodeb is greater than INTRA _ DC, the following applies:

the variables minAB and maxAB are derived as follows:

min (candirapredModeA, candirapredModeB) [ equation 16]

MaxAB (candiraPredModeA, candiraPredModeB) [ equation 17]

-if candxtrapredmodea and candxtrapredmodeb are both greater than INTRA _ DC, candModeList [ x ] is derived as follows, where x ═ 0.. 4.

candModelist [0] ═ candIntrarPredModea [ equation 18]

candModelist [1] ═ candIntrarPredModeB [ equation 19]

candModelist [2] ═ INTRA _ DC [ equation 20]

If maxAB-minAB is in the range of 2 to 62 (inclusive), then the following applies:

candModelist [3] ((maxAB + 61)% 64) [ equation 21]

candModelist [4] ((maxAB-1)% 64) [ equation 22]

Otherwise, the following applies:

candModelist [3] ((maxAB + 60)% 64) [ equation 23]

candModelist [4] ═ 2+ ((maxAB)% 64) [ equation 24]

Else (candirapredmodea or candirapredmodeb is greater than INTRA _ DC), candModeList [ x ] is derived as follows, where x ═ 0.. 4:

candModelist [0] ═ maxAB [ equation 25]

candModelist [1] ═ INTRA _ DC [ equation 26]

candModelist [2] ═ 2+ ((maxAB + 61)% 64) [ equation 27]

candModelist [3] ((maxAB-1)% 64) [ equation 28]

candModelist [4] ((maxAB + 60)% 64) [ equation 29]

Otherwise, the following applies:

candModelist [0] ═ INTRA _ DC [ equation 30]

candModelist [1] ═ INTRA _ ANGULAR50 [ equation 31]

candModelist [2] ═ INTRA _ ANGULAR18 [ equation 32]

candModelist [3] ═ INTRA _ ANGULAR46 [ equation 33]

candModelist [4] ═ INTRA _ ANGULAR54 [ equation 34]

4. IntraPredModey [ xCb ] [ yCb ] was derived by applying the following procedure:

-if intra _ luma _ mpm _ flag [ xCb ] [ yCb ] is equal to 1, IntraPredModeY [ xCb ] [ yCb ] is set equal to candModelist [ intra _ luma _ mpm _ idx [ xCb ] [ yCb ] ].

Else, derive IntraPredModeY [ xCb ] [ yCb ] by applying the following ordered steps:

1) where candModeList [ i ] is greater than candModeList [ j ], i ═ 0..3, and for each i, j ═ (i + 1.. 4), then two values are swapped as follows:

(candModelist [ i ], candModelist [ j ]) Swap (candModelist [ i ], candModelist [ j ]) [ equation 35]

2) IntraPredModey [ xCb ] [ yCb ] was derived by the following ordered procedure:

IntraPredModey [ xCb ] [ yCb ] is set equal to intra _ luma _ mpm _ remaining [ xCb ] [ yCb ].

intrapredmodey [ xCb ] [ yCb ] increasing by 1

For i equal to 0 to 4 inclusive, increment the value of IntraPredModey [ xCb ] [ yCb ] by 1 when IntraPredModey [ xCb ] [ yCb ] is greater than or equal to candModelist [ i ].

Else (INTRA _ luma _ not _ player _ flag [ xCb ] [ yCb ] equal to 0), IntraPredModey [ xCb ] [ yCb ] is set equal to INTRA _ PLANAR.

The variable IntraPredModeY [ x ] [ y ] is set equal to IntraPredModeY [ xCb ] [ yCb ], where x ═ xcb.. xCb + cbWidth-1 and y ═ ycb.. yCb + cbHeight-1.

1.12 Multi-reference line (MRL) Intra prediction

Multi-reference line (MRL) intra prediction uses more reference lines for intra prediction. In fig. 11, an example of 4 reference lines is depicted, where samples of segments a and F are not taken from reconstructed neighboring samples, but are padded with the closest samples from segments B and E, respectively. HEVC intra prediction uses the nearest reference line (i.e., reference line 0). In MRL, additional lines (reference line 1 and reference line 3) are used.

The index (mrl _ idx) of the selected reference line is signaled and used to generate the intra predictor. For reference lines idx greater than 0, only the extra reference line mode is included in the MPM list and only the MPM index is signaled with no remaining modes. The reference line index is signaled before the intra prediction mode, in case a non-zero reference line index is signaled, the plane and DC mode are excluded from the intra prediction mode.

For the first line of blocks inside the CTU, the MRL is disabled to prevent the use of extended reference samples outside the current CTU line. Also, when the additional line is used, the PDPC is disabled.

1.13 Intra-frame Subdivision (ISP)

An intra sub-partition (ISP) tool divides a luma intra prediction block vertically or horizontally into 2 or 4 sub-partitions according to block size. For example, the minimum block size of an ISP is 4x8 (or 8x 4). If the block size is greater than 4x8 (or 8x4), the corresponding block is divided by 4 sub-partitions. Fig. 12 shows an example of two possibilities. All sub-partitions fulfill the condition of having at least 16 samples.

TABLE 2-1 entropy codec coefficient set size

Size of block	Coefficient set size
		1×N,N≥16	1×16
N×1,N≥16	16×1
		2×N,N≥8	2×8
N×2,N≥8	8×2
		All other possible M N cases	4×4

For each sub-partition, reconstructed samples are obtained by adding a residual signal to the prediction signal. Here, the residual signal is generated through processes such as entropy decoding, inverse quantization, and inverse transformation. Thus, the reconstructed sample values for each sub-partition may be used to generate a prediction for the next sub-partition, and each sub-partition is processed iteratively. In addition, the first sub-partition to be processed is a sub-partition containing the CU's upper left sample point, and then continues down (horizontal partition) or right (vertical partition). As a result, the reference samples used for generating the sub-divided prediction signals are located only on the left and upper sides of the line. All sub-partitions share the same intra mode. The following is a summary of the ISP's interactions with other codec tools.

Multiple Reference Lines (MRL): if the block has an MRL index other than 0, then the ISP codec mode will be inferred to be 0 and therefore the ISP mode information will not be sent to the decoder.

Entropy coding and decoding coefficient group size: modifying the size of the entropy codec sub-blocks has been modified such that they have 16 samples in all possible cases, as shown in table 2-1. Note that the new size only affects blocks generated by the ISP, where one of the scales is smaller than 4 samples. In all other cases, the coefficient sets remain on the 4 × 4 scale.

And (3) CBF coding and decoding: at least one of the sub-partitions is assumed to have a non-zero CBF. Thus, if n is the number of sub-segmentations, and the first sub-segmentation has produced zero CBF, the CBF of the nth sub-segmentation is inferred to be 1.

The MPM used: in blocks of ISP mode codec, the MPM flag will be inferred to be 1, the MPM list is modified to exclude DC mode and prioritize horizontal intra mode for ISP horizontal, vertical intra mode for vertical partitioning.

Transforming the size constraint: all ISP transforms of length greater than 16 points use DCT-II.

PDPC: when the CU uses the ISP codec mode, the PDPC filter will not be applied to the resulting sub-partitions.

MTS flag: if the CU uses ISP codec mode, the MTS CU flag will be set to 0 and will not be sent to the decoder. Thus, the decoder will not perform RD tests for each resulting sub-partition, for different available transforms. The selection of the transformation of the ISP mode will instead be fixed and selected according to the utilized intra mode, processing order and block size. Thus, no signaling is required. For example, let t_HAnd t_VHorizontal and vertical transforms selected for the w × h sub-partitions, respectively, where w is the width and h is the height. Then, the transformation is selected according to the following rules:

if w-1 or h-1, there is no horizontal or vertical transform, respectively.

DCT-II if w is 2 or w >32

DCT-II if h 2 or h >32

Otherwise, the transform is selected as in Table 2-2.

TABLE 2-2-transform selection depends on Intra mode

Disadvantages of the existing embodiments

The MPM list is designed for the natural sequence. On screen content codec, the vertical mode and the horizontal mode are two main modes, so different MPM list constructions may be required in screen content codec. Also, on mixed content, more than one MPM list may be needed, which may be able to switch based on whether the current block is a screen content block.

Exemplary method for maximum possible mode list construction for Screen content codec

The following should be considered as an example to explain the general concept. These examples are provided merely to facilitate an understanding of the disclosed technology and should not be construed in a narrow manner. Furthermore, the embodiments may be combined in any manner.

1. Two or more MPM list construction methods may be used for conventional intra mode coding.

A. In one example, the first MPM list may be a current list and the second MPM list may be derived from the first list.

I. In one example, to generate the second MPM list, the first MPM list may be generated first, after which reordering may be applied to those modes in the first MPM list.

a) In one example, the planes and/or DC patterns may be placed in different locations in the list.

in one example, to generate the second MPM list, the first MPM list may be generated first, after which one or more modes in the list may be replaced.

a) In one example, the planar and/or DC modes may be replaced with angular modes (e.g., horizontal or vertical).

b) In one example, some modes other than the vertical mode and/or the horizontal mode may be replaced by the vertical mode and/or the horizontal mode.

b. In one example, the second MPM list may follow the logic of the first MPM list, but replace some non-angle modes with angle modes.

c. In one example, whether more than one MPM list is allowed may be signaled in the set of SPS/VPS/PPS/picture header/slice group header/maximum codec unit (LCU)/Codec Unit (CU)/LCU row/LCU level.

d. In one example, the number of allowed MPM lists may be predefined or signaled to the decoder.

i. In one example, the number of MPM lists is a fixed value, e.g., 2.

in one example, the number of MPM lists may be signaled to the decoder in a group of SPS/VPS/PPS/picture header/slice group header/maximum codec unit (LCU)/Codec Unit (CU)/LCU row/LCU level.

e. In one example, one MPM list is used for natural content codec and another for screen content codec.

f. In one example, which MPM list to use may be based on

i. Video content (e.g., screen content or natural content)

A message signaled in a group of SPS/VPS/PPS/picture header/slice group header/maximum codec unit (LCU)/Codec Unit (CU)/LCU row/LCU level or other video unit.

a) In one example, a flag may be signaled in a slice header to indicate a list to select for coding a conventional intra prediction mode.

b) In one example, the MPM list may depend on a message informing of video content information, such as screen content or camera capture content.

Location of Prediction Unit (PU)

Block size of a current block and/or its neighboring blocks

v. block shape of current block and/or its neighboring blocks

Prediction modes (intra/inter) of neighboring blocks of the current block

Intra prediction modes of neighboring blocks of a current block

Motion vector of neighboring block of current block

Indication of QR-BDPCM mode of neighboring blocks to a current block

Indication of the QR-BDPCM mode of the transform type of the current block (e.g., indicated by tu _ mts _ idx)

Current quantization parameter of current block and/or its neighboring blocks

Indication of color format (e.g. 4:2:0, 4:4:4)

Coding and decoding tree structure

Slice/slice group type and/or picture type

2. The MPM list can be switched at slice/picture/CU/LCU group/block level. Whether to switch the MPM list may be based on

a. Video content (e.g., screen content or natural content)

b. Message signaled in a group of SPS/VPS/PPS/Picture header/slice group header/Largest Coding Unit (LCU)/Coding Unit (CU)/LCU line/LCU levels

c. Location of Prediction Unit (PU)

d. Block size of current block and/or its adjacent block

e. Block shape of current block and/or its adjacent block

f. Prediction modes of the neighbouring blocks of the current block (intra/inter)

g. Intra prediction modes of neighboring blocks of a current block

h. Motion vector of neighboring block of current block

i. Indication of QR-BDPCM mode of neighboring blocks of a current block

j. Indication of the QR-BDPCM mode of the transform type of the current block (e.g., indicated by tu _ mts _ idx)

k. Current quantization parameter of current block and/or its adjacent block

Indication of color format (e.g. 4:2:0, 4:4:4)

m. coding and decoding tree structure

Slice/slice group type and/or picture type

Color component (e.g. luminance or chrominance)

3. In the construction of the MPM list, a specific angle mode (e.g., a vertical mode and/or a horizontal mode) may always be in the MPM list.

a. In one example, the particular angle mode may be inserted in a given position in the MPM list, e.g., the first.

b. In one example, the vertical mode and the horizontal mode may always be in the first two positions in the MPM list in order.

c. In one example, the horizontal mode and the vertical mode may always be in the first two positions in the MPM list in order.

d. In one example, the MPM list may always be { INTRA _ anflag 50, INTRA _ anflag 18, DC, plane, INTRA _ anflag 34, INTRA _ anflag 66}

e. In one example, the MPM list may always be { INTRA _ anflag 18, INTRA _ anflag 50, plane, DC, INTRA _ anflag 34, INTRA _ anflag 66}

f. In one example, the MPM list may always be { INTRA _ anflag 50, INTRA _ anflag 18, plane, DC, INTRA _ anflag 34, INTRA _ anflag 66}

g. In one example, the MPM list may always be { plane, DC, INTRA _ anflag 18, INTRA _ anflag 50, INTRA _ anflag 34, INTRA _ anflag 66 }.

h. In one example, if neither the left-side neighboring INTRA mode nor the above-side neighboring INTRA mode is available, the MPM list may be { INTRA _ anflag 50, INTRA _ anflag 18, DC, plane, INTRA _ anflag 34, INTRA _ anflag 66}

i. In one example, if neither the left-side neighboring INTRA mode nor the above-side neighboring INTRA mode is available, the MPM list may be { INTRA _ anflag 50, INTRA _ anflag 18, DC, plane, INTRA _ anflag 34, INTRA _ anflag 2}

j. In one example, if the left-side neighboring INTRA mode and the above-side neighboring INTRA mode are the same, and this mode is a non-angle INTRA mode, the MPM list may be { DC, plane, INTRA _ ANGULAR50, INTRA _ ANGULAR18, INTRA _ ANGULAR34, INTRA _ ANGULAR66}

k. In one example, if the left-side neighboring intra-mode and the above-side neighboring intra-mode are the same, and such a mode a is an angle intra-mode, the MPM list may be constructed based on the direction of a.

i. In one example, if a > -2 and a < -18, the MPM list may always be { INTRA _ anpilot 18, INTRA _ anpilot 2, INTRA _ anpilot 50, DC, plane, INTRA _ anpilot 66}

in one example, if a > -18 and a < ═ 34, the MPM list may always be { INTRA _ angul 18, INTRA _ angul 34, INTRA _ angul 50, DC, plane, INTRA _ angul 2}

in one example, if a > -34 and a < ═ 50, the MPM list may always be { INTRA _ anflag 50, INTRA _ anflag 34, INTRA _ anflag 18, DC, plane, INTRA _ anflag 66}

in one example, if a > -50 and a < -66, the MPM list may always be { INTRA _ anflag 50, INTRA _ anflag 66, INTRA _ anflag 18, DC, plane, INTRA _ anflag 34}

In one example, the left-side neighboring intra mode and the above-side neighboring intra mode are different modes, one of which is a non-angle mode.

i. In one example, further, the MPM list may be { DC, plane, INTRA _ anflag 50, INTRA _ anflag 18, INTRA _ anflag 34, INTRA _ anflag 66}

m. in one example, the left-side neighboring intra mode and the above-side neighboring intra mode are different modes, both of which are angular modes.

I. In one example, the left Max represents the maximum value between the two intra modes.

in one example, if Max > < 2 and Max < 18, the MPM list may be { INTRA _ anpilot 18, INTRA _ anpilot 2, INTRA _ anpilot 50, DC, plane, INTRA _ anpilot 66}

in one example, if Max > -18 and Max < ═ 34, the MPM list may be { INTRA _ anpilot 18, INTRA _ anpilot 34, INTRA _ anpilot 50, DC, plane, INTRA _ anpilot 2}

in one example, if Max > -34 and Max < ═ 50, the MPM list may be { INTRA _ anflag 50, INTRA _ anflag 34, INTRA _ anflag 18, DC, plane, INTRA _ anflag 66}

In one example, if Max > -50 and Max < ═ 66, the MPM list may be { INTRA _ anpilot 50, INTRA _ anpilot 66, INTRA _ anpilot 18, DC, plane, INTRA _ anpilot 34}

N. in one example, at the top of the current design on the MPM list build, the vertical mode and/or horizontal mode may replace one existing mode in the MPM list.

i. In one example, in the MPM list, a mode having a direction closest to the vertical mode may be replaced with the vertical mode.

in one example, in the MPM list, the mode having the direction closest to the horizontal mode may be replaced with the horizontal mode.

In one example, the location of the vertical mode and/or the horizontal mode in the MPM list may be based on

i. Message signaled in a group of SPS/VPS/PPS/Picture header/slice group header/Largest Coding Unit (LCU)/Coding Unit (CU)/LCU line/LCU levels

Location of Prediction Unit (PU).

Block size of current block and/or its neighboring blocks

Block shape of current block and/or its neighboring blocks

v. prediction modes (intra/inter) of neighboring blocks of the current block

Intra prediction mode of neighboring blocks of the current block

Motion vector of neighboring block of current block

Indication of QR-BDPCM mode of neighboring blocks to a current block

indication of the QR-BDPCM mode of the transform type of the current block (e.g., indicated by tu _ mts _ idx)

x. current quantization parameter of current block and/or its neighboring blocks

Indication of color format (e.g., 4:2:0, 4:4:4)

xi. coding and decoding tree structure

Slice/slice group type and/or picture type

The intra mode of RDPCM may be set to a mode other than the first mode in the MPM list. In this case, the prediction signal may be generated from one intra mode, and the stored intra modes may correspond to each other. The stored intra modes may be used to predict intra modes of successive coded blocks. In some embodiments, the stored intra mode may be different from the intra mode used in the intra prediction process of the current block.

a. In one example, the intra-mode of the RDPCM codec block is set as the second MPM candidate (e.g., corresponding to intra _ luma _ MPM _ idx ═ 0)

b. In one example, the intra-mode of the RDPCM codec block is set to any MPM candidate on the MPM list except the first one (e.g., corresponding to intra _ luma _ MPM _ idx ═ 0)

C. In one example, whether and/or how to apply the above methods may be based on

i. Message signaled in a group of SPS/VPS/PPS/Picture header/slice group header/Largest Coding Unit (LCU)/Coding Unit (CU)/LCU line/LCU levels

Location of Prediction Unit (PU).

Block size of current block and/or its neighboring blocks

Block shape of current block and/or its neighboring blocks

v. prediction modes (intra/inter) of neighboring blocks of the current block

Intra prediction mode of neighboring blocks of the current block

Motion vector of neighboring block of current block

Indication of QR-BDPCM mode of neighboring blocks to a current block

indication of the QR-BDPCM mode of the transform type of the current block (e.g., indicated by tu _ mts _ idx)

x. current quantization parameter of current block and/or its neighboring blocks

Indication of color format (e.g., 4:2:0, 4:4:4)

xi. coding and decoding tree structure

Slice/slice group type and/or picture type

5. The position of the planar mode may not always be the first in the MPM list.

a. In one example, the planar mode may be placed at the last position in the MPM list.

b. In one example, the plane mode may be placed anywhere in the MPM list except the first.

c. In one example, the plane mode may not be allowed to be placed in the first position in the MPM list.

d. In one example, whether and/or how to apply the above methods may be based on

i. Message signaled in the group of SPS/VPS/PPS/picture header/slice group header/Largest Codec Unit (LCU)/Codec Unit (CU)/LCU row/LCU level ii.

Block size of current block and/or its neighboring blocks

Block shape of current block and/or its neighboring blocks

v. prediction modes (intra/inter) of neighboring blocks of the current block

Intra prediction mode of neighboring blocks of the current block

Motion vector of neighboring block of current block

Indication of QR-BDPCM mode of neighboring blocks to a current block

indication of the QR-BDPCM mode of the transform type of the current block (e.g., indicated by tu _ mts _ idx)

x. current quantization parameter of current block and/or its neighboring blocks

Indication of color format (e.g., 4:2:0, 4:4:4)

xi. coding and decoding tree structure

Slice/slice group type and/or picture type

Binarization of the MPM index (e.g., intra _ luma _ MPM _ idx) may not use truncated unary binarization.

a. In one example, the codeword length of the MPM index (e.g., intra _ luma _ MPM _ idx) may be fixed.

Signaling of mpm index (e.g., intra _ luma _ mpm _ idx) may use Exp-Golomb binarization of order K.

i. In one example, K is an integer (e.g., 2) and may be based on

a) Location of a Prediction Unit (PU).

b) Block size of current block and/or its adjacent block

c) Block shape of current block and/or its adjacent block

d) Prediction modes of the neighbouring blocks of the current block (intra/inter)

e) Intra prediction modes of neighboring blocks of a current block

f) Motion vector of neighboring block of current block

g) Indication of QR-BDPCM mode of neighboring blocks of a current block

h) Indication of the QR-BDPCM mode of the transform type of the current block (e.g., indicated by tu _ mts _ idx)

i) Current quantization parameter of current block and/or its adjacent block

j) Indication of color format (e.g., 4:2:0, 4:4:4)

k) Coding and decoding tree structure

l) slice/slice group type and/or picture type

7. To encode an indication of an MPM candidate in the list, a first flag may be encoded followed by a remaining index. The first flag is used to indicate whether the selected intra prediction mode is the same as or different from the first MPM candidate in the list.

The semantic of intra _ luma _ not _ planar _ flag can be interpreted as whether the intra prediction mode is not the first MPM candidate in the MPM list, but not as whether it is not the planar mode.

b. Optionally, the first flag may be conditionally coded.

i. In one example, if the block represents screen content, the first flag may not be coded.

c. Optionally, in addition, if the selected mode is not the first MPM candidate in the list, signaling of the remaining MPM indices (e.g., intra _ luma _ MPM _ idx) may be further signaled.

8. The proposed vertical and horizontal intra prediction in the MPM list construction process may correspond to directly copying reference samples in the intra prediction process.

a. In one example, the vertical intra prediction may be generated by directly copying reference samples in the same column.

b. In one example, the horizontal intra prediction may be generated by directly copying reference samples in the same row.

c. In one example, the indication of whether to apply the above prediction may be based on

i. Video content (e.g., screen content or natural content)

Message signaled in the group of SPS/VPS/PPS/Picture header/slice group header/Largest Codec Unit (LCU)/Codec Unit (CU)/LCU line/LCU level

Location of a Prediction Unit (PU).

Block size of a current block and/or its neighboring blocks

v. block shape of current block and/or its neighboring blocks

Prediction modes (intra/inter) of neighboring blocks of the current block

Intra prediction modes of neighboring blocks of a current block

Motion vector of neighboring block of current block

indication of QR-BDPCM mode of neighboring blocks of a current block

Indication of the QR-BDPCM mode of the transform type of the current block (e.g., indicated by tu _ mts _ idx)

Current quantization parameter of current block and/or its neighboring blocks

Indication of color format (e.g. 4:2:0, 4:4:4)

xiii. independent/dual codec tree structure

Slice/slice group type and/or picture type

xv. color component (e.g., may apply only to chrominance or luminance components)

9. The above method can also be applied to other intra prediction methods, such as affine intra prediction method (also called matrix-based intra prediction method), intra sub-partition prediction (ISP), multi-reference line (MRL), and the like.

Fig. 13a and 13b show a flow chart of an exemplary method of video processing. Referring to FIG. 13, method 1300 includes, at step 1302, constructing a first mode list of intra codec modes based on intra modes of neighboring blocks to the current video block. The method 1300 further includes, at step 1304, constructing a second mode list of intra-coding modes based on the first mode list. Method 1300 also includes, at step 1306, performing intra mode coding for the current video block using at least one of the first mode list or the second mode list.

In some embodiments, wherein the first mode list and the second mode list are MPM (most probable mode) lists. In some embodiments, constructing the second pattern list includes reordering patterns of the first pattern list. In some embodiments, constructing the second pattern list includes replacing one or more patterns of the first pattern list. In some embodiments, the second mode list has the same logic as the first mode list. In some embodiments, the method further comprises: an indication of whether both the first mode list and the second mode list are allowed is included in a bitstream representation of the current video block. In some implementations, whether to use the first mode list or the second mode list is determined based on a message in an SPS, VPS, PPS, picture header, slice group header, Largest Coding Unit (LCU), Coding Unit (CU), LCU line, LCU group, or other video unit. In some embodiments, performing intra-mode codec comprises switching from the first mode list to the second mode list or switching from the second mode list to the first mode list. In some implementations, the switching occurs at a slice, picture, CU, LCU, group of LCUs, or block level. In some embodiments, the handover is based on at least one of: a. video content, b. a message signaled in an SPS, VPS, PPS, picture header, slice group header, maximum codec unit (LCU), Codec Unit (CU), LCU row, LCU group, c. location of a Prediction Unit (PU), d. block size of the current video block and/or neighboring blocks, e. block shape of the current video block and/or neighboring blocks, f. prediction mode of neighboring blocks of the current video block, g. intra prediction mode of neighboring blocks of the current video block, h. motion vector of neighboring blocks of the current video block, i. indication of QR-BDPCM mode of neighboring blocks of the current video block, j. indication of QR-BDPCM mode of transform type of the current video block, k. current quantization parameter of the current video block and/or neighboring blocks, l. indication of color format, m. decoded coding tree structure, n. slice, slice group header and/or picture type, or o. color components. In some embodiments, each of the first and second pattern lists includes a pattern index, and the binarization of the pattern index is performed without using truncated unary binarization. In some embodiments, at least one of the first mode list and the second mode list is applied to perform an intra prediction method, an affine intra prediction method, an intra sub partition prediction (ISP), or a Multiple Reference Line (MRL).

Referring to fig. 13b, method 1320 includes constructing a mode list of the intra coding mode to include the candidate mode based on intra modes of neighboring blocks to the current video block, wherein constructing the mode list includes locating the candidate mode at a position in the mode list based on a type of the candidate mode. In some embodiments, locating the candidate mode comprises locating a particular angular mode at a given position in the mode list. In some embodiments, locating the candidate mode includes locating the vertical mode and the horizontal mode, or the horizontal mode and the vertical mode, in order at the first two positions in the mode list. In some embodiments, locating the candidate mode comprises locating an intra mode of RDPCM (residual differential pulse code modulation) to a position in the mode list other than the first position. In some embodiments, the prediction signal is generated from an intra mode. In some embodiments, another intra mode is used to predict the intra mode of a subsequent coded block. In some embodiments, locating the candidate mode comprises locating the planar mode to a position in the mode list other than the first position. In some embodiments, the mode list includes a mode index, and the binarization of the mode index is performed without using truncated unary binarization. In some embodiments, the method further comprises indicating whether the selected intra prediction mode is the same as or different from the first candidate mode in the mode list. In some embodiments, the method further comprises generating intra-prediction samples based on the pattern list by copying reference samples in the same column or the same row. In some embodiments, the mode list is applied to perform an intra prediction method, an affine intra prediction method, an intra sub partition prediction (ISP), or a Multiple Reference Line (MRL).

Fig. 14 is a block diagram of the video processing apparatus 1200. Device 1200 may be used to implement one or more methods described herein. The apparatus 1200 may be included in a smartphone, tablet, computer, internet of things (IoT) receiver, and the like. The apparatus 1200 may include one or more processors 1202, one or more memories 1204, and video processing hardware 1206. The processor 1202 may be configured to implement one or more of the methods described herein (including, but not limited to, methods 1100 and 1150). The memory(s) 1204 may be used for storing data and code for implementing the methods and techniques described herein. Video processing hardware 1206 may be used to implement some of the techniques described in this document in hardware circuits.

In some embodiments, the video codec method may be implemented using an apparatus implemented on a hardware platform as described in connection with fig. 14.

Fig. 15 is a flow diagram of an example method 1500 of video processing. The method 1500 includes, at 1502, constructing two or more Maximum Possible Mode (MPM) lists for an intra-coding mode of a current block for a transition between the current block of video and a bitstream representation of the current block, wherein the two or more MPM lists include at least a first MPM list constructed using a first construction method and a second MPM list constructed using a second construction method different from the first construction method; and at 1504, performing a conversion using one MPM list selected from the constructed two or more MPM lists.

In some examples, the first MPM list is a current MPM list for the current block, and the second MPM list is derived from the first MPM list.

In some examples, a first MPM list is first generated, and a second MPM list is generated by reordering those modes in the first MPM list.

In some examples, the planar mode and/or the DC mode are placed in different positions in the first MPM list.

In some examples, a first MPM list is first generated, and a second MPM list is generated by substituting one or more modes in the first MPM list.

In some examples, an angular mode is utilized instead of a planar mode and/or a DC mode.

In some examples, the angular mode includes at least one of a horizontal mode and a vertical mode.

In some examples, some modes other than the horizontal mode and the vertical mode in the first MPM list are replaced with the horizontal mode and/or the vertical mode.

In one example, the second MPM list adheres to the logic of the first MPM list, with some non-angle modes being replaced with angle modes.

In some examples, whether two or more MPM lists are allowed is signaled in at least one of a Video Parameter Set (VPS), a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), a picture header, a slice group header, a Largest Coding Unit (LCU), a Coding Unit (CU), a row of LCUs, and a group level of LCUs.

In some examples, the number of allowed MPM lists is predefined or signaled to the decoder.

In some examples, the number of MPM lists is a fixed value.

In some embodiments, the fixed value is 2.

In some examples, the number of MPM lists is signaled in at least one of a VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU row, and LCU group level.

In some examples, one of the first MPM list and the second MPM list is used for natural content codec and the other is used for screen content codec.

In some examples, which MPM list to select depends on one or more of:

i. video content including at least one of screen content and natural content;

messages signaled in VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU row, LCU group, or other video unit;

location of a Prediction Unit (PU);

a block size of the current block and/or its neighboring blocks;

v. block shape of the current block and/or its neighboring blocks;

prediction mode, including intra-mode and inter-mode of neighboring blocks of the current block;

intra prediction modes of neighboring blocks of the current block;

motion vectors of neighboring blocks of the current block;

an indication of quantized residual block differential pulse code modulation (QR-BDPCM) mode of neighboring blocks to the current block;

an indication of a QR-BDPCM mode of a transform type for a current block;

a current quantization parameter of the current block and/or its neighboring blocks;

an indication of a color format of the current block;

coding and decoding tree structure;

slice, slice group type, and/or picture type.

In some examples, a flag is signaled in a slice header to indicate a list to select for coding a conventional intra prediction mode.

In some examples, the MPM list depends on a message indicating video content information, the video content information including at least one of screen content or camera capture content.

In some examples, the color format of the current block is 4:2:0 or 4:4: 4.

In some examples, switching of the MPM list at slice, picture, CU, LCU group, or block level may be allowed.

In some examples, whether to switch the MPM list depends on at least one of:

video content comprising at least one of screen content and natural content;

a message signaled in VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU line, LCU group, or other video unit;

location of Prediction Unit (PU);

xix block size of the current block and/or its neighboring blocks;

xx. block shape of the current block and/or its neighboring blocks;

prediction modes, including intra-modes and inter-modes of neighboring blocks of the current block;

xxii. intra prediction modes of neighboring blocks of the current block;

xxiii. motion vectors of neighboring blocks of the current block;

xxiv. an indication of a quantized residual block differential pulse code modulation (QR-BDPCM) mode of a neighboring block to the current block;

xxv. an indication of a QR-BDPCM mode of a transform type for a current block;

xxvi. the current quantization parameter of the current block and/or its neighboring blocks;

xxvii. an indication of the color format of the current block;

xxviii. codec tree structure;

slice, slice group type and/or picture type;

a color component comprising at least one of a luma component and a chroma component;

in some examples, the particular angle mode is always in the MPM list during construction of the MPM list.

In some examples, the particular angular mode is a vertical mode and/or a horizontal mode.

In some examples, the particular angle mode is inserted at a given position in the MPM list.

In some examples, the particular angle mode is inserted at a first position in the MPM list.

In some examples, the vertical mode and the horizontal mode are always the first two positions in the MPM list in order.

In some examples, the horizontal mode and the vertical mode are always the first two positions in the MPM list in order.

In some examples, the MPM list is always { INTRA _ anflag 50, INTRA _ anflag 18, DC, plane, INTRA _ anflag 34, INTRA _ anflag 66 }.

In some examples, the MPM list is always { INTRA _ anflag 18, INTRA _ anflag 50, plane, DC, INTRA _ anflag 34, INTRA _ anflag 66 }.

In some examples, the MPM list is always { INTRA _ anflag 50, INTRA _ anflag 18, plane, DC, INTRA _ anflag 34, INTRA _ anflag 66 }.

In some examples, the MPM list is always { plane, DC, INTRA _ anflag 18, INTRA _ anflag 50, INTRA _ anflag 34, INTRA _ anflag 66 }.

In some examples, if neither the left-neighboring INTRA-mode nor the above-neighboring INTRA-mode of the current block is available, the MPM list is { INTRA _ anflag 50, INTRA _ anflag 18, DC, plane, INTRA _ anflag 34, INTRA _ anflag 66 }.

In some examples, if neither the left-neighboring INTRA-mode nor the above-neighboring INTRA-mode of the current block is available, the MPM list is { INTRA _ anflag 50, INTRA _ anflag 18, DC, plane, INTRA _ anflag 34, INTRA _ anflag 2 }.

In some examples, if the left-neighboring INTRA-mode and the above-neighboring INTRA-mode of the current block are the same, and such a mode is a non-angle INTRA-mode, the MPM list is { DC, plane, INTRA _ ANGULAR50, INTRA _ ANGULAR18, INTRA _ ANGULAR34, INTRA _ ANGULAR66 }.

In some examples, if the left-side neighboring intra-mode and the above-side neighboring intra-mode of the current block are the same, and such mode a is an angular intra-mode, the MPM list is constructed based on the direction of such mode a.

In some examples, if a > -2 and a < ═ 18, the MPM list is always { INTRA _ ANGULAR18, INTRA _ ANGULAR2, INTRA _ ANGULAR50, DC, plane, INTRA _ ANGULAR66 }.

In some examples, if a > -18 and a < ═ 34, the MPM list is always { INTRA _ ANGULAR18, INTRA _ ANGULAR34, INTRA _ ANGULAR50, DC, plane, INTRA _ ANGULAR2 }.

In some examples, if a > -34 and a < ═ 50, the MPM list is always { INTRA _ ANGULAR50, INTRA _ ANGULAR34, INTRA _ ANGULAR18, DC, plane, INTRA _ ANGULAR66 }.

In some examples, if a > -50 and a < -66, the MPM list is always { INTRA _ ANGULAR50, INTRA _ ANGULAR66, INTRA _ ANGULAR18, DC, plane, INTRA _ ANGULAR34 }.

In some examples, if the left-neighboring INTRA-mode and the above-neighboring INTRA-mode of the current block are different modes, and one of them is a non-angle mode, the MPM list is { DC, plane, INTRA _ ANGULAR50, INTRA _ ANGULAR18, INTRA _ ANGULAR34, INTRA _ ANGULAR66 }.

In some examples, the left-side neighboring intra mode and the above-side neighboring intra mode of the current block are different modes, and they are both angle modes, then Max represents the maximum value between the two intra modes.

In some examples, if Max > -2 and Max < ═ 18, then the MPM list is { INTRA _ ANGULAR18, INTRA _ ANGULAR2, INTRA _ ANGULAR50, DC, plane, INTRA _ ANGULAR66 }.

In some examples, if Max > -18 and Max < ═ 34, the MPM list may be { INTRA _ ANGULAR18, INTRA _ ANGULAR34, INTRA _ ANGULAR50, DC, plane, INTRA _ ANGULAR2 }.

In some examples, if Max > -34 and Max < ═ 50, the MPM list may be { INTRA _ ANGULAR50, INTRA _ ANGULAR34, INTRA _ ANGULAR18, DC, plane, INTRA _ ANGULAR66 }.

In some examples, if Max > -50 and Max < ═ 66, the MPM list may be { INTRA _ ANGULAR50, INTRA _ ANGULAR66, INTRA _ ANGULAR18, DC, plane, INTRA _ ANGULAR34 }.

In some examples, one existing mode in the MPM list is replaced by a vertical mode and/or a horizontal mode.

In some examples, one existing mode in the MPM list that has the closest direction to the vertical mode is replaced by the vertical mode.

In some examples, one existing mode in the MPM list having the closest direction to the horizontal mode is replaced by the horizontal mode.

In some examples, the position of the vertical mode and/or the horizontal mode in the MPM list depends on at least one of:

messages signaled in VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU row, LCU group, or other video unit;

xv. predicting the location of a unit (PU);

xvi. the block size of the current block and/or its neighboring blocks;

xvii. block shape of the current block and/or its neighboring blocks;

a prediction mode comprising intra-modes and inter-modes of neighboring blocks of the current block;

xix. intra prediction mode of a neighboring block of the current block;

xx. motion vectors of neighboring blocks of the current block;

xxi. an indication of a quantized residual block differential pulse code modulation (QR-BDPCM) mode of a neighboring block to the current block;

xxii. indication of the QR-BDPCM mode of the transform type of the current block;

xxiii. current quantization parameter of the current block and/or its neighboring blocks;

xxiv. an indication of a color format of the current block;

xxv. a coding and decoding tree structure;

slice, slice group type and/or picture type.

In some examples, when the current block is a Residual Differential Pulse Code Modulation (RDPCM) codec block, an intra-mode of the RDPCM codec block is set to a mode in the MPM list other than the first mode.

In some examples, the intra-mode of the RDPCM codec block is set to the second MPM mode in the MPM list with an indication of intra _ luma _ MPM _ idx ═ 0.

In some examples, the intra-mode of the RDPCM codec block is set to any mode in the MPM list other than the first mode with an indication of intra _ luma _ MPM _ idx ═ 0.

In some examples, whether and/or how to set the intra-mode of the RDPCM codec block depends on at least one of:

messages signaled in VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU row, LCU group, or other video unit;

xv. predicting the location of a unit (PU);

xvi. the block size of the current block and/or its neighboring blocks;

xvii. block shape of the current block and/or its neighboring blocks;

a prediction mode comprising intra-modes and inter-modes of neighboring blocks of the current block;

xix. intra prediction mode of a neighboring block of the current block;

xx. motion vectors of neighboring blocks of the current block;

xxi. an indication of a quantized residual block differential pulse code modulation (QR-BDPCM) mode of a neighboring block to the current block;

xxii. indication of the QR-BDPCM mode of the transform type of the current block;

xxiii. current quantization parameter of the current block and/or its neighboring blocks;

xxiv. an indication of a color format of the current block;

xxv. a coding and decoding tree structure;

slice, slice group type and/or picture type.

In some examples, the position of the planar mode is not always the first position in the MPM list.

In some examples, the planar mode is put to the last position in the MPM list.

In some examples, the plane mode is placed anywhere in the MPM list except the first position.

In some examples, the plane mode is not allowed to be placed in the first position in the MPM list.

In some examples, whether and/or how to set the location of the planar mode depends on at least one of:

messages signaled in VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU row, LCU group, or other video unit;

xv. predicting the location of a unit (PU);

xvi. the block size of the current block and/or its neighboring blocks;

xvii. block shape of the current block and/or its neighboring blocks;

a prediction mode comprising intra-modes and inter-modes of neighboring blocks of the current block;

xix. intra prediction mode of a neighboring block of the current block;

xx. motion vectors of neighboring blocks of the current block;

xxi. an indication of a quantized residual block differential pulse code modulation (QR-BDPCM) mode of a neighboring block to the current block;

xxii. indication of the QR-BDPCM mode of the transform type of the current block;

xxiii. current quantization parameter of the current block and/or its neighboring blocks;

xxiv. an indication of a color format of the current block;

xxv. a coding and decoding tree structure;

slice, slice group type and/or picture type.

In some examples, the binarization of the MPM indices in the MPM list does not use truncated unary binarization.

In some examples, the MPM index is intra _ luma _ MPM _ idx.

In some examples, the codeword length of the MPM index is fixed.

In some examples, the signaling of the MPM index uses Exp-Golomb binarization with order K, K being an integer.

In some examples, K is 2.

In some examples, K depends on at least one of:

position of a Prediction Unit (PU);

xiv. the block size of the current block and/or its neighboring blocks;

xv. block shape of the current block and/or its neighboring blocks;

prediction modes, including intra-modes and inter-modes of neighboring blocks of the current block;

xvii. intra prediction modes of neighboring blocks of the current block;

xviii. motion vectors of neighboring blocks of the current block;

an indication of quantized residual block differential pulse code modulation (QR-BDPCM) mode of neighboring blocks to the current block;

xx. an indication of the QR-BDPCM mode of the transform type of the current block;

xxi. the current quantization parameter of the current block and/or its neighboring blocks;

xxii. an indication of a color format of the current block;

xxiii. codec tree structure;

band, slice group type, and/or picture type.

In some examples, the indication of the MPM candidates in the MPM list is coded, wherein a first flag is coded followed by the remaining MPM indices of the MPM candidates, the first flag indicating whether the selected intra prediction mode is the same as or different from the first MPM candidate in the MPM list.

In some examples, the first flag is an intra _ luma _ not _ plane _ flag, where the semantics of the intra _ luma _ not _ plane _ flag are interpreted as whether the selected intra prediction mode is not the first MPM candidate in the MPM list, but not as whether it is not a planar mode.

In some examples, the first flag is conditionally coded.

In some examples, the first flag is not coded if the current block has screen content.

In some examples, if the selected intra prediction mode is not the first MPM candidate in the MPM list, the remaining MPM indices are further signaled.

In some examples, the MPM list construction process includes vertical intra prediction and/or horizontal intra prediction, which correspond to directly copying reference samples in the intra prediction process.

In some examples, the vertical intra prediction is generated by directly copying reference samples in the same column.

In some examples, the horizontal intra prediction is generated by directly copying reference samples in the same row.

In some examples, applying vertical intra prediction and/or horizontal intra prediction depends on at least one of:

video content comprising at least one of screen content and natural content;

a message signaled in VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU line, LCU group, or other video unit;

location of Prediction Unit (PU);

xix block size of the current block and/or its neighboring blocks;

xx. block shape of the current block and/or its neighboring blocks;

prediction modes, including intra-modes and inter-modes of neighboring blocks of the current block;

xxii. intra prediction modes of neighboring blocks of the current block;

xxiii. motion vectors of neighboring blocks of the current block;

xxiv. an indication of a quantized residual block differential pulse code modulation (QR-BDPCM) mode of a neighboring block to the current block;

xxv. an indication of a QR-BDPCM mode of a transform type for a current block;

xxvi. the current quantization parameter of the current block and/or its neighboring blocks;

xxvii. an indication of the color format of the current block;

xxviii. codec tree structure;

slice, slice group type and/or picture type;

a color component comprising at least one of a luma component and a chroma component;

in some examples, the first construction method and/or the second construction method further includes at least one of an affine intra prediction method, a matrix-based intra prediction method, an intra sub-partition prediction (ISP), and a multi-reference line (MRL).

In some examples, the color format of the current block is 4:2:0 or 4:4: 4.

In some examples, the conversion generates a current block of video from the bitstream representation.

In some examples, the transform generates a bitstream representation from a current block of video.

From the foregoing, it will be appreciated that specific embodiments of the technology disclosed herein have been described herein for purposes of illustration, but that various modifications may be made without deviating from the scope of the disclosed technology. Accordingly, the technology disclosed herein is not limited, except as by the appended claims.

Implementations of the subject matter and the functional operations described in this patent document can be implemented in various systems, digital electronic circuitry, or computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible and non-transitory computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine readable storage device, a machine readable storage substrate, a memory device, a composition of matter effecting a machine readable propagated signal, or a combination of one or more of them. The terms "data processing unit" or "data processing apparatus" encompass all apparatuses, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

A computer program (also known as a program, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer does not necessarily have such a device. Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

It is intended that the specification, together with the drawings, be considered exemplary only, with examples being shown therein. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Further, the use of "or" is intended to include "and/or" unless the context clearly dictates otherwise.

While this patent document contains many specifics, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features specific to particular embodiments of particular inventions. Certain features that are described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although certain features may be described above as acting in certain combinations and even initially claimed as such, one or more features of a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

Also, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Moreover, the separation of various system components in the embodiments described herein should not be understood as requiring such separation in all embodiments.

Only a few implementations and examples have been described and other implementations, enhancements and variations can be made based on what is described and illustrated in this patent document.

Claims (79)

1. A method of processing video, comprising:

constructing, for a transition between a current block of video and a bitstream representation of the current block, two or more Maximum Possible Modes (MPMs) lists for an intra-coding mode of the current block, wherein the two or more MPM lists include at least a first MPM list constructed using a first construction method and a second MPM list constructed using a second construction method different from the first construction method; and

the conversion is performed using one MPM list selected from the constructed two or more MPM lists.

2. The method of claim 1, wherein the first MPM list is a current MPM list for the current block, the second MPM list being derived from the first MPM list.

3. The method of claim 2, wherein the first MPM list is generated first, and the second MPM list is generated by reordering those modes in the first MPM list.

4. The method of claim 3, wherein planar mode and/or DC mode are put in different positions in the first MPM list.

5. The method of claim 2, wherein the first MPM list is first generated, and the second MPM list is generated by substituting one or more modes in the first MPM list.

6. The method of claim 5, wherein an angular mode is used instead of a planar mode and/or a DC mode.

7. The method of claim 6, wherein the angular mode comprises at least one of a horizontal mode and a vertical mode.

8. The method of claim 5, wherein some modes other than the horizontal mode and the vertical mode in the first MPM list are replaced with horizontal modes and/or vertical modes.

9. The method of claim 1, wherein the second MPM list adheres to the logic of the first MPM list, with some non-angular modes being replaced with angular modes.

10. The method of any of claims 1-9, wherein signaling whether two or more MPM lists are allowed is in at least one of a Video Parameter Set (VPS), a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), a picture header, a slice group header, a Largest Coding Unit (LCU), a Coding Unit (CU), a row of LCUs, and a group level of LCUs.

11. The method according to any of claims 1-10, wherein the number of allowed MPM lists is predefined or signaled to a decoder.

12. The method of claim 11, wherein the number of MPM lists is a fixed value.

13. The method of claim 12, wherein the fixed value is 2.

14. The method of claim 11, wherein the number of MPM lists is signaled in at least one of a VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU row, and LCU group level.

15. The method of any one of claims 1-14, wherein one of the first and second MPM lists is used for natural content codec and the other is used for screen content codec.

16. The method of any one of claims 1-15, wherein which MPM list is selected depends on one or more of:

i. video content including at least one of screen content and natural content;

messages signaled in VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU row, LCU group, or other video unit;

location of a Prediction Unit (PU);

a block size of the current block and/or its neighboring blocks;

v. block shape of the current block and/or its neighboring blocks;

prediction mode comprising intra-mode and inter-mode of neighboring blocks to the current block;

intra prediction modes of neighboring blocks to the current block;

motion vectors of neighboring blocks of the current block;

an indication of quantized residual block differential pulse code modulation (QR-BDPCM) mode of neighboring blocks to the current block;

an indication of a QR-BDPCM mode of a transform type for the current block;

a current quantization parameter of the current block and/or its neighboring blocks;

an indication of a color format of the current block;

coding and decoding tree structure;

slice, slice group type, and/or picture type.

17. The method of claim 16, wherein a flag is signaled in a slice header to indicate a list to select for coding a conventional intra prediction mode.

18. The method of claim 16, wherein the MPM list depends on a message indicating video content information including at least one of screen content or camera capture content.

19. The method of claim 16, wherein the color format of the current block is 4:2:0 or 4:4: 4.

20. The method of any of claims 1-19, wherein the MPM list is allowed to switch at a slice, picture, CU, LCU, group of LCUs, or block level.

21. The method of claim 20, wherein whether to switch the MPM list depends on at least one of:

i. video content including at least one of screen content and natural content;

messages signaled in VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU row, LCU group, or other video unit;

location of a Prediction Unit (PU);

a block size of the current block and/or its neighboring blocks;

v. block shape of the current block and/or its neighboring blocks;

prediction mode comprising intra-mode and inter-mode of neighboring blocks to the current block;

intra prediction modes of neighboring blocks to the current block;

motion vectors of neighboring blocks of the current block;

an indication of quantized residual block differential pulse code modulation (QR-BDPCM) mode of neighboring blocks to the current block;

an indication of a QR-BDPCM mode of a transform type for the current block;

a current quantization parameter of the current block and/or its neighboring blocks;

an indication of a color format of the current block;

coding and decoding tree structure;

slice, slice group type and/or picture type;

xv. include a color component of at least one of a luminance component and a chrominance component.

22. The method of any one of claims 1-21, wherein a particular angular mode is always in the MPM list during construction of the MPM list.

23. The method according to claim 22, wherein the specific angular mode is a vertical mode and/or a horizontal mode.

24. The method of claim 22 or 23, wherein the particular angular mode is inserted at a given position in the MPM list.

25. The method of claim 24, wherein the particular angular mode is inserted at a first position in the MPM list.

26. The method of claim 23, wherein the vertical mode and the horizontal mode are always sequentially at the first two positions in the MPM list.

27. The method of claim 23, wherein the horizontal mode and the vertical mode are always sequentially at the first two positions in the MPM list.

28. The method of claim 22, wherein the MPM list is always { INTRA _ angul 50, INTRA _ angul 18, DC, plane, INTRA _ angul 34, INTRA _ angul 66 }.

29. The method of claim 22, wherein the MPM list is always { INTRA _ anflag 18, INTRA _ anflag 50, plane, DC, INTRA _ anflag 34, INTRA _ anflag 66 }.

30. The method of claim 22, wherein the MPM list is always { INTRA _ anflag 50, INTRA _ anflag 18, plane, DC, INTRA _ anflag 34, INTRA _ anflag 66 }.

31. The method of claim 22, wherein the MPM list is always { plane, DC, INTRA _ anflag 18, INTRA _ anflag 50, INTRA _ anflag 34, INTRA _ anflag 66 }.

32. The method of claim 22, wherein the MPM list is { INTRA _ anflag 50, INTRA _ anflag 18, DC, plane, INTRA _ anflag 34, INTRA _ anflag 66}, if neither left-side neighboring INTRA-mode nor above-side neighboring INTRA-mode of the current block is available.

33. The method of claim 22, wherein the MPM list is { INTRA _ anflag 50, INTRA _ anflag 18, DC, plane, INTRA _ anflag 34, INTRA _ anflag 2}, if neither left-side neighboring INTRA-mode nor above-side neighboring INTRA-mode of the current block is available.

34. The method of claim 22, wherein if the left-neighboring INTRA-mode and the above-neighboring INTRA-mode of the current block are the same and such a mode is a non-angle INTRA-mode, the MPM list is { DC, plane, INTRA _ anflag 50, INTRA _ anflag 18, INTRA _ anflag 34, INTRA _ anflag 66 }.

35. The method of claim 22, wherein if the left-side neighboring intra-mode and the above-side neighboring intra-mode of the current block are the same, and such mode a is an angular intra-mode, the MPM list is constructed based on a direction of such mode a.

36. The method of claim 35, wherein if a > -2 and a < ═ 18, the MPM list is { INTRA _ ANGULAR18, INTRA _ ANGULAR2, INTRA _ ANGULAR50, DC, plane, INTRA _ ANGULAR66 }.

37. The method of claim 35, wherein if a > -18 and a < ═ 34, the MPM list is { INTRA _ ANGULAR18, INTRA _ ANGULAR34, INTRA _ ANGULAR50, DC, plane, INTRA _ ANGULAR2 }.

38. The method of claim 35, wherein if a > -34 and a < ═ 50, the MPM list is { INTRA _ ANGULAR50, INTRA _ ANGULAR34, INTRA _ ANGULAR18, DC, plane, INTRA _ ANGULAR66 }.

39. The method of claim 35, wherein if a > -50 and a < ═ 66, the MPM list is { INTRA _ ANGULAR50, INTRA _ ANGULAR66, INTRA _ ANGULAR18, DC, plane, INTRA _ ANGULAR34 }.

40. The method of claim 22, wherein if the left-neighboring INTRA-mode and the above-neighboring INTRA-mode of the current block are different modes, and one of them is a non-angle mode, the MPM list is { DC, plane, INTRA _ anflag 50, INTRA _ anflag 18, INTRA _ anflag 34, INTRA _ anflag 66 }.

41. The method of claim 22, wherein the left-side neighboring intra mode and the above-side neighboring intra mode of the current block are different modes and both of them are angle modes, then wherein Max represents a maximum value between the two intra modes.

42. The method of claim 41, wherein the MPM list is { INTRA _ ANGULAR18, INTRA _ ANGULAR2, INTRA _ ANGULAR50, DC, plane, INTRA _ ANGULAR66}, if Max > -2 and Max < -18.

43. The method of claim 41, wherein if Max > 18 and Max < ═ 34, the MPM list may be { INTRA _ ANGULAR18, INTRA _ ANGULAR34, INTRA _ ANGULAR50, DC, plane, INTRA _ ANGULAR2 }.

44. The method of claim 41, wherein if Max > -34 and Max < -50, the MPM list may be { INTRA _ ANGULAR50, INTRA _ ANGULAR34, INTRA _ ANGULAR18, DC, plane, INTRA _ ANGULAR66 }.

45. The method of claim 41, wherein if Max > -50 and Max < -66, the MPM list may be { INTRA _ ANGULAR50, INTRA _ ANGULAR66, INTRA _ ANGULAR18, DC, plane, INTRA _ ANGULAR34 }.

46. The method of claim 23, wherein one existing mode in the MPM list is replaced by a vertical mode and/or a horizontal mode.

47. The method of claim 46, wherein one existing mode in the MPM list having a closest direction to a vertical mode is replaced by the vertical mode.

48. The method of claim 46, wherein one existing mode in the MPM list having a closest direction to a horizontal mode can be replaced by the horizontal mode.

49. The method of claim 23, wherein the position of a vertical mode and/or a horizontal mode in the MPM list depends on at least one of:

i. a message signaled in a VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU line, LCU group, or other video unit;

location of a Prediction Unit (PU);

a block size of the current block and/or its neighboring blocks;

block shape of the current block and/or its neighboring blocks;

v. prediction modes, including intra-mode and inter-mode of neighboring blocks of the current block;

intra prediction modes of neighboring blocks to the current block;

a motion vector of a neighboring block of the current block;

an indication of quantized residual block differential pulse code modulation (QR-BDPCM) mode of a neighboring block to the current block;

an indication of a QR-BDPCM mode of a transform type for the current block;

x. a current quantization parameter of the current block and/or its neighboring blocks;

an indication of a color format of the current block;

coding and decoding tree structure;

slice, slice group type and/or picture type.

50. The method of any of claims 1-49, wherein, when the current block is a Residual Differential Pulse Code Modulation (RDPCM) coded block, an intra-mode of the RDPCM coded block is set to a mode in the MPM list other than the first mode.

51. The method of claim 50, wherein the intra-mode of the RDPCM coded block is set to a second MPM mode in the MPM list with an indication of intra _ luma _ MPM _ idx-0.

52. The method of claim 50, wherein the intra-mode of the RDPCM coded block is set to any mode in the MPM list other than the first mode with an indication of intra _ luma _ MPM _ idx ═ 0.

53. The method of any of claims 50-52, wherein whether and/or how to set the intra-mode of the RDPCM coded block depends on at least one of:

i. a message signaled in a VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU line, LCU group, or other video unit;

location of a Prediction Unit (PU);

a block size of the current block and/or its neighboring blocks;

block shape of the current block and/or its neighboring blocks;

v. prediction modes, including intra-mode and inter-mode of neighboring blocks of the current block;

intra prediction modes of neighboring blocks to the current block;

a motion vector of a neighboring block of the current block;

an indication of quantized residual block differential pulse code modulation (QR-BDPCM) mode of a neighboring block to the current block;

an indication of a QR-BDPCM mode of a transform type for the current block;

x. a current quantization parameter of the current block and/or its neighboring blocks;

an indication of a color format of the current block;

coding and decoding tree structure;

slice, slice group type and/or picture type.

54. The method of any one of claims 1-49, wherein the position of planar mode is not always the first position in the MPM list.

55. The method of claim 54, wherein the planar mode is placed at the last position in the MPM list.

56. The method of claim 54, wherein the planar mode is placed anywhere in the MPM list except for a first location.

57. The method of claim 54, wherein the planar mode is not allowed to be placed in the first position in the MPM list.

58. The method according to any of claims 54-57, wherein whether and/or how to set the position of the planar mode depends on at least one of:

i. a message signaled in a VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU line, LCU group, or other video unit;

location of a Prediction Unit (PU);

a block size of the current block and/or its neighboring blocks;

block shape of the current block and/or its neighboring blocks;

v. prediction modes, including intra-mode and inter-mode of neighboring blocks of the current block;

intra prediction modes of neighboring blocks to the current block;

a motion vector of a neighboring block of the current block;

an indication of quantized residual block differential pulse code modulation (QR-BDPCM) mode of a neighboring block to the current block;

an indication of a QR-BDPCM mode of a transform type for the current block;

x. a current quantization parameter of the current block and/or its neighboring blocks;

an indication of a color format of the current block;

coding and decoding tree structure;

slice, slice group type and/or picture type.

59. The method of any one of claims 1-58, wherein binarization of MPM indices in the MPM list does not use truncated unary binarization.

60. The method of claim 59, wherein the MPM index is intra _ luma _ MPM _ idx.

61. The method of claim 59 or 60, wherein a codeword length of the MPM index is fixed.

62. The method of claim 59 or 60, wherein the signaling of the MPM index uses Exp-Golomb binarization with order K, K being an integer.

63. The method of claim 62, wherein K is 2.

64. The method of claim 62, wherein K depends on at least one of:

i. a location of a Prediction Unit (PU);

a block size of the current block and/or its neighboring blocks;

block shape of the current block and/or its neighboring blocks;

prediction mode comprising intra-mode and inter-mode of neighboring blocks to the current block;

v. intra prediction modes of neighboring blocks of the current block;

a motion vector of a neighboring block of the current block;

an indication of a quantized residual block differential pulse code modulation (QR-BDPCM) mode of a neighboring block to the current block;

an indication of a QR-BDPCM mode of a transform type for the current block;

a current quantization parameter of the current block and/or its neighboring blocks;

x. an indication of a color format of the current block;

coding and decoding tree structure;

slice, slice group type, and/or picture type.

65. The method of any one of claims 1-64, wherein an indication of an MPM candidate in the MPM list is coded, with a first flag to indicate whether a selected intra-prediction mode is the same or different from a first MPM candidate in the MPM list, followed by remaining MPM indices of the MPM candidate.

66. The method of claim 65, wherein the first flag is an intra _ luma _ not _ plane _ flag, wherein a semantic of the intra _ luma _ not _ plane _ flag is interpreted as whether the selected intra prediction mode is not the first MPM candidate in the MPM list, and is not interpreted as whether it is not a planar mode.

67. The method of claim 65 or 66, wherein the first flag is conditionally coded.

68. The method of claim 67, wherein the first flag is not coded if the current block has screen content.

69. The method of claim 67, wherein if the selected intra-prediction mode is not the first MPM candidate in the MPM list, further signaling remaining MPM indices.

70. The method of any one of claims 1-69, wherein the MPM list construction process comprises vertical intra prediction and/or horizontal intra prediction, which correspond to directly copying reference samples in an intra prediction process.

71. The method of claim 70, wherein the vertical intra prediction is generated by directly copying reference samples in the same column.

72. The method of claim 70, wherein the horizontal intra prediction is generated by directly copying reference samples in the same row.

73. The method of any of claims 70-72, wherein applying the vertical intra prediction and/or the horizontal intra prediction depends on at least one of:

i. video content including at least one of screen content and natural content;

messages signaled in VPS, SPS, PPS, picture header, slice group header, LCU, CU, LCU row, LCU group, or other video unit;

location of a Prediction Unit (PU);

a block size of the current block and/or its neighboring blocks;

v. block shape of the current block and/or its neighboring blocks;

prediction mode comprising intra-mode and inter-mode of neighboring blocks to the current block;

intra prediction modes of neighboring blocks to the current block;

motion vectors of neighboring blocks of the current block;

an indication of quantized residual block differential pulse code modulation (QR-BDPCM) mode of neighboring blocks to the current block;

an indication of a QR-BDPCM mode of a transform type for the current block;

a current quantization parameter of the current block and/or its neighboring blocks;

an indication of a color format of the current block;

coding and decoding tree structure;

slice, slice group type and/or picture type;

xv. include a color component of at least one of a luminance component and a chrominance component.

74. The method of any of claims 1-73, wherein the first and/or second construction methods further comprise at least one of an affine intra prediction method, a matrix-based intra prediction method, an intra sub-partition prediction (ISP), and a multi-reference line (MRL).

75. The method of any of claims 21, 49, 53, 58, 64, 73, wherein the color format of the current block is 4:2:0 or 4:4: 4.

76. The method of any of claims 1-75, wherein the converting generates the current block of video from the bitstream representation.

77. The method of any of claims 1-75, wherein the converting generates the bitstream representation from the current block of video.

78. An apparatus in a video system comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any of claims 1-77.

79. A computer program product stored on a non-transitory computer readable medium, the computer program product comprising program code for implementing the method of any one of claims 1 to 77.

Images