WO2020065508A1 - Chroma mode intra coding - Google Patents
Chroma mode intra coding Download PDFInfo
- Publication number
- WO2020065508A1 WO2020065508A1 PCT/IB2019/058066 IB2019058066W WO2020065508A1 WO 2020065508 A1 WO2020065508 A1 WO 2020065508A1 IB 2019058066 W IB2019058066 W IB 2019058066W WO 2020065508 A1 WO2020065508 A1 WO 2020065508A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- block
- mode
- chroma
- intra
- coding
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/103—Selection of coding mode or of prediction mode
- H04N19/11—Selection of coding mode or of prediction mode among a plurality of spatial predictive coding modes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/184—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being bits, e.g. of the compressed video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
Definitions
- This patent document relates to video coding techniques, devices and systems.
- Devices, systems and methods related to digital video coding, and specifically, intra mode coding and chroma mode coding for images and video based of past (e.g., historical or statistical) information are described.
- the described methods may be applied to both the existing video coding standards (e.g., High Efficiency Video Coding (HEVC)) and future video coding standards (e.g., Versatile Video Coding (WC)) or codecs.
- HEVC High Efficiency Video Coding
- WC Versatile Video Coding
- the disclosed technology may be used to provide a method for video processing.
- This example method includes selecting, for a conversion between a current block of visual media data and a bitstream representation of the current block, a first intra prediction mode based on at least a first set of history intra coding information that includes statistical information of a set of intra prediction modes, and performing the conversion based on the first intra prediction mode.
- the disclosed technology may be used to provide a method for video processing.
- This example method includes, selecting, for a conversion between a current block of visual media data and a bitstream representation of the current block, a first intra prediction mode based on at least intra prediction modes associated with non-adjacent neighbors of the current block.
- the method also includes performing the conversion based on the first intra prediction mode.
- the disclosed technology may be used to provide a method for video processing.
- This example method includes selecting, for a conversion between a current block of visual media data and a bitstream representation of the current block, an intra prediction mode based on at least one of spatial neighboring blocks to the current block, and performing the conversion based on the intra prediction mode.
- the at least one of the spatial neighboring blocks is different from a first block that is located to a left of a first row of the current block and a second block that is located above a first column of the current block.
- the disclosed technology may be used to provide a method for video processing.
- This example method includes generating a final prediction block for a conversion between a current block of visual media data and a bitstream representation of the current block and performing the conversion based on the final prediction block. At least a portion of the final prediction block is generated based on a combination of a first prediction block and a second prediction block that are based on reconstructed samples from an image segment that comprises the current block.
- the disclosed technology may be used to provide a method for video processing.
- This example method includes maintaining a first table of motion candidates during a conversion between a current block of visual media data and a bitstream representation of the current block and determining, based on at least the first table of motion candidates, motion information for the current block that is coded using an intra block copy (IBC) mode in which at least one motion vector is directed at an image segment that comprises the current block.
- IBC intra block copy
- the disclosed technology may be used to provide a method for video processing.
- This example method for video processing includes determining, for a conversion between a current block of video data and a bitstream representation of the current block, a first coding mode for a chroma component of the current block based on at least one or more coding modes of a block that is not adjacent to the current block.
- the method also includes performing, based on the first coding mode, the conversion between the bitstream representation and the current block.
- the disclosed technology may be used to provide a method for video processing.
- This example method for video processing includes determining, for a conversion between a bitstream representation of a chroma block and the chroma block, an intra coding mode based on a spatial neighboring block of a luma block that corresponds to the chroma block.
- the method also includes performing, based on the intra coding mode, a conversion between the bitstream representation and the chroma block.
- the disclosed technology may be used to provide a method for video coding.
- This example method for video coding includes selecting, for a bitstream representation of a current block of visual media data, a first intra prediction mode from a set of intra prediction modes based on a first set of past intra coding information, and processing, based on the first intra prediction mode, the bitstream representation to generate the current block.
- the disclosed technology may be used to provide a method for video coding.
- This example method for video coding includes selecting, for a bitstream representation of a current block of visual media data, an intra prediction mode from at least one of adjacent or non-adjacent spatial neighboring blocks to the current block, and processing, based on the intra prediction mode, the bitstream representation to generate the current block.
- the disclosed technology may be used to provide a method for video coding.
- This example method for video coding includes generating, for a bitstream representation of a current block of visual media data, a first prediction block and a second prediction block that are based on reconstructed samples from an image segment that comprises the current block, generating at least a portion of a final prediction block based on a linear function of the first and second prediction blocks, and processing, based on the final prediction block, the bitstream representation to generate the current block.
- the disclosed technology may be used to provide a method for video coding.
- This example method for video coding includes selecting, for a bitstream representation of a current block that comprises a luma component and a chroma component, a first coding mode for the chroma component, and performing, based on the first coding mode, a conversion between the bitstream representation and the current block.
- the disclosed technology may be used to provide a method for video coding.
- This example method for video coding includes selecting, for a bitstream representation of a current block that comprises a luma component and a chroma component, an intra coding mode from a spatial neighboring block to the luma component, adding the intra coding mode to a chroma intra mode list, and performing, based on the intra coding mode, a conversion between the bitstream representation and the current block.
- the above-described method is embodied in the form of processor-executable code and stored in a computer-readable program medium.
- a device that is configured or operable to perform the above-described method.
- the device may include a processor that is
- a video decoder apparatus may implement a method as described herein.
- FIG. 1 shows an example of 67 intra prediction modes.
- FIG. 2 shows an example of neighboring blocks used for intra mode prediction.
- FIGS. 3A and 3B show examples of reference samples for wide-angle intra prediction modes for non-square blocks.
- FIG. 4 shows an example of a discontinuity when using wide-angle intra prediction.
- FIGS. 5A-5D show examples of samples used by a position-dependent intra prediction combination (PDPC) method.
- PDPC position-dependent intra prediction combination
- FIG. 6 shows an example of locations of samples used for the derivation of the weights of the linear model.
- FIG. 7 shows an example of classifying neighboring samples into two groups.
- FIG. 8A shows an example of a chroma sample and its corresponding luma samples.
- FIG. 8B shows an example of down filtering for the cross-component linear model
- FIGS. 9 A and 9B show examples of a chroma block and its corresponding luma block.
- FIG. 10 shows an example of intra prediction modes with default modes.
- FIGS. 11 A and 11B show examples of corresponding sub-blocks for a chroma CB.
- FIG. 12 shows an example of an intra-picture block copy.
- FIGS. 13A-13F show examples of different adjacent blocks that may be selected for intra mode coding.
- FIG. 14 shows a flowchart of an example method for intra-mode coding based on past information in accordance with the disclosed technology.
- FIG. 15 shows a flowchart of another example method for intra-mode coding based on past information in accordance with the disclosed technology.
- FIG. 16 shows a flowchart of yet another example method for intra- mode coding based on past information in accordance with the disclosed technology.
- FIG. 17 is a block diagram of an example of a hardware platform for implementing a visual media decoding or a visual media encoding technique described in the present document.
- FIG. 18 is a block diagram of an example video processing system in which disclosed techniques may be implemented.
- FIG. 19 shows a flowchart of an example method for video processing in accordance with the disclosed technology.
- FIG. 20 shows a flowchart of another example method for video processing in accordance with the disclosed technology.
- FIG. 21 shows a flowchart of another example method for video processing in accordance with the disclosed technology.
- FIG. 22 shows a flowchart of another example method for video processing in accordance with the disclosed technology.
- FIG. 23 shows a flowchart of another example method for video processing in accordance with the disclosed technology.
- FIG. 24 shows a flowchart of an example method for chroma mode coding based on past information.
- FIG. 25 shows a flowchart of another example method for chroma mode coding based on past information.
- FIG. 26 shows a flowchart of an example method for video processing in accordance with the disclosed technology.
- FIG. 27 shows a flowchart of yet another example method for video processing in accordance with the disclosed technology.
- Video codecs typically include an electronic circuit or software that compresses or decompresses digital video, and are continually being improved to provide higher coding efficiency.
- a video codec converts uncompressed video to a compressed format or vice versa.
- the compressed format usually conforms to a standard video compression specification, e.g., the High Efficiency Video Coding (HEVC) standard (also known as H.265 or MPEG-H Part 2), the Versatile Video Coding (VVC) standard to be finalized, or other current and/or future video coding standards.
- HEVC High Efficiency Video Coding
- VVC Versatile Video Coding
- Embodiments of the disclosed technology may be applied to existing video coding standards (e.g., HEVC, H.265) and future standards to improve runtime performance. Section headings are used in the present document to improve readability of the description and do not in any way limit the discussion or the embodiments (and/or implementations) to the respective sections only.
- the number of directional intra modes is extended from 33, as used in HEVC, to 65.
- the additional directional modes are depicted as red dotted arrows in FIG. 1, and the planar and DC modes remain the same.
- These denser directional intra prediction modes apply for all block sizes and for both luma and chroma intra predictions.
- Conventional angular intra prediction directions are defined from 45 degrees to -135 degrees in clockwise direction as shown in FIG. 1.
- VTM2 several conventional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes for the non- square blocks.
- the replaced modes are signaled using the original method and remapped to the indexes of wide angular modes after parsing.
- the total number of intra prediction modes is unchanged (e.g., 67), and the intra mode coding is unchanged.
- every intra-coded block has a square shape and the length of each of its side is a power of 2. Thus, no division operations are required to generate an intra-predictor using DC mode.
- blocks can have a rectangular shape that necessitates the use of a division operation per block in the general case. To avoid division operations for DC prediction, only the longer side is used to compute the average for non-square blocks.
- MPMs 3 Most Probable Modes
- a and B For neighbor intra modes (A and B), two neighbouring blocks, located in left and above are considered.
- the left and above blocks are those who are connected to the top-left sample of current block, as shown in FIG. 2.
- An initial MPM list is formed by performing pruning process for two neighboring intra modes. The pruning process is used to remove duplicated modes so that only unique modes can be included into the MPM list. If two neighboring modes are different each other, one of the default modes (e.g., PLANA (0), DC (1), ANGULAR50 (e.g., 50)) is added to the MPM list after the pruning check with the existing two MPMs. When the two neighboring modes are the same, either the default modes or the derived modes are added to the MPM list after the pruning check.
- the detailed generation process of three MPM list is derived as follows:
- An additional pruning process is used to remove duplicated modes so that only unique modes can be included into the MPM list.
- FLC Fixed Length Code
- conventional angular intra prediction directions are defined from 45 degrees to -135 degrees in clockwise direction.
- VTM2 several conventional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes for non square blocks.
- the replaced modes are signaled using the original method and remapped to the indexes of wide angular modes after parsing.
- the total number of intra prediction modes for a certain block is unchanged, e.g., 67, and the intra mode coding is unchanged.
- top reference with length 2W+1 and the left reference with length 2H+1, are defined as shown in the examples in FIGS. 3 A and 3B.
- the mode number of replaced mode in wide-angular direction mode is dependent on the aspect ratio of a block.
- the replaced intra prediction modes are illustrated in Table 1.
- two vertically-adjacent predicted samples may use two non- adjacent reference samples in the case of wide-angle intra prediction.
- low-pass reference samples filter and side smoothing are applied to the wide-angle prediction to reduce the negative effect of the increased gap Dr a .
- PDPC position dependent intra prediction combination
- PDPC is an intra prediction method which invokes a combination of the un-filter ed boundary reference samples and HEVC style intra prediction with filtered boundary reference samples.
- PDPC is applied to the following intra modes without signaling: planar, DC, horizontal, vertical, bottom-left angular mode and its eight adjacent angular modes, and top-right angular mode and its eight adjacent angular modes.
- the prediction sample pred(x,y ) is predicted using an intra prediction mode (DC, planar, angular) and a linear combination of reference samples according to the Equation as follows:
- pred(x,y) (wL R-i, y + wTxRx,- 1 - wTL xi?-i,-i+(64 - wL - wT+wTL)xpred(x,y )
- R x, - I, R- ⁇ , y represent the reference samples located at the top and left of current sample (x, y), respectively, and R- represents the reference sample located at the top-left corner of the current block.
- FIGS. 5A-5D illustrate the definition of reference samples (Rx,- 1, R- ⁇ , y and R-i -i) for PDPC applied over various prediction modes.
- the prediction sample pred (x’, y’) is located at (x’, y’) within the prediction block.
- Table 2 Examples of PDPC weights according to prediction modes
- HEVC chroma coding five modes (including one direct mode (DM) which is the intra prediction mode from the top-left corresponding luma block and four default modes) are allowed for a chroma block.
- the two color components share the same intra prediction mode.
- two new methods have been proposed, including: cross-component linear model (CCLM) prediction mode and multiple DMs.
- CCLM cross-component linear model
- CCLM cross component linear model
- pred c ⁇ i,j represents the predicted chroma samples in a CU and rec L ' i,j ) represents the downsampled reconstructed luma samples of the same CU for color formats 4:2:0 or 4:2:2 while rec L ' i,j ) represents the reconstructed luma samples of the same CU for color format 4:4:4.
- CCLM parameters a and b are derived by minimizing the regression error between the neighboring reconstructed luma and chroma samples around the current block as follows:
- L(n ) represents the down-sampled (for color formats 4:2:0 or 4:2:2) or original
- C(n ) represents the top and left neighboring reconstructed chroma samples
- value of N is equal to twice of the minimum of width and height of the current chroma coding block.
- the above two equations are applied directly.
- the neighboring samples of the longer boundary are first subsampled to have the same number of samples as for the shorter boundary.
- FIG. 6 shows the location of the left and above
- this regression error minimization computation is performed as part of the decoding process, not just as an encoder search operation, so no syntax is used to convey the a and b values.
- the CCLM prediction mode also includes prediction between the two chroma components, e.g., the Cr (red-difference) component is predicted from the Cb (blue- difference) component.
- the CCLM Cb-to- Cr prediction is applied in residual domain. This is implemented by adding a weighted reconstructed Cb residual to the original Cr intra prediction to form the final Cr prediction:
- pred C * r (i,j ) pred Cr (i,j ) + a resi cb '(i,j ) (4)
- resi cb '(i,j ) presents the reconstructed Cb residue sample at position (i,j).
- the scaling factor a may be derived in a similar way as in the CCLM luma-to-chroma prediction. The only difference is an addition of a regression cost relative to a default a value in the error function so that the derived scaling factor is biased towards a default value of -0.5 as follows:
- Cb(n ) represents the neighboring reconstructed Cb samples
- Cr(n ) represents the neighboring reconstructed Cr samples
- l is equal to ⁇ Cb(n)) » 9.
- the CCLM luma-to-chroma prediction mode is added as one additional chroma intra prediction mode.
- one more RD cost check for the chroma components is added for selecting the chroma intra prediction mode.
- intra prediction modes other than the CCLM luma-to-chroma prediction mode is used for the chroma components of a CU
- CCLM Cb-to-Cr prediction is used for Cr component prediction.
- the single model CCLM mode employs one linear model for predicting the chroma samples from the luma samples for the whole CU, while in MMLM, there can be two models.
- neighboring luma samples and neighboring chroma samples of the current block are classified into two groups, each group is used as a training set to derive a linear model (i.e., a particular a and b are derived for a particular group). Furthermore, the samples of the current luma block are also classified based on the same rule for the classification of neighboring luma samples.
- FIGS. 8 A and 8B show an example of a chroma sample and its corresponding luma samples, and an example of of down filtering for the cross-component linear model (CCLM) in the Joint Exploration Model (JEM), respectively.
- CCLM cross-component linear model
- JEM Joint Exploration Model
- a candidate list of chroma intra prediction modes may be derived first, and wherein three parts may be included:
- One direct mode which is set to the one intra luma prediction mode associated with luma CB covering the co-located top-left position of a chroma block.
- DM direct mode
- FIGS. 9 A and 9B An example is shown in FIGS. 9 A and 9B with the co-located position denoted as“TL”
- syntax and semantics of intra mode related information may be specified as follows:
- Table 3 Example of a coding unit (CU) syntax
- intra_luma_mpm_remainder[ x0 ][ yO ] specify the intra prediction mode for luma samples.
- the array indices xO, yO specify the location ( xO , yO ) of the top-left luma sample of the considered prediction block relative to the top-left luma sample of the picture.
- intra_luma_mpm_flag[ xO ][ yO ] is equal to 1
- the intra prediction mode is inferred from a neighbouring intra-predicted prediction unit according to the specification (e.g., clause 8.2.2).
- intra_chroma_pred_mode[ xO ][ yO ] specifies the intra prediction mode for chroma samples.
- the array indices xO, yO specify the location ( xO, yO ) of the top-left luma sample of the considered prediction block relative to the top-left luma sample of the picture.
- an input to this process is a luma location ( xPb , yPb ) specifying the top-left sample of the current luma prediction block relative to the top-left luma sample of the current picture.
- the luma intra prediction mode IntraPredModeY[ xPb ][ yPb ] is derived.
- Table 4 specifies the value for the intra prediction mode and the associated names.
- Table 4 Example specification of intra prediction modes and associated names
- IntraPredModeY[ xPb ][ yPb ] labelled 0..66 represents directions of predictions as shown in FIG. 1.
- IntraPredModeY[ xPb ][ yPb ] is derived by the following ordered steps:
- the candidate intra prediction mode candlntraPredModeX is derived as follows:
- candlntraPredModeX is set equal to INTRA DC.
- variable availableX is equal to FALSE.
- candlntraPredModeB is equal to candlntraPredModeA, the following applies:
- candModeList[ 0 ] and candModeList[ 1 ] are derived as follows:
- candModeList[ 2 ] is set equal to INTRA_ANGULAR50.
- ntraPredModeY[ xPb ][ yPb ] is derived by applying the following procedure:
- IntraPredModeY[ xPb ][ yPb ] is set equal to
- IntraPredModeY[ xPb ][ yPb ] is derived by the following ordered steps:
- IntraPredModeY[ xPb ][ yPb ] is set equal to
- IntraPredModeY[ xPb ] [ yPb ] is incremented by one.
- an input to this process is a luma location ( xPb, yPb ) specifying the top-left sample of the current chroma prediction block relative to the top-left luma sample of the current picture.
- the chroma intra prediction mode IntraPredModeC[ xPb ][ yPb ] is derived.
- the chroma intra prediction mode [00112] In some embodiments, the chroma intra prediction mode
- IntraPredModeC[ xPb ][ yPb ] is derived using intra_chroma_pred_mode[ xPb ][ yPb ] and
- IntraPredModeY[ xPb ][ yPb ] as specified in Table 5 and Table 6.
- MDM Multiple Direct Modes
- Intra prediction modes derived from luma CBs covering the co-located five positions of the current chroma block are defined as: center, top-left, top-right, bottom-left and bottom-right 4x4 block within the corresponding luma block of current chroma block for I slices.
- P and B slices there is no need to check these five sub-blocks since they shall be within one CB.
- An example of five co-located luma positions is shown in FIGS. 11 A and 11B, they are denoted by CR, TL, TR, BL and BR, respectively.
- Mode 50 Mode 2
- Mode 34 Mode 34
- Mode 66 Mode 10
- Mode 26 Mode 26 (with Mode 66, Mode 10 and Mode 26 being added)
- the pruning process is applied whenever a new chroma intra mode is added to the candidate list.
- the chroma intra mode candidate list size is always set to 10.
- IBC Intra block copy
- CPR current picture referencing
- the reference samples correspond to the reconstructed samples of the current picture prior to in-loop filter operations.
- the in-loop filters refer to both deblocking and sample adaptive offset (SAO) filters.
- SAO sample adaptive offset
- the use of the IBC mode is signaled at both sequence and picture level.
- the IBC mode is enabled at sequence parameter set (SPS)
- SPS sequence parameter set
- the IBC mode is enabled at picture level, the current reconstructed picture is treated as a reference picture. Therefore, no syntax change on block level is needed in HEVC SCC on top of the existing HEVC inter mode to signal the use of the IBC mode.
- the following is a list of features for the IBC mode in HEVC SCC:
- IBC mode e.g., 2Nx2N or 2N/N or Nx2N or NxN and Asymmetrical Motion Partitioning (AMP) modes.
- AMP Asymmetrical Motion Partitioning
- merge and skip modes are also available for the IBC mode.
- the merge candidate list construction is unified, containing merge candidates from the neighboring positions that are either coded in the IBC mode or the HEVC inter mode.
- the current block under merge or skip mode can merge into either an IBC mode coded neighbor or otherwise an HEVC inter mode coded one.
- Block vector prediction and coding schemes for the IBC mode reuse the schemes used for motion vector prediction and coding in the HEVC inter mode (AMVP and MVD coding).
- some embodiments comprise additional features (or treatments) for IBC modes, including:
- the motion vector for the IBC mode also referred as block vector, is coded with integer-pel precision, but stored in memory in quarter-pel precision after decoding as quarter-pel precision is required in interpolation and deblocking stages.
- the stored vector predictor When used in motion vector prediction for the IBC mode, the stored vector predictor will be right shifted by 2.
- a slice with the IBC mode enabled will be treated as a P or B slice, even though no other picture than the current picture itself is used as a reference picture.
- IBC could take care of repeated patterns which are far from the current block and intra coding mode could take care of reducing less spatial correlation. However, either of the two could only be beneficial for a certain case.
- Embodiments of the presently disclosed technology overcome drawbacks of existing implementations, thereby providing video coding with higher coding efficiencies but lower computational complexity.
- the examples of the disclosed technology provided below explain general concepts, and are not meant to be interpreted as limiting. In an example, unless explicitly indicated to the contrary, the various features described in these examples may be combined.
- Examples of intra mode codins based on historical information [00141] Examples of intra mode codins based on historical information [00142] Example 1.
- a table with statistical information of intra prediction modes (IPMs) may be maintained in the encoding/decoding process to help intra mode coding.
- the table includes the occurrence and/or frequency of each
- the table may have entries for a partial set of allowed IPMs.
- IPMs from spatial neighboring blocks are firstly added to the MPM list in order. Afterwards, a few IPMs from the table may be added in order after pruning. If the list is not full, derived IPMs may be further added.
- MPM list size may be extended.
- IPMs from spatial neighboring blocks and derived modes may be firstly added to the MPM list in order. Afterwards, a few IPMs from the table may be added in order after pruning.
- IPMs included in a MPM list may be reordered based on the table.
- the MPM list is firstly constructed based on
- MPM list is re-ordered based on the descending order of occurrence or frequency associated with IPMs recorded in the table. In such a way, a IPM in the MPM list with highest occurrence or frequency will be adjusted to be in the first position of the MPM list.
- Example 2 Different tables may be used for different color components.
- a table is used for luma intra-mode coding and another table is used for chroma intra-mode coding.
- the occurrence of a mode has a maximum value (e.g. 255).
- all occurrence number in the table is right shift by a number such as 2.
- all occurrence number in the table subtract a number, such as the minimum occurrence number in the table.
- Example 4 Instead of utilizing IPMs of adjacent neighboring blocks for coding a block, intra prediction modes of non-adjacent neighboring blocks may be utilized.
- IPMs of non-adjacent neighboring blocks may be added to the MPM list.
- pruning of IPMs may be applied. If an IPM of a non-adjacent neighboring block is different from any of IPMs in the MPMS list, it may be added to the MPM list.
- MPM list size may be extended.
- IPMs from adjacent neighboring blocks and derived modes may be firstly added to the MPM list in order. Afterwards, IPMs from non- adjacent neighboring blocks may be added in order after pruning.
- IPMs of non-adjacent neighboring blocks may be used to re-order the non-
- MPM IPMs For those IPMs associated with non-adjacent neighboring blocks, higher priority may be assigned, e.g., less bits may be used to code them.
- Example 5 Coding of an IPM associated with current block may depend on the table, IPMs from adjacent spatial neighboring blocks, IPMs from non-adjacent spatial neighboring blocks.
- Example 6 Instead of using of the above and left neighboring blocks relative to the top-left position of current block for intra mode coding (such as using the intra prediction modes of above and left neighboring blocks to construct a MPM list), different positions may be utilized.
- the left neighboring block relative to the top-left position of current block is replaced by the left neighboring block relative to the center position of the first column of current block, as shown in FIG. 13D.
- Example 7 Instead of utilizing the fixed M relative neighboring blocks for all kinds of blocks, selection of M neighboring blocks may depend on block shape and/or block size.
- the intra prediction modes of above and left neighboring blocks relative to the to-left position of current block for intra mode coding may be used for coding intra prediction mode of current block.
- the intra prediction modes of above and/or left neighboring blocks relative to the center position of the first row and/or the first column of current block for intra mode coding may be used for coding intra prediction mode of current block.
- Example 8 Multiple prediction blocks may be generated with reconstructed samples from the same tile/slice/picture containing current block.
- two prediction blocks may be firstly generated and the final prediction block is generated from the two prediction blocks wherein one of them is generated by an intra prediction mode and the other is generated by a motion vector pointing to the same tile/slice/picture (e.g., intra block copy; and the motion vector is also known as block vector) containing current block.
- a motion vector pointing to the same tile/slice/picture (e.g., intra block copy; and the motion vector is also known as block vector) containing current block.
- average of the two prediction blocks i.e., equal weights for the two prediction blocks
- weights may be signaled in SPS/PPS/VPS/picture or sequence or slice header/tile/groups of CTUs/CTU/CU/PU/TU.
- Weights applied to the two prediction blocks may be pre-defined.
- Example 9 The above methods may be applied under certain conditions.
- Example 10 When IBC is treated as inter mode (i.e., with at least one motion vector to derive the prediction block), History-based Motion Vector Prediction (HMVP) methods, which utilize previously coded motion information for motion information prediction may be used.
- HMVP methods allow the encoding/decoding process to be performed based on historical data (e.g., the blocks that have been processed).
- One or multiple tables with motion information from previously coded blocks are maintained during the encoding/decoding process. Each table can be associated with a counter to track the number of motion candidates stored in the table.
- the associated motion information in tables may be used (e.g., by being added to the motion candidate lists) depending on the coding characteristics of the block (e.g., whether the block is IBC coded).
- the tables may be updated based on the coding characteristics of the block. That is, the updating of the tables is based on the encoding/decoding order.
- the tables can be reset (e.g., removing the stored candidates and setting the associated counter to 0) before processing a new slice, a new largest coding unit (LCU) row, or a new tile.
- look-up-tables may be updated with motion information from
- look-up-tables are not allowed to be updated with motion information from IBC-coded blocks.
- a look-up table may include motion information from
- IBC-coded blocks and other kinds of inter-coded blocks.
- a motion candidate in the look-up table is derived from an IBC coded block, it may be used to encode/decode current IBC coded block.
- a first look-up table can include motion information for an IBC-coded block.
- a second, different look-up table can include motion information for a block that is not IBC-coded (e.g., the block can be coded with an inter mode).
- the first and second look-up tables can be updated after the corresponding conversions are completed.
- a first subsequent conversion of an IBC coded block can be performed using the updated first look-up table, and a second subsequent conversion of an inter coded block can be performed using the updated second look-up table.
- pruning may be applied before updating look-up tables as described in Example 10 by adding a motion candidate obtained from a coded block.
- the new motion candidate to be added need to be pruned to all existing motion candidates in selected tables. That is, the new motion candidate is compared against all existing motion candidates in selected tables to make sure that there are no duplicated candidates.
- the new motion candidates may be only pruned to a certain number of existing motion candidates. For example, it may be compared against the last m motion candidates ( ⁇ m is an integer) in a look-up table before selectively adding it as the last entry of the LUT.
- the selected table is the one that is used to construct the motion candidate list for the coded block where the motion candidate is obtained from.
- the motion candidate can be used to update partial or all of available look-up tables.
- the look-up tables as described in Example 10 may be used when a block is coded with merge or AMVP mode. That is, the current block can be coded using an IBC merge mode or an IBC AMVP mode.
- non-adjacent candidates e.g., merge candidates or AMVP candidates
- merge candidates or AMVP candidates can be combined in the look-up tables.
- m candidates non-adjacent and/or adjacent candidates
- n motion candidates from look-up tables may be added (e.g., when the list is not full). Both m and n are positive integers (> 0).
- m candidates non-adjacent and/or adjacent candidates
- n motion candidates from look-up tables may be checked.
- the checked candidates may be added when the list is not full.
- the adjacent or non-adjacent candidates and motion candidates from look-up tables may be added in an interleaved way.
- one non-adjacent block is located outside a certain range (e.g., outside the current LCU row), it would be replaced by a motion candidate in a look-up table.
- non-adjacent candidates have a higher priority compared to motion candidates from look-up tables.
- motion candidates from look-up tables have a higher priority compared to non-adjacent candidates. Motion candidates from look-up tables are first checked. If the total number of candidates are still less than the maximumly allowed number, the non-adjacent blocks are checked to add non-adjacent candidates to the merge candidate list.
- look-up tables may be constructed, and/or updated to store intra prediction modes from previously coded blocks and look-up tables may be used for coding/decoding an intra-coded block.
- a variable i.e., cnt
- cnt may be further assigned for each intra prediction mode to record how many times the intra prediction mode has been used in previously coded blocks.
- intra prediction modes of non-adjacent blocks may be used as intra prediction mode predictors for coding an intra-coded block.
- look-up tables and non-adjacent based methods may be jointly utilized.
- the intra prediction modes in either look-up tables or non-adjacent blocks may be used in the MPM list construction process.
- the intra prediction modes in either look-up tables or non-adjacent blocks may be used to re-order non-MPM intra prediction modes.
- a motion candidate list for an IBC-coded block can be constructed as follows:
- the last available motion candidate is then added to the list.
- Example 11 it is proposed that for chroma mode coding, the default modes (e.g., DC/Planar/HOR/VER in HEVC and VVC) may be replaced by modes derived from look-up table or non-adjacent chroma blocks.
- the default modes e.g., DC/Planar/HOR/VER in HEVC and VVC
- modes derived from look-up table or non-adjacent chroma blocks may be replaced by modes derived from look-up table or non-adjacent chroma blocks.
- the default modes may be still added to the chroma mode list if the list is not full after checking those modes from look-up table or non-adjacent chroma blocks.
- Example 12 it is proposed that for chroma mode coding, the default modes (e.g., DC/Planar/HOR/VER in HEVC and WC) may be replaced by modes derived from non-adjacent luma blocks (e.g., a luma block that is not the corresponding luma block of the chroma block).
- the default modes e.g., DC/Planar/HOR/VER in HEVC and WC
- non-adjacent luma blocks e.g., a luma block that is not the corresponding luma block of the chroma block.
- the default modes may be still added to the chroma mode list if the list is not full after checking those modes from non-adjacent luma blocks.
- Example 13 In one example, it is proposed that the intra modes of spatial
- neighboring blocks of the corresponding luma block may be also added to the chroma intra mode list.
- FIG. 13A-13F The examples shown in FIG. 13A-13F may be used wherein the current luma block is the corresponding luma block of a chroma block.
- the neighbor’s luma intra modes may be added to the chroma mode list. That is, intra modes of a neighboring luma block of the current luma block may be added to the chroma mode list.
- FIG. 14 shows a flowchart of an example method for cross-component prediction.
- the method 1400 includes, at step 1410, selecting, for a bitstream representation of a current block of visual media data, a first intra prediction mode from a set of intra prediction modes based on a first set of past intra coding information.
- the first set of past intra coding information includes historical intra coding information.
- the first set of past intra coding information comprises statistical intra coding information.
- the statistical intra coding information comprises a number of occurrences of each of the set of intra prediction modes over a time duration
- the method 1400 further includes the steps of constructing a most probable mode (MPM) list based on neighboring blocks of the current block, and reordering the MPM list based on the number of occurrences.
- MPM most probable mode
- the method 1400 includes, at step 1420, processing, based on the first intra prediction mode, the bitstream representation to generate the current block.
- the current block comprises a luma component and a chroma component
- the first set of past intra coding information is used for intra-mode coding the luma component
- a second set of past intra coding information is used for intra-mode coding the chroma component.
- the luma component is based on a first coding tree
- the chroma component is based on a different second coding tree.
- the number of occurrences is incremented for an intra prediction mode of the set of intra prediction modes that corresponds to the first intra prediction mode.
- the method 1400 further includes the step of determining that at least one of the number of occurrences is equal to a maximum number of occurrences.
- the method further includes right shifting each of the number of occurrences by a predefined number.
- the method further includes subtracting a minimum number of occurrences from each of the number of occurrences.
- the method 1400 further includes the step of adding a first subset of the set of intra prediction modes to a most probable mode (MPM) list.
- the method 1400 may further include increasing a size of an MPM list, and adding a first subset of the set of intra prediction modes to the MPM list, wherein the first subset of intra prediction modes correspond to blocks that are spatial neighbors of the current block.
- the method 1400 may further include pruning the MPM list, and adding a second subset of the set of intra prediction modes to the MPM list, wherein an ordering of intra prediction modes in the second subset is maintained in the MPM list.
- FIG. 15 shows a flowchart of another example method for cross-component prediction.
- the method 1500 includes, at step 1510, selecting, for a bitstream representation of a current block of visual media data, an intra prediction mode from at least one of adjacent or non- adjacent spatial neighboring blocks to the current block.
- the at least one of the adjacent spatial neighboring blocks is an above neighboring block relative to a top-left position of the current block, or an above neighboring block relative to a center position of a first row of the current block, or an above neighboring block relative to a top-right position of the current block, or a left neighboring block relative to a top-left position of the current block, or a left neighboring block relative to a center position of a first column of the current block, or a left neighboring block relative to a bottom-left position of the current block.
- the method 1500 includes, at step 1520, processing, based on the intra prediction mode, the bitstream representation to generate the current block.
- FIG. 16 shows a flowchart of yet another example method for cross-component prediction.
- the method 1600 includes, at step 1610, generating, for a bitstream representation of a current block of visual media data, a first prediction block and a second prediction block that are based on reconstructed samples from an image segment that comprises the current block.
- the method 1600 includes, at step 1620, generating at least a portion of a final prediction block based on a linear function of the first and second prediction blocks.
- the final prediction block is an average of the first and second prediction blocks.
- one portion of the final prediction block is based on the linear function of the first and second prediction blocks, and the remaining portion is copied directly from the first or second prediction blocks.
- the weights of the linear function are signaled in a sequence parameter set (SPS), a picture parameter set (PPS), a video parameter set (VPS), a slice header, a tile header, a group of coding tree units (CTUs), a coding unit (CU), a prediction unit (PU) or a transform unit (TU).
- the method 1600 includes, at step 1630, processing, based on the final prediction block, the bitstream representation to generate the current block.
- FIG. 24 shows a flowchart of an example method for chroma mode coding based on past information.
- the method 2400 includes, at step 2410, selecting, for a bitstream
- the method 2400 includes, at step 2420, performing, based on the first coding mode, a conversion between the bitstream representation and the current block.
- the conversion generates the current block from the bitstream representation (e.g., as might be implemented in a video decoder).
- the conversion generates the bitstream representation from the current block (e.g., as might be implemented in a video encoder).
- the first coding mode for the chroma component is a default mode, which may be a DC mode, a planar mode, a vertical mode or a horizontal mode.
- the method 2400 may further include the step of replacing the default mode with a second coding mode derived from a look-up table or a non-adjacent chroma block.
- the default mode is added to a chroma mode list upon determining that the chroma mode list comprises the second coding mode from the non-adjacent chroma block and is not full.
- the method 2400 may further include the step of replacing the default mode with a second coding mode derived from a non-adjacent luma block.
- the default mode is added to a chroma mode list upon determining that the chroma mode list comprises the second coding mode from the non-adjacent luma block and is not full.
- FIG. 25 shows a flowchart of another example method for chroma mode coding based on past information.
- the method 2500 includes, at step 2510, selecting, for a bitstream representation of a current block that comprises a luma component and a chroma component, an intra coding mode from a spatial neighboring block to the luma component.
- the method 2500 includes, at step 2520, adding the intra coding mode to a chroma intra mode list.
- the method 2500 includes, at step 2530, performing, based on the intra coding mode, a conversion between the bitstream representation and the current block.
- the spatial neighboring block may be an above neighboring block relative to a top-left position of the current block, an above neighboring block relative to a center position of a first row of the current block, an above neighboring block relative to a top-right position of the current block, a left neighboring block relative to a center position of a first column of the current block, or a left neighboring block relative to a bottom-left position of the current block.
- FIG. 17 is a block diagram of a video processing apparatus 1700.
- the apparatus 1700 may be used to implement one or more of the methods described herein.
- the apparatus 1700 may be embodied in a smartphone, tablet, computer, Internet of Things (IoT) receiver, and so on.
- the apparatus 1700 may include one or more processors 1702, one or more memories 1704 and video processing hardware 1706.
- the processor(s) 1702 may be configured to implement one or more methods (including, but not limited to, methods 1400, 1500 and 1600) described in the present document.
- the memory (memories) 1704 may be used for storing data and code used for implementing the methods and techniques described herein.
- the video processing hardware 1706 may be used to implement, in hardware circuitry, some techniques described in the present document.
- the video coding methods may be implemented using an apparatus that is implemented on a hardware platform as described with respect to FIG. 17.
- FIG. 18 is a block diagram showing an example video processing system 1800 in which various techniques disclosed herein may be implemented.
- the system 1800 may include input 1802 for receiving video content.
- the video content may be received in a raw or uncompressed format, e.g., 8 or 10 bit multi-component pixel values, or may be in a compressed or encoded format.
- the input 1802 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interface include wired interfaces such as Ethernet, passive optical network (PON), etc. and wireless interfaces such as Wi-Fi or cellular interfaces.
- the system 1800 may include a coding component 1804 that may implement the various coding or encoding methods described in the present document.
- the coding component 1804 may reduce the average bitrate of video from the input 1802 to the output of the coding component 1804 to produce a coded representation of the video.
- the coding techniques are therefore sometimes called video compression or video transcoding techniques.
- the output of the coding component 1804 may be either stored, or transmitted via a communication connected, as represented by the component 1806.
- the stored or communicated bitstream (or coded) representation of the video received at the input 1802 may be used by the component 1808 for generating pixel values or displayable video that is sent to a display interface 1810.
- the process of generating user-viewable video from the bitstream representation is sometimes called video decompression.
- certain video processing operations are referred to as “coding” operations or tools, it will be appreciated that the coding tools or operations are used at an encoder and corresponding decoding tools or operations that reverse the results of the coding will be performed by
- Examples of a peripheral bus interface or a display interface may include universal serial bus (ETSB) or high definition multimedia interface (HDMI) or Displayport, and so on.
- Examples of storage interfaces include SATA (serial advanced technology attachment), PCI,
- IDE interface and the like.
- the techniques described in the present document may be embodied in various electronic devices such as mobile phones, laptops, smartphones or other devices that are capable of performing digital data processing and/or video display.
- FIG. 19 shows a flowchart of an example method 1900 for video coding in accordance with the disclosed technology.
- the method 1900 includes, at operation 1910, selecting, for a conversion between a current block of visual media data and a bitstream representation of the current block, a first intra prediction mode based on at least a first set of history intra coding information that includes statistical information of a set of intra prediction modes.
- the method 1900 also includes, at operation 1920, performing the conversion based on the first intra prediction mode.
- the conversion includes encoding the current block to generate the bitstream representation.
- the conversion includes decoding the bitstream representation to generate the current block.
- the statistical intra coding information comprises a number of occurrences of each of the set of intra prediction modes over a time duration. In some embodiments, the statistical intra coding information comprises a number of occurrences of a part of the set of intra prediction modes over a time duration.
- the method further includes, comprising, after performing the conversion, updating the set of history intra coding information.
- the updating comprises accumulating the number of occurrences of the first intra prediction mode by k in the first set of history intra coding information, k being a positive integer.
- a subsequent block of the current block is processed based on the updated history intra coding information.
- each intra prediction mode in the first set of history intra coding information is associated with a limit for the number of occurrences.
- the method includes, after the number of occurrences reaches the limit, reducing the number of occurrences based on a predefined rule.
- the predefined rule comprises dividing the number of occurrences by a first predefined value, or subtracting a second predefined value from the number of occurrences.
- the method includes constructing a most probable mode (MPM) list based on at least the set of intra prediction modes.
- the method includes adding a subset of the set of intra prediction modes to the MPM list.
- the method includes adding intra prediction modes associated with spatial neighbors of the current block to the MPM list, adding a subset of the set of intra prediction modes to the MPM list, and pruning the MPM list.
- the method includes adding derived intra prediction modes to the MPM list.
- the method includes increasing a size of the MPM list.
- the method includes reordering the MPM list based on the set of history intra coding information.
- the MPM list is reordered based on a descending order of the statistical information of the set of intra prediction modes.
- the current block comprises a luma component and a chroma component.
- the first set of history intra coding information is used for intra-mode coding the luma component.
- a second set of history intra coding information is used for intra-mode coding the chroma component.
- the luma component is based on a first coding tree and the chroma component is based on a second coding tree different from the first coding tree.
- selecting the first intra prediction mode is further based on intra prediction modes of spatial neighbors of the current block or intra prediction modes of non- adjacent neighbors of the current block.
- the history intra coding information is stored in a table.
- FIG. 20 shows a flowchart of another example method 2000 for video processing in accordance with the disclosed technology.
- the method 2000 includes, at operation 2010, selecting, for a conversion between a current block of visual media data and a bitstream representation of the current block, a first intra prediction mode based on at least intra prediction modes associated with non-adjacent neighbors of the current block.
- the method 2000 also includes, at operation 2020, performing the conversion based on the first intra prediction mode.
- the conversion includes encoding the current block to generate the bitstream representation. In some embodiments, the conversion includes decoding the bitstream representation to generate the current block.
- the method further includes constructing, for the conversion between the current block and the bitstream representation of the current block, a most probable mode (MPM) list based on the intra prediction modes associated with the non-adjacent neighbors of the current block. The conversion is performed further based on the MPM list.
- constructing the MPM list includes adding intra prediction modes associated with spatial neighbors of the current block to the MPM list and pruning the MPM list.
- constructing the MPM list includes adding derived intra prediction modes to the MPM list.
- the method includes increasing a size of the MPM list.
- the method includes reordering intra prediction modes that are not in the MPM list according to the intra prediction modes associated with the non-adjacent neighbors of the current block.
- the intra prediction modes associated with the non-adjacent neighbors of the current block are associated with a higher priority for the reordering.
- FIG. 21 shows a flowchart of an example method 2100 for video processing in accordance with the disclosed technology.
- the method 2100 includes, at operation 2110, selecting, for a conversion between a current block of visual media data and a bitstream representation of the current block, an intra prediction mode based on at least one of spatial neighboring blocks of the current block.
- the at least one of the spatial neighboring blocks is different from a first block that is located to a left of a first row of the current block and a second block that is located above a first column of the current block.
- the method 2100 also includes, at operation 2110, performing the conversion based on the intra prediction mode.
- the conversion comprises encoding the current block to generate the bitstream representation. In some embodiments, the conversion comprises decoding the bitstream representation to generate the current block.
- the at least one of the spatial neighboring blocks includes a block adjacent to the current block. In some embodiments, the at least one of the spatial neighboring blocks includes a block non-adjacent to the current block. In some embodiments, the at least one of the adjacent spatial neighboring blocks includes a block adjacent to a top-left position of the current block. In some embodiments, the at least one of the adjacent spatial neighboring blocks includes a block adjacent to a center position of a first row of the current block. In some embodiments, the at least one of the adjacent spatial neighboring blocks includes a block adjacent to a center position of a first column of the current block.
- the at least one of the spatial neighboring blocks is selected based on one or more dimensions of the current block.
- the current block has a square shape, and the at least one of the adjacent spatial neighboring blocks includes a block adjacent to a top-left position of the current block.
- the selected intra prediction mode can be added to a most-probable-mode list of the current block.
- the current block has a non-square shape
- the at least one of the spatial neighboring blocks includes a block adjacent to a center position of a first row of the current block or a block adjacent to a center position of a first column of the current block.
- FIG. 22 shows a flowchart of an example method 2200 for video processing in accordance with the disclosed technology.
- the method 2200 includes, at operation 2210, generating a final prediction block for a conversion between a current block of visual media data and a bitstream representation of the current block.
- the method 2200 includes, at operation 2220, performing the conversion based on the final prediction block. At least a portion of the final prediction block is generated based on a combination of a first prediction block and a second prediction block that are based on reconstructed samples from an image segment that comprises the current block.
- the conversion comprises encoding the current block to generate the bitstream representation.
- the conversion comprises decoding the bitstream representation to generate the current block.
- the first prediction block is generated according to an intra prediction mode.
- the second prediction block is generated based on a motion vector pointing to the image segment.
- the second prediction block is generated based on an intra block copy technology in which the reconstructed samples pointed by the motion vector are copied.
- the second prediction block is generated based on an intra block copy technology in which motion compensation is applied to the reconstructed samples pointed by the motion vector before applying an in-loop filtering operation.
- the reconstructed samples are generated before an in-loop filtering operation is applied.
- a remaining portion of the final prediction block is copied directly from the first or second prediction block.
- the combination of the first and the second prediction blocks comprises a linear function of the first and second prediction blocks.
- the linear function comprises an average of the first and second prediction blocks.
- the linear function includes a weighted linear function of the first and the second prediction blocks. In some embodiments, different weights are applied for different positions relative to the first or the second prediction block. In some embodiments, different weights are applied for different intra prediction modes.
- weights of the weighted linear function are signaled in a sequence parameter set (SPS), a picture parameter set (PPS), a video parameter set (VPS), a slice header, a tile header, a group of coding tree units (CTUs), a coding unit (CU), a prediction unit (PU) or a transform unit (TU).
- SPS sequence parameter set
- PPS picture parameter set
- VPS video parameter set
- CTUs coding tree units
- CU coding unit
- PU prediction unit
- TU transform unit
- weights of the weighted linear function are predefined.
- the image segment includes a picture, a slice or a tile.
- the first prediction block and the second prediction block are generated for a selected color component of the current block.
- the selected color component includes a luma color component.
- the first prediction block and the second prediction block are generated upon determining that a size or a shape of the current block satisfies a predefined criterion.
- FIG. 23 shows a flowchart of an example method 2300 for video processing in accordance with the disclosed technology.
- the method 2300 includes, at operation 2310, maintaining a first table of motion candidates during a conversion between a current block of visual media data and a bitstream representation of the current block.
- the method 2300 includes, at operation 2320, determining, based on at least the first table of motion candidates, motion information for the current block that is coded using an intra block copy (IBC) mode in which at least one motion vector is directed at an image segment that comprises the current block.
- IBC intra block copy
- the method 2300 also includes, at operation 2330, performing the conversion based on the motion information.
- the conversion includes encoding the current block to generate the bitstream representation.
- the conversion includes decoding the bitstream representation to generate the current block.
- the method includes selectively updating, based on a rule, the first table of motion candidates using the motion information for the conversion.
- the rule specifies adding the motion information to the first table of motion candidates.
- the rule specifies excluding the motion information from the first table of motion candidates.
- the method includes updating, based on the motion
- the method includes performing a second conversion between a second block of visual media data and the bitstream representation of the second block using the updated first table of motion candidates.
- the second block can be coded using the intra block copy mode.
- the first table of motion candidates includes only motion information from blocks that are coded using the intra block copy mode.
- the first table of motion candidates is only used for processing blocks that are coded using the intra block copy mode.
- the first table of motion candidates includes motion information from blocks that are coded using the intra block copy mode and excludes motion information from blocks that are coded using other techniques.
- the first table of motion candidates includes motion information from blocks that are coded using the intra block copy mode and motion information from blocks that are coded using other techniques.
- the second conversion is performed using motion candidates associated with IBC coded blocks in the updated first table of motion candidates.
- the method includes maintaining, for a third conversion between a third block of visual media data and the bitstream representation of the third block, a second table of motion candidates, performing the third conversion between the third block and the bitstream representation of the third block based on the second table of motion candidates.
- the third conversion is performed without using the updated first table of motion candidates.
- the second table includes motion information from blocks that are encoded using other techniques that are different than the IBC mode.
- the third block is coded using a technique that is different than the IBC mode.
- the technique includes an inter mode.
- the method includes updating the second table of motion candidates using motion information for the third conversion.
- the method includes performing a fourth conversion between an IBC coded block based on the updated first table and performing a fifth conversion between an inter coded block based on the updated second table.
- the method includes comparing a motion candidate associated with the current block with a number of entries in the first table.
- the first table is updated based on the comparing.
- the number of entries corresponds to all entries in the first tables.
- the number of entries is m, m being an integer, and the m entries are last m entries in the first table.
- updating the first table comprises adding the motion candidate to the first table.
- the first table includes no duplicated entries.
- the method includes determining that the current block is further coded using an IBC merge mode or an IBC Advanced Motion Vector Prediction (AMVP) mode.
- the method includes determining a list of motion candidates for the current block by combining the first table of motion candidates and a set of adjacent or non- adjacent candidates of the current block and performing the conversion based on the list.
- the combining comprises checking m candidates from the set of adjacent or non- adjacent candidates, and checking n motion candidates from the first table of motion candidates, wherein m and n are positive integers.
- the method includes adding at least one of the m candidates from the set of adjacent or non-adjacent candidates to the list of motion candidates upon determining that the list is not full. In some embodiments, the method includes adding at least one of the n motion candidates from the first table of motion candidates to the list of motion candidates upon determining that the list is not full.
- the combining comprises checking candidates from the set of adjacent or non-adjacent candidates and the first table of motion candidates in an interleaved manner.
- the method includes checking a candidate from the set of adjacent or non-adjacent candidates prior to checking a motion candidate from the first table.
- the checking comprises checking a candidate from the set of adjacent or non-adjacent candidates and replacing, based on a coding characteristic of an adjacent or non- adjacent block associated with the adjacent or non-adjacent candidate, the candidate with a motion candidate from the first table of motion candidates.
- the coding characteristic of the adjacent or non-adjacent block indicates that the adjacent or non-adjacent block is located outside a predefined range. In some embodiments, the coding characteristic of the adjacent or non-adjacent block indicates that the adjacent or non-adjacent block is intra coded. In some embodiments, adding a motion candidate further comprising adding the motion candidate to the list upon determining that the motion candidate from the first table of motion candidates is different from at least one candidate from the set of adjacent or non-adjacent candidates. In some embodiments, the set of adjacent or non-adjacent candidate has a higher priority than the first table of motion candidates.
- the list of motion candidates comprises multiple merge candidates used for IBC merge-coded blocks. In some embodiments, the list of motion candidates comprises multiple AMVP candidates used for IBC AMVP-coded blocks. In some embodiments, the list of motion candidates comprises intra prediction modes used for intra- coded blocks. In some embodiments, a size of the first table is pre-defined or signaled in the bitstream representation. In some embodiments, the first table is associated with a counter that indicates a number of available motion candidates in the first table.
- the method includes resetting the first table before processing a new slice, a new LCU row, or a new tile.
- the counter is set to zero before processing a new slice, a new LCU row, or a new tile.
- FIG. 26 shows a flowchart of an example method 2600 for video processing in accordance with the disclosed technology.
- the method 2600 includes, at operation 2610, determining, for a conversion between a current block of video data and a bitstream
- the method 2600 also includes, at operation 2620, performing, based on the first coding mode, the conversion between the bitstream representation and the current block.
- the conversion generates the current block from the bitstream representation. In some embodiments, the conversion generates the bitstream representation from the current block.
- the first coding mode is derived from a table of motion candidates or a non-adjacent chroma block. In some embodiments, the first coding mode is derived from a luma block that is different than a corresponding luma block of the chroma component. In some embodiments, the first coding mode is derived from a luma block non- adjacent to a corresponding luma block of the chroma component. In some embodiments, the first coding mode is determined to be a first default coding mode.
- the method includes adding the first default coding mode to a chroma mode candidate list of the current block. In some embodiments, the method further includes determining a second default coding mode that includes one of a DC mode, a planar mode, a vertical mode or a horizontal mode. In some embodiments, the second default coding mode is added to a chroma mode list upon determining that the chroma mode list is not full after checking the first default mode.
- a motion candidate in the table of motion candidates corresponds to an intra prediction mode of a previously coded block.
- the method includes updating the table of motion candidates after the conversion.
- the method includes performing a second conversion between a subsequent block of video data and a bitstream representation of the subsequent block based on the updated table.
- FIG. 27 shows a flowchart of an example method 2700 for video processing in accordance with the disclosed technology.
- the method 2700 includes, at operation 2710, determining, for a conversion between a bitstream representation of a chroma block and the chroma block, an intra coding mode based on a spatial neighboring block of a luma block that corresponds to the chroma block.
- the method 2700 also includes, at operation 2720, performing, based on the intra coding mode, a conversion between the bitstream representation and the chroma block.
- the conversion generates the chroma block from the bitstream representation. In some embodiments, the conversion generates the bitstream representation from the chroma block. In some embodiments, the method includes adding the intra coding mode to a chroma intra mode list of the chroma block.
- the spatial neighboring block includes an above neighboring block relative to a top-left position of the chroma block. In some embodiments, the spatial neighboring block includes a left neighboring block relative to a top-left position of the chroma block. In some embodiments, the spatial neighboring block includes an above neighboring block relative to a top-right position of the chroma block. In some embodiments, the spatial neighboring block includes a left neighboring block relative to a bottom-left position of the chroma block. In some embodiments, the spatial neighboring block includes a block adjacent to a center position of a first row of the chroma block. In some embodiments, the spatial neighboring block is a block adjacent to a center position of a first column of the chroma block.
- Implementations of the subject matter and the functional operations described in this patent document can be implemented in various systems, digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them.
- Implementations of the subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible and non-transitory computer readable medium for execution by, or to control the operation of, data processing apparatus.
- the computer readable medium can be a machine- readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them.
- data processing unit or“data processing apparatus” encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers.
- the apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.
- a computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.
- a computer program does not necessarily correspond to a file in a file system.
- a program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code).
- a computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a
- processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer.
- a processor will receive instructions and data from a read only memory or a random access memory or both.
- the essential elements of a computer are a processor for performing instructions and one or more memory devices for storing instructions and data.
- a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks.
- mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks.
- a computer need not have such devices.
- Computer readable media suitable for storing computer program instructions and data include all forms of nonvolatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices.
- semiconductor memory devices e.g., EPROM, EEPROM, and flash memory devices.
- the processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
Devices, systems and methods for digital video coding, which includes chroma mode intra coding, are described. In a representative aspect, a method for video processing includes determining, for a conversion between a current block of video data and a bitstream representation of the current block, a first coding mode for a chroma component of the current block based on at least one or more coding modes of a block that is not adjacent to the current block. The method further includes performing, based on the first coding mode, the conversion between the bitstream representation and the current block.
Description
CHROMA MODE IM RA CODING
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] Under the applicable patent law and/or rules pursuant to the Paris Convention, this application is made to timely claim the priority to and benefit of International Patent Application No. PCT/CN2018/107180, filed on September 24, 2018. For all purposes under the U.S. law, the entire contents of the before-mentioned patent application are incorporated by reference as part of the disclosure of this patent document.
TECHNICAL FIELD
[0002] This patent document relates to video coding techniques, devices and systems.
BACKGROUND
[0003] In spite of the advances in video compression, digital video still accounts for the largest bandwidth use on the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, it is expected that the bandwidth demand for digital video usage will continue to grow.
SUMMARY
[0004] Devices, systems and methods related to digital video coding, and specifically, intra mode coding and chroma mode coding for images and video based of past (e.g., historical or statistical) information are described. The described methods may be applied to both the existing video coding standards (e.g., High Efficiency Video Coding (HEVC)) and future video coding standards (e.g., Versatile Video Coding (WC)) or codecs.
[0005] In one representative aspect, the disclosed technology may be used to provide a method for video processing. This example method includes selecting, for a conversion between a current block of visual media data and a bitstream representation of the current block, a first intra prediction mode based on at least a first set of history intra coding information that includes statistical information of a set of intra prediction modes, and performing the conversion based on the first intra prediction mode.
[0006] In another representative aspect, the disclosed technology may be used to provide a method for video processing. This example method includes, selecting, for a conversion between
a current block of visual media data and a bitstream representation of the current block, a first intra prediction mode based on at least intra prediction modes associated with non-adjacent neighbors of the current block. The method also includes performing the conversion based on the first intra prediction mode.
[0007] In another representative aspect, the disclosed technology may be used to provide a method for video processing. This example method includes selecting, for a conversion between a current block of visual media data and a bitstream representation of the current block, an intra prediction mode based on at least one of spatial neighboring blocks to the current block, and performing the conversion based on the intra prediction mode. The at least one of the spatial neighboring blocks is different from a first block that is located to a left of a first row of the current block and a second block that is located above a first column of the current block.
[0008] In another representative aspect, the disclosed technology may be used to provide a method for video processing. This example method includes generating a final prediction block for a conversion between a current block of visual media data and a bitstream representation of the current block and performing the conversion based on the final prediction block. At least a portion of the final prediction block is generated based on a combination of a first prediction block and a second prediction block that are based on reconstructed samples from an image segment that comprises the current block.
[0009] In another representative aspect, the disclosed technology may be used to provide a method for video processing. This example method includes maintaining a first table of motion candidates during a conversion between a current block of visual media data and a bitstream representation of the current block and determining, based on at least the first table of motion candidates, motion information for the current block that is coded using an intra block copy (IBC) mode in which at least one motion vector is directed at an image segment that comprises the current block. The method also includes performing the conversion based on the motion information.
[0010] In one representative aspect, the disclosed technology may be used to provide a method for video processing. This example method for video processing includes determining, for a conversion between a current block of video data and a bitstream representation of the current block, a first coding mode for a chroma component of the current block based on at least one or more coding modes of a block that is not adjacent to the current block. The method also
includes performing, based on the first coding mode, the conversion between the bitstream representation and the current block.
[0011] In another representative aspect, the disclosed technology may be used to provide a method for video processing. This example method for video processing includes determining, for a conversion between a bitstream representation of a chroma block and the chroma block, an intra coding mode based on a spatial neighboring block of a luma block that corresponds to the chroma block. The method also includes performing, based on the intra coding mode, a conversion between the bitstream representation and the chroma block.
[0012] In another representative aspect, the disclosed technology may be used to provide a method for video coding. This example method for video coding includes selecting, for a bitstream representation of a current block of visual media data, a first intra prediction mode from a set of intra prediction modes based on a first set of past intra coding information, and processing, based on the first intra prediction mode, the bitstream representation to generate the current block.
[0013] In another representative aspect, the disclosed technology may be used to provide a method for video coding. This example method for video coding includes selecting, for a bitstream representation of a current block of visual media data, an intra prediction mode from at least one of adjacent or non-adjacent spatial neighboring blocks to the current block, and processing, based on the intra prediction mode, the bitstream representation to generate the current block.
[0014] In yet another representative aspect, the disclosed technology may be used to provide a method for video coding. This example method for video coding includes generating, for a bitstream representation of a current block of visual media data, a first prediction block and a second prediction block that are based on reconstructed samples from an image segment that comprises the current block, generating at least a portion of a final prediction block based on a linear function of the first and second prediction blocks, and processing, based on the final prediction block, the bitstream representation to generate the current block.
[0015] In another representative aspect, the disclosed technology may be used to provide a method for video coding. This example method for video coding includes selecting, for a bitstream representation of a current block that comprises a luma component and a chroma component, a first coding mode for the chroma component, and performing, based on the first
coding mode, a conversion between the bitstream representation and the current block.
[0016] In another representative aspect, the disclosed technology may be used to provide a method for video coding. This example method for video coding includes selecting, for a bitstream representation of a current block that comprises a luma component and a chroma component, an intra coding mode from a spatial neighboring block to the luma component, adding the intra coding mode to a chroma intra mode list, and performing, based on the intra coding mode, a conversion between the bitstream representation and the current block.
[0017] In yet another representative aspect, the above-described method is embodied in the form of processor-executable code and stored in a computer-readable program medium.
[0018] In yet another representative aspect, a device that is configured or operable to perform the above-described method is disclosed. The device may include a processor that is
programmed to implement this method.
[0019] In yet another representative aspect, a video decoder apparatus may implement a method as described herein.
[0020] The above and other aspects and features of the disclosed technology are described in greater detail in the drawings, the description and the claims.
BRIEF DESCRIPTION OF THE DRAWINGS
[0021] FIG. 1 shows an example of 67 intra prediction modes.
[0022] FIG. 2 shows an example of neighboring blocks used for intra mode prediction.
[0023] FIGS. 3A and 3B show examples of reference samples for wide-angle intra prediction modes for non-square blocks.
[0024] FIG. 4 shows an example of a discontinuity when using wide-angle intra prediction.
[0025] FIGS. 5A-5D show examples of samples used by a position-dependent intra prediction combination (PDPC) method.
[0026] FIG. 6 shows an example of locations of samples used for the derivation of the weights of the linear model.
[0027] FIG. 7 shows an example of classifying neighboring samples into two groups.
[0028] FIG. 8A shows an example of a chroma sample and its corresponding luma samples.
[0029] FIG. 8B shows an example of down filtering for the cross-component linear model
(CCFM) in the Joint Exploration Model (JEM).
[0030] FIGS. 9 A and 9B show examples of a chroma block and its corresponding luma block.
[0031] FIG. 10 shows an example of intra prediction modes with default modes.
[0032] FIGS. 11 A and 11B show examples of corresponding sub-blocks for a chroma CB.
[0033] FIG. 12 shows an example of an intra-picture block copy.
[0034] FIGS. 13A-13F show examples of different adjacent blocks that may be selected for intra mode coding.
[0035] FIG. 14 shows a flowchart of an example method for intra-mode coding based on past information in accordance with the disclosed technology.
[0036] FIG. 15 shows a flowchart of another example method for intra-mode coding based on past information in accordance with the disclosed technology.
[0037] FIG. 16 shows a flowchart of yet another example method for intra- mode coding based on past information in accordance with the disclosed technology.
[0038] FIG. 17 is a block diagram of an example of a hardware platform for implementing a visual media decoding or a visual media encoding technique described in the present document.
[0039] FIG. 18 is a block diagram of an example video processing system in which disclosed techniques may be implemented.
[0040] FIG. 19 shows a flowchart of an example method for video processing in accordance with the disclosed technology.
[0041] FIG. 20 shows a flowchart of another example method for video processing in accordance with the disclosed technology.
[0042] FIG. 21 shows a flowchart of another example method for video processing in accordance with the disclosed technology.
[0043] FIG. 22 shows a flowchart of another example method for video processing in accordance with the disclosed technology.
[0044] FIG. 23 shows a flowchart of another example method for video processing in accordance with the disclosed technology.
[0045] FIG. 24 shows a flowchart of an example method for chroma mode coding based on past information.
[0046] FIG. 25 shows a flowchart of another example method for chroma mode coding based on past information.
[0047] FIG. 26 shows a flowchart of an example method for video processing in accordance
with the disclosed technology.
[0048] FIG. 27 shows a flowchart of yet another example method for video processing in accordance with the disclosed technology.
DETAILED DESCRIPTION
[0049] Due to the increasing demand of higher resolution video, video coding methods and techniques are ubiquitous in modern technology. Video codecs typically include an electronic circuit or software that compresses or decompresses digital video, and are continually being improved to provide higher coding efficiency. A video codec converts uncompressed video to a compressed format or vice versa. There are complex relationships between the video quality, the amount of data used to represent the video (determined by the bit rate), the complexity of the encoding and decoding algorithms, sensitivity to data losses and errors, ease of editing, random access, and end-to-end delay (latency). The compressed format usually conforms to a standard video compression specification, e.g., the High Efficiency Video Coding (HEVC) standard (also known as H.265 or MPEG-H Part 2), the Versatile Video Coding (VVC) standard to be finalized, or other current and/or future video coding standards.
[0050] Embodiments of the disclosed technology may be applied to existing video coding standards (e.g., HEVC, H.265) and future standards to improve runtime performance. Section headings are used in the present document to improve readability of the description and do not in any way limit the discussion or the embodiments (and/or implementations) to the respective sections only.
1 Examples of intra prediction in WC
1.1 Intra mode coding with 67 intra prediction modes
[0051] To capture the arbitrary edge directions presented in natural video, the number of directional intra modes is extended from 33, as used in HEVC, to 65. The additional directional modes are depicted as red dotted arrows in FIG. 1, and the planar and DC modes remain the same. These denser directional intra prediction modes apply for all block sizes and for both luma and chroma intra predictions.
[0052] Conventional angular intra prediction directions are defined from 45 degrees to -135 degrees in clockwise direction as shown in FIG. 1. In VTM2, several conventional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes for the non-
square blocks. The replaced modes are signaled using the original method and remapped to the indexes of wide angular modes after parsing. The total number of intra prediction modes is unchanged (e.g., 67), and the intra mode coding is unchanged.
[0053] In HEVC, every intra-coded block has a square shape and the length of each of its side is a power of 2. Thus, no division operations are required to generate an intra-predictor using DC mode. In VTV2, blocks can have a rectangular shape that necessitates the use of a division operation per block in the general case. To avoid division operations for DC prediction, only the longer side is used to compute the average for non-square blocks.
1.2 Examples of intra mode coding
[0054] In some embodiments, and to keep the complexity of the MPM list generation low, an intra mode coding method with 3 Most Probable Modes (MPMs) is used. The following three aspects are considered to the MPM lists:
[0055] — Neighbor intra modes;
[0056] — Derived intra modes; and
[0057] — Default intra modes.
[0058] For neighbor intra modes (A and B), two neighbouring blocks, located in left and above are considered. The left and above blocks are those who are connected to the top-left sample of current block, as shown in FIG. 2. An initial MPM list is formed by performing pruning process for two neighboring intra modes. The pruning process is used to remove duplicated modes so that only unique modes can be included into the MPM list. If two neighboring modes are different each other, one of the default modes (e.g., PLANA (0), DC (1), ANGULAR50 (e.g., 50)) is added to the MPM list after the pruning check with the existing two MPMs. When the two neighboring modes are the same, either the default modes or the derived modes are added to the MPM list after the pruning check. The detailed generation process of three MPM list is derived as follows:
[0059] If two neighboring candidate modes (i.e., A == B) are same,
[0060] If A is less than 2, candModeList [3] = { 0, 1, 50}.
[0061] Otherwise, candModeList[ 0 ] = { A, 2 + ( ( A + 6l ) % 64 ),
2 + ( ( A - 1 ) % 64 ) }
[0062] Otherwise,
[0063] If neither of A and B is equal to 0, candModeList [3] = { A, B, 0}.
[0064] Otherwise, if neither of A and B is equal to 1, candModeList [3] = { A, B, 1 }.
[0065] Otherwise, candModeList [3] = { A, B, 50}.
[0066] An additional pruning process is used to remove duplicated modes so that only unique modes can be included into the MPM list. For entropy coding of the 64 non-MPM modes, a 6-bit Fixed Length Code (FLC) is used.
1.3 Wide-angle intra prediction for non-square blocks
[0067] In some embodiments, conventional angular intra prediction directions are defined from 45 degrees to -135 degrees in clockwise direction. In VTM2, several conventional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes for non square blocks. The replaced modes are signaled using the original method and remapped to the indexes of wide angular modes after parsing. The total number of intra prediction modes for a certain block is unchanged, e.g., 67, and the intra mode coding is unchanged.
[0068] To support these prediction directions, the top reference with length 2W+1, and the left reference with length 2H+1, are defined as shown in the examples in FIGS. 3 A and 3B.
[0069] In some embodiments, the mode number of replaced mode in wide-angular direction mode is dependent on the aspect ratio of a block. The replaced intra prediction modes are illustrated in Table 1.
Table 1 : Intra prediction modes replaced by wide-angle modes
[0070] As shown in FIG. 4, two vertically-adjacent predicted samples may use two non- adjacent reference samples in the case of wide-angle intra prediction. Hence, low-pass reference samples filter and side smoothing are applied to the wide-angle prediction to reduce the negative effect of the increased gap Dra.
1.4 Examples of position dependent intra prediction combination (PDPC)
[0071] In the VTM2, the results of intra prediction of planar mode are further modified by a position dependent intra prediction combination (PDPC) method. PDPC is an intra prediction
method which invokes a combination of the un-filter ed boundary reference samples and HEVC style intra prediction with filtered boundary reference samples. PDPC is applied to the following intra modes without signaling: planar, DC, horizontal, vertical, bottom-left angular mode and its eight adjacent angular modes, and top-right angular mode and its eight adjacent angular modes.
[0072] The prediction sample pred(x,y ) is predicted using an intra prediction mode (DC, planar, angular) and a linear combination of reference samples according to the Equation as follows:
[0073] pred(x,y)=(wL R-i,y + wTxRx,- 1 - wTL xi?-i,-i+(64 - wL - wT+wTL)xpred(x,y )
+ 32 ) » shift
[0074] Herein, Rx,- I, R-\,y represent the reference samples located at the top and left of current sample (x, y), respectively, and R- represents the reference sample located at the top-left corner of the current block.
[0075] In some embodiments, and if PDPC is applied to DC, planar, horizontal, and vertical intra modes, additional boundary filters are not needed, as required in the case of HEVC DC mode boundary filter or horizontal/vertical mode edge filters.
[0076] FIGS. 5A-5D illustrate the definition of reference samples (Rx,- 1, R-\,y and R-i -i) for PDPC applied over various prediction modes. The prediction sample pred (x’, y’) is located at (x’, y’) within the prediction block. The coordinate x of the reference sample Rx,- 1 is given by: x = x’
+ y’ + 1 , and the coordinate y of the reference sample R-i,y is similarly given by: y = x’ +y’ + 1 [0077] In some embodiments, the PDPC weights are dependent on prediction modes and are shown in Table 2, where S = shift.
Table 2: Examples of PDPC weights according to prediction modes
1.5 Examples of chroma coding
[0078] In HEVC chroma coding, five modes (including one direct mode (DM) which is the intra prediction mode from the top-left corresponding luma block and four default modes) are allowed for a chroma block. The two color components share the same intra prediction mode.
[0079] In some embodiments, and different from the design in HEVC, two new methods have been proposed, including: cross-component linear model (CCLM) prediction mode and multiple DMs.
1.5.1 Examples of the cross-component linear model (CCLM)
[0080] In some embodiments, and to reduce the cross-component redundancy, a cross component linear model (CCLM) prediction mode (also referred to as LM), is used in the JEM, for which the chroma samples are predicted based on the reconstructed luma samples of the same CU by using a linear model as follows:
[0081] predc(i,j ) = a recL' i,j) + b (1)
[0082] Here, predc{i,j ) represents the predicted chroma samples in a CU and recL' i,j ) represents the downsampled reconstructed luma samples of the same CU for color formats 4:2:0 or 4:2:2 while recL' i,j ) represents the reconstructed luma samples of the same CU for color format 4:4:4. CCLM parameters a and b are derived by minimizing the regression error between the neighboring reconstructed luma and chroma samples around the current block as follows:
[0085] Here, L(n ) represents the down-sampled (for color formats 4:2:0 or 4:2:2) or original
(for color format 4:4:4) top and left neighboring reconstructed luma samples, C(n ) represents the top and left neighboring reconstructed chroma samples, and value of N is equal to twice of the minimum of width and height of the current chroma coding block.
[0086] In some embodiments, and for a coding block with a square shape, the above two equations are applied directly. In other embodiments, and for a non-square coding block, the neighboring samples of the longer boundary are first subsampled to have the same number of samples as for the shorter boundary. FIG. 6 shows the location of the left and above
reconstructed samples and the sample of the current block involved in the CCLM mode.
[0087] In some embodiments, this regression error minimization computation is performed as part of the decoding process, not just as an encoder search operation, so no syntax is used to convey the a and b values.
[0088] In some embodiments, the CCLM prediction mode also includes prediction between the two chroma components, e.g., the Cr (red-difference) component is predicted from the Cb
(blue- difference) component. Instead of using the reconstructed sample signal, the CCLM Cb-to- Cr prediction is applied in residual domain. This is implemented by adding a weighted reconstructed Cb residual to the original Cr intra prediction to form the final Cr prediction:
[0089] predC * r(i,j ) = predCr(i,j ) + a resicb'(i,j ) (4)
[0090] Here, resicb'(i,j ) presents the reconstructed Cb residue sample at position (i,j).
[0091] In some embodiments, the scaling factor a may be derived in a similar way as in the CCLM luma-to-chroma prediction. The only difference is an addition of a regression cost relative to a default a value in the error function so that the derived scaling factor is biased towards a default value of -0.5 as follows:
[0093] Here, Cb(n ) represents the neighboring reconstructed Cb samples, Cr(n ) represents the neighboring reconstructed Cr samples, and l is equal to
· Cb(n)) » 9.
[0094] In some embodiments, the CCLM luma-to-chroma prediction mode is added as one additional chroma intra prediction mode. At the encoder side, one more RD cost check for the chroma components is added for selecting the chroma intra prediction mode. When intra prediction modes other than the CCLM luma-to-chroma prediction mode is used for the chroma components of a CU, CCLM Cb-to-Cr prediction is used for Cr component prediction.
1.5.2 Examples of a multiple model CCLM
[0095] In the JEM, there are two CCLM modes: the single model CCLM mode and the multiple model CCLM mode (MMLM). As indicated by the name, the single model CCLM mode employs one linear model for predicting the chroma samples from the luma samples for the whole CU, while in MMLM, there can be two models.
[0096] In MMLM, neighboring luma samples and neighboring chroma samples of the current block are classified into two groups, each group is used as a training set to derive a linear model (i.e., a particular a and b are derived for a particular group). Furthermore, the samples of the current luma block are also classified based on the same rule for the classification of neighboring luma samples.
[0097] FIG. 7 shows an example of classifying the neighboring samples into two groups. Threshold is calculated as the average value of the neighboring reconstructed luma samples. A neighboring sample with Rec'i\x,y] <= Threshold is classified into group 1; while a neighboring
sample with Rec'i\x,y\ > Threshold is classified into group 2.
[0099] FIGS. 8 A and 8B show an example of a chroma sample and its corresponding luma samples, and an example of of down filtering for the cross-component linear model (CCLM) in the Joint Exploration Model (JEM), respectively.
1.5.3 Examples of chroma coding in WC
[00100] With regard to existing implementations, CCLM as in JEM is adopted in VTM-2.0, whereas MM-CCLM in JEM has not been adopted in VTM-2.0.
[00101] In some embodiments, for chroma intra coding, a candidate list of chroma intra prediction modes may be derived first, and wherein three parts may be included:
[00102] (1) One direct mode (DM) which is set to the one intra luma prediction mode associated with luma CB covering the co-located top-left position of a chroma block. An example is shown in FIGS. 9 A and 9B with the co-located position denoted as“TL”
[00103] (2) One cross-component linear model (CCLM) mode
[00104] (3) Four default modes (DC, Planar, Horizontal, and Vertical modes, as highlighted in FIG. 1). In an example, if one of the four default modes is identical to the DM mode, it is replaced by the intra prediction mode with largest mode index, e.g., indicated by the dashed line depicted in FIG. 10.
1.6 Examples of syntax, semantics and decoding in WC2
[00105] In some embodiments, the syntax and semantics of intra mode related information may be specified as follows:
Table 3: Example of a coding unit (CU) syntax
[00106] Examples of coding unit semantics. In some embodiments, the syntax elements intra luma mpm flag[ xO ][ yO ], intra_luma_mpm_idx[ x0 ][ yO ] and
intra_luma_mpm_remainder[ x0 ][ yO ] specify the intra prediction mode for luma samples. The array indices xO, yO specify the location ( xO , yO ) of the top-left luma sample of the considered prediction block relative to the top-left luma sample of the picture. When
intra_luma_mpm_flag[ xO ][ yO ] is equal to 1, the intra prediction mode is inferred from a neighbouring intra-predicted prediction unit according to the specification (e.g., clause 8.2.2).
[00107] In some embodiments, intra_chroma_pred_mode[ xO ][ yO ] specifies the intra prediction mode for chroma samples. The array indices xO, yO specify the location ( xO, yO ) of the top-left luma sample of the considered prediction block relative to the top-left luma sample of the picture.
[00108] Examples of deriving the luma intra prediction mode. In some embodiments, an input to this process is a luma location ( xPb , yPb ) specifying the top-left sample of the current luma prediction block relative to the top-left luma sample of the current picture. In this process, the luma intra prediction mode IntraPredModeY[ xPb ][ yPb ] is derived. Table 4 specifies the value for the intra prediction mode and the associated names.
Table 4: Example specification of intra prediction modes and associated names
[00109] In the context of Table 4, the intra prediciton mode INTRA CCLM is only applicable to chroma components, and IntraPredModeY[ xPb ][ yPb ] labelled 0..66 represents directions of predictions as shown in FIG. 1. In some embodiments, IntraPredModeY[ xPb ][ yPb ] is derived by the following ordered steps:
1. The neighbouring locations ( xNbA, yNbA ) and ( xNbB, yNbB ) are set equal to
( xPb - 1, yPb ) and ( xPb, yPb - 1 ), respectively.
2. For X being replaced by either A or B, the variables candlntraPredModeX are derived as follows:
- The availability derivation process for a block is invoked with the location
( xCurr, yCurr ) set equal to ( xPb, yPb ) and the neighbouring location
( xNbY, yNbY ) set equal to ( xNbX, yNbX ) as inputs, and the output is assigned to availableX.
- The candidate intra prediction mode candlntraPredModeX is derived as follows:
- If one or more of the following conditions are true, candlntraPredModeX is set equal to INTRA DC.
- The variable availableX is equal to FALSE.
- CuPredMode[ xNbX ][ yNbX ] is not equal to MODE_INTRA.
- X is equal to B and yPb - 1 is less than
( ( yPb » CtbLog2SizeY ) « CtbLog2SizeY ).
- Otherwise, candlntraPredModeX is set equal to
IntraPredModeY[ xNbX ][ yNbX ].
3. The candModeList[ x ] with x = 0..2 is derived as follows:
- If candlntraPredModeB is equal to candlntraPredModeA, the following applies:
- If candlntraPredModeA is less than 2 (i.e., equal to INTRA PLANAR or INTRA_DC), candModeList[ x ] with x = 0..2 is derived as follows:
candModeList[ 0 ] = INTRA PLANAR (8-1) candModeList[ 1 ] = INTRA DC (8-2) candModeList[ 2 ] = INTRA ANGULAR50 (8-3)
- Otherwise, candModeList[ x ] with x = 0..2 is derived as follows:
candModeList[ 0 ] = candlntraPredModeA (8-4) candModeList[ 1 ] = 2 + ( ( candlntraPredModeA + 61 ) % 64 ) (8-5) candModeList[ 2 ] = 2 + ( ( candlntraPredModeA - 1 ) % 64 ) (8-6)
- Otherwise (candlntraPredModeB is not equal to candlntraPredModeA), the
following applies:
- candModeList[ 0 ] and candModeList[ 1 ] are derived as follows:
candModeList[ 0 ] = candlntraPredModeA (8-7) candModeList[ 1 ] = candlntraPredModeB (8-8)
- If neither of candModeList[ 0 ] and candModeList[ 1 ] is equal to
INTRA PLANAR, candModeList[ 2 ] is set equal to INTRA PLANAR,
- Otherwise, if neither of candModeList[ 0 ] and candModeList[ 1 ] is equal to
INTRA DC, candModeList[ 2 ] is set equal to INTRA DC,
- Otherwise, candModeList[ 2 ] is set equal to INTRA_ANGULAR50.
ntraPredModeY[ xPb ][ yPb ] is derived by applying the following procedure:
- If intra_luma_mpm_flag[ xPb ][ yPb ] is equal to 1, the
IntraPredModeY[ xPb ][ yPb ] is set equal to
candModeList[ intra_luma_mpm_idx[ xPb ][ yPb ] ].
- Otherwise, IntraPredModeY[ xPb ][ yPb ] is derived by applying the following
ordered steps:
1. The array candModeList[ x ], x = 0..2 is modified by the following ordered steps: i. When candModeList[ 0 ] is greater than candModeList[ 1 ], both values are swapped as follows:
( candModeList[ 0 ], candModeList[ 1 ] ) = Swap( candModeList[ 0 ], candModeList[ 1 ] ) (8-9) ii. When candModeList[ 0 ] is greater than candModeList[ 2 ], both values are swapped as follows:
( candModeList[ 0 ], candModeList[ 2 ] ) = Swap( candModeList[ 0 ], candModeList[ 2 ] ) (8-10) iii. When candModeList[ 1 ] is greater than candModeList[ 2 ], both values are swapped as follows:
( candModeList[ 1 ], candModeList[ 2 ] ) = Swap( candModeList[ 1 ], candModeList[ 2 ] ) (8-11)
2. IntraPredModeY[ xPb ][ yPb ] is derived by the following ordered steps:
i. IntraPredModeY[ xPb ][ yPb ] is set equal to
intra_luma_mpm_remainder[ xPb ][ yPb ].
ii. For i equal to 0 to 2, inclusive, when IntraPredModeY[ xPb ][ yPb ] is
greater than or equal to candModeList[ i ], the value of
IntraPredModeY[ xPb ] [ yPb ] is incremented by one.
[00110] The variable IntraPredModeY[ x ][ y ] with x = xPb..xPb + cbWidth - 1 and y = yPb..yPb + cbHeight - 1 is set to be equal to IntraPredModeY[ xPb ][ yPb ].
[00111] Examples of deriving the chroma intra prediction mode. In some embodiments, an input to this process is a luma location ( xPb, yPb ) specifying the top-left sample of the current chroma prediction block relative to the top-left luma sample of the current picture. In this process, the chroma intra prediction mode IntraPredModeC[ xPb ][ yPb ] is derived.
[00112] In some embodiments, the chroma intra prediction mode
IntraPredModeC[ xPb ][ yPb ] is derived using intra_chroma_pred_mode[ xPb ][ yPb ] and
IntraPredModeY[ xPb ][ yPb ] as specified in Table 5 and Table 6.
1.7 Examples of multiple direct modes (DMs)
[00113] An existing implementation (e.g., JVET-D0111) was recently adopted by JEM. Due to the separate block partitioning structure for luma and chroma components in I slices, one chroma CB may correspond to several luma CBs. Therefore, to further improve the coding efficiency, a Multiple Direct Modes (MDM) method for chroma intra coding is proposed. In
MDM, the list of chroma mode candidates includes the following three parts:
[00114] (1) the CCLM mode
[00115] (2) Intra prediction modes derived from luma CBs covering the co-located five positions of the current chroma block: the five positions to be checked in order are defined as: center, top-left, top-right, bottom-left and bottom-right 4x4 block within the corresponding luma block of current chroma block for I slices. For P and B slices, there is no need to check these five sub-blocks since they shall be within one CB. An example of five co-located luma positions is shown in FIGS. 11 A and 11B, they are denoted by CR, TL, TR, BL and BR, respectively.
[00116] (3) Chroma prediction modes from spatial neighboring blocks to unify the luma and chroma mode derivation process (with removals and additions notated):
[00117] o 5 chroma prediction modes from left, above, below-left, above-right, and above-left spatial neighboring blocks of current block, and the locations of the five spatial blocks as in the merge mode (with Planar and DC modes being removed)
[00118] o With the planar and DC modes being added
[00119] o Derived modes are added, those intra modes are obtained by adding -1 or +1 to the angular modes which are already included into the list
[00120] o Default modes are added in the order of: Vertical (Mode 18), Horizontal
(Mode 50), Mode 2, Mode 34, Mode 66, Mode 10, Mode 26 (with Mode 66, Mode 10 and Mode 26 being added)
[00121] o The addition of the rule that if any of the four default modes (Planar,
Horizontal, Vertical and DC modes) is not included in the list, the missing default modes are used to replace the last one or more candidates.
[00122] In some embodiments, the pruning process is applied whenever a new chroma intra mode is added to the candidate list. In this contribution, the chroma intra mode candidate list size is always set to 10.
1.8 Examples of intra block copy
[00123] In some embodiments, HEVC screen content coding (SCC) extensions employ a new coding tool, Intra block copy (IBC), also named intra picture block compensation or current picture referencing (CPR), is a very efficient technique in terms of coding performance improvement. IBC is a block matching technique wherein a current prediction block is predicted from a reference block located in the already reconstructed regions of the same picture.
[00124] In IBC, a displacement vector (referred as block vector or BV, or motion vector) is used to signal the relative displacement from the position of the current block to that of the reference block. Further, the previously reconstructed reference block within the same picture is added to the prediction errors to form the reconstructed current block. In this technique, the reference samples correspond to the reconstructed samples of the current picture prior to in-loop filter operations. In HEVC, the in-loop filters refer to both deblocking and sample adaptive offset (SAO) filters. An example of the intra block compensation is shown in FIG. 12.
[00125] In some embodiments, the use of the IBC mode is signaled at both sequence and picture level. When the IBC mode is enabled at sequence parameter set (SPS), it can be enabled at picture level. When the IBC mode is enabled at picture level, the current reconstructed picture is treated as a reference picture. Therefore, no syntax change on block level is needed in HEVC SCC on top of the existing HEVC inter mode to signal the use of the IBC mode.
[00126] In some embodiments, the following is a list of features for the IBC mode in HEVC SCC:
[00127] — All prediction unit partition modes in the HEVC inter mode are supported in the
IBC mode, e.g., 2Nx2N or 2N/N or Nx2N or NxN and Asymmetrical Motion Partitioning (AMP) modes.
[00128] — Merge and skip modes are also available for the IBC mode. The merge candidate list construction is unified, containing merge candidates from the neighboring positions that are either coded in the IBC mode or the HEVC inter mode. Depending on the selected merge index, the current block under merge or skip mode can merge into either an IBC mode coded neighbor or otherwise an HEVC inter mode coded one.
[00129] — Block vector prediction and coding schemes for the IBC mode reuse the schemes used for motion vector prediction and coding in the HEVC inter mode (AMVP and MVD coding).
[00130] — Residue transform, quantization and coefficient coding are the same as in the
HEVC inter mode.
[00131] In addition to the features enumerated above, some embodiments comprise additional features (or treatments) for IBC modes, including:
[00132] — The motion vector for the IBC mode, also referred as block vector, is coded with integer-pel precision, but stored in memory in quarter-pel precision after decoding as
quarter-pel precision is required in interpolation and deblocking stages. When used in motion vector prediction for the IBC mode, the stored vector predictor will be right shifted by 2.
[00133] — A slice with the IBC mode enabled will be treated as a P or B slice, even though no other picture than the current picture itself is used as a reference picture.
[00134] — Weighted prediction is disabled for the IBC mode as there was no clear compression benefit observed.
2 Examples of drawbacks in existing implementations
[00135] The current intra mode coding implementations in either JEM or VTM exhibit at least the following issues:
[00136] • For luma coding, only prediction modes of adjacent spatial neighboring blocks are considered. The lack of statistic intra prediction information of previously coded blocks results in limited coding performance improvements.
[00137] • In an existing implementation, it is proposed to check more neighboring blocks to be added to MPM list for better coding of intra prediction modes. The complexity is also increased due to multiple times of access of adjacent spatial neighboring blocks.
[00138] • For chroma coding, the multiple DMs solution could bring additional coding gains at the cost of higher complexity since the neighboring chroma modes need to be stored which was not needed in HEVC.
[00139] • IBC could take care of repeated patterns which are far from the current block and intra coding mode could take care of reducing less spatial correlation. However, either of the two could only be beneficial for a certain case.
3 Example methods for intra mode coding based on past information
[00140] Embodiments of the presently disclosed technology overcome drawbacks of existing implementations, thereby providing video coding with higher coding efficiencies but lower computational complexity. Intra mode coding based on past information, and as described in the present document, may enhance both existing and future video coding standards, is elucidated in the following examples described for various implementations. The examples of the disclosed technology provided below explain general concepts, and are not meant to be interpreted as limiting. In an example, unless explicitly indicated to the contrary, the various features described in these examples may be combined.
[00141] Examples of intra mode codins based on historical information
[00142] Example 1. In one example, a table with statistical information of intra prediction modes (IPMs) may be maintained in the encoding/decoding process to help intra mode coding.
[00143] (a) In one example, the table includes the occurrence and/or frequency of each
IPM. Alternatively, the table may have entries for a partial set of allowed IPMs.
[00144] (b) Partial of IPMs listed in the table may be added as new MPMs to the MPM list.
[00145] (i) In one example, IPMs from spatial neighboring blocks are firstly added to the MPM list in order. Afterwards, a few IPMs from the table may be added in order after pruning. If the list is not full, derived IPMs may be further added.
[00146] (ii) In one example, MPM list size may be extended. In this case, IPMs from spatial neighboring blocks and derived modes may be firstly added to the MPM list in order. Afterwards, a few IPMs from the table may be added in order after pruning.
[00147] (c) IPMs included in a MPM list may be reordered based on the table.
[00148] (i) In one example, the MPM list is firstly constructed based on
neighboring blocks and derived modes (e.g., as described in Section 1.2). Afterwards, MPM list is re-ordered based on the descending order of occurrence or frequency associated with IPMs recorded in the table. In such a way, a IPM in the MPM list with highest occurrence or frequency will be adjusted to be in the first position of the MPM list.
[00149] Example 2 Different tables may be used for different color components.
[00150] (a) For example, a table is used for luma intra-mode coding and another table is used for chroma intra-mode coding.
[00151] (b) Alternatively, only one table is used for luma intra-mode coding.
[00152] (c) Alternatively, whether to use multiple tables or one table depend on whether separate coding trees are used for luma and chroma, respectively.
[00153] Example 3 The frequency or occurrence of one mode M in the table is accumulated by K (e.g. K = 1) after decoding a block with mode M.
[00154] (a) In one example, the occurrence of a mode has a maximum value (e.g. 255).
[00155] (b) If at least one occurrence of a mode reaches the maximum value, a regulation
(or normalization) process is applied.
[00156] (i) In one example, all occurrence number in the table is right shift by a number such as 2.
[00157] (ii) In another example, all occurrence number in the table subtract a
number, such as the minimum occurrence number in the table.
[00158] Example 4 Instead of utilizing IPMs of adjacent neighboring blocks for coding a block, intra prediction modes of non-adjacent neighboring blocks may be utilized.
[00159] (a) IPMs of non-adjacent neighboring blocks may be added to the MPM list.
Alternatively, pruning of IPMs may be applied. If an IPM of a non-adjacent neighboring block is different from any of IPMs in the MPMS list, it may be added to the MPM list.
[00160] (b) In one example, MPM list size may be extended. In this case, IPMs from adjacent neighboring blocks and derived modes may be firstly added to the MPM list in order. Afterwards, IPMs from non- adjacent neighboring blocks may be added in order after pruning.
[00161] (c) IPMs of non-adjacent neighboring blocks may be used to re-order the non-
MPM IPMs. In one example, for those IPMs associated with non-adjacent neighboring blocks, higher priority may be assigned, e.g., less bits may be used to code them.
[00162] Example 5 Coding of an IPM associated with current block may depend on the table, IPMs from adjacent spatial neighboring blocks, IPMs from non-adjacent spatial neighboring blocks.
[00163] Examples of selection of adjacent neighboring blocks
[00164] Example 6 Instead of using of the above and left neighboring blocks relative to the top-left position of current block for intra mode coding (such as using the intra prediction modes of above and left neighboring blocks to construct a MPM list), different positions may be utilized.
[00165] (a) In one example, the above neighboring block relative to the top-left position of current block is replaced by the above neighboring block relative to the center position of the first row of current block, as shown in FIG. 13 A.
[00166] (b) In one example, the left neighboring block relative to the top-left position of current block is replaced by the left neighboring block relative to the center position of the first column of current block, as shown in FIG. 13D.
[00167] Example 7 Instead of utilizing the fixed M relative neighboring blocks for all kinds of blocks, selection of M neighboring blocks may depend on block shape and/or block size.
[00168] (a) In one example, if the block is a square block, the intra prediction modes of above and left neighboring blocks relative to the to-left position of current block for intra mode coding may be used for coding intra prediction mode of current block.
[00169] (b) In one example, if the block is a non-square block, the intra prediction modes
of above and/or left neighboring blocks relative to the center position of the first row and/or the first column of current block for intra mode coding may be used for coding intra prediction mode of current block.
[00170] Examples of multiple prediction blocks for one intra-coded block
[00171] Example 8 Multiple prediction blocks may be generated with reconstructed samples from the same tile/slice/picture containing current block.
[00172] (a) In one example, two prediction blocks may be firstly generated and the final prediction block is generated from the two prediction blocks wherein one of them is generated by an intra prediction mode and the other is generated by a motion vector pointing to the same tile/slice/picture (e.g., intra block copy; and the motion vector is also known as block vector) containing current block.
[00173] (b) In one example, linear function is applied to the two prediction blocks to derive a final intra prediction block.
[00174] (i) Alternatively, only partial of the final prediction block is generated from the multiple (e.g., two) prediction blocks and the remaining part is directly copied from one of the multiple prediction blocks.
[00175] (ii) In one example, average of the two prediction blocks (i.e., equal weights for the two prediction blocks) is used as the final intra prediction block.
[00176] (iii) In one example, different weight for different position relative to one block, and/or for different intra prediction mode may be applied.
[00177] (iv) In one example, weights may be signaled in SPS/PPS/VPS/picture or sequence or slice header/tile/groups of CTUs/CTU/CU/PU/TU.
[00178] (v) Weights applied to the two prediction blocks may be pre-defined.
[00179] Example 9 The above methods may be applied under certain conditions.
[00180] (a) It may be applied for certain color components, e.g., only the luma component.
[00181] (b) It may be applied for certain block size/block shape etc.
[00182] Example 10. When IBC is treated as inter mode (i.e., with at least one motion vector to derive the prediction block), History-based Motion Vector Prediction (HMVP) methods, which utilize previously coded motion information for motion information prediction may be used. The HMVP methods allow the encoding/decoding process to be performed based on historical data (e.g., the blocks that have been processed). One or multiple tables with motion
information from previously coded blocks are maintained during the encoding/decoding process. Each table can be associated with a counter to track the number of motion candidates stored in the table. During the encoding/decoding of one block, the associated motion information in tables may be used (e.g., by being added to the motion candidate lists) depending on the coding characteristics of the block (e.g., whether the block is IBC coded). After encoding/decoding the block, the tables may be updated based on the coding characteristics of the block. That is, the updating of the tables is based on the encoding/decoding order. In some embodiments, the tables can be reset (e.g., removing the stored candidates and setting the associated counter to 0) before processing a new slice, a new largest coding unit (LCU) row, or a new tile.
[00183] (a) In one example, look-up-tables may be updated with motion information from
IBC-coded blocks.
[00184] (b) Alternatively, look-up-tables are not allowed to be updated with motion information from IBC-coded blocks.
[00185] (c) In one example, for the IBC mode, motion information from previously coded blocks are not allowed to predict the motion information of IBC-coded blocks.
[00186] (d) In one example, only motion information from previously IBC-coded blocks can be used to predict motion information of the current IBC-coded block. One look-up table is used to include only motion information from IBC-coded blocks and such look-up table is only used for IBC-coded blocks.
[00187] (e) Alternatively, for the IBC mode, motion information from previously coded blocks are allowed to predict the motion information of IBC-coded blocks.
[00188] (i) In one example, a look-up table may include motion information from
IBC-coded blocks and other kinds of inter-coded blocks. For example, if a motion candidate in the look-up table is derived from an IBC coded block, it may be used to encode/decode current IBC coded block.
[00189] (ii) Alternatively, multiple look-up tables may be required and at least one of them only includes motion information from IBC-coded blocks and one of them only includes motion information from other kinds of inter-coded blocks. For example, a first look-up table can include motion information for an IBC-coded block. A second, different look-up table can include motion information for a block that is not IBC-coded (e.g., the block can be coded with an inter mode). The first and second look-up tables can be updated after the corresponding
conversions are completed. In some embodiments, a first subsequent conversion of an IBC coded block can be performed using the updated first look-up table, and a second subsequent conversion of an inter coded block can be performed using the updated second look-up table.
[00190] In some embodiments, before updating look-up tables as described in Example 10 by adding a motion candidate obtained from a coded block, pruning may be applied.
[00191] (a) In one example, the new motion candidate to be added need to be pruned to all existing motion candidates in selected tables. That is, the new motion candidate is compared against all existing motion candidates in selected tables to make sure that there are no duplicated candidates.
[00192] (b) Alternatively, the new motion candidates may be only pruned to a certain number of existing motion candidates. For example, it may be compared against the last m motion candidates (< m is an integer) in a look-up table before selectively adding it as the last entry of the LUT.
[00193] (c) In some embodiments, the selected table is the one that is used to construct the motion candidate list for the coded block where the motion candidate is obtained from. In some embodiments, the motion candidate can be used to update partial or all of available look-up tables.
[00194] In some embodiments, the look-up tables as described in Example 10 may be used when a block is coded with merge or AMVP mode. That is, the current block can be coded using an IBC merge mode or an IBC AMVP mode.
[00195] In some embodiments, besides adjacent candidates, non-adjacent candidates (e.g., merge candidates or AMVP candidates) can be combined in the look-up tables.
[00196] (a) In one example, m candidates (non-adjacent and/or adjacent candidates) and n motion candidates from look-up tables may be added (e.g., when the list is not full). Both m and n are positive integers (> 0).
[00197] (b) In one example, m candidates (non-adjacent and/or adjacent candidates) may be checked and/or n motion candidates from look-up tables may be checked. The checked candidates may be added when the list is not full.
[00198] (c) In one example, the adjacent or non-adjacent candidates and motion candidates from look-up tables may be added in an interleaved way.
[00199] (i) The order of checking the adjacent or non-adjacent candidates is kept
unchanged. One candidate is checked is followed by a motion candidate from a look-up table.
[00200] (ii) The order of checking non-adjacent blocks is kept unchanged.
However, if one non-adjacent block is located outside a certain range (e.g., outside the current LCU row), it would be replaced by a motion candidate in a look-up table.
[00201] (iii) In one example, the order of checking non-adjacent blocks is kept unchanged. However, if one adjacent or non-adjacent block is coded with intra mode or intra block copy mode, it would be replaced by a motion candidate in a look-up table (e.g., upon determining that the motion candidate in the look-up table is different from the adjacent or non- adjacent block).
[00202] (d) In one example, non-adjacent candidates have a higher priority compared to motion candidates from look-up tables.
[00203] (i) In this case, all the non-adjacent blocks are checked firstly. If the total number of candidates are still less than the maximumly allowed number, then motion candidates from look-up tables are further checked.
[00204] (ii) Alternatively, motion candidates from look-up tables have a higher priority compared to non-adjacent candidates. Motion candidates from look-up tables are first checked. If the total number of candidates are still less than the maximumly allowed number, the non-adjacent blocks are checked to add non-adjacent candidates to the merge candidate list.
[00205] (iii) Similarly, when both non-adjacent method and look-up-table-based method are allowed in the Advanced Motion Vector Prediction (AMVP) coding process, the priority could be handled in a similar way as described in above examples.
[00206] In some embodiments, similar to the usage of look-up tables with motion candidates for motion vector prediction, it is proposed that one or multiple look-up tables may be constructed, and/or updated to store intra prediction modes from previously coded blocks and look-up tables may be used for coding/decoding an intra-coded block.
[00207] (a) Number of entries for each LUT is the same, e.g., equal to total number of allowed intra predictions.
[00208] (b) A variable (i.e., cnt) may be further assigned for each intra prediction mode to record how many times the intra prediction mode has been used in previously coded blocks.
[00209] (c) When updating a look-up table with a selected intra prediction mode, the associated cnt can be modified, such as increasing by 1.
[00210] In some embodiments, intra prediction modes of non-adjacent blocks may be used as intra prediction mode predictors for coding an intra-coded block.
[00211] In some embodiments, look-up tables and non-adjacent based methods may be jointly utilized. In one example, the intra prediction modes in either look-up tables or non-adjacent blocks may be used in the MPM list construction process. Alternatively, the intra prediction modes in either look-up tables or non-adjacent blocks may be used to re-order non-MPM intra prediction modes.
[00212] In some embodiments, a motion candidate list for an IBC-coded block can be constructed as follows:
[00213] (a) If there exist unique block vectors (also referred to as displacement vectors or motion vectors as discussed above) that are associated with two spatial neighboring blocks of the current block, add them to the list.
[00214] (2) Check the last available motion candidate in the table (that is, the latest candidate).
If there is no duplicated candidate in the list, the last available motion candidate is then added to the list.
[00215] (3) If the list is not full, add the remaining motion candidates in the look-up table.
[00216] Examples of chroma mode codins
[00217] Example 11. In one example, it is proposed that for chroma mode coding, the default modes (e.g., DC/Planar/HOR/VER in HEVC and VVC) may be replaced by modes derived from look-up table or non-adjacent chroma blocks.
[00218] (a) Alternatively, the default modes may be still added to the chroma mode list if the list is not full after checking those modes from look-up table or non-adjacent chroma blocks.
[00219] Example 12 In one example, it is proposed that for chroma mode coding, the default modes (e.g., DC/Planar/HOR/VER in HEVC and WC) may be replaced by modes derived from non-adjacent luma blocks (e.g., a luma block that is not the corresponding luma block of the chroma block).
[00220] (a) Alternatively, the default modes may be still added to the chroma mode list if the list is not full after checking those modes from non-adjacent luma blocks.
[00221] Example 13 In one example, it is proposed that the intra modes of spatial
neighboring blocks of the corresponding luma block may be also added to the chroma intra mode list.
[00222] (a) The left and/or above neighboring blocks of the top-left sample located at the corresponding luma block are checked and their intra modes may be used. For example, the left and/or above neighboring block of‘TL’ shown in FIG. 11 A may be checked.
[00223] (b) The examples shown in FIG. 13A-13F may be used wherein the current luma block is the corresponding luma block of a chroma block. Alternatively, the neighbor’s luma intra modes may be added to the chroma mode list. That is, intra modes of a neighboring luma block of the current luma block may be added to the chroma mode list.
[00224] The examples described above may be incorporated in the context of the methods described below, e.g., methods 1400, 1500 and 1600, which may be implemented at a video encoder and/or decoder.
[00225] FIG. 14 shows a flowchart of an example method for cross-component prediction.
The method 1400 includes, at step 1410, selecting, for a bitstream representation of a current block of visual media data, a first intra prediction mode from a set of intra prediction modes based on a first set of past intra coding information. In some embodiments, the first set of past intra coding information includes historical intra coding information.
[00226] In some embodiments, and in the context of Example 1, the first set of past intra coding information comprises statistical intra coding information. In one example, the statistical intra coding information comprises a number of occurrences of each of the set of intra prediction modes over a time duration, and the method 1400 further includes the steps of constructing a most probable mode (MPM) list based on neighboring blocks of the current block, and reordering the MPM list based on the number of occurrences.
[00227] The method 1400 includes, at step 1420, processing, based on the first intra prediction mode, the bitstream representation to generate the current block.
[00228] In some embodiments, and in the context of Example 2, the current block comprises a luma component and a chroma component, the first set of past intra coding information is used for intra-mode coding the luma component, and a second set of past intra coding information is used for intra-mode coding the chroma component. In an example, the luma component is based on a first coding tree, and the chroma component is based on a different second coding tree.
[00229] In some embodiments, and in the context of Example 3, the number of occurrences is incremented for an intra prediction mode of the set of intra prediction modes that corresponds to the first intra prediction mode. Then, the method 1400 further includes the step of determining
that at least one of the number of occurrences is equal to a maximum number of occurrences. In one example, the method further includes right shifting each of the number of occurrences by a predefined number. In another example, the method further includes subtracting a minimum number of occurrences from each of the number of occurrences.
[00230] In some embodiments, and in the context of Example 4, the method 1400 further includes the step of adding a first subset of the set of intra prediction modes to a most probable mode (MPM) list. In other embodiments, the method 1400 may further include increasing a size of an MPM list, and adding a first subset of the set of intra prediction modes to the MPM list, wherein the first subset of intra prediction modes correspond to blocks that are spatial neighbors of the current block. In these cases, the method 1400 may further include pruning the MPM list, and adding a second subset of the set of intra prediction modes to the MPM list, wherein an ordering of intra prediction modes in the second subset is maintained in the MPM list.
[00231] FIG. 15 shows a flowchart of another example method for cross-component prediction. The method 1500 includes, at step 1510, selecting, for a bitstream representation of a current block of visual media data, an intra prediction mode from at least one of adjacent or non- adjacent spatial neighboring blocks to the current block.
[00232] In some embodiments, and in the context of Example 6, the at least one of the adjacent spatial neighboring blocks is an above neighboring block relative to a top-left position of the current block, or an above neighboring block relative to a center position of a first row of the current block, or an above neighboring block relative to a top-right position of the current block, or a left neighboring block relative to a top-left position of the current block, or a left neighboring block relative to a center position of a first column of the current block, or a left neighboring block relative to a bottom-left position of the current block.
[00233] The method 1500 includes, at step 1520, processing, based on the intra prediction mode, the bitstream representation to generate the current block.
[00234] FIG. 16 shows a flowchart of yet another example method for cross-component prediction. The method 1600 includes, at step 1610, generating, for a bitstream representation of a current block of visual media data, a first prediction block and a second prediction block that are based on reconstructed samples from an image segment that comprises the current block.
[00235] The method 1600 includes, at step 1620, generating at least a portion of a final prediction block based on a linear function of the first and second prediction blocks. In some
embodiments, the final prediction block is an average of the first and second prediction blocks.
[00236] In some embodiments, and in the context of Example 8, one portion of the final prediction block is based on the linear function of the first and second prediction blocks, and the remaining portion is copied directly from the first or second prediction blocks. In some embodiments, the weights of the linear function are signaled in a sequence parameter set (SPS), a picture parameter set (PPS), a video parameter set (VPS), a slice header, a tile header, a group of coding tree units (CTUs), a coding unit (CU), a prediction unit (PU) or a transform unit (TU).
[00237] The method 1600 includes, at step 1630, processing, based on the final prediction block, the bitstream representation to generate the current block.
[00238] FIG. 24 shows a flowchart of an example method for chroma mode coding based on past information. The method 2400 includes, at step 2410, selecting, for a bitstream
representation of a current block that comprises a luma component and a chroma component, a first coding mode for the chroma component.
[00239] The method 2400 includes, at step 2420, performing, based on the first coding mode, a conversion between the bitstream representation and the current block. In some embodiments, the conversion generates the current block from the bitstream representation (e.g., as might be implemented in a video decoder). In other embodiments, the conversion generates the bitstream representation from the current block (e.g., as might be implemented in a video encoder).
[00240] In some embodiments, and in the context of Example 11, the first coding mode for the chroma component is a default mode, which may be a DC mode, a planar mode, a vertical mode or a horizontal mode.
[00241] In some embodiments, the method 2400 may further include the step of replacing the default mode with a second coding mode derived from a look-up table or a non-adjacent chroma block. In an example, and in the context of Example 11, the default mode is added to a chroma mode list upon determining that the chroma mode list comprises the second coding mode from the non-adjacent chroma block and is not full.
[00242] In some embodiments, the method 2400 may further include the step of replacing the default mode with a second coding mode derived from a non-adjacent luma block. In an example, and in the context of Example 12, the default mode is added to a chroma mode list upon determining that the chroma mode list comprises the second coding mode from the non-adjacent luma block and is not full.
[00243] FIG. 25 shows a flowchart of another example method for chroma mode coding based on past information. The method 2500 includes, at step 2510, selecting, for a bitstream representation of a current block that comprises a luma component and a chroma component, an intra coding mode from a spatial neighboring block to the luma component.
[00244] The method 2500 includes, at step 2520, adding the intra coding mode to a chroma intra mode list.
[00245] The method 2500 includes, at step 2530, performing, based on the intra coding mode, a conversion between the bitstream representation and the current block.
[00246] In some embodiments, and in the context of Examples 6 and 13, the spatial neighboring block may be an above neighboring block relative to a top-left position of the current block, an above neighboring block relative to a center position of a first row of the current block, an above neighboring block relative to a top-right position of the current block, a left neighboring block relative to a center position of a first column of the current block, or a left neighboring block relative to a bottom-left position of the current block.
5 Example implementations of the disclosed technology
[00247] FIG. 17 is a block diagram of a video processing apparatus 1700. The apparatus 1700 may be used to implement one or more of the methods described herein. The apparatus 1700 may be embodied in a smartphone, tablet, computer, Internet of Things (IoT) receiver, and so on. The apparatus 1700 may include one or more processors 1702, one or more memories 1704 and video processing hardware 1706. The processor(s) 1702 may be configured to implement one or more methods (including, but not limited to, methods 1400, 1500 and 1600) described in the present document. The memory (memories) 1704 may be used for storing data and code used for implementing the methods and techniques described herein. The video processing hardware 1706 may be used to implement, in hardware circuitry, some techniques described in the present document.
[00248] In some embodiments, the video coding methods may be implemented using an apparatus that is implemented on a hardware platform as described with respect to FIG. 17.
[00249] FIG. 18 is a block diagram showing an example video processing system 1800 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of the system 1800. The system 1800 may include input 1802 for receiving video content. The video content may be received in a raw or uncompressed
format, e.g., 8 or 10 bit multi-component pixel values, or may be in a compressed or encoded format. The input 1802 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interface include wired interfaces such as Ethernet, passive optical network (PON), etc. and wireless interfaces such as Wi-Fi or cellular interfaces.
[00250] The system 1800 may include a coding component 1804 that may implement the various coding or encoding methods described in the present document. The coding component 1804 may reduce the average bitrate of video from the input 1802 to the output of the coding component 1804 to produce a coded representation of the video. The coding techniques are therefore sometimes called video compression or video transcoding techniques. The output of the coding component 1804 may be either stored, or transmitted via a communication connected, as represented by the component 1806. The stored or communicated bitstream (or coded) representation of the video received at the input 1802 may be used by the component 1808 for generating pixel values or displayable video that is sent to a display interface 1810. The process of generating user-viewable video from the bitstream representation is sometimes called video decompression. Furthermore, while certain video processing operations are referred to as “coding” operations or tools, it will be appreciated that the coding tools or operations are used at an encoder and corresponding decoding tools or operations that reverse the results of the coding will be performed by a decoder.
[00251] Examples of a peripheral bus interface or a display interface may include universal serial bus (ETSB) or high definition multimedia interface (HDMI) or Displayport, and so on. Examples of storage interfaces include SATA (serial advanced technology attachment), PCI,
IDE interface, and the like. The techniques described in the present document may be embodied in various electronic devices such as mobile phones, laptops, smartphones or other devices that are capable of performing digital data processing and/or video display.
[00252] FIG. 19 shows a flowchart of an example method 1900 for video coding in accordance with the disclosed technology. The method 1900 includes, at operation 1910, selecting, for a conversion between a current block of visual media data and a bitstream representation of the current block, a first intra prediction mode based on at least a first set of history intra coding information that includes statistical information of a set of intra prediction modes. The method 1900 also includes, at operation 1920, performing the conversion based on the first intra prediction mode.
[00253] In some embodiments, the conversion includes encoding the current block to generate the bitstream representation. In some embodiments, the conversion includes decoding the bitstream representation to generate the current block. In some embodiments, the statistical intra coding information comprises a number of occurrences of each of the set of intra prediction modes over a time duration. In some embodiments, the statistical intra coding information comprises a number of occurrences of a part of the set of intra prediction modes over a time duration.
[00254] In some embodiments, the method further includes, comprising, after performing the conversion, updating the set of history intra coding information. In some embodiments, the updating comprises accumulating the number of occurrences of the first intra prediction mode by k in the first set of history intra coding information, k being a positive integer. In some embodiments, a subsequent block of the current block is processed based on the updated history intra coding information.
[00255] In some embodiments, each intra prediction mode in the first set of history intra coding information is associated with a limit for the number of occurrences. In some embodiments, the method includes, after the number of occurrences reaches the limit, reducing the number of occurrences based on a predefined rule. In some embodiments, the predefined rule comprises dividing the number of occurrences by a first predefined value, or subtracting a second predefined value from the number of occurrences.
[00256] In some embodiments, the method includes constructing a most probable mode (MPM) list based on at least the set of intra prediction modes. In some embodiments, the method includes adding a subset of the set of intra prediction modes to the MPM list. In some embodiments, the method includes adding intra prediction modes associated with spatial neighbors of the current block to the MPM list, adding a subset of the set of intra prediction modes to the MPM list, and pruning the MPM list. In some embodiments, the method includes adding derived intra prediction modes to the MPM list. In some embodiments, the method includes increasing a size of the MPM list. In some embodiments, the method includes reordering the MPM list based on the set of history intra coding information. In some embodiments, the MPM list is reordered based on a descending order of the statistical information of the set of intra prediction modes.
[00257] In some embodiments, the current block comprises a luma component and a chroma
component. The first set of history intra coding information is used for intra-mode coding the luma component. In some embodiments, a second set of history intra coding information is used for intra-mode coding the chroma component. In some embodiments, the luma component is based on a first coding tree and the chroma component is based on a second coding tree different from the first coding tree.
[00258] In some embodiments, selecting the first intra prediction mode is further based on intra prediction modes of spatial neighbors of the current block or intra prediction modes of non- adjacent neighbors of the current block. In some embodiments, the history intra coding information is stored in a table.
[00259] FIG. 20 shows a flowchart of another example method 2000 for video processing in accordance with the disclosed technology. The method 2000 includes, at operation 2010, selecting, for a conversion between a current block of visual media data and a bitstream representation of the current block, a first intra prediction mode based on at least intra prediction modes associated with non-adjacent neighbors of the current block. The method 2000 also includes, at operation 2020, performing the conversion based on the first intra prediction mode.
[00260] In some embodiments, the conversion includes encoding the current block to generate the bitstream representation. In some embodiments, the conversion includes decoding the bitstream representation to generate the current block.
[00261] In some embodiments, the method further includes constructing, for the conversion between the current block and the bitstream representation of the current block, a most probable mode (MPM) list based on the intra prediction modes associated with the non-adjacent neighbors of the current block. The conversion is performed further based on the MPM list. In some embodiments, constructing the MPM list includes adding intra prediction modes associated with spatial neighbors of the current block to the MPM list and pruning the MPM list. In some embodiments, constructing the MPM list includes adding derived intra prediction modes to the MPM list. In some embodiments, the method includes increasing a size of the MPM list.
[00262] In some embodiments, the method includes reordering intra prediction modes that are not in the MPM list according to the intra prediction modes associated with the non-adjacent neighbors of the current block. In some embodiments, the intra prediction modes associated with the non-adjacent neighbors of the current block are associated with a higher priority for the reordering.
[00263] FIG. 21 shows a flowchart of an example method 2100 for video processing in accordance with the disclosed technology. The method 2100 includes, at operation 2110, selecting, for a conversion between a current block of visual media data and a bitstream representation of the current block, an intra prediction mode based on at least one of spatial neighboring blocks of the current block. The at least one of the spatial neighboring blocks is different from a first block that is located to a left of a first row of the current block and a second block that is located above a first column of the current block. The method 2100 also includes, at operation 2110, performing the conversion based on the intra prediction mode.
[00264] In some embodiments, the conversion comprises encoding the current block to generate the bitstream representation. In some embodiments, the conversion comprises decoding the bitstream representation to generate the current block.
[00265] In some embodiments, the at least one of the spatial neighboring blocks includes a block adjacent to the current block. In some embodiments, the at least one of the spatial neighboring blocks includes a block non-adjacent to the current block. In some embodiments, the at least one of the adjacent spatial neighboring blocks includes a block adjacent to a top-left position of the current block. In some embodiments, the at least one of the adjacent spatial neighboring blocks includes a block adjacent to a center position of a first row of the current block. In some embodiments, the at least one of the adjacent spatial neighboring blocks includes a block adjacent to a center position of a first column of the current block.
[00266] In some embodiments, the at least one of the spatial neighboring blocks is selected based on one or more dimensions of the current block. In some embodiments, the current block has a square shape, and the at least one of the adjacent spatial neighboring blocks includes a block adjacent to a top-left position of the current block. The selected intra prediction mode can be added to a most-probable-mode list of the current block.
[00267] In some embodiments, the current block has a non-square shape, and the at least one of the spatial neighboring blocks includes a block adjacent to a center position of a first row of the current block or a block adjacent to a center position of a first column of the current block.
[00268] FIG. 22 shows a flowchart of an example method 2200 for video processing in accordance with the disclosed technology. The method 2200 includes, at operation 2210, generating a final prediction block for a conversion between a current block of visual media data and a bitstream representation of the current block. The method 2200 includes, at operation
2220, performing the conversion based on the final prediction block. At least a portion of the final prediction block is generated based on a combination of a first prediction block and a second prediction block that are based on reconstructed samples from an image segment that comprises the current block. In some embodiments, the conversion comprises encoding the current block to generate the bitstream representation. In some embodiments, the conversion comprises decoding the bitstream representation to generate the current block.
[00269] In some embodiments, the first prediction block is generated according to an intra prediction mode. In some embodiments, the second prediction block is generated based on a motion vector pointing to the image segment. In some embodiments, the second prediction block is generated based on an intra block copy technology in which the reconstructed samples pointed by the motion vector are copied. In some embodiments, the second prediction block is generated based on an intra block copy technology in which motion compensation is applied to the reconstructed samples pointed by the motion vector before applying an in-loop filtering operation.
[00270] In some embodiments, the reconstructed samples are generated before an in-loop filtering operation is applied. In some embodiments, a remaining portion of the final prediction block is copied directly from the first or second prediction block. In some embodiments, the combination of the first and the second prediction blocks comprises a linear function of the first and second prediction blocks. In some embodiments, the linear function comprises an average of the first and second prediction blocks. In some embodiments, the linear function includes a weighted linear function of the first and the second prediction blocks. In some embodiments, different weights are applied for different positions relative to the first or the second prediction block. In some embodiments, different weights are applied for different intra prediction modes. In some embodiments, weights of the weighted linear function are signaled in a sequence parameter set (SPS), a picture parameter set (PPS), a video parameter set (VPS), a slice header, a tile header, a group of coding tree units (CTUs), a coding unit (CU), a prediction unit (PU) or a transform unit (TU). In some embodiments, weights of the weighted linear function are predefined.
[00271] In some embodiments, the image segment includes a picture, a slice or a tile. In some embodiments, the first prediction block and the second prediction block are generated for a selected color component of the current block. In some embodiments, the selected color
component includes a luma color component. In some embodiments, the first prediction block and the second prediction block are generated upon determining that a size or a shape of the current block satisfies a predefined criterion.
[00272] FIG. 23 shows a flowchart of an example method 2300 for video processing in accordance with the disclosed technology. The method 2300 includes, at operation 2310, maintaining a first table of motion candidates during a conversion between a current block of visual media data and a bitstream representation of the current block. The method 2300 includes, at operation 2320, determining, based on at least the first table of motion candidates, motion information for the current block that is coded using an intra block copy (IBC) mode in which at least one motion vector is directed at an image segment that comprises the current block. The method 2300 also includes, at operation 2330, performing the conversion based on the motion information. In some embodiments, the conversion includes encoding the current block to generate the bitstream representation. In some embodiments, the conversion includes decoding the bitstream representation to generate the current block.
[00273] In some embodiments, the method includes selectively updating, based on a rule, the first table of motion candidates using the motion information for the conversion. In some embodiments, the rule specifies adding the motion information to the first table of motion candidates. In some embodiments, the rule specifies excluding the motion information from the first table of motion candidates.
[00274] In some embodiments, the method includes updating, based on the motion
information for the current block, the first table of motion candidates after the conversion. In some embodiments, the method includes performing a second conversion between a second block of visual media data and the bitstream representation of the second block using the updated first table of motion candidates. The second block can be coded using the intra block copy mode. In some embodiments, the first table of motion candidates includes only motion information from blocks that are coded using the intra block copy mode. In some embodiments, the first table of motion candidates is only used for processing blocks that are coded using the intra block copy mode. In some embodiments, the first table of motion candidates includes motion information from blocks that are coded using the intra block copy mode and excludes motion information from blocks that are coded using other techniques. In some embodiments, the first table of motion candidates includes motion information from blocks that are coded using the intra block
copy mode and motion information from blocks that are coded using other techniques. In some embodiments, the second conversion is performed using motion candidates associated with IBC coded blocks in the updated first table of motion candidates.
[00275] In some embodiments, the method includes maintaining, for a third conversion between a third block of visual media data and the bitstream representation of the third block, a second table of motion candidates, performing the third conversion between the third block and the bitstream representation of the third block based on the second table of motion candidates. In some embodiments, the third conversion is performed without using the updated first table of motion candidates. In some embodiments, the second table includes motion information from blocks that are encoded using other techniques that are different than the IBC mode. In some embodiments, the third block is coded using a technique that is different than the IBC mode. In some embodiments, the technique includes an inter mode. In some embodiments, the method includes updating the second table of motion candidates using motion information for the third conversion. In some embodiments, the method includes performing a fourth conversion between an IBC coded block based on the updated first table and performing a fifth conversion between an inter coded block based on the updated second table.
[00276] In some embodiments, the method includes comparing a motion candidate associated with the current block with a number of entries in the first table. The first table is updated based on the comparing. In some embodiments, the number of entries corresponds to all entries in the first tables. In some embodiments, the number of entries is m, m being an integer, and the m entries are last m entries in the first table. In some embodiments, updating the first table comprises adding the motion candidate to the first table. In some embodiments, the first table includes no duplicated entries.
[00277] In some embodiments, the method includes determining that the current block is further coded using an IBC merge mode or an IBC Advanced Motion Vector Prediction (AMVP) mode. In some embodiments, the method includes determining a list of motion candidates for the current block by combining the first table of motion candidates and a set of adjacent or non- adjacent candidates of the current block and performing the conversion based on the list. In some embodiments, the combining comprises checking m candidates from the set of adjacent or non- adjacent candidates, and checking n motion candidates from the first table of motion candidates, wherein m and n are positive integers. In some embodiments, the method includes adding at least
one of the m candidates from the set of adjacent or non-adjacent candidates to the list of motion candidates upon determining that the list is not full. In some embodiments, the method includes adding at least one of the n motion candidates from the first table of motion candidates to the list of motion candidates upon determining that the list is not full.
[00278] In some embodiments, the combining comprises checking candidates from the set of adjacent or non-adjacent candidates and the first table of motion candidates in an interleaved manner. In some embodiments, the method includes checking a candidate from the set of adjacent or non-adjacent candidates prior to checking a motion candidate from the first table. In some embodiments, the checking comprises checking a candidate from the set of adjacent or non-adjacent candidates and replacing, based on a coding characteristic of an adjacent or non- adjacent block associated with the adjacent or non-adjacent candidate, the candidate with a motion candidate from the first table of motion candidates. In some embodiments, the coding characteristic of the adjacent or non-adjacent block indicates that the adjacent or non-adjacent block is located outside a predefined range. In some embodiments, the coding characteristic of the adjacent or non-adjacent block indicates that the adjacent or non-adjacent block is intra coded. In some embodiments, adding a motion candidate further comprising adding the motion candidate to the list upon determining that the motion candidate from the first table of motion candidates is different from at least one candidate from the set of adjacent or non-adjacent candidates. In some embodiments, the set of adjacent or non-adjacent candidate has a higher priority than the first table of motion candidates.
[00279] In some embodiments, the list of motion candidates comprises multiple merge candidates used for IBC merge-coded blocks. In some embodiments, the list of motion candidates comprises multiple AMVP candidates used for IBC AMVP-coded blocks. In some embodiments, the list of motion candidates comprises intra prediction modes used for intra- coded blocks. In some embodiments, a size of the first table is pre-defined or signaled in the bitstream representation. In some embodiments, the first table is associated with a counter that indicates a number of available motion candidates in the first table.
[00280] In some embodiments, the method includes resetting the first table before processing a new slice, a new LCU row, or a new tile. In some embodiments, the counter is set to zero before processing a new slice, a new LCU row, or a new tile.
[00281] FIG. 26 shows a flowchart of an example method 2600 for video processing in
accordance with the disclosed technology. The method 2600 includes, at operation 2610, determining, for a conversion between a current block of video data and a bitstream
representation of the current block, a first coding mode for a chroma component of the current block based on at least one or more coding modes of a block that is not adjacent to the current block. The method 2600 also includes, at operation 2620, performing, based on the first coding mode, the conversion between the bitstream representation and the current block.
[00282] In some embodiments, the conversion generates the current block from the bitstream representation. In some embodiments, the conversion generates the bitstream representation from the current block.
[00283] In some embodiments, the first coding mode is derived from a table of motion candidates or a non-adjacent chroma block. In some embodiments, the first coding mode is derived from a luma block that is different than a corresponding luma block of the chroma component. In some embodiments, the first coding mode is derived from a luma block non- adjacent to a corresponding luma block of the chroma component. In some embodiments, the first coding mode is determined to be a first default coding mode.
[00284] In some embodiments, the method includes adding the first default coding mode to a chroma mode candidate list of the current block. In some embodiments, the method further includes determining a second default coding mode that includes one of a DC mode, a planar mode, a vertical mode or a horizontal mode. In some embodiments, the second default coding mode is added to a chroma mode list upon determining that the chroma mode list is not full after checking the first default mode.
[00285] In some embodiments, a motion candidate in the table of motion candidates corresponds to an intra prediction mode of a previously coded block. In some embodiments, the method includes updating the table of motion candidates after the conversion. In some embodiments, the method includes performing a second conversion between a subsequent block of video data and a bitstream representation of the subsequent block based on the updated table.
[00286] FIG. 27 shows a flowchart of an example method 2700 for video processing in accordance with the disclosed technology. The method 2700 includes, at operation 2710, determining, for a conversion between a bitstream representation of a chroma block and the chroma block, an intra coding mode based on a spatial neighboring block of a luma block that corresponds to the chroma block. The method 2700 also includes, at operation 2720, performing,
based on the intra coding mode, a conversion between the bitstream representation and the chroma block.
[00287] In some embodiments, the conversion generates the chroma block from the bitstream representation. In some embodiments, the conversion generates the bitstream representation from the chroma block. In some embodiments, the method includes adding the intra coding mode to a chroma intra mode list of the chroma block.
[00288] In some embodiments, the spatial neighboring block includes an above neighboring block relative to a top-left position of the chroma block. In some embodiments, the spatial neighboring block includes a left neighboring block relative to a top-left position of the chroma block. In some embodiments, the spatial neighboring block includes an above neighboring block relative to a top-right position of the chroma block. In some embodiments, the spatial neighboring block includes a left neighboring block relative to a bottom-left position of the chroma block. In some embodiments, the spatial neighboring block includes a block adjacent to a center position of a first row of the chroma block. In some embodiments, the spatial neighboring block is a block adjacent to a center position of a first column of the chroma block.
[00289] From the foregoing, it will be appreciated that specific embodiments of the presently disclosed technology have been described herein for purposes of illustration, but that various modifications may be made without deviating from the scope of the invention. Accordingly, the presently disclosed technology is not limited except as by the appended claims.
[00290] Implementations of the subject matter and the functional operations described in this patent document can be implemented in various systems, digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Implementations of the subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible and non-transitory computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine- readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them. The term“data processing unit” or“data processing apparatus” encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable
processor, a computer, or multiple processors or computers. The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.
[00291] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code).
A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a
communication network.
[00292] The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).
[00293] Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read only memory or a random access memory or both. The essential elements of a computer are a processor for performing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks. However, a computer need not have such devices.
Computer readable media suitable for storing computer program instructions and data include all forms of nonvolatile memory, media and memory devices, including by way of example
semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.
[00294] It is intended that the specification, together with the drawings, be considered example only. As used herein, the use of“or” is intended to include“and/or”, unless the context clearly indicates otherwise.
[00295] While this patent document contains many specifics, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple
embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.
[00296] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Moreover, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all
embodiments.
[00297] Only a few implementations and examples are described and other implementations, enhancements and variations can be made based on what is described and illustrated in this patent document.
Claims
1. A method for video processing, comprising:
determining, for a conversion between a current block of video data and a bitstream representation of the current block, a first coding mode for a chroma component of the current block based on at least one or more coding modes of a block that is not adjacent to the current block; and
performing, based on the first coding mode, the conversion between the bitstream representation and the current block.
2. The method of claim 1 , wherein the conversion generates the current block from the bitstream representation.
3. The method of claim 1, wherein the conversion generates the bitstream representation from the current block.
4. The method of any of claims 1 to 3, wherein the first coding mode is derived from a table of motion candidates or a non-adjacent chroma block.
5. The method of any of claims 1 to 4, wherein the first coding mode is derived from a luma block that is different than a corresponding luma block of the chroma component.
6. The method of any of claims 1 to 3, wherein the first coding mode is derived from a luma block non-adjacent to a corresponding luma block of the chroma component.
7. The method of claim 1 to 6, wherein the first coding mode is determined to be a first default coding mode.
8. The method of claim 7, further comprising:
adding the first default coding mode to a chroma mode candidate list of the current block.
9. The method of claim 7 or 8, further comprising:
determining a second default coding mode that includes one of a DC mode, a planar mode, a vertical mode or a horizontal mode.
10. The method of claim 9, wherein the second default coding mode is added to a chroma mode list upon determining that the chroma mode list is not full after checking the first default mode.
11. The method of any of claims 1 to 4, wherein a motion candidate in the table of motion candidates corresponds to an intra prediction mode of a previously coded block.
12. The method of any of claim 1 to 4, comprising:
updating the table of motion candidates after the conversion.
13. The method of claim 12, comprising:
performing a second conversion between a subsequent block of video data and a bitstream representation of the subsequent block based on the updated table.
14. A method for video processing, comprising:
determining, for a conversion between a bitstream representation of a chroma block and the chroma block, an intra coding mode based on a spatial neighboring block of a luma block that corresponds to the chroma block; and
performing, based on the intra coding mode, a conversion between the bitstream representation and the chroma block.
15. The method of claim 14, wherein the conversion generates the chroma block from the bitstream representation.
16. The method of claim 14, wherein the conversion generates the bitstream representation from the chroma block.
17. The method of any of claims 14 to 16, comprising:
adding the intra coding mode to a chroma intra mode list of the chroma block.
18. The method of any of claims 14 to 17, wherein the spatial neighboring block includes an above neighboring block relative to a top-left position of the chroma block.
19. The method of any of claims 14 to 17, wherein the spatial neighboring block includes a left neighboring block relative to a top-left position of the chroma block.
20. The method of any of claims 14 to 17, wherein the spatial neighboring block includes an above neighboring block relative to a top-right position of the chroma block.
21. The method of any of claims 14 to 17, wherein the spatial neighboring block includes a left neighboring block relative to a bottom-left position of the chroma block.
22. The method of any of claims 14 to 17, wherein the spatial neighboring block includes a block adjacent to a center position of a first row of the chroma block.
23. The method of any of claims 14 to 17, wherein the spatial neighboring block is a block adjacent to a center position of a first column of the chroma block.
24. An apparatus in a video system comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to implement the method in any one of claims 1 to 23.
25. A computer program product stored on a non-transitory computer readable media, the computer program product including program code for carrying out the method in any one of claims 1 to 23.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN2018107180 | 2018-09-24 | ||
CNPCT/CN2018/107180 | 2018-09-24 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2020065508A1 true WO2020065508A1 (en) | 2020-04-02 |
Family
ID=68136476
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/IB2019/058066 WO2020065508A1 (en) | 2018-09-24 | 2019-09-24 | Chroma mode intra coding |
Country Status (3)
Country | Link |
---|---|
CN (1) | CN110944198B (en) |
TW (1) | TWI838406B (en) |
WO (1) | WO2020065508A1 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114071158A (en) * | 2020-07-29 | 2022-02-18 | 腾讯科技(深圳)有限公司 | Motion information list construction method, device and equipment in video coding and decoding |
WO2024120307A1 (en) * | 2022-12-07 | 2024-06-13 | Mediatek Inc. | Method and apparatus of candidates reordering of inherited cross-component models in video coding system |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116193125B (en) * | 2020-04-30 | 2024-07-23 | 北京达佳互联信息技术有限公司 | Method and apparatus for prediction related residual scaling for video coding |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20170332084A1 (en) * | 2016-05-13 | 2017-11-16 | Qualcomm Incorporated | Neighbor based signaling of intra prediction modes |
WO2018116925A1 (en) * | 2016-12-21 | 2018-06-28 | シャープ株式会社 | Intra prediction image generating device, image decoding device, and image coding device |
US20180199061A1 (en) * | 2015-07-14 | 2018-07-12 | Kai Zhang | Method and Apparatus of Advanced Intra Prediction for Chroma Components in Video and Image Coding |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR101611409B1 (en) * | 2011-09-28 | 2016-04-14 | 한국전자통신연구원 | Method for encoding/decoding an intra prediction mode and apparatus for the same |
KR20170058837A (en) * | 2015-11-19 | 2017-05-29 | 한국전자통신연구원 | Method and apparatus for encoding/decoding of intra prediction mode signaling |
-
2019
- 2019-09-24 CN CN201910904951.4A patent/CN110944198B/en active Active
- 2019-09-24 WO PCT/IB2019/058066 patent/WO2020065508A1/en active Application Filing
- 2019-09-24 TW TW108134477A patent/TWI838406B/en active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20180199061A1 (en) * | 2015-07-14 | 2018-07-12 | Kai Zhang | Method and Apparatus of Advanced Intra Prediction for Chroma Components in Video and Image Coding |
US20170332084A1 (en) * | 2016-05-13 | 2017-11-16 | Qualcomm Incorporated | Neighbor based signaling of intra prediction modes |
WO2018116925A1 (en) * | 2016-12-21 | 2018-06-28 | シャープ株式会社 | Intra prediction image generating device, image decoding device, and image coding device |
US20190327466A1 (en) * | 2016-12-21 | 2019-10-24 | Sharp Kabushiki Kaisha | Intra prediction image generation apparatus, image decoding apparatus, and image coding apparatus |
Non-Patent Citations (2)
Title |
---|
CHEN J ET AL: "Algorithm description of Joint Exploration Test Model 6 (JEM6)", 6. JVET MEETING; 31-3-2017 - 7-4-2017; HOBART; (THE JOINT VIDEO EXPLORATION TEAM OF ISO/IEC JTC1/SC29/WG11 AND ITU-T SG.16 ); URL: HTTP://PHENIX.INT-EVRY.FR/JVET/,, no. JVET-F1001, 31 May 2017 (2017-05-31), XP030150793 * |
KOTRA (HUAWEI) A M ET AL: "CE3.2.2: Intra mode signaling with priority based MPM and non-MPM list construction", no. JVET-K0365, 11 July 2018 (2018-07-11), XP030199461, Retrieved from the Internet <URL:http://phenix.int-evry.fr/jvet/doc_end_user/documents/11_Ljubljana/wg11/JVET-K0365-v3.zip JVET-K0365.docx> [retrieved on 20180711] * |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114071158A (en) * | 2020-07-29 | 2022-02-18 | 腾讯科技(深圳)有限公司 | Motion information list construction method, device and equipment in video coding and decoding |
WO2024120307A1 (en) * | 2022-12-07 | 2024-06-13 | Mediatek Inc. | Method and apparatus of candidates reordering of inherited cross-component models in video coding system |
Also Published As
Publication number | Publication date |
---|---|
TW202017378A (en) | 2020-05-01 |
CN110944198A (en) | 2020-03-31 |
TWI838406B (en) | 2024-04-11 |
CN110944198B (en) | 2023-12-12 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11870980B2 (en) | Selection of adjacent neighboring block for intra coding | |
CN114339221B (en) | Convolutional neural network based filter for video encoding and decoding | |
JP7403555B2 (en) | Matrix derivation in intracoding mode | |
KR20210150387A (en) | Determination of chroma coding mode based on matrix-based intra prediction | |
US20230132613A1 (en) | Matrix-based intra prediction using upsampling | |
JP2024003124A (en) | Restricted upsampling process in matrix-based intra prediction | |
US20230037931A1 (en) | Context coding for matrix-based intra prediction | |
US11490089B2 (en) | Transform bypass coded residual blocks in digital video | |
KR20220038690A (en) | Weighting factors for predictive sample filtering in intra mode | |
CN115211113A (en) | Intra-frame prediction of multi-parameter models | |
WO2020065508A1 (en) | Chroma mode intra coding | |
JP2023164589A (en) | Matrix-based intra prediction using filtering | |
CN118413644A (en) | Convolutional neural network based filter for video encoding and decoding | |
CN117121481A (en) | Individual tree codec limitations |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 19782736 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 02.09.2021) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 19782736 Country of ref document: EP Kind code of ref document: A1 |