CN116195253A

CN116195253A - Block vector processing in intra block copy codec

Info

Publication number: CN116195253A
Application number: CN202180050329.7A
Authority: CN
Inventors: 刘鸿彬; 王悦; 李俊儒
Original assignee: Douyin Vision Co Ltd
Current assignee: Douyin Vision Co Ltd
Priority date: 2020-08-20
Filing date: 2021-08-19
Publication date: 2023-05-30
Also published as: WO2022037628A1

Abstract

Methods, systems, and devices for Intra Block Copy (IBC) using block vector processing are described. An example method of video processing includes: for a transition between video including a current video block and a bitstream of the video, determining availability of a non-immediately adjacent neighboring block for Block Vector (BV) based encoding and decoding according to a rule based on an overlap condition between a current region including the current video block and a neighboring region including the non-immediately adjacent neighboring block; and performing a conversion based on the determination.

Description

Block vector processing in intra block copy codec

Cross Reference to Related Applications

The present application claims priority and benefit from international patent application No. pct/CN2020/110325 filed 8/20/2020, in time, in accordance with applicable patent laws and/or rules of the paris convention. The entire disclosure of the foregoing application is incorporated by reference as part of the disclosure of this application for all purposes in accordance with law.

Technical Field

This patent document relates to image and video coding.

Background

Digital video occupies the largest bandwidth usage on the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth requirements for digital video usage are expected to continue to increase.

Disclosure of Invention

The present document discloses techniques for Intra Block Copy (IBC) using block vector processing that can be used by image and video encoders and decoders to perform image or video encoding, decoding, or processing.

In one example aspect, a video processing method is disclosed. The method comprises the following steps: for a transition between a video including a current video block and a bitstream of the video, determining availability of a non-immediately adjacent block for Block Vector (BV) based encoding and decoding according to a rule based on an overlap condition between a current region including the current video block and an adjacent region including the non-immediately adjacent block of the current video block; and performing a conversion according to the determination.

In another example aspect, another video processing method is disclosed. The method comprises the following steps: for a transition between a video including a current video block and a bitstream of the video, determining whether a sample string vector of the current video block is used to predict a block vector and/or a string vector of a subsequent video block associated with the current video block; and performing a conversion based on the determination.

In yet another example aspect, another video processing method is disclosed. The method comprises the following steps: for a transition between a video including a current video block and a bitstream of the video, determining that one or more sample string vectors that are not immediately adjacent to the block are available for prediction of a block vector of the current video block; and performing a conversion based on the determination.

In yet another example aspect, another video processing method is disclosed. The method comprises the following steps: for a transition between a video including a current video block and a bitstream of the video, determining that one or more block vectors that are not immediately adjacent to the block are available for prediction of a sample string vector of the current video block; and performing a conversion based on the determination.

In yet another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video including a current video block and a bitstream of the video, wherein one or more syntax elements are used in the bitstream to indicate a block vector or a sample string vector for predicting the current video block according to a format rule.

In yet another example aspect, another video processing method is disclosed. The method comprises the following steps: a conversion between a video comprising a video block and a bitstream of the video is performed, wherein the conversion complies with a rule, and wherein the rule specifies that the same list construction process is used during the conversion to determine a block vector prediction list or a sample string vector prediction list of the video block.

In yet another example aspect, another video processing method is disclosed. The method comprises the following steps: for a transition between a video including a current video block and a bitstream of the video, determining whether a sample string vector or a block vector of an associated block is used to predict a block vector of the current video block according to a rule; and performing a conversion based on the determination, wherein the associated block comprises a parent block or at least one sibling block of the current video block, wherein the sample string vector is derived in a sample string prediction mode and the block vector is derived in an intra block copy mode.

In yet another example aspect, another video processing method is disclosed. The method comprises the following steps: for a transition between a video including a current video block and a bitstream of the video, determining that at least one block vector generated from an intra block copy mode for encoding and decoding the current video block is used in an estimation process of a sample string vector of the current video block; and performing a conversion based on the determination, wherein the current video block is further encoded using the sample string prediction mode.

In yet another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video including a current video block and a bitstream of the video, wherein the bitstream conforms to a format rule, and wherein the format rule specifies that a block vector index indicating a block vector in a block vector prediction list used by the current video block is constrained to be less than a number of block vectors in the block vector prediction list.

In yet another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video including a current video block and a bitstream of the video, wherein the bitstream conforms to a format rule, and wherein the format rule specifies that a sample string vector index indicating a block vector or a sample string vector in an intra-history motion vector prediction list used by the current video block is constrained to be less than a number of elements in the intra-history motion vector prediction list.

In yet another example aspect, a video encoder apparatus is disclosed. The video encoder includes a processor configured to implement the above-described method.

In yet another example aspect, a video decoder apparatus is disclosed. The video decoder includes a processor configured to implement the above-described method.

In yet another example aspect, a computer-readable medium storing code is disclosed. Which code embodies one of the methods described herein in the form of processor executable code.

These and other features will be described in this document.

Drawings

Fig. 1 shows an example of Intra Block Copy (IBC).

FIG. 2 illustrates example locations of spatial merge candidates.

Fig. 3 shows an example of non-immediately adjacent blocks.

Fig. 4 shows another example of a non-immediately adjacent block.

Fig. 5 shows an example of sample string prediction.

Fig. 6 shows an example of a usable non-immediately adjacent block area.

Fig. 7 is a block diagram illustrating an example video processing system in which various techniques disclosed herein may be implemented.

FIG. 8 is a block diagram of an example hardware platform for video processing.

Fig. 9 is a block diagram illustrating an example video codec system capable of implementing some embodiments of the present disclosure.

Fig. 10 is a block diagram illustrating an example of an encoder capable of implementing some embodiments of the present disclosure.

Fig. 11 is a block diagram illustrating an example of a decoder capable of implementing some embodiments of the present disclosure.

Fig. 12-21 illustrate flowcharts of example methods of video processing.

Detailed Description

The section headings are used herein for ease of understanding and do not limit the applicability of the techniques and embodiments disclosed in each section to that section only. Furthermore, the H.266 term is used in some descriptions only for ease of understanding and not to limit the scope of the disclosed technology. Thus, the techniques described herein are also applicable to other video codec protocols and designs.

1. Introduction to the invention

The techniques described in this document may be used to encode and decode visual media data, such as images or video, commonly referred to in this document as video. In particular, it relates to intra-block copying in video codecs. It can be applied in existing video codec standards such as HEVC or the upcoming standard (multi-function video codec, audio video standard 3). It may also be used for future video codec standards or video codecs.

2. Preliminary discussion

Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO/IEC standards. The ITU-T specifies H.261 and H.263, the ISO/IEC specifies MPEG-1 and MPEG-4 Visual, and the two organizations jointly specify the H.264/MPEG-2 video and H.264/MEPG-4 advanced video codec (Advanced Video Coding, AVC) and the H.265/HEVC standards. Since h.262, the video codec standard was based on a hybrid video codec structure in which temporal prediction plus transform coding was used. To explore future video codec technologies beyond HEVC, a joint video exploration team (Joint Video Exploration Team, jfet) was jointly established by VCEG and MPEG in 2015. Thereafter, jfet takes many new approaches and applies them to reference software named joint exploration model (Joint Exploration Model, JEM). In month 4 of 2018, a joint video expert team (Joint Video Expert Team, jfet) was established between VCEG (Q6/16) and ISO/IEC JTC1 SC29/WG11 (MPEG) to address the VVC standard that reduces 50% bit rate compared to HEVC.

The latest version of the VVC draft, i.e. the multi-function video codec (draft 9), can be found at the following web sites:

http://phenix.it-sudparis.eu/jvet/doc_end_user/documents/18_Alpbach/wg11/JVET-R2001-v10.zip

the latest reference software for VVC, named VTM, can be found at the following web sites:

https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/tags/VTM-9.0

2.1 History-based merger candidate derivation

History-based MVP (HMVP) merge candidates are added to a merge table after spatial MVP and TMVP. In this method, motion information of a previous codec block is stored in a table and used as MVP of a current CU. A table with a plurality of HMVP candidates is maintained during the encoding/decoding process. When a new CTU row is encountered, the table is reset (emptied). Whenever there is a non-sub-intra-coded CU, the associated motion information is added to the last entry of the table as a new HMVP candidate.

The HMVP table size S is set to 6, which indicates that a maximum of 6 history-based MVP (HMVP) candidates can be added to the table. When new motion candidates are inserted into the table, a constrained first-in-first-out (FIFO) rule is used, wherein a redundancy check is first applied to look up whether the same HMVP is present in the table. If the same HMVP is found, it is deleted from the table and then all HMVP candidates are moved forward.

HMVP candidates may be used in the merge candidate table construction process. The latest few HMVP candidates in the table are checked in order and inserted after the TMVP candidates in the candidate table. Redundancy check is applied to HMVP candidates to spatial or temporal merge candidates.

In order to reduce the number of redundancy check operations, the following simplifications are introduced:

1) Whether the number of HMPV candidates for merge table generation is set to (N < =4)? M: (8-N), wherein N indicates the number of existing candidates in the merge table and M indicates the number of available HMVP candidates in the table.

2) Once the total number of available merge candidates reaches the maximum allowed merge candidates minus 1, the build process of the merge candidate table from the HMVP is terminated.

The HMVP concept also extends to block vector prediction in intra block copy (mode).

2.2 intra block copying

Intra Block Copy (IBC), also known as current picture reference, has been adopted in HEVC screen content codec extensions (HEVC Screen Content Coding extensions, HEVC-SCC) and current VVC test model (VTM-4.0). IBC extends the concept of motion compensation from inter-frame coding to intra-frame coding. As shown in fig. 1, when IBC is applied, the current block is predicted from a reference block in the same picture. The samples in the reference block must have been reconstructed before the current block is encoded or decoded. While IBC is not efficient for most camera shot sequences, IBC exhibits significant codec gain for screen content. The reason is that there are many repeated patterns in the screen content picture, such as icons and text characters. IBC can effectively eliminate redundancy between these repeated patterns. In HEVC-SCC, an inter Coding Unit (CU) may apply IBC if a current picture is selected as a reference picture. In this case, the MV is renamed as a Block Vector (BV), which always has integer pixel precision. For compatibility with the main level HEVC, the current picture is marked as a "long-term" reference picture in a decoded picture buffer (Decoded Picture Buffer, DPB). It should be noted that inter-view reference pictures are also labeled as "long-term" reference pictures in the multi-view/3D video codec standard similarly.

After the BV finds its reference block, the prediction may be generated by copying the reference block. The residual may be obtained by subtracting the reference pixel from the original signal. The transform and quantization may then be applied as in other codec modes.

However, when the reference block is outside the picture, or overlaps with the current block, or outside the reconstruction region, or outside the active region limited by some constraints, some or all of the pixel values are not defined. Basically, there are two solutions to deal with such problems. One is to prohibit this, e.g. in terms of bitstream consistency. Another is to apply padding to those undefined pixel values. The following sections describe the solution in detail.

2.3 IBC in HEVC screen content codec extensions

In the screen content codec extension of HEVC, when a block uses the current picture as a reference, it should be ensured that the entire reference block is within the available reconstruction area, as shown in the following specification text:

the variables offsetX and offsetY are derived as follows:

offsetX＝(ChromaArrayType＝＝0)？0:(mvCLX[0]&0x72:0) (8-104)

offsetY＝(ChromaArrayType＝＝0)？0:(mvCLX[1]&0x72:0) (8-105)

the requirement of bitstream consistency is that when the reference picture is the current picture, the luma motion vector mvLX should obey the following constraints:

-when invoking the deduction procedure of the availability of the z-scan order block specified in clause 6.4.1, wherein (xCurr, yCurr) is set equal to (xCb, yCb) and the neighboring luminance positions (xNbY, yNbY) are set equal to (xpb+ (mvLX [0] > > 2) -offsetX, yPb + (mvLX [1] > > 2) -offsetY) as input, the output shall be equal to TRUE.

When invoking the deduction procedure of the availability of the z-scan order block specified in clause 6.4.1, wherein (xCurr, yCurr) is set equal to (xCb, yCb) and the neighboring luminance positions (xNbY, yNbY) are set equal to (xpb+ (mvLX [0] > > 2) +npbw-1+offsetx, ypb+ (mvLX [1] > > 2) +npbh-1+offsety) as input, the output shall be equal to TRUE.

One or both of the following conditions should be true:

the value of- (mvLX [0] > 2) +nPbW+xB1+offsetX is less than or equal to 0.

The value of- (mvLX [1] > 2) +nPbH+yB1+offsetY is less than or equal to 0.

The following conditions should be true:

(xPb+(mvLX[0]>>2)+nPbSw-1+offsetX)/CtbSizeY-xCurr/CtbSizeY<＝yCurr/CtbSizeY-(yPb+(mvLX[1]>>2)+nPbSh-1+offsetY)/CtbSizeY (8-106)

therefore, a case where the reference block overlaps with the current block or the reference block is out of picture does not occur. No reference block or prediction block needs to be filled.

2.4 IBC in VVC test model

In the current VVC test model (i.e., VTM-4.0 design), the entire reference block should be consistent with the current Coding Tree Unit (CTU) and not overlap with the current block. Thus, there is no need to fill in reference blocks or prediction blocks. The IBC flag is encoded and decoded as a prediction mode of the current CU. Thus, there are three prediction modes per CU in total: mode_intra, mode_inter, and mode_ibc.

2.4.1 IBC merge mode

In IBC merge mode, the index to an entry in the IBCmerge candidate table is parsed from the bitstream. The construction of the IBC merge table can be summarized as follows in the following order of steps:

Step 1: spatial candidates are derived.

Step 2: history-based block vector prediction (HBVP) candidates are inserted.

Step 3: the pairwise average candidates are inserted.

In the derivation of the spatial merge candidates, up to four merge candidates are selected from candidates located at the positions shown in fig. 2. The deduced sequence is A ₁ 、B ₁ 、B ₀ 、A ₀ And B ₂ . Position B ₂ Only in position A ₁ 、B ₁ ，B ₀ And A ₀ Is not available (e.g., because B) ₂ Belonging to another slice or slice) or not using IBC mode codec. In addition position A ₁ After the candidates, redundancy check is performed on the insertion of the remaining candidates to ensure that candidates having the same motion information in the table are excluded, thereby improving the codec efficiency. In order to reduce the computational complexity, not all possible candidate pairs are considered in the redundancy check mentioned. Instead, only the pairs connected by arrows in fig. 2 are considered, and candidates are only added to the table if the corresponding candidates for redundancy check do not have the same motion information.

After inserting the spatial candidates, if the size of the IBC merge table is still smaller than the size of the largest IBC merge table, IBC candidates from the HBVP table may be inserted. A redundancy check needs to be performed when inserting HBVP candidates.

Finally, the pairwise average candidates are inserted into the IBC merge table.

When the reference block identified by the merge candidate is outside the picture, or overlaps with the current block, or is outside the reconstructed region, or is outside the active region limited by some constraint, the merge candidate is referred to as an invalid merge candidate.

Note that the invalid merge candidate may be inserted into the IBC merge table.

2.4.2 IBC AMVP mode

In IBC advanced motion vector prediction (Advanced Motion Vector Prediction, AMVP) mode, the AMVP index pointing to an entry in the IBC AMVP table is parsed from the bitstream. The construction of the IBC AMVP table may be summarized as follows in the following order of steps:

step 1: spatial candidates are derived.

Check A ₀ 、A ₁ Until an available candidate is found.

Check B ₀ 、B ₁ 、B ₂ Until an available candidate is found.

Step 2: the HBVP candidate is inserted.

Step 3: zero candidates are inserted.

After inserting the spatial candidates, if the size of the IBC AMVP table is still smaller than the size of the maximum IBC AMVP table, IBC candidates from the HBVP table may be inserted.

Finally, zero candidates are inserted into the IBC AMVP table.

2.5 IBC AMVP mode in AVS3

In the av codec standard 3 (Audio Video coding Standard, avs 3), an HBVP table is maintained to store BVs of previous codec blocks. For each entry of the HBVP table, information of the block associated with the BV is stored in addition to the BV, including the width and height of the block and the coordinates of the left-hand sample of the block (relative to the left-hand sample of the picture). At the same time, a counter indicating how many times a BV is encountered is also stored in the entry. Hereinafter, the coordinates of the left-hand sample point of the block are also used as the coordinates of the block.

In the IBC AMVP mode, when an IBC AMVP (advanced motion vector prediction) table is constructed for a current block, first, BVs in the HBVP table are sequentially checked and classified into 7 classes. Each class may contain at most one BV, if multiple BVs are classified into the same class, the class uses the latest inspected BV.

For BV, if the size (e.g., width x height) of the block associated with BV is greater than or equal to 64, it is classified as class 0.

For BV, if its counter is greater than or equal to 3, it is classified into the first class.

For BV, the classification is also performed in the following order:

if its horizontal coordinates are smaller than the horizontal coordinates of the current block and its vertical coordinates are smaller than the vertical coordinates of the current block, it is classified into a fourth class, for example, the upper left class.

Otherwise, if its horizontal coordinate is greater than or equal to the horizontal coordinate of the current block plus the width of the current block, it is classified into a fifth class, for example, an upper right class.

Otherwise, if its vertical coordinates are greater than or equal to the vertical coordinates of the current block plus the height of the current block, it is classified into a sixth class, for example, the lower left class.

Otherwise, if its vertical coordinates are smaller than that of the current block, it is classified into a third class, such as the upper class.

Otherwise, if its horizontal coordinates are smaller than that of the current block, it is classified into a second class, for example, a left class.

Second, class 0-6 BVs are inserted into the AMVP table in order. If the class is not empty, then after pruning with the AMVP candidates that have been inserted, the corresponding BV may be added to the AMVP table.

In the BV estimation process, an initial BV is first determined. Then, a one-dimensional vertical BV search, a one-dimensional horizontal BV search, and a two-dimensional BV search are successfully performed to find the optimal BV. Each BV search phase starts with the same initial BV. In a one-dimensional vertical BV search, the vertical BV component is constrained to be less than or equal to y-H. Similarly, in a one-dimensional horizontal BV search, the horizontal BV component is constrained to be less than or equal to x-W.

2.6 Sample string prediction in AVS3

In M4503, a sample string prediction method is proposed in which a current video block is divided into one or more groups of samples (for example, a group of samples is referred to as a sample string), and each sample string is predicted by copying a group of pixels in a current picture including the current video block. Meanwhile, samples may not be predicted and may be directly encoded, and such samples are referred to as mismatched samples. An example is shown in fig. 5, where a CU is divided into two strings of samples, and each string is predicted by a set of pixels identified by a sample String Vector (SV). For a CU that is encoded and decoded in the sample string prediction mode, no residual is encoded and decoded. The strings of samples are scanned in traversal order and for each string of samples, a string length and a string vector are encoded. For non-matching samples, their values are directly encoded and decoded.

The SV of the sample string may be directly from the intra-frame historical motion vector prediction list updated by the historical BV and SV. When predicting the SV of the sample string from the intra-frame historical motion vector prediction list, no SV differences are further signaled.

In the sample string prediction method, the SV of a block is not used to predict its subsequent codec block.

3. Technical problem to be solved by the disclosed technical solution

1. The Block Vector (BV) of a non-immediately adjacent block is not used when constructing the IBC merge table or the AMVP table, which is inefficient.

2. In AVS3, when classifying BVs in the HBVP table, the block size (e.g., width of block x height) associated with the BV is compared to a fixed value (e.g., 64) to determine whether the BV should be classified into class 0, regardless of the size of the current block, which may be unreasonable.

3. In AVS3, very strict constraints are applied to the vertical BV component and the horizontal BV component in a one-dimensional BV search phase, which is inefficient.

4. In the string-sample prediction method, SV is not used to predict the subsequent block, which is inefficient.

5. Syntax elements related to SV and BV are designed differently.

Sv prediction list and BV prediction list are constructed in different ways.

4. Example solutions and embodiments

The following items should be taken as examples explaining the general concepts. These items should not be interpreted narrowly. Furthermore, these items may be combined in any manner.

The coordinates of the current block (e.g., the coordinates of the left-hand sample of the block) are denoted as (x, y), and the width and height of the current block are denoted as W and H, respectively. Coordinates of non-immediately adjacent samples are expressed as (x-deltaX, y-deltaY), where deltaX and deltaY are positive integers, negative integers, or 0, and non-immediately adjacent blocks are S1 x S2 (S1 and S2 are integers, e.g., s1=s2=4) blocks of covered samples. Assume that the current CTU row containing the current block starts at coordinate (0, cturowy). In AVS3, a list of intra-frame historical motion vector predictions is maintained and updated by histories BV and SV. The intra-frame history motion vector prediction list may be used for BV and SV codecs.

1. It is proposed that when predicting the BV of a current block, it is possible to use the BV of its non-immediately adjacent block.

a. It is proposed that BVs of non-immediately adjacent blocks may be inserted into an IBC merge table or/and IBC AMVP table (e.g. block vector prediction table).

b. In one example, the location of non-immediately adjacent blocks may depend on the width or/and height of the current block.

i. For example, when constructing an IBC merge table or/and an IBC AMVP table, non-immediately adjacent blocks of overlay locations (x-M, y-M), (x-M, y+H/2), (x-M, y+H), (x+W/2, y-M), (x+W, y-M) may be examined, where M is an integer, as shown in FIG. 3. For example, m=8.

1. Alternatively, in constructing the IBC merge table or/and the IBC AMVP table, non-immediately adjacent blocks of the overlay locations (x-M, y-M), (x-M, y+H-1), (x-M, y+H), (x+W-1, y-M), (x+W, y-M) may be examined.

2. Or, alternatively, non-immediately adjacent blocks of overlay locations (x-M, y), (x, y-M), (x-M, y+ 3*H/2), (x-M, y+ 2*H), (x+ 3*W/2, y-M), (x+2 x w, y-M) may be examined when constructing the IBC merge table or/and IBC AMVP table.

For example, in constructing an IBC merge table or/and an IBC AMVP table, the overlay locations (x-M-1, y-M-1), (x-M-1, y-M-1+ (H+M)/2), (x-M-1, y+H), (x-M-1+ (W+M)/2, y-M-1), (x+W, y-M-1) may be examined for non-immediately adjacent blocks, where M is an integer, as shown in FIG. 4. For example, m=8.

1. Alternatively, in constructing the IBC merge table or/and the IBC AMVP table, non-immediately adjacent blocks of the overlay locations (x-M-1, y-M-1), (x-M-1, y+H), (x+W-1, y-M-1), (x+W, y-M-1) may be examined.

2. Or, alternatively, the non-immediately adjacent blocks of the overlay locations (x-M-1, y), (x, y-M-1), (x-M-1, y-M-1+3 x (h+m)/2), (x-M-1, y+2 x h+m), (x-M-1+3 x (w+m)/2, y-M-1), (x+2 x w+m, y-M-1) may be examined when constructing the IBC merge table or/and IBC AMVP table.

c. In one example, checking how many non-immediately adjacent blocks may depend on the shape or size of the current block.

d. In one example, checking how many non-immediately adjacent blocks may depend on the coordinates of the current block.

e. In one example, when the non-immediately adjacent block and the current block are in two different CTU rows, the BV of the non-immediately adjacent block may not be used to predict the BV of the current block.

f. In one example, when the non-immediately neighboring block and the current block are in two different CTU rows, and the difference between the vertical coordinates of the current block and the current CTU row (e.g., ctuRowY) is less than or equal to Th, the BV of the non-immediately neighboring block may not be used to predict the BV of the current block.

i. For example, th is equal to 0. In this case, the current block is located in the top row of the current CTU row.

For example, th is equal to 4.

For example, th is equal to M-4, where M is defined as above to indicate the location of a non-immediately adjacent block.

g. In one example, when the non-immediately adjacent block and the current block are in two different CTU rows, the position of the non-immediately adjacent block may be cropped to be within the same CTU row as the current block, and the BV at the cropped position may be used to predict the BV of the current block.

h. In one example, when the non-immediately adjacent block and the current block are in two different CTU rows, the vertical position of the non-immediately adjacent block may be clipped to within the distance of the vertical coordinates of the current CTU row, and the BV at the clipped position may be used to predict the BV of the current block.

i. For example, the vertical position of a non-immediately adjacent block may be clipped to ctuRowY-Th2, where Th2 is an integer. For example, th2 is equal to 0, 4 or 8.

i. In one example, when the non-immediately adjacent block and the current block are in two different slices/sub-pictures, the BV of the non-immediately adjacent block may not be used to predict the BV of the current block.

i. Alternatively, the location of the non-immediately adjacent block may be cropped to be within the same slice/sub-picture as the current block, and the BV at the cropped location may be used to predict the BV of the current block.

j. In one example, when the non-immediately adjacent block and the current block are in two different CTUs or regions (e.g., rectangular regions having a width RW and a height RH), the BV of the non-immediately adjacent block may not be used to predict the BV of the current block.

i. Alternatively, the location of a non-immediately adjacent block may be cropped to be within the same CTU as the current block, and the BV at the cropped location may be used to predict the BV of the current block.

k. In one example, when a non-immediately adjacent block is outside N (e.g., n=8) adjacent lines above the current block, the BV of the block may not be used to predict the current block. In fig. 6, an example is shown, only BVs of non-immediately adjacent blocks in the "available non-immediately adjacent block area" may be used to predict BVs of the current block.

In one example, when a non-immediately adjacent neighboring block is outside of M (e.g., m=8) neighboring columns to the left of the current block, the BV of the block may not be used to predict the BV of the current block. An example is shown in fig. 6.

m. in one example, when a non-immediately adjacent block is outside the L-shaped adjacent region of the current block, the BV of the block may not be used to predict the BV of the current block.

n. in one example, when a reference block of a current block, identified by a BV of a non-immediately neighboring block, is not completely included in a current CTU or current region (e.g., a rectangular region having a width RW and a height RH) including the current block, such BV may not be used to predict the BV of the current block.

2. It is proposed that the order of checking for non-immediately adjacent blocks may depend on the relative position of the adjacent blocks with respect to the current block.

a. In one example, the order of checking for non-immediately adjacent blocks may be as follows: upper left neighbor block, upper right neighbor block, lower left neighbor block, upper neighbor block, and left neighbor block of the current block.

i. For example, the non-immediately adjacent blocks of the overlay locations (x-M, y-M), (x-M, y+H), (x+W/2, y-M), (x-M, y+H/2) are examined in the order of (x-M, y-M), (x-M, y+H/2), (x-M, y+H), (x+W/2, y-M), (x+W, y-M).

For example, the non-immediately adjacent blocks of the overlay locations (x-M-1, y-M-1), (x+W, y-M-1), (x-M+ (W+M)/2, y-M-1), (x-M-1, y-M+ (H+M)/2) are examined in the order of (x-M-1, y-M-1), (x-M-1, y-M+ (H+M)/2), (x-M-1, y+H), (x-M+ (W+M)/2, y-M-1), (x+W, y-M-1).

b. In one example, the order of checking for non-immediately adjacent blocks may be as follows: the left neighbor block, the upper left neighbor block, the upper right neighbor block, and the lower left neighbor block of the current block.

i. For example, the non-immediately adjacent blocks of the overlay locations (x-M, y-M), (x-M, y+H/2), (x-W/2, y-M), (x-M, y-M), (x+W, y-M), and (x-M, y+H) are examined in the order of (x-M, y+H/2), (x+W/2, y-M), (x+W, y-M).

For example, the non-immediately adjacent blocks of the overlay locations (x-M-1, y-M-1), (x-M-1, y-M-m+ (H+M)/2), (x-M+ (W+M)/2, y-M-1), (x-M-1, y-M-1), (x+W, y-M-1) and (x-M-1, y+H) are examined in the order of (x-M-1, y-M+ (H+M)/2), (x-M-1, y+H), (x-M+ (W+M)/2, y-M-1) and (x+W, y-M-1).

c. In one example, the order of checking for non-immediately adjacent blocks may be as follows: the left neighbor block, the upper right neighbor block, the lower left neighbor block, and the upper left neighbor block of the current block.

d. In one example, the order of checking for non-immediately adjacent blocks may be as follows: the lower left neighbor block, the upper right neighbor block, the upper neighbor block, and the upper left neighbor block of the current block.

e. In one example, the order of checking for non-immediately adjacent blocks may be as follows: the upper left neighbor block, the upper right neighbor block, and the lower left neighbor block of the current block.

f. In one example, the order of checking for non-immediately adjacent blocks may be as follows: the upper left neighbor block, the upper neighbor block, the left neighbor block, the upper right neighbor block, and the lower left neighbor block of the current block.

g. In one example, the order of checking for non-immediately adjacent blocks may be as follows: upper neighbor block, left neighbor block, upper right neighbor block, and lower left neighbor block of the current block.

h. In one example, non-immediately adjacent blocks may be divided into multiple groups, candidates in each group are checked in a predefined order, and up to N (N is an integer, e.g., n=1) candidates from one group may be inserted into the IBC merge table or/and IBCAMVP table.

i. For example, non-immediately adjacent blocks may be divided into two groups: { lower left, left } -adjacent block, { upper right, upper left } -adjacent block.

For example, non-immediately adjacent blocks may be divided into two groups: { lower left, upper left } -adjacent block, { upper right, upper } -adjacent block.

i. In one example, the order of checking for non-immediately adjacent blocks may depend on the distance from the adjacent block to the current block.

i. For example, the distance may be defined as the distance from the left-hand sample of the neighboring block to the left-hand sample of the current block.

1. The distance may be defined as the sum of the horizontal distance and the vertical distance from the left-hand sample of the neighboring block to the left-hand sample of the current block.

2. The distance may be defined as the sum of the square of the horizontal distance and the square of the vertical distance from the left-hand sample of the neighboring block to the left-hand sample of the current block.

For example, non-immediately adjacent blocks may be examined in ascending order of distance.

For example, non-immediately adjacent blocks may be examined in descending order of distance.

j. In one example, the order of checking for non-immediately adjacent blocks may depend on the size or shape of the current block.

i. For example, for a block with W > M1H (e.g., m1=2), an upper neighbor block, an upper right neighbor block, and an upper left neighbor block may be given higher priority than a lower left neighbor block and a lower left neighbor block.

For example, for a block with W > M1H (e.g., m1=2), an upper neighbor block, an upper right neighbor block, and an upper left neighbor block may be given lower priority than a lower left neighbor block and a lower left neighbor block.

For example, for a block with H > M1W (e.g., m1=2), an upper neighbor block, an upper right neighbor block, and an upper left neighbor block may be given higher priority than a lower left neighbor block and an upper left neighbor block.

For example, for a block with H > M1W (e.g., m1=2), an upper neighbor block, an upper right neighbor block, and an upper left neighbor block may be given lower priority than a lower left neighbor block and a lower left neighbor block.

k. In one example, the order of checking for non-immediately adjacent blocks may depend on the size of the adjacent blocks.

i. For example, non-immediately adjacent blocks may be examined in ascending order of size (width x height).

For example, non-immediately adjacent blocks may be examined in descending order of size (width x height). 3. It is proposed that inserting BVs of non-immediately adjacent blocks into the IBC merge table or/and IBC AMVP table may depend on availability of BVs from the HBVP table or/and immediately adjacent blocks BV.

a. In one example, BVs of non-immediately adjacent blocks are inserted after BVs from HBVP tables.

i. Alternatively, BVs that are not immediately adjacent blocks are inserted before BVs from the HBVP table.

Alternatively, BVs from non-immediately adjacent blocks are interleaved with BVs from the HBVP table.

b. In one example, BVs in non-immediately adjacent blocks are inserted after BVs of immediately adjacent blocks.

i. Alternatively, BVs that are not immediately adjacent blocks are inserted before BVs that are immediately adjacent blocks.

Alternatively, BVs that are not immediately adjacent blocks are staggered from BVs that are immediately adjacent blocks.

c. In one example, after inserting a BV from the HBVP table or/and a BV of an immediately adjacent block, when there is no empty entry in the IBC merge/AMVP table, no BV of a non-immediately adjacent block is inserted.

d. In one example, BVs that are not immediately adjacent blocks may be divided into classes in a similar manner as BVs from the HBVP table.

i. For example, non-immediately adjacent blocks may be classified into 5 classes including an upper left class, an upper right class, a lower left class, an upper class, and a left class according to the relative positions of the adjacent blocks and the current block. One or more non-immediately adjacent blocks may be classified into a class.

in one example, when the HBVP table does not contain any available BV in the first category, BV of non-immediately adjacent blocks belonging to the first category (if available) may be used instead.

1. In one example, BVs belonging to one or more non-immediately adjacent blocks of the first category may be checked in a predefined order until an available BV is found or all BVs are checked.

2. The BVs of one or more non-immediately adjacent blocks belonging to the first category may be checked in a predefined order until the BVs in the first category are inserted into the IBC merge/AMVP table or all BVs are checked.

in one example, when there is an available BV from both the HBVP table and a non-immediately adjacent block belonging to the first category, which BV to use may depend on the distance from the block associated with the BV to the current block (similar to that defined in item 2.e).

1. For example, BVs may be checked in descending order of distance until an available BV is found or all BVs are checked.

2. For example, BVs may be checked in descending distance order until they are inserted into the IBC merge/AMVP table or all BVs are checked.

3. For example, BVs may be checked in ascending distance order until an available BV is found or all BVs are checked.

4. For example, BVs may be checked in ascending distance order until they are inserted into the IBC merge/AMVP table or all BVs are checked.

e. In one example, when the HBVP table does not contain any of the available BVs in the first category (e.g., the first category may be one of

category

0, 1, 2, 3, 4, 5, or 6), BVs that are not immediately adjacent blocks may be used for the first category.

i. In one example, when the HBVP table does not contain any available BVs in the first category, the BVs of the first set of non-immediately adjacent blocks may be checked sequentially until an available BV is found, or all BVs are checked.

in one example, when the HBVP table does not contain any available BV in the second category, the BVs of the second set of non-immediately adjacent blocks may be checked sequentially until an available BV is found. When the first class is different from the second class, the first set of non-immediately adjacent blocks may be different from the second set of non-immediately adjacent blocks.

1. Alternatively, the first set of non-immediately adjacent blocks may be the same as the second set of non-immediately adjacent blocks.

in one example, if a non-immediately adjacent block belongs to a first non-immediately adjacent block group, it may not belong to a second non-immediately adjacent block group that is different from the first non-immediately adjacent block group.

in one example, when a BV of a first non-immediately adjacent block is used for a first class, the BV may not be checked again for a second class.

1. Alternatively, when a BV of a first non-immediately adjacent block is checked for a first class, the BV may not be checked again for a second class.

f. In one example, before inserting a BV from a non-immediately adjacent block, the BV may be compared to one or more BVs already inserted in the IBC merge/AMVP table.

i. In one example, if a BV from a non-immediately adjacent block is the same as one of the one or more BVs that have been inserted into the IBC merge/AMVP table, it is not inserted into the IBC merge/AMVP.

in one example, if a BV from a non-immediately adjacent block is similar to one of the one or more BVs that have been inserted into the IBC merge/AMVP table, it is not inserted into the IBC merge/AMVP.

in one example, such comparison may be made for one or more BVs from non-immediately adjacent blocks.

Alternatively, no comparison is made.

4. It is proposed that it is possible to decide whether or not to divide BV from the HBVP table into N (N is a non-negative integer, e.g., n=0) class according to the block size (denoted BvBlkSize) associated with BV and the size of the current block (denoted CurBlkSize).

a. In one example, BV may be classified as class N when BvBlkSize is greater than or equal to a factor CurBlkSize, where the factor is a positive number. For example, the factor is equal to 1.

b. In one example, when BvBlkSize is greater than a factor CurBlkSize, BV may be classified as class N, where the factor is a positive number. For example, the factor is equal to 1.

c. In one example, BV may be classified as class N when BvBlkSize is less than or equal to a factor CurBlkSize, where the factor is a positive number. For example, the factor is equal to 1.

d. In one example, when BvBlkSize is less than a factor CurBlkSize, BV may be classified as an nth class, where the factor is a positive number. For example, the factor is equal to 1.

e. In one example, when BvBlkSize is equal to a factor currblksize, BV may be classified as the nth class, where the factor is a positive number. For example, the factor is equal to 1.

f. Alternatively, it may be decided whether a BV from the HBVP table should be classified as the nth class based on the block size associated with the BV and the size of the current block.

5. It is proposed that the range of BV components in a one-dimensional BV search may depend only on the coordinates of the current block.

a. Alternatively, or in addition, the range of BV components in a one-dimensional BV search may not depend on the size of the current block.

b. In one example, in a one-dimensional vertical BV search, the vertical BV component is constrained to be less than or equal to y-N1 (N1 is an integer, e.g., n1=0, 8, or-8).

c. In one example, in a one-dimensional horizontal BV search, the horizontal BV component is constrained to be less than or equal to x-N2 (N2 is an integer, such as n2=0, 8, or-8).

d. Alternatively, the range of BV components in a one-dimensional BV search may depend on both the size and coordinates of the current block.

i. For example, in a one-dimensional vertical BV search, the vertical BV component is constrained to be less than or equal to y+h-N1 (N1 is an integer, e.g., n1=0, 8, or-8).

For example, in a one-dimensional horizontal BV search, the horizontal BV component is constrained to be less than or equal to x+w-N2 (N2 is an integer, such as n2=0, 8, or-8).

Or, alternatively, the range of BV components may be further dependent on the starting BV (startBvX, startBvY).

1. For example, in a one-dimensional vertical BV search, the vertical BV component is constrained to be less than or equal to y+startbvy+k1H-N1. N1 is an integer, such as n1=0, 8 or-8, and K1 is an integer, such as k1=2, 3, 4.

2. For example, in a one-dimensional horizontal BV search, the horizontal BV component is constrained to be less than or equal to x+startbvx+k2 x W-N2. N2 is an integer, such as n2=0, 8 or-8, and K2 is an integer, such as k2=2, 3, 4.

6. It is proposed that the SV (sample string vector) of a block can be used to predict BV or/and SV of its subsequent codec block.

a. In one example, one representative SV may be selected for a block that is encoded and decoded in the sample string prediction mode, and the one representative SV may be stored for the block.

i. For example, the SV of the first sample string may be selected as the representative SV.

For example, the SV of the last sample string may be selected as the representative SV.

For example, the SV of a sample string covering the center position of a block may be selected as a representative SV.

For example, the SV of the sample string covering the representative position of the block may be selected as the representative SV.

For example, the SV of the sample string having the greatest length may be selected as the representative SV. If there are multiple strings of samples having the maximum length, the BV of one of the multiple strings of samples may be selected as the representative SV.

For example, the average of BVs of a plurality of strings of spots may be used as the representative SV.

b. In one example, multiple SVs may be stored for blocks that are encoded and decoded in the sample string prediction mode.

i. For example, for each MxN sub-block, SV of a sample string covering most of the samples of the sub-block is stored for the sub-block. If there are multiple strings of samples covering most of the samples of the sub-block, then a BV of one of the multiple strings of samples may be stored for the sub-block.

7. It is proposed that when predicting the BV of a block, its SV of a non-immediately adjacent block may be used.

a. It is proposed that SVs of non-immediately adjacent blocks may be inserted into the IBC merge list or/and IBC AMVP list.

b. The method described in bulleted 1 may be used to insert SVs into the IBC merge list or/and IBC AMVP list.

c. In addition, or alternatively, SVs immediately adjacent to the neighboring block may be used to predict BV of the current block.

8. It is proposed that when predicting the SV of a block, it is possible to use BV of its non-immediately adjacent block.

a. In addition, or when predicting the SV of a block, the SV of its non-immediately adjacent block may be used.

b. In addition, or when predicting the SV of a block, the SV or/and BV of its immediately adjacent block may be used.

9. Unified BV and SV signaling is presented.

a. In one example, a syntax element for indicating SV is used to indicate BV.

b. In one example, a syntax element for indicating BV is used to indicate SV.

c. In one example, a syntax is signaled to indicate whether an SV or BV is included in a predefined SV/BV set.

i. For example, the predefined SV/BV set may depend on whether the syntax to be encoded is SV or BV.

For example, the predefined set of SVs may depend on whether the current sample string contains all or part of the samples of the current block.

For example, the predefined set of SVs may depend on whether the current sample string starts from an even or an odd row.

For example, for BV or SV of a sample string containing all samples of a block, the predefined SV/BV set may include { (0, -H), (-W, 0) }, where W and H are the width and height, respectively, of the current block.

1. Alternatively, the predefined set of SVs/BVs may comprise (0, -H).

2. Alternatively, the predefined set of SVs/BVs may comprise (0, -H) or (-W, 0).

For example, for SVs of a sample string containing only partial samples of a block, a predefined set of SVs may include { (0, -1), (W, 0) }.

1. Alternatively, the predefined set of SVs/BVs may include { (0, -1) }.

For example, for SVs of a sample string starting from an even row of blocks, the predefined set of SVs may include { (0, -1), (-W, 0) }.

For example, for SVs of a sample string starting from an odd row of blocks, the predefined set of SVs may include { (0, -1), (W, 0) }.

In addition, or when an SV or BV is included in a predefined set of SVs/BVs, wherein the set contains more than one entry, an index indicating which SV/BV in the set is used may be further signaled.

d. In one example, for a block that is encoded and decoded in IBC mode, a syntax may be signaled to indicate whether the BV of the block is from a BV prediction list.

i. In addition, or only when such syntax indicates that the BV of the block is from the BV prediction list, the BV index indicating which BV in the BV prediction list is used by the current block may be signaled.

e. In one example, for blocks that are encoded in IBC mode, the intra-frame historical motion vector prediction list may be used directly to encode BV and the BV prediction list may not be constructed.

i. For example, the BV index may be signaled to indicate which BV/SV in the intra historical motion vector prediction list is used for the block.

1. In addition, or if BV/SV in the intra historical motion vector prediction list is used for the block, the BV difference may not be signaled and may be derived to zero.

2. In addition, or alternatively, the BV index is constrained in the coherence bitstream to be less than the number of elements in the intra historical motion vector prediction list.

f. In one example, for a block that is encoded and decoded in IBC mode, a syntax may be signaled to indicate whether the BV of the block is from an intra historical motion vector prediction list.

i. In addition, or alternatively, the BV index may be signaled only if such syntax indicates that the BV of the block is from an intra historical motion vector prediction list.

in addition, or when there are no elements in the intra-frame historical motion vector prediction list, such syntax is constrained to false, i.e., the BV of the block is constrained to not be from the intra-frame historical motion vector prediction list.

g. In one example, BV (0, 0) may not be inserted into the BV prediction list.

10. A unified BV prediction list construction process and an SV prediction list construction process are provided.

a. In one example, the SV prediction list construction process is aligned with the BV prediction list construction process.

b. In one example, the BV prediction list construction process is aligned with the SV prediction list construction process.

11. It is proposed that the SV (derived in sample string prediction mode) or/and BV (derived in intra block copy mode) of the parent block or/and at least one sibling block (if available) of the current block may be used for the BV estimation process of the current block.

a. In one example, the SV or/and BV of the parent block or/and at least one sibling block (if available) of the current block may be considered as a candidate BV in the BV estimation process.

i. In addition, alternatively, the SV or/and BV of the parent block or/and at least one sibling block (if available) of the current block may be tested before all other candidate BVs.

in addition, or if the minimum cost achieved by the SV or/and BV of the parent block or/and at least one sibling block (if available) of the current block is less than a threshold, all other candidate BVs may be skipped.

b. In addition, or alternatively, the SV or/and BV of at least one descendant of at least one sibling block of the current block may be used for the BV estimation process of the current block.

12. It is proposed that when an intra block copy mode is performed before a sample string prediction mode of a block, BV obtained in the intra block copy mode can be used for SV estimation process of a current block and vice versa.

a. In one example, N (e.g., n=1, 2,3, etc.) BVs that achieve the smallest cost in intra block copy mode may be considered as candidate SVs in the SV estimation process.

i. In addition, or alternatively, N BVs may be tested before all other candidate SVs.

Additionally, or alternatively, if the minimum cost achieved by N BVs during SV estimation is less than a threshold, all other candidate SVs may be skipped.

N BVs may be considered in the SV estimation process before checking the sample string prediction mode even if the intra block copy mode is not the current best mode.

b. In one example, when the sample string contains all the samples of the block, N BVs that achieve the smallest cost in intra block copy mode may be used for the SV estimation process.

c. In one example, when the sample string contains partial samples of a block, N BVs that achieve the smallest cost in intra block copy mode may be used in the SV estimation process.

d. In one example, N may be different for a first sample string containing all samples of a block and a second sample string containing a portion of the samples of the block.

e. N may be different for different sized blocks.

13. It is proposed to constrain the BV index (e.g., cbvp_index indicating which BV in the BV prediction list is used by the current block) to be less than the number of BVs in the BV prediction list in the coherency bitstream.

a. In one example, when there are N (N is an integer) BVs in the BV prediction list, the BV index is constrained to be less than or equal to N-1.

b. In addition, or when there is no element in the BV prediction list, it is proposed to restrict an indication of whether the BV of the current block is from the BV prediction list to false, i.e., the BV of the current block is restricted to not be from the BV prediction list.

14. It is proposed to constrain the SV index (e.g., sv_direction_idx indicating which SV/BV in the intra historical motion vector prediction list is used by the sample string) to be smaller than the number of elements in the intra historical motion vector prediction list in the consistent bitstream.

a. In addition, or when there is no element in the intra-history motion vector prediction list, it is proposed to constrain an indication (e.g., sv_direction_flag) of whether or not the SV of the current sample string is from the intra-history motion vector prediction list to false, i.e., the SV of the current sample string is constrained to not be from the intra-history motion vector prediction list.

5. Examples

The following are some example embodiments of some inventive aspects summarized above in this section, to which the VVC specification may be applied. Most relevant parts that have been added or modified are indicated in bold, underlined and italics, e.g. "use a"

", some deleted parts are indicated by italics with strikers, such as" based on ++>

B). Still other variations are editable themselves and thus are not highlighted.

5.1 example #1

9.5.6.3.2 section vector prediction

First, the category blockmotionclass Y (y=0 to 6) is constructed according to the following steps:

a) The number of selectable block copy intra prediction history motion information candidates, numallowedhbvpc and, is initialized to Min (CntHbvp, numOfHbvpCand).

b) If numalowedhbvpc and is equal to 0 and both MvPredXBv and MvPredYBv are equal to 0, the export process is ended, otherwise the following steps continue.

c) Otherwise, Y is from 0 to 6, and the number of motion information candidates cntClassY for intra prediction of block copy in each class of blockMotionClassY is initialized to 0. For x=0 to numallowedhbvpand-1, hbvpcoandatelist [ X ] is classified as follows:

1) If widthCandX is greater than or equal to the weightcandx

HbvpCanandateList [ X ]]To blockMotionClass0 and add the value of cntClassY to 1;

2) If cntccandx is greater than or equal to 3, then HbvpCandidateList [ X ] is added to blockMotionClass1 and the value of cntClassY is added to 1;

3) If xCandX is less than xCur and yCandX is less than yCur, then HbvpCandidateList [ X ] is added to blockMotionClass4 and the value of cntClassY is added to 1;

4) Otherwise, if xCandX is greater than or equal to xcur+widthcur, adding HbvpCandidateList [ X ] to blockMotionClass5, and adding the value of cntClassY to 1;

5) Otherwise, if yCandX is greater than or equal to yCur+HeghtCur, then adding HbvpCandidateList [ X ] to blockMotionClass6, and adding the value of cntClassY to 1;

6) Otherwise, if yCandX is less than yCur, then HbvpCandidateList [ X ] is added to blockMotionClass3 and the value of cntClassY is added to 1;

7) Otherwise HbvpCandidateList [ X ] is added to blockMotionClass2 and the value of cntClassY is added to 1.

Then, the block vector of the intra copy motion information in the blockMotionClassY is labeled bvClassY, and the candidate class table cbvpcanddataelist is derived according to the following method:

a) cntCbvp, cntCheck and Y are initialized to 0;

b) If the value of cntClassY is greater than 0, the following steps are performed:

1) Initializing candIdx to 0;

2) If cntCbvp is equal to 0, then the block vector of cbvcanddatelist [ cntCbvp ] is equal to bvClassY, cntCbwp is incremented by 1, and step c) is performed;

3) Otherwise, if the block vector of CbvpCandidateList [ candIdx ] is the same as bvClassY, go to step c);

4) Otherwise, adding 1 to candIdx, and if candIdx is smaller than cntCheck, turning to step 3);

5) Otherwise, the block vector of CbvpCanddateList [ cntCbvp ] is equal to bvClassY, and cntCbvp is incremented by 1.

c) The value of Y is added to 1 and then the following steps are performed:

1) If the value of Y is less than or equal to 2, setting the value of cntCheck to cntCbvp, and then continuing to execute step b);

2) Otherwise, if Y is less than 7, continuing to execute the step b);

3) Otherwise, end

The process of deriving CbvpCanddateList.

/>

If cntCbvp is equal to 0, both MvPredXBv and MvPredYBv are equal to 0. Otherwise, mvPredXBv and MvPredYBv are equal to the abscissa and ordinate, respectively, of CbvpCandidateList [ CbvpIndex ].

Fig. 7 is a block diagram illustrating an example video processing system 7000 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of system 7000. The system 7000 may include an input 7002 for receiving video content. The video content may be received in an original or uncompressed format, such as 8 or 10 bit multi-component pixel values, or may be received in a compressed or encoded format. Input 7002 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as ethernet, passive Optical Network (PON), etc., and wireless interfaces such as Wi-Fi or cellular interfaces.

The system 7000 can include a codec component 7004 that can implement the various codec or encoding methods described in this document. The codec component 7004 may reduce the average bit rate of the video from the input 7002 to the output of the codec component 7004 to produce a codec representation of the video. Thus, codec technology is sometimes referred to as video compression or video transcoding technology. The output of codec component 7004 may be stored or transmitted via a communication connection as represented by component 7006. The stored or communicatively transmitted bit stream (or codec) representation of the video received at the input 7002 may be used by the component 7008 to generate pixel values or displayable video that is transmitted to the display interface 7010. The process of generating user-viewable video from a bitstream representation is sometimes referred to as video decompression. Further, while certain video processing operations are referred to as "codec" operations or tools, it will be appreciated that a codec tool or operation is used at the encoder and that a corresponding decoding tool or operation that inverts the encoding results will be performed by the decoder.

Examples of the peripheral bus interface or the display interface may include a Universal Serial Bus (USB) or a High Definition Multimedia Interface (HDMI) or a display port (Displayport), or the like. Examples of storage interfaces include SATA (serial advanced technology attachment), PCI, IDE interfaces, and the like. The techniques described in this document may be embodied in various electronic devices such as mobile phones, laptops, smartphones, or other devices capable of performing digital data processing and/or video display.

Fig. 8 is a block diagram of a video processing apparatus 8000. Device 8000 may be used to implement one or more of the methods described herein. The apparatus 8000 may be embodied in a smart phone, tablet, computer, internet of things (IoT) receiver, or the like. The apparatus 8000 may include one or more processors 8002, one or more memories 8004, and video processing hardware 8006. Processor(s) 8002 may be configured to implement one or more of the methods described in this document (e.g., in fig. 12-21). Memory(s) 8004 may be used to store data and code for implementing the methods and techniques described herein. Video processing hardware 8006 may be used to implement some of the techniques described in this document in hardware circuitry. In some embodiments, hardware 8006 may be partially or wholly located in one or more processors 8002, such as a graphics processor.

Fig. 9 is a block diagram illustrating an example video codec system 100 that may utilize the techniques of this disclosure.

As shown in fig. 9, the video codec system 100 may include a source device 110 and a target device 120. The source device 110 generates encoded video data, wherein the source device 110 may be referred to as a video encoding device. The target device 120 may decode the encoded video data generated by the source device 110, wherein the target device 120 may be referred to as a video decoding device. Source device 110 may include a video source 112, a video encoder 114, and an input/output (I/O) interface 116.

Video source 112 may include sources such as a video capture device, an interface to receive video data from a video content provider, and/or a computer graphics system to generate video data, or a combination of these sources. The video data may include one or more pictures. Video encoder 114 encodes video data from video source 112 to generate a bitstream. The bitstream may include a sequence of bits that form a codec representation of the video data. The bitstream may include the encoded pictures and related data. A codec picture is a codec representation of a picture. The related data may include sequence parameter sets, picture parameter sets, and other syntax structures. The I/O interface 116 may include a modulator/demodulator (modem) and/or a transmitter. The encoded video data may be transmitted directly to the target device 120 via the I/O interface 116 over the network 130 a. The encoded video data may also be stored on storage medium/server 130b for access by target device 120.

The target device 120 may include an I/O interface 126, a video decoder 124, and a display device 122.

The I/O interface 126 may include a receiver and/or a modem. The I/O interface 126 may obtain encoded video data from the source device 110 or the storage medium/server 130 b. Video decoder 124 may decode the encoded video data. The display device 122 may display the decoded video data to a user. The display device 122 may be integrated with the target device 120 or may be external to the target device 120, the target device 120 being configured to interface with an external display device.

Video encoder 114 and video decoder 124 may operate in accordance with video compression standards, such as the High Efficiency Video Coding (HEVC) standard, the Versatile Video Coding (VVC) standard, and other current and/or other standards.

Fig. 10 is a block diagram illustrating an example of a video encoder 200, which video encoder 200 may be video encoder 114 in system 100 shown in fig. 9.

Video encoder 200 may be configured to perform any or all of the techniques of this disclosure. In the example shown in fig. 10, the video encoder 200 includes a plurality of functional components. The techniques described in this disclosure may be shared among the various components of video encoder 200. In some examples, the processor may be configured to perform any or all of the techniques described in this disclosure.

Functional components of the video encoder 200 may include a segmentation unit 201, a prediction unit 202, which may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205, and intra prediction unit 206, a residual generation unit 207, a transform unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213, and an entropy encoding unit 214.

In other examples, video encoder 200 may include more, fewer, or different functional components. In one example, the prediction unit 202 may include an Intra Block Copy (IBC) unit. The IBC unit may perform prediction in an IBC mode, in which at least one reference picture is a picture in which the current video block is located.

Furthermore, some components, such as the motion estimation unit 204 and the motion compensation unit 205, may be highly integrated, but are represented separately in the example of fig. 10 for descriptive purposes.

The segmentation unit 201 may segment a picture into one or more video blocks. The video encoder 200 and the video decoder 300 may support various video block sizes.

The mode selection unit 203 may, for example, select one of the codec modes (intra or inter) based on the error result and provide the resulting intra or inter codec block to the residual generation unit 207 to generate residual block data and the reconstruction unit 212 reconstructs the codec block to use as a reference picture. In some examples, mode selection unit 203 may select a Combination of Intra and Inter Prediction (CIIP) modes, where the prediction is based on an inter prediction signal and an intra prediction signal. In the case of inter prediction, the mode selection unit 203 may also select the resolution (e.g., sub-pixel or integer-pixel precision) of the motion vector for the block.

In order to perform inter prediction on the current video block, the motion estimation unit 204 may generate motion information of the current video block by comparing one or more reference frames from the buffer 213 with the current video block. The motion compensation unit 205 may determine a predicted video block of the current video block based on motion information of pictures other than the picture associated with the current video block from the buffer 213 and the decoding samples.

The motion estimation unit 204 and the motion compensation unit 205 may perform different operations on the current video block, e.g., depending on whether the current video block is in an I-slice, a P-slice, or a B-slice.

In some examples, motion estimation unit 204 may perform unidirectional prediction on the current video block, and motion estimation unit 204 may search for a reference picture of table 0 or 1 for a reference video block of the current video block. Motion estimation unit 204 may then generate a reference index indicating a reference picture in table 0 or table 1 that includes the reference video block and a motion vector indicating a spatial displacement between the current video block and the reference video block. The motion estimation unit 204 may output the reference index, the prediction direction indicator, and the motion vector as motion information of the current video block. The motion compensation unit 205 may generate a predicted video block of the current block based on the reference video block indicated by the motion information of the current video block.

In other examples, motion estimation unit 204 may perform bi-prediction on the current video block, motion estimation unit 204 may search for a reference picture in table 0 for a reference video block of the current video block, and may also search for a reference picture in table 1 for a reference video block of the current video block. Motion estimation unit 204 may then generate reference indices indicative of the reference pictures in table 0 and table 1, including the reference video block and a motion vector indicative of the spatial displacement between the reference video block and the current video block. The motion estimation unit 204 may output the reference index and the motion vector of the current video block as motion information of the current video block. The motion compensation unit 205 may generate a predicted video block of the current video block based on the reference video block indicated by the motion information of the current video block.

In some examples, motion estimation unit 204 may output the complete set of motion information for use in a decoding process of a decoder.

In some examples, motion estimation unit 204 may not output the complete set of motion information for the current video. Instead, motion estimation unit 204 may signal motion information for the current video block with reference to motion information for another video block. For example, motion estimation unit 204 may determine that the motion information of the current video block is sufficiently similar to the motion information of neighboring video blocks.

In one example, motion estimation unit 204 may indicate a value in a syntax structure associated with the current video block that indicates to video decoder 300 that the current video block has the same motion information as another video block.

In another example, motion estimation unit 204 may identify another video block and a motion vector difference (motion vector difference, MVD) in a syntax structure associated with the current video block. The motion vector difference represents the difference between the motion vector of the current video block and the motion vector of the indicated video block. The video decoder 300 may determine a motion vector of the current video block using the indicated motion vector of the video block and the motion vector difference.

As described above, the video encoder 200 may predictively signal motion vectors. Two examples of prediction signaling techniques that may be implemented by video encoder 200 include Advanced Motion Vector Prediction (AMVP) and merge mode signaling.

The intra prediction unit 206 may perform intra prediction on the current video block. When the intra prediction unit 206 performs intra prediction on a current video block, the intra prediction unit 206 may generate prediction data of the current video block based on decoding samples of other video blocks in the same picture. The prediction data of the current video block may include a prediction video block and various syntax elements.

The residual generation unit 207 may generate residual data of the current video block by subtracting (e.g., indicated by a negative sign) the predicted video block(s) of the current video block from the current video block. The residual data of the current video block may include residual video blocks corresponding to different sample components of samples in the current video block.

In other examples, for the current video block, for example, in the skip mode, there may be no residual data of the current video block, and the residual generation unit 207 may not perform the subtraction operation.

Transform processing unit 208 may generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to the residual video block associated with the current video block.

After transform processing unit 208 generates a transform coefficient video block associated with the current video block, quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (quantization parameter, QP) values associated with the current video block.

The inverse quantization unit 210 and the inverse transform unit 211 may apply inverse quantization and inverse transform, respectively, to the transform coefficient video block to reconstruct a residual video block from the transform coefficient video block. The reconstruction unit 212 may add the reconstructed residual video block to corresponding samples from the one or more prediction video blocks generated by the prediction unit 202 to generate a reconstructed video block associated with the current block that is stored in the buffer 213.

After the reconstruction unit 212 reconstructs the video blocks, a loop filtering operation may be performed to reduce video block artifacts in the video blocks.

The entropy encoding unit 214 may receive data from other functional components of the video encoder 200. When the entropy encoding unit 214 receives data, the entropy encoding unit 214 may perform one or more entropy encoding operations to generate entropy encoded data and output a bitstream that includes the entropy encoded data.

Fig. 11 is a block diagram illustrating an example of a video decoder 300, which video decoder 300 may be video decoder 114 in system 100 shown in fig. 9.

The video decoder 300 may be configured to perform any or all of the techniques of this disclosure. In the example shown in fig. 11, the video decoder 300 includes a plurality of functional components. The techniques described in this disclosure may be shared among the various components of video decoder 300. In some examples, the processor may be configured to perform any or all of the techniques described in this disclosure.

In the example shown in fig. 11, the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra prediction unit 303, an inverse quantization unit 304, an inverse transformation unit 305, and a reconstruction unit 306 and a buffer 307. In some examples, video decoder 300 may perform a decoding channel that is generally opposite to the encoding channel described with respect to video encoder 200 (fig. 8).

The entropy decoding unit 301 may take the encoded bitstream. The encoded bitstream may include entropy encoded video data (e.g., encoded blocks of video data). The entropy decoding unit 301 may decode the entropy-encoded video data, and from the entropy-decoded video data, the motion compensation unit 302 may determine motion information including a motion vector, a motion vector precision, a reference picture table index, and other motion information. For example, the motion compensation unit 302 may determine such information by performing AMVP and merge modes.

The motion compensation unit 302 may generate a motion compensation block, and may perform interpolation based on the interpolation filter. An identifier for the interpolation filter at sub-pixel accuracy may be included in the syntax element.

Motion compensation unit 302 may calculate an interpolation of sub-integer pixels of the reference block using interpolation filters used by video encoder 200 during encoding of the video block. The motion compensation unit 302 may determine an interpolation filter used by the video encoder 200 according to the received syntax information and generate a prediction block using the interpolation filter.

The motion compensation unit 302 may use some syntax information to determine the size of the blocks used to encode the frame(s) and/or slice(s) of the encoded video sequence, partition information describing how each macroblock of a picture of the encoded video sequence is partitioned, mode indications how each partition is encoded, one or more reference frames (and reference frame tables) for each inter-codec block, and other information to decode the encoded video sequence.

The intra prediction unit 303 may form a prediction block from spatially neighboring blocks using, for example, an intra prediction mode received in a bitstream. The inverse quantization unit 303 inversely quantizes, i.e., dequantizes, the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 301. The inverse transformation unit 303 applies an inverse transformation.

The reconstruction unit 306 may add the residual block to a corresponding prediction block generated by the motion compensation unit 202 or the intra prediction unit 303 to form a decoded block. Deblocking filters may also be applied to filter the decoded blocks, if desired, to remove blocking artifacts. The decoded video blocks are then stored in a buffer 307 which provides a reference block for subsequent motion compensation/intra prediction and also generates decoded video for presentation on a display device

Fig. 12-21 illustrate example methods that can implement the technical solutions described above in embodiments such as that illustrated in fig. 7-11.

Fig. 12 shows a flow chart of an example method 1200 of video processing. The method 1200 includes, at operation 1210, determining availability of a non-immediately adjacent block for Block Vector (BV) based encoding and decoding according to a rule based on an overlap condition between a current region including the current video block and an adjacent region including the non-immediately adjacent block of the current video block for a transition between a video including the current video block and a bitstream of the video.

The method 1200 includes, at operation 1220, performing a conversion based on the determination.

Fig. 13 illustrates a flow chart of an example method 1300 of video processing. The method 1300 includes, at operation 1310, determining, for a transition between a video including a current video block and a bitstream of the video, whether a sample string vector of the current video block is used to predict a block vector and/or a string vector of a subsequent video block associated with the current video block.

The method 1300 includes, at operation 1320, performing a conversion based on the determination.

Fig. 14 shows a flowchart of an example method 1400 of video processing. The method 1400 includes, at operation 1410, determining that one or more sample string vectors that are not immediately adjacent to a block are available for prediction of a block vector of a current video block for a transition between a video comprising the current video block and a bitstream of the video.

The method 1400 includes, at operation 1420, performing a conversion based on the determination.

Fig. 15 illustrates a flow chart of an example method 1500 of video processing. The method 1500 includes, at operation 1510, determining that one or more block vectors that are not immediately adjacent to a block are available for prediction of a sample string vector of a current video block for a transition between a video comprising the current video block and a bitstream of the video.

The method 1500 includes, at operation 1520, performing a conversion based on the determination.

Fig. 16 illustrates a flow chart of an example method 1600 of video processing. Method 1600 includes, at operation 1610, performing a conversion between a video including a current video block and a bitstream of the video in which one or more syntax elements are used to indicate a block vector or a sample string vector for predicting the current video block according to a format rule.

Fig. 17 shows a flow chart of an example method 1700 of video processing. The method 1700 includes, at operation 1710, performing a conversion between a video including a video block and a bitstream of the video, the conversion conforming to a rule specifying that the same list construction process be used during the conversion to determine a block vector prediction list or a sample string vector prediction list of the video block.

Fig. 18 shows a flow chart of an example method 1800 of video processing. The method 1800 includes, at operation 1810, determining, for a transition between a video including a current video block and a bitstream of the video, whether a sample string vector or a block vector of an associated block including a parent block or at least one sibling block of the current video block is used to predict a block vector of the current video block according to a rule, the sample string vector being derived in a sample string prediction mode and the block vector being derived in an intra block copy mode.

The method 1800 includes, at operation 1820, performing a conversion based on the determination.

Fig. 19 illustrates a flow chart of an example method 1900 of video processing. Method 1900 includes, at operation 1910, determining that at least one block vector generated from an intra block copy mode for encoding and decoding a current video block is used for an estimation process of a sample string vector of the current video block for a transition between a video including the current video block and a bitstream of the video, the current video block being further encoded using a sample string prediction mode.

Method 1900 includes, at operation 1920, performing a conversion based on the determination.

Fig. 20 shows a flow chart of an example method 2000 of video processing. The method 2000 includes, at operation 2010, performing a conversion between a video including a current video block and a bitstream of the video, the bitstream conforming to a format rule specifying that a block vector index indicating a block vector in a block vector prediction list used by the current video block is constrained to be less than a number of block vectors in the block vector prediction list.

Fig. 21 illustrates a flow chart of an example method 2100 of video processing. Method 2100 includes, at operation 2110, performing a conversion between a video including a current video block and a bitstream of the video, the bitstream conforming to a format rule specifying that a sample string vector index indicating a block vector or a sample string vector in an intra-frame historical motion vector prediction list used by the current video block is constrained to be less than a number of elements in the intra-frame historical motion vector prediction list.

The following solutions illustrate example embodiments of the techniques discussed in the previous section (e.g., items 1-14).

A list of solutions preferred by some embodiments is provided next.

S1, a video processing method comprises the following steps: for a transition between a video including a current video block and a bitstream of the video, determining availability of a non-immediately adjacent block for Block Vector (BV) based encoding and decoding according to a rule based on an overlap condition between a current region including the current video block and an adjacent region including the non-immediately adjacent block of the current video block; and performing a conversion based on the determination.

S2. the method according to solution S1, wherein non-immediately adjacent neighboring blocks are available as the overlap condition specifies that the current region does not overlap with the neighboring region, and wherein the current region is a first Codec Tree Unit (CTU) and the neighboring region is a second CTU.

S3, according to the method of the solution S1, wherein the non-adjacent block is available because the block vector of the non-adjacent block identifies the reference block of the current video block and the overlap condition specifies that the reference block and the current region partially overlap.

S4. the method according to solution S1, wherein, since the neighborhood comprises N adjacent lines directly above the current video block and excludes lines above the N adjacent lines, non-immediately adjacent blocks are available, and wherein N is a positive integer.

S5. the method according to solution S1, wherein, since the neighborhood comprises N adjacent columns directly to the left of the current video block and excludes columns to the left of the N adjacent columns, non-immediately adjacent blocks are available, and wherein N is a positive integer.

S6. the method according to solution S4 or S5, wherein n=8.

S7. the method according to solution S1, wherein, since the neighboring area comprises a first L-shaped neighboring area immediately adjacent to the current video block and excludes a second L-shaped area, non-immediately adjacent neighboring blocks are available, and wherein the first L-shaped area is between the second L-shaped area and the current video block.

S8, a video processing method comprises the following steps: for a transition between a video including a current video block and a bitstream of the video, determining whether a sample string vector of the current video block is used to predict a block vector and/or a string vector of a subsequent video block associated with the current video block; and performing a conversion based on the determination.

S9. the method according to solution S8, wherein the current video block is encoded and decoded using a sample string prediction mode, and wherein the sample string vector is a representative string vector stored for the current video block.

S10. the method according to solution S9, wherein the representative string vector comprises a string vector of the first sample string.

S11. the method according to solution S9, wherein the representative string vector comprises a string vector of the last sample string.

S12. the method according to solution S9, wherein the representative string vector comprises a string vector of strings of samples covering the center position of the current video block.

S13. the method according to solution S9, wherein the representative string vector comprises a string vector of strings of samples having a maximum length.

S14. the method according to solution S8, wherein the current video block is encoded and decoded using a sample string prediction mode, and wherein a plurality of string vectors are stored for the current video block.

S15, a video processing method comprises the following steps: for a transition between a video including a current video block and a bitstream of the video, determining that one or more sample string vectors that are not immediately adjacent to the block are available for prediction of a block vector of the current video block; and performing a conversion based on the determination.

S16. the method according to solution S15, wherein one or more sample string vectors are inserted into an Inter Block Copy (IBC) merge list or an IBC Advanced Motion Vector Prediction (AMVP) list.

S17. the method according to solution S15, wherein the prediction of the block vector of the current video block is further based on one or more sample string vectors of immediately adjacent blocks.

S18, a video processing method comprises the following steps: for a transition between a video including a current video block and a bitstream of the video, determining that one or more block vectors that are not immediately adjacent to the block are available for prediction of a sample string vector of the current video block; and performing a conversion based on the determination.

S19. the method according to solution S18, wherein the prediction of the sample string vector of the current video block is further based on one or more sample string vectors of non-immediately adjacent blocks.

S20. the method according to solution S18, wherein the prediction of the sample string vector of the current video block is further based on one or more block vectors or one or more sample string vectors of immediately adjacent blocks.

S21. a method of video processing, comprising performing a conversion between a video comprising a current video block and a bitstream of the video, wherein one or more syntax elements are used in the bitstream to indicate a block vector or a sample string vector for predicting the current video block according to a format rule.

S22. the method according to solution S21, wherein the format rules specify that the one or more syntax elements comprise syntax elements indicating a sample string vector.

S23. the method according to solution S21, wherein the format rules specify that the one or more syntax elements comprise syntax elements indicating a block vector.

S24. the method according to solution S21, wherein the format rules specify that the one or more syntax elements comprise syntax elements indicating whether a block vector or a sample string vector is comprised in a predefined set of block vectors or a predefined set of sample string vectors, respectively.

S25. the method according to solution S21, wherein the current video block is encoded and decoded in Intra Block Copy (IBC) mode, and wherein the format rules specify that the one or more syntax elements include syntax elements indicating whether the block vector is from a block vector prediction list.

S26. the method according to solution S21, wherein the current video block is encoded and decoded in Intra Block Copy (IBC) mode, and wherein the block vector is encoded and decoded based on an intra historical motion vector prediction list.

S27. the method according to solution S21, wherein the current video block is encoded and decoded in Intra Block Copy (IBC) mode, and wherein the format rules specify whether the one or more syntax elements include syntax elements indicating whether the block vector is from an intra historical motion vector prediction list.

S28. the method according to solution S21, wherein the block vector is a (0, 0) block vector that is not inserted into the block vector prediction list.

S29. a method of video processing, comprising performing a conversion between a video comprising video blocks and a bitstream of the video, wherein the conversion complies with a rule, and wherein the rule specifies that the same list construction process is used during the conversion to determine a block vector prediction list or a sample string vector prediction list of video blocks.

S30. the method according to solution S29, wherein the same list construction process is aligned with the block vector prediction list construction process.

S31. the method according to solution S29, wherein the same list construction process is aligned with the sample string vector prediction list construction process.

S32, a video processing method comprises the following steps: for a transition between a video including a current video block and a bitstream of the video, determining whether a sample string vector or a block vector of an associated block is used to predict a block vector of the current video block according to a rule; and performing a conversion based on the determination, wherein the associated block comprises a parent block or at least one sibling block of the current video block, wherein the sample string vector is derived in a sample string prediction mode and the block vector is derived in an intra block copy mode.

S33. the method according to solution S32, wherein the rule specifies that the sample string vector or the block vector is a candidate in the estimation of the block vector of the current video block.

S34. the method according to solution S33, wherein the rule specifies that all other candidate block vectors are skipped for the estimation process when determining that the sample string vector or block vector achieves the minimum cost.

S35. the method according to solution S32, wherein the associated block further comprises at least one descendant of the parent block or the at least one sibling block.

S36, a video processing method comprises the following steps: for a transition between a video including a current video block and a bitstream of the video, determining that at least one block vector generated from an intra block copy mode for encoding and decoding the current video block is used for an estimation process of a sample string vector of the current video block; and performing a conversion based on the determination, wherein the current video block is further encoded using the sample string prediction mode.

S37. the method according to solution S36, wherein the current video block is encoded and decoded using an intra block copy mode, and then encoded and decoded using a sample string prediction mode.

S38. the method according to solution S36, wherein the current video block is encoded and decoded using a sample string prediction mode, and subsequently decoded using an intra block copy mode.

S39. the method according to any of the solutions S36 to S38, wherein at least one block vector comprises N block vectors achieving a minimum cost in intra block copy mode, and wherein N is a positive integer.

S40. the method according to solution S39, wherein the value of N is based on one or more sizes of the current video block.

S41. a method of video processing, comprising performing a conversion between a video comprising a current video block and a bitstream of the video, wherein the bitstream complies with a format rule, and wherein the format rule specifies that a block vector index indicating a block vector in a block vector prediction list used by the current video block is constrained to be less than a number of block vectors in the block vector prediction list.

S42. the method according to solution S41, wherein the block vector prediction list comprises N block vectors, and wherein the block vector index is less than or equal to (N-1).

S43. the method according to solution S41 or S42, wherein the block vector index is cbvp_index.

S44. a method of video processing, comprising performing a conversion between a video comprising a current video block and a bitstream of the video, wherein the bitstream complies with a format rule, and wherein the format rule specifies that a sample string vector index indicating a block vector or a sample string vector in an intra historical motion vector prediction list used by the current video block is constrained to be less than a number of elements in the intra historical motion vector prediction list.

S45. the method according to solution S44, wherein the sample string vector index is sv_direction_idx.

S46. the method according to any of the solutions S1 to S45, wherein the converting comprises decoding video from a bitstream.

S47. the method according to any of the solutions S1 to S45, wherein the converting comprises encoding the video into a bitstream.

S48. a method of storing a bitstream representing a video to a computer-readable recording medium, comprising generating a bitstream from the video according to the method described in any one or more of the solutions S1 to S45, and storing the bitstream in the computer-readable recording medium.

S49. a video processing apparatus comprising a processor configured to implement the method according to any one or more of solutions S1 to S48.

S50. a computer readable medium storing instructions that, when executed, cause a processor to implement a method according to one or more of solutions S1 to S48.

S51. a computer readable medium stores a bitstream generated according to any one or more of the solutions S1 to S48.

S52. a video processing apparatus for storing a bitstream, wherein the video processing apparatus is configured to implement the method according to any one or more of the solutions S1 to S48.

Next another list of solutions preferred by some embodiments is provided.

P1. a video processing method, comprising: for a transition between a video block of a video and a codec representation of the video, building a list of motion candidates by adding one or more block vectors corresponding to one or more non-immediately adjacent blocks of a current video block according to rules; and performing a conversion based on the list of motion candidates.

P2. the method according to solution P1, wherein the list comprises an intra block copy merge list.

P3. the method according to any one of the solutions P1 to P2, wherein the list comprises an advanced motion vector predictor list.

P4. the method according to any one of the solutions P1 to P3, wherein the rule specifies an order in which the motion candidate is checked for one or more non-immediately adjacent blocks based on the position of the one or more non-immediately adjacent blocks relative to the current video block.

P5. the method according to solution P4, wherein the order comprises first checking an upper left neighbor block, then an upper right neighbor block, then a lower left neighbor block, then an upper neighbor block, and a left neighbor block of the current block.

P6. the method according to solution P4, wherein the sequence comprises: the left neighbor block, the upper left neighbor block, the upper right neighbor block, and the lower left neighbor block of the current block.

P7. a method of video processing, comprising: for a transition between a current video block of a video and a bitstream representation of the video, determining whether a condition of a block vector of one or more block vectors that are not immediately adjacent to the block is satisfied, wherein the condition depends on an availability of the block vector from a history-based block vector prediction list or an availability of the block vector that is immediately adjacent to the block; and performing a conversion according to the determination.

P8. the method according to solution P7, wherein the condition for adding one or more block vectors that are not immediately adjacent blocks is that all history-based block vectors are inserted into the list.

P9. a video processing method comprising: for a transition between a current video block of a video and a codec representation of the video, determining whether a block vector in a history-based block vector predictor (HBVP) list is classified as an nth class according to a rule that depends on a block size associated with the block vector or a size of the current video block; and performing a conversion based on the determination.

P10. the method according to solution P9, wherein the rule specifies classifying the block vector if the block size associated with the block vector is a factor multiple of the size of the current video block.

P11. the method according to solution P10, wherein the factor is equal to 1.

P12. a method of video processing, comprising: for a transition between a current video block of a video and a codec representation of the video, determining a one-dimensional search range for determining a block vector based on a rule, wherein the rule is based on an attribute of the current video block; and performing a conversion according to the determination.

P13. the method according to solution P12, wherein the attribute comprises coordinates of the current video block, and wherein the attribute is sufficient to determine the search range.

P14. the method according to solution P12, wherein the attributes comprise the size of the current video block.

P15. the method according to any one of the solutions P1 to P14, wherein performing the conversion comprises encoding the video to generate an encoded representation.

P16. the method according to any one of the solutions P1 to P14, wherein performing the conversion comprises parsing and decoding the encoded representation to generate the video.

P17. a video decoding apparatus comprising a processor configured to implement the method according to one or more of the solutions P1 to P16.

P18. a video coding device comprising a processor configured to implement the method according to one or more of the solutions P1 to P16.

P19. a computer program product storing computer code which, when executed by a processor, causes the processor to carry out the method according to any one of the solutions P1 to P16.

In this document, the term "video processing" may refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm may be applied during conversion from a pixel representation of video to a corresponding bit stream representation, and vice versa. For example, as defined by the syntax, the bitstream representation (or simply bitstream) of the current video block may correspond to bits that are co-located or distributed in different locations within the bitstream. For example, a macroblock may be encoded according to transformed and encoded error residuals and may also use bits in the header and other fields in the bitstream.

The disclosure and other solutions, examples, embodiments, modules and functional operations described in this document may be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this document and their structural equivalents, or in combinations of one or more of them. The disclosure and other embodiments may be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible and non-volatile computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them. The term "data processing unit" or "data processing apparatus" includes all means, devices, and machines for processing data, including for example, a programmable processor, a computer, or multiple processors or groups of computers. The apparatus may include, in addition to hardware, code that creates an execution environment for a computer program, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. The propagated signal is an artificially generated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus.

A computer program (also known as a program, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. The computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers at one site or distributed across multiple sites and interconnected by a communication network.

The processes and logic flows described in this document can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processing and logic flows may also be performed by, and apparatus may be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer does not necessarily have such a device. Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disk; CD ROM and DVD-ROM discs. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

While this patent document contains many specifics, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features of particular embodiments of particular inventions. Certain features that are described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various functions that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination and the combination of the claims may be directed to a subcombination or variation of a subcombination.

Also, although operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Moreover, the separation of various system components in the embodiments described herein should not be understood as requiring such separation in all embodiments.

Only a few implementations and examples are described, and other implementations, enhancements, and variations may be made based on what is described and illustrated in this patent document.

Claims

1. A method of video processing, comprising:

for a transition between a video including a current video block and a bitstream of the video, determining availability of a non-immediately adjacent block of the current video block for Block Vector (BV) based coding according to a rule based on an overlap condition between a current region including the current video block and an adjacent region including the non-immediately adjacent block of the current video block; and

the conversion is performed based on the determination.

2. The method of claim 1, wherein the non-immediately adjacent block is available because the overlap condition specifies that the current region does not overlap with the adjacent region, and wherein the current region is a first Codec Tree Unit (CTU) and the adjacent region is a second CTU.

3. The method of claim 1, wherein the non-immediately adjacent block is available because the block vector of the non-immediately adjacent block identifies a reference block of the current video block and the overlap condition specifies that the reference block and the current region partially overlap.

4. The method of claim 1, wherein the non-immediately adjacent neighboring block is available because the neighboring region includes N neighboring lines directly above the current video block and excludes lines above the N neighboring lines, and wherein N is a positive integer.

5. The method of claim 1, wherein the non-immediately adjacent neighboring block is available because the neighboring region includes N neighboring columns directly to the left of the current video block and excludes columns to the left of the N neighboring columns, and wherein N is a positive integer.

6. The method of claim 4 or 5, wherein N = 8.

7. The method of claim 1, wherein the non-immediately adjacent neighboring block is available because the neighboring region comprises a first L-shaped neighboring region immediately adjacent to the current video block and excludes a second L-shaped region, and wherein the first L-shaped region is between the second L-shaped region and the current video block.

8. A method of video processing, comprising:

for a transition between a video including a current video block and a bitstream of the video, determining whether a sample string vector of the current video block is used to predict a block vector and/or a string vector of a subsequent video block associated with the current video block; and

Based on the determination, a conversion is performed.

9. The method of claim 8, wherein the current video block is encoded using a sample string prediction mode, and wherein the sample string vector is a representative string vector stored for the current video block.

10. The method of claim 9, wherein the representative string vector comprises a string vector of a first sample string.

11. The method of claim 9, wherein the representative string vector comprises a string vector of a last sample string.

12. The method of claim 9, wherein the representative string vector comprises a string vector of strings of samples covering a center position of the current video block.

13. The method of claim 9, wherein the representative string vector comprises a string vector having a maximum length of strings of samples.

14. The method of claim 8, wherein the current video block is encoded using a sample string prediction mode, and wherein a plurality of string vectors are stored for the current video block.

15. A method of video processing, comprising:

for a transition between a video comprising a current video block and a bitstream of the video, determining that one or more sample string vectors that are not immediately adjacent to a block are available for prediction of a block vector of the current video block; and

Based on the determination, a conversion is performed.

16. The method of claim 15, wherein the one or more sample string vectors are inserted into an Inter Block Copy (IBC) merge list or an IBC Advanced Motion Vector Prediction (AMVP) list.

17. The method of claim 15, wherein the prediction of the block vector of the current video block is further based on one or more sample string vectors immediately adjacent to the block.

18. A method of video processing, comprising:

for a transition between a video including a current video block and a bitstream of the video, determining that one or more block vectors that are not immediately adjacent to a neighboring block are available for prediction of a sample string vector of the current video block; and

based on the determination, a conversion is performed.

19. The method of claim 18, wherein the prediction of the sample string vector of the current video block is further based on one or more sample string vectors of the non-immediately adjacent blocks.

20. The method of claim 18, wherein the prediction of the sample string vector of the current video block is further based on one or more block vectors or one or more sample string vectors of immediately adjacent blocks.

21. A method of video processing, comprising:

Performs a conversion between a video including a current video block and a bit stream of the video,

wherein one or more syntax elements are used in the bitstream to indicate a block vector or a sample string vector, according to a format rule, that is used for prediction of the current video block.

22. The method of claim 21, wherein the format rules specify that the one or more syntax elements include syntax elements indicating the sample string vector.

23. The method of claim 21, wherein the format rules specify that the one or more syntax elements include syntax elements indicating the block vector.

24. The method of claim 21, wherein the format rules specify that the one or more syntax elements include syntax elements indicating whether the block vector or the sample string vector is included in a predefined set of block vectors or a predefined set of sample string vectors, respectively.

25. The method of claim 21, wherein the current video block is encoded in an Intra Block Copy (IBC) mode, and wherein the format rules specify that the one or more syntax elements include a syntax element indicating whether the block vector is from a block vector prediction list.

26. The method of claim 21, wherein the current video block is encoded in an Intra Block Copy (IBC) mode, and wherein the block vector is encoded based on an intra historical motion vector prediction list.

27. The method of claim 21, wherein the current video block is encoded in an Intra Block Copy (IBC) mode, and wherein the format rules specify that the one or more syntax elements include a syntax element indicating whether the block vector is from an intra historical motion vector prediction list.

28. The method of claim 21, wherein the block vector is a (0, 0) block vector that is not inserted into a block vector prediction list.

29. A method of video processing, comprising:

performs conversion between a video including video blocks and a bit stream of the video,

wherein the conversion complies with the rules, and

wherein the rules provide that the same list construction process is used during the conversion to determine a block vector prediction list or a sample string vector prediction list of the video block.

30. The method of claim 29, wherein the same list construction process is aligned with a block vector prediction list construction process.

31. The method of claim 29, wherein the same list construction process is aligned with a sample string vector prediction list construction process.

32. A method of video processing, comprising:

for a transition between a video including a current video block and a bitstream of the video, determining whether a sample string vector of an associated block or a block vector of the associated block is used to predict a block vector of the current video block according to a rule; and

based on the determination a conversion is performed,

wherein the associated block comprises a parent block or at least one sibling block of the current video block, wherein the sample string vector is derived in a sample string prediction mode and the block vector is derived in an intra block copy mode.

33. The method of claim 32, wherein the rule specifies that the sample string vector or the block vector is a candidate in an estimation process of a block vector of the current video block.

34. The method of claim 33, wherein the rule specifies that, in determining that the sample string vector or the block vector achieves a minimum cost, all other candidate block vectors are skipped for the estimation process.

35. The method of claim 32, wherein the associated block further comprises at least one descendant of the parent block or the at least one sibling block.

36. A method of video processing, comprising:

for a transition between a video including a current video block and a bitstream of the video, determining that at least one block vector generated from an intra block copy mode for encoding and decoding the current video block is used for an estimation process of a sample string vector of the current video block; and

based on the determination a conversion is performed,

wherein the current video block is further encoded using a sample string prediction mode.

37. The method of claim 36, wherein the current video block is encoded using the intra block copy mode and subsequently encoded using the sample string prediction mode.

38. The method of claim 36, wherein the current video block is encoded using the sample string prediction mode and subsequently encoded using the intra block copy mode.

39. The method of any of claims 36-38, wherein the at least one block vector comprises N block vectors that achieve a minimum cost in the intra block copy mode, and wherein N is a positive integer.

40. The method of claim 39, wherein the value of N is based on one or more dimensions of the current video block.

41. A method of video processing, comprising:

wherein the bit stream complies with a format rule, and

wherein the format rules specify that a block vector index indicating a block vector in a block vector prediction list used by the current video block is constrained to be less than a number of block vectors in the block vector prediction list.

42. The method of claim 41, wherein the block vector prediction list comprises N block vectors, and wherein the block vector index is less than or equal to (N-1).

43. A method according to claim 41 or 42 wherein the block vector index is cbvp_index.

44. A method of video processing, comprising:

wherein the bit stream complies with a format rule, and

wherein the format rules specify that a sample string vector index indicating a block vector or a sample string vector in an intra-frame historical motion vector prediction list used by the current video block is constrained to be less than the number of elements in the intra-frame historical motion vector prediction list.

45. The method of claim 44, wherein the sample string vector index is sv_direction_idx.

46. The method of any one of claims 1 to 45, wherein the converting comprises decoding the video from the bitstream.

47. The method of any one of claims 1 to 45, wherein the converting comprises encoding the video into the bitstream.

48. A method of storing a bitstream representing video to a computer readable recording medium, comprising:

generating the bitstream from the video according to the method described in any one or more of claims 1 to 45; and

the bit stream is stored in the computer readable recording medium.

49. A video processing apparatus comprising a processor configured to implement the method of any one or more of claims 1 to 48.

50. A computer readable medium having instructions stored thereon that, when executed, cause a processor to implement the method of one or more of claims 1 to 48.

51. A computer readable medium storing a bitstream generated according to any one or more of claims 1 to 48.

52. A video processing apparatus for storing a bitstream, wherein the video processing apparatus is configured to implement a method according to any one or more of claims 1 to 48.