TWI709332B

TWI709332B - Motion prediction based on updated motion vectors

Info

Publication number: TWI709332B
Application number: TW108125901A
Authority: TW
Inventors: 劉鴻彬; 張莉; 張凱; 王悅
Original assignee: 大陸商北京字節跳動網絡技術有限公司; 美商字節跳動有限公司
Priority date: 2018-07-20
Filing date: 2019-07-22
Publication date: 2020-11-01
Also published as: WO2020016858A1; TW202023283A; CN110740327B; CN110740321B; WO2020016859A2; CN110740332A; WO2020016857A1; CN110740321A; CN110740327A; TW202008787A; TWI734147B; WO2020016859A3; CN110740332B; TW202008786A; TWI723472B

Abstract

A method for video processing, comprising: determining a current block with motion vectors; generating updated motion vectors based on specific prediction mode; and performing, based on the updated motion vectors, a conversion between the current block and a bitstream representation of a video data including the current block, wherein the specific prediction mode includes one or more of bi-directional optical flow (BIO) refinement, a decoder-side motion vector refinement (DMVR), frame-rate up conversion (FRUC) techniques or a template matching technique.

Description

基於更新的運動矢量的運動預測Motion prediction based on updated motion vector

本文件涉及視訊編碼和視訊解碼技術、設備和系統。 [相關申請的交叉引用] 根據適用的專利法和/或依據巴黎公約的規則，本申請要求於2018年7月20日提交的題為「基於更新的運動矢量的運動預測」的國際專利申請第PCT/CN2018/096384號的優先權和權益。該國際專利申請第PCT/CN2018/096384號的全部公開內容通過引用併入作為本申請的公開內容的一部分。This document deals with video encoding and video decoding technologies, equipment and systems. [Cross references to related applications] In accordance with the applicable patent law and/or the rules of the Paris Convention, this application requires the international patent application No. PCT/CN2018/096384 filed on July 20, 2018, entitled "Motion prediction based on updated motion vectors" Priorities and rights. The entire disclosure of the International Patent Application No. PCT/CN2018/096384 is incorporated by reference as part of the disclosure of this application.

儘管視訊壓縮有所進步，但數位視訊仍佔網際網路和其他數位通信網路上最大的頻寬使用。隨著能夠接收和顯示視訊的所連接的使用者設備的數量增加，預計數位視訊使用的頻寬需求將繼續增長。Despite improvements in video compression, digital video still accounts for the largest bandwidth usage on the Internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, it is expected that the demand for bandwidth used by digital video will continue to grow.

描述了與數位視訊編碼有關的設備、系統和方法，並且具體地，描述了基於更新的運動向量的運動預測。所描述的方法可以應用於現有視訊編碼標準（例如，高效視訊編碼（HEVC））和未來視訊編碼標準或視訊轉碼器。The devices, systems, and methods related to digital video coding are described, and specifically, motion prediction based on updated motion vectors is described. The method described can be applied to existing video coding standards (for example, High Efficiency Video Coding (HEVC)) and future video coding standards or video codecs.

在一個代表性方面，所公開的技術可以用於提供視訊編碼的方法。該方法包括確定具有運動向量的當前塊；基於特定預測模式生成更新的運動向量；以及基於所述更新的運動向量，執行所述當前塊與包括所述當前塊的視訊資料的位元流表示之間的轉換，其中所述特定預測模式包括以下中的一個或多個：雙向光流（BIO）細化、解碼器側運動向量細化（DMVR）、畫面播放速率上轉換（FRUC）技術或模板匹配技術。In a representative aspect, the disclosed technology can be used to provide a video encoding method. The method includes determining a current block with a motion vector; generating an updated motion vector based on a specific prediction mode; and, based on the updated motion vector, performing one of the current block and a bitstream representation of video data including the current block The specific prediction mode includes one or more of the following: bidirectional optical flow (BIO) refinement, decoder-side motion vector refinement (DMVR), frame rate up-conversion (FRUC) technology or template Matching technology.

在另一個代表性方面，所公開的技術可以用於提供用於處理視訊資料的另一種方法。該方法包括確定當前塊和對應的相鄰塊；構造所述當前塊的最終預測塊作為從所述當前塊的運動向量導出的預測塊和從一個或多個相鄰塊的運動向量導出的預測塊的加權和；以及基於所述當前塊的最終預測塊，執行所述當前塊與包括所述當前塊的視訊資料的位元流表示之間的轉換；其中短階內插濾波器用於構造所述當前塊的最終預測塊。In another representative aspect, the disclosed technology can be used to provide another method for processing video data. The method includes determining a current block and a corresponding neighboring block; constructing a final prediction block of the current block as a prediction block derived from the motion vector of the current block and a prediction derived from the motion vector of one or more neighboring blocks The weighted sum of the blocks; and based on the final prediction block of the current block, performing the conversion between the current block and the bitstream representation of the video data including the current block; wherein a short-order interpolation filter is used to construct the The final prediction block of the current block.

在又一個代表性方面，上述方法以處理器可執行代碼的形式實施並儲存在電腦可讀程式介質中。In yet another representative aspect, the above method is implemented in the form of processor executable code and stored in a computer-readable program medium.

在又一個代表性方面，公開了一種被配置或可操作以執行上述方法的設備。該設備可以包括被程式設計為實現該方法的處理器。In yet another representative aspect, a device configured or operable to perform the above method is disclosed. The device may include a processor programmed to implement the method.

在又一個代表性方面，視訊解碼器裝置可以實現如本文中所描述的方法。In yet another representative aspect, the video decoder device may implement the method as described herein.

在附圖、說明書和申請專利範圍中更詳細地描述了所公開技術的上述方面和特徵以及其他方面和特徵。The above-mentioned aspects and features and other aspects and features of the disclosed technology are described in more detail in the accompanying drawings, specification and the scope of patent application.

由於對更高解析度視訊的需求的增加，視訊編碼方法和技術在現代技術中普遍存在。視訊轉碼器通常包括壓縮或解壓縮數位視訊的電子電路或軟體，並且不斷改進以提供更高的編碼效率。視訊轉碼器將未壓縮視訊轉換為壓縮格式，反之亦然。視訊品質、用於表示視訊的資料量（由位元速率確定）、編碼和解碼演算法的複雜度、對資料丟失和錯誤的敏感性、編輯的簡易性，隨機訪問和端到端延遲（延遲）之間存在複雜的關係。壓縮格式通常符合標準視訊壓縮規範，例如，高效視訊編碼（HEVC）標準（也稱為H.265或MPEG-H第2部分）、要完成的通用視訊編碼標準、或其他當前和/或未來的視訊編碼標準。Due to the increasing demand for higher-resolution video, video coding methods and technologies are common in modern technologies. Video codecs usually include electronic circuits or software that compress or decompress digital video, and are continuously improved to provide higher coding efficiency. Video codec converts uncompressed video to compressed format and vice versa. Video quality, the amount of data used to represent the video (determined by the bit rate), the complexity of the encoding and decoding algorithms, the sensitivity to data loss and errors, the ease of editing, random access and end-to-end latency (latency) ) There is a complicated relationship. The compression format usually conforms to standard video compression specifications, for example, the High Efficiency Video Coding (HEVC) standard (also known as H.265 or MPEG-H Part 2), the general video coding standard to be completed, or other current and/or future Video coding standard.

所公開的技術的實施例可以應用於現有視訊編碼標準（例如，HEVC，H.265）和未來的標準以改進壓縮性能。在本文件中使用節標題以改進描述的可讀性，並且不以任何方式將討論或實施例（和/或實現方式）限制於僅相應的節。The embodiments of the disclosed technology can be applied to existing video coding standards (for example, HEVC, H.265) and future standards to improve compression performance. Section headings are used in this document to improve the readability of the description, and do not limit the discussion or embodiments (and/or implementations) to only the corresponding section in any way.

1.在HEVC/H.265中的幀間預測的示例1. Example of inter prediction in HEVC/H.265

多年來，視訊編碼標準已經顯著改進，並且現在部分地提供高編碼效率和對更高解析度的支援。諸如HEVC和H.265的最新標準基於利用時間預測加變換編碼的混合視訊編碼結構。Over the years, video coding standards have improved significantly and now partly provide high coding efficiency and support for higher resolutions. The latest standards such as HEVC and H.265 are based on a hybrid video coding structure using temporal prediction plus transform coding.

1.1幀間預測的示例1.1 Example of inter prediction

每個幀間預測的PU（預測單元）具有對於一個或兩個參考圖像列表的運動參數。在一些實施例中，運動參數包括運動向量和參考圖像索引。在其他實施例中，也可以使用inter_pred_idc來信令通知對兩個參考圖像列表中一個的使用。在另外的其他實施例中，運動向量可以明確地被編碼為相對於預測器的增量。Each inter-predicted PU (prediction unit) has motion parameters for one or two reference image lists. In some embodiments, the motion parameters include motion vectors and reference image indexes. In other embodiments, inter_pred_idc can also be used to signal the use of one of the two reference image lists. In still other embodiments, the motion vector can be explicitly coded as an increment relative to the predictor.

當使用跳過（skip）模式編碼CU時，一個PU與該CU相關聯，並且不存在顯著的殘差係數，不存在編碼的運動向量增量或參考圖像索引。指定Merge模式，由此從相鄰PU——包括空間和時間候選——獲得用於當前PU的運動參數。Merge模式可以應用於任何幀間預測的PU，而不僅適用於跳過模式。Merge模式的替代方案是運動參數的顯式傳輸（explicit transmission），其中運動向量、每個參考圖像列表的對應參考圖像索引、參考圖像列表使用被每個PU地明確地信令通知。When a CU is coded in skip mode, one PU is associated with the CU, and there are no significant residual coefficients, no coded motion vector increments or reference image indexes. Specify the Merge mode, thereby obtaining motion parameters for the current PU from neighboring PUs, including spatial and temporal candidates. The Merge mode can be applied to any inter-predicted PU, not only to the skip mode. An alternative to the Merge mode is the explicit transmission of motion parameters, where the motion vector, the corresponding reference image index of each reference image list, and the reference image list usage are explicitly signaled by each PU.

當信令指示要使用兩個參考圖像列表中的一個時，PU從一個樣點塊產生。這被稱為「單向預測」。單向預測可用於P條帶和B條帶。When the signaling indicates that one of the two reference picture lists is to be used, the PU is generated from one sample block. This is called "one-way prediction". One-way prediction can be used for P-band and B-band.

當信令指示要使用兩個參考圖像列表時，PU從兩個樣點塊產生。這被稱為「雙向預測」。雙向預測僅可用於B條帶。When the signaling indicates that two reference picture lists are to be used, the PU is generated from two sample blocks. This is called "two-way prediction". Bi-directional prediction can only be used for band B.

1.1.1構建Merge模式的候選的實施例1.1.1 Examples of candidates for constructing Merge mode

當使用Merge模式預測PU時，從位元流解析出指向Merge候選列表（merge candidates list）中的條目的索引，並且該索引被用於檢索運動信息。該列表的構建可以根據以下步驟順序進行總結：When the Merge mode is used to predict the PU, an index pointing to an entry in the Merge candidate list (merge candidates list) is parsed from the bit stream, and the index is used to retrieve motion information. The construction of the list can be summarized according to the following sequence of steps:

步驟1：初始候選推導步驟1.1：空間候選推導步驟1.2：空間候選的冗餘校驗步驟1.3：時間候選推導Step 1: Initial candidate derivation Step 1.1: Spatial candidate derivation Step 1.2: Redundancy check of spatial candidates Step 1.3: Time candidate derivation

步驟2：附加候選*** 步驟2.1：創建雙向預測候選步驟2.2：***零運動候選Step 2: Additional candidate insertion Step 2.1: Create bidirectional prediction candidates Step 2.2: Insert zero motion candidates

圖1示出了基於上面總結的步驟順序構建Merge候選列表的示例。對於空間Merge候選推導，在位於五個不同位置的候選中選擇最多四個Merge候選。對於時間Merge候選推導，在兩個候選中選擇最多一個Merge候選。由於在解碼器處假設每個PU的候選的數量為常數，因此當候選的數量未達到在條帶標頭中信令通知的最大Merge候選數量（MaxNumMergeCand）時，生成附加的候選。由於候選的數量是恒定的，因此使用二進位一元截斷（TU）來編碼最佳Merge候選的索引。如果CU的尺寸等於8，則當前CU的所有PU共用單個Merge候選列表，該單個Merge候選列表與2N×2N預測單元的Merge候選列表相同。Figure 1 shows an example of constructing a Merge candidate list based on the sequence of steps summarized above. For the derivation of spatial Merge candidates, at most four Merge candidates are selected from candidates located at five different positions. For the temporal Merge candidate derivation, at most one Merge candidate is selected from the two candidates. Since the number of candidates for each PU is assumed to be constant at the decoder, when the number of candidates does not reach the maximum number of Merge candidates (MaxNumMergeCand) signaled in the slice header, additional candidates are generated. Since the number of candidates is constant, binary unary truncation (TU) is used to encode the index of the best Merge candidate. If the size of the CU is equal to 8, all PUs of the current CU share a single Merge candidate list, which is the same as the Merge candidate list of the 2N×2N prediction unit.

1.1.2構建空間Merge候選1.1.2 Build Space Merge Candidate

在空間Merge候選的推導中，在位於圖2中描繪的位置中的候選中選擇最多四個Merge候選。推導的順序是A₁ 、B₁ 、B₀ 、A₀ 和B₂ 。僅當位置A₁ 、B₁ 、B₀ 、A₀ 的任何PU不可用時（例如，因為該PU屬於另一個條帶（slice）或片（tile））或者是幀內編碼時，才考慮位置B₂ 。在添加位置A₁ 處的候選之後，對剩餘候選的添加進行冗餘校驗，該冗餘校驗確保具有相同運動信息的候選被排除在列表之外，從而改進編碼效率。In the derivation of spatial Merge candidates, a maximum of four Merge candidates are selected among the candidates located in the positions depicted in FIG. 2. The sequence of derivation is A ₁ , B ₁ , B ₀ , A ₀ and B ₂ . Only when any PU at positions A ₁ , B ₁ , B ₀ , A ₀ is not available (for example, because the PU belongs to another slice or tile) or is intra-coded, the position is considered B ₂ . After the addition of the candidate position at A _1, of the remaining candidate is added redundancy check, CRC to ensure that the candidate has the same motion information is excluded from the list, thereby improving coding efficiency.

為了降低計算複雜度，在所提到的冗餘校驗中並未考慮所有可能的候選對。相反，僅考慮與圖3中的箭頭連結的對，並且如果用於冗餘校驗的對應候選具有不同的運動信息，則該候選僅被添加到列表中。重複的運動信息的另一個來源是與不同於2Nx2N的劃分相關聯的「第二PU」。作為示例，圖4A和4B分別描繪了針對N×2N和2N×N的情況的第二PU。當當前PU被劃分為N×2N時，位置A₁ 處的候選不被考慮用於列表構建。在一些實施例中，添加該候選可以導致具有相同運動信息的兩個預測單元，這對於在編碼單元中僅具有一個PU是冗餘的。類似地，當當前PU被劃分為2N×N時，不考慮位置B₁ 。In order to reduce the computational complexity, not all possible candidate pairs are considered in the mentioned redundancy check. In contrast, only the pair connected with the arrow in FIG. 3 is considered, and if the corresponding candidate used for redundancy check has different motion information, the candidate is only added to the list. Another source of repeated motion information is the "second PU" associated with a partition other than 2Nx2N. As an example, FIGS. 4A and 4B depict the second PU for the cases of N×2N and 2N×N, respectively. When the current PU is divided into N×2N, the candidate at position A ₁ is not considered for list construction. In some embodiments, adding this candidate may result in two prediction units with the same motion information, which is redundant for having only one PU in the coding unit. Similarly, when the current PU is divided into 2N×N, the position B _{1 is} not considered.

1.1.3構建時間Merge候選1.1.3 Merge Candidates at Build Time

在此步驟中，只有一個候選被添加到列表中。特別地，在該時間Merge候選的推導中，基於共位（co-located）的PU來導出縮放的運動向量，該共位的PU屬於相對於給定參考圖像列表內的當前圖像具有最小POC差異的圖像。在條帶標頭中明確地信令通知用於共位的PU的推導的參考圖像列表。In this step, only one candidate is added to the list. In particular, in the derivation of the temporal Merge candidate, the scaled motion vector is derived based on the co-located PU. The co-located PU has the smallest value relative to the current image in the given reference image list. POC difference image. The derivation of the reference picture list for the co-located PU is explicitly signaled in the slice header.

圖5示出了用於時間Merge候選（如虛線）的縮放的運動向量的推導的示例，該用於時間Merge候選（如虛線）的縮放的運動向量使用POC距離tb和td從共位的PU的運動向量縮放，其中tb被定義為當前圖像的參考圖像與該當前圖像之間的POC差異，並且td被定義為共位元圖像的參考圖像與該共位元圖像之間的POC差異。時間Merge候選的參考圖像索引被設置為等於零。對於B條帶，獲得兩個運動向量並將其組合以產生雙向預測Merge候選，該兩個運動向量中的一個用於參考圖像列表0而另一個用於參考圖像列表1。FIG. 5 shows an example of the derivation of the scaled motion vector for temporal Merge candidates (such as dashed lines). The scaled motion vector for temporal Merge candidates (such as dashed lines) uses POC distances tb and td from a co-located PU The motion vector scaling, where tb is defined as the POC difference between the reference image of the current image and the current image, and td is defined as the difference between the reference image of the co-bit image and the co-bit image POC difference between. The reference image index of the temporal Merge candidate is set equal to zero. For the B slice, two motion vectors are obtained and combined to generate a bi-directional predictive Merge candidate, one of the two motion vectors is used for reference image list 0 and the other is used for reference image list 1.

如圖6所示，在屬於參考幀的共位的PU(Y)中，在候選C₀ 和C₁ 之間選擇用於時間候選的位置。如果位置C₀ 處的PU不可用、被幀內編碼的或在當前CTU之外，則使用位置C₁ 。否則，在時間Merge候選的推導中使用位置C₀ 。As shown in FIG. 6, in the co-located PU (Y) belonging to the reference frame, the position for the temporal candidate is selected between the candidates C ₀ and C ₁ . If the PU at position C ₀ is not available, is intra-coded, or is outside the current CTU, position C ₁ is used. Otherwise, the position C ₀ is used in the derivation of the temporal Merge candidate.

1.1.4構建附加類型的Merge候選1.1.4 Build additional types of Merge candidates

除了空時Merge候選之外，還存在兩種附加類型的Merge候選：組合的雙向預測Merge候選和零Merge候選。通過利用空時Merge候選來生成組合的雙向預測Merge候選。組合的雙向預測Merge候選僅用於B條帶。通過將初始候選的第一參考圖像列表運動參數與另一個候選的第二參考圖像列表運動參數組合來生成組合的雙向預測候選。如果這兩個元組提供不同的運動假設，則它們將形成一個新的雙向預測候選。In addition to space-time Merge candidates, there are two additional types of Merge candidates: combined bidirectional predictive Merge candidates and zero Merge candidates. The combined bidirectional predictive Merge candidate is generated by using the space-time Merge candidate. The combined bi-directional prediction Merge candidate is only used for the B band. The combined bidirectional prediction candidate is generated by combining the first reference image list motion parameter of the initial candidate with the second reference image list motion parameter of another candidate. If these two tuples provide different motion hypotheses, they will form a new bi-prediction candidate.

圖7示出了該過程的示例，在該過程中原始列表（710，在左方）中具有mvL0和refIdxL0或mvL1和refIdxL1的兩個候選被用於創建組合的雙向預測Merge候選，該組合的雙向預測Merge候選被添加到最終列表（720，在右方）。Figure 7 shows an example of this process in which two candidates with mvL0 and refIdxL0 or mvL1 and refIdxL1 in the original list (710, on the left) are used to create a combined bidirectional prediction Merge candidate. The bi-predictive Merge candidate is added to the final list (720, on the right).

零運動候選被***以填充Merge候選列表中的剩餘條目，並且因此達到MaxNumMergeCand容量。這些候選具有零空間位移和參考圖像索引，該參考圖像索引從零開始並且每當新的零運動候選被添加到列表時增加。這些候選使用的參考幀的數量是分別用於單向和雙向預測的1和2。在一些實施例中，不對這些候選執行冗餘校驗。Zero motion candidates are inserted to fill the remaining entries in the Merge candidate list, and thus reach the MaxNumMergeCand capacity. These candidates have a zero spatial displacement and a reference image index, which starts at zero and increases every time a new zero motion candidate is added to the list. The number of reference frames used by these candidates is 1 and 2 for unidirectional and bidirectional prediction, respectively. In some embodiments, no redundancy check is performed on these candidates.

1.1.5用於並行處理的運動估計區域的示例1.1.5 Examples of motion estimation regions for parallel processing

為了加速編碼過程，可以並存執行運動估計，由此同時導出給定區域內的所有預測單元的運動向量。來自空間相鄰的Merge候選的推導可能干擾並行處理，因為一個預測單元直到其相關聯的運動估計完成才能從相鄰PU導出運動參數。為了減輕編碼效率和處理等待時間之間的折衷，可以定義運動估計區域（MER）。MER的尺寸可以在圖像參數集（PPS）中使用「log2_parallel_merge_level_minus2」語法元素來信令通知。當MER被定義時，落入同一區域的Merge候選被標記為不可用，並且因此在列表構建中也不被考慮。In order to speed up the encoding process, motion estimation can be performed concurrently, thereby simultaneously deriving the motion vectors of all prediction units in a given area. The derivation from spatially neighboring Merge candidates may interfere with parallel processing, because a prediction unit cannot derive motion parameters from neighboring PUs until its associated motion estimation is completed. In order to alleviate the trade-off between coding efficiency and processing latency, a motion estimation area (MER) can be defined. The size of the MER can be signaled in the picture parameter set (PPS) using the "log2_parallel_merge_level_minus2" syntax element. When MER is defined, Merge candidates that fall into the same area are marked as unavailable, and therefore are not considered in the list construction.

1.2 高級運動向量預測（AMVP）的實施例1.2 Examples of Advanced Motion Vector Prediction (AMVP)

AMVP利用運動向量與相鄰PU的空時相關性，該空時相關性用於運動參數的顯式傳輸。通過下述操作來構建運動向量候選列表：首先校驗左方、上方在時間上相鄰PU位置的可用性，移除冗餘候選，並添加零向量，以使候選列表為恒定長度。然後，編碼器可以從候選列表中選擇最佳預測器，並傳輸指示所選候選的對應索引。與Merge索引信令類似，使用一元截斷來編碼最佳運動向量候選的索引。在這種情況下要編碼的最大值是2（參見圖8）。在以下的節中，提供了關於運動向量預測候選的推導過程的細節。AMVP utilizes the space-time correlation between motion vectors and neighboring PUs, and the space-time correlation is used for explicit transmission of motion parameters. The motion vector candidate list is constructed by the following operations: firstly, check the availability of temporally adjacent PU positions on the left and above, remove redundant candidates, and add zero vectors to make the candidate list a constant length. The encoder can then select the best predictor from the candidate list and transmit the corresponding index indicating the selected candidate. Similar to Merge index signaling, unary truncation is used to encode the index of the best motion vector candidate. The maximum value to be encoded in this case is 2 (see Figure 8). In the following section, details on the derivation process of motion vector prediction candidates are provided.

1.2.1構建運動向量預測候選的示例1.2.1 Example of constructing motion vector prediction candidates

圖8總結了用於運動向量預測候選的推導過程，並且可以針對具有refidx作為輸入的每個參考圖像列表來實現。Figure 8 summarizes the derivation process for motion vector prediction candidates, and can be implemented for each reference image list with refidx as input.

在運動向量預測中，考慮兩種類型的運動向量候選：空間運動向量候選和時間運動向量候選。如圖2先前示出的，對於空間運動向量候選推導，最終基於位於五個不同位置的每個PU的運動向量來導出兩個運動向量候選。In motion vector prediction, two types of motion vector candidates are considered: spatial motion vector candidates and temporal motion vector candidates. As shown previously in FIG. 2, for the spatial motion vector candidate derivation, two motion vector candidates are finally derived based on the motion vector of each PU located at five different positions.

對於時間運動向量候選推導，從基於兩個不同的共位位置導出的兩個候選中選擇一個運動向量候選。在製作空時候選的第一列表之後，移除列表中的重複的運動向量候選。如果潛在候選的數量大於2，則從列表中移除其在相關聯的參考圖像列表內的參考圖像索引大於1的運動向量候選。如果空時運動向量候選的數量小於2，則將附加的零運動向量候選添加到列表中。For the derivation of temporal motion vector candidates, one motion vector candidate is selected from two candidates derived based on two different co-located positions. After making the first list selected when empty, the repeated motion vector candidates in the list are removed. If the number of potential candidates is greater than 2, the motion vector candidate whose reference image index in the associated reference image list is greater than 1 is removed from the list. If the number of space-time motion vector candidates is less than 2, then additional zero motion vector candidates are added to the list.

1.2.2構建空間運動向量候選1.2.2 Construction of spatial motion vector candidates

在空間運動向量候選的推導中，在五個潛在候選中考慮最多兩個候選，該五個潛在候選來自位於如圖2先前示出的位置的PU，這些位置與運動Merge的那些位置相同。當前PU的左側的推導順序被定義為A₀ 、A₁ 以及縮放的A₀ 、縮放的A₁ 。當前PU的上側的推導順序被定義為B₀ 、B₁ 、B₂ 、縮放的B₀ 、縮放的B₁ 、縮放的B₂ 。因此，對於每一側，存在四種可用作運動向量候選的情況，其中兩種情況不需要使用空間縮放，並且兩種情況使用空間縮放。四種不同的情況總結如下。In the derivation of the spatial motion vector candidates, at most two candidates are considered among the five potential candidates, which are from the PU located at the positions previously shown in FIG. 2, which are the same as those of the motion Merge. The derivation order of the left side of the current PU is defined as A ₀ , A ₁ and scaled A ₀ , scaled A ₁ . The derivation order of the upper side of the current PU is defined as B ₀ , B ₁ , B ₂ , scaled B ₀ , scaled B ₁ , scaled B ₂ . Therefore, for each side, there are four cases that can be used as motion vector candidates, two of which do not need to use spatial scaling, and two cases use spatial scaling. The four different situations are summarized below.

無空間縮放（1）相同的參考圖像列表，以及相同的參考圖像（相同的POC）（2）不同的參考圖像列表，但是相同的參考圖像（相同的POC）No space zoom (1) The same reference image list, and the same reference image (same POC) (2) Different reference image list, but the same reference image (same POC)

空間縮放（3）相同的參考圖像列表，但不同的參考圖像（不同的POC）（4）不同的參考圖像列表，以及不同的參考圖像（不同的POC）Space zoom (3) The same reference image list, but different reference images (different POC) (4) Different reference image lists, and different reference images (different POC)

首先校驗無空間縮放情況，接下來校驗允許空間縮放的情況。不管參考圖像列表如何，當POC在相鄰PU的參考圖像與當前PU的參考圖像之間是不同的時，考慮空間縮放。如果左方候選的所有PU都不可用或者是被幀內編碼的，則允許對上方運動向量進行縮放，以幫助左方和上方MV候選的並行推導。否則，對上側運動向量不允許空間縮放。First, verify the absence of spatial scaling, and then verify the permitted spatial scaling. Regardless of the reference image list, when the POC is different between the reference image of the adjacent PU and the reference image of the current PU, spatial scaling is considered. If all the PUs of the left candidate are unavailable or are intra-coded, the upper motion vector is allowed to be scaled to help the parallel derivation of the left and upper MV candidates. Otherwise, spatial scaling is not allowed for the upper motion vector.

如圖9中的示例所示，對於空間縮放情況，以與時間縮放類似的方式縮放相鄰PU的運動向量。一個差異在於當前PU的參考圖像列表和索引被給出以作為輸入；實際縮放過程與時間縮放過程相同。As shown in the example in FIG. 9, for the spatial scaling case, the motion vectors of adjacent PUs are scaled in a similar manner to temporal scaling. One difference is that the reference image list and index of the current PU are given as input; the actual scaling process is the same as the time scaling process.

1.2.3構建時間運動向量候選1.2.3 Construction of temporal motion vector candidates

除了參考圖像索引推導之外，用於時間Merge候選的推導的所有過程與用於空間運動向量候選的推導的過程相同（如圖6中的示例所示）。在一些實施例中，將參考圖像索引信令通知給解碼器。Except for the reference image index derivation, all the processes used for the derivation of temporal Merge candidates are the same as those used for the derivation of spatial motion vector candidates (as shown in the example in FIG. 6). In some embodiments, the reference image index is signaled to the decoder.

2.在聯合探索模型（JEM）中的幀間預測方法的示例2. Examples of inter prediction methods in the Joint Exploration Model (JEM)

在一些實施例中，使用稱為聯合探索模型（JEM）的參考軟體來探索未來的視訊編碼技術。在JEM中，在數個編碼工具中採用基於子塊的預測，諸如仿射預測、替代時間運動向量預測（ATMVP）、空時運動向量預測（STMVP）、雙向光流（BIO）、畫面播放速率上轉換（FRUC）、局部自我調整運動向量解析度（LAMVR）、重疊塊的運動補償（OBMC）、局部亮度補償（LIC）和解碼器側運動向量細化（DMVR）。In some embodiments, reference software called Joint Exploration Model (JEM) is used to explore future video coding technologies. In JEM, sub-block-based prediction is used in several coding tools, such as affine prediction, alternative temporal motion vector prediction (ATMVP), space-time motion vector prediction (STMVP), bidirectional optical flow (BIO), picture playback rate Up-conversion (FRUC), local self-adjusting motion vector resolution (LAMVR), overlapping block motion compensation (OBMC), local luminance compensation (LIC) and decoder-side motion vector refinement (DMVR).

2.1基於子CU的運動向量預測的示例2.1 Example of motion vector prediction based on sub-CU

在具有四叉樹加二叉樹（QTBT）的JEM中，每個CU可以具有針對每個預測方向的至多一個運動參數的集合。在一些實施例中，通過將大CU劃分為子CU並且導出大CU的所有子CU的運動信息，在編碼器中考慮兩個子CU級別運動向量預測方法。替代時間運動向量預測（ATMVP）方法允許每個CU從比共位元參考圖像中的當前CU小的多個塊中獲取運動信息的多個集合。在空時運動向量預測（STMVP）方法中，通過使用時間運動向量預測器和空間相鄰運動向量來遞迴地導出子CU的運動向量。在一些實施例中，並且為了保留用於子CU運動預測的更準確的運動場，可以禁用參考幀的運動壓縮。In JEM with quadtree plus binary tree (QTBT), each CU may have at most one set of motion parameters for each prediction direction. In some embodiments, by dividing the large CU into sub-CUs and deriving the motion information of all sub-CUs of the large CU, two sub-CU-level motion vector prediction methods are considered in the encoder. The Alternative Temporal Motion Vector Prediction (ATMVP) method allows each CU to obtain multiple sets of motion information from multiple blocks smaller than the current CU in the co-bit reference image. In the space-time motion vector prediction (STMVP) method, the motion vector of the sub-CU is recursively derived by using a temporal motion vector predictor and a spatial neighboring motion vector. In some embodiments, and in order to preserve a more accurate motion field for sub-CU motion prediction, motion compression of reference frames may be disabled.

2.1.1替代時間運動向量預測（ATMVP）的示例2.1.1 Examples of alternative temporal motion vector prediction (ATMVP)

在ATMVP方法中，通過從小於當前CU的塊中獲取運動信息（包括運動向量和參考索引）的多個集合來修改時間運動向量預測（TMVP）方法。In the ATMVP method, the Temporal Motion Vector Prediction (TMVP) method is modified by acquiring multiple sets of motion information (including motion vectors and reference indexes) from blocks smaller than the current CU.

圖10示出了CU 1000的ATMVP運動預測過程的示例。ATMVP方法以兩個步驟預測CU 1000內的子CU 1001的運動向量。第一步驟是使用時間向量識別參考圖像1050中的對應塊1051。參考圖像1050也稱為運動源圖像。第二步驟是將當前CU 1000劃分為子CU 1001，並從對應於每個子CU的塊中獲得運動向量以及每個子CU的參考索引。FIG. 10 shows an example of the ATMVP motion prediction process of the CU 1000. The ATMVP method predicts the motion vector of the sub-CU 1001 in the CU 1000 in two steps. The first step is to use the time vector to identify the corresponding block 1051 in the reference image 1050. The reference image 1050 is also called a moving source image. The second step is to divide the current CU 1000 into sub-CUs 1001, and obtain the motion vector and the reference index of each sub-CU from the block corresponding to each sub-CU.

在第一步驟中，參考圖像1050和對應塊由當前CU 1000的空間相鄰塊的運動信息確定。為了避免相鄰塊的重複掃描過程，使用當前CU 1000的Merge候選列表中的第一Merge候選。第一可用運動向量及其相關聯的參考索引被設置為運動源圖像的時間向量和索引。這樣，與TMVP相比，可以更準確地識別對應塊，其中對應塊（有時稱為共位塊）總是相對於當前CU處於右下或中心位置。In the first step, the reference image 1050 and the corresponding block are determined by the motion information of the spatial neighboring blocks of the current CU 1000. In order to avoid the repeated scanning process of adjacent blocks, the first Merge candidate in the Merge candidate list of the current CU 1000 is used. The first available motion vector and its associated reference index are set as the time vector and index of the motion source image. In this way, compared with TMVP, the corresponding block can be identified more accurately, where the corresponding block (sometimes called a co-located block) is always in the lower right or center position relative to the current CU.

在第二步驟中，通過向當前CU的座標添加時間向量，由運動源圖像1050中的時間向量識別子CU 1051的對應塊。對於每個子CU，其對應塊的運動信息（例如，覆蓋中心樣點的最小運動網格）用於導出子CU的運動信息。在識別出對應的N×N塊的運動信息之後，以與HEVC的TMVP相同的方式將其轉換為當前子CU的運動向量和參考索引，在該方式中應用運動縮放和其他過程。例如，解碼器校驗是否滿足低延遲條件（例如，當前圖像的所有參考圖像的POC小於當前圖像的POC）並且可能使用運動向量MVx（例如，對應於參考圖像列表X的運動向量）用於預測每個子CU的運動向量MVy（例如，其中X等於0或1並且Y等於1-X）。In the second step, by adding a time vector to the coordinates of the current CU, the corresponding block of the sub-CU 1051 is identified from the time vector in the motion source image 1050. For each sub-CU, the motion information of its corresponding block (for example, the smallest motion grid covering the center sample point) is used to derive the motion information of the sub-CU. After the motion information of the corresponding N×N block is identified, it is converted into the motion vector and reference index of the current sub-CU in the same manner as the TMVP of HEVC, in which motion scaling and other processes are applied. For example, the decoder checks whether the low-delay condition is satisfied (for example, the POC of all reference pictures of the current picture is less than the POC of the current picture) and may use the motion vector MVx (for example, the motion vector corresponding to the reference picture list X ) Is used to predict the motion vector MVy of each sub-CU (for example, where X is equal to 0 or 1 and Y is equal to 1-X).

2.1.2空時運動向量預測（STMVP）的示例2.1.2 Example of Space-Time Motion Vector Prediction (STMVP)

在STMVP方法中，按照光柵掃描順序遞迴地導出子CU的運動向量。圖11示出了具有四個子塊和相鄰塊的一個CU的示例。考慮包括四個4×4子CU A（1101）、B（1102）、C（1103）和D（1104）的8×8 CU 1100。當前幀中的相鄰4×4塊標記為（1111）、b（1112）、c（1113）和d（1114）。In the STMVP method, the motion vectors of the sub-CUs are recursively derived in the raster scan order. Fig. 11 shows an example of one CU with four sub-blocks and neighboring blocks. Consider an 8×8 CU 1100 that includes four 4×4 sub-CUs A (1101), B (1102), C (1103), and D (1104). The adjacent 4×4 blocks in the current frame are marked as (1111), b(1112), c(1113) and d(1114).

子CU A的運動推導通過識別其兩個空間鄰居開始。第一鄰居是子CU A 1101上方的N×N塊（塊c 1113）。如果該塊c（1113）不可用或者是幀內編碼，則校驗（從左到右，從塊c 1113開始）子CU A（1101）上方的其他N×N塊。第二鄰居是子CU A 1101左側的塊（塊b 1112）。如果塊b（1112）不可用或者是幀內編碼，則校驗（從上到下，從塊b 1112開始）子CU A 1101左側的其他塊。從每個列表的相鄰塊獲得的運動信息被縮放到給定列表的第一參考幀。接下來，通過遵循與HEVC中指定的TMVP推導相同的過程來導出子塊A 1101的時間運動向量預測器（TMVP）。在塊D 1104處的共位元塊的運動信息被相應地獲取和縮放。最後，在檢索和縮放運動信息之後，對每個參考列表分開平均所有可用運動向量。平均運動向量被指定為當前子CU的運動向量。The motion derivation of sub CU A starts by identifying its two spatial neighbors. The first neighbor is the N×N block above the sub CU A 1101 (block c 1113). If the block c (1113) is not available or is intra-coded, then check (from left to right, starting from block c 1113) the other N×N blocks above sub-CU A (1101). The second neighbor is the block to the left of the sub CU A 1101 (block b 1112). If block b (1112) is unavailable or is intra-coded, then check (from top to bottom, starting from block b 1112) other blocks on the left of sub-CU A 1101. The motion information obtained from the neighboring blocks of each list is scaled to the first reference frame of a given list. Next, the temporal motion vector predictor (TMVP) of sub-block A 1101 is derived by following the same process as the TMVP derivation specified in HEVC. The motion information of the co-bit block at block D 1104 is acquired and scaled accordingly. Finally, after retrieving and scaling the motion information, all available motion vectors are separately averaged for each reference list. The average motion vector is designated as the motion vector of the current sub-CU.

2.1.3子CU運動預測模式信令通知的示例2.1.3 Example of sub-CU motion prediction mode signaling notification

在一些實施例中，子CU模式被啟用作為附加的Merge候選，並且不需要附加的語法元素來信令通知模式。添加兩個附加的Merge候選以合併每個CU的候選列表以表示ATMVP模式和STMVP模式。在其他實施例中，如果序列參數集指示啟用了ATMVP和STMVP，則可以使用多達七個Merge候選。附加Merge候選的編碼邏輯與HM中的Merege候選的編碼邏輯相同，這意味著，對於P或B條帶中的每個CU，兩個附加Merge候選可能需要再進行兩次RD校驗。在一些實施例中，例如JEM，Merge索引的所有二進位位元由CABAC（基於上下文的自我調整二進位算術編碼）進行上下文編碼。在其他實施例中，例如，HEVC，僅對第一個二進位數字（bin）進行上下文編碼，並且對剩餘的二進位數字進行上下文旁路編碼。In some embodiments, the sub-CU mode is enabled as an additional Merge candidate, and no additional syntax elements are required to signal the mode. Add two additional Merge candidates to merge the candidate list of each CU to represent ATMVP mode and STMVP mode. In other embodiments, if the sequence parameter set indicates that ATMVP and STMVP are enabled, up to seven Merge candidates may be used. The coding logic of the additional Merge candidate is the same as that of the Merege candidate in the HM, which means that for each CU in the P or B slice, two additional Merge candidates may need to perform two more RD checks. In some embodiments, such as JEM, all binary bits indexed by Merge are context-encoded by CABAC (Context-based Self-Adjusting Binary Arithmetic Coding). In other embodiments, for example, HEVC, only the first binary number (bin) is context-encoded, and the remaining binary numbers are context-bypass-encoded.

2.2自我調整運動向量差異解析度的示例2.2 Example of self-adjusting motion vector difference resolution

在一些實施例中，當條帶標頭中的use_integer_mv_flag等於0時，以四分之一亮度樣點為單位，信令通知（PU的運動向量與預測的運動向量之間的）運動向量差異（MVD）。在JEM中，引入了局部自我調整運動向量解析度（LAMVR）。在JEM中，MVD可以以四分之一亮度樣點、整數亮度樣點或四個亮度樣點為單位進行編碼。在編碼單元（CU）級別控制MVD解析度，並且向具有至少一個非零MVD分量的每個CU，有條件地信令通知MVD解析度標誌。In some embodiments, when the use_integer_mv_flag in the slice header is equal to 0, the motion vector difference (between the motion vector of the PU and the predicted motion vector) is signaled in units of a quarter luminance sample ( MVD). In JEM, local self-adjusting motion vector resolution (LAMVR) is introduced. In JEM, MVD can be coded in units of one-quarter luminance samples, integer luminance samples, or four luminance samples. The MVD resolution is controlled at the coding unit (CU) level, and each CU with at least one non-zero MVD component is conditionally signaled with an MVD resolution flag.

對於具有至少一個非零MVD分量的CU，信令通知第一標記，以指示是否在CU中使用四分之一亮度樣點MV精度。當第一標誌（等於1）指示未使用四分之一亮度樣點MV精度時，信令通知另一個標誌，以指示是否使用整數亮度樣點MV精度或四個亮度樣點MV精度。For a CU with at least one non-zero MVD component, a first flag is signaled to indicate whether to use a quarter-luminance sample MV accuracy in the CU. When the first flag (equal to 1) indicates that one-quarter luminance sample MV accuracy is not used, another flag is signaled to indicate whether to use integer luminance sample MV accuracy or four luminance sample MV accuracy.

當CU的第一MVD解析度標誌為零，或未針對CU編碼（意味著CU中的所有MVD均為零）時，四分之一亮度樣點MV解析度被用於該CU。當CU使用整數亮度樣點MV精度或四個亮度樣點MV精度時，該CU的AMVP候選列表中的MVP被取整到對應的精度。When the first MVD resolution flag of the CU is zero or is not coded for the CU (meaning that all MVDs in the CU are zero), the MV resolution of a quarter of the luminance sample is used for the CU. When a CU uses integer luminance sample MV precision or four luminance sample MV precision, the MVP in the AMVP candidate list of the CU is rounded to the corresponding precision.

在編碼器中，使用CU級別RD校驗來確定將哪個MVD解析度將用於CU。換言之，對於每個MVD解析度，執行CU級RD校驗三次。為了加快編碼器速度，在JEM中應用以下編碼方案。In the encoder, a CU level RD check is used to determine which MVD resolution will be used for the CU. In other words, for each MVD resolution, CU-level RD verification is performed three times. In order to speed up the encoder, the following coding scheme is applied in JEM.

在具有正常四分之一亮度樣點MVD解析度的CU的RD校驗期間，儲存該當前CU的運動信息（整數亮度樣點精度）。對於具有整數亮度樣點和4個亮度樣點MVD解析度的相同CU，儲存的運動信息（取整後）被用作RD校驗期間進一步小範圍運動向量細化的起點，使得耗時的運動估計過程不重複三次。During the RD verification period of a CU with a normal quarter luminance sample MVD resolution, the motion information of the current CU (integer luminance sample accuracy) is stored. For the same CU with integer luma samples and 4 luma samples with MVD resolution, the stored motion information (rounded) is used as the starting point for further refinement of small-range motion vectors during RD verification, making time-consuming motion The estimation process is not repeated three times.

有條件地調用具有4個亮度樣點MVD解析度的CU的RD校驗。對於CU，當整數亮度樣點MVD解析度的RD成本遠大於四分之一亮度樣點MVD解析度的RD成本時，跳過該CU的4個亮度樣點MVD解析度的RD校驗。Conditionally invoke the RD check of the CU with 4 luminance samples MVD resolution. For a CU, when the RD cost of the integer luminance sample MVD resolution is much greater than the RD cost of a quarter luminance sample MVD resolution, skip the RD check of the CU's 4 luminance sample MVD resolution.

2.3較高的運動向量儲存精度的示例2.3 Examples of higher motion vector storage accuracy

在HEVC中，運動向量精度是四分之一像素（pel）（用於4:2:0視訊的四分之一亮度樣點和八分之一色度樣點）。在JEM中，內部運動向量儲存和Merge候選的精度增加到1/16像素。較高的運動向量精度（1/16像素）用於以跳過模式/Merge模式編碼的CU的運動補償幀間預測。對於使用正常AMVP模式編碼的CU，使用整數像素或四分之一像素運動。In HEVC, the motion vector accuracy is one-quarter pixel (pel) (one-quarter luminance sample and one-eighth chrominance sample used for 4:2:0 video). In JEM, the accuracy of internal motion vector storage and Merge candidates is increased to 1/16 pixel. Higher motion vector accuracy (1/16 pixel) is used for motion compensation inter prediction of CU coded in skip mode/Merge mode. For CUs coded using the normal AMVP mode, integer pixel or quarter pixel motion is used.

具有與HEVC運動補償內插濾波器相同的濾波器長度和歸一化因數的SHVC上採樣內插濾波器，被用作附加分數像素位置的運動補償內插濾波器。在JEM中色度分量運動向量精度是1/32樣點，通過使用兩個相鄰1/16像素分數位置的濾波器的平均值，來導出1/32像素分數位置的附加內插濾波器。The SHVC up-sampling interpolation filter, which has the same filter length and normalization factor as the HEVC motion compensation interpolation filter, is used as a motion compensation interpolation filter for adding fractional pixel positions. In JEM, the accuracy of the chrominance component motion vector is 1/32 sample points. By using the average value of the filters of two adjacent 1/16 pixel score positions, an additional interpolation filter at 1/32 pixel score positions is derived.

2.4重疊塊的運動補償（OBMC）的示例2.4 Example of overlapping block motion compensation (OBMC)

在JEM中，OBMC可以在CU級別使用語法元素進行開關。當JEM中使用OBMC時，對除去CU的右側邊界和底部邊界的所有運動補償（MC）塊邊界執行OBMC。此外，它應用於亮度和色度分量。JEM中，MC塊對應於編碼塊。當CU使用子CU模式（包括子CU Merge、仿射以及FRUC模式）編碼時，CU的每個子塊是MC塊。為了統一處理CU的邊界，在子塊的尺寸設置為4x4的情況下，對所有MC塊邊界以子塊級別執行OBMC，如圖12A和圖12B所示。In JEM, OBMC can be switched at the CU level using syntax elements. When OBMC is used in JEM, OBMC is performed on all motion compensation (MC) block boundaries excluding the right and bottom boundaries of the CU. In addition, it applies to luminance and chrominance components. In JEM, the MC block corresponds to the coding block. When the CU is encoded using the sub-CU mode (including sub-CU Merge, affine, and FRUC modes), each sub-block of the CU is an MC block. In order to process the boundaries of the CU uniformly, when the size of the sub-block is set to 4x4, OBMC is performed at the sub-block level on all MC block boundaries, as shown in FIGS. 12A and 12B.

圖12A示出了CU/PU邊界處的子塊，陰影子塊是應用OBMC的位置。類似地，圖12B示出了ATMVP模式中的子PU。FIG. 12A shows the sub-blocks at the CU/PU boundary, and the shaded sub-block is the location where OBMC is applied. Similarly, FIG. 12B shows the sub-PU in the ATMVP mode.

當OBMC應用於當前子塊時，除了當前MV之外，四個相鄰子塊的向量（如果可用且與當前運動向量不完全相同）也會被用於導出當前子塊的預測塊。組合這些基於多個運動向量的多個預測塊以生成當前子塊的最終預測信號。When OBMC is applied to the current sub-block, in addition to the current MV, the vectors of the four adjacent sub-blocks (if available and not exactly the same as the current motion vector) will also be used to derive the prediction block of the current sub-block. These multiple prediction blocks based on multiple motion vectors are combined to generate the final prediction signal of the current sub-block.

基於相鄰子塊的運動向量的預測塊被表示為P_N ，其中N指示相鄰上方、下方、左側和右側的子塊的索引，並且基於當前子塊的運動向量的預測塊被表示為PC。當P_N 基於相鄰子塊的運動信息且該運動信息與當前子塊的運動信息相同時，不從P_N 執行OBMC。否則，將P_N 的每個樣點添加到PC中的相同樣點中，即將四行/列P_N 添加到PC。加權因數{1/4, 1/8, 1/16, 1/32}用於P_N ，並且加權因數{3/4, 7/8, 15/16, 31/32}用於PC。例外是對於小MC塊（即，當編碼塊的高度或寬度等於4或CU使用子CU模式編碼時）僅將P_N 的兩行/列添加到PC。在這種情況下，加權因數{1/4, 1/8}用於P_N ，並且加權因數{3/4, 7/8}用於PC。對於基於垂直（水平）相鄰子塊的運動向量生成的P_N ，將P_N 的相同行（列）中的樣點添加到具有相同加權因數的PC。The prediction block based on the motion vector of the adjacent sub-block is represented as P _N , where N indicates the index of the adjacent upper, lower, left, and right sub-blocks, and the prediction block based on the motion vector of the current sub-block is represented as PC . When the same P _N adjacent sub block based on the motion information and the motion information of the motion information of the current sub-block is not performing OBMC from P _N. Otherwise, add each sample point of P _N to the same point in the PC, that is, add four rows/columns P _N to the PC. Weighting factors {1/4, 1/8, 1/16, 1/32} are used for P _N , and weighting factors {3/4, 7/8, 15/16, 31/32} are used for PC. The exception is that for small MC blocks (ie, when the height or width of the coded block is equal to 4 or the CU is coded using the sub-CU mode), only two rows/columns of P _N are added to the PC. In this case, weighting factors {1/4, 1/8} are used for P _N , and weighting factors {3/4, 7/8} are used for PC. For _PN generated based on the motion vectors of vertically (horizontal) adjacent sub-blocks, samples in the same row (column) of _PN are added to PCs with the same weighting factor.

在JEM中，對於尺寸小於或等於256個亮度樣點的CU，信令通知CU級別標誌以指示是否對當前CU應用OBMC。對於尺寸大於256個亮度樣點或未使用AMVP模式編碼的CU，預設情況下應用OBMC。在編碼器處，當OBMC應用於CU時，在運動估計階段期間考慮其影響。由OBMC使用頂部相鄰塊和左相鄰塊的運動信息形成的預測信號用於補償當前CU的原始信號的頂部邊界和左側邊界，並且然後應用正常運動估計過程。In JEM, for a CU whose size is less than or equal to 256 luminance samples, a CU level flag is signaled to indicate whether to apply OBMC to the current CU. For CUs whose size is greater than 256 luminance samples or are not coded in AMVP mode, OBMC is applied by default. At the encoder, when OBMC is applied to the CU, its impact is considered during the motion estimation phase. The prediction signal formed by OBMC using the motion information of the top neighboring block and the left neighboring block is used to compensate the top boundary and the left boundary of the original signal of the current CU, and then the normal motion estimation process is applied.

2.5局部亮度補償（LIC）的示例2.5 Example of Local Luminance Compensation (LIC)

LIC基於用於亮度變化的線性模型，使用縮放因數a和偏移b。並且，針對每個幀間模式編碼的編碼單元（CU）自我調整地啟用或禁用LIC。LIC is based on a linear model for brightness changes, using a scaling factor a and an offset b. And, the LIC is enabled or disabled in self-adjustment for each coding unit (CU) coded in the inter mode.

當LIC應用於CU時，採用最小平方誤差方法，通過使用當前CU的相鄰樣點及其對應的參考樣點來導出參數a和b。圖13示出了用於導出IC演算法的參數的相鄰樣點的示例。具體地，並且如圖13所示，使用了該CU的子採樣（2:1子採樣）的相鄰樣點和參考圖像中的對應樣點（其由當前CU或子CU的運動信息識別）。IC參數被導出並被分別應用於每個預測方向。When LIC is applied to CU, the least square error method is adopted, and the parameters a and b are derived by using the neighboring samples of the current CU and the corresponding reference samples. Figure 13 shows an example of neighboring samples used to derive the parameters of the IC algorithm. Specifically, and as shown in Fig. 13, the adjacent samples of the sub-sampling (2:1 sub-sampling) of the CU and the corresponding samples in the reference image (which are identified by the motion information of the current CU or sub-CU) are used. ). The IC parameters are derived and applied to each prediction direction separately.

當使用Merge模式對CU進行編碼時，以類似於Merge模式中的運動信息複製的方式從相鄰塊複製LIC標誌；否則，向該CU信令通知LIC標誌以指示LIC是否適用。When the CU is encoded in the Merge mode, the LIC flag is copied from adjacent blocks in a manner similar to the motion information copy in the Merge mode; otherwise, the LIC flag is signaled to the CU to indicate whether the LIC is applicable.

當對圖像啟用LIC時，需要附加的CU級別RD校驗以確定是否對CU應用LIC。當對CU啟用LIC時，分別對整數像素運動搜索和分數像素運動搜索，使用均值移除的絕對差和（mean-removed sum of absolute diffefference，MR-SAD）以及均值移除的絕對哈達瑪變換差和（mean-removed sum of absolute Hadamard-transformed difference，MR-SATD），而不是SAD和SATD。When LIC is enabled for images, additional CU-level RD checks are required to determine whether to apply LIC to CU. When LIC is enabled for CU, use mean-removed sum of absolute diffefference (MR-SAD) and absolute Hadamard transform difference of mean-removed sum of absolute difference (MR-SAD) for integer pixel motion search and fractional pixel motion search respectively Sum (mean-removed sum of absolute Hadamard-transformed difference, MR-SATD) instead of SAD and SATD.

為了降低編碼複雜度，在JEM中應用以下編碼方案。In order to reduce coding complexity, the following coding scheme is applied in JEM.

當當前圖像與其參考圖像之間不存在明顯的亮度變化時，對整個圖像禁用LIC。為了識別這種情況，在編碼器處，計算當前圖像與該當前圖像的每個參考圖像的長條圖。如果當前圖像與該當前圖像的每個參考圖像之間的長條圖差異小於給定閾值，則對當前圖像禁用LIC；否則，對當前圖像啟用LIC。When there is no obvious brightness change between the current image and its reference image, LIC is disabled for the entire image. In order to recognize this situation, at the encoder, a bar graph of the current image and each reference image of the current image is calculated. If the bar graph difference between the current image and each reference image of the current image is less than a given threshold, LIC is disabled for the current image; otherwise, LIC is enabled for the current image.

2.6仿射運動補償預測的示例2.6 Example of affine motion compensation prediction

在HEVC中，僅將平移運動模型應用於運動補償預測（MCP）。然而，相機和物件可以具有多種運動，例如放大/縮小、旋轉、透視運動和其他不規則的運動。在另一方面，JEM應用簡化的仿射變換運動補償預測。圖14示出了由兩個控制點運動向量V0和V1描述的塊1400的仿射運動場的示例。塊1400的運動向量場（MVF）可以由以下等式描述：In HEVC, only the translational motion model is applied to motion compensation prediction (MCP). However, cameras and objects can have multiple movements, such as zoom in/out, rotation, perspective motion, and other irregular motions. On the other hand, JEM applies simplified affine transform motion compensation prediction. FIG. 14 shows an example of the affine motion field of the block 1400 described by two control point motion vectors V0 and V1. The motion vector field (MVF) of block 1400 can be described by the following equation:

(1)

如圖14所示，（v_0x , v_0y ）是左頂角控制點的運動向量，（v_1x , v_1y ）是右頂角控制點的運動向量。為了簡化運動補償預測，可以應用基於子塊的仿射變換預測。子塊尺寸

如以下等式導出：As shown in Figure 14, (v _0x , v _0y ) is the motion vector of the control point at the left vertex, (v _1x , v _1y ) is the motion vector of the control point at the right vertex. In order to simplify the motion compensation prediction, sub-block-based affine transform prediction can be applied. Sub-block size

Derived as the following equation:

(2)

其中MvPre是運動向量分數精度（例如，在JEM中為1/16），（v_2x , v_2y ）是左下控制點的運動向量，根據等式（1）計算。如果需要，可以向下調整M和N，以使其分別為w和h的除數。Where MvPre is the score accuracy of the motion vector (for example, 1/16 in JEM), (v _2x , v _2y ) is the motion vector of the lower left control point, calculated according to equation (1). If necessary, you can adjust M and N downwards so that they are divisors of w and h, respectively.

圖15示出了塊1500的每個子塊的仿射MVF的示例。為了導出每個M×N子塊的運動向量，可以根據等式（1）計算每個子塊的中心樣點的運動向量，並將其取整為運動向量分數精度（例如，在JEM中為1/16）。然後，可以應用運動補償內插濾波器，以利用導出的運動向量生成每個子塊的預測。在MCP之後，每個子塊的高精度運動向量以與正常運動向量相同的精度被取整並保存。FIG. 15 shows an example of the affine MVF of each sub-block of the block 1500. In order to derive the motion vector of each M×N sub-block, the motion vector of the center sample point of each sub-block can be calculated according to equation (1) and rounded to the accuracy of the motion vector score (for example, 1 in JEM) /16). Then, a motion compensation interpolation filter can be applied to generate a prediction for each sub-block using the derived motion vector. After the MCP, the high-precision motion vector of each sub-block is rounded and saved with the same accuracy as the normal motion vector.

在JEM中，存在兩種仿射運動模式：AF_INTER模式和AF_MERGE模式。對於寬度和高度均大於8的CU，可以應用AF_INTER模式。在位元流中信令通知CU級別的仿射標誌，以指示是否使用AF_INTER模式。在AF_INTER模式中，使用相鄰塊構建具有運動向量對

的候選列表。In JEM, there are two affine motion modes: AF_INTER mode and AF_MERGE mode. For CUs whose width and height are both greater than 8, AF_INTER mode can be applied. The CU-level affine flag is signaled in the bit stream to indicate whether to use the AF_INTER mode. In AF_INTER mode, adjacent blocks are used to construct pairs with motion vectors

List of candidates.

圖16示出了用於在AF_INTER模式中的塊1600的運動向量預測（MVP）的示例。如圖16所示，從塊A、B或C的運動向量中選擇

。來自相鄰塊的運動向量可以根據參考列表來縮放。運動向量還可以根據相鄰塊的參考的圖像順序計數（POC）、當前CU的參考的POC和當前CU的POC之間的關係來縮放。從相鄰子塊D和E中選擇

的方法是類似的。如果候選列表的數量小於2，則由通過重複每個AMVP候選而構建的運動向量對來填充該列表。當候選列表大於2時，首先可以根據相鄰運動向量（例如，基於候選對中的兩個運動向量的相似性）對候選進行分類。在一些實現方式中，保留前兩個候選。在一些實現方式中，速率失真（RD）成本校驗用於確定選擇哪個運動向量對候選作為當前CU的控制點運動向量預測（CPMVP）。可以在位元流中信令通知指示候選列表中的CPMVP的位置的索引。在確定當前仿射CU的CPMVP之後，應用仿射運動估計，並找到控制點運動向量（CPMV）。然後在位元流中信令通知CPMV與CPMVP的差異。FIG. 16 shows an example of motion vector prediction (MVP) for block 1600 in AF_INTER mode. As shown in Figure 16, select from the motion vectors of block A, B or C

. Motion vectors from neighboring blocks can be scaled according to the reference list. The motion vector can also be scaled according to the relationship between the referenced picture order count (POC) of the neighboring block, the referenced POC of the current CU, and the POC of the current CU. Select from adjacent sub-blocks D and E

The method is similar. If the number of the candidate list is less than 2, the list is filled by a pair of motion vectors constructed by repeating each AMVP candidate. When the candidate list is greater than 2, the candidates can be classified according to adjacent motion vectors (for example, based on the similarity of the two motion vectors in the candidate pair). In some implementations, the first two candidates are retained. In some implementations, the rate distortion (RD) cost check is used to determine which motion vector pair candidate is selected as the control point motion vector prediction (CPMVP) of the current CU. The index indicating the position of the CPMVP in the candidate list may be signaled in the bit stream. After determining the CPMVP of the current affine CU, apply affine motion estimation and find the control point motion vector (CPMV). Then signal the difference between CPMV and CPMVP in the bit stream.

當在AF_MERGE模式中應用CU時，它從有效的相鄰重建塊獲得使用仿射模式編碼的第一塊。圖17A示出了當前CU 1700的候選塊的選擇順序的示例。如圖17A所示，選擇順序可以是從當前CU 1700的左方（1701）、上方（1702）、右上方（1703）、左下方（1704）到左上方（1705）。圖17B示出了在AF_MERGE模式中的當前CU 1700的候選塊的另一個示例。如圖17B所示，如果相鄰左下塊1701以仿射模式編碼，則導出包含子塊1701的CU的左頂角、右上角和左底角的運動向量

、

和

。基於

、

和

來計算當前CU 1700的左頂角的運動向量

。可以相應地計算當前CU的右上方的運動向量

。When the CU is applied in the AF_MERGE mode, it obtains the first block encoded using the affine mode from the valid neighboring reconstructed block. FIG. 17A shows an example of the selection order of candidate blocks of the current CU 1700. As shown in FIG. 17A, the selection order may be from the left (1701), the upper (1702), the upper right (1703), the lower left (1704) to the upper left (1705) of the current CU 1700. FIG. 17B shows another example of candidate blocks of the current CU 1700 in the AF_MERGE mode. As shown in FIG. 17B, if the adjacent lower left block 1701 is coded in affine mode, the motion vectors of the top left corner, the top right corner, and the bottom left corner of the CU containing the sub-block 1701 are derived

,

with

. based on

,

with

To calculate the motion vector of the top left corner of the current CU 1700

. The motion vector of the upper right of the current CU can be calculated accordingly

.

在根據等式（1）中的仿射運動模型計算當前CU的CPMV

和

之後，可以生成該當前CU的MVF。為了識別當前CU是否使用AF_MERGE模式編碼，當存在至少一個相鄰塊以仿射模式編碼時，可以在位元流中信令通知仿射標誌。Calculate the CPMV of the current CU according to the affine motion model in equation (1)

with

After that, the MVF of the current CU can be generated. In order to identify whether the current CU uses AF_MERGE mode coding, when there is at least one neighboring block coded in affine mode, an affine flag can be signaled in the bit stream.

2.7模式匹配的運動向量推導（PMMVD）的示例2.7 Example of Motion Vector Derivation (PMMVD) for Pattern Matching

PMMVD模式是一種基於畫面播放速率上轉換（FRUC）方法的特殊Merge模式。使用該模式，塊的運動信息不被信令通知，而是在解碼器側導出。PMMVD mode is a special Merge mode based on the FRUC method. Using this mode, the motion information of the block is not signaled, but is derived on the decoder side.

當CU的Merge標誌為真時，可以向該CU信令通知FRUC標誌。當FRUC標誌為假時，可以信令通知Merge索引，並使用常規Merge模式。當FRUC標誌為真時，可以信令通知附加的FRUC模式標誌以指示將使用哪種方法（例如，雙邊匹配或模板匹配）來導出該塊的運動信息。When the Merge flag of the CU is true, the FRUC flag can be signaled to the CU. When the FRUC flag is false, the Merge index can be signaled and the regular Merge mode is used. When the FRUC flag is true, an additional FRUC mode flag can be signaled to indicate which method (for example, bilateral matching or template matching) will be used to derive the motion information of the block.

在編碼器側，關於是否對CU使用FRUC Merge模式的決定是基於如對正常Merge候選那樣所做的RD成本選擇。例如，通過使用RD成本選擇來校驗CU的多種匹配模式（例如，雙邊匹配和模板匹配）。導致最小成本的匹配模式與其他CU模式進一步比較。如果FRUC匹配模式是最有效的模式，則對於CU將FRUC標誌設置為真，並且使用有關匹配模式。On the encoder side, the decision on whether to use the FRUC Merge mode for the CU is based on the RD cost selection as done for the normal Merge candidates. For example, by using RD cost selection to verify multiple matching modes of CU (for example, bilateral matching and template matching). The matching mode that results in the least cost is further compared with other CU modes. If the FRUC matching mode is the most effective mode, set the FRUC flag to true for the CU, and use the relevant matching mode.

通常，FRUC Merge模式中的運動推導過程有兩個步驟。首先執行CU級別運動搜索，接下來執行子CU級別運動細化。在CU級別，基於雙邊匹配或模板匹配為整個CU導出初始運動向量。首先，生成MV候選列表，並且選擇導致最小匹配成本的候選作為進一步CU級別細化的起點。然後，圍繞起始點執行基於雙邊匹配或模板匹配的局部搜索。將導致最小匹配成本的MV作為整個CU的MV。隨後，運動信息在子CU級別進一步細化，其中導出的CU運動向量作為起點。Generally, the motion derivation process in FRUC Merge mode has two steps. The CU level motion search is performed first, and then the sub-CU level motion refinement is performed. At the CU level, the initial motion vector is derived for the entire CU based on bilateral matching or template matching. First, a list of MV candidates is generated, and the candidate that leads to the smallest matching cost is selected as the starting point for further CU level refinement. Then, a local search based on bilateral matching or template matching is performed around the starting point. The MV that leads to the smallest matching cost is taken as the MV of the entire CU. Subsequently, the motion information is further refined at the sub-CU level, with the derived CU motion vector as the starting point.

例如，針對

CU運動信息推導執行以下推導處理。在第一階段，導出整體

CU的MV。在第二階段，CU進一步劃分為

子CU。如等式（3）中計算

的值，

是預定義的劃分深度，其在JEM中默認設置為3。然後導出每個子CU的MV。For example, for

The CU motion information derivation executes the following derivation processing. In the first stage, export the whole

CU’s MV. In the second stage, CU is further divided into

Child CU. As calculated in equation (3)

The value of

It is a predefined division depth, which is set to 3 by default in JEM. Then export the MV of each sub-CU.

} (3)

圖18示出了畫面播放速率上轉換（FRUC）方法中使用的雙邊匹配的示例。雙邊匹配用於通過在兩個不同參考圖像（1810,1811）中沿當前CU（1800）的運動軌跡找到兩個塊之間的最接近匹配，來導出當前CU的運動信息。在連續運動軌跡的假設下，指向兩個參考塊的運動向量MV0（1801）和MV1（1802）與在當前圖像和兩個參考圖像之間的時間距離——例如TD0（1803）和TD1（1804）——成比例。在一些實施例中，當當前圖像（1800）在時間上在兩個參考圖像（1810, 1811）之間並且從當前圖像到兩個參考圖像的時間距離相同時，雙邊匹配變為基於鏡像的雙向MV。FIG. 18 shows an example of bilateral matching used in the frame rate up conversion (FRUC) method. Bilateral matching is used to derive the motion information of the current CU by finding the closest match between two blocks along the motion trajectory of the current CU (1800) in two different reference images (1810, 1811). Under the assumption of continuous motion trajectories, the motion vectors MV0 (1801) and MV1 (1802) pointing to the two reference blocks and the time distance between the current image and the two reference images—for example, TD0 (1803) and TD1 (1804)-Proportional. In some embodiments, when the current image (1800) is between two reference images (1810, 1811) in time and the time distance from the current image to the two reference images is the same, the bilateral matching becomes Two-way MV based on mirroring.

圖19示出了畫面播放速率上轉換（FRUC）方法中使用的模板匹配的示例。模板匹配可以用於通過找到在當前圖像中的模板（例如，當前CU的頂部相鄰塊和/或左方相鄰塊）與參考圖像1910中的塊（例如，具有與模板相同的尺寸）之間的最接近匹配，來導出當前CU 1900的運動信息。除了上述FRUC Merge模式之外，模板匹配也可以適用於AMVP模式。在JEM和HEVC兩者中，AMVP有兩個候選。使用模板匹配方法，可以導出新的候選。如果由模板匹配的新導出的候選與第一現有AMVP候選不同，則將其***AMVP候選列表的最開始，並且然後將列表尺寸設置為2（例如，通過移除第二現有AMVP候選）。當應用於AMVP模式時，僅應用CU級別搜索。FIG. 19 shows an example of template matching used in the frame rate up conversion (FRUC) method. Template matching can be used to find the difference between the template in the current image (for example, the top adjacent block and/or the left adjacent block of the current CU) and the block in the reference image 1910 (for example, having the same size as the template). The closest match between the two to derive the current CU 1900 motion information. In addition to the aforementioned FRUC Merge mode, template matching can also be applied to the AMVP mode. In both JEM and HEVC, AMVP has two candidates. Using the template matching method, new candidates can be derived. If the newly derived candidate matched by the template is different from the first existing AMVP candidate, it is inserted at the very beginning of the AMVP candidate list, and then the list size is set to 2 (for example, by removing the second existing AMVP candidate). When applied to AMVP mode, only CU level search is applied.

CU級別的MV候選集合可以包括：（1）如果當前CU處於AMVP模式，則為原始AMVP候選，（2）所有Merge候選，（3）內插MV域（稍後描述）中的數個MV，以及頂部和左方相鄰的運動向量。The MV candidate set at the CU level may include: (1) if the current CU is in AMVP mode, it is the original AMVP candidate, (2) all Merge candidates, (3) interpolated several MVs in the MV domain (described later), And the motion vectors adjacent to the top and left.

當使用雙邊匹配時，Merge候選的每個有效MV可以被用作輸入，以在假設雙邊匹配的情況下生成MV對。例如，Merge候選的一個有效MV是在參考列表A中的（MVa，ref_a ）。然後，在其他參考列表B中找到其配對雙邊MV的參考圖像ref_b ，使得ref_a 和ref_b 在時間上位於當前圖像的不同側。如果參考列表B中這樣的ref_b 不可用，則ref_b 被確定為與ref_a 不同的參考，並且ref_b 到當前圖像的時間距離是列表B中的最小值。在確定ref_b 之後，基於當前圖像與ref_a 、ref_b 之間的時間距離通過縮放MVa來導出MVb。When bilateral matching is used, each valid MV of the Merge candidate can be used as input to generate MV pairs assuming bilateral matching. For example, a valid MV of the Merge candidate is in the reference list A (MVa, ref _a ). Then, find the reference image ref _{b of} its paired bilateral MV in the other reference list B, so that ref _a and ref _b are located on different sides of the current image in time. If such ref _b in reference list B is not available, ref _b is determined to be _a different reference from ref _a , and the time distance from ref _b to the current image is the minimum value in list B. After determining ref _b , MVb is derived by scaling MVa based on the time distance between the current image and ref _a and ref _b .

在一些實現方式中，來自內插MV域的四個MV也可以被添加到CU級別候選列表。更具體地，添加當前CU的位置(0, 0)、(W/2, 0)、(0, H/2)和(W/2, H/2)處的內插MV。當FRUC應用於AMVP模式時，原始AMVP候選也被添加到CU級別MV候選集合。在一些實現方式中，在CU級別，用於AMVP CU的15個MV、用於Merge CU的13個MV可以被添加到候選列表。In some implementations, four MVs from the interpolated MV domain can also be added to the CU-level candidate list. More specifically, the interpolated MVs at the positions (0, 0), (W/2, 0), (0, H/2), and (W/2, H/2) of the current CU are added. When FRUC is applied to AMVP mode, the original AMVP candidate is also added to the CU-level MV candidate set. In some implementations, at the CU level, 15 MVs for AMVP CU and 13 MVs for Merge CU can be added to the candidate list.

子CU級別的MV候選集合包括：（1）從CU級別搜索確定的MV，（2）頂部、左方、左頂和右頂的相鄰MV，（3）來自參考圖像的並列MV的縮放版本，（4）一個或多個ATMVP候選（例如，最多四個），以及（5）一個或多個STMVP候選（例如，最多四個）。來自參考圖像的縮放MV如下導出。遍歷兩個列表中的參考圖像。參考圖像中的子CU的並列位置處的MV被縮放到起始CU級別MV的參考。ATMVP和STMVP候選可以是前四個候選。在子CU級別，一個或多個MV（例如，最多十七個）被添加到候選列表中。The MV candidate set at the sub-CU level includes: (1) MV determined from the CU level search, (2) adjacent MVs at the top, left, top left, and top right, (3) scaling of the parallel MVs from the reference image Version, (4) one or more ATMVP candidates (for example, up to four), and (5) one or more STMVP candidates (for example, up to four). The zoomed MV from the reference image is derived as follows. Traverse the reference images in the two lists. The MV at the juxtaposed position of the sub-CU in the reference image is scaled to the reference of the starting CU level MV. The ATMVP and STMVP candidates can be the first four candidates. At the sub-CU level, one or more MVs (for example, up to seventeen) are added to the candidate list.

內插MV域的生成Generation of interpolated MV domain

在對幀進行編碼之前，基於單邊ME為整個圖像生成內插運動域。然後，運動域可以稍後用作CU級別或子CU級別MV候選。Before encoding the frame, an interpolated motion field is generated for the entire image based on a single-sided ME. Then, the motion domain can be used as a CU-level or sub-CU-level MV candidate later.

在一些實施例中，兩個參考列表中的每個參考圖像的運動域以4×4塊級別遍歷。圖20示出了在FRUC方法中的單向運動估計（ME）2000的示例。對於每個4×4塊，如果與塊相關聯的運動通過當前圖像中的4×4塊並且該塊尚未被分配任何內插運動，則參考塊的運動根據時間距離TD0和TD1（與HEVC中的TMVP的MV縮放的方式相同的方式）縮放到當前圖像，並且將縮放的運動分配給當前幀中的塊。如果無縮放的MV被分配到4×4塊，則在內插的運動域中將塊的運動標記為不可用。In some embodiments, the motion domain of each reference image in the two reference lists is traversed at a 4×4 block level. FIG. 20 shows an example of one-way motion estimation (ME) 2000 in the FRUC method. For each 4×4 block, if the motion associated with the block passes through the 4×4 block in the current image and the block has not been assigned any interpolated motion, the motion of the reference block is based on the temporal distance TD0 and TD1 (with HEVC The MV zoom in the TMVP in the same way) zoom to the current image, and assign the zoomed motion to the block in the current frame. If an unscaled MV is allocated to a 4×4 block, the motion of the block is marked as unavailable in the interpolated motion domain.

內插和匹配成本Interpolation and matching costs

當運動向量指向分數樣點位置時，需要運動補償內插。為了降低複雜度，雙邊匹配和模板匹配兩者可以使用雙線性內插而不是常規的8階HEVC內插。When the motion vector points to the position of the fractional sample point, motion compensation interpolation is required. In order to reduce complexity, both bilateral matching and template matching can use bilinear interpolation instead of conventional 8-order HEVC interpolation.

匹配成本的計算在不同的步驟有點不同。當從CU級別的候選集合中選擇候選時，匹配成本可以是雙邊匹配或模板匹配的絕對差值和（SAD）。在確定起始MV之後，如下計算子CU級別搜索的雙邊匹配的匹配成本

：The calculation of the matching cost is a bit different in different steps. When selecting candidates from the candidate set at the CU level, the matching cost may be the sum of absolute differences (SAD) of bilateral matching or template matching. After the starting MV is determined, the matching cost of the bilateral matching of the sub-CU level search is calculated as follows

:

(4)

其中

是一個加權因數。在一些實施例中，

根據經驗設置為4，MV和MV^s 分別指示當前MV和起始MV。SAD可以仍用作子CU級別搜索的模板匹配的匹配成本。among them

Is a weighting factor. In some embodiments,

Set to 4 based on experience, MV and MV ^s indicate the current MV and the starting MV respectively. SAD can still be used as the matching cost of template matching for sub-CU level searches.

在FRUC模式中，MV通過僅使用亮度樣點導出。導出的運動將用於MC幀間預測的亮度和色度。在確定MV之後，使用用於亮度的8階內插濾波器和用於色度的4階內插濾波器來執行最終MC。In FRUC mode, MV is derived by using only luminance samples. The derived motion will be used for the luma and chroma of MC inter prediction. After the MV is determined, the final MC is performed using an 8-order interpolation filter for luminance and a 4-order interpolation filter for chrominance.

MV細化是以雙邊匹配成本或模板匹配成本為準則的基於模式的MV搜索。在JEM中，支援兩種搜索模式——分別用於CU級別和子CU級別的MV細化的無限制的中心偏置菱形搜索（unrestricted center-biased diamond search，UCBDS）和自我調整交叉搜索（adaptive cross search）。對於CU級別和子CU級別MV細化，以四分之一亮度樣點MV精度直接搜索MV，並且接下來以八分之一亮度樣點MV細化。對於CU步驟和子CU步驟的MV細化的搜索範圍被設置為等於8個亮度樣點。MV refinement is a pattern-based MV search based on bilateral matching cost or template matching cost. In JEM, two search modes are supported-unrestricted center-biased diamond search (UCBDS) and adaptive cross search for CU-level and sub-CU-level MV refinement. search). For CU-level and sub-CU-level MV refinement, the MV is directly searched with a quarter-luminance sample MV accuracy, and then the MV is refined with one-eighth sample MV. The search range of MV refinement for the CU step and the sub-CU step is set equal to 8 luminance samples.

在雙邊匹配Merge模式中，應用雙向預測，因為基於在兩個不同參考圖像中沿當前CU的運動軌跡的兩個塊之間的最接近匹配來導出CU的運動信息。在模板匹配Merge模式中，編碼器可以在針對CU的來自列表0的單向預測、來自列表1的單向預測或者雙向預測之中進行選擇。選擇可以基於模板匹配成本，如下：如果costBi >= factor * min (cost0, cost1) 使用雙向預測；否則，如果cost0 >= cost1 使用來自列表0的單向預測；否則，使用來自列表1的單向預測；In the bilateral matching Merge mode, bi-directional prediction is applied because the motion information of the CU is derived based on the closest match between two blocks along the motion trajectory of the current CU in two different reference images. In the template matching Merge mode, the encoder can choose among unidirectional prediction from list 0, unidirectional prediction from list 1, or bidirectional prediction for the CU. Selection can be based on template matching cost, as follows: If costBi >= factor * min (cost0, cost1) Use bidirectional prediction; Otherwise, if cost0 >= cost1 Use one-way prediction from list 0; otherwise, Use the one-way forecast from Listing 1;

其中cost0是列表0模板匹配的SAD，cost1是列表1模板匹配的SAD，costBi是雙向預測模板匹配的SAD。例如，當factor的值等於1.25時，這意味著選擇過程偏向於雙向預測。幀間預測方向選擇可以應用於CU級別模板匹配過程。Where cost0 is the SAD matched by the template in list 0, cost1 is the SAD matched by the template in list 1, and costBi is the SAD matched by the bidirectional prediction template. For example, when the value of factor is equal to 1.25, it means that the selection process is biased towards bidirectional prediction. Inter-frame prediction direction selection can be applied to the CU-level template matching process.

2.8雙向光流（BIO）的示例2.8 Example of bidirectional optical flow (BIO)

雙向光流（BIO）方法是在用於雙向預測的逐塊運動補償之上執行的逐樣點運動細化。在一些實現方式中，樣點級別運動細化不使用信令通知。The bidirectional optical flow (BIO) method is a sample-by-sample motion refinement performed on top of the block-by-block motion compensation used for bidirectional prediction. In some implementations, sample level motion refinement does not use signaling.

讓

是在塊運動補償之後的參考k (k=0, 1)的亮度值，並且將

、

分別表示為

梯度的水平分量和垂直分量。假設光流有效，則運動向量場

由以下等式給出。Let

Is the luminance value of reference k (k=0, 1) after block motion compensation, and will

,

Denoted as

The horizontal and vertical components of the gradient. Assuming the optical flow is valid, the motion vector field

It is given by the following equation.

(5)

將此光流等式與Hermite內插組合，得到每個樣點的運動軌跡，其結果是匹配函數值

和偏導數

、

的唯一的三階多項式。在t=0時此多項式的值是BIO預測：Combine this optical flow equation with Hermite interpolation to get the trajectory of each sample point, and the result is the matching function value

And partial derivatives

,

The only third-order polynomial. At t=0, the value of this polynomial is the BIO prediction:

(6)

圖21示出了雙向光流（BIO）方法中的光流軌跡的示例。其中，

和

表示到參考幀的距離。距離

和

基於Ref0和Ref1的POC來計算：

=POC(當前)−POC(Ref0),

=POC(Ref1)−POC(當前)。如果兩個預測都來自相同的時間方向（既可以是來自過去的，也可以是來自未來的），則符號是不同的（例如，

）。在這種情況下，如果預測不是來自相同的時刻（例如，

），則應用BIO。兩個參考區域都具有非零運動（例如，

）並且塊運動向量與時間距離（例如，

）成正比。FIG. 21 shows an example of the optical flow trajectory in the bidirectional optical flow (BIO) method. among them,

with

Indicates the distance to the reference frame. distance

with

Calculate based on the POC of Ref0 and Ref1:

=POC(current)−POC(Ref0),

=POC(Ref1)−POC(current). If both predictions are from the same time direction (either from the past or from the future), the signs are different (for example,

). In this case, if the prediction is not from the same moment (for example,

), then apply BIO. Both reference regions have non-zero motion (e.g.,

) And the distance between the block motion vector and time (for example,

) Is directly proportional.

通過最小化點A和點B中的值之間的差異

來確定運動向量場

。圖22A和圖22B示出了運動軌跡和參考幀平面的交點的示例。模型僅使用對於

的局部泰勒展開的第一線性項：By minimizing the difference between the values in point A and point B

To determine the motion vector field

. 22A and 22B show examples of intersections of the motion trajectory and the reference frame plane. The model is only used for

The first linear term of the local Taylor expansion:

(7)

上述等式中的所有值都取決於表示為

的樣點位置。假設運動在局部周圍區域是一致的，可以在以當前預測點

為中心的(2M+1)x(2M+1)方形窗口Ω內最小化

，其中M等於2：All values in the above equation depend on the expression as

The sample point position. Assuming that the motion is consistent in the local surrounding area, the current prediction point can be used

Minimize within the centered (2M+1)x(2M+1) square window Ω

, Where M is equal to 2:

(8)

對於此優化問題，JEM使用簡化方法，首先在垂直方向上進行最小化，並且然後在水平方向上進行最小化。這導致For this optimization problem, JEM uses a simplified method, which first minimizes in the vertical direction, and then minimizes in the horizontal direction. Which resulted in

(9)

(10)

其中，among them,

(11)

為了避免除以零或非常小的值，在等式（9）和等式（10）中引入正則化參數r和m。In order to avoid dividing by zero or very small values, regularization parameters r and m are introduced in equation (9) and equation (10).

(12)

(13)

其中

是視訊樣點的位元深度。among them

Is the bit depth of the video sample point.

為了保持BIO的記憶體訪問與常規雙向預測運動補償相同，針對當前塊內的位置計算所有預測和梯度值

。圖22A示出了塊2200外部的訪問位置的示例。如圖22A所示，在等式（9）中，以預測塊的邊界上的當前預測點為中心的(2M+1)x(2M+1)方形視窗Ω需要訪問塊外的位置。在JEM中，塊外的

的值被設置為等於塊內最近的可用值。例如，這可以實現為填充區域2201，如圖22B所示。In order to keep the memory access of BIO the same as conventional bidirectional predictive motion compensation, all prediction and gradient values are calculated for the position in the current block

. FIG. 22A shows an example of an access location outside the block 2200. As shown in FIG. 22A, in equation (9), the (2M+1)×(2M+1) square window Ω centered on the current prediction point on the boundary of the prediction block needs to access a location outside the block. In JEM, outside the block

The value of is set equal to the nearest available value in the block. For example, this can be implemented as a filled area 2201, as shown in Figure 22B.

使用BIO，可以針對每個樣品來細化運動場是可能的。為了降低計算複雜度，可以在JEM中使用基於塊的BIO設計。可以基於4x4塊來計算運動細化。在基於塊的BIO中，可以聚合4x4塊中的所有樣點在等式（9）中的s_n 值，並且然後將聚合的s_n 值用於4x4塊的導出的BIO運動向量偏移。更具體地，以下公式可以用於基於塊的BIO推導：With BIO, it is possible to refine the sports field for each sample. In order to reduce computational complexity, block-based BIO design can be used in JEM. Motion refinement can be calculated based on 4x4 blocks. In block-based BIO, it is possible to aggregate the _sn values in equation (9) for all samples in the 4x4 block, and then use the aggregated _sn values for the derived BIO motion vector offset of the 4x4 block. More specifically, the following formula can be used for block-based BIO derivation:

(14)

其中b_k 表示屬於預測塊的第k個4×4塊的樣品的集合。由((s_n , b_k )>>4)替換等式（9）和等式（10）中的s_n 以導出相關聯的運動向量偏移。Where b _k represents a set of samples belonging to the kth 4×4 block of the prediction block. Replace s _n in equation (9) and equation (10) by ((s _n , b _k )>>4) to derive the associated motion vector offset.

在一些情況下，BIO的MV組（regiment）可能由於雜訊或不規則運動而不可靠。因此，在BIO中，MV組的量級被限制為閾值。基於當前圖像的參考圖像是否都來自一個方向來確定閾值。例如，如果當前圖像的所有參考圖像都來自一個方向，則閾值被設置為

；否則，它被設置為

。In some cases, BIO's MV regimen may be unreliable due to noise or irregular movement. Therefore, in BIO, the magnitude of the MV group is limited to a threshold. The threshold is determined based on whether the reference images of the current image all come from one direction. For example, if all reference images of the current image come from one direction, the threshold is set to

; Otherwise, it is set to

.

可以同時計算BIO的梯度與運動補償內插，該運動補償內插使用與HEVC運動補償過程一致的操作（例如，2D可分離有限脈衝回應FIR）。在一些實施例中，根據塊運動向量的分數部分，此2D可分離FIR的輸入是與運動補償過程和分數位置（fracX, fracY）相同的參考幀樣點。對於水平梯度

，首先對應於具有去縮放偏移d−8的分數位置fracY，使用BIOfilterS垂直內插信號，然後對應於具有去縮放偏移18−d的分數位置fracX，在水平方向上應用梯度濾波器BIOfilterG。對於垂直梯度

，首先對應於具有去縮放偏移d−8的分數位置fracY，使用BIOfilterG垂直應用梯度濾波器，然後對應於具有去縮放偏移18−d的分數位置fracX，在水平方向上使用BIOfilterS來執行信號位移。用於梯度計算BIOfilterG和信號位移BIOfilterF的內插濾波器的長度可以較短（例如，6階）以維持合理的複雜度。表格1示出了可以用於BIO中塊運動向量的不同分數位置的梯度計算的示例濾波器。表格2示出了可以用於BIO中預測信號生成的示例內插濾波器。The BIO gradient and motion compensation interpolation can be calculated at the same time, and the motion compensation interpolation uses operations consistent with the HEVC motion compensation process (for example, 2D separable finite impulse response FIR). In some embodiments, according to the fractional part of the block motion vector, the input of this 2D separable FIR is the same reference frame sample as the motion compensation process and fractional location (fracX, fracY). For horizontal gradient

, First corresponds to the fractional position fracY with a de-scaling offset d−8, using BIOfilterS to interpolate the signal vertically, and then corresponding to the fractional position fracX with a de-scaling offset of 18-d, the gradient filter BIOfilterG is applied in the horizontal direction. For vertical gradient

, First corresponding to the fractional position fracY with a de-scaling offset d−8, using BIOfilterG to apply the gradient filter vertically, and then corresponding to the fractional position fracX with a de-scaling offset of 18-d, using BIOfilterS to perform the signal in the horizontal direction Displacement. The length of the interpolation filter used for gradient calculation BIOfilterG and signal displacement BIOfilterF can be shorter (for example, 6th order) to maintain reasonable complexity. Table 1 shows example filters that can be used for gradient calculation of different score positions of block motion vectors in BIO. Table 2 shows example interpolation filters that can be used for prediction signal generation in BIO.

表格 1 BIO中梯度計算的示例性濾波器

Table 1 Exemplary filters for gradient calculation in BIO

表格2 BIO中預測信號生成的示例性內插濾波器

Table 2 Exemplary interpolation filter for prediction signal generation in BIO

在JEM中，當兩個預測來自不同的參考圖像時，BIO可以應用於所有雙向預測的塊。當對CU啟用局部亮度補償（LIC）時，可以禁用BIO。In JEM, when two predictions come from different reference images, BIO can be applied to all bidirectionally predicted blocks. When local brightness compensation (LIC) is enabled for the CU, BIO can be disabled.

在一些實施例中，OBMC在正常MC過程之後應用於塊。為了降低計算複雜度，在OBMC過程期間不可以應用BIO。這意味著當使用其自己的MV時BIO應用於塊的MC過程，當在OBMC過程中使用相鄰塊的MV時BIO不應用於MC過程。In some embodiments, OBMC is applied to the block after the normal MC process. In order to reduce computational complexity, BIO cannot be applied during the OBMC process. This means that BIO is applied to the MC process of a block when its own MV is used, and BIO is not applied to the MC process when the MV of an adjacent block is used in the OBMC process.

2.9解碼器側運動向量細化（DMVR）的示例2.9 Decoder-side motion vector refinement (DMVR) example

在雙向預測操作中，為了預測一個塊區域，分別使用列表0的運動向量（MV）和列表1的MV形成的兩個預測塊被組合以形成單個預測信號。在解碼器側運動向量細化（DMVR）方法中，雙向預測的兩個運動向量通過雙邊模板匹配過程進一步細化。雙邊模板匹配應用於解碼器中，以在雙邊模板和參考圖像中的重建樣點之間執行基於失真的搜索，以便在不傳輸附加的運動信息的情況下獲得細化的MV。In the bidirectional prediction operation, in order to predict one block area, two prediction blocks formed using the motion vector (MV) of List 0 and the MV of List 1 are combined to form a single prediction signal. In the decoder-side motion vector refinement (DMVR) method, the two motion vectors of bidirectional prediction are further refined through a bilateral template matching process. Bilateral template matching is applied in the decoder to perform a distortion-based search between the bilateral template and the reconstructed samples in the reference image to obtain a refined MV without transmitting additional motion information.

如圖23所示，在DMVR中，分別從列表0的初始MV0和列表1的MV1生成雙邊模板，作為兩個預測塊的加權組合（即平均）。模板匹配操作包括計算生成的模板與參考圖像中的樣點區域（在初始預測塊周圍）之間的成本度量。對於兩個參考圖像中的每一個，產生最小模板成本的MV被視為該列表的更新MV以替換原始模板。在JEM中，對於每個列表，搜索九個MV候選。該九個MV候選包括原始MV和8個周圍MV，其中一個亮度樣點在水平或垂直方向上或在兩個方向上偏移到原始MV。最後，如圖23所示，兩個新的MV，即MV0’和MV1’，被用於生成最終的雙向預測結果。絕對差值和（SAD）用作成本度量。As shown in Figure 23, in DMVR, a bilateral template is generated from the initial MV0 of list 0 and MV1 of list 1, respectively, as a weighted combination (ie, average) of two prediction blocks. The template matching operation includes calculating the cost metric between the generated template and the sample area (around the initial prediction block) in the reference image. For each of the two reference images, the MV that produces the smallest template cost is regarded as the updated MV of the list to replace the original template. In JEM, for each list, nine MV candidates are searched. The nine MV candidates include the original MV and 8 surrounding MVs, and one of the luminance samples is offset to the original MV in the horizontal or vertical direction or in both directions. Finally, as shown in Figure 23, two new MVs, namely MV0' and MV1', are used to generate the final bidirectional prediction result. The sum of absolute differences (SAD) is used as a cost metric.

DMVR被應用於雙向預測的Merge模式，在不傳輸附加的語法元素的情況下使用來自過去的參考圖像中的一個MV和來自將來的參考圖像中的另一個MV。在JEM中，當為CU啟用LIC、仿射運動、FRUC或子CU Merge候選時，不應用DMVR。DMVR is applied to the Merge mode of bidirectional prediction, using one MV from the past reference image and another MV from the future reference image without transmitting additional syntax elements. In JEM, when LIC, affine motion, FRUC, or sub-CU Merge candidates are enabled for CU, DMVR is not applied.

3．CABAC修改的示例3. CABAC modified example

在JEM中，與HEVC中的設計相比，CABAC包含以下三個主要變化： - 修改的變換係數的上下文建模 - 具有依賴於上下文的更新速度的多假設概率估計 - 上下文模型的自我調整初始化In JEM, compared with the design in HEVC, CABAC contains the following three main changes: -Context modeling of modified transform coefficients -Multi-hypothesis probability estimation with context-dependent update speed -Self-tuning initialization of the context model

3.1變換係數的上下文建模的示例3.1 Example of context modeling of transform coefficients

在HEVC中，使用非重疊係數群組（CG）對編碼塊的變換係數進行編碼，並且每個CG包含編碼塊的4×4塊的係數。編碼塊內的CG和CG內的變換係數根據預定義的掃描順序進行編碼。具有至少一個非零變換係數的CG的變換係數級別的編碼可以被分成多個掃描通道。在第一通道中，編碼第一個bin（由bin0表示，也稱為significant_coeff_flag，其指示係數的量級大於0）。接下來，可以應用用於上下文編碼第二/第三bin（分別由bin1和bin2表示，也稱為coeff_abs_greater1_flag和coeff_abs_greater2_flag）的兩個掃描通道。最後，如果需要，調用用於編碼符號信息和剩餘值（也稱為coeff_abs_level_remaining）的多於兩個掃描通道。只有前三個掃描通道中的bin以常規模式編碼，並且這些bin在下面的描述中稱為常規bin。In HEVC, a non-overlapping coefficient group (CG) is used to encode transform coefficients of a coding block, and each CG contains coefficients of a 4×4 block of the coding block. The CG in the coding block and the transform coefficients in the CG are coded according to a predefined scanning order. The coding of the transform coefficient level of the CG with at least one non-zero transform coefficient can be divided into multiple scanning channels. In the first pass, encode the first bin (represented by bin0, also called significant_coeff_flag, which indicates that the magnitude of the coefficient is greater than 0). Next, two scanning channels for the second/third bin of context coding (represented by bin1 and bin2, also called coeff_abs_greater1_flag and coeff_abs_greater2_flag) can be applied. Finally, if needed, call more than two scan channels for encoding symbol information and remaining values (also called coeff_abs_level_remaining). Only the bins in the first three scan channels are coded in regular mode, and these bins are called regular bins in the following description.

在JEM中，改變常規bin的上下文建模。當在第i個掃描通道（i為0,1,2）中對bin i進行編碼時，上下文索引取決於由本地模板覆蓋的鄰域中的先前編碼係數的第i個bin的值。具體地，基於相鄰係數的第i個bin的總和來確定上下文索引。In JEM, change the context modeling of regular bins. When bin i is encoded in the i-th scan channel (i is 0, 1, 2), the context index depends on the value of the i-th bin of the previously encoded coefficient in the neighborhood covered by the local template. Specifically, the context index is determined based on the sum of the i-th bin of adjacent coefficients.

如圖24所示，本地模板包含多達五個空間相鄰變換係數，其中x指示當前變換係數的位置，並且xi（i為0至4）指示其五個鄰居。為了捕獲不同頻率的變換係數的特性，可以將一個編碼塊劃分為多達三個區域，並且無論編碼塊尺寸如何，劃分方法都是固定的。例如，當對亮度變換係數的bin0進行編碼時，將一個編碼塊劃分為使用不同顏色標記的三個區域，並列出分配給每個區域的上下文索引。亮度和色度分量以類似的方式處理，但具有分開的上下文模型集。此外，亮度分量的bin0（即，有效標誌）的上下文模型選擇進一步取決於變換尺寸。As shown in Figure 24, the local template contains up to five spatially adjacent transform coefficients, where x indicates the position of the current transform coefficient, and xi (i is 0 to 4) indicates its five neighbors. In order to capture the characteristics of transform coefficients of different frequencies, one coding block can be divided into up to three regions, and the division method is fixed regardless of the size of the coding block. For example, when encoding bin0 of the luminance transform coefficient, one encoding block is divided into three regions marked with different colors, and the context index assigned to each region is listed. Luminance and chrominance components are handled in a similar way, but with separate sets of context models. In addition, the context model selection of bin0 (ie, valid flag) of the luminance component further depends on the transform size.

3.2多假設概率估計的示例3.2 Example of multi-hypothesis probability estimation

二進位算術編碼器基於與每個上下文模型相關聯的兩個概率估計P0和P1應用「多假設」概率更新模型，並且以不同的自我調整速率獨立地更新如下：The binary arithmetic encoder applies a "multi-hypothesis" probability update model based on the two probability estimates P0 and P1 associated with each context model, and updates the model independently at different self-adjusting rates as follows:

(15)

其中，

和

（j = 0,1）分別表示解碼bin之前和之後的概率。變數Mi（為4,5,6,7）是控制索引等於i的上下文模型的概率更新速度的參數；並且k表示概率的精度（本文等於15）。among them,

with

(J = 0, 1) respectively represent the probability before and after decoding the bin. The variable Mi (4, 5, 6, 7) is a parameter that controls the probability update speed of the context model with an index equal to i; and k represents the accuracy of the probability (this article is equal to 15).

用於二進位算術編碼器中的間隔細分的概率估計P是來自兩個假設的估計的平均值：The probability estimate P used for interval subdivision in the binary arithmetic encoder is the average of the estimates from two hypotheses:

(16)

在JEM中，控制每個上下文模型的概率更新速度的等式（15）中使用的參數Mi的值被分配如下：In JEM, the value of the parameter Mi used in equation (15) that controls the rate of probability update of each context model is assigned as follows:

在編碼器側，記錄與每個上下文模型相關聯的編碼的bin。在對一個條帶進行編碼之後，對於索引等於i的每個上下文模型，計算使用不同的Mi值（為4,5,6,7）的速率成本，並選擇提供最小速率成本的一個Mi值。為簡單起見，僅在遇到條帶類型和條帶級別量化參數的新組合時才執行該選擇過程。On the encoder side, the encoded bin associated with each context model is recorded. After encoding a slice, for each context model with an index equal to i, calculate the rate cost using different Mi values (4, 5, 6, 7), and select a Mi value that provides the smallest rate cost. For simplicity, this selection process is performed only when a new combination of band type and band level quantization parameters is encountered.

針對每個上下文模型i信令通知1位元標誌以指示Mi是否不同於預設值4。當標誌為1時，使用兩個位元來指示Mi是否等於5、6或7。A 1-bit flag is signaled for each context model i to indicate whether Mi is different from the preset value 4. When the flag is 1, two bits are used to indicate whether Mi is equal to 5, 6, or 7.

3.3上下文模型的初始化的示例3.3 Example of the initialization of the context model

代替在HEVC中使用用於上下文模型的初始化的固定表格，可以通過從先前編碼的圖像複製狀態來初始化用於幀間編碼的條帶的上下文模型的初始概率狀態。更具體地，在對每個圖像的中心定位的CTU進行編碼之後，儲存所有上下文模型的概率狀態以用作後續圖像上的對應上下文模型的初始狀態。在JEM中，從具有與當前條帶相同的條帶類型和相同的條帶級別QP的先前編碼的圖像的儲存狀態中複製每個幀間編碼條帶的初始狀態集。這缺乏損耗穩固性，但在當前的JEM方案中用於編碼效率實驗目的。Instead of using a fixed table for the initialization of the context model in HEVC, the initial probability state of the context model for the slice of inter coding can be initialized by copying the state from the previously coded image. More specifically, after encoding the CTU located at the center of each image, the probability states of all context models are stored to be used as the initial state of the corresponding context models on subsequent images. In JEM, the initial state set of each inter-coded slice is copied from the storage state of the previously encoded image having the same slice type and the same slice level QP as the current slice. This lacks loss stability, but is used for coding efficiency experiments in the current JEM scheme.

4.有關實施例和方法的示例4. Examples of embodiments and methods

與所公開的技術有關的方法包括擴展的LAMVR，其中支持的運動向量解析度範圍從1/4像素到4像素（1/4像素、1/2像素、1像素、2像素和4像素）。當信令通知MVD信息時，在CU級別信令通知關於運動向量解析度的信息。Methods related to the disclosed technology include extended LAMVR, where the supported motion vector resolution ranges from 1/4 pixel to 4 pixels (1/4 pixel, 1/2 pixel, 1 pixel, 2 pixel, and 4 pixel). When MVD information is signaled, information on the resolution of the motion vector is signaled at the CU level.

取決於CU的解析度，調整CU的運動向量（MV）和運動向量預測器（MVP）兩者。如果應用的運動向量解析度表示為R（R可以是¼、½、1、2、4），則MV（MV_x ，MV_y ）和MVP（MVP_x ，MVP_y ）表示如下：Depending on the resolution of the CU, both the motion vector (MV) and the motion vector predictor (MVP) of the CU are adjusted. If the resolution of the applied motion vector is expressed as R (R can be ¼, ½, 1, 2, 4), MV (MV _x , MV _y ) and MVP (MVP _x , MVP _y ) are expressed as follows:

(MV_x ,MV_y )=(Round(MV_x /(R*4))*(R*4),Round(MV_y /(R*4))*(R*4))(MV _x ,MV _y )=(Round(MV _x /(R*4))*(R*4),Round(MV _y /(R*4))*(R*4))

(MVP_x ,MVP_y )=(Round(MVP_x /(R*4))*(R*4),Round(MVP_y /(R*4))*(R*4))(MVP _x ,MVP _y )=(Round(MVP _x /(R*4))*(R*4),Round(MVP _y /(R*4))*(R*4))

因為運動向量預測器和MV兩者都由自我調整解析度調整，因此MVD（MVD_x ，MVD_y ）也與解析度對準，並且根據解析度信令通知如下：Because both the motion vector predictor and the MV are adjusted by the self-adjusting resolution, the MVD (MVD _x , MVD _y ) is also aligned with the resolution, and is signaled as follows according to the resolution:

(MVD_x ,MVD_y )=((MV_x –MVP_x )/(R*4), (MV_y –MVP_y )/R*4))(MVD _x ,MVD _y )=((MV _x --MVP _x )/(R*4), (MV _y --MVP _y )/R*4))

在該提議中，運動向量解析度索引（MVR索引）指示MVP索引以及運動向量解析度。結果，所提議的方法沒有MVP索引信令。表格示出了MVR索引的每個值所表示的內容。In this proposal, the motion vector resolution index (MVR index) indicates the MVP index and the motion vector resolution. As a result, the proposed method does not have MVP index signaling. The table shows what each value of the MVR index represents.

表格 3 MVR索引表示的示例

Table 3 Examples of MVR Index Representation

在雙向預測的情況下，AMVR針對每種解析度具有3種模式。AMVR雙向索引指示是否信令通知每個參考列表（列表0或列表1）的MVD_x 、MVD_y 。AMVR雙向索引的示例定義如下表所示。In the case of bidirectional prediction, AMVR has 3 modes for each resolution. The AMVR bidirectional index indicates whether to signal MVD _x and MVD _{y of} each reference list (list 0 or list 1). The example definition of AMVR bidirectional index is shown in the following table.

表格 4 AMVR雙向索引的示例

Table 4 Example of AMVR bidirectional index

5.現有實現方式的示例5. Examples of existing implementations

在使用BIO的一個現有實現方式中，列表0中的參考塊/子塊（由refblk0表示）和列表1中的參考塊/子塊（refblk1）之間的計算的MV——由(v_x , v_y )表示——僅用於當前塊/子塊的運動補償，並且不用於未來編碼塊的運動預測、去方塊、OBMC等，這可能是低效的。In an existing implementation using BIO, the calculated MV between the reference block/sub-block (represented by refblk0) in list 0 and the reference block/sub-block (refblk1) in list 1 is represented by (v _x , v _y ) means that it is only used for motion compensation of the current block/sub-block, and not used for motion prediction, deblocking, OBMC, etc. of future coded blocks, which may be inefficient.

在使用OBMC的另一個現有實現方式中，對於AMVP模式，在編碼器處確定是否對小塊（寬度*高度>= 256）啟用OBMC，並且信令通知解碼器。這增加了編碼器的複雜度。同時，對於給定的塊/子塊，當啟用OBMC時，它總是應用於亮度和色度，這可能導致編碼效率下降。In another existing implementation using OBMC, for AMVP mode, it is determined at the encoder whether to enable OBMC for small blocks (width * height >= 256), and the decoder is signaled. This increases the complexity of the encoder. At the same time, for a given block/sub-block, when OBMC is enabled, it is always applied to luminance and chrominance, which may lead to a decrease in coding efficiency.

6.用於基於更新的MV的運動預測的示例方法6. Example method for motion prediction based on updated MV

當前公開的技術的實施例克服了現有實現方式的缺點，從而提供具有更高編碼效率的視訊編碼。基於所公開的技術，使用更新的運動向量的運動預測可以增強現有和未來的視訊編碼標準，在以下針對各種實現方式所描述的示例中闡明。以下提供的所公開技術的示例解釋了一般概念，並不意味著被解釋為限制性的。在一個示例中，除非明確地相反指示，否則可以組合這些示例中描述的各種特徵。The embodiments of the currently disclosed technology overcome the shortcomings of existing implementations, thereby providing video coding with higher coding efficiency. Based on the disclosed technology, motion prediction using updated motion vectors can enhance existing and future video coding standards, as illustrated in the examples described below for various implementations. The examples of the disclosed technology provided below explain general concepts and are not meant to be interpreted as restrictive. In one example, unless explicitly indicated to the contrary, the various features described in these examples may be combined.

關於術語，來自列表0和列表1的當前圖像的參考圖像分別表示為Ref0和Ref1。取

=POC(當前)−POC(Ref0)，

= POC(Ref1)−POC(當前)，並分別由refblk0和refblk1表示來自Ref0和Ref1的當前塊的參考塊。對於當前塊中的子塊中， refblk0中其對應子塊的指向refblk1的MV由(v_x , v_y )表示。Ref0和Ref1中的子塊的MV分別由(mvL0_x , mvL0_y )和(mvL1_x , mvL1_y )表示。如本專利文件中所述，可以將用於運動預測的更新的基於運動向量的方法擴展到現有和未來的視訊編碼標準中。Regarding the terminology, the reference pictures of the current pictures from List 0 and List 1 are denoted as Ref0 and Ref1, respectively. take

=POC(current)−POC(Ref0),

= POC(Ref1)−POC(current), and refblk0 and refblk1 represent the reference blocks of the current block from Ref0 and Ref1, respectively. For the sub-block in the current block, the MV of the corresponding sub-block in refblk0 that points to refblk1 is represented by (v _x , v _y ). The MVs of the sub-blocks in Ref0 and Ref1 are represented by (mvL0 _x , mvL0 _y ) and (mvL1 _x , mvL1 _y ), respectively. As described in this patent document, the updated motion vector-based method for motion prediction can be extended to existing and future video coding standards.

示例1。提議修改BIO編碼塊的運動信息（例如，與運動補償中使用的不同），其可以在稍後使用，諸如在隨後運動預測（例如，TMVP）過程中。Example 1. It is proposed to modify the motion information of the BIO coding block (for example, different from that used in motion compensation), which can be used later, such as in a subsequent motion prediction (for example, TMVP) process.

（a）在一個示例中，提議縮放在BIO中導出的MV(v_x , v_y )並將其相加到當前塊/子塊的原始MV（mvLX_x , mvLX_y ）（X = 0或1）。更新的MV計算如下：mvL0’_x = -v_x * (

/(

+

)) + mvL0_x ，mvL0’_y = -v_y * (

/(

+

)) + mvL0_y ，以及mvL1’_x = v_x * (

/(

+

)) + mvL1_x ，mvL1’_y = v_y * (

/(

+

)) + mvL1_y 。（i）在一個示例中，更新的MV用於未來的運動預測（如在AMVP、Merge和仿射模式中）、去方塊（deblocking）、OBMC等。（ii）替代地，更新的MV僅可以用於按解碼順序對其非緊隨的CU/PU的運動預測。（iii）替代地，更新的MV僅可以在AMVP、Merge或仿射模式中用作TMVP。(A) In an example, it is proposed to scale the MV (v _x , v _y ) derived in BIO and add it to the original MV (mvLX _x , mvLX _y ) of the current block/sub-block (X = 0 or 1 ). The updated MV is calculated as follows: mvL0' _x = -v _x * (

/(

+

)) + mvL0 _x , mvL0' _y = -v _y * (

/(

+

)) + mvL0 _y , and mvL1' _x = v _x * (

/(

+

)) + mvL1 _x , mvL1' _y = v _y * (

/(

+

)) + mvL1 _y . (I) In one example, the updated MV is used for future motion prediction (such as in AMVP, Merge, and affine modes), deblocking, OBMC, etc. (Ii) Alternatively, the updated MV can only be used for motion prediction of its non-immediate CU/PU in decoding order. (Iii) Alternatively, the updated MV can only be used as TMVP in AMVP, Merge or Affine mode.

示例2。提議對於BIO、DMVR、FRUC、模板匹配或需要從位元流中導出的那些更新MV（或包括MV和/或參考圖像的運動信息）的其他方法，使用更新的運動信息可能受到約束。Example 2. It is proposed that for BIO, DMVR, FRUC, template matching, or other methods that need to be derived from the bitstream to update the MV (or include the motion information of the MV and/or the reference image), the use of the updated motion information may be restricted.

（a）如果可以在子塊級別更新運動信息，則可以儲存不同子塊的更新和未更新的運動信息兩者。在一個示例中，可以儲存一些子塊的更新的運動信息，並且對於其他剩餘的子塊，儲存未更新的運動信息。(A) If the motion information can be updated at the sub-block level, both updated and unupdated motion information of different sub-blocks can be stored. In one example, the updated motion information of some sub-blocks may be stored, and for other remaining sub-blocks, the unupdated motion information may be stored.

（b）在一個示例中，如果在子塊級別更新MV（或運動信息），則僅針對內部子塊（即，不在PU/CU/CTU邊界處的子塊）儲存更新的MV，並且然後將其用於運動預測、去方塊、OBMC等，如圖25所示。(B) In one example, if the MV (or motion information) is updated at the sub-block level, the updated MV is stored only for the inner sub-block (ie, the sub-block not at the PU/CU/CTU boundary), and then It is used for motion prediction, deblocking, OBMC, etc., as shown in Figure 25.

（c）在一個示例中，更新的MV或運動信息不用於運動預測和OBMC。替代地，另外，更新的MV或運動信息不用於去方塊。(C) In one example, the updated MV or motion information is not used for motion prediction and OBMC. Alternatively, in addition, the updated MV or motion information is not used for deblocking.

（d）在一個示例中，更新的MV或運動信息僅用於運動補償和時間運動預測，如TMVP/ATMVP。(D) In an example, the updated MV or motion information is only used for motion compensation and temporal motion prediction, such as TMVP/ATMVP.

示例3。提議取決於編碼模式、運動信息、PU/CU/塊的尺寸或位置隱式地啟用/禁用OBMC，並且因此不信令通知OBMC標誌。Example 3. It is proposed to implicitly enable/disable OBMC depending on the coding mode, motion information, size or location of PU/CU/block, and therefore not signal the OBMC flag.

（a）在一個示例中，如果滿足以下條件中的一個，則對於在AMVP模式或AFFINE_INTER模式中編碼的PU/CU/塊禁用OBMC（其中w和h是PU/CU/塊的寬度和高度）。（i）w×h >= T （ii）w >= T && h >= T(A) In an example, if one of the following conditions is met, OBMC is disabled for PU/CU/blocks encoded in AMVP mode or AFFINE_INTER mode (where w and h are the width and height of PU/CU/block) . (I) w×h >= T (Ii) w >= T && h >= T

（b）在一個示例中，對於在Merge模式和AFFINE_MERGE模式中編碼的PU/CU/塊總是啟用OBMC。(B) In one example, OBMC is always enabled for PU/CU/blocks encoded in Merge mode and AFFINE_MERGE mode.

（c）替代地，另外，垂直和水平OBMC分開被禁用/啟用。如果PU/CU/塊的高度小於T，則禁用垂直OBMC。如果PU/CU/塊的寬度小於T，則禁用水平OBMC。(C) Alternatively, in addition, vertical and horizontal OBMC are separately disabled/enabled. If the height of PU/CU/block is less than T, vertical OBMC is disabled. If the width of PU/CU/block is less than T, horizontal OBMC is disabled.

（d）在一個示例中，在對於頂部CTU邊界處的PU/CU/塊/子塊的OBMC中不使用來自上方行的相鄰MV。(D) In one example, the adjacent MV from the upper row is not used in the OBMC for the PU/CU/block/sub-block at the top CTU boundary.

（e）在一個示例中，在對於左側CTU邊界處的PU/CU/塊/子塊的OBMC中不使用來自左側列的相鄰MV。(E) In one example, the adjacent MV from the left column is not used in the OBMC for the PU/CU/block/sub-block at the left CTU boundary.

（f）在一個示例中，僅啟用對於單向預測的PU/CU/塊/子塊的OBMC。(F) In one example, only OBMC for uni-directionally predicted PU/CU/block/sub-block is enabled.

（g）在一個示例中，對於其MVD解析度大於或等於整數像素的PU/CU/塊禁用OBMC。(G) In one example, OBMC is disabled for PU/CU/blocks whose MVD resolution is greater than or equal to integer pixels.

示例4。提議OBMC是否被啟用可以取決於當前PU/CU/塊/子塊及其相鄰PU/CU/塊/子塊的運動信息。Example 4. It is proposed that whether OBMC is enabled or not may depend on the motion information of the current PU/CU/block/sub-block and its neighboring PU/CU/block/sub-block.

（a）在一個示例中，如果相鄰PU/CU/塊/子塊具有與當前PU/CU/塊/子塊相當不同（quite different）的運動信息，則其運動信息不在OBMC中使用。（i）在一個示例中，相鄰PU/CU/塊/子塊具有與當前PU/CU/塊/子塊不同的預測方向或參考圖像。（ii）在一個示例中，相鄰PU/CU/塊/子塊具有與當前PU/CU/塊/子塊的相同的預測方向和參考圖像，然而，在預測方向X（X = 0或1）上的相鄰PU/CU/塊/子塊和當前PU/CU/塊/子塊之間的絕對水平/垂直MV差異大於給定閾值MV_TH。(A) In one example, if an adjacent PU/CU/block/sub-block has motion information quite different from the current PU/CU/block/sub-block, its motion information is not used in OBMC. (I) In one example, the neighboring PU/CU/block/sub-block has a different prediction direction or reference image from the current PU/CU/block/sub-block. (Ii) In one example, the adjacent PU/CU/block/sub-block has the same prediction direction and reference image as the current PU/CU/block/sub-block, however, in the prediction direction X (X=0 or 1) The absolute horizontal/vertical MV difference between the adjacent PU/CU/block/sub-block and the current PU/CU/block/sub-block is greater than the given threshold MV_TH.

（b）替代地，如果相鄰PU/CU/塊/子塊具有與當前PU/CU/塊/子塊類似（similar）的運動信息，則其運動信息不在OBMC中使用。（i）在一個示例中，相鄰PU/CU/塊/子塊具有與當前PU/CU/塊/子塊相同的預測方向和參考圖像，並且在所有預測方向上的相鄰PU/CU/塊/子塊和當前PU/CU/塊/子塊之間的絕對水平/垂直MV差異小於給定閾值MV_TH。(B) Alternatively, if the adjacent PU/CU/block/sub-block has motion information similar to the current PU/CU/block/sub-block, its motion information is not used in OBMC. (I) In one example, the adjacent PU/CU/block/sub-block has the same prediction direction and reference image as the current PU/CU/block/sub-block, and the adjacent PU/CU in all prediction directions The absolute horizontal/vertical MV difference between /block/subblock and the current PU/CU/block/subblock is less than a given threshold MV_TH.

實施例5。提議在ATMVP/STMVP、仿射模式或PU/CU內的每個子塊（尺寸為N×M）具有單獨的運動信息的其他模式中，可以以與子塊尺寸不同的塊尺寸執行OBMC。Example 5. It is proposed that in ATMVP/STMVP, affine mode, or other modes in which each sub-block (size N×M) within the PU/CU has separate motion information, OBMC can be performed with a block size different from the sub-block size.

（a）在一個示例中，子塊尺寸為4×4，並且OBMC僅在8×8塊邊界處執行。(A) In one example, the sub-block size is 4×4, and OBMC is only executed at 8×8 block boundaries.

示例6。提議在OBMC中處理多少行/列可以取決於PU/CU/塊/子塊尺寸。Example 6. It is proposed how many rows/columns to be processed in OBMC may depend on the PU/CU/block/sub-block size.

（a）在一個示例中，如果PU/CU/塊/子塊的寬度大於N，則處理PU/CU/塊/子塊的4個左側列；否則，僅處理PU/CU/塊/子塊的2（或1）個左側列。(A) In an example, if the width of PU/CU/block/sub-block is greater than N, then process the 4 left columns of PU/CU/block/sub-block; otherwise, only process PU/CU/block/sub-block The 2 (or 1) left column.

（b）在一個示例中，如果PU/CU/塊/子塊的高度大於N，則處理PU/CU/塊/子塊的4個上方行；否則，僅處理PU/CU/塊/子塊的2（或1）個上方行。(B) In an example, if the height of PU/CU/block/sub-block is greater than N, then process the 4 upper rows of PU/CU/block/sub-block; otherwise, only process PU/CU/block/sub-block 2 (or 1) of the upper row.

示例7。提議獨立地啟用/禁用用於亮度和色度分量的OBMC，並且示例2和3中描述的規則可以單獨地應用於每個分量。Example 7. It is proposed to independently enable/disable OBMC for luma and chroma components, and the rules described in Examples 2 and 3 can be applied to each component individually.

示例8。提議在使用相鄰運動信息生成預測塊時使用短階內插濾波器（如雙線性、4階或6階濾波器）。Example 8. It is proposed to use short-order interpolation filters (such as bilinear, 4th or 6th order filters) when generating prediction blocks using adjacent motion information.

（a）在一個示例中，不對稱6階濾波器用於亮度分量。對於子像素位置，左側/上側的4個像素和右側/下側的2個像素用於內插。(A) In one example, an asymmetric 6th order filter is used for the luminance component. For the sub-pixel position, 4 pixels on the left/upper side and 2 pixels on the right/lower side are used for interpolation.

示例9。所提議的方法可以應用於某些模式、塊尺寸/形狀和/或某些子塊尺寸。Example 9. The proposed method can be applied to certain modes, block sizes/shapes and/or certain sub-block sizes.

（a）所提議的方法可以應用於某些模式，諸如傳統的平移運動（即，禁用仿射模式）。(A) The proposed method can be applied to certain modes, such as traditional translational motion (ie, affine mode is disabled).

（b）所提議的方法可以應用於某些塊尺寸。（i）在一個示例中，它僅應用於具有w×h≥T的塊，其中w和h是當前塊的寬度和高度。（ii）在另一個示例中，它僅應用於w≥T&&h≥T的塊。(B) The proposed method can be applied to certain block sizes. (I) In an example, it only applies to blocks with w×h≥T, where w and h are the width and height of the current block. (Ii) In another example, it only applies to blocks with w≥T&&h≥T.

實施例10。所提議的方法可以應用於所有色度分量。替代地，它們可以僅應用於某些色度分量。例如，它們可以僅應用於亮度分量。Example 10. The proposed method can be applied to all chrominance components. Alternatively, they can only be applied to certain chrominance components. For example, they can be applied only to the luminance component.

以上描述的示例可以併入在下面描述的方法的上下文中，例如，方法2600和方法2700，其可以在視訊解碼器處實現。The examples described above can be incorporated in the context of the methods described below, for example, method 2600 and method 2700, which can be implemented at a video decoder.

圖26示出了用於視訊解碼的示例性方法的流程圖。方法2600包括，在步驟2610，接收視訊資料的當前塊的位元流表示。Figure 26 shows a flowchart of an exemplary method for video decoding. The method 2600 includes, in step 2610, receiving a bitstream representation of the current block of video data.

方法2600包括，在步驟2620，基於第一運動向量和第一參考運動向量和第二參考運動向量的加權和來分別生成更新的第一參考運動向量和更新的第二參考運動向量。在一些實施例中，基於來自第一參考塊的第一參考運動向量和來自第二參考塊的第二參考運動向量來導出第一運動向量，並且當前塊與第一參考塊和第二參考塊相關聯。The method 2600 includes, in step 2620, generating an updated first reference motion vector and an updated second reference motion vector based on the first motion vector and the weighted sum of the first reference motion vector and the second reference motion vector, respectively. In some embodiments, the first motion vector is derived based on the first reference motion vector from the first reference block and the second reference motion vector from the second reference block, and the current block differs from the first reference block and the second reference block Associated.

方法2600包括，在步驟2630，基於更新的第一參考運動向量和更新的第二參考運動向量來處理位元流表示從而生成當前塊。The method 2600 includes, at step 2630, processing the bitstream representation based on the updated first reference motion vector and the updated second reference motion vector to generate the current block.

在一些實施例中，並且如在示例1的上下文中所描述的，基於使用第一參考運動向量和第二參考運動向量的雙向光流（BIO）細化來導出第一運動向量。在示例中，加權和包括基於當前塊、第一參考塊和第二參考塊的圖像順序計數（POC）的權重。In some embodiments, and as described in the context of Example 1, the first motion vector is derived based on bidirectional optical flow (BIO) refinement using the first reference motion vector and the second reference motion vector. In an example, the weighted sum includes a weight based on the picture order count (POC) of the current block, the first reference block, and the second reference block.

在一些實施例中，該處理可以基於雙向光流（BIO）細化、解碼器側運動向量細化（DMVR）、畫面播放速率上轉換（FRUC）技術或模板匹配技術。在一個示例中，對不在當前塊的邊界上的內部子塊生成更新的第一參考運動向量和更新的第二參考運動向量。在另一示例中，對當前塊的子塊的子集生成更新的第一參考運動向量和更新的第二參考運動向量。In some embodiments, the processing may be based on bidirectional optical flow (BIO) refinement, decoder-side motion vector refinement (DMVR), frame rate up conversion (FRUC) technology, or template matching technology. In one example, the updated first reference motion vector and the updated second reference motion vector are generated for internal sub-blocks not on the boundary of the current block. In another example, the updated first reference motion vector and the updated second reference motion vector are generated for a subset of the sub-blocks of the current block.

在一些實施例中，處理不包括運動預測或重疊塊運動補償（OBMC）。In some embodiments, the processing does not include motion prediction or overlapping block motion compensation (OBMC).

圖27示出了用於視訊解碼的另一個示例性方法的流程圖。方法2700包括，在步驟2710，接收視訊資料的當前塊的位元流表示。FIG. 27 shows a flowchart of another exemplary method for video decoding. Method 2700 includes, in step 2710, receiving a bitstream representation of the current block of video data.

方法2700包括，在步驟2720，在不信令通知OBMC標誌的情況下，基於當前塊的特性通過選擇性地使用重疊塊運動補償（OBMC）來處理該位元流表示從而生成當前塊。The method 2700 includes, in step 2720, without signaling the OBMC flag, based on the characteristics of the current block by selectively using overlapping block motion compensation (OBMC) to process the bitstream representation to generate the current block.

在一些實施例中，特性包括圖像中當前塊的維度或當前塊的位置。在其他實施例中，特性包括當前塊的運動信息。在一個示例中，如果當前塊的運動信息與相鄰塊的運動信息不同，則可以不使用OBMC。在另一示例中，如果當前塊的運動信息與相鄰塊的運動信息相同，則可以使用OBMC。In some embodiments, the characteristics include the dimensions of the current block or the position of the current block in the image. In other embodiments, the characteristics include motion information of the current block. In one example, if the motion information of the current block is different from the motion information of neighboring blocks, OBMC may not be used. In another example, if the motion information of the current block is the same as the motion information of the neighboring block, OBMC can be used.

在一些實施例中，並且如在示例7的上下文中所描述的，OBMC可以獨立地應用於亮度和色度分量。在一個示例中，OBMC被應用於當前塊的色度分量，並且其中OBMC不被應用於當前塊的亮度分量。在另一個示例中，OBMC被應用於當前塊的亮度分量，並且其中OBMC不被應用於當前塊的色度分量。In some embodiments, and as described in the context of Example 7, OBMC can be independently applied to the luminance and chrominance components. In one example, OBMC is applied to the chrominance component of the current block, and wherein OBMC is not applied to the luma component of the current block. In another example, OBMC is applied to the luminance component of the current block, and wherein OBMC is not applied to the chrominance component of the current block.

在一些實施例中，並且如在示例6的上下文中所描述的，處理位元流表示包括使用OBMC處理當前塊的預定數目的行或列，並且其中預定數目基於當前塊的子塊的尺寸。In some embodiments, and as described in the context of Example 6, the processing bitstream representation includes processing a predetermined number of rows or columns of the current block using OBMC, and wherein the predetermined number is based on the size of the sub-block of the current block.

7．所公開的技術的示例實現方式7. Example implementation of the disclosed technology

圖28是視訊處理裝置2800的框圖。裝置2800可以用於實現本文描述的一種或多種方法。裝置2800可以實施在智慧手機、平板電腦、電腦、物聯網（IoT）接收器等中。裝置2800可以包括一個或多個處理器2802、一個或多個記憶體2804和視訊處理電路2806。（一個或多個）處理器2802可以被配置為實現本文件中描述的一種或多種方法（包括但不限於方法2600和2700）。（一個或多個）記憶體2804可以用於儲存用於實現本文描述的方法和技術的資料和代碼。視訊處理電路2806可以用於在硬體電路中實現本文件中描述的一些技術。FIG. 28 is a block diagram of the video processing device 2800. The apparatus 2800 can be used to implement one or more methods described herein. The device 2800 may be implemented in a smart phone, a tablet computer, a computer, an Internet of Things (IoT) receiver, etc. The device 2800 may include one or more processors 2802, one or more memories 2804, and a video processing circuit 2806. The processor(s) 2802 may be configured to implement one or more methods described in this document (including but not limited to methods 2600 and 2700). The memory(s) 2804 can be used to store data and codes used to implement the methods and techniques described herein. The video processing circuit 2806 can be used to implement some of the techniques described in this document in a hardware circuit.

在一些實施例中，視訊編碼方法可以使用在如關於圖28所描述的硬體平臺上實現的裝置來實現。In some embodiments, the video encoding method may be implemented using a device implemented on a hardware platform as described with respect to FIG. 28.

圖29示出了用於視訊處理的另一個示例性方法的流程圖。方法2900包括，在步驟2910，確定具有運動向量的當前塊。FIG. 29 shows a flowchart of another exemplary method for video processing. The method 2900 includes, in step 2910, determining the current block having a motion vector.

方法2900包括，在步驟2920，基於特定預測模式生成更新的運動向量。The method 2900 includes, in step 2920, generating an updated motion vector based on a specific prediction mode.

方法2900包括，在步驟2930，基於所述更新的運動向量，執行所述當前塊與包括所述當前塊的視訊資料的位元流表示之間的轉換。The method 2900 includes, in step 2930, performing a conversion between the current block and a bitstream representation of video data including the current block based on the updated motion vector.

圖30示出了用於處理視訊資料的另一個示例性方法的流程圖。方法3000包括，在步驟3010，確定當前塊和對應的相鄰塊。FIG. 30 shows a flowchart of another exemplary method for processing video data. The method 3000 includes, at step 3010, determining the current block and the corresponding neighboring block.

方法3000包括，在步驟3020，構造所述當前塊的最終預測塊作為從所述當前塊的運動向量導出的預測塊和從一個或多個相鄰塊的運動向量導出的預測塊的加權和。例如，短階內插濾波器可以用於構造所述當前塊的最終預測塊。The method 3000 includes, in step 3020, constructing the final prediction block of the current block as a weighted sum of the prediction block derived from the motion vector of the current block and the prediction block derived from the motion vector of one or more neighboring blocks. For example, a short-order interpolation filter can be used to construct the final prediction block of the current block.

方法3000包括，在步驟3030，基於所述當前塊的最終預測塊，執行所述當前塊與包括所述當前塊的視訊資料的位元流表示之間的轉換。The method 3000 includes, in step 3030, performing a conversion between the current block and a bitstream representation of video data including the current block based on the final prediction block of the current block.

可以在以下示例列表中描述本文件中公開的各種實施例和技術。The various embodiments and techniques disclosed in this document can be described in the following list of examples.

1、一種視訊處理方法，包括：確定具有運動向量的當前塊；基於特定預測模式生成更新的運動向量；以及基於所述更新的運動向量，執行所述當前塊與包括所述當前塊的視訊資料的位元流表示之間的轉換，其中所述特定預測模式包括以下中的一個或多個：雙向光流（BIO）細化、解碼器側運動向量細化（DMVR）、畫面播放速率上轉換（FRUC）技術或模板匹配技術。1. A video processing method, including: Determine the current block with the motion vector; Generate updated motion vectors based on a specific prediction mode; and Based on the updated motion vector, perform conversion between the current block and the bit stream representation of the video data including the current block, wherein the specific prediction mode includes one or more of the following: bidirectional optical flow (BIO) refinement, decoder side motion vector refinement (DMVR), frame rate up conversion (FRUC) technology or template matching technology.

2、如示例1所述的方法，其中對於所述當前塊的子塊的子集生成所述更新的運動向量。2. The method of example 1, wherein the updated motion vector is generated for a subset of sub-blocks of the current block.

3、如示例2所述的方法，其中對於所述當前塊的子塊的子集儲存未更新的運動向量和所述更新的運動向量，並且對於所述當前塊的剩餘的子塊僅儲存所述未更新的運動向量。3. The method according to example 2, wherein the unupdated motion vector and the updated motion vector are stored for a subset of the sub-blocks of the current block, and only the remaining sub-blocks of the current block are stored. State the motion vector that has not been updated.

4、如示例2所述的方法，其中對於所述當前塊的子塊的子集儲存所述更新的運動向量，並且對於所述當前塊的剩餘的子塊儲存未更新的運動向量。4. The method of example 2, wherein the updated motion vector is stored for a subset of sub-blocks of the current block, and an unupdated motion vector is stored for the remaining sub-blocks of the current block.

5、如示例2所述的方法，其中對於不在PU、CU或CTU邊界處的內部子塊僅儲存所述更新的運動向量。5. The method according to example 2, wherein only the updated motion vector is stored for internal sub-blocks not at the boundary of PU, CU or CTU.

6、如示例5所述的方法，其中所述更新的運動向量用於運動預測、去方塊或重疊塊的運動補償（OBMC）。6. The method of example 5, wherein the updated motion vector is used for motion prediction, deblocking or motion compensation (OBMC) of overlapping blocks.

7、如示例1所述的方法，其中所述更新的運動向量不用於運動預測和重疊塊的運動補償（OBMC）。7. The method of example 1, wherein the updated motion vector is not used for motion prediction and motion compensation of overlapping blocks (OBMC).

8、如示例1所述的方法，其中所述更新的運動向量不用於去方塊。8. The method according to example 1, wherein the updated motion vector is not used for deblocking.

9、如示例1所述的方法，其中所述更新的運動向量用於運動補償和/或時間運動預測。9. The method according to example 1, wherein the updated motion vector is used for motion compensation and/or temporal motion prediction.

10、如示例9所述的方法，其中所述時間運動預測包括時間運動向量預測（TMVP）和/或替代時間運動向量預測（ATMVP）。10. The method of example 9, wherein the temporal motion prediction includes temporal motion vector prediction (TMVP) and/or alternative temporal motion vector prediction (ATMVP).

11、一種處理視訊資料的方法，包括：確定當前塊和對應的相鄰塊；和構造所述當前塊的最終預測塊作為從所述當前塊的運動向量導出的預測塊和從一個或多個相鄰塊的運動向量導出的預測塊的加權和；以及基於所述當前塊的最終預測塊，執行所述當前塊與包括所述當前塊的視訊資料的位元流表示之間的轉換；其中短階內插濾波器用於構造所述當前塊的最終預測塊。11. A method for processing video data, including: Determine the current block and the corresponding neighboring block; and Constructing the final prediction block of the current block as a weighted sum of the prediction block derived from the motion vector of the current block and the prediction block derived from the motion vector of one or more neighboring blocks; and Based on the final prediction block of the current block, performing a conversion between the current block and the bitstream representation of the video data including the current block; The short-order interpolation filter is used to construct the final prediction block of the current block.

12、如示例11所述的方法，其中所述短階內插濾波器是雙線、4階或6階濾波器。12. The method of example 11, wherein the short-order interpolation filter is a two-wire, 4th or 6th order filter.

13、如示例12所述的方法，其中所述短階內插濾波器是不對稱6階濾波器，並且所述濾波器用於亮度分量，並且其中對於子像素位置，左側/上側的4個像素和右側/下側的2個像素用於內插。13. The method according to example 12, wherein the short-order interpolation filter is an asymmetric 6-order filter, and the filter is used for the luminance component, and wherein for the sub-pixel position, 4 pixels on the left/upper side And the 2 pixels on the right/lower side are used for interpolation.

14、如示例1至13中任一項所述的方法，其中所述方法應用於平移運動，並且禁用仿射模式。14. The method according to any one of examples 1 to 13, wherein the method is applied to translational motion, and the affine mode is disabled.

15、如示例1至13中任一項所述的方法，其中所述方法僅應用於w×h≥T的塊，其中w和h是所述當前塊的寬度和高度，並且T是閾值。15. The method according to any one of examples 1 to 13, wherein the method is only applied to a block of w×h≥T, where w and h are the width and height of the current block, and T is a threshold.

16、如示例1至13中任一項所述的方法，其中所述方法僅應用於w≥T且h≥T的塊，其中w和h是所述當前塊的寬度和高度，並且T是閾值。16. The method according to any one of examples 1 to 13, wherein the method is only applied to blocks with w≥T and h≥T, where w and h are the width and height of the current block, and T is Threshold.

17、如示例1至16中任一項所述的方法，其中所述方法應用於所有色度分量。17. The method according to any one of examples 1 to 16, wherein the method is applied to all chrominance components.

18、如示例1至16中任一項所述的方法，其中所述方法僅應用於亮度分量。18. The method according to any one of examples 1 to 16, wherein the method is only applied to the luminance component.

19、一種視訊系統中的裝置，包括處理器和其上具有指令的非暫時性記憶體，其中所述指令在由所述處理器執行時使所述處理器實現示例1至18中任一項所述的方法。19. A device in a video system, comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions when executed by the processor enable the processor to implement any one of Examples 1 to 18 The method described.

20、一種儲存在非暫時性電腦可讀介質上的電腦程式產品，所述電腦程式產品包括用於實施如示例1至18中任一項所述的方法的程式碼。20. A computer program product stored on a non-transitory computer readable medium, the computer program product including program code for implementing the method described in any one of Examples 1 to 18.

從前述內容可以理解，本文已經出於說明的目的描述了本公開技術的具體實施例，但是在不脫離本發明的範圍的情況下可以進行各種修改。因此，除了所附申請專利範圍之外，本發明所公開的技術不受限制。It can be understood from the foregoing that specific embodiments of the disclosed technology have been described herein for illustrative purposes, but various modifications can be made without departing from the scope of the present invention. Therefore, the technology disclosed in the present invention is not limited except for the scope of the attached application.

本專利文件描述的主題和功能操作的實現方式可以以各種系統實現，以數位電子電路實現，或者以電腦軟體、韌體或硬體實現，包括本說明書中公開的結構及其結構等同物，或者以它們中的一個或多個的組合實現。本說明書所描述的主題的實現方式可以實現為一個或多個電腦程式產品，即，在有形和非暫時性電腦可讀介質上編碼的一個或多個電腦程式指令模組，用於由資料處理裝置執行或控制資料處理裝置的操作。電腦可讀介質可以是機器可讀存放裝置、機器可讀儲存基板、記憶體設備、影響機器可讀傳播信號的物質組合、或者它們中的一個或多個的組合。術語「資料處理單元」或「資料處理裝置」涵蓋用於處理資料的所有裝置、設備和機器，包括例如可程式設計處理器、電腦或多個處理器或電腦。除了硬體之外，該裝置還可以包括為所討論的電腦程式創建執行環境的代碼，例如，構成處理器韌體、協定疊、資料庫管理系統、作業系統、或者它們中的一個或多個的組合的代碼。The subject and function operation described in this patent document can be implemented in various systems, implemented in digital electronic circuits, or implemented in computer software, firmware, or hardware, including the structure disclosed in this specification and its structural equivalents, or Realize in a combination of one or more of them. The implementation of the subject described in this specification can be implemented as one or more computer program products, that is, one or more computer program instruction modules encoded on tangible and non-transitory computer-readable media for data processing The device executes or controls the operation of the data processing device. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of substances that affect a machine-readable propagation signal, or a combination of one or more of them. The term "data processing unit" or "data processing device" covers all devices, equipment, and machines used to process data, including, for example, a programmable processor, computer, or multiple processors or computers. In addition to hardware, the device may also include code that creates an execution environment for the computer program in question, for example, constituting processor firmware, protocol stack, database management system, operating system, or one or more of them The code of the combination.

電腦程式（也稱為程式、軟體、軟體應用、腳本或代碼）可以以任何形式的程式設計語言編寫，包括編譯或解釋語言，並且可以以任何形式來部署電腦程式，包括作為獨立程式或作為適合在計算環境中使用的模組、元件、子常式或其他單元。電腦程式不一定對應於檔案系統中的檔。程式可以儲存在保存其他程式或資料的檔的一部分中（例如，儲存在標記語言檔中的一個或多個腳本），儲存在專用於所討論的程式的單個檔中，或儲存在多個協調檔中（例如，儲存一個或多個模組、副程式或代碼部分的檔）。可以部署電腦程式以在一個電腦上或在位於一個網站上或分佈在多個網站上並由通信網路互連的多個電腦上執行。Computer programs (also called programs, software, software applications, scripts or codes) can be written in any form of programming language, including compiled or interpreted languages, and computer programs can be deployed in any form, including as stand-alone programs or as suitable Modules, components, subroutines, or other units used in the computing environment. Computer programs do not necessarily correspond to files in the file system. Programs can be stored in part of a file that holds other programs or data (for example, one or more scripts stored in a markup language file), in a single file dedicated to the program in question, or in multiple coordination In a file (for example, a file that stores one or more modules, subprograms, or code parts). A computer program can be deployed to be executed on one computer or on multiple computers located on one website or distributed on multiple websites and interconnected by a communication network.

本說明書中描述的過程和邏輯流程可以由執行一個或多個電腦程式的一個或多個可程式設計處理器執行，以通過對輸入資料進行操作並生成輸出來執行功能。過程和邏輯流程也可以由專用邏輯電路執行，並且裝置也可以實現為專用邏輯電路，例如FPGA（現場可程式設計閘陣列）或ASIC（專用積體電路）。The processes and logic flows described in this specification can be executed by one or more programmable processors that execute one or more computer programs to perform functions by operating on input data and generating output. The process and logic flow can also be executed by a dedicated logic circuit, and the device can also be implemented as a dedicated logic circuit, such as FPGA (Field Programmable Gate Array) or ASIC (Dedicated Integrated Circuit).

舉例來說，適合於執行電腦程式的處理器包括通用和專用微處理器、以及任何種類的數位電腦的任何一個或多個處理器。通常，處理器將從唯讀記憶體或隨機存取記憶體或兩者接收指令和資料。電腦的基本元件是用於執行指令的處理器和用於儲存指令和資料的一個或多個記憶體設備。通常，電腦還將包括或可操作地耦合到用於儲存資料的一個或多個大型存放區設備，例如磁片、磁光碟或光碟，以從該一個或多個大型存放區設備接收資料，或將資料傳遞到該一個或多個大型存放區設備，或者既接收又傳遞資料。然而，電腦不需要具有這樣的設備。適用於儲存電腦程式指令和資料的電腦可讀介質包括所有形式的非易失性記憶體、介質和記憶體設備，舉例來說，包括半導體記憶體設備，例如EPROM、EEPROM和快閃記憶體設備。處理器和記憶體可以由專用邏輯電路補充或併入專用邏輯電路中。For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, the processor will receive commands and data from read-only memory or random access memory or both. The basic components of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, the computer will also include or be operatively coupled to one or more large storage area devices for storing data, such as floppy disks, magneto-optical discs or optical discs, to receive data from the one or more large storage area devices, or Deliver data to the one or more large storage area devices, or both receive and deliver data. However, the computer does not need to have such equipment. Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM and flash memory devices . The processor and memory can be supplemented by or incorporated into a dedicated logic circuit.

旨在將說明書與附圖一起僅視為示例性的，其中示例性意味著示例。如本文所使用的，單數形式「一」、「一個」和「該」旨在也包括複數形式，除非上下文另有明確說明。另外，除非上下文另有明確說明，否則「或」的使用旨在包括「和/或」。It is intended to treat the description together with the drawings as merely exemplary, where exemplary means example. As used herein, the singular forms "a", "an" and "the" are intended to also include the plural form, unless the context clearly dictates otherwise. In addition, the use of "or" is intended to include "and/or" unless the context clearly dictates otherwise.

雖然本專利文件包含許多細節，但這些細節不應被解釋為對任何發明或可要求保護的範圍的限制，而是作為特定於特定發明的特定實施例的特徵的描述。在本專利文件中，在分開的實施例的上下文中描述的某些特徵也可以在單個實施例中組合實現。相反，在單個實施例的上下文中描述的各種特徵也可以分開地或以任何合適的子組合在多個實施例中實現。此外，儘管上面的特徵可以描述為以某些組合起作用並且甚至最初如此要求保護，但是在一些情況下，可以從所要求保護的組合中去除來自該組合的一個或多個特徵，並且所要求保護的組合可以指向子組合或子組合的變型。Although this patent document contains many details, these details should not be construed as limitations on the scope of any invention or claimable, but as descriptions of features specific to specific embodiments of specific inventions. In this patent document, certain features described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. In addition, although the above features may be described as functioning in certain combinations and even initially claimed as such, in some cases, one or more features from the combination may be removed from the claimed combination, and the claimed The combination of protection can be directed to a sub-combination or a variant of the sub-combination.

類似地，雖然在附圖中以特定順序描繪了操作，但是這不應該被理解為要求以所示的特定順序或按循序執行這樣的操作，或者執行所有示出的操作，以實現期望的結果。此外，在本專利文件中描述的實施例中的各種系統元件的分離不應被理解為在所有實施例中都要求這樣的分離。Similarly, although operations are depicted in a specific order in the drawings, this should not be understood as requiring that such operations be performed in the specific order shown or in a sequential order, or that all the operations shown are performed to achieve the desired result . In addition, the separation of various system elements in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

僅描述了幾個實現方式和示例，並且可以基於本專利文件中描述和示出的內容來做出其他實現方式、增強和變型。Only a few implementations and examples are described, and other implementations, enhancements and modifications can be made based on the content described and shown in this patent document.

2800‧‧‧裝置 2802‧‧‧處理器 2804‧‧‧記憶體 2806‧‧‧視訊處理電路 2600、2700、2900、3000‧‧‧方法 2610～2630、2710～2720、2910～2930、3010～3030‧‧‧步驟2800‧‧‧device 2802‧‧‧Processor 2804‧‧‧Memory 2806‧‧‧Video Processing Circuit 2600, 2700, 2900, 3000‧‧‧Method 2610～2630, 2710～2720, 2910～2930, 3010～3030‧‧‧Step

圖1示出了構建Merge候選列表的示例。圖2示出了空間候選的位置的示例。圖3示出了經受空間Merge候選的冗餘校驗的候選對的示例。圖4A和圖4B示出了基於當前塊的尺寸和形狀的第二預測單元PU的位置的示例。圖5示出了用於時間Merge候選的運動向量縮放的示例。圖6示出了用於時間Merge候選的候選位置的示例。圖7示出了生成組合的雙向預測Merge候選的示例。圖8示出了構建運動向量預測候選的示例。圖9示出了用於空間運動向量候選的運動向量縮放的示例。圖10示出了使用用於編碼單元（CU）的替代時間運動向量預測（ATMVP）演算法的運動預測的示例。圖11示出了具有由空時運動向量預測（STMVP）演算法使用的子塊和相鄰塊的編碼單元（CU）的示例。圖12A和圖12B示出了當使用重疊塊的運動補償（OBMC）演算法時子塊的示例快照。圖13示出了用於導出局部亮度補償（LIC）演算法的參數的相鄰樣點的示例。圖14示出了簡化的仿射運動模型的示例。圖15示出了每個子塊的仿射運動向量場（MVF）的示例。圖16示出了用於AF_INTER仿射運動模式的運動向量預測（MVP）的示例。圖17A和17B示出了用於AF_MERGE仿射運動模式的示例候選。圖18示出了在模式匹配運動向量推導（PMMVD）模式中的雙邊匹配的示例，模式匹配運動向量推導（PMMVD）模式是基於畫面播放速率上轉換（FRUC）演算法的特殊Merge模式。圖19示出了在FRUC演算法中的模板匹配的示例。圖20示出了在FRUC演算法中的單向運動估計的示例。圖21示出了由雙向光流（BIO）演算法使用的光流軌跡的示例。圖22A和22B示出了使用不具有塊擴展的雙向光流（BIO）演算法的示例快照。圖23示出了基於雙邊模板匹配的解碼器側運動向量細化（DMVR）演算法的示例。圖24示出了在變換係數上下文建模中使用的模板定義的示例。圖25示出了在PU/CU中的內部和邊界子塊的示例。圖26示出了根據當前所公開的技術的用於視訊編碼的示例方法的流程圖。圖27示出了根據當前所公開的技術的用於視訊編碼的另一個示例方法的流程圖。圖28是用於實現本文件中描述的視覺媒體解碼或視覺媒體編碼技術的硬體平臺的示例的框圖。圖29示出了根據當前所公開的技術的用於視訊處理的再一個示例方法的流程圖。圖30示出了根據當前所公開的技術的用於處理視訊資料的又一個示例方法的流程圖。Figure 1 shows an example of constructing a Merge candidate list. Fig. 2 shows an example of the position of the spatial candidate. Fig. 3 shows an example of a candidate pair subjected to redundancy check of spatial Merge candidates. 4A and 4B show examples of the position of the second prediction unit PU based on the size and shape of the current block. Fig. 5 shows an example of motion vector scaling for temporal Merge candidates. Fig. 6 shows an example of candidate positions for temporal Merge candidates. FIG. 7 shows an example of generating combined bidirectional prediction Merge candidates. Fig. 8 shows an example of constructing motion vector prediction candidates. Fig. 9 shows an example of motion vector scaling for spatial motion vector candidates. FIG. 10 shows an example of motion prediction using an alternative temporal motion vector prediction (ATMVP) algorithm for coding units (CU). FIG. 11 shows an example of a coding unit (CU) with sub-blocks and neighboring blocks used by a space-time motion vector prediction (STMVP) algorithm. Figures 12A and 12B show example snapshots of sub-blocks when using an overlapping block motion compensation (OBMC) algorithm. Figure 13 shows an example of neighboring samples used to derive the parameters of the Local Luminance Compensation (LIC) algorithm. Fig. 14 shows an example of a simplified affine motion model. FIG. 15 shows an example of an affine motion vector field (MVF) of each sub-block. FIG. 16 shows an example of motion vector prediction (MVP) for AF_INTER affine motion mode. 17A and 17B show example candidates for the AF_MERGE affine motion mode. FIG. 18 shows an example of bilateral matching in the pattern matching motion vector derivation (PMMVD) mode. The pattern matching motion vector derivation (PMMVD) mode is a special Merge mode based on the frame rate up conversion (FRUC) algorithm. Figure 19 shows an example of template matching in the FRUC algorithm. Figure 20 shows an example of one-way motion estimation in the FRUC algorithm. Figure 21 shows an example of the optical flow trajectory used by the Bidirectional Optical Flow (BIO) algorithm. Figures 22A and 22B show example snapshots using a bidirectional optical flow (BIO) algorithm without block expansion. Figure 23 shows an example of a decoder-side motion vector refinement (DMVR) algorithm based on bilateral template matching. Fig. 24 shows an example of template definition used in transform coefficient context modeling. Fig. 25 shows an example of inner and boundary sub-blocks in PU/CU. FIG. 26 shows a flowchart of an example method for video encoding according to the currently disclosed technology. FIG. 27 shows a flowchart of another example method for video encoding according to the currently disclosed technology. FIG. 28 is a block diagram of an example of a hardware platform used to implement the visual media decoding or visual media encoding technology described in this document. FIG. 29 shows a flowchart of still another example method for video processing according to the currently disclosed technology. FIG. 30 shows a flowchart of yet another example method for processing video data according to the currently disclosed technology.

2900‧‧‧方法 2900‧‧‧Method

2910~2930‧‧‧步驟 2910~2930‧‧‧Step

Claims

一種視訊處理方法，包括：確定具有運動向量的當前塊；基於特定預測模式生成更新的運動向量；以及基於所述更新的運動向量，執行所述當前塊與包括所述當前塊的視訊資料的位元流表示之間的轉換，其中所述特定預測模式包括以下中的一個或多個：雙向光流(BIO)細化、解碼器側運動向量細化(DMVR)、畫面播放速率上轉換(FRUC)技術或模板匹配技術，其中對於所述當前塊的子塊的子集生成所述更新的運動向量。 A video processing method includes: determining a current block having a motion vector; generating an updated motion vector based on a specific prediction mode; and executing the position of the current block and video data including the current block based on the updated motion vector Conversion between meta-stream representations, where the specific prediction mode includes one or more of the following: bidirectional optical flow (BIO) refinement, decoder-side motion vector refinement (DMVR), frame rate up conversion (FRUC) ) Technology or template matching technology, wherein the updated motion vector is generated for a subset of the sub-blocks of the current block.

如申請專利範圍第1項所述的方法，其中對於所述當前塊的子塊的子集儲存未更新的運動向量和所述更新的運動向量，並且對於所述當前塊的剩餘的子塊僅儲存所述未更新的運動向量。 The method according to item 1 of the scope of patent application, wherein the unupdated motion vector and the updated motion vector are stored for the subset of the sub-blocks of the current block, and only the remaining sub-blocks of the current block are Store the unupdated motion vector.

如申請專利範圍第1項所述的方法，其中對於所述當前塊的子塊的子集儲存所述更新的運動向量，並且對於所述當前塊的剩餘的子塊儲存未更新的運動向量。 The method according to claim 1, wherein the updated motion vector is stored for a subset of the sub-blocks of the current block, and an unupdated motion vector is stored for the remaining sub-blocks of the current block.

如申請專利範圍第1項所述的方法，其中對於不在PU、CU或CTU邊界處的內部子塊僅儲存所述更新的運動向量。 The method according to the first item of the scope of patent application, wherein only the updated motion vector is stored for internal sub-blocks not located at the boundary of PU, CU, or CTU.

如申請專利範圍第4項所述的方法，其中所述更新的運動向量用於運動預測、去方塊或重疊塊的運動補償(OBMC)。 The method according to claim 4, wherein the updated motion vector is used for motion prediction, deblocking or overlapping block motion compensation (OBMC).

一種視訊處理方法，包括：確定具有運動向量的當前塊；基於特定預測模式生成更新的運動向量；以及基於所述更新的運動向量，執行所述當前塊與包括所述當前塊的視訊資料的位元流表示之間的轉換，其中所述特定預測模式包括以下中的一個或多個：雙向光流(BIO)細化、解碼器側運動向量細化(DMVR)、畫面播放速率上轉換(FRUC)技術或模板匹配技術，其中所述更新的運動向量不用於運動預測和重疊塊的運動補償(OBMC)。 A video processing method, including: Determine a current block with a motion vector; generate an updated motion vector based on a specific prediction mode; and based on the updated motion vector, perform conversion between the current block and the bitstream representation of the video data including the current block , Wherein the specific prediction mode includes one or more of the following: bidirectional optical flow (BIO) refinement, decoder side motion vector refinement (DMVR), frame rate up conversion (FRUC) technology or template matching technology, The updated motion vector is not used for motion prediction and overlapping block motion compensation (OBMC).

一種視訊處理方法，包括：確定具有運動向量的當前塊；基於特定預測模式生成更新的運動向量；以及基於所述更新的運動向量，執行所述當前塊與包括所述當前塊的視訊資料的位元流表示之間的轉換，其中所述特定預測模式包括以下中的一個或多個：雙向光流(BIO)細化、解碼器側運動向量細化(DMVR)、畫面播放速率上轉換(FRUC)技術或模板匹配技術，其中所述更新的運動向量不用於去方塊。 A video processing method includes: determining a current block having a motion vector; generating an updated motion vector based on a specific prediction mode; and executing the position of the current block and video data including the current block based on the updated motion vector Conversion between meta-stream representations, where the specific prediction mode includes one or more of the following: bidirectional optical flow (BIO) refinement, decoder-side motion vector refinement (DMVR), frame rate up conversion (FRUC) ) Technology or template matching technology, wherein the updated motion vector is not used for deblocking.

一種視訊處理方法，包括：確定具有運動向量的當前塊；基於特定預測模式生成更新的運動向量；以及基於所述更新的運動向量，執行所述當前塊與包括所述當前塊的視訊資料的位元流表示之間的轉換，其中所述特定預測模式包括以下中的一個或多個：雙向光流(BIO)細化、解碼器側運動向量細化(DMVR)、畫面播放速率上轉換(FRUC)技術或模板匹配技術，其中所述更新的運動向量用於運動補償和/或時間運動預測；並且其中所述時間運動預測包括時間運動向量預測(TMVP)和/或替代時間運動向量預測(ATMVP)。 A video processing method includes: determining a current block with a motion vector; generating an updated motion vector based on a specific prediction mode; and Based on the updated motion vector, perform conversion between the current block and the bit stream representation of the video data including the current block, wherein the specific prediction mode includes one or more of the following: bidirectional optical flow (BIO) refinement, decoder-side motion vector refinement (DMVR), frame rate up conversion (FRUC) technology or template matching technology, wherein the updated motion vector is used for motion compensation and/or temporal motion prediction; and The temporal motion prediction includes temporal motion vector prediction (TMVP) and/or alternative temporal motion vector prediction (ATMVP).

一種處理視訊資料的方法，包括：確定當前塊和對應的相鄰塊；構造所述當前塊的最終預測塊作為從所述當前塊的運動向量導出的預測塊和從一個或多個相鄰塊的運動向量導出的預測塊的加權和；以及基於所述當前塊的最終預測塊，執行所述當前塊與包括所述當前塊的視訊資料的位元流表示之間的轉換；其中短階內插濾波器用於構造所述當前塊的最終預測塊。 A method for processing video data, including: determining a current block and a corresponding neighboring block; constructing a final prediction block of the current block as a prediction block derived from a motion vector of the current block and from one or more neighboring blocks The weighted sum of the prediction block derived from the motion vector of the current block; and based on the final prediction block of the current block, the conversion between the current block and the bit stream representation of the video data including the current block is performed; The interpolation filter is used to construct the final prediction block of the current block.

如申請專利範圍第9項所述的方法，其中所述短階內插濾波器是雙線、4階或6階濾波器。 The method according to item 9 of the scope of patent application, wherein the short-order interpolation filter is a two-line, 4th or 6th order filter.

如申請專利範圍第10項所述的方法，其中所述短階內插濾波器是不對稱6階濾波器，並且所述濾波器用於亮度分量，並且其中對於子像素位置，左側/上側的4個像素和右側/下側的2個像素用於內插。 The method according to the 10th item of the scope of patent application, wherein the short-order interpolation filter is an asymmetric 6th-order filter, and the filter is used for the luminance component, and wherein for the sub-pixel position, the left/upper 4 2 pixels on the right/lower side are used for interpolation.

如申請專利範圍第1至11項中任一項所述的方法，其中所述方法應用於平移運動，並且禁用仿射模式。 The method according to any one of the claims 1 to 11, wherein the method is applied to translational motion, and the affine mode is disabled.

如申請專利範圍第1至11項中任一項所述的方法，其中所述方法僅應用於w×h

T的塊，其中w和h是所述當前塊的寬度和高度，並且T是閾值。 The method described in any one of items 1 to 11 in the scope of the patent application, wherein the method is only applied to w×h

A block of T, where w and h are the width and height of the current block, and T is the threshold.

如申請專利範圍第1至11項中任一項所述的方法，其中所述方法僅應用於w

T且h

T的塊，其中w和h是所述當前塊的寬度和高度，並且T是閾值。 The method described in any one of items 1 to 11 in the scope of the patent application, wherein the method is only applied to w

T and h

如申請專利範圍第1至11項中任一項所述的方法，其中所述方法應用於所有色度分量。 The method according to any one of the claims 1 to 11, wherein the method is applied to all chrominance components.

如申請專利範圍第1至11項中任一項所述的方法，其中所述方法僅應用於亮度分量。 The method described in any one of items 1 to 11 in the scope of the patent application, wherein the method is only applied to the luminance component.

一種視訊系統中的裝置，包括處理器和其上具有指令的非暫時性記憶體，其中所述指令在由所述處理器執行時使所述處理器實現如申請專利範圍第1至16項中任一項所述的方法。 A device in a video system, comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions, when executed by the processor, enable the processor to implement as described in items 1 to 16 of the scope of patent application Any one of the methods.

一種儲存在非暫時性電腦可讀介質上的電腦程式產品，所述電腦程式產品包括用於實施如申請專利範圍第1至16項中任一項所述的方法的程式碼。 A computer program product stored on a non-transitory computer readable medium, the computer program product including program code for implementing the method described in any one of the 1 to 16 patents.

一種計算機可讀儲存介質，所述計算機可讀儲存介質用於儲存程序代碼，其中所述程序代碼由計算機執行時，所述計算機實現申請專利範圍第1至16項中任一項所述的方法。 A computer-readable storage medium for storing program code, wherein when the program code is executed by a computer, the computer implements the method described in any one of items 1 to 16 in the scope of patent application .