WO2022061680A1

WO2022061680A1 - Inter-frame prediction method, encoder, decoder, and computer storage medium

Info

Publication number: WO2022061680A1
Application number: PCT/CN2020/117559
Authority: WO
Inventors: 谢志煌
Original assignee: Oppo广东移动通信有限公司
Priority date: 2020-09-24
Filing date: 2020-09-24
Publication date: 2022-03-31
Also published as: CN116250232A

Abstract

Disclosed in embodiments of the present application are an inter-frame prediction method, an encoder, a decoder, and a computer storage medium. The method comprises: a decoder parsing a code stream to obtain a prediction mode parameter of a current block; when the prediction mode parameter indicates using an inter-frame prediction mode to determine an inter-frame prediction value of the current block, determining a first prediction value of a sub-block of the current block, the sub-block comprising one or more pixel points; for a pixel point in the sub-block, determining a first motion vector deviation between a motion vector of the pixel point and a motion vector of the sub-block; determining a filter parameter according to the first motion vector deviation; using a separable filter to perform filtering processing on the basis of the filter parameter and the first prediction value to obtain a prediction value of the pixel point; and determining a second prediction value of the sub-block according to the prediction value of the pixel point, the second prediction value being used for determining a reconstruction value of the sub-block.

Description

帧间预测方法、编码器、解码器以及计算机存储介质Inter-frame prediction method, encoder, decoder, and computer storage medium

技术领域technical field

本申请涉及视频编解码技术领域，尤其涉及一种帧间预测方法、编码器、解码器以及计算机存储介质。The present application relates to the technical field of video coding and decoding, and in particular, to an inter-frame prediction method, an encoder, a decoder, and a computer storage medium.

背景技术Background technique

在视频编解码领域，为了兼顾性能和代价，一般情况下，在多功能视频编码(Versatile Video Coding，VVC)及数字音视频编解码技术标准工作组(Audio Video coding Standard Workgroup of China，AVS)中的仿射预测是基于子块来实现的。目前，提出了使用光流原理的预测修正(prediction refinement with optical flow，PROF)使用光流原理对基于子块的仿射预测进行修正，进而提高了压缩性能。In the field of video coding and decoding, in order to take into account performance and cost, in general, in the Versatile Video Coding (VVC) and Digital Audio Video Coding Standard Workgroup of China (AVS) The affine prediction of is implemented based on sub-blocks. At present, prediction refinement with optical flow (PROF) using optical flow principle is proposed to modify sub-block-based affine prediction, thereby improving the compression performance.

然而，PROF的应用是基于子块内的像素点的运动矢量与子块运动矢量的偏差非常小的情况，也就是说，子块内的像素点的运动矢量与子块运动矢量的偏差非常小的情况下使用PROF的光流计算方法是有效的，但是，由于PROF依赖于基准位置的水平方向和垂直方向的梯度，在实际位置离基准位置较远的情况下，基准位置的水平和垂直方向的梯度并不能真实的反映基准位置和实际位置之间的水平和垂直方向的梯度，因此，在子块内的像素点的运动矢量与子块运动矢量的偏差较大的情况下，该方法就不是特别有效了。However, the application of PROF is based on the fact that the deviation between the motion vector of the pixel in the sub-block and the motion vector of the sub-block is very small, that is to say, the deviation of the motion vector of the pixel in the sub-block and the motion vector of the sub-block is very small. The optical flow calculation method using PROF is effective in the case of The gradient does not truly reflect the gradient in the horizontal and vertical directions between the reference position and the actual position. Therefore, when the deviation between the motion vector of the pixel in the sub-block and the motion vector of the sub-block is large, this method will Not particularly effective anymore.

发明内容SUMMARY OF THE INVENTION

本申请提出一种帧间预测方法、编码器、解码器以及计算机存储介质，可以大大提升编码性能，从而提高了编解码效率。The present application proposes an inter-frame prediction method, an encoder, a decoder, and a computer storage medium, which can greatly improve encoding performance and thus improve encoding and decoding efficiency.

本申请的技术方案是这样实现的：The technical solution of the present application is realized as follows:

第一方面，本申请实施例提供了一种帧间预测方法，应用于解码器，该方法包括：In a first aspect, an embodiment of the present application provides an inter-frame prediction method, which is applied to a decoder, and the method includes:

解析码流，获取当前块的预测模式参数；Parse the code stream to obtain the prediction mode parameters of the current block;

当所述预测模式参数指示使用帧间预测模式确定所述当前块的帧间预测值时，确定所述当前块的子块的第一预测值；其中，所述子块包括一个或多个像素点；When the prediction mode parameter indicates that the inter prediction value of the current block is determined using the inter prediction mode, a first prediction value of a sub-block of the current block is determined; wherein the sub-block includes one or more pixels point;

对于所述子块内的一像素点，确定所述像素点的运动矢量与所述子块的运动矢量之间的第一运动矢量偏差；For a pixel in the sub-block, determining a first motion vector deviation between the motion vector of the pixel and the motion vector of the sub-block;

根据所述第一运动矢量偏差确定滤波参数；determining filtering parameters according to the first motion vector deviation;

基于所述滤波参数和所述第一预测值，利用可分离滤波器进行滤波处理，获得所述像素点的预测值；Based on the filtering parameters and the first predicted value, filtering is performed using a separable filter to obtain the predicted value of the pixel point;

根据所述像素点的预测值，确定所述子块的第二预测值；其中，所述第二预测值用于确定所述子块的重建值。According to the predicted value of the pixel point, a second predicted value of the sub-block is determined; wherein, the second predicted value is used to determine the reconstructed value of the sub-block.

第二方面，本申请实施例提供了一种帧间预测方法，应用于编码器，该方法包括：In a second aspect, an embodiment of the present application provides an inter-frame prediction method, which is applied to an encoder, and the method includes:

确定当前块的预测模式参数；determine the prediction mode parameter of the current block;

根据所述像素点的预测值，确定所述子块的第二预测值；其中，所述第二预测值用于确定所述子块的残差。According to the predicted value of the pixel point, a second predicted value of the sub-block is determined; wherein, the second predicted value is used to determine the residual of the sub-block.

第三方面，本申请实施例提供了一种解码器，所述解码器包括解析部分，第一确定部分；In a third aspect, an embodiment of the present application provides a decoder, where the decoder includes a parsing part and a first determining part;

所述解析部分，配置为解析码流，获取当前块的预测模式参数；The parsing part is configured to parse the code stream to obtain the prediction mode parameter of the current block;

所述第一确定部分，配置为当所述预测模式参数指示使用帧间预测模式确定所述当前块的帧间预测值时，确定所述当前块的子块的第一预测值；其中，所述子块包括一个或多个像素点；对于所述子块内的一像素点，确定所述像素点的运动矢量与所述子块的运动矢量之间的第一运动矢量偏差；根据所述第一运动矢量偏差确定滤波参数；基于所述滤波参数和所述第一预测值，利用可分离滤波器进行滤波处理，获得所述像素点的预测值；根据所述像素点的预测值，确定所述子块的第二预测值；其中，所述第二预测值用于确定所述子块的重建值。The first determination part is configured to determine the first prediction value of the sub-block of the current block when the prediction mode parameter indicates that the inter prediction value of the current block is determined using the inter prediction mode; wherein, the the sub-block includes one or more pixel points; for a pixel point in the sub-block, determine the first motion vector deviation between the motion vector of the pixel point and the motion vector of the sub-block; according to the The first motion vector deviation determines a filter parameter; based on the filter parameter and the first predicted value, a separable filter is used to perform filtering processing to obtain the predicted value of the pixel point; according to the predicted value of the pixel point, determine the second prediction value of the sub-block; wherein the second prediction value is used to determine the reconstruction value of the sub-block.

第四方面，本申请实施例提供了一种解码器，所述解码器包括第一处理器、存储有所述第一处理器可执行指令的第一存储器，当所述指令被执行时，所述第一处理器执行时实现如上所述的帧间预测方法。In a fourth aspect, an embodiment of the present application provides a decoder. The decoder includes a first processor and a first memory storing an executable instruction of the first processor. When the instruction is executed, the When executed by the first processor, the above-mentioned inter-frame prediction method is implemented.

第五方面，本申请实施例提供了一种编码器，所述编码器包括第二确定部分；In a fifth aspect, an embodiment of the present application provides an encoder, and the encoder includes a second determination part;

所述第二确定部分，配置为确定当前块的预测模式参数；当所述预测模式参数指示使用帧间预测模式确定所述当前块的帧间预测值时，确定所述当前块的子块的第一预测值；其中，所述子块包括一个或多个像素点；对于所述子块内的一像素点，确定所述像素点的运动矢量与所述子块的运动矢量之间的第一运动矢量偏差；根据所述第一运动矢量偏差确定滤波参数；基于所述滤波参数和所述第一预测值，利用可分离滤波器进行滤波处理，获得所述像素点的预测值；根据所述像素点的预测值，确定所述子块的第二预测值；其中，所述第二预测值用于确定所述子块的残差。The second determination part is configured to determine the prediction mode parameter of the current block; when the prediction mode parameter indicates that the inter prediction value of the current block is determined by using the inter prediction mode, determine the sub-block of the current block. The first predicted value; wherein, the sub-block includes one or more pixel points; for a pixel point in the sub-block, determine the first prediction value between the motion vector of the pixel point and the motion vector of the sub-block a motion vector deviation; determining a filtering parameter according to the first motion vector deviation; performing filtering processing with a separable filter based on the filtering parameter and the first predicted value, to obtain the predicted value of the pixel point; The predicted value of the pixel point is used to determine the second predicted value of the sub-block; wherein, the second predicted value is used to determine the residual of the sub-block.

第六方面，本申请实施例提供了一种编码器，所述编码器包括第二处理器、存储有所述第二处理器可执行指令的第二存储器，当所述指令被执行时，所述第二处理器执行时实现如上所述的帧间预测方法。In a sixth aspect, an embodiment of the present application provides an encoder, the encoder includes a second processor, and a second memory storing an executable instruction of the second processor, when the instruction is executed, the When executed by the second processor, the inter-frame prediction method as described above is implemented.

第七方面，本申请实施例提供了一种计算机存储介质，该计算机存储介质存储有计算机程序，所述计算机程序被第一处理器和第二处理器执行时，实现如上所述的帧间预测方法。In a seventh aspect, an embodiment of the present application provides a computer storage medium, where the computer storage medium stores a computer program, and when the computer program is executed by the first processor and the second processor, the above-mentioned inter-frame prediction is implemented method.

本申请实施例所提供的一种帧间预测方法、编码器、解码器以及计算机存储介质，解码器解析码流，获取当前块的预测模式参数；当预测模式参数指示使用帧间预测模式确定当前块的帧间预测值时，确定当前块的子块的第一预测值；其中，子块包括一个或多个像素点；对于子块内的一像素点，确定像素点的运动矢量与子块的运动矢量之间的第一运动矢量偏差；根据第一运动矢量偏差确定滤波参数；基于滤波参数和第一预测值，利用可分离滤波器进行滤波处理，获得像素点的预测值；根据像素点的预测值，确定子块的第二预测值；其中，第二预测值用于确定子块的重建值。也就是说，本申请提出的帧间预测方法，可以在基于子块的预测之后，针对运动矢量与子块的运动矢量有偏差的像素点，可以在基于子块的第一预测值的基础上，利用可分离滤波器进行基于点的二次预测，获得第二预测值。其中，可分离滤波器可以分离为水平方向的一维滤波器和垂直方向的一维滤波器，滤波后可以获得该像素点对应的新的预测值，最终获得子块修正后的第二预测值，本申请提出的帧间预测方法能够很好的适用于全部场景，大大提升编码性能，从而提高了编解码效率。An inter-frame prediction method, an encoder, a decoder, and a computer storage medium provided by the embodiments of the present application, the decoder parses the code stream, and obtains the prediction mode parameter of the current block; when the prediction mode parameter indicates that the inter-frame prediction mode is used to determine the current block When the inter-frame prediction value of the block, determine the first prediction value of the sub-block of the current block; wherein, the sub-block includes one or more pixels; for a pixel in the sub-block, determine the motion vector of the pixel and the sub-block The first motion vector deviation between the motion vectors of the The predicted value of the sub-block is determined, and the second predicted value is used to determine the reconstruction value of the sub-block. That is to say, the inter-frame prediction method proposed in the present application can, after the prediction based on the sub-block, for the pixel points whose motion vector is deviated from the motion vector of the sub-block, can be based on the first prediction value of the sub-block on the basis of , use a separable filter to perform point-based secondary prediction to obtain a second predicted value. Among them, the separable filter can be separated into a one-dimensional filter in the horizontal direction and a one-dimensional filter in the vertical direction. After filtering, a new predicted value corresponding to the pixel point can be obtained, and finally a second predicted value after sub-block correction can be obtained. , the inter-frame prediction method proposed in this application can be well applied to all scenarios, greatly improving the encoding performance, thereby improving the encoding and decoding efficiency.

附图说明Description of drawings

图1为像素的插值示意图；Figure 1 is a schematic diagram of pixel interpolation;

图2为子块插值的示意图一；2 is a schematic diagram 1 of sub-block interpolation;

图3为子块插值的示意图二；3 is a schematic diagram 2 of sub-block interpolation;

图4为样本位置示意图；Figure 4 is a schematic diagram of the sample location;

图5为本申请实施例提供的一种视频编码***的组成框图示意图；5 is a schematic block diagram of the composition of a video coding system provided by an embodiment of the present application;

图6为本申请实施例提供的一种视频解码***的组成框图示意图；6 is a schematic block diagram of the composition of a video decoding system provided by an embodiment of the present application;

图7为帧间预测方法的实现流程示意图一；7 is a schematic diagram 1 of the implementation flow of the inter-frame prediction method;

图8为确定运动矢量偏差的示意图；8 is a schematic diagram of determining a motion vector deviation;

图9为滤波处理的示意图一；FIG. 9 is a schematic diagram 1 of filtering processing;

图10为滤波处理的示意图二；10 is a schematic diagram 2 of filtering processing;

图11为双向预测场景中的二次预测示意图；11 is a schematic diagram of secondary prediction in a bidirectional prediction scenario;

图12为扩展当前子块的示意图；12 is a schematic diagram of extending the current sub-block;

图13为4x4子块的示意图；Figure 13 is a schematic diagram of a 4x4 sub-block;

图14为替换像素的示意图；14 is a schematic diagram of a replacement pixel;

图15为帧间预测方法的实现流程示意图二；15 is a schematic diagram 2 of the implementation flow of the inter-frame prediction method;

图16为解码器的组成结构示意图一；16 is a schematic diagram of the composition structure of a decoder;

图17为解码器的组成结构示意图二；FIG. 17 is a schematic diagram of the composition structure of the decoder 2;

图18为编码器的组成结构示意图一；FIG. 18 is a schematic diagram of the composition structure of the encoder one;

图19为编码器的组成结构示意图二。FIG. 19 is a second schematic diagram of the composition and structure of the encoder.

具体实施方式detailed description

下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述。可以理解的是，此处所描述的具体实施例仅仅用于解释相关申请，而非对该申请的限定。另外还需要说明的是，为了便于描述，附图中仅示出了与有关申请相关的部分。The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. It should be understood that the specific embodiments described herein are only used to explain the related application, but not to limit the application. In addition, it should be noted that, for the convenience of description, only the parts related to the relevant application are shown in the drawings.

目前，通用的视频编解码标准基于都采用基于块的混合编码框架。视频图像中的每一帧被分割成相同大小(比如128×128，64×64等)的正方形的最大编码单元(Largest Coding Unit，LCU)，每个最大编码单元还可以根据规则划分成矩形的编码单元(Coding Unit，CU)；而且编码单元可能还会划分成更小的预测单元(Prediction Unit，PU)。具体地，混合编码框架可以包括有预测、变换(Transform)、量化(Quantization)、熵编码(EntropyCoding)、环路滤波(In Loop Filter)等模块；其中，预测模块可以包括帧内预测(intra prediction)和帧间预测(inter prediction)，帧间预测可以包括运动估计(motion estimation)和运动补偿(motion compensation)。由于视频图像的一个帧内相邻像素之间存在很强的相关性，在视频编解码技术中使用帧内预测方式能够消除相邻像素之间的空间冗余；但是由于视频图像中的相邻帧之间也存在着很强的相似性，在视频编解码技术中使用帧间预测方式消除相邻帧之间的时间冗余，从而能够提高编码效率。下述本申请将以帧间预测进行详细描述。Currently, common video coding standards are based on the block-based hybrid coding framework. Each frame in the video image is divided into square largest coding units (Largest Coding Units, LCUs) of the same size (such as 128×128, 64×64, etc.), and each LCU can also be divided into rectangular ones according to the rules. Coding unit (Coding Unit, CU); and the coding unit may be divided into smaller prediction units (Prediction Unit, PU). Specifically, the hybrid coding framework may include modules such as prediction, transform (Transform), quantization (Quantization), entropy coding (EntropyCoding), loop filtering (In Loop Filter), etc.; wherein, the prediction module may include intra prediction (intra prediction) ) and inter prediction, which can include motion estimation and motion compensation. Since there is a strong correlation between adjacent pixels in a frame of a video image, the use of intra-frame prediction in video coding and decoding technology can eliminate the spatial redundancy between adjacent pixels; There is also a strong similarity between frames. In the video coding and decoding technology, the inter-frame prediction method is used to eliminate the temporal redundancy between adjacent frames, so as to improve the coding efficiency. The present application will be described in detail below in terms of inter prediction.

帧间预测即使用已经编/解码的帧来预测当前帧中需要编/解码的部分，在基于块的编解码框架里，需要编/解码的部分通常是编码单元或预测单元。这里把需要编/解码的编码单元或预测单元统称为当前块。平移运动是视频中的一种常见而且简单的运动方式，所以平移的预测也是视频编解码中的一种传统的预测方法。视频中的平移运动可以理解为一部分内容随着时间的变化，从一个帧上的某一个位置移动到了另一个帧上的某一个位置。平移的一个简单的单向预测可以用某一帧与当前帧之间的一个运动矢量(motion vector，MV)来表示。这里所说的某一帧为当前帧的一个参考帧，当前块通过这个包含参考帧和运动矢量的运动信息可以找到参考帧上的一个与当前块大小相同的参考块，把这个参考块作为当前块的预测块。理想的平移运动中，当前块的内容在不同帧之间没有变形、旋转等变化以及亮度颜色的变化等，然而，视频中的内容并不总是符合这样的理想情况。双向预测在一定程度上可以解决上述问题。通常的双向预测是指双向的平移的预测。双向预测即是用两个参考帧和运动矢量的运动信息分别从两个参考帧(两个参考帧可能是同一个参考帧)找到两个与当前块大小相同的参考块，利用这两个参考块生成当前块的预测块。生成方法包括平均、加权平均以及一些其他的计算等。Inter-frame prediction is to use the already encoded/decoded frame to predict the part that needs to be encoded/decoded in the current frame. In the block-based encoding/decoding framework, the part that needs to be encoded/decoded is usually a coding unit or a prediction unit. Here, the coding unit or prediction unit that needs to be encoded/decoded is collectively referred to as the current block. Translation motion is a common and simple motion method in video, so translation prediction is also a traditional prediction method in video coding and decoding. Panning motion in video can be understood as a part of the content moving from a certain position on one frame to a certain position on another frame over time. A simple unidirectional prediction of translation can be represented by a motion vector (MV) between a certain frame and the current frame. A certain frame mentioned here is a reference frame of the current frame. The current block can find a reference block on the reference frame with the same size as the current block through the motion information including the reference frame and motion vector, and use this reference block as the current block. The predicted block for the block. In an ideal translation motion, the content of the current block does not change from frame to frame, such as deformation, rotation, etc., as well as changes in brightness and color, etc. However, the content in the video does not always meet such an ideal situation. Bidirectional prediction can solve the above problems to a certain extent. The usual bidirectional prediction refers to bidirectional translation prediction. Bidirectional prediction is to use the motion information of two reference frames and motion vectors to find two reference blocks with the same size as the current block from two reference frames (the two reference frames may be the same reference frame), and use these two reference frames. The block generates a predicted block for the current block. Generation methods include averaging, weighted averaging, and some other calculations.

在本申请中，可以认为预测是运动补偿的一部分，有些文献会把本申请中的预测称为运动补偿，如本申请所说仿射预测，有些文献会叫作仿射运动补偿。In this application, prediction can be considered as a part of motion compensation, and some documents will call the prediction in this application motion compensation, and as mentioned in this application, affine prediction, some documents will call it affine motion compensation.

旋转、放大、缩小、扭曲、形变等也是在视频中常见的变化，然而，普通的平移预测并不能很好地处理这种变化，于是仿射(affine)预测模型被应用于视频编解码中，如VVC和AVS中的仿射，其中，VVC和AVS3的仿射预测模型相似。在旋转、放大、缩小、扭曲、形变等变化中，可以认为当前块不是所有点都使用相同的MV，因此需要导出每一个点的MV。仿射预测模型用少量的几个参数通过计算导出每一个点的MV。VVC和AVS3的仿射预测模型都使用了2控制点(4参数)和3控制点(6参数)模型。2控制点即当前块的左上角和右上角2个控制点，3控制点即当前块的左上角、右上角和左下角3个控制点。一般情况下，由于每个MV包括一个x分量和一个y分量，所以2个控制点有4个参数，3个控制点有6个参数。Rotation, enlargement, reduction, distortion, deformation, etc. are also common changes in video. However, ordinary translation prediction cannot handle such changes well, so affine prediction models are applied to video codecs. Such as affine in VVC and AVS, where the affine prediction models of VVC and AVS3 are similar. In changes such as rotation, enlargement, reduction, distortion, deformation, etc., it can be considered that not all points of the current block use the same MV, so the MV of each point needs to be derived. The affine prediction model derives the MV of each point by calculation with a small number of parameters. Both VVC and AVS3 affine prediction models use 2-control point (4-parameter) and 3-control-point (6-parameter) models. The 2 control point is the 2 control points in the upper left corner and the upper right corner of the current block, and the 3 control point is the 3 control points in the upper left corner, the upper right corner and the lower left corner of the current block. In general, since each MV includes an x-component and a y-component, 2 control points have 4 parameters, and 3 control points have 6 parameters.

根据仿射预测模型可以为每一个像素点导出一个MV，每一个像素点都可以在参考帧中找到其对应的位置，如果这个位置不是整像素点，那么需要通过插值的方法得出这个分像素点的值。现在视频编解码标准中使用的插值方法通常都是有限长单位冲激响应(Finite Impulse Response，FIR)滤波器来实现，而按这种方式实现复杂度(成本)是很高的。例如，在AVS3中，对亮度分量使用8抽头的插值滤波器，而且普通模式分像素精度是1/4像素的，affine模式的分像素精度是1/16像素的。对每一个符合1/16像素精度的分像素点，需要使用水平方向的8个整像素和垂直方向8个整像素即64个整像素插值得到。图1为像素的插值示意图，如图1所示，圆形像素是想要得到的分像素点，深色的正方形像素是该分像素对应的整像素的位置，它们两个之间的矢量就是分像素的运动矢量，浅色的正方形像素是对圆形分像素点的插值需要用到的像素，要得到该分像素点的值，需要这些8x8的浅色的正方形像素区域的像素值进行插值，也包含深色的像素点。According to the affine prediction model, an MV can be derived for each pixel, and each pixel can find its corresponding position in the reference frame. If the position is not an integer pixel, then the sub-pixel needs to be obtained by interpolation. point value. The interpolation methods used in the current video coding and decoding standards are usually implemented by finite-length unit impulse response (Finite Impulse Response, FIR) filters, and the complexity (cost) to implement in this way is very high. For example, in AVS3, an 8-tap interpolation filter is used for the luminance component, and the sub-pixel accuracy of normal mode is 1/4 pixel, and the sub-pixel accuracy of affine mode is 1/16 pixel. For each sub-pixel point that meets the 1/16 pixel precision, it needs to be obtained by using 8 integer pixels in the horizontal direction and 8 integer pixels in the vertical direction, that is, 64 integer pixels. Figure 1 is a schematic diagram of pixel interpolation. As shown in Figure 1, the circular pixel is the desired sub-pixel point, the dark square pixel is the position of the integer pixel corresponding to the sub-pixel, and the vector between them is The motion vector of the sub-pixels, the light-colored square pixels are the pixels that need to be used for the interpolation of the circular sub-pixel points. To get the value of the sub-pixel point, the pixel values of these 8x8 light-colored square pixel areas need to be interpolated. , which also contains dark pixels.

在传统的平移的预测中，当前块的每个像素点的MV是相同的。如果进一步地引入子块的概念，子块的大小如4x4，8x8等。图2为子块插值的示意图一，4x4的块插值需要使用的像素区域如图2所示。图3为子块插值的示意图二，8x8的块插值需要使用的像素区域如图3所示。In traditional translation prediction, the MV of each pixel of the current block is the same. If the concept of sub-block is further introduced, the size of the sub-block is 4x4, 8x8, etc. FIG. 2 is a schematic diagram 1 of sub-block interpolation. The pixel area that needs to be used for 4×4 block interpolation is shown in FIG. 2 . FIG. 3 is a schematic diagram 2 of sub-block interpolation. The pixel area that needs to be used for 8×8 block interpolation is shown in FIG. 3 .

如果一个子块中的每个像素点的MV是相同的。那么一个子块中的像素点可以一起进行插值，从而分摊带宽、使用同一个相位的滤波器以及共享插值过程的中间值。但是如果每一个像素点使用一个MV，那么带宽就会增加，而且可能会使用不同相位的滤波器以及不能共享插值过程的中间值。If the MV of each pixel in a sub-block is the same. The pixels in a sub-block can then be interpolated together, sharing the bandwidth, using filters of the same phase, and sharing the intermediate values of the interpolation process. But if you use one MV per pixel, the bandwidth will increase, and you may use filters of different phases and intermediate values that cannot share the interpolation process.

基于点的仿射预测代价很高，因此，为了兼顾性能和代价，在VVC和AVS3中的仿射预测是基于子块来实现的。AVS3中的子块大小有4x4和8x8两种大小，VVC中使用4x4的子块大小。每一个子块有一个MV，子块内部的像素点共享同一个MV。从而对子块内部的所有像素点统一做插值。通过上述方法，基于子块的仿射预测与其他基于子块的预测方法的运动补偿复杂度相近。Point-based affine prediction is expensive. Therefore, in order to balance performance and cost, affine prediction in VVC and AVS3 is implemented based on sub-blocks. The sub-block size in AVS3 is 4x4 and 8x8, and the 4x4 sub-block size is used in VVC. Each sub-block has an MV, and the pixels inside the sub-block share the same MV. Therefore, all the pixels inside the sub-block are uniformly interpolated. With the above method, the sub-block-based affine prediction is similar in motion compensation complexity to other sub-block-based prediction methods.

可见，基于子块的仿射预测方法中，子块内部的像素点共享同一个MV，其中，确定这个共享MV的方法是取当前子块中心的MV。而对4x4、8x8等水平垂直方向上至少有一个是偶数个像素的子块来说，其中心其实是落在非整数像素点上的。目前的标准中都取一个整数像素的位置，例如，如对4x4子块，取距离左上角位置为(2，2)的像素点。对8x8子块，取距离左上角位置为(4，4)的像素点。It can be seen that in the sub-block-based affine prediction method, the pixels inside the sub-block share the same MV, and the method for determining this shared MV is to take the MV of the center of the current sub-block. For 4x4, 8x8 and other sub-blocks with at least one even number of pixels in the horizontal and vertical directions, the center of the sub-block actually falls on a non-integer pixel point. In the current standard, the position of an integer pixel is taken. For example, for a 4x4 sub-block, the pixel point at the position (2, 2) from the upper left corner is taken. For the 8x8 sub-block, take the pixel at the position (4, 4) from the upper left corner.

仿射预测模型可以根据当前块所使用的控制点(2个控制点或3个控制点)导出每一个像素点的MV。在基于子块的仿射预测中，根据上一段中像素点计算出这个位置的MV，作为该子块的MV。为了推导每个子块的运动矢量，可以将每个子块的中心采样的运动矢量取整到1/16精度，然后进行运动补偿。The affine prediction model can derive the MV of each pixel based on the control points (2 control points or 3 control points) used by the current block. In the sub-block-based affine prediction, the MV of this position is calculated according to the pixels in the previous segment, as the MV of the sub-block. In order to derive the motion vector of each sub-block, the motion vector sampled at the center of each sub-block can be rounded to 1/16 precision, and then motion compensation is performed.

随着技术的发展，一个叫做使用光流的预测改进技术PROF的方法被提出。这个技术可以在不增加带宽的情况下，对基于块的仿射预测的预测值进行改进。在基于子块的仿射预测做完后，对每一个已完成基于子块的仿射预测的像素点计算其水平方向和垂直方向的梯度。在VVC中PROF计算梯度时使用的是3抽头的滤波器[-1，0，1]，其计算方法和双向光流(Bi-directional Optical flow，BDOF)的方法相同。之后，对每一个像素点，计算其运动矢量偏差，该运动矢量偏差即为当前像素点的运动矢量与整个子块使用MV的差值。这些运动矢量偏差都可以根据仿射预测模型的公式计算得出。由于公式的特性，一些子块的相同位置的运动矢量偏差是相同的，对这些子块，只需要计算出一组运动矢量偏差，其他子块可以直接复用这些值。对每一个像素点，使用该点的水平垂直梯度及运动矢量偏差(包括水平方向的偏差和垂直方向的偏差)计算出该像素点的预测值的修正值，然后将原预测值，即基于子块的仿射预测的预测值，加上预测值的修正值，便可以得到修正后的预测值。With the development of technology, a method called prediction improvement technique PROF using optical flow was proposed. This technique can improve the prediction value of block-based affine prediction without increasing bandwidth. After the sub-block-based affine prediction is completed, the gradients in the horizontal and vertical directions are calculated for each pixel for which the sub-block-based affine prediction has been completed. In VVC, PROF uses a 3-tap filter [-1, 0, 1] when calculating the gradient, and its calculation method is the same as that of Bi-directional Optical flow (BDOF). Afterwards, for each pixel point, its motion vector deviation is calculated, and the motion vector deviation is the difference between the motion vector of the current pixel point and the MV used by the entire sub-block. These motion vector deviations can be calculated according to the formula of the affine prediction model. Due to the characteristics of the formula, the motion vector deviations at the same position of some sub-blocks are the same. For these sub-blocks, only a set of motion vector deviations need to be calculated, and other sub-blocks can directly reuse these values. For each pixel, use the horizontal and vertical gradients of the point and the motion vector deviation (including the deviation in the horizontal direction and the deviation in the vertical direction) to calculate the corrected value of the predicted value of the pixel, and then use the original predicted value, that is, based on the sub- The predicted value of the affine prediction of the block is added to the modified value of the predicted value, and the modified predicted value can be obtained.

计算水平垂直方向的梯度时，使用[-1，0，1]滤波器，也就是对当前像素点，水平方向会使用到其左边距离为一的像素点和右边距离为一的像素点的预测值，垂直方向会使用到其上边距离为一的像素点和下边距离为一的像素点的预测值。如果当前像素点是当前块的边界位置，那么上述的几个像素点中有的会超过当前块的边界一个像素距离。使用当前块的边界的预测值向外填充一个像素距离的位置以满足梯度计算，从而不需要额外地增加上述超过当前块的边界一个像素距离的预测值。由于梯度计算只需要使用基于子块的仿射预测的预测值，所以不需要增加额外的带宽。When calculating the gradient in the horizontal and vertical directions, the [-1, 0, 1] filter is used, that is, for the current pixel, the horizontal direction will use the prediction of the pixels with a distance of 1 on the left and a pixel with a distance of 1 on the right. value, the vertical direction will use the predicted value of the pixel with a distance of one above and a distance of one below. If the current pixel is the boundary position of the current block, some of the above-mentioned pixel points will exceed the boundary of the current block by a distance of one pixel. The predicted value of the boundary of the current block is used to fill the position of one pixel distance outward to satisfy the gradient calculation, so that there is no need to additionally increase the predicted value of one pixel distance beyond the boundary of the current block. Since the gradient computation only needs to use the predicted value of the subblock-based affine prediction, no additional bandwidth needs to be added.

在VVC标准文本中，对根据控制点的MV导出当前块的各个子块的MV以及子块内的各像素点的运动矢量偏差。VVC中每个子块使用的作为子块MV的像素点都相同，所以只需要导出一组子块的运动矢量偏差，其他子块可以复用该子块。进一步地，在VVC标准文本中，对PROF流程的描述，PROF对运动矢量偏差的计算包含在上面的流程中。In the VVC standard text, the MV of each sub-block of the current block and the motion vector deviation of each pixel in the sub-block are derived from the MV of the control point. Each sub-block in VVC uses the same pixel points as the sub-block MV, so it is only necessary to derive the motion vector deviation of a group of sub-blocks, and other sub-blocks can reuse the sub-block. Further, in the VVC standard text, the description of the PROF process, the calculation of the motion vector deviation by PROF is included in the above process.

AVS3的仿射预测在计算子块的MV时，和VVC的基本原理相同，但AVS3对当前块的左上角子块A，右上角子块B和左下角子块C有特殊的处理。AVS3's affine prediction has the same basic principle as VVC when calculating the MV of a sub-block, but AVS3 has special processing for the upper-left sub-block A, upper-right sub-block B and lower-left sub-block C of the current block.

下面是AVS3标准文本对仿射运动单元子块运动矢量阵列的导出的描述：The following is the description of the AVS3 standard text for the derivation of affine motion unit subblock motion vector arrays:

如果仿射控制点运动矢量组中有3个运动矢量，那么运动矢量组可以表示为mvsAffine(mv0，mv1，mv2)；如果仿射控制点运动矢量组中有2个运动矢量，那么运动矢量组可以表示为mvsAffine(mv0，mv1)。接着，可以按照以下步骤导出仿射运动单元子块运动矢量阵列：If there are 3 motion vectors in the affine control point motion vector group, then the motion vector group can be expressed as mvsAffine(mv0, mv1, mv2); if there are 2 motion vectors in the affine control point motion vector group, then the motion vector group Can be expressed as mvsAffine(mv0, mv1). Next, the affine motion unit subblock motion vector array can be derived as follows:

1、计算变量dHorX、dVerX、dHorY和dVerY：1. Calculate the variables dHorX, dVerX, dHorY and dVerY:

dHorX＝(mv1_x-mv0_x)<<(7-Log(width))；dHorX=(mv1_x-mv0_x)<<(7-Log(width));

dHorY＝(mv1_y-mv0_y)<<(7-Log(width))；dHorY=(mv1_y-mv0_y)<<(7-Log(width));

若运动矢量组为mvsAffine(mv0，mv1，mv2)，则：If the motion vector group is mvsAffine(mv0, mv1, mv2), then:

dVerX＝(mv2_x-mv0_x)<<(7-Log(height))；dVerX=(mv2_x-mv0_x)<<(7-Log(height));

dVerY＝(mv2_y-mv0_y)<<(7-Log(height))；dVerY=(mv2_y-mv0_y)<<(7-Log(height));

若运动矢量组为mvsAffine(mv0，mv1)，则：If the motion vector group is mvsAffine(mv0, mv1), then:

dVerX＝-dHorY；dVerX=-dHorY;

dVerY＝dHorX；dVerY=dHorX;

需要说明的是，图4为样本位置示意图，如图4所示，(xE，yE)是当前预测单元亮度预测块左上角样本在当前图像的亮度样本矩阵中的位置，当前预测单元的宽度和高度分别是width和height，每个子块的宽度和高度分别是subwidth和subheight，当前预测单元亮度预测块的左上角样本所在的子块为A，右上角样本所在的子块为B，左下角样本所在的子块为C。It should be noted that FIG. 4 is a schematic diagram of sample positions. As shown in FIG. 4 , (xE, yE) is the position of the upper left sample of the luminance prediction block of the current prediction unit in the luminance sample matrix of the current image, and the width of the current prediction unit and The heights are width and height respectively, the width and height of each sub-block are subwidth and subheight respectively, the sub-block where the upper-left sample of the luminance prediction block of the current prediction unit is located is A, the sub-block where the upper-right sample is located is B, and the lower-left sample is located. The sub-block is C.

2.1、如果当前预测单元的预测参考模式是‘Pred_List01’或AffineSubblockSizeFlag等于1(AffineSubblockSizeFlag用于指示子块尺寸的大小)，则subwidth和subheight均等于8，(x，y)是尺寸为8x8的子块左上角位置的坐标，则可以计算每个8x8亮度子块的运动矢量mvE(mvE_x，mvE_y)：2.1. If the prediction reference mode of the current prediction unit is 'Pred_List01' or AffineSubblockSizeFlag is equal to 1 (AffineSubblockSizeFlag is used to indicate the size of the subblock), then both subwidth and subheight are equal to 8, and (x, y) is a subblock of size 8x8 The coordinates of the upper left corner position, the motion vector mvE (mvE_x, mvE_y) of each 8x8 luminance sub-block can be calculated:

如果子块是A，则xPos和yPos均等于0；If the subblock is A, both xPos and yPos are equal to 0;

如果子块是B，则xPos等于width，yPos等于0；If the subblock is B, then xPos is equal to width and yPos is equal to 0;

如果子块为C且mvsAffine中有3个运动矢量，则xPos等于0，yPos等于height；If the sub-block is C and there are 3 motion vectors in mvsAffine, then xPos is equal to 0 and yPos is equal to height;

否则，xPos等于(x-xE)+4，yPos等于(y-yE)+4；Otherwise, xPos is equal to (x-xE)+4, and yPos is equal to (y-yE)+4;

因此，当前8x8子块的运动矢量mvE为：Therefore, the motion vector mvE of the current 8x8 subblock is:

mvE_x＝Clip3(-131072，131071，Rounding((mv0_x<<7)+dHorX×xPos+dVerX×yPos，7))；mvE_x=Clip3(-131072, 131071, Rounding((mv0_x<<7)+dHorX×xPos+dVerX×yPos,7));

mvE_y＝Clip3(-131072，131071，Rounding((mv0_y<<7)+dHorY×xPos+dVerY×yPos，7))；mvE_y=Clip3(-131072, 131071, Rounding((mv0_y<<7)+dHorY×xPos+dVerY×yPos,7));

2.2、如果当前预测单元的预测参考模式是‘Pred_List0’或‘Pred_List1’，且AffineSubblockSizeFlag等于0，则subwidth和subheight均等于4，(x，y)是尺寸为4x4的子块左上角位置的坐标，计算每个4x4亮度子块的运动矢量mvE(mvE_x，mvE_y)：2.2. If the prediction reference mode of the current prediction unit is 'Pred_List0' or 'Pred_List1', and AffineSubblockSizeFlag is equal to 0, then both subwidth and subheight are equal to 4, (x, y) is the coordinate of the upper left corner of the subblock with size 4x4, Compute the motion vector mvE(mvE_x, mvE_y) for each 4x4 luma subblock:

如果子块是C且mvAffine中有3个运动矢量，则xPos等于0，yPos等height；If the sub-block is C and there are 3 motion vectors in mvAffine, xPos is equal to 0, yPos, etc. height;

否则，xPos等于(x-xE)+2，yPos等于(y-yE)+2；Otherwise, xPos is equal to (x-xE)+2, and yPos is equal to (y-yE)+2;

因此，当前4x4子块的运动矢量mvE为：Therefore, the motion vector mvE of the current 4x4 subblock is:

mvE_y＝Clip3(-131072，131071，Rounding((mv0_y<<7)+dHorY×xPos+dVerY×yPos，7))。mvE_y=Clip3(-131072, 131071, Rounding((mv0_y<<7)+dHorY×xPos+dVerY×yPos,7)).

下面是AVS3文本对于仿射预测样本导出及亮度、色度样本插值的描述：The following is the description of the AVS3 text for affine prediction sample export and luminance and chrominance sample interpolation:

如果当前预测单元亮度预测块左上角样本在当前图像的亮度样本矩阵中的位置为(xE，yE)。If the position of the upper left sample of the luma prediction block of the current prediction unit in the luma sample matrix of the current image is (xE, yE).

如果当前预测单元的预测参考模式是‘PRED_List0’且AffineSubblockSizeFlag的值为0，mv0E0是MvArrayL0运动矢量集合在(xE+x，yE+y)位置的、4x4单元的LO运动矢量。亮度预测样本矩阵predMatrixL0中的元素predMatrixL0[x][y]的值，是参考图像队列0中参考索引为RefIdxL0的1/16精度亮度样本矩阵中位置为(((xE+x)<<4)+mv0E0_x，(yE+y)<<4)+mv0E0_y)))的样本值，色度预测样本矩阵predMatrixL0中的元素predMatrixL0[x][y]的值，是参考图像队列0中参考索引为RefIdxL0的1/32精度色度样本矩阵中位置为(((xE+2x)<<4)+MvC_x，(yE+2y)<<4)+MvC_y)))的样本值。其中，x1＝((xE+2x)>>3)<<3，y1＝((yE+2y)>>3)<<3，mv1E0是MvArrayL0运动矢量集合在(x1，y1)位置的4x4单元的LO运动矢量，mv2E0是MvArrayL0运动矢量集合在(x1+4，y1)位置的4x4单元的LO运动矢量，mv3E0是MvArrayL0运动矢量集合在(x1，y1+4)位置的4x4单元的LO运动矢量，mv4E0是MvArrayL0运动矢量集合在(x1+4，y1+4)位置的4x4单元的LO运动矢量。If the prediction reference mode of the current prediction unit is 'PRED_List0' and the value of AffineSubblockSizeFlag is 0, mv0E0 is the LO motion vector of the 4x4 unit with the MvArrayL0 motion vector set at (xE+x, yE+y) position. The value of the element predMatrixL0[x][y] in the luma prediction sample matrix predMatrixL0 is the position in the 1/16-precision luma sample matrix with the reference index RefIdxL0 in the reference image queue 0 (((xE+x)<<4) +mv0E0_x, the sample value of (yE+y)<<4)+mv0E0_y))), the value of the element predMatrixL0[x][y] in the chrominance prediction sample matrix predMatrixL0, the reference index in the reference image queue 0 is RefIdxL0 The sample value at position (((xE+2x)<<4)+MvC_x, (yE+2y)<<4)+MvC_y))) in the 1/32 precision chroma sample matrix of . Among them, x1=((xE+2x)>>3)<<3, y1=((yE+2y)>>3)<<3, mv1E0 is the 4x4 unit of MvArrayL0 motion vector set at (x1, y1) position The LO motion vector of the , mv4E0 is the LO motion vector of the 4x4 unit of the MvArrayL0 motion vector set at the (x1+4, y1+4) position.

MvC_x＝(mv1E0_x+mv2E0_x+mv3E0_x+mv4E0_x+2)>>2MvC_x=(mv1E0_x+mv2E0_x+mv3E0_x+mv4E0_x+2)>>2

MvC_y＝(mv1E0_y+mv2E0_y+mv3E0_y+mv4E0_y+2)>>2MvC_y=(mv1E0_y+mv2E0_y+mv3E0_y+mv4E0_y+2)>>2

如果当前预测单元的预测参考模式是‘PRED_List0’且AffineSubblockSizeFlag的值为1，mv0E0是MvArrayL0运动矢量集合在(xE+x，yE+y)位置的、8x8单元的LO运动矢量。亮度预测样本矩阵predMatrixL0中的元素predMatrixL0[x][y]的值，是参考图像队列0中参考索引为RefIdxL0的1/16精度亮度样本矩阵中位置为(((xE+x)<<4)+mv0E0_x，(yE+y)<<4)+mv0E0_y)))的样本值，色度预测样本矩阵predMatrixL0中的元素predMatrixL0[x][y]的值是参考图像队列0中参考索引为RefIdxL0的1/32精度色度样本矩阵中位置为(((xE+2x)<<4)+MvC_x，(yE+2y)<<4)+MvC_y)))的样本值。其中，MvC_x等于mv0E0_x，MvC_y等于mv0E0。If the prediction reference mode of the current prediction unit is 'PRED_List0' and the value of AffineSubblockSizeFlag is 1, mv0E0 is the LO motion vector of an 8x8 unit with the MvArrayL0 motion vector set at (xE+x, yE+y) position. The value of the element predMatrixL0[x][y] in the luma prediction sample matrix predMatrixL0 is the position in the 1/16-precision luma sample matrix with the reference index RefIdxL0 in the reference image queue 0 (((xE+x)<<4) +mv0E0_x, the sample value of (yE+y)<<4)+mv0E0_y))), the value of the element predMatrixL0[x][y] in the chrominance prediction sample matrix predMatrixL0 is the reference index in the reference image queue 0 is RefIdxL0 The sample value at position (((xE+2x)<<4)+MvC_x, (yE+2y)<<4)+MvC_y))) in the 1/32 precision chroma sample matrix. Among them, MvC_x is equal to mv0E0_x, and MvC_y is equal to mv0E0.

如果当前预测单元的预测参考模式是‘PRED_List1’且AffineSubblockSizeFlag的值为0，mv0E1是MvArrayL1运动矢量集合在(xE+x，yE+y)位置的、4x4单元的L1运动矢量。亮度预测样本矩阵predMatrixL1中的元素predMatrixL1[x][y]的值，是参考图像队列1中参考索引为RefIdxL1的1/16精度亮度样本矩阵中位置为(((xE+x)<<4)+mv0E1_x，(yE+y)<<4)+mv0E1_y)))的样本值，色度预测样本矩阵predMatrixL1中的元素predMatrixL1[x][y]的值是参考图像队列1中参考索引为RefIdxL1的1/32精度色度样本矩阵中位置为(((xE+2x)<<4)+MvC_x，(yE+2y)<<4)+MvC_y)))的样本值。其中，x1＝((xE+2x)>>3)<<3，y1＝((yE+2y)>>3)<<3，mv1E1是MvArrayL1运动矢量集合在(x1，y1)位置的4x4单元的L1运动矢量，mv2E1是MvArrayL1运动矢量集合在(x1+4，y1)位置的4x4单元的L1运动矢量，mv3E1是MvArrayL1运动矢量集合在(x1，y1+4)位置的4x4单元的L1运动矢量，mv4E1是MvArrayL1运动矢量集合在(x1+4，y1+4)位置的4x4单元的L1运动矢量。If the prediction reference mode of the current prediction unit is 'PRED_List1' and the value of AffineSubblockSizeFlag is 0, mv0E1 is the L1 motion vector of the 4x4 unit with the MvArrayL1 motion vector set at (xE+x, yE+y) position. The value of the element predMatrixL1[x][y] in the luma prediction sample matrix predMatrixL1 is the position in the 1/16-precision luma sample matrix with the reference index RefIdxL1 in the reference image queue 1 (((xE+x)<<4) +mv0E1_x, the sample value of (yE+y)<<4)+mv0E1_y))), the value of the element predMatrixL1[x][y] in the chrominance prediction sample matrix predMatrixL1 is the reference index in the reference image queue 1 RefIdxL1 The sample value at position (((xE+2x)<<4)+MvC_x, (yE+2y)<<4)+MvC_y))) in the 1/32 precision chroma sample matrix. Among them, x1=((xE+2x)>>3)<<3, y1=((yE+2y)>>3)<<3, mv1E1 is the 4x4 unit of MvArrayL1 motion vector set at (x1, y1) position , mv2E1 is the L1 motion vector of the 4x4 unit of the MvArrayL1 motion vector set at the (x1+4, y1) position, mv3E1 is the MvArrayL1 motion vector set at the (x1, y1+4) position of the 4x4 unit L1 motion vector of the unit , mv4E1 is the L1 motion vector of the 4x4 unit of the MvArrayL1 motion vector set at the (x1+4, y1+4) position.

MvC_x＝(mv1E1_x+mv2E1_x+mv3E1_x+mv4E1_x+2)>>2MvC_x=(mv1E1_x+mv2E1_x+mv3E1_x+mv4E1_x+2)>>2

MvC_y＝(mv1E1_y+mv2E1_y+mv3E1_y+mv4E1_y+2)>>2MvC_y=(mv1E1_y+mv2E1_y+mv3E1_y+mv4E1_y+2)>>2

如果当前预测单元的预测参考模式是‘PRED_List1’且AffineSubblockSizeFlag的值为1，mv0E1是MvArrayL1运动矢量集合在(xE+x，yE+y)位置的、8x8单元的L1运动矢量。亮度预测样本矩阵predMatrixL1中的元素predMatrixL1[x][y]的值，是参考图像队列1中参考索引为RefIdxL1的1/16精度亮度样本矩阵中位置为(((xE+x)<<4)+mv0E1_x，(yE+y)<<4)+mv0E1_y)))的样本值，色度预测样本矩阵predMatrixL1中的元素predMatrixL1[x][y]的值是参考图像队列1中参考索引为RefIdxL1的1/32精度色度样本矩阵中位置为(((xE+2x)<<4)+MvC_x，(yE+2y)<<4)+MvC_y)))的样本值。其中MvC_x等于mv0E1_x，MvC_y等于mv0E1。If the prediction reference mode of the current prediction unit is 'PRED_List1' and the value of AffineSubblockSizeFlag is 1, mv0E1 is the L1 motion vector of the 8x8 unit with the MvArrayL1 motion vector set at (xE+x, yE+y) position. The value of the element predMatrixL1[x][y] in the luma prediction sample matrix predMatrixL1 is the position in the 1/16-precision luma sample matrix with the reference index RefIdxL1 in the reference image queue 1 (((xE+x)<<4) +mv0E1_x, the sample value of (yE+y)<<4)+mv0E1_y))), the value of the element predMatrixL1[x][y] in the chrominance prediction sample matrix predMatrixL1 is the reference index in the reference image queue 1 RefIdxL1 The sample value at position (((xE+2x)<<4)+MvC_x, (yE+2y)<<4)+MvC_y))) in the 1/32 precision chroma sample matrix. where MvC_x is equal to mv0E1_x and MvC_y is equal to mv0E1.

如果当前预测单元的预测模式是‘PRED_List01’，mv0E0是MvArrayL0运动矢量集合在(xE+x，yE+y)位置的8x8单元的L0运动矢量，mv0E1是MvArrayL1运动矢量集合在(x，y)位置的、8x8单元的L1运动矢量。亮度预测样本矩阵predMatrixL0中的元素predMatrixL0[x][y]的值是参考图像队列0中参考索引为RefIdxL0的1/16精度亮度样本矩阵中位置为(((xE+x)<<4)+mv0E0_x，(yE+y)<<4)+mv0E0_y)))的样本值，色度预测样本矩阵predMatrixL0中的元素predMatrixL0[x][y]的值是参考图像队列0中参考索引为RefIdxL0的1/32精度色度样本矩阵中位置为(((xE+2x)<<4)+MvC0_x，(yE+2y)<<4)+MvC0_y)))的样本值，亮度预测样本矩阵predMatrixL1中的元素predMatrixL1[x][y]的值是参考图像队列1中参考索引为RefIdxL1的1/16精度亮度样本矩阵中位置为((((xE+x)<<4)+mv0E1_x，(yE+y)<<4)+mv0E1_y))))的样本值，色度预测样本矩阵predMatrixL1中的元素predMatrixL1[x][y]的值是参考图像队列1中参考索引为RefIdxL1的1/32精度色度样本矩阵中位置为(((xE+2x)<<4)+MvC1_x，(yE+2y)<<4)+MvC1_y)))的样本值。其中MvC0_x等于mv0E0_x，MvC0_y等于mv0E0_y，MvC1_x等于mv0E1_x，MvC1_y等于mv0E1_y。If the prediction mode of the current prediction unit is 'PRED_List01', mv0E0 is the L0 motion vector of the 8x8 unit with the MvArrayL0 motion vector set at (xE+x, yE+y) position, mv0E1 is the MvArrayL1 motion vector set at the (x, y) position , an L1 motion vector of 8x8 cells. The value of the element predMatrixL0[x][y] in the luma prediction sample matrix predMatrixL0 is the position in the 1/16-precision luma sample matrix with the reference index RefIdxL0 in the reference image queue 0 (((xE+x)<<4)+ mv0E0_x, the sample value of (yE+y)<<4)+mv0E0_y))), the value of the element predMatrixL0[x][y] in the chrominance prediction sample matrix predMatrixL0 is 1 whose reference index is RefIdxL0 in the reference image queue 0 The sample value at the position of (((xE+2x)<<4)+MvC0_x, (yE+2y)<<4)+MvC0_y))) in the /32-precision chroma sample matrix, the element in the luma prediction sample matrix predMatrixL1 The value of predMatrixL1[x][y] is the position in the 1/16-precision luma sample matrix with reference index RefIdxL1 in reference image queue 1 ((((xE+x)<<4)+mv0E1_x, (yE+y) <<4)+mv0E1_y)))) sample value, the value of the element predMatrixL1[x][y] in the chroma prediction sample matrix predMatrixL1 is the 1/32 precision chroma sample with the reference index RefIdxL1 in the reference image queue 1 The sample value at position (((xE+2x)<<4)+MvC1_x, (yE+2y)<<4)+MvC1_y))) in the matrix. Where MvC0_x is equal to mv0E0_x, MvC0_y is equal to mv0E0_y, MvC1_x is equal to mv0E1_x, and MvC1_y is equal to mv0E1_y.

其中，参考图像的亮度1/16精度样本矩阵和色度1/32精度样本矩阵中各个位置的元素值通过如下的仿射亮度样本插值过程和仿射色度样本插值过程所定义的插值方法得到。参考图像外的整数样本应使用该图像内距离该样本最近的整数样本(边缘或角样本)代替，即运动矢量能指向参考图像外的样本。Among them, the element values of each position in the luminance 1/16 precision sample matrix and the chrominance 1/32 precision sample matrix of the reference image are obtained by the interpolation method defined by the following affine luminance sample interpolation process and affine chrominance sample interpolation process. . Integer samples outside the reference image should be replaced with the nearest integer samples (edge or corner samples) in the image to the sample, that is, the motion vector can point to samples outside the reference image.

具体地，仿射亮度样本插值过程如下：Specifically, the affine luminance sample interpolation process is as follows:

A，B，C，D是相邻整像素样本，dx与dy是整像素样本A周边分像素样本a(dx，dy)与A的水平和垂直距离，dx等于fx&15，dy等于fy&15，其中(fx，fy)是该分像素样本在1/16精度的亮度样本矩阵中的坐标。整像素Ax，y的周边有的255个分像素样本ax，y(dx，dy)。A, B, C, D are adjacent integer pixel samples, dx and dy are the horizontal and vertical distances between sub-pixel samples a(dx, dy) and A around integer pixel sample A, dx is equal to fx&15, dy is equal to fy&15, where ( fx, fy) are the coordinates of the sub-pixel sample in the 1/16 precision luminance sample matrix. There are 255 sub-pixel samples ax, y (dx, dy) around the whole pixel Ax, y.

具体地，样本位置ax，0(x＝1～15)由水平方向上距离插值点最近得8个整数值滤波得到，预测值的获取方式如下：Specifically, the sample position ax, 0 (x=1-15) is obtained by filtering the 8 integer values closest to the interpolation point in the horizontal direction, and the method of obtaining the predicted value is as follows:

ax，0＝Clip1((fL[x][0]×A-3，0+fL[x][1]×A-2，0+fL[x][2]×A-1，0+fL[x][3]×A0，0+fL[x][4]×A1，0+fL[x][5]×A2，0+fL[x][6]×A3，0+fL[x][7]×A4，0+32)>>6)。ax,0=Clip1((fL[x][0]×A-3,0+fL[x][1]×A-2,0+fL[x][2]×A-1,0+fL [x][3]×A0, 0+fL[x][4]×A1, 0+fL[x][5]×A2, 0+fL[x][6]×A3, 0+fL[x ][7]×A4, 0+32)>>6).

具体地，样本位置a0，y(y＝1～15)由垂直方向上距离插值点最近得8个整数值滤波得到，预测值的获取方式如下：Specifically, the sample positions a0, y (y=1-15) are obtained by filtering the 8 integer values closest to the interpolation point in the vertical direction, and the predicted values are obtained in the following manner:

a0，y＝Clip1((fL[y][0]×A0，-3+fL[y][1]×A-2，0+fL[y][2]×A-1，0+fL[y][3]×A0，0+fL[y][4]×A1，0+fL[y][5]×A2，0+fL[y][6]×A3，0+fL[y][7]×A-4，0+32)>>6)。a0,y=Clip1((fL[y][0]×A0,-3+fL[y][1]×A-2,0+fL[y][2]×A-1,0+fL[ y][3]×A0, 0+fL[y][4]×A1, 0+fL[y][5]×A2, 0+fL[y][6]×A3, 0+fL[y] [7]×A-4, 0+32)>>6).

具体地，样本位置ax，y(x＝1～15，y＝1～15)的预测值的获取方式如下：Specifically, the acquisition method of the predicted value of the sample position ax, y (x=1-15, y=1-15) is as follows:

ax，y＝Clip1((fL[y][0]×a'x，y-3+fL[y][1]×a'x，y-2+fL[y][2]×a'x，y-1+fL[y][3]×a'x，y+fL[y][4]×a'x，y+1+fL[y][5]×a'x，y+2+fL[y][6]×a'x，y+3+fL[y][7]×a'x，y+4+(1<<(19-BitDepth)))>>(20-BitDepth))。ax, y=Clip1((fL[y][0]×a'x, y-3+fL[y][1]×a'x, y-2+fL[y][2]×a'x , y-1+fL[y][3]×a'x, y+fL[y][4]×a'x, y+1+fL[y][5]×a'x, y+2 +fL[y][6]×a'x, y+3+fL[y][7]×a'x, y+4+(1<<(19-BitDepth)))>>(20-BitDepth )).

其中：in:

a'x，y＝(fL[x][0]×A-3，y+fL[x][1]×A-2，y+fL[x][2]×A-1，y+fL[x][3]×A0，y+fL[x][4]×A1，y+fL[x][5]×A2，y+fL[x][6]×A3，y+fL[x][7]×A4，y+((1<<(BitDepth-8))>>1))>>(BitDepth-8)。a'x, y=(fL[x][0]×A-3, y+fL[x][1]×A-2, y+fL[x][2]×A-1, y+fL [x][3]×A0, y+fL[x][4]×A1, y+fL[x][5]×A2, y+fL[x][6]×A3, y+fL[x ][7]×A4, y+((1<<(BitDepth-8))>>1))>>(BitDepth-8).

亮度插值滤波器系数如表1所示：The luminance interpolation filter coefficients are shown in Table 1:

表1Table 1

具体地，仿射色度样本插值过程如下：Specifically, the affine chroma sample interpolation process is as follows:

A，B，C，D是相邻整像素样本，dx与dy是整像素样本A周边分像素样本a(dx，dy)与A的水平和垂直距离，dx等于fx&31，dy等于fy&31，其中(fx，fy)是该分像素样本在1/32精度的色度样本矩阵中的坐标。整像素Ax，y的周边有1023个分像素样本ax，y(dx，dy)。A, B, C, D are adjacent integer pixel samples, dx and dy are the horizontal and vertical distances between sub-pixel samples a(dx, dy) and A around integer pixel sample A, dx is equal to fx&31, dy is equal to fy&31, where ( fx, fy) are the coordinates of the sub-pixel sample in the 1/32-precision chroma sample matrix. There are 1023 sub-pixel samples ax,y(dx,dy) around the whole pixel Ax,y.

具体地，对于dx等于0或dy等于0的分像素点，可直接用色度整像素插值得到，对于dx不等于0且dy不等于0的点，使用整像素行(dy等于0)上的分像素进行计算：Specifically, for the sub-pixel points where dx is equal to 0 or dy is equal to 0, it can be obtained directly by chrominance integer pixel interpolation. Calculate by pixel:

if(dx＝＝0){if(dx==0){

ax，y(0，dy)＝Clip3(0，(1<<BitDepth)-1，(fC[dy][0]×Ax，y-1+fC[dy][1]×Ax，y+fC[dy][2]×Ax，y+1+fC[dy][3]×Ax，y+2+32)>>6)ax,y(0,dy)=Clip3(0,(1<<BitDepth)-1,(fC[dy][0]×Ax,y-1+fC[dy][1]×Ax,y+fC [dy][2]×Ax, y+1+fC[dy][3]×Ax, y+2+32)>>6)

}}

else if(dy＝＝0){else if(dy==0){

ax，y(dx，0)＝Clip3(0，(1<<BitDepth)-1，(fC[dx][0]×Ax-1，y+fC[dx][1]×Ax，y+fC[dx][2]×Ax+1，y+fC[dx][3]×Ax+2，y+32)>>6)ax,y(dx,0)=Clip3(0,(1<<BitDepth)-1,(fC[dx][0]×Ax-1,y+fC[dx][1]×Ax,y+fC [dx][2]×Ax+1, y+fC[dx][3]×Ax+2, y+32)>>6)

}}

else{else{

ax，y(dx，dy)＝Clip3(0，(1<<BitDepth)-1，(C[dy][0]×a'x，y-1(dx，0)+C[dy][1]×a'x，y(dx，0)+C[dy][2]×a'x，y+1(dx，0)+C[dy][3]×a'x，y+2(dx，0)+(1<<(19-BitDepth)))>>(20-BitDepth))ax,y(dx,dy)=Clip3(0,(1<<BitDepth)-1,(C[dy][0]×a'x,y-1(dx,0)+C[dy][1 ]×a'x,y(dx,0)+C[dy][2]×a'x,y+1(dx,0)+C[dy][3]×a'x,y+2( dx, 0)+(1<<(19-BitDepth)))>>(20-BitDepth))

}}

其中，a'x，y(dx，0)是整像素行上的分像素的临时值，定义为：a'x，y(dx，0)＝(fC[dx][0]×Ax-1，y+fC[dx][1]×Ax，y+fC[dx][2]×Ax+1，y+fC[dx][3]×Ax+2，y+((1<<(BitDepth-8))>>1))>>(BitDepth-8)。Among them, a'x,y(dx,0) is the temporary value of the sub-pixel on the whole pixel row, which is defined as: a'x,y(dx,0)=(fC[dx][0]×Ax-1 , y+fC[dx][1]×Ax, y+fC[dx][2]×Ax+1, y+fC[dx][3]×Ax+2, y+((1<<(BitDepth- 8))>>1))>>(BitDepth-8).

色度插值滤波器系数如表2所示：The chroma interpolation filter coefficients are shown in Table 2:

表2Table 2

常见的仿射预测的方法可以包括以下步骤：Common affine prediction methods can include the following steps:

步骤101、确定控制点的运动矢量。Step 101: Determine the motion vector of the control point.

步骤102、根据控制点的运动矢量确定子块的运动矢量。Step 102: Determine the motion vector of the sub-block according to the motion vector of the control point.

步骤103、根据子块的运动矢量对子块进行预测。Step 103: Predict the sub-block according to the motion vector of the sub-block.

目前，在通过PROF改进基于块的仿射预测的预测值时，具体可以包括以下步骤：At present, when improving the predicted value of block-based affine prediction through PROF, the following steps can be specifically included:

步骤104、根据控制点的运动矢量与子块的运动矢量确定子块内每个位置与子块的运动矢量偏差。Step 104: Determine the motion vector deviation between each position in the sub-block and the sub-block according to the motion vector of the control point and the motion vector of the sub-block.

步骤105、使用基于子块的预测值导出每一个位置计算水平和垂直方向的梯度。Step 105 , using the sub-block-based prediction value to derive each position to calculate the gradients in the horizontal and vertical directions.

步骤106、利用光流原理，根据每一个位置的运动矢量偏差和水平、垂直方向的梯度，计算每一个位置的预测值的偏差值。Step 106: Using the principle of optical flow, calculate the deviation value of the predicted value of each position according to the motion vector deviation of each position and the gradients in the horizontal and vertical directions.

步骤107、对每一个位置基于子块的预测值加上预测值的偏差值，得到修正后的预测值。Step 107: Add the deviation value of the predicted value to the predicted value based on the sub-block for each position to obtain the revised predicted value.

目前，在通过PROF改进基于块的仿射预测的预测值时，具体还可以包括以下步骤：At present, when improving the predicted value of block-based affine prediction through PROF, the following steps can be specifically included:

步骤109、根据控制点的运动矢量确定子块的运动矢量及子块内每个位置与子块的运动矢量的偏差。Step 109: Determine the motion vector of the sub-block and the deviation between each position in the sub-block and the motion vector of the sub-block according to the motion vector of the control point.

使用光流原理的预测修正PROF可以使用光流原理对基于子块的仿射预测进行修正，提高了压缩性能。然而，PROF的应用是基于子块内的像素点的运动矢量与子块运动矢量的偏差非常小的情况，也就是说，子块内的像素点的运动矢量与子块运动矢量的偏差非常小的情况下使用PROF的光流计算方法是有效的，但是，由于PROF依赖于基准位置的水平方向和垂直方向的梯度，在实际位置离基准位置较远的情况下，基准位置的水平和垂直方向的梯度并不能真实的反映基准位置和实际位置之间的水平和垂直方向的梯度，因此，在子块内的像素点的运动矢量与子块运动矢量的偏差较大的情况下，该方法就不是特别有效了。Prediction Correction Using Optical Flow Principle PROF can use optical flow principle to correct sub-block-based affine prediction, which improves the compression performance. However, the application of PROF is based on the fact that the deviation between the motion vector of the pixel in the sub-block and the motion vector of the sub-block is very small, that is to say, the deviation of the motion vector of the pixel in the sub-block and the motion vector of the sub-block is very small. The optical flow calculation method using PROF is effective in the case of The gradient does not truly reflect the gradient in the horizontal and vertical directions between the reference position and the actual position. Therefore, when the deviation between the motion vector of the pixel in the sub-block and the motion vector of the sub-block is large, this method will Not particularly effective anymore.

由此可见，现有的PROF修正预测值方法并不严谨，在对仿射预测进行改善时，并不能很好的适用于全部场景，编码性能有待提升。It can be seen that the existing PROF correction prediction method is not rigorous, and when improving affine prediction, it cannot be well applied to all scenarios, and the coding performance needs to be improved.

为了解决现有技术中存在的缺陷，在本申请的实施例中，可以在基于子块的预测之后，针对运动矢量与子块的运动矢量有偏差的像素点，可以在基于子块的第一预测值的基础上，利用可分离滤波器进行基于点的二次预测，获得第二预测值。其中，可分离滤波器可以分离为水平方向的一维滤波器和垂直方向的一维滤波器，滤波后可以获得该像素点对应的新的预测值，最终获得子块修正后的第二预测值，本申请提出的帧间预测方法能够很好的适用于全部场景，大大提升编码性能，从而提高了编解码效率。In order to solve the defects in the prior art, in the embodiments of the present application, after the sub-block-based prediction, for the pixels whose motion vectors are deviated from the motion vector of the sub-block, the first sub-block-based On the basis of the predicted value, use a separable filter to perform point-based secondary prediction to obtain a second predicted value. Among them, the separable filter can be separated into a one-dimensional filter in the horizontal direction and a one-dimensional filter in the vertical direction. After filtering, a new predicted value corresponding to the pixel point can be obtained, and finally a second predicted value after sub-block correction can be obtained. , the inter-frame prediction method proposed in this application can be well applied to all scenarios, greatly improving the encoding performance, thereby improving the encoding and decoding efficiency.

应理解，本申请实施例提供一种视频编码***，图5为本申请实施例提供的一种视频编码***的组成框图示意图，如图5所示，该视频编码***11可以包括：变换单元111、量化单元112、模式选择和编码控制逻辑单元113、帧内预测单元114、帧间预测单元115(包括：运动补偿和运动估计)、反量化单元116、反变换单元117、环路滤波单元118、编码单元119和解码图像缓存单元110；针对输入的原始视频信号，通过编码树块(Coding Tree Unit，CTU)的划分可以得到一个视频重建块，通过模式选择和编码控制逻辑单元113确定编码模式，然后，对经过帧内或帧间预测后得到的残差像素信息，通过变换单元111、量化单元112对该视频重建块进行变换，包括将残差信息从像素域变换到变换域，并对所得的变换系数进行量化，用以进一步减少比特率；帧内预测单元114用于对该视频重建块进行帧内预测；其中，帧内预测单元114用于确定该视频重建块的最优帧内预测模式(即目标预测模式)；帧间预测单元115用于执行所接收的视频重建块相对于一或多个参考帧中的一或多个块的帧间预测编码，以提供时间预测信息；其中吗，运动估计为产生运动向量的过程，运动向量可以估计该视频重建块的运动，然后，运动补偿基于由运动估计所确定的运动向量执行运动补偿；在确定帧间预测模式之后，帧间预测单元115还用于将所选择的帧间预测数据提供到编码单元119，而且，将所计算确定的运动向量数据也发送到编码单元119；此外，反量化单元116和反变换单元117用于该视频重建块的重构建，在像素域中重构建残差块，该重构建残差块通过环路滤波单元118去除方块效应伪影，然后，将该重构残差块添加到解码图像缓存单元110的帧中的一个预测性块，用以产生经重构建的视频重建块；编码单元119是用于编码各种编码参数及量化后的变换系数。而解码图像缓存单元110用于存放重构建的视频重建块，用于预测参考。随着视频图像编码的进行，会不断生成新的重构建的视频重建块，这些重构建的视频重建块都会被存放在解码图像缓存单元110中。It should be understood that an embodiment of the present application provides a video encoding system, and FIG. 5 is a schematic block diagram of the composition of a video encoding system provided by an embodiment of the present application. As shown in FIG. 5 , the video encoding system 11 may include: a transformation unit 111 , quantization unit 112, mode selection and coding control logic unit 113, intra prediction unit 114, inter prediction unit 115 (including motion compensation and motion estimation), inverse quantization unit 116, inverse transform unit 117, loop filtering unit 118 , encoding unit 119 and decoded image buffering unit 110; for the original video signal of the input, a video reconstruction block can be obtained by the division of the coding tree block (Coding Tree Unit, CTU), and the encoding mode is determined by mode selection and encoding control logic unit 113 , and then, for the residual pixel information obtained after intra-frame or inter-frame prediction, the video reconstruction block is transformed by the transform unit 111 and the quantization unit 112, including transforming the residual information from the pixel domain to the transform domain, and converting the residual information to the transform domain. The obtained transform coefficients are quantized to further reduce the bit rate; the intra-frame prediction unit 114 is used to perform intra-frame prediction on the video reconstruction block; wherein, the intra-frame prediction unit 114 is used to determine the optimal intra-frame frame of the video reconstruction block prediction mode (ie, target prediction mode); inter prediction unit 115 is used to perform inter prediction encoding of the received video reconstruction block relative to one or more blocks in one or more reference frames to provide temporal prediction information; Among them, motion estimation is the process of generating a motion vector, which can estimate the motion of the video reconstruction block, and then, motion compensation performs motion compensation based on the motion vector determined by the motion estimation; after determining the inter prediction mode, the inter The prediction unit 115 is also used to supply the selected inter-frame prediction data to the coding unit 119, and also to send the calculated motion vector data to the coding unit 119; in addition, the inverse quantization unit 116 and the inverse transform unit 117 are used for The reconstruction of the video reconstruction block reconstructs a residual block in the pixel domain, the reconstructed residual block is passed through the loop filter unit 118 to remove the blocking artifacts, and then the reconstructed residual block is added to the decoded image buffer. A predictive block in the frame of unit 110 is used to generate a reconstructed video reconstruction block; encoding unit 119 is used to encode various encoding parameters and quantized transform coefficients. The decoded image buffer unit 110 is used for storing reconstructed video reconstruction blocks for prediction reference. As the video image encoding proceeds, new reconstructed video reconstruction blocks are continuously generated, and these reconstructed video reconstruction blocks are all stored in the decoded image buffer unit 110 .

本申请实施例还提供一种视频解码***，图6为本申请实施例提供的一种视频解码***的组成框图示意图，如图6所示，该视频解码***12可以包括：解码单元121、反变换单元127，与反量化单元122、帧内预测单元123、运动补偿单元124、环路滤波单元125和解码图像缓存单元126单元；输入的视频信号经过视频编码***11进行编码处理之后，输出该视频信号的码流；该码流输入视频解码***12中，首先经过解码单元121，用于得到解码后的变换系数；针对该变换系数通过反变换单元127与反量化单元122进行处理，以便在像素域中产生残差块；帧内预测单元123可用于基于所确定的帧内预测方向和来自当前帧或图片的先前经解码块的数据而产生当前视频解码块的预测数据；运动补偿单元124是通过剖析运动向量和其他关联语法元素来确定用于视频解码块的预测信息，并使用该预测信息以产生正被解码的视频解码块的预测性块；通过对来自反变换单元127与反量化单元122的残差块与由帧内预测单元123或运动补偿单元124产生的对应预测性块进行求和，而形成解码的视频块；该解码的视频信号通过环路滤波单元125以便去除方块效应伪影，可以改善视频质量；然后将经解码的视频块存储于解码图像缓存单元126中，解码图像缓存单元126存储用于后续帧内预测或运动补偿的参考图像，同时也用于视频信号的输出，得到所恢复的原始视频信号。An embodiment of the present application further provides a video decoding system. FIG. 6 is a schematic block diagram of the composition of a video decoding system provided by an embodiment of the present application. As shown in FIG. 6 , the video decoding system 12 may include: a decoding unit 121, an inverse The transformation unit 127, together with the inverse quantization unit 122, the intra-frame prediction unit 123, the motion compensation unit 124, the loop filtering unit 125 and the decoded image buffer unit 126; The code stream of the video signal; the code stream is input into the video decoding system 12, and firstly passes through the decoding unit 121 to obtain the decoded transform coefficient; the transform coefficient is processed by the inverse transform unit 127 and the inverse quantization unit 122, so that the A residual block is generated in the pixel domain; intra-prediction unit 123 may be used to generate prediction data for the current video decoding block based on the determined intra-prediction direction and data from previously decoded blocks of the current frame or picture; motion compensation unit 124 is by parsing motion vectors and other associated syntax elements to determine prediction information for the video decoding block, and using the prediction information to generate the predictive block of the video decoding block being decoded; The residual block of unit 122 is summed with the corresponding predictive blocks produced by intra-prediction unit 123 or motion compensation unit 124 to form a decoded video block; the decoded video signal is passed through in-loop filtering unit 125 in order to remove blockiness Artifacts, which can improve video quality; the decoded video blocks are then stored in decoded image buffer unit 126, which stores reference images for subsequent intra prediction or motion compensation, as well as for video signal output to get the restored original video signal.

本申请实施例提供的一种帧间预测方法主要作用于视频编码***11的帧间预测单元215和视频解码***12的帧间预测单元，即运动补偿单元124；也就是说，如果在视频编码***11能够通过本申请实施例提供的帧间预测方法得到一个较好的预测效果，那么，对应地，在视频解码***12，也能够改善视频解码恢复质量。An inter-frame prediction method provided in this embodiment of the present application mainly acts on the inter-frame prediction unit 215 of the video coding system 11 and the inter-frame prediction unit of the video decoding system 12, that is, the motion compensation unit 124; The system 11 can obtain a better prediction effect through the inter-frame prediction method provided in the embodiment of the present application, and correspondingly, the video decoding system 12 can also improve the quality of video decoding and restoration.

基于此，下面结合附图和实施例对本申请的技术方案进一步详细阐述。在进行详细阐述之前，需要说明的是，说明书通篇中提到的“第一”、“第二”、“第三”等，仅仅是为了区分不同的特征，不具有限定优先级、先后顺序、大小关系等功能。Based on this, the technical solutions of the present application are further elaborated below with reference to the accompanying drawings and embodiments. Before going into detail, it should be noted that the "first", "second", "third", etc. mentioned throughout the specification are only for distinguishing different features, and do not have a limited priority or sequence. , size relationship and other functions.

需要说明的是，本实施例以AVS3标准为基础进行示例性说明，本申请提出的帧间预测方法同样可以适用于VVC等其他编码标准技术，不申请对此不作具体限定。It should be noted that this embodiment is exemplified based on the AVS3 standard, and the inter-frame prediction method proposed in this application can also be applied to other coding standard technologies such as VVC, which is not specifically limited in this application.

本申请实施例提供一种帧间预测方法，该方法应用于视频解码设备，即解码器。该方法所实现的功能可以通过解码器中的第一处理器调用计算机程序来实现，当然计算机程序可以保存在第一存储器中，可见，该解码器至少包括第一处理器和第一存储器。An embodiment of the present application provides an inter-frame prediction method, which is applied to a video decoding device, that is, a decoder. The functions implemented by the method can be implemented by the first processor in the decoder calling a computer program, and of course the computer program can be stored in the first memory. It can be seen that the decoder includes at least a first processor and a first memory.

进一步地，在本申请的实施例中，图7为帧间预测方法的实现流程示意图一，如图7所示，解码器进行帧间预测的方法可以包括以下步骤：Further, in the embodiment of the present application, FIG. 7 is a schematic diagram 1 of the implementation flow of the inter-frame prediction method. As shown in FIG. 7 , the method for performing the inter-frame prediction by the decoder may include the following steps:

步骤201、解析码流，获取当前块的预测模式参数。Step 201: Parse the code stream to obtain the prediction mode parameter of the current block.

在本申请的实施例中，解码器可以先解析二进制码流，从而获得当前块的预测模式参数。其中，预测模式参数可以用于对当前块所使用的预测模式进行确定。In the embodiment of the present application, the decoder may first parse the binary code stream to obtain the prediction mode parameter of the current block. The prediction mode parameter may be used to determine the prediction mode used by the current block.

需要说明的是，待解码图像可以划分为多个图像块，而当前待解码的图像块可以称为当前块(可以用CU表示)，与当前块相邻的图像块可以称为相邻块；即在待解码图像中，当前块与相邻块之间具有相邻关系。这里，每个当前块可以包括第一图像分量、第二图像分量和第三图像分量，也即当前块表示待解码图像中当前待进行第一图像分量、第二图像分量或者第三图像分量预测的图像块。It should be noted that the image to be decoded can be divided into multiple image blocks, and the current image block to be decoded can be called the current block (which can be represented by CU), and the image blocks adjacent to the current block can be called adjacent blocks; That is, in the image to be decoded, the current block and adjacent blocks have an adjacent relationship. Here, each current block may include a first image component, a second image component, and a third image component, that is, the current block indicates that the prediction of the first image component, the second image component or the third image component in the image to be decoded is currently performed image block.

其中，假定当前块进行第一图像分量预测，而且第一图像分量为亮度分量，即待预测图像分量为亮度分量，那么当前块也可以称为亮度块；或者，假定当前块进行第二图像分量预测，而且第二图像分量为色度分量，即待预测图像分量为色度分量，那么当前块也可以称为色度块。Wherein, it is assumed that the current block performs the first image component prediction, and the first image component is a luminance component, that is, the image component to be predicted is a luminance component, then the current block may also be called a luminance block; or, it is assumed that the current block performs the second image component prediction prediction, and the second image component is a chrominance component, that is, the image component to be predicted is a chrominance component, then the current block may also be called a chrominance block.

进一步地，在本申请的实施例中，预测模式参数不仅可以指示当前块采用的预测模式，还可以指示与该预测模式相关的参数。Further, in this embodiment of the present application, the prediction mode parameter may not only indicate the prediction mode adopted by the current block, but may also indicate parameters related to the prediction mode.

可以理解的是，在本申请的实施例中，预测模式可以包括有帧间预测模式、传统帧内预测模式以及非传统帧内预测模式等。It can be understood that, in the embodiment of the present application, the prediction mode may include an inter prediction mode, a traditional intra prediction mode, a non-traditional intra prediction mode, and the like.

也就是说，在编码侧，编码器可以选取最优的预测模式对当前块进行预编码，在这过程中就可以确定出当前块的预测模式，然后确定用于指示预测模式的预测模式参数，从而将相应的预测模式参数写入码流，由编码器传输到解码器。That is to say, on the encoding side, the encoder can select the optimal prediction mode to pre-encode the current block. In the process, the prediction mode of the current block can be determined, and then the prediction mode parameters used to indicate the prediction mode can be determined. Thereby, the corresponding prediction mode parameters are written into the code stream and transmitted from the encoder to the decoder.

相应地，在解码器侧，解码器通过解析码流便可以直接获取到当前块的预测模式参数，并根据解析获得的预测模式参数确定当前块所使用的预测模式，以及该预测模式对应的相关参数。Correspondingly, on the decoder side, the decoder can directly obtain the prediction mode parameter of the current block by parsing the code stream, and determine the prediction mode used by the current block according to the prediction mode parameter obtained by parsing, and the correlation corresponding to the prediction mode. parameter.

进一步地，在本申请的实施例中，解码器在解析获得预测模式参数之后，可以基于预测模式参数确定当前块是否使用帧间预测模式。Further, in the embodiment of the present application, after the decoder obtains the prediction mode parameter by parsing, it may determine whether the current block uses the inter prediction mode based on the prediction mode parameter.

步骤202、当预测模式参数指示使用帧间预测模式确定当前块的帧间预测值时，确定当前块的子块的第一预测值；其中，子块包括一个或多个像素点。 Step 202 , when the prediction mode parameter indicates that the inter prediction value of the current block is determined by using the inter prediction mode, determine the first prediction value of the sub-block of the current block, wherein the sub-block includes one or more pixels.

步骤203、对于子块内的一像素点，确定像素点的运动矢量与子块的运动矢量之间的第一运动矢量偏差。Step 203: For a pixel in the sub-block, determine the first motion vector deviation between the motion vector of the pixel and the motion vector of the sub-block.

在本申请的实，解码器在解析获得预测模式参数之后，如果解析获得的预测模式参数指示当前块使用帧间预测模式确定当前块的帧间预测值，那么解码器可以先确定出当前块的每一个子块的第一预测值，同时，还需要确定出像素点的运动矢量与子块的运动矢量之间的第一运动矢量偏差。其中，每一个子块均包括有一个或多个像素点(sample)，每一个像素点都对应有一个像素位置和一个像素值。In the present application, after the decoder obtains the prediction mode parameter by parsing, if the prediction mode parameter obtained by parsing indicates that the current block uses the inter prediction mode to determine the inter prediction value of the current block, the decoder can first determine the For the first predicted value of each sub-block, at the same time, it is also necessary to determine the first motion vector deviation between the motion vector of the pixel point and the motion vector of the sub-block. Wherein, each sub-block includes one or more pixel points (sample), and each pixel point corresponds to a pixel position and a pixel value.

示例性的，在本申请的实施例中，解码器在确定当前块的子块的第一预测值时，可以先确定当前块的子块的运动矢量，然后再基于该运动矢量确定出该子块对应的第一预测值。Exemplarily, in the embodiment of the present application, when determining the first predicted value of the sub-block of the current block, the decoder may first determine the motion vector of the sub-block of the current block, and then determine the sub-block based on the motion vector. The first predicted value corresponding to the block.

示例性的，在本申请的实施例中，解码器在确定当前块的子块的第一预测值时，也可以先确定子块与当前块之间的第二运动矢量偏差；然后再基于第二运动矢量偏差，确定子块的第一预测值。Exemplarily, in this embodiment of the present application, when determining the first predicted value of the sub-block of the current block, the decoder may first determine the second motion vector deviation between the sub-block and the current block; 2. The motion vector offset determines the first predicted value of the sub-block.

也就是说，在本申请中，解码器确定当前块的子块的第一预测值可能有多种方式，既可以在计算子块的运动矢量之后，利用运动矢量进一步计算第一预测值，也可以直接计算子块的第一预测值。例如，可以针对一个子块计算获得该子块与当前块(整个编码单元或预测单元)之间的运动矢量偏差(或者水平方向和垂直方向的偏差)，然后根据这个与当前块的运动矢量偏差确定该子块的第一预测值。进一步地，在获得该子块的第一预测值之后，该子块的运动矢量可以认为是当前块的运动矢量加上与该子块的运动矢量偏差。That is to say, in this application, the decoder may determine the first predicted value of the sub-block of the current block in several ways. After calculating the motion vector of the sub-block, the first predicted value can be further calculated by using the motion vector, or The first predicted value of the sub-block can be directly calculated. For example, the motion vector deviation (or the deviation in the horizontal direction and the vertical direction) between the sub-block and the current block (the entire coding unit or prediction unit) can be calculated and obtained for a sub-block, and then the motion vector deviation from the current block can be obtained according to this motion vector deviation. A first predicted value for the sub-block is determined. Further, after the first predicted value of the sub-block is obtained, the motion vector of the sub-block can be regarded as the motion vector of the current block plus a deviation from the motion vector of the sub-block.

可见，在一些场景中，即使不先进行运动矢量的计算，也可以确定出子块的第一预测值量。例如，双向光流，在AVS中简称BIO，在VVC中简称BDOF，利用双向光流计算子块与当前块的运动矢量偏差，然后利用光流原理根据梯度、运动矢量偏差以及当前的预测值得到新的预测值。It can be seen that, in some scenarios, even without first calculating the motion vector, the first predictor amount of the sub-block can be determined. For example, bidirectional optical flow, referred to as BIO in AVS and BDOF in VVC, uses bidirectional optical flow to calculate the motion vector deviation between the sub-block and the current block, and then uses the optical flow principle to obtain the gradient, motion vector deviation and the current predicted value. new predicted value.

需要说明的是，在本申请中，在一些场景中，针对一个块(编码块或预测块)，如4x4的块，作为一个整体进行计算，如运动矢量偏差的计算、预测值的修正等的计算，也可以按照本申请中的子块的思想来处理。It should be noted that in this application, in some scenarios, a block (coding block or prediction block), such as a 4x4 block, is calculated as a whole, such as the calculation of the motion vector deviation, the correction of the predicted value, etc. The calculation can also be processed according to the idea of the sub-block in this application.

进一步地，在本申请中，解码器在进行像素点的运动矢量与子块的运动矢量之间的第一运动矢量偏差的确定时，相应地，既可以基于计算获得的子块的运动矢量确定第一运动矢量偏差，也可以不进行子块的运动矢量的计算而直接确定第一运动矢量偏差。也就是说，解码器可以在不确定像素点的运动矢量和子块的运动矢量的情况下直接进行第一运动矢量偏差的计算。例如，解码器可以直接根据整个当前块的运动矢量和每个子块的运动矢量偏差，导出每个像素点的运动矢量与子块的运动矢量之间的第一运动矢量偏差。Further, in the present application, when the decoder determines the first motion vector deviation between the motion vector of the pixel point and the motion vector of the sub-block, correspondingly, it can be determined based on the motion vector of the sub-block obtained by calculation. For the first motion vector offset, the first motion vector offset may be directly determined without calculating the motion vector of the sub-block. That is to say, the decoder can directly calculate the first motion vector deviation without determining the motion vector of the pixel point and the motion vector of the sub-block. For example, the decoder can directly derive the first motion vector deviation between the motion vector of each pixel and the motion vector of the sub-block according to the motion vector of the entire current block and the motion vector deviation of each sub-block.

示例性的，图8为确定运动矢量偏差的示意图，如图8所示，如果子块sub_block0相对于整个当前块的运动矢量偏差是(dmv_x_sub0，dmv_y_sub0)，子块sub_block1相对于整个当前块的运动矢量偏差是(dmv_x_sub1，dmv_y_sub1)，子块sub_block2相对于整个当前块的运动矢量偏差是(dmv_x_sub2，dmv_y_sub2)，子块sub_block3相对于整个当前块的运动矢量偏差是(dmv_x_sub3，dmv_y_sub3)，那么sub_block0只中与sub_block1相邻的像素1与sub_block0的运动矢量偏差为((dmv_x_sub1-dmv_x_sub0)/2，(dmv_y_sub1-dmv_y_sub0)/2)，sub_block0中与sub_block1、sub_block2、sub_block3都相邻的像素2与sub_block0的运动矢量偏差为((dmv_x_sub1+dmv_x_sub2+dmv_x_sub3-3xdmv_x_sub0)/4，(dmv_y_sub1+dmv_y_sub2+dmv_y_sub3-3xdmv_y_sub0)/4)。Exemplarily, FIG. 8 is a schematic diagram of determining the motion vector deviation. As shown in FIG. 8 , if the motion vector deviation of the sub-block sub_block0 relative to the entire current block is (dmv_x_sub0, dmv_y_sub0), the motion of the sub-block sub_block1 relative to the entire current block is The vector deviation is (dmv_x_sub1, dmv_y_sub1), the motion vector deviation of sub-block sub_block2 relative to the entire current block is (dmv_x_sub2, dmv_y_sub2), and the motion vector deviation of sub-block sub_block3 relative to the entire current block is (dmv_x_sub3, dmv_y_sub3), then sub_block0 is only The motion vector deviation between the pixel 1 adjacent to sub_block1 and sub_block0 is ((dmv_x_sub1-dmv_x_sub0)/2, (dmv_y_sub1-dmv_y_sub0)/2), and the pixel 2 adjacent to sub_block1, sub_block2, sub_block3 in sub_block0 and sub_block0 The motion vector offset is ((dmv_x_sub1+dmv_x_sub2+dmv_x_sub3-3xdmv_x_sub0)/4, (dmv_y_sub1+dmv_y_sub2+dmv_y_sub3-3xdmv_y_sub0)/4).

需要说明的是，在本申请的实施例中，当前块为当前帧中待解码的图像块，当前帧以图像块的形式按一定顺序依次进行解码，该当前块为当前帧内按该顺序下一时刻待解码的图像块。当前块可具有多种规格尺寸，例如16×16、32×32或32×16等规格，其中数字表示当前块上像素点的行数和列数。It should be noted that, in the embodiments of the present application, the current block is the image block to be decoded in the current frame, the current frame is decoded in the form of image blocks in a certain order, and the current block is the image block in the current frame in this order. The image block to be decoded at a time. The current block may have various sizes, such as 16×16, 32×32, or 32×16, where the numbers represent the number of rows and columns of pixels on the current block.

进一步对，在本申请的实施例中，当前块可以划分为多个子块，其中，每一个子块的尺寸大小都是相同的，子块为较小规格的像素点集合。子块的尺寸可以为8x8或4x4。Further, in the embodiments of the present application, the current block may be divided into a plurality of sub-blocks, wherein the size of each sub-block is the same, and the sub-block is a set of pixel points of a smaller specification. The size of the sub-block can be 8x8 or 4x4.

示例性的，在本申请中，当前块的尺寸为16×16，可以划分为4个尺寸均为8x8的子块。Exemplarily, in this application, the size of the current block is 16×16, which can be divided into 4 sub-blocks each with a size of 8×8.

可以理解的是，在本申请的实施例中，在解码器解析码流获取到预测模式参数指示使用帧间预测模式确定当前块的帧间预测值的情况下，就可以继续采用本申请实施例所提供的帧间预测方法。It can be understood that, in the embodiments of the present application, when the decoder parses the code stream and obtains the prediction mode parameter indicating that the inter-frame prediction mode is used to determine the inter-frame prediction value of the current block, the embodiment of the present application can continue to be used. The provided inter prediction method.

在本申请的实施例中，进一步地，当预测模式参数指示使用帧间预测模式确定当前块的帧间预测值时，解码器确定当前块的子块的运动矢量的方法可以包括以下步骤：In the embodiment of the present application, further, when the prediction mode parameter indicates that the inter prediction value of the current block is determined using the inter prediction mode, the method for the decoder to determine the motion vector of the sub-block of the current block may include the following steps:

步骤301、解析码流，获取当前块的仿射模式参数和预测参考模式。Step 301: Parse the code stream to obtain the affine mode parameter and prediction reference mode of the current block.

步骤302、当仿射模式参数指示使用仿射模式时，确定控制点模式和子块尺寸参数。Step 302: When the affine mode parameter indicates to use the affine mode, determine the control point mode and the sub-block size parameter.

步骤303、根据预测参考模式、控制点模式以及子块尺寸参数，确定运动矢量。Step 303: Determine a motion vector according to the prediction reference mode, the control point mode and the sub-block size parameter.

在本申请的实施例中，解码器在解析获得预测模式参数之后，如果解析获得的预测模式参数指示当前块使用帧间预测模式确定当前块的帧间预测值，那么解码器可以通过解析码流，获得仿射模式参数和预测参考模式。In this embodiment of the present application, after the decoder obtains the prediction mode parameter by parsing, if the prediction mode parameter obtained by parsing indicates that the current block uses the inter prediction mode to determine the inter prediction value of the current block, the decoder can parse the code stream by , to obtain the affine mode parameters and the prediction reference mode.

需要说明的是，在本申请的实施例中，仿射模式参数用于对是否使用仿射模式进行指示。具体地，仿射模式参数可以为仿射运动补偿允许标志affine_enable_flag，解码器通过仿射模式参数的取值的确定，可以进一步确定是否使用仿射模式。It should be noted that, in the embodiments of the present application, the affine mode parameter is used to indicate whether to use the affine mode. Specifically, the affine mode parameter may be the affine motion compensation enable flag affine_enable_flag, and the decoder may further determine whether to use the affine mode by determining the value of the affine mode parameter.

也就是说，在本申请中，仿射模式参数可以为一个二值变量。若仿射模式参数的取值为1，则指示使用仿射模式；若仿射模式参数的取值为0，则指示不使用仿射模式。That is, in this application, the affine mode parameter may be a binary variable. If the value of the affine mode parameter is 1, it indicates that the affine mode is used; if the value of the affine mode parameter is 0, it indicates that the affine mode is not used.

可以理解的是，在本申请中，解码器解析码流，如果未解析得到仿射模式参数，那么也可以理解为指示不使用仿射模式。It can be understood that, in this application, the decoder parses the code stream, and if the affine mode parameter is not obtained by parsing, it can also be understood as indicating that the affine mode is not used.

示例性的，在本申请中，仿射模式参数的取值可以等于仿射运动补偿允许标志affine_enable_flag的值，如果affine_enable_flag的值为‘1’，表示可使用仿射运动补偿；如果affine_enable_flag的值为‘0’，表示不应使用仿射运动补偿。Exemplarily, in this application, the value of the affine mode parameter may be equal to the value of the affine motion compensation enable flag affine_enable_flag, if the value of affine_enable_flag is '1', it means that affine motion compensation can be used; if the value of affine_enable_flag is '0', indicating that affine motion compensation should not be used.

进一步地，在本申请的实施例中，如果解码器解析码流所获取的仿射模式参数指示使用仿射模式，那么解码器可以进行获取控制点模式和子块尺寸参数。Further, in the embodiment of the present application, if the affine mode parameter obtained by the decoder parsing the code stream indicates that the affine mode is used, the decoder may obtain the control point mode and the sub-block size parameter.

需要说明的是，在本申请的实施例中，控制点模式用于对控制点的个数进行确定。在仿射模型中，一个子块可以有2个控制点或者3个控制点，相应地，控制点模式可以为2个控制点对应的控制点模式，或者为3个控制点对应的控制点模式。即控制点模式可以包括4参数模式和6参数模式。It should be noted that, in the embodiments of the present application, the control point mode is used to determine the number of control points. In the affine model, a sub-block can have 2 control points or 3 control points, correspondingly, the control point pattern can be the control point pattern corresponding to 2 control points, or the control point pattern corresponding to 3 control points . That is, the control point mode can include a 4-parameter mode and a 6-parameter mode.

可以理解的是，在本申请的实施例中，对于AVS3标准，如果当前块使用了仿射模式，那么解码器还需要确定出当前块在仿射模式中控制点的个数进行确定，从而可以确定出使用的是4参数(2个控制点)模式，还是6参数(3个控制点)模式。It can be understood that, in the embodiments of the present application, for the AVS3 standard, if the current block uses the affine mode, the decoder also needs to determine the number of control points in the affine mode of the current block to determine, so that it can be determined. Determines whether 4-parameter (2 control points) mode or 6-parameter (3 control points) mode is used.

进一步地，在本申请的实施例中，如果解码器解析码流所获取的仿射模式参数指示使用仿射模式，那么解码器可以通过解析码流，进一步获取子块尺寸参数。Further, in the embodiment of the present application, if the affine mode parameter obtained by the decoder parsing the code stream indicates that the affine mode is used, the decoder can further obtain the sub-block size parameter by parsing the code stream.

具体地，子块尺寸参数可以通过仿射预测子块尺寸标志affine_subblock_size_flag确定，解码器通过解析码流，获得子块尺寸标志，并根据子块标志的取值确定当前块的子块的尺寸大小。其中，子块的尺寸大小可以为8x8或4x4。具体地，在本申请中，子块尺寸标志可以为一个二值变量。若子块尺寸标志的取值为1，则指示子块尺寸参数为8x8；若子块尺寸标志的取值为0，则指示子块尺寸参数为4x4。Specifically, the subblock size parameter can be determined by the affine prediction subblock size flag affine_subblock_size_flag. The decoder obtains the subblock size flag by parsing the code stream, and determines the subblock size of the current block according to the value of the subblock flag. The size of the sub-block may be 8x8 or 4x4. Specifically, in this application, the sub-block size flag may be a binary variable. If the value of the subblock size flag is 1, it indicates that the subblock size parameter is 8x8; if the value of the subblock size flag is 0, it indicates that the subblock size parameter is 4x4.

示例性的，在本申请中，子块尺寸标志的取值可以等于仿射预测子块尺寸标志affine_subblock_size_flag的值，如果affine_subblock_size_flag的值为‘1’，则当前块划分为尺寸为8x8的子块；如果affine_subblock_size_flag的值为‘0’，则当前块划分为尺寸为4x4的子块。Exemplarily, in this application, the value of the subblock size flag may be equal to the value of the affine prediction subblock size flag affine_subblock_size_flag, if the value of affine_subblock_size_flag is '1', the current block is divided into subblocks with a size of 8x8; If the value of affine_subblock_size_flag is '0', the current block is divided into sub-blocks of size 4x4.

可以理解的是，在本申请中，解码器解析码流，如果未解析得到子块尺寸标志，那么也可以理解为当前块划分为4x4的子块。也就是说，如果码流中不存在affine_subblock_size_flag，可以直接将子块尺寸标志的取值设置为0。It can be understood that, in this application, the decoder parses the code stream, and if the sub-block size flag is not obtained by parsing, it can also be understood that the current block is divided into 4×4 sub-blocks. That is to say, if affine_subblock_size_flag does not exist in the code stream, the value of the subblock size flag can be directly set to 0.

进一步地，在本申请的实施例中，解码器在确定控制点模式和子块尺寸参数之后，便可以根据预测参考模式、控制点模式以及子块尺寸参数，进一步确定出当前块中的子块的运动矢量。Further, in the embodiment of the present application, after determining the control point mode and the sub-block size parameter, the decoder can further determine the size of the sub-block in the current block according to the prediction reference mode, the control point mode and the sub-block size parameter. Motion vector.

具体地，在本申请的实施例中，解码器可以先根据预测参考模式确定控制点运动矢量组；然后可以基于控制点运动矢量组、控制点模式以及子块尺寸参数，确定出子块的运动矢量。Specifically, in the embodiment of the present application, the decoder may first determine the control point motion vector group according to the prediction reference mode; and then may determine the motion of the subblock based on the control point motion vector group, the control point mode and the subblock size parameter vector.

可以理解的是，在本申请的实施例中，控制点运动矢量组可以用于对控制点的运动矢量进行确定。It can be understood that, in the embodiment of the present application, the control point motion vector group may be used to determine the motion vector of the control point.

需要说明的是，在本申请的实施例中，解码器可以按照上述方法，遍历当前块中的每一个子块，利用每一个子块的控制点运动矢量组、控制点模式以及子块尺寸参数，确定出每一个子块的运动矢量，从而可以根据每一个子块的运动矢量构建获得运动矢量集合。It should be noted that, in the embodiment of the present application, the decoder can traverse each sub-block in the current block according to the above method, and use the control point motion vector group, control point mode and sub-block size parameter of each sub-block , the motion vector of each sub-block is determined, so that the motion vector set can be obtained according to the construction of the motion vector of each sub-block.

可以理解的是，在本申请的实施例中，当前块的运动矢量集合中可以包括当前块的每一个子块的运动矢量。It can be understood that, in this embodiment of the present application, the motion vector set of the current block may include the motion vector of each sub-block of the current block.

进一步地，在本申请的实施例中，解码器在根据控制点运动矢量组、控制点模式以及子块尺寸参数，确定运动矢量时，可以先根据控制点运动矢量组、控制点模式以及当前块的尺寸参数，确定差值变量；然后可以基于预测模式参数和子块尺寸参数，获得用于确定子块的运动矢量的位置；最后，便可以利用差值变量和用于确定子块的运动矢量的位置，确定子块的运动矢量，进而可以获得当前块的多个子块的运动矢量集合。Further, in the embodiment of the present application, when the decoder determines the motion vector according to the control point motion vector group, the control point mode and the sub-block size parameter, it can first determine the motion vector according to the control point motion vector group, the control point mode and the current block. Then, based on the prediction mode parameter and the sub-block size parameter, the position of the motion vector for determining the sub-block can be obtained; finally, the difference variable and the motion vector for determining the sub-block can be used. position, determine the motion vector of the sub-block, and then obtain the motion vector set of multiple sub-blocks of the current block.

示例性的，在本申请中，差值变量可以包括4个变量，具体为dHorX、dVerX、dHorY和dVerY，在计算差值变量时，解码器需要先确定出控制点运动矢量组，其中，控制点运动矢量组可以对控制点的运动矢量进行表征。Exemplarily, in this application, the difference variable may include 4 variables, specifically dHorX, dVerX, dHorY, and dVerY. When calculating the difference variable, the decoder needs to first determine the control point motion vector group, where the control point The point motion vector group can characterize the motion vector of the control point.

具体地，如果控制点模式为6参数模式，即存在3个控制点，那么控制点运动矢量组可以为包括3个运动矢量的运动矢量组，表示为mvsAffine(mv0，mv1，mv2)；如果控制点模式为4参数模式，即存在2个控制点，那么控制点运动矢量组可以为包括2个运动矢量的运动矢量组，表示为mvsAffine(mv0，mv1)。Specifically, if the control point mode is a 6-parameter mode, that is, there are 3 control points, then the control point motion vector group may be a motion vector group including 3 motion vectors, expressed as mvsAffine(mv0, mv1, mv2); if the control point If the point mode is a 4-parameter mode, that is, there are two control points, the control point motion vector group may be a motion vector group including two motion vectors, which is expressed as mvsAffine(mv0, mv1).

接着，解码器便可以利用控制点运动矢量组进行差值变量的计算：Then, the decoder can use the control point motion vector group to calculate the difference variable:

dHorX＝(mv1_x-mv0_x)<<(7-Log(width))；dHorX=(mv1_x-mv0_x)<<(7-Log(width));

dHorY＝(mv1_y-mv0_y)<<(7-Log(width))；dHorY=(mv1_y-mv0_y)<<(7-Log(width));

dVerX＝(mv2_x-mv0_x)<<(7-Log(height))；dVerX=(mv2_x-mv0_x)<<(7-Log(height));

dVerY＝(mv2_y-mv0_y)<<(7-Log(height))；dVerY=(mv2_y-mv0_y)<<(7-Log(height));

dVerX＝-dHorY；dVerX=-dHorY;

dVerY＝dHorX。dVerY=dHorX.

其中，width和height分别为当前块的宽度和高度，即当前块的尺寸参数，具体地，当前块的尺寸参数可以为解码器通过解析码流获取的。The width and height are respectively the width and height of the current block, that is, the size parameter of the current block. Specifically, the size parameter of the current block may be obtained by the decoder by parsing the code stream.

进一步地，在本申请的实施例中，解码器在确定出差值变量之后，可以接着基于预测模式参数和子块尺寸参数，获得用于确定子块的运动矢量的位置。具体地，解码器可以通过字块尺寸标志确定出子块的尺寸，同时可以通过预测模式参数确定具体使用哪一种预测模式，然后便可以根据子块的尺寸和使用的预测模式，确定出用于确定子块的运动矢量的位置。Further, in the embodiment of the present application, after determining the difference value variable, the decoder may then obtain the position for determining the motion vector of the sub-block based on the prediction mode parameter and the sub-block size parameter. Specifically, the decoder can determine the size of the sub-block through the word block size flag, and can determine which prediction mode to use through the prediction mode parameter, and then can determine the size of the sub-block and the prediction mode used. used to determine the position of the motion vector of the sub-block.

示例性的，在本申请中，如果当前块的预测参考模式的取值为2，即为第三参考模式‘Pred_List01’，或者，子块尺寸标志的取值为1，即子块的宽度subwidth和高度subheight均等于8，则(x，y)是8×8子块左上角位置的坐标，那么可以通过以下方式获得用于确定子块的运动矢量的位置的坐标xPos和yPos：Exemplarily, in this application, if the value of the prediction reference mode of the current block is 2, it is the third reference mode 'Pred_List01', or, the value of the subblock size flag is 1, that is, the width subwidth of the subblock. and height subheight are both equal to 8, then (x, y) is the coordinates of the upper left corner of the 8×8 sub-block, then the coordinates xPos and yPos used to determine the position of the motion vector of the sub-block can be obtained in the following ways:

如果子块是当前块的左上角的控制点，则xPos和yPos均等于0；If the sub-block is the control point in the upper left corner of the current block, both xPos and yPos are equal to 0;

如果子块是当前块的右上角的控制点，则xPos等于width，yPos等于0；If the subblock is the control point in the upper right corner of the current block, xPos equals width and yPos equals 0;

如果子块是当前块的左下角的控制点，控制点运动矢量组可以为包括3个运动矢量的运动矢量组，则xPos等于0，yPos等于height；If the sub-block is the control point at the lower left corner of the current block, the control point motion vector group may be a motion vector group including 3 motion vectors, then xPos is equal to 0, and yPos is equal to height;

否则，xPos等于(x-xE)+4，yPos等于(y-yE)+4。Otherwise, xPos is equal to (x-xE)+4 and yPos is equal to (y-yE)+4.

示例性的，在本申请中，如果当前块的预测参考模式的取值0或1，即为第一参考模式‘Pred_List0’或第二参考模式‘Pred_List1’，且子块尺寸标志的取值为0，即子块的宽度subwidth和高度subheight均等于4，(x，y)是4×4子块左上角位置的坐标，那么可以通过以下方式获得用于确定子块的运动矢量的位置的坐标xPos和yPos：Exemplarily, in this application, if the prediction reference mode of the current block is 0 or 1, it is the first reference mode 'Pred_List0' or the second reference mode 'Pred_List1', and the value of the subblock size flag is 0, that is, the width subwidth and height subheight of the subblock are both equal to 4, (x, y) is the coordinates of the upper left corner of the 4×4 subblock, then the coordinates used to determine the position of the motion vector of the subblock can be obtained in the following way xPos and yPos:

否则，xPos等于(x-xE)+2，yPos等于(y-yE)+2。Otherwise, xPos is equal to (x-xE)+2 and yPos is equal to (y-yE)+2.

进一步地，在本申请的实施例中，解码器计算获得用于确定子块的运动矢量的位置之后，接着可以基于用于确定子块的运动矢量的位置和差值变量确定出该子块的运动矢量，最后，遍历当前块的每一个子块，获得每一个子块的运动矢量，便可以构建获得当前块的多个子块的运动矢量集合。Further, in the embodiment of the present application, after the decoder calculates and obtains the position of the motion vector used for determining the sub-block, it can then determine the position of the sub-block based on the position and the difference variable for determining the motion vector of the sub-block. Motion vector, and finally, traverse each sub-block of the current block to obtain the motion vector of each sub-block, then a motion vector set for obtaining multiple sub-blocks of the current block can be constructed.

示例性的，在本申请中，解码器在获得用于确定子块的运动矢量的位置xPos和yPos之后，可以通过以下方式确定出子块的运动矢量mvE(mvE_x，mvE_y)Exemplarily, in this application, after obtaining the positions xPos and yPos for determining the motion vector of the sub-block, the decoder may determine the motion vector mvE of the sub-block in the following manner (mvE_x, mvE_y)

需要说明的是，在本申请中，在确定子块内的每个位置与子块的运动矢量的偏差时，如果当前块使用的是仿射预测模型，可以根据仿射预测模型的公式计算出子块内的每个位置的运动矢量，与子块的运动矢量相减得到它们的偏差。如果子块的运动矢量都选择子块内同一位置的运动矢量，如4x4的块使用距离左上角(2，2)的位置，8x8的块使用距离左上角(4，4)的位置，根据现在标准包括VVC和AVS3中使用的仿射模型，每个子块相同位置的运动矢量偏差都是相同的。但是AVS在左上角，右上角，以及3控制点的情况下的左下角(上述AVS3文本中的，如图4所示的A，B，C位置)与其他块使用的位置不同，相应地在计算左上角，右上角，以及3控制点的情况下的左下角的子块的运动矢量偏差时与其他块也不同。具体如实施例。It should be noted that, in this application, when determining the deviation between each position in the sub-block and the motion vector of the sub-block, if the current block uses an affine prediction model, it can be calculated according to the formula of the affine prediction model. The motion vector of each position within the sub-block is subtracted from the motion vector of the sub-block to obtain their offset. If the motion vectors of the sub-blocks all select the motion vector at the same position in the sub-block, for example, the 4x4 block uses the position from the upper left corner (2, 2), and the 8x8 block uses the position from the upper left corner (4, 4). The standard includes the affine model used in VVC and AVS3, where the motion vector bias for the same position of each sub-block is the same. But AVS is in the upper left corner, upper right corner, and lower left corner in the case of 3 control points (the A, B, C positions in the above AVS3 text, as shown in Figure 4) are different from the positions used by other blocks, correspondingly in The calculation of the motion vector deviation of the upper left corner, upper right corner, and lower left sub-block in the case of three control points is also different from that of other blocks. Specific examples are as follows.

可以理解的是，本申请实施例提出的帧间预测方法对解码器执行步骤202和步骤203的顺序不进行限定，也就是说，在本申请中，解码器可以先执行步骤202，然后执行步骤203，也可以先执行步骤203，再执行步骤202，还可以同时执行步骤202和步骤203。It can be understood that the inter-frame prediction method proposed in this embodiment of the present application does not limit the order in which the decoder performs step 202 and step 203, that is, in this application, the decoder may perform step 202 first, and then perform step 203. 203, step 203 may also be performed first, and then step 202 may be performed, or step 202 and step 203 may be performed simultaneously.

进一步地，在本申请的实施例中，解码器在基于运动矢量确定子块的第一预测值时，可以先确定样本矩阵；其中，样本矩阵包括亮度样本矩阵和色度样本矩阵；然后可以根据预测参考模式、子块尺寸参数、样本矩阵以及运动矢量集合，确定第一预测值。Further, in the embodiment of the present application, when the decoder determines the first predicted value of the sub-block based on the motion vector, it can first determine the sample matrix; wherein, the sample matrix includes a luminance sample matrix and a chrominance sample matrix; The prediction reference mode, the sub-block size parameter, the sample matrix, and the motion vector set determine the first prediction value.

需要说明的是，在本申请的实施例中，解码器在根据预测参考模式、子块尺寸参数、样本矩阵以及运动矢量集合，确定第一预测值时，可以先根据预测参考模式和子块尺寸参数，从运动矢量集合中确定目标运动矢量；然后可以利用预测参考模式对应的参考图像队列和参考索引样本矩阵以及目标运动矢量，确定预测样本矩阵；其中，预测样本矩阵包括多个子块的第一预测值。It should be noted that, in the embodiments of the present application, when the decoder determines the first prediction value according to the prediction reference mode, the sub-block size parameter, the sample matrix, and the motion vector set, it can first determine the first prediction value according to the prediction reference mode and the sub-block size parameter. , determine the target motion vector from the motion vector set; then the reference image queue corresponding to the prediction reference mode and the reference index sample matrix and the target motion vector can be used to determine the prediction sample matrix; wherein, the prediction sample matrix includes the first prediction of a plurality of sub-blocks value.

具体地，在本申请的实施例中，样本矩阵可以包括亮度样本矩阵和色度样本矩阵，相应地，解码器确定出的预测样本矩阵可以包括亮度预测样本矩阵和色度预测样本矩阵，其中，亮度预测样本矩阵包括多个子块的第一亮度预测值，色度预测样本矩阵包括多个子块的第一色度预测值，第一亮度预测值和第一色度预测值构成子块的第一预测值。Specifically, in the embodiment of the present application, the sample matrix may include a luma sample matrix and a chroma sample matrix. Correspondingly, the predicted sample matrix determined by the decoder may include a luma predicted sample matrix and a chrominance predicted sample matrix, wherein, The luma prediction sample matrix includes the first luma predicted values of multiple sub-blocks, the chrominance prediction sample matrix includes the first chrominance predicted values of the multiple sub-blocks, and the first luma predicted value and the first chrominance predicted value constitute the first luma predicted value of the sub-block. Predictive value.

示例性的，在本申请中，假设当前块左上角样本在当前图像的亮度样本矩阵中的位置为(xE，yE)。如果当前块的预测参考模式取值为0，即使用第一参考模式‘PRED_List0’，且子块尺寸标志的取值为0，即子块尺寸参数为4×4，那么目标运动矢量mv0E0是当前块的运动矢量集合在(xE+x，yE+y)位置的4×4子块的运动矢量。亮度预测样本矩阵predMatrixL0中的元素predMatrixL0[x][y]的值是参考图像队列0中参考索引为RefIdxL0的1/16精度亮度样本矩阵中位置为(((xE+x)<<4)+mv0E0_x，(yE+y)<<4)+mv0E0_y)))的样本值，色度预测样本矩阵predMatrixL0中的元素predMatrixL0[x][y]的值是参考图像队列0中参考索引为RefIdxL0的1/32精度色度样本矩阵中位置为(((xE+2×x)<<4)+MvC_x，(yE+2×y)<<4)+MvC_y)))的样本值。其中，x1＝((xE+2×x)>>3)<<3，y1＝((yE+2×y)>>3)<<3，mv1E0是当前块的运动矢量集合在(x1，y1)位置的4×4单元的运动矢量，mv2E0是当前块的运动矢量集合在(x1+4，y1)位置的4×4单元的运动矢量，mv3E0是当前块的运动矢量集合在(x1，y1+4)位置的4×4单元的运动矢量，mv4E0是当前块的运动矢量集合在(x1+4，y1+4)位置的4×4单元的运动矢量。Exemplarily, in this application, it is assumed that the position of the upper left corner sample of the current block in the luminance sample matrix of the current image is (xE, yE). If the prediction reference mode of the current block is 0, that is, the first reference mode 'PRED_List0' is used, and the sub-block size flag is 0, that is, the sub-block size parameter is 4×4, then the target motion vector mv0E0 is the current The motion vector of the block sets the motion vectors of the 4x4 sub-blocks at the (xE+x, yE+y) position. The value of the element predMatrixL0[x][y] in the luma prediction sample matrix predMatrixL0 is the position in the 1/16-precision luma sample matrix with the reference index RefIdxL0 in the reference image queue 0 (((xE+x)<<4)+ mv0E0_x, the sample value of (yE+y)<<4)+mv0E0_y))), the value of the element predMatrixL0[x][y] in the chrominance prediction sample matrix predMatrixL0 is 1 whose reference index is RefIdxL0 in the reference image queue 0 The sample value at position (((xE+2×x)<<4)+MvC_x, (yE+2×y)<<4)+MvC_y))) in the /32-precision chroma sample matrix. Among them, x1=((xE+2×x)>>3)<<3, y1=((yE+2×y)>>3)<<3, mv1E0 is the motion vector set of the current block in (x1, y1) is the motion vector of the 4×4 unit at the position, mv2E0 is the motion vector of the 4×4 unit of the current block’s motion vector set at the (x1+4, y1) position, and mv3E0 is the current block’s motion vector set at (x1, y1) The motion vector of the 4×4 unit at the position of y1+4), mv4E0 is the motion vector of the 4×4 unit of the motion vector set of the current block at the position of (x1+4, y1+4).

具体地，MvC_x和MvC_y可以通过以下方式确定：Specifically, MvC_x and MvC_y can be determined in the following ways:

示例性的，在本申请中，假设当前块左上角样本在当前图像的亮度样本矩阵中的位置为(xE，yE)，如果当前块的预测参考模式取值为0，即使用第一参考模式‘PRED_List0’，且子块尺寸标志的取值为1，即子块尺寸参数为8×8，那么目标运动矢量mv0E0是当前块的运动矢量集合在(xE+x，yE+y)位置的8×8单元的运动矢量。亮度预测样本矩阵predMatrixL0中的元素predMatrixL0[x][y]的值是参考图像队列0中参考索引为RefIdxL0的1/16精度亮度样本矩阵中位置为(((xE+x)<<4)+mv0E0_x，(yE+y)<<4)+mv0E0_y)))的样本值，色度预测样本矩阵predMatrixL0中的元素predMatrixL0[x][y]的值是参考图像队列0中参考索引为RefIdxL0的1/32精度色度样本矩阵中位置为(((xE+2×x)<<4)+MvC_x，(yE+2×y)<<4)+MvC_y)))的样本值。其中，MvC_x等于mv0E0_x，MvC_y等于mv0E0。Exemplarily, in this application, it is assumed that the position of the upper left corner sample of the current block in the luminance sample matrix of the current image is (xE, yE), if the prediction reference mode of the current block is 0, that is, the first reference mode is used. 'PRED_List0', and the value of the sub-block size flag is 1, that is, the sub-block size parameter is 8 × 8, then the target motion vector mv0E0 is the motion vector set of the current block at (xE+x, yE+y) position 8 Motion vector of ×8 elements. The value of the element predMatrixL0[x][y] in the luma prediction sample matrix predMatrixL0 is the position in the 1/16-precision luma sample matrix with the reference index RefIdxL0 in the reference image queue 0 (((xE+x)<<4)+ mv0E0_x, the sample value of (yE+y)<<4)+mv0E0_y))), the value of the element predMatrixL0[x][y] in the chrominance prediction sample matrix predMatrixL0 is 1 whose reference index is RefIdxL0 in the reference image queue 0 The sample value at position (((xE+2×x)<<4)+MvC_x, (yE+2×y)<<4)+MvC_y))) in the /32-precision chroma sample matrix. Among them, MvC_x is equal to mv0E0_x, and MvC_y is equal to mv0E0.

示例性的，在本申请中，假设当前块左上角样本在当前图像的亮度样本矩阵中的位置为(xE，yE)。如果当前块的预测参考模式取值为1，即使用第二参考模式‘PRED_List1’，且子块尺寸标志的取值为0，即子块尺寸参数为4×4，那么目标运动矢量mv0E1是当前块的运动矢量集合在(xE+x，yE+y)位置的4×4单元的运动矢量。亮度预测样本矩阵predMatrixL1中的元素predMatrixL1[x][y]的值是参考图像队列1中参考索引为RefIdxL1的1/16精度亮度样本矩阵中位置为(((xE+x)<<4)+mv0E1_x，(yE+y)<<4)+mv0E1_y)))的样本值，色度预测样本矩阵predMatrixL1中的元素predMatrixL1[x][y]的值是参考图像队列1中参考索引为RefIdxL1的1/32精度色度样本矩阵中位置为(((xE+2×x)<<4)+MvC_x，(yE+2×y)<<4)+MvC_y)))的样本值。其中，x1＝((xE+2×x)>>3)<<3，y1＝((yE+2×y)>>3)<<3，mv1E1是MvArray运动矢量集合在(x1，y1)位置的4×4单元的运动矢量，mv2E1是MvArray运动矢量集合在(x1+4，y1)位置的4×4单元的运动矢量，mv3E1是MvArray运动矢量集合在(x1，y1+4)位置的4×4单元的运动矢量，mv4E1是MvArray运动矢量集合在(x1+4，y1+4)位置的4×4单元的运动矢量。Exemplarily, in this application, it is assumed that the position of the upper left corner sample of the current block in the luminance sample matrix of the current image is (xE, yE). If the prediction reference mode of the current block is 1, that is, the second reference mode 'PRED_List1' is used, and the sub-block size flag is 0, that is, the sub-block size parameter is 4×4, then the target motion vector mv0E1 is the current The motion vector of the block is a set of motion vectors of 4×4 units at the (xE+x, yE+y) position. The value of the element predMatrixL1[x][y] in the luma prediction sample matrix predMatrixL1 is the 1/16-precision luma sample matrix whose reference index is RefIdxL1 in the reference image queue 1. The position in the matrix is (((xE+x)<<4)+ mv0E1_x, the sample value of (yE+y)<<4)+mv0E1_y))), the value of the element predMatrixL1[x][y] in the chrominance prediction sample matrix predMatrixL1 is 1 whose reference index is RefIdxL1 in the reference image queue 1 The sample value at position (((xE+2×x)<<4)+MvC_x, (yE+2×y)<<4)+MvC_y))) in the /32-precision chroma sample matrix. Among them, x1=((xE+2×x)>>3)<<3, y1=((yE+2×y)>>3)<<3, mv1E1 is the MvArray motion vector set at (x1, y1) The motion vector of the 4×4 unit of the position, mv2E1 is the motion vector of the 4×4 unit of the MvArray motion vector set at the (x1+4, y1) position, and mv3E1 is the MvArray motion vector set at the (x1, y1+4) position. Motion vector of 4×4 unit, mv4E1 is the motion vector of 4×4 unit of MvArray motion vector set at (x1+4, y1+4) position.

示例性的，在本申请中，假设当前块左上角样本在当前图像的亮度样本矩阵中的位置为(xE，yE)。如果当前块的预测参考模式取值为1，即使用第二参考模式‘PRED_List1’，且子块尺寸标志的取值为1，即子块尺寸参数为8×8，那么目标运动矢量mv0E1是当前块的运动矢量集合在(xE+x，yE+y)位置的8×8单元的运动矢量。亮度预测样本矩阵predMatrixL1中的元素predMatrixL1[x][y]的值是参考图像队列1中参考索引为RefIdxL1的1/16精度亮度样本矩阵中位置为(((xE+x)<<4)+mv0E1_x，(yE+y)<<4)+mv0E1_y)))的样本值，色度预测样本矩阵predMatrixL1中的元素predMatrixL1[x][y]的值是参考图像队列1中参考索引为RefIdxL1的1/32精度色度样本矩阵中位置为(((xE+2×x)<<4)+MvC_x，(yE+2×y)<<4)+MvC_y)))的样本值。其中MvC_x等于mv0E1_x，MvC_y等于mv0E1。Exemplarily, in this application, it is assumed that the position of the upper left corner sample of the current block in the luminance sample matrix of the current image is (xE, yE). If the prediction reference mode of the current block is 1, that is, the second reference mode 'PRED_List1' is used, and the sub-block size flag is 1, that is, the sub-block size parameter is 8×8, then the target motion vector mv0E1 is the current The motion vector of the block is a set of motion vectors of 8×8 units at the (xE+x, yE+y) position. The value of the element predMatrixL1[x][y] in the luma prediction sample matrix predMatrixL1 is the 1/16-precision luma sample matrix whose reference index is RefIdxL1 in the reference image queue 1. The position in the matrix is (((xE+x)<<4)+ mv0E1_x, the sample value of (yE+y)<<4)+mv0E1_y))), the value of the element predMatrixL1[x][y] in the chrominance prediction sample matrix predMatrixL1 is 1 whose reference index is RefIdxL1 in the reference image queue 1 The sample value at position (((xE+2×x)<<4)+MvC_x, (yE+2×y)<<4)+MvC_y))) in the /32-precision chroma sample matrix. where MvC_x is equal to mv0E1_x and MvC_y is equal to mv0E1.

示例性的，在本申请中，假设当前块左上角样本在当前图像的亮度样本矩阵中的位置为(xE，yE)。如果当前块的预测参考模式取值为2，即使用第三参考模式‘PRED_List01’，那么目标运动矢量mv0E0是当前块的运动矢量集合在(xE+x，yE+y)位置的8×8单元的运动矢量，目标运动矢量mv0E1是当前块的运动矢量集合在(x，y)位置的8×8单元的运动矢量。亮度预测样本矩阵predMatrixL0中的元素predMatrixL0[x][y]的值是参考图像队列0中参考索引为RefIdxL0的1/16精度亮度样本矩阵中位置为(((xE+x)<<4)+mv0E0_x，(yE+y)<<4)+mv0E0_y)))的样本值，色度预测样本矩阵predMatrixL0中的元素predMatrixL0[x][y]的值是参考图像队列0中参考索引为RefIdxL0的1/32 精度色度样本矩阵中位置为(((xE+2×x)<<4)+MvC0_x，(yE+2×y)<<4)+MvC0_y)))的样本值，亮度预测样本矩阵predMatrixL1中的元素predMatrixL1[x][y]的值是参考图像队列1中参考索引为RefIdxL1的1/16精度亮度样本矩阵中位置为((((xE+x)<<4)+mv0E1_x，(yE+y)<<4)+mv0E1_y))))的样本值，色度预测样本矩阵predMatrixL1中的元素predMatrixL1[x][y]的值是参考图像队列1中参考索引为RefIdxL1的1/32精度色度样本矩阵中位置为(((xE+2×x)<<4)+MvC1_x，(yE+2×y)<<4)+MvC1_y)))的样本值。其中MvC0_x等于mv0E0_x，MvC0_y等于mv0E0_y，MvC1_x等于mv0E1_x，MvC1_y等于mv0E1_y。Exemplarily, in this application, it is assumed that the position of the upper left corner sample of the current block in the luminance sample matrix of the current image is (xE, yE). If the prediction reference mode of the current block is 2, that is, the third reference mode 'PRED_List01' is used, then the target motion vector mv0E0 is an 8×8 unit where the motion vector set of the current block is located at (xE+x, yE+y) position The target motion vector mv0E1 is the motion vector of the 8×8 unit where the motion vector of the current block is set at the (x, y) position. The value of the element predMatrixL0[x][y] in the luma prediction sample matrix predMatrixL0 is the position in the 1/16-precision luma sample matrix with the reference index RefIdxL0 in the reference image queue 0 (((xE+x)<<4)+ mv0E0_x, the sample value of (yE+y)<<4)+mv0E0_y))), the value of the element predMatrixL0[x][y] in the chrominance prediction sample matrix predMatrixL0 is 1 whose reference index is RefIdxL0 in the reference image queue 0 /32 The sample value at the position of (((xE+2×x)<<4)+MvC0_x, (yE+2×y)<<4)+MvC0_y))) in the precision chroma sample matrix, the luminance prediction sample matrix The value of the element predMatrixL1[x][y] in predMatrixL1 is the 1/16-precision luma sample matrix whose reference index is RefIdxL1 in the reference image queue 1. The position in the matrix is ((((xE+x)<<4)+mv0E1_x, ( The sample value of yE+y)<<4)+mv0E1_y)))), the value of the element predMatrixL1[x][y] in the chrominance prediction sample matrix predMatrixL1 is 1/32 of the reference index in the reference image queue 1 as RefIdxL1 The sample value at position (((xE+2×x)<<4)+MvC1_x, (yE+2×y)<<4)+MvC1_y))) in the precision chroma sample matrix. Where MvC0_x is equal to mv0E0_x, MvC0_y is equal to mv0E0_y, MvC1_x is equal to mv0E1_x, and MvC1_y is equal to mv0E1_y.

需要说明的是，在申请的实施例中，样本矩阵中的亮度样本矩阵可以为1/16精度亮度样本矩阵，样本矩阵中的色度样本矩阵可以为1/32精度色度样本矩阵。It should be noted that, in the embodiment of the application, the luminance sample matrix in the sample matrix may be a 1/16 precision luminance sample matrix, and the chrominance sample matrix in the sample matrix may be a 1/32 precision chrominance sample matrix.

可以理解的是，在本申请的实施例中，对于不同的预测参考模式，解码器通过解析码流所获取的参考图像队列和参考索引是不相同的。It can be understood that, in the embodiment of the present application, for different prediction reference modes, the reference picture queue and reference index obtained by the decoder by parsing the code stream are different.

进一步地，在本申请的实施例中，解码器确定样本矩阵时，可以先获取亮度插值滤波器系数和色度插值滤波器系数；然后可以基于亮度插值滤波器系数确定亮度样本矩阵，同时，可以基于色度插值滤波器系数确定色度样本矩阵。Further, in the embodiment of the present application, when the decoder determines the sample matrix, it can first obtain the luminance interpolation filter coefficients and the chrominance interpolation filter coefficients; then the luminance sample matrix can be determined based on the luminance interpolation filter coefficients, and at the same time, it can A matrix of chroma samples is determined based on the chroma interpolation filter coefficients.

示例性的，在本申请中，解码器在确定亮度样本矩阵时，获取的亮度插值滤波器系数如上述表1所示，然后按照像素点的位置和样本位置，计算获得亮度样本矩阵。Exemplarily, in this application, when the decoder determines the luminance sample matrix, the obtained luminance interpolation filter coefficients are shown in Table 1, and then the luminance sample matrix is obtained by calculation according to the pixel position and the sample position.

具体地，样本位置a _x，0(x＝1～15)由水平方向上距离插值点最近得8个整数值滤波得到，预测值的获取方式如下： Specifically, the sample position a _{x, 0} (x=1～15) is obtained by filtering the 8 integer values closest to the interpolation point in the horizontal direction, and the method of obtaining the predicted value is as follows:

a _x，0＝Clip1((fL[x][0]×A _-3，0+fL[x][1]×A _-2，0+fL[x][2]×A _-1，0+fL[x][3]×A _0，0+fL[x][4]×A _1，0+fL[x][5]×A _2，0+fL[x][6]×A _3， ₀+fL[x][7]×A _4，0+32)>>6)。 a _x,0 =Clip1((fL[x][0]×A _-3,0 +fL[x][1]×A- _2,0 +fL[x][2]×A- _1,0 + fL[x][3]×A _0,0 +fL[x][4]×A _1,0 +fL[x][5]×A _2,0 +fL[x][6]×A _3, ₀ +fL[x][7]×A 4, ₀ +32)>>6).

具体地，样本位置a _0，y(y＝1～15)由垂直方向上距离插值点最近得8个整数值滤波得到，预测值的获取方式如下： Specifically, the sample position a _{0, y} (y=1～15) is obtained by filtering the 8 integer values closest to the interpolation point in the vertical direction, and the method of obtaining the predicted value is as follows:

a _0，y＝Clip1((fL[y][0]×A _0，-3+fL[y][1]×A _-2，0+fL[y][2]×A _-1，0+fL[y][3]×A _0，0+fL[y][4]×A _1，0+fL[y][5]×A _2，0+fL[y][6]×A _3， ₀+fL[y][7]×A _-4，0+32)>>6)。 a _0,y =Clip1((fL[y][0]×A _0,-3 +fL[y][1]×A- _2,0 +fL[y][2]×A- _1,0 + fL[y][3]×A _0,0 +fL[y][4]×A _1,0 +fL[y][5]×A _2,0 +fL[y][6]×A _3, ₀ +fL[y][7]×A _-4 , 0 +32)>>6).

具体地，样本位置a _x，y(x＝1～15，y＝1～15)的预测值的获取方式如下： Specifically, the acquisition method of the predicted value of the sample position a _{x, y} (x=1-15, y=1-15) is as follows:

a _x，y＝Clip1((fL[y][0]×a' _x，y-3+fL[y][1]×a' _x，y-2+fL[y][2]× _a'x，y-1+fL[y][3]× _a'x，y+fL[y][4]× _a'x，y+1+fL[y][5]× _a'x，y+2+fL[y][6]× _a'x， _y+3+fL[y][7]× _a'x，y+4+(1<<(19-BitDepth)))>>(20-BitDepth))。 a _{x, y =} Clip1((fL[y][0]×a' _{x, y-3} +fL[y][1]×a' _{x, y-2} +fL[y][2]× _{a' x, y-1} +fL[y][3] × _{a'x, y} +fL[y][4] × _{a'x, y+1} +fL[y][5] × _{a'x, y} + 2+fL[y][6]× _a'x, _y+3 +fL[y][7]× _{a'x, y+4} +(1<<(19-BitDepth)))>>(20- BitDepth)).

其中：in:

a' _x，y＝(fL[x][0]×A _-3，y+fL[x][1]×A _-2，y+fL[x][2]×A _-1，y+fL[x][3]×A _0，y+fL[x][4]×A _1，y+fL[x][5]×A _2，y+fL[x][6]×A _3， _y+fL[x][7]×A _4，y+((1<<(BitDepth-8))>>1))>>(BitDepth-8)。 a' _{x, y} = (fL[x][0]×A _{-3, y} +fL[x][1]×A _{-2, y} +fL[x][2]×A _{-1, y+} fL[ x][3]×A _{0, y} +fL[x][4]×A _{1, y} +fL[x][5]×A _{2, y} +fL[x][6]×A _3, _y + fL[x][7]×A _{4, y+} ((1<<(BitDepth-8))>>1))>>(BitDepth-8).

示例性的，在本申请中，解码器在确定色度样本矩阵时，可以先解析码流获得色度插值滤波器系数如上述表2所示，然后按照像素点的位置和样本位置，计算获得色度样本矩阵。Exemplarily, in this application, when the decoder determines the chrominance sample matrix, it can first parse the code stream to obtain the chrominance interpolation filter coefficients as shown in Table 2 above, and then calculate and obtain according to the pixel position and the sample position. Chroma sample matrix.

if(dx＝＝0){if(dx==0){

a _x，y(0，dy)＝Clip3(0，(1<<BitDepth)-1，(fC[dy][0]×A _x，y-1+fC[dy][1]×A _x，y+fC[dy][2]×A _x，y+1+fC[dy][3]×A _x， _y+2+32)>>6) a _{x, y} (0, dy)=Clip3(0, (1<<BitDepth)-1, (fC[dy][0]× _{Ax, y} -1+fC[dy][1]× _{Ax, y} +fC[dy][2]×A _{x, y} +1+fC[dy][3]×A _x, _y +2+32)>>6)

}}

else if(dy＝＝0){else if(dy==0){

a _x，y(dx，0)＝Clip3(0，(1<<BitDepth)-1，(fC[dx][0]×A _x-1，y+fC[dx][1]×A _x，y+fC[dx][2]×A _x+1，y+fC[dx][3]×A _x+2， _y+32)>>6) a _x,y (dx,0)=Clip3(0,(1<<BitDepth)-1,(fC[dx][0]× _Ax-1,y +fC[dx][1]× _{Ax, y} +fC[dx][2]×A _{x+1, y} +fC[dx][3]×A _x+2, _y+32 )>>6)

}}

else{else{

a _x，y(dx，dy)＝Clip3(0，(1<<BitDepth)-1，(C[dy][0]×a' _x，y-1(dx，0)+C[dy][1]×a' _x，y(dx，0)+C[dy][2]×a' _x， _y+1(dx，0)+C[dy][3]×a' _x，y+2(dx，0)+(1<<(19-BitDepth)))>>(20-BitDepth)) a _x,y (dx,dy)=Clip3(0,(1<<BitDepth)-1,(C[dy][0]×a' _x,y-1 (dx,0)+C[dy][ 1]×a' _{x, y} (dx, 0)+C[dy][2]×a' _x, _y +1(dx, 0)+C[dy][3]×a' _{x, y} +2 (dx, 0)+(1<<(19-BitDepth)))>>(20-BitDepth))

}}

其中，a' _x，y(dx，0)是整像素行上的分像素的临时值，定义为：a' _x，y(dx，0)＝(fC[dx][0]×A _x-1，y+fC[dx][1]×A _x， _y+fC[dx][2]×A _x+1，y+fC[dx][3]×A _x+2，y+((1<<(BitDepth-8))>>1))>>(BitDepth-8)。 Among them, a' _{x, y} (dx, 0) is the temporary value of the sub-pixel on the whole pixel row, which is defined as: a' _{x, y} (dx, 0)=(fC[dx][0]×A _{x- 1, y} +fC[dx][1]× _Ax, _y +fC[dx][2]× _{Ax+1, y+} fC[dx][3]× _{Ax+2, y} +((1<<(BitDepth-8))>>1))>>(BitDepth-8).

进一步地，在本申请的实施例中，解码器在确定像素点的运动矢量与子块的运动矢量之间的第一运动矢量偏差时，可以先解析码流，获取二次预测参数；如果二次预测参数指示使用二次预测，那么解码器可以基于差值变量确定每一个像素点的运动矢量与子块的运动矢量之间的第一运动矢量偏差。Further, in the embodiment of the present application, when determining the first motion vector deviation between the motion vector of the pixel point and the motion vector of the sub-block, the decoder can first parse the code stream to obtain the secondary prediction parameter; The secondary prediction parameter indicates that secondary prediction is used, then the decoder can determine the first motion vector deviation between the motion vector of each pixel and the motion vector of the sub-block based on the difference variable.

具体地，在本申请的实施例中，解码器在基于差值变量确定每一个像素点的运动矢量与子块的运动矢量之间的第一运动矢量偏差时，也可以按照上述方法，根据控制点运动矢量组、控制点模式以及当前块的尺寸参数，确定出dHorX、dVerX、dHorY和dVerY这4个差值变量，然后在利用差值变量进一步确定出子块中的每一个像素点所对应的第一运动矢量偏差。Specifically, in the embodiment of the present application, when the decoder determines the first motion vector deviation between the motion vector of each pixel point and the motion vector of the sub-block based on the difference variable, it can also follow the above method, according to the control Point motion vector group, control point mode and size parameters of the current block, determine the four difference variables dHorX, dVerX, dHorY and dVerY, and then use the difference variable to further determine the corresponding pixel in the sub-block. The first motion vector deviation of .

示例性的，在本申请中，width和height分别为解码器获取的当前块的宽度和高度，利用子块尺寸参数所确定出子块的宽度subwidth和高度subheight。假设(i，j)为子块内部的任意像素点的像素位置的坐标，其中，i的取值范围是0～(subwidth-1)，j的取值范围是0～(subheight-1)，那么，可以通过以下方法计算4不同类型的子块的内部每个像素(i，j)位置的运动矢量偏差：Exemplarily, in this application, width and height are the width and height of the current block obtained by the decoder, respectively, and the width subwidth and height subheight of the subblock are determined by using the subblock size parameter. Suppose (i, j) is the coordinate of the pixel position of any pixel in the sub-block, where the value range of i is 0～(subwidth-1), the value range of j is 0～(subheight-1), Then, the motion vector deviation of each pixel (i, j) position inside the 4 different types of sub-blocks can be calculated by the following method:

如果子块是当前块的左上角的控制点A，那么(i，j)像素的运动矢量偏差dMvA[i][j]：If the sub-block is the control point A in the upper left corner of the current block, then the motion vector deviation dMvA[i][j] of the (i, j) pixel:

dMvA[i][j][0]＝dHorX×i+dVerX×jdMvA[i][j][0]=dHorX×i+dVerX×j

dMvA[i][j][1]＝dHorY×i+dVerY×j；dMvA[i][j][1]=dHorY×i+dVerY×j;

如果子块是当前块的右上角的控制点B，那么(i，j)像素的运动矢量偏差dMvB[i][j]：If the sub-block is the control point B in the upper right corner of the current block, then the motion vector deviation dMvB[i][j] of the (i, j) pixel:

dMvB[i][j][0]＝dHorX×(i-subwidth)+dVerX×jdMvB[i][j][0]=dHorX×(i-subwidth)+dVerX×j

dMvB[i][j][1]＝dHorY×(i-subwidth)+dVerY×j；dMvB[i][j][1]=dHorY×(i-subwidth)+dVerY×j;

如果子块是当前块的左下角的控制点C，控制点运动矢量组可以为包括3个运动矢量的运动矢量组，那么(i，j)像素的运动矢量偏差dMvC[i][j]：If the sub-block is the control point C in the lower left corner of the current block, the control point motion vector group may be a motion vector group including 3 motion vectors, then the motion vector deviation dMvC[i][j] of the (i, j) pixel:

dMvC[i][j][0]＝dHorX×i+dVerX×(j-subheight)dMvC[i][j][0]=dHorX×i+dVerX×(j-subheight)

dMvC[i][j][1]＝dHorY×i+dVerY×(j-subheight)；dMvC[i][j][1]=dHorY×i+dVerY×(j-subheight);

否则，(i，j)像素的运动矢量偏差dMvN[i][j]：Otherwise, the motion vector deviation dMvN[i][j] of the (i, j) pixel:

dMvN[i][j][0]＝dHorX×(i–(subwidth>>1))+dVerX×(j–(subheight>>1))dMvN[i][j][0]=dHorX×(i–(subwidth>>1))+dVerX×(j–(subheight>>1))

dMvN[i][j][1]＝dHorY×(i–(subwidth>>1))+dVerY×(j–(subheight>>1))。dMvN[i][j][1]=dHorY×(i−(subwidth>>1))+dVerY×(j−(subheight>>1)).

其中，dMvX[i][j][0]表示运动矢量偏差在水平分量的偏差值，dMvX[i][j][1]表示运动矢量偏差在垂直分量的偏差值。X为A，B，C或N。Among them, dMvX[i][j][0] represents the deviation value of the motion vector deviation in the horizontal component, and dMvX[i][j][1] represents the deviation value of the motion vector deviation in the vertical component. X is A, B, C or N.

可以理解的是，在本申请的实施例中，解码器在基于差值变量确定每一个像素点的运动矢量与子块的运动矢量之间的第一运动矢量偏差之后，便可以利用子块内的全部像素点对应的全部第一运动矢量偏差，构建出该子块对应的第一运动矢量偏差矩阵。可见，第一运动矢量偏差矩阵中包括有子块与任意一个内部的像素点之间的第一运动矢量偏差，即第一运动矢量偏差。It can be understood that, in the embodiment of the present application, after determining the first motion vector deviation between the motion vector of each pixel point and the motion vector of the sub-block based on the difference variable, the decoder can use the sub-block within the All the first motion vector deviations corresponding to all the pixels of , construct the first motion vector deviation matrix corresponding to the sub-block. It can be seen that the first motion vector deviation matrix includes the first motion vector deviation between the sub-block and any internal pixel point, that is, the first motion vector deviation.

进一步地，在本申请的实施例中，如果解码器解析码流获得的二次预测参数指示不使用二次预测，那么，解码器可以选择直接将上述步骤202中所获得的当前块的子块的第一预测值作为子块的第二预测值，而不再进行步骤203至步骤206的处理。Further, in this embodiment of the present application, if the secondary prediction parameter obtained by the decoder parsing the code stream indicates that secondary prediction is not used, then the decoder can directly select the sub-block of the current block obtained in the above step 202. The first predicted value of , is used as the second predicted value of the sub-block, and the processing from step 203 to step 206 is no longer performed.

具体地，在本本申请的实施例中，如果二次预测参数指示不使用二次预测，那么解码器可以利用预测样本矩阵确定出第二预测值。其中，预测样本矩阵包括多个子块的第一预测值，解码器可以将像素点所在的子块的第一预测值确定为自身的第二预测值。Specifically, in the embodiment of the present application, if the secondary prediction parameter indicates that secondary prediction is not used, the decoder may determine the second prediction value by using the prediction sample matrix. The prediction sample matrix includes the first prediction values of multiple sub-blocks, and the decoder may determine the first prediction value of the sub-block where the pixel point is located as its own second prediction value.

示例性的，在本申请中，如果当前块的预测参考模式取值为0或1，即使用第一参考模式‘PRED_List0’，或使用第二参考模式‘PRED_List1’，那么可以直接从包括有一个亮度预测样本矩阵和两个色度预测样本矩阵的预测样本矩阵中，选择像素点所在的子块的第一预测值，并将该第一预测值确定为像素点的帧间预测值，即第二预测值。Exemplarily, in this application, if the prediction reference mode of the current block takes a value of 0 or 1, that is, the first reference mode 'PRED_List0' or the second reference mode 'PRED_List1' is used, it can be directly obtained from including a In the prediction sample matrix of the luminance prediction sample matrix and the two chrominance prediction sample matrices, the first prediction value of the sub-block where the pixel point is located is selected, and the first prediction value is determined as the inter-frame prediction value of the pixel point, that is, the first prediction value of the pixel point. Two predicted values.

示例性的，在本申请中，如果当前块的预测参考模式取值为2，即使用第三参考模式‘PRED_List01’，那么可以先对预测样本矩阵中所包括的2个亮度预测样本矩阵(2组共4个色度预测样本矩阵)进行均值运算，得到1个平均后的亮度预测样本(2个平均后的色度预测样本)，最后便从这个平均后的亮度预测样本(2个平均后的色度预测样本)中，选择像素点所在的子块的第一预测值，并将该第一预测值确定为像素点的帧间预测值，即第二预测值。Exemplarily, in this application, if the prediction reference mode of the current block is 2, that is, the third reference mode 'PRED_List01' is used, then the two luma prediction sample matrices (2 A total of 4 chrominance prediction sample matrices in the group) are averaged to obtain 1 averaged luminance prediction sample (2 averaged chrominance prediction samples), and finally from the averaged luminance prediction sample (2 averaged In the chrominance prediction sample), the first prediction value of the sub-block where the pixel point is located is selected, and the first prediction value is determined as the inter-frame prediction value of the pixel point, that is, the second prediction value.

步骤204、根据第一运动矢量偏差确定滤波参数。Step 204: Determine filtering parameters according to the first motion vector deviation.

在本申请的实施例中，解码器在确定子块的第一预测值和像素点的运动矢量与子块的运动矢量之间的第一运动矢量偏差之后，便可以根据第一运动矢量偏差进一步确定出可分离滤波器的滤波参数。In the embodiment of the present application, after determining the first prediction value of the sub-block and the first motion vector deviation between the motion vector of the pixel point and the motion vector of the sub-block, the decoder can further The filtering parameters of the separable filter are determined.

需要说明的是，在本申请的实施例中，可分离滤波器可以为二维滤波器，具体地，可分离滤波器可以为横竖可分离的二维滤波器或水平垂直可分离的二维滤波器，可以由水平方向的一维滤波器和垂直方向的一维滤波器构成。It should be noted that, in the embodiments of the present application, the separable filter may be a two-dimensional filter, and specifically, the separable filter may be a two-dimensional filter that can be separated horizontally and vertically or a two-dimensional filter that can be separated horizontally and vertically The filter can be composed of a one-dimensional filter in the horizontal direction and a one-dimensional filter in the vertical direction.

也就是说，在本申请中，如果基于像素点的二次预测使用可分离滤波器，那么该可分离滤波器可以在水平方向和垂直方向分别进行滤波处理，最终获得的滤波结果即为像素点对应的新的预测值。That is to say, in this application, if a separable filter is used for pixel-based secondary prediction, the separable filter can be filtered in the horizontal and vertical directions respectively, and the final filtering result is the pixel. the corresponding new predicted value.

可以理解的是，在本申请中，作为水平垂直可分离的可分离滤波器，可以对二维像素点在两个方向上分别进行滤波处理，具体地，可分离滤波器可以先进行一个方向(如水平方向或垂直方向)的滤波，得到该方向对应的中间值，然后对该中间值进行另一个方向(如垂直方向或水平方向)的滤波，得到最终的滤波结果。It can be understood that, in this application, as a horizontally and vertically separable separable filter, the two-dimensional pixel point can be filtered in two directions respectively. Specifically, the separable filter can be performed in one direction ( Such as horizontal direction or vertical direction) filtering, obtain the middle value corresponding to the direction, and then filter the middle value in another direction (such as vertical direction or horizontal direction) to obtain the final filtering result.

需要说明的是，在本申请中，目前多种常见的编解码场景已经使用了可分离滤波器，如帧间基于块的预测的插值滤波、帧间基于子块的预测的插值滤波，也包括上述仿射基于子块的预测的插值滤波等。It should be noted that, in this application, separable filters have been used in various common encoding and decoding scenarios, such as interpolation filtering of inter-frame block-based prediction, interpolation filtering of inter-frame sub-block-based prediction, including The above-described affine interpolation filtering and the like are based on prediction of subblocks.

进一步地，在本申请中，可分离滤波器的滤波参数可以包括该可分离滤波器的滤波器系数，还可以包括该可分离滤波器的滤波器相位，本申请不作具体限定。Further, in this application, the filtering parameters of the separable filter may include the filter coefficient of the separable filter, and may also include the filter phase of the separable filter, which is not specifically limited in this application.

需要说明的是，在本申请的实施例中，解码器在根据第一运动矢量偏差确定滤波参数时，可以先确定滤波模式；然后再基于滤波模式，利用第一运动矢量偏差确定滤波参数；其中，第一运动矢量偏差包括水平偏差和垂直偏差。It should be noted that, in the embodiment of the present application, when determining the filtering parameters according to the first motion vector deviation, the decoder may first determine the filtering mode; then based on the filtering mode, the first motion vector deviation is used to determine the filtering parameters; wherein , the first motion vector deviation includes horizontal deviation and vertical deviation.

具体地，在本申请中，滤波模式可以包括水平滤波、垂直滤波以及双向分离滤波。也就是说，在本申请中，解码器在使用可分离滤波器对子块中的像素点进行二次预测时，可以选择使用该可分离滤波器的一个方向进行滤波处理，例如，仅仅进行水平方向的滤波，或者仅仅进行垂直方向的滤波；也可以选择使用该可分离滤波器的两个方向分别进行滤波处理，例如分别进行垂直滤波和水平滤波。Specifically, in this application, the filtering modes may include horizontal filtering, vertical filtering, and bidirectional separation filtering. That is to say, in this application, when the decoder uses the separable filter to perform secondary prediction on the pixels in the sub-block, it can choose to use one direction of the separable filter to perform filtering processing, for example, only perform horizontal filtering. filtering in the vertical direction, or only filtering in the vertical direction; you can also choose to use the two directions of the separable filter to perform filtering processing separately, for example, perform vertical filtering and horizontal filtering respectively.

可以理解的是，在本申请的实施例中，解码器在确定滤波模式时，可以先确定水平参数和垂直参数；然后再将水平参数和垂直参数分别与预设水平阈值和预设垂直阈值进行比较，获得比较结果，最终可以根据比较结果确定使用的滤波模式。It can be understood that, in the embodiment of the present application, when the decoder determines the filtering mode, it can first determine the horizontal parameter and the vertical parameter; Compare, obtain the comparison result, and finally determine the filtering mode to be used according to the comparison result.

需要说明的是，在本申请的实施例中，水平参数和垂直参数可以为当前块在水平方向和垂直方向上的运动信息；水平参数和垂直参数也可以为子块在水平方向和垂直方向上的运动信息；水平参数和垂直参数还可以为像素点在水平方向和垂直方向上的运动信息。It should be noted that, in the embodiment of the present application, the horizontal parameter and the vertical parameter may be the motion information of the current block in the horizontal and vertical directions; the horizontal parameter and the vertical parameter may also be the horizontal and vertical directions of the sub-block. The motion information of the pixel point; the horizontal parameter and the vertical parameter can also be the motion information of the pixel point in the horizontal direction and the vertical direction.

具体地，在本申请中，运动信息可以为多种不同的参数，例如，运动信息可以包括当前块、或子块、或像素点在水平方向和垂直方向上的运动矢量或运动矢量偏差。Specifically, in this application, the motion information may be various parameters, for example, the motion information may include the motion vector or motion vector deviation of the current block, or sub-block, or pixels in the horizontal and vertical directions.

示例性的，在本申请中，在将水平参数和垂直参数分别与预设水平阈值和预设垂直阈值进行比较，获得比较结果之后，如果水平参数大于或者等于预设水平阈值，且垂直参数小于预设垂直阈值，那么可以确定滤波模式为水平滤波；如果水平参数小于预设水平阈值，且垂直参数大于或者等于预设垂直阈值，那么可以确定滤波模式为垂直滤波；如果水平参数大于或者等于预设水平阈值，且垂直参数大于或者等于预设垂直阈值，那么可以确定滤波模式为双向分离滤波。Exemplarily, in the present application, after comparing the horizontal parameter and the vertical parameter with the preset horizontal threshold and the preset vertical threshold, respectively, and obtaining the comparison result, if the horizontal parameter is greater than or equal to the preset horizontal threshold, and the vertical parameter is less than If the vertical threshold is preset, it can be determined that the filtering mode is horizontal filtering; if the horizontal parameter is less than the preset horizontal threshold, and the vertical parameter is greater than or equal to the preset vertical threshold, then the filtering mode can be determined to be vertical filtering; if the horizontal parameter is greater than or equal to the preset If the horizontal threshold is set, and the vertical parameter is greater than or equal to the preset vertical threshold, it can be determined that the filtering mode is bidirectional separation filtering.

可以理解的是，在本申请中，在将水平参数和垂直参数分别与预设水平阈值和预设垂直阈值进行比较，获得比较结果之后，如果水平参数小于预设水平阈值，且垂直参数小于预设垂直阈值，那么解码器可以认为不需要进行二次预测处理，从而可以不使用可分离滤波器进行滤波处理。It can be understood that, in the present application, after comparing the horizontal parameter and the vertical parameter with the preset horizontal threshold and the preset vertical threshold respectively, and obtaining the comparison result, if the horizontal parameter is less than the preset horizontal threshold, and the vertical parameter is less than the preset horizontal threshold If the vertical threshold is set, the decoder can think that the secondary prediction processing is not required, so that the filtering processing can be performed without using a separable filter.

也就是说，在本申请中，基于一定的预设条件，解码器可以选择只使用一个方向的滤波器，例如只使用水平方向的滤波器或只使用垂直方向的滤波器；也可以两个方向都不使用，即跳过二次预测。其中，该预设条件可以是基于预设水平阈值和预设垂直阈值进行比较，例如，某一个或某几个值为0，或某一个或某几个值小于(或小于等于，或大于，或大于等于)某个阈值，而比较处理的作用范围可以是整个编码单元或预测单元，即当前块，也可以是每一个子块或每一个像素点。示例性的，如果比较处理的作用范围是整个编码单元或预测单元，一种可能的限制是整个编码单元或预测单元的某一个方向的最大运动矢量偏差小于一个阈值，或某一个方向的某一个参数小于一个阈值。如果整个编码单元或预测单元的水平方向的最大运动矢量偏差小于一个阈值，那么整个编码单元或预测单元只进行垂直方向的滤波；如果整个编码单元或预测单元的垂直方向的最大运动矢量偏差小于一个阈值，那么整个编码单元或预测单元只进行水平方向的滤波；如果整个编码单元或预测单元的两个方向的最大运动矢量偏差都小于一个阈值，那么整个编码单元或预测单元水平方向和垂直方向都不进行滤波。That is to say, in this application, based on certain preset conditions, the decoder can choose to use only one filter in one direction, for example, only use the filter in the horizontal direction or only use the filter in the vertical direction; Neither are used, i.e. secondary prediction is skipped. Wherein, the preset condition may be based on the comparison between the preset horizontal threshold and the preset vertical threshold, for example, one or several values are 0, or one or several values are less than (or less than or equal to, or greater than, or greater than or equal to) a certain threshold, and the scope of action of the comparison processing may be the entire coding unit or prediction unit, that is, the current block, or may be each sub-block or each pixel. Exemplarily, if the scope of the comparison processing is the entire coding unit or the prediction unit, a possible limitation is that the maximum motion vector deviation in a certain direction of the entire coding unit or the prediction unit is less than a threshold, or a certain direction in a certain direction. The parameter is less than a threshold. If the maximum motion vector deviation in the horizontal direction of the entire coding unit or prediction unit is less than a threshold, then the entire coding unit or prediction unit is only filtered in the vertical direction; if the maximum motion vector deviation in the vertical direction of the entire coding unit or prediction unit is less than a If the maximum motion vector deviation in both directions of the entire coding unit or prediction unit is less than a threshold, then the entire coding unit or prediction unit is horizontally and vertically filtered. No filtering is performed.

可选的，在本申请中，解码器在使用运动矢量偏差进行可分离滤波器的滤波参数的确定过程中，可以对该运动矢量偏差在水平方向上的水平偏差和在垂直方向上的垂直偏差进行大小的限制。具体地，解码器可以针对水平方向和垂直方向分别设置最大值和最小值。具体地，如果水平偏差或垂直偏差大于对应方向上的最大值，那么可以将水平偏差或垂直偏差设置为对应的最大值，如果水平偏差或垂直偏差小于对应方向上的最小值，那么可以将水平偏差或垂直偏差设置为对应的最小值。其中，水平方向或垂直方向上的最大值的绝对值可以是1像素，水平方向或垂直方向上的最小值的绝对值可以是1/2像素。Optionally, in the present application, in the process of determining the filtering parameters of the separable filter using the motion vector deviation, the decoder can determine the horizontal deviation of the motion vector deviation in the horizontal direction and the vertical deviation in the vertical direction. Make size restrictions. Specifically, the decoder can set the maximum value and the minimum value for the horizontal direction and the vertical direction, respectively. Specifically, if the horizontal deviation or vertical deviation is greater than the maximum value in the corresponding direction, the horizontal deviation or vertical deviation can be set to the corresponding maximum value, and if the horizontal deviation or vertical deviation is smaller than the minimum value in the corresponding direction, then the horizontal deviation or vertical deviation can be set to the corresponding maximum value. The bias or vertical bias is set to the corresponding minimum value. The absolute value of the maximum value in the horizontal direction or the vertical direction may be 1 pixel, and the absolute value of the minimum value in the horizontal direction or the vertical direction may be 1/2 pixel.

进一步地，在本申请中，由于可分离滤波器的滤波参数可以包括多种不同类型参数，例如滤波器系数或滤波器相位，因此，解码器在进行可分离滤波器的滤波参数的确定时，可以采用多种不同的方式。Further, in this application, since the filtering parameters of the separable filter may include a variety of different types of parameters, such as filter coefficients or filter phases, when the decoder determines the filtering parameters of the separable filter, This can be done in a number of different ways.

示例性的，在本申请中，如果滤波参数为滤波器系数，那么解码器在基于滤波模式，利用第一运动矢量偏差确定滤波参数时，可以先解析码流，获取比例参数；然后可以根据比例参数和第一运动矢量偏差，按照滤波模式确定像素点对应的滤波器系数。Exemplarily, in this application, if the filter parameter is a filter coefficient, then when the decoder determines the filter parameter based on the filter mode and the first motion vector deviation, it can first parse the code stream to obtain the scale parameter; The parameter is offset from the first motion vector, and the filter coefficient corresponding to the pixel is determined according to the filter mode.

需要说明的是，在本申请中，比例参数包括至少一个比例值，其中，至少一个比例值均为非零实数。It should be noted that, in this application, the scale parameter includes at least one scale value, wherein at least one scale value is a non-zero real number.

可以理解的是，在本申请中，解码器在根据比例参数和第一运动矢量偏差，按照滤波模式确定像素点对应的滤波器系数时，如果滤波模式为水平滤波，那么解码器可以根据比例参数和水平偏差，按照预设计算规则计算滤波器系数。也就是说，在对像素点进行二次预测时，对于水平滤波的滤波模式，解码器可以利用像素点的水平偏差，按照预设计算规则来分别计算该像素点的滤波器系数和与该像素点水平相邻的相邻像素点的滤波器系数。It can be understood that, in this application, when the decoder determines the filter coefficient corresponding to the pixel point according to the filter mode according to the scale parameter and the first motion vector deviation, if the filter mode is horizontal filtering, then the decoder can be based on the scale parameter. and the horizontal deviation, the filter coefficients are calculated according to the preset calculation rules. That is to say, when performing secondary prediction on a pixel, for the filtering mode of horizontal filtering, the decoder can use the horizontal deviation of the pixel to calculate the filter coefficient of the pixel and the correlation with the pixel according to the preset calculation rule. The filter coefficients of adjacent pixels that are horizontally adjacent to each other.

可以理解的是，在本申请中，解码器在根据比例参数和第一运动矢量偏差，按照滤波模式确定像素点对应的滤波器系数时，如果滤波模式为垂直滤波，那么解码器可以根据比例参数和垂直偏差，按照预设计算规则计算滤波器系数。也就是说，在对像素点进行二次预测时，对于垂直滤波的滤波模式，解码器可以利用像素点的垂直偏差，按照预设计算规则来分别计算该像素点的滤波器系数和与该像素点垂直相邻的相邻像素点的滤波器系数。It can be understood that in this application, when the decoder determines the filter coefficients corresponding to the pixels according to the filter mode according to the scale parameter and the first motion vector deviation, if the filter mode is vertical filtering, the decoder can use the scale parameter according to the scale parameter. and vertical deviation, the filter coefficients are calculated according to the preset calculation rules. That is to say, when performing secondary prediction on a pixel, for the filtering mode of vertical filtering, the decoder can use the vertical deviation of the pixel to calculate the filter coefficient of the pixel and the correlation with the pixel according to the preset calculation rule. The filter coefficients of the adjacent pixels that are vertically adjacent to the point.

可以理解的是，在本申请中，解码器在根据比例参数和第一运动矢量偏差，按照滤波模式确定像素点对应的滤波器系数时，如果滤波模式为双向分离滤波，那么解码器可以根据比例参数、水平偏差以及垂直偏差，分别按照预设计算规则计算滤波器系数。也就是说，在对像素点进行二次预测时，对于双向分离滤波的滤波模式，解码器可以利用像素点的垂直偏差和水平偏差，按照预设计算规则来分别计算该像素点的滤波器系数，以及与该像素点水平相邻和垂直相邻的相邻像素点的滤波器系数。It can be understood that in this application, when the decoder determines the filter coefficient corresponding to the pixel point according to the filter mode according to the scale parameter and the first motion vector deviation, if the filter mode is bidirectional separation filtering, then the decoder can be based on the scale. Parameters, horizontal deviation and vertical deviation, respectively calculate filter coefficients according to preset calculation rules. That is to say, when performing secondary prediction on a pixel, for the filtering mode of bidirectional separation filtering, the decoder can use the vertical deviation and horizontal deviation of the pixel to calculate the filter coefficient of the pixel according to the preset calculation rule. , and the filter coefficients of the adjacent pixels horizontally and vertically adjacent to the pixel.

需要说明的是，在本申请中，预设计算规则可以包括多种不同的计算方式，如加法运算、减法运算、乘法运动等。其中，对于不同的像素点，可以使用不同的计算方式进行滤波器系数的计算。也就是说，在计算一个像素点的滤波器系数，以及与该像素点水平相邻(和\或垂直相邻)的相邻像素点的滤波器系数时，可以选择使用不同的计算方式。It should be noted that, in this application, the preset calculation rule may include various calculation methods, such as addition operation, subtraction operation, multiplication movement, and the like. Among them, for different pixel points, different calculation methods can be used to calculate the filter coefficients. That is to say, when calculating the filter coefficient of a pixel and the filter coefficients of the adjacent pixels horizontally adjacent to the pixel (and \ or vertically adjacent), you can choose to use different calculation methods.

进一步地，在本申请的实施例中，解码器基于像素点的第一运动矢量偏差计算获得的滤波器系数中的任意一个滤波器系数，可以为该第一运动矢量偏差的一次函数(多项式)、二次函数(多项式)或者高次函数(多项式)，本申请不作具体限定。Further, in the embodiment of the present application, any one of the filter coefficients in the filter coefficients obtained by the decoder based on the calculation of the first motion vector deviation of the pixel point may be a linear function (polynomial) of the first motion vector deviation , a quadratic function (polynomial), or a higher-order function (polynomial), which is not specifically limited in this application.

也就是说，在本申请中，解码器在按照预设计算规则中不同的计算方法计算获得多个像素点对应的多个滤波器系数中，部分滤波器系数可以为第一运动矢量偏差的一次函数(多项式)，即两者为线性关系，还可以为第一运动矢量偏差的二次函数(多项式)或高次函数(多项式)，即两者为非线性关系。That is to say, in the present application, when the decoder calculates and obtains multiple filter coefficients corresponding to multiple pixels according to different calculation methods in the preset calculation rules, some filter coefficients may be one time of the deviation of the first motion vector. The function (polynomial), that is, the two are in a linear relationship, and may also be a quadratic function (polynomial) or a higher-order function (polynomial) of the first motion vector deviation, that is, the two are in a non-linear relationship.

可见，在本申请的实施例中，在对一个像素点进行二次预测时，解码器基于比例参数和该像素点的第一运动矢量偏差，按照预设计算规则计算获得的滤波器系数是多个不同的滤波器系数，具体地，对于该像素点和与该像素点相邻的其他像素点，每个像素点对应一个滤波器系数。It can be seen that, in the embodiment of the present application, when performing secondary prediction on a pixel, the decoder calculates the filter coefficients obtained according to the preset calculation rule based on the scale parameter and the deviation of the first motion vector of the pixel. different filter coefficients, specifically, for the pixel point and other pixel points adjacent to the pixel point, each pixel point corresponds to one filter coefficient.

示例性的，在本申请中，假设像素点的第一运动矢量偏差为(dmv_x，dmv_y)，其中，如果该像素点的像素位置的坐标为(i，j)，那么dmv_x可以表示为dMvX[i][j][0]，即表示第一运动矢量偏差在水平分量的偏差值，即水平偏差，dmv_y可以表示为dMvX[i][j][1]，即表示第一运动矢量偏差在垂直分量的偏差值，即垂直偏差。这里的dmv_x，dmv_y都是其实际的大小，即dmv_x，dmv_y的1表示1个像素的距离，dmv_x，dmv_y是小数或者分数。Exemplarily, in this application, it is assumed that the first motion vector deviation of the pixel point is (dmv_x, dmv_y), wherein, if the coordinates of the pixel position of the pixel point are (i, j), then dmv_x can be expressed as dMvX[ i][j][0], which means the deviation value of the first motion vector deviation in the horizontal component, that is, the horizontal deviation, dmv_y can be expressed as dMvX[i][j][1], which means that the first motion vector deviation is in The deviation value of the vertical component, that is, the vertical deviation. Here, dmv_x and dmv_y are their actual sizes, that is, 1 of dmv_x and dmv_y represents the distance of 1 pixel, and dmv_x and dmv_y are decimals or fractions.

例如，表3和表4分别为基于第一运动矢量偏差(dmv_x，dmv_y)所获得的滤波器系数。如表3所示，对于可分离滤波器，按照像素点的第一运动矢量偏差(水平偏差为dmv_x)和不同的比例参数，如value1、value2、value3、value4、value5、value6，可以获得该像素点和2个与该像素点左、右相邻的像素点对应的3个滤波器系数。For example, Table 3 and Table 4 are the filter coefficients obtained based on the first motion vector deviations (dmv_x, dmv_y), respectively. As shown in Table 3, for the separable filter, the pixel can be obtained according to the first motion vector deviation of the pixel point (the horizontal deviation is dmv_x) and different scale parameters, such as value1, value2, value3, value4, value5, value6 point and 3 filter coefficients corresponding to 2 pixels adjacent to the left and right of the pixel.

其中，比例参数value1、value2、value3、value4、value5、value6一般是整数、小数或分数，一种可能的情况是value1、value2、value3、value4、value5、value6都是2的幂，如2，1，1/2，1/4，1/8等。Among them, the proportional parameters value1, value2, value3, value4, value5, and value6 are generally integers, decimals or fractions. A possible situation is that value1, value2, value3, value4, value5, and value6 are all powers of 2, such as 2, 1 , 1/2, 1/4, 1/8, etc.

表3table 3

像素点pixel	滤波器系数filter coefficients
左Left	dmv_x×value1+value2dmv_x×value1+value2
中心center	dmv_x×value3+value4dmv_x×value3+value4
右right	dmv_x×value5+value6dmv_x×value5+value6

如表4所示，对于可分离滤波器，按照像素点的第一运动矢量偏差(垂直偏差为dmv_y)和不同的比例参数，如value7、value8、valu9、value10、valu11、value12，可以获得该像素点和2个与该像素点上、下相邻的像素点对应的3个滤波器系数。As shown in Table 4, for the separable filter, according to the first motion vector deviation of the pixel point (the vertical deviation is dmv_y) and different scale parameters, such as value7, value8, valu9, value10, valu11, value12, the pixel can be obtained. point and 2 3 filter coefficients corresponding to the adjacent pixels above and below the pixel.

其中，比例参数value7、value8、valu9、value10、valu11、value12一般是整数、小数或分数，一种可能的情况是value7、value8、valu9、value10、valu11、value12都是2的幂，如2，1，1/2，1/4，1/8等。Among them, the proportional parameters value7, value8, valu9, value10, valu11, and value12 are generally integers, decimals or fractions. A possible situation is that value7, value8, valu9, value10, valu11, and value12 are all powers of 2, such as 2, 1 , 1/2, 1/4, 1/8, etc.

表4Table 4

像素点pixel	滤波器系数filter coefficients
上superior	dmv_y×value7+value8dmv_y×value7+value8
中心center	dmv_y×value9+value10dmv_y×value9+value10
下Down	dmv_y×value11+value12dmv_y×value11+value12

示例性的，在本申请中，解码器可以令value1为-1/2，令value4为1，令value5为1/2，其余比例参数为0，则基于表3可以获得表5如下所示：Exemplarily, in this application, the decoder can set value1 to be -1/2, set value4 to be 1, set value5 to be 1/2, and the rest of the scale parameters to be 0. Based on Table 3, Table 5 can be obtained as follows:

表5table 5

像素点pixel	滤波器系数filter coefficients
左Left	-dmv_x/2-dmv_x/2
中心 center	11
右right	dmv_x/2dmv_x/2

令value7为-1/2，令value10为1，令value11为1/2，其余比例参数为0，则基于表4可以获得表6如下所示：Let value7 be -1/2, let value10 be 1, let value11 be 1/2, and the rest of the scale parameters are 0, then based on table 4, table 6 can be obtained as follows:

表6Table 6

像素点pixel	滤波器系数filter coefficients
上superior	-dmv_y/2-dmv_y/2
中心 center	11
下Down	dmv_y/2dmv_y/2

在本申请中，可分离滤波器的滤波器系数可以通过待预测的像素点的第一运动矢量偏差确定。其中，滤波器系数可以表示为第一运动矢量偏差的多项式的形式，除了如上述表3和表4中的一次多项式，还可以为二次多项式、三次多项式等。例如，表7和表8分别为基于第一运动矢量偏差(dmv_x，dmv_y)所获得的滤波器系数。如表7所示，对于可分离滤波器，按照像素点的第一运动矢量偏差(水平偏差为dmv_x)和不同的比例参数，可以获得该像素点和2个与该像素点左、右相邻的像素点对应的3个滤波器系数。In the present application, the filter coefficient of the separable filter can be determined by the first motion vector deviation of the pixel point to be predicted. Wherein, the filter coefficients can be expressed in the form of polynomials of the first motion vector deviation, and in addition to the first-order polynomials in Tables 3 and 4 above, they can also be second-order polynomials, third-order polynomials, and the like. For example, Table 7 and Table 8 are the filter coefficients obtained based on the first motion vector deviations (dmv_x, dmv_y), respectively. As shown in Table 7, for the separable filter, according to the first motion vector deviation of the pixel point (the horizontal deviation is dmv_x) and different scale parameters, the pixel point and two adjacent to the left and right of the pixel point can be obtained. The 3 filter coefficients corresponding to the pixels of .

表7Table 7

像素点pixel	滤波器系数filter coefficients
左Left	dmv_x×dmv_x×value3+dmv_x×value4+value5dmv_x×dmv_x×value3+dmv_x×value4+value5
中心center	dmv_x×dmv_x×value6+dmv_x×value7+value8dmv_x×dmv_x×value6+dmv_x×value7+value8
右right	dmv_x×dmv_x×value1+dmv_x×value2+value9dmv_x×dmv_x×value1+dmv_x×value2+value9

如表8所示，对于可分离滤波器，按照像素点的第一运动矢量偏差(垂直偏差为dmv_y)和不同的比例参数，可以获得该像素点和2个与该像素点上、下相邻的像素点对应的3个滤波器系数。As shown in Table 8, for the separable filter, according to the first motion vector deviation of the pixel point (the vertical deviation is dmv_y) and different scale parameters, the pixel point and two adjacent pixels above and below the pixel point can be obtained. The 3 filter coefficients corresponding to the pixels of .

表8Table 8

像素点pixel	滤波器系数filter coefficients
上superior	dmv_y×dmv_y×value3+dmv_y×value4+value5dmv_y×dmv_y×value3+dmv_y×value4+value5
中心center	dmv_y×dmv_y×value6+dmv_y×value7+value8dmv_y×dmv_y×value6+dmv_y×value7+value8
下Down	dmv_y×dmv_y×value1+dmv_y×value2+value9dmv_y×dmv_y×value1+dmv_y×value2+value9

示例性的，在本申请中，解码器可以直接将中心的像素点的滤波器系数设置为1。令value3和令value1为1，令value4为-1/2，令value6为-2，令value8为1，令value2为1/2，其余比例参数为0，则基于表7可以获得表9如下所示：Exemplarily, in this application, the decoder may directly set the filter coefficient of the center pixel to 1. Let value3 and value1 be 1, let value4 be -1/2, let value6 be -2, let value8 be 1, let value2 be 1/2, and the rest of the scale parameters are 0, then based on Table 7, you can obtain Table 9 as follows Show:

表9Table 9

像素点pixel	滤波器系数filter coefficients
左Left	dmv_x×dmv_x-dmv_x/2dmv_x×dmv_x-dmv_x/2
中心center	1-2×dmv_x×dmv_x1-2×dmv_x×dmv_x
右right	dmv_x×dmv_x+dmv_x/2dmv_x×dmv_x+dmv_x/2

相应地，基于表8可以获得表10如下所示：Accordingly, based on Table 8, Table 10 can be obtained as follows:

表10Table 10

像素点pixel	滤波器系数filter coefficients
上superior	dmv_y×dmv_y-dmv_y/2dmv_y×dmv_y-dmv_y/2
中心center	1-2×dmv_y×dmv_y1-2×dmv_y×dmv_y
下Down	dmv_y×dmv_y+dmv_y/2dmv_y×dmv_y+dmv_y/2

需要说明的是，在本申请中，解码器基于像素点的第一运动矢量偏差，按照预设计算规则计算获得的滤波器系数可以认为是一个关于dmv_x或dmv_y的函数(多项式)。但是由于可分离滤波器在水平方向或垂直方向上的滤波器仅仅有3个抽头，因此，当分像素点偏离中心较多时，使用相同的计算规则，即利用相同的简单函数(如一次、二次函数)计算获得的滤波器系数准确性可能较差。It should be noted that, in this application, the filter coefficients obtained by the decoder based on the first motion vector deviation of the pixels and calculated according to the preset calculation rule can be considered as a function (polynomial) about dmv_x or dmv_y. However, since the filter of the separable filter has only 3 taps in the horizontal direction or the vertical direction, when the sub-pixel points are far from the center, the same calculation rule is used, that is, the same simple function (such as primary, secondary, etc.) is used. The filter coefficients obtained by the function) may be less accurate.

为了提高滤波器系数的准确性，在本申请中，在使用像素点的第一运动矢量偏差计算滤波器系数时，可以采用分段函数来取代原有的简单函数，也就是说，可以通过分段函数计算至少一个滤波器系数。In order to improve the accuracy of the filter coefficients, in the present application, when calculating the filter coefficients using the first motion vector deviation of the pixels, a piecewise function can be used to replace the original simple function, that is, the The segment function computes at least one filter coefficient.

可以理解的是，在本申请中，分段函数可以称作样条函数(spline function)。示例性的，当dmv_x或dmv_y的值(绝对值)小于(或等于)一个阈值时，可以使用一个函数(多项式)导出对应的滤波器系数，而当dmv_x或dmv_y的值(绝对值)大于(或等于)该阈值时，使用另外一个函数(多项式)导出对应的滤波器系数。即采用两段式的分段函数进行滤波器系数的计算。It can be understood that in this application, the piecewise function may be referred to as a spline function. Exemplarily, when the value (absolute value) of dmv_x or dmv_y is less than (or equal to) a threshold, a function (polynomial) can be used to derive the corresponding filter coefficient, and when the value (absolute value) of dmv_x or dmv_y is greater than ( or equal to) the threshold, use another function (polynomial) to derive the corresponding filter coefficients. That is, a two-stage piecewise function is used to calculate the filter coefficients.

进一步地，在本申请的实施例中，也可以采用三段式或多段式的分段函数进行滤波器系数的计算。Further, in the embodiments of the present application, a three-segment or multi-segment piecewise function may also be used to calculate the filter coefficient.

具体地，在本申请中，在计算像素点对应的多个滤波器系数中的不同滤波器系数时，可以使用相同的分段函数，也可以使用不同的分段函数；在计算像素点对应的多个滤波器系数中的不同滤波器系数时，针对dmv_x或dmv_y所使用的的阈值可能全部相同也可能不全相同。Specifically, in this application, when calculating different filter coefficients among multiple filter coefficients corresponding to a pixel, the same piecewise function or different piecewise functions can be used; For different filter coefficients among the multiple filter coefficients, the thresholds used for dmv_x or dmv_y may or may not all be the same.

进一步地，在本申请的实施例中，解码器在计算滤波器系数的过程中，还可以使用预设上限值和\或预设下限值对最终的计算结果进行限制。例如，当基于第一运动矢量偏差计算获得的滤波器系数大于或者等于预设上限值时，直接将该预设上限值作为对应的滤波器系数进行导出，或者，当基于第一运动矢量偏差计算获得的滤波器系数小于或者等于预设下限值时，直接将该预设下限值作为对应的滤波器系数进行导出。Further, in the embodiments of the present application, in the process of calculating the filter coefficients, the decoder may also use a preset upper limit value and\or a preset lower limit value to limit the final calculation result. For example, when the filter coefficient calculated based on the first motion vector deviation is greater than or equal to the preset upper limit value, the preset upper limit value is directly derived as the corresponding filter coefficient, or when the first motion vector When the filter coefficient obtained by the deviation calculation is less than or equal to the preset lower limit value, the preset lower limit value is directly derived as the corresponding filter coefficient.

可以理解的是，本申请提出的分段函数方式或限制大小方式不仅可以作用于可分离滤波器中，也可以应用于常见的二维滤波器，即水平垂直不可分离的二维滤波器，具体地，针对对滤波器的至少一个系数，可以按照分段函数进行计算，或者，可以按照预设上限值和预设下限值对最终计算结果的大小进行限制。It can be understood that the piecewise function method or the limited size method proposed in this application can not only act on separable filters, but also can be applied to common two-dimensional filters, that is, two-dimensional filters that cannot be separated horizontally and vertically. Preferably, for at least one coefficient of the filter, the calculation may be performed according to a piecewise function, or the size of the final calculation result may be limited according to a preset upper limit value and a preset lower limit value.

需要说明的是，在本申请中，对滤波器系数的大小进行限制的方式可以理解为更加具体的分段函数，因此，解码器在计算滤波器系数时，可以同时采用分段函数的方式和限制大小的方式。It should be noted that in this application, the method of limiting the size of the filter coefficients can be understood as a more specific piecewise function. Therefore, when the decoder calculates the filter coefficients, the method of the piecewise function and Ways to limit size.

示例性的，在本申请中，基于第一运动矢量偏差(dmv_x，dmv_y)，对于可分离滤波器，按照像素点的第一运动矢量偏差(水平偏差为dmv_x)和不同的比例参数，使用分段函数进行计算，可以获得该像素点和2个与该像素点左、右相邻的像素点对应的3个滤波器系数如表11所示：Exemplarily, in this application, based on the first motion vector deviation (dmv_x, dmv_y), for the separable filter, according to the first motion vector deviation of the pixel point (the horizontal deviation is dmv_x) and different scale parameters, use the separation filter. The segment function is calculated, and the pixel point and the 3 filter coefficients corresponding to the 2 pixel points adjacent to the left and right of the pixel point can be obtained as shown in Table 11:

表11Table 11

示例性的，在本申请中，对于可分离滤波器，按照像素点的第一运动矢量偏差(垂直偏差为dmv_y)和不同的比例参数，使用分段函数进行计算，可以获得该像素点和2个与该像素点上、下相邻的像素点对应的3个滤波器系数如表12所示：Exemplarily, in this application, for the separable filter, the pixel point and 2 The three filter coefficients corresponding to the adjacent pixels above and below the pixel are shown in Table 12:

表12Table 12

需要说明的是，在本申请的实施例中，目前常见的8抽头的滤波器所对应的整像素点和分像素点的运动矢量在水平和垂直方向均为非负的，且大小均属于0像素到1像素之间，即dmv_x，dmv_y不可以为负的。而在本申请中，可分离滤波器对应的整像素点和分像素点的运动矢量在水平和垂直方向都可以为负的，即dmv_x，dmv_y可以为负的。It should be noted that, in the embodiments of the present application, the motion vectors of the integer pixel and sub-pixel corresponding to the common 8-tap filter are non-negative in the horizontal and vertical directions, and the size is 0 Between pixel and 1 pixel, that is, dmv_x, dmv_y cannot be negative. In this application, the motion vectors of the whole pixel and the sub-pixel corresponding to the separable filter may be negative in both the horizontal and vertical directions, that is, dmv_x and dmv_y may be negative.

在本申请的实施例中，对不同的子块，滤波器可以使用不同的多项式或不同的比例参数。一个例子是，4x4的子块使用如表3表4所示的一次函数导出滤波器系数，8x8的子块使用如表7表8所示的二次函数导出滤波器系数。一个具体的例子是4x4的子块使用如表5表6所示的一次函数导出滤波器系数，8x8的子块使用如表9表10所示的二次函数导出滤波器系数。另一个具体的例子是4x4的子块使用如表5表6所示的一次函数导出滤波器系数，8x8的子块使用如表13表14所示的一次函数导出滤波器系数。上述不同的子块可以是不同大小的子块，也可以是不同位置的子块，如左上角的子块A，右上角的子块B，左下角的子块C和其他子块。In the embodiment of the present application, for different sub-blocks, the filter may use different polynomials or different scale parameters. An example is that the 4x4 sub-block uses a linear function as shown in Table 3 and Table 4 to derive filter coefficients, and the 8x8 sub-block uses a quadratic function as shown in Table 7 and Table 8 to derive filter coefficients. A specific example is that the 4x4 sub-block uses a linear function as shown in Table 5 and Table 6 to derive filter coefficients, and the 8x8 sub-block uses a quadratic function as shown in Table 9 and Table 10 to derive filter coefficients. Another specific example is that the 4x4 sub-block uses the linear functions shown in Table 5 and Table 6 to derive the filter coefficients, and the 8x8 sub-block uses the linear functions shown in Table 13 and Table 14 to derive the filter coefficients. The above-mentioned different sub-blocks may be sub-blocks of different sizes or sub-blocks of different positions, such as sub-block A in the upper left corner, sub-block B in the upper-right corner, sub-block C in the lower-left corner and other sub-blocks.

解码器可以令value1为-1/4，令value4为1，令value5为1/4，其余比例参数为0，则基于表3可以获得表13如下所示：The decoder can set value1 to be -1/4, value4 to be 1, value5 to be 1/4, and the rest of the scale parameters to be 0. Based on Table 3, Table 13 can be obtained as follows:

表13Table 13

像素位置pixel position	滤波器系数filter coefficients
左Left	-dmv_x/4-dmv_x/4
中心 center	11
右right	dmv_x/4dmv_x/4

令value7为-1/4，令value10为1，令value11为1/4，其余比例参数为0，则基于表4可以获得表14如下所示：Let value7 be -1/4, let value10 be 1, let value11 be 1/4, and the rest of the scale parameters are 0, then based on Table 4, Table 14 can be obtained as follows:

表14Table 14

像素位置pixel position	滤波器系数filter coefficients
上superior	-dmv_y/4-dmv_y/4
中心 center	11
下Down	dmv_y/4dmv_y/4

进一步地，在本申请中，如果滤波参数为滤波器相位，那么解码器在基于滤波模式，利用第一运动矢量偏差确定滤波参数时，可以先确定偏差与相位的映射表，然后再根据偏差与相位的映射表和第一运动矢量偏差，按照滤波模式确定像素点对应的滤波器相位。Further, in the present application, if the filter parameter is the filter phase, then when the decoder determines the filter parameter based on the filter mode and utilizes the first motion vector deviation, it can first determine the mapping table of the deviation and the phase, and then according to the deviation and The phase mapping table and the first motion vector deviation determine the filter phase corresponding to the pixel point according to the filter mode.

需要说明的是，在本申请中，解码器在确定偏差与相位的映射表时，既可以通过训练获得偏差与相位的映射表，也可以计算获得偏差与相位的映射表。It should be noted that, in the present application, when the decoder determines the mapping table of deviation and phase, it can obtain the mapping table of deviation and phase through training, and can also obtain the mapping table of deviation and phase by calculation.

也就是说，在本申请的实施例中，偏差与相位的映射表既可以是通过一定的计算公式推导获得的，也可以是直接训练获得的。其中，在计算获得偏差与相位的映射表时，也可以按照分段函数的方式进行推导，还可以根据其他更加复杂的公式推导出来。当然，也可以有不必根据公式推导出来的滤波器相位，而是训练出来的滤波器相位。That is to say, in the embodiment of the present application, the mapping table of deviation and phase may be obtained by deduction through a certain calculation formula, or obtained by direct training. Wherein, when calculating and obtaining the mapping table of deviation and phase, it can also be deduced in the manner of piecewise function, and can also be deduced according to other more complex formulas. Of course, there may also be a filter phase that is not necessarily derived from the formula, but a trained filter phase.

可以理解的是，在本申请中，解码器在根据偏差与相位的映射表和第一运动矢量偏差，按照滤波模式确定像素点对应的滤波器相位时，如果滤波模式为水平滤波，那么解码器可以按照水平偏差查询偏差与相位的映射表，获得滤波器相位。也就是说，在对像素点进行二次预测时，对于水平滤波的滤波模式，解码器可以利用像素点的水平偏差，直接从偏差与相位的映射表中查询获得该像素点的滤波器相位和与该像素点水平相邻的相邻像素点的滤波器相位。It can be understood that, in this application, when the decoder determines the filter phase corresponding to the pixel point according to the filter mode according to the mapping table of deviation and phase and the first motion vector deviation, if the filter mode is horizontal filtering, then the decoder The filter phase can be obtained by querying the mapping table of deviation and phase according to the horizontal deviation. That is to say, when performing secondary prediction on a pixel, for the filtering mode of horizontal filtering, the decoder can use the horizontal deviation of the pixel to directly query the mapping table of deviation and phase to obtain the filter phase sum of the pixel. The filter phase of the adjacent pixels horizontally adjacent to this pixel.

可以理解的是，在本申请中，解码器在根据偏差与相位的映射表和第一运动矢量偏差，按照滤波模式确定像素点对应的滤波器相位时，如果滤波模式为垂直滤波，那么解码器可以按照垂直偏差查询偏差与相位的映射表，获得滤波器相位。也就是说，在对像素点进行二次预测时，对于垂直滤波的滤波模式，解码器可以利用像素点的垂直偏差，直接从偏差与相位的映射表中查询获得该像素点的滤波器相位和与该像素点垂直相邻的相邻像素点的滤波器相位。It can be understood that, in this application, when the decoder determines the filter phase corresponding to the pixel point according to the filter mode according to the mapping table of deviation and phase and the first motion vector deviation, if the filter mode is vertical filtering, then the decoder The filter phase can be obtained by querying the mapping table of deviation and phase according to the vertical deviation. That is to say, when performing secondary prediction on pixels, for the filtering mode of vertical filtering, the decoder can use the vertical deviation of the pixel to directly query the mapping table of deviation and phase to obtain the filter phase and The filter phase of the adjacent pixels vertically adjacent to this pixel.

可以理解的是，在本申请中，解码器在根据偏差与相位的映射表和第一运动矢量偏差，按照滤波模式确定像素点对应的滤波器相位时，如果滤波模式为双向分离滤波，那么解码器可以按照水平偏差和垂直偏差，分别查询偏差与相位的映射表，获得滤波器相位。也就是说，在对像素点进行二次预测时，对于双向分离滤波的滤波模式，解码器可以利用像素点的垂直偏差和水平偏差，直接从偏差与相位的映射表中查询获得该像素点的滤波器相位、与该像素点垂直相邻和水平相邻的相邻像素点的滤波器相位。It can be understood that, in this application, when the decoder determines the filter phase corresponding to the pixel point according to the filtering mode according to the mapping table of deviation and phase and the first motion vector deviation, if the filtering mode is bidirectional separation filtering, then decoding According to the horizontal deviation and vertical deviation, the filter can query the mapping table of deviation and phase respectively to obtain the filter phase. That is to say, when performing secondary prediction on pixels, for the filtering mode of bidirectional separation filtering, the decoder can use the vertical deviation and horizontal deviation of the pixel to directly query the mapping table of deviation and phase to obtain the pixel's value. Filter phase, the filter phase of the adjacent pixels vertically and horizontally adjacent to this pixel.

在本申请的实施例中，如果运动矢量偏差的精度较高，或者在进行滤波时使用的运动矢量偏差的精度较高，或者dmv_x和/或dmv_y的可能的值很多，或者根据运动矢量偏差计算系数复杂度不高，那么采用通过运动矢量偏差和比例参数计算滤波器系数的方式进行滤波参数的确定比较合理。然而，如果运动矢量偏差的精度不高，或者在进行滤波时使用的运动矢量偏差的精度不高，或者说dmv_x和/或dmv_y的可能的值不是很多，或者根据运动矢量偏差计算系数复杂度较高，那么可以优选将滤波参数确定为滤波器相位，即将可分离滤波器做成多个相位，每个相位有一组固定的系数的形式。In the embodiment of the present application, if the precision of the motion vector deviation is high, or the precision of the motion vector deviation used in filtering is high, or there are many possible values of dmv_x and/or dmv_y, or the calculation is performed according to the motion vector deviation If the coefficient complexity is not high, it is reasonable to determine the filter parameters by calculating the filter coefficients through the motion vector deviation and the scale parameter. However, if the accuracy of the motion vector offset is not high, or the accuracy of the motion vector offset used in filtering is not high, or the possible values of dmv_x and/or dmv_y are not many, or the calculation of the coefficients based on the motion vector offset is more complex If the filter parameter is high, then the filter parameter can be preferably determined as the filter phase, that is, the separable filter is made into a plurality of phases, and each phase has the form of a set of fixed coefficients.

具体地，在本申请中，dmv_x或dmv_y的每一个值对应一个相位，其中，解码器可以根据dmv_x或dmv_y的取值确定二次预测处理时使用的相位。如果滤波器中间的系数是常数，那么也可以用dmv_x或dmv_y的每一绝对值对应一个相位，是负数的话，与相同绝对值的正数相比把上下或左右的滤波器系数对掉即可。Specifically, in this application, each value of dmv_x or dmv_y corresponds to a phase, wherein the decoder can determine the phase used in the secondary prediction process according to the value of dmv_x or dmv_y. If the coefficient in the middle of the filter is constant, then each absolute value of dmv_x or dmv_y can also be used to correspond to a phase. If it is a negative number, compare the upper and lower or left and right filter coefficients with the positive number of the same absolute value. .

需要说明的是，在本申请中，如果dmv_x和/或dmv_y的可能的值很多，可以通过降低精度，如从1/64像素精度降低到1/32像素精度，或限制最大值，如原来最大值为1像素，现在将最大值限制在1/2像素等来减少dmv_x和/或dmv_y的可能的值，从而避免滤波器的相位数过多。It should be noted that in this application, if there are many possible values of dmv_x and/or dmv_y, the precision can be reduced, such as from 1/64 pixel precision to 1/32 pixel precision, or the maximum value can be limited, such as the original maximum A value of 1 pixel, now limits the maximum value to 1/2 pixel etc. to reduce possible values of dmv_x and/or dmv_y, thus avoiding too many phases for the filter.

示例性的，如果进行滤波时的运动矢量偏差精度是1/16像素精度，最大值是16/16像素，最小值是-16/16像素，那么总共有33个可能的dmv_x或dmv_y的值。基于上述表11和表12所示的分段函数，解码器按照水平偏差和垂直偏差计算获得偏差与相位的映射表如表15所示，其中，对于水平方向，dmv对应像素点在水平方向的运动矢量差值dmv_x，coef0是该像素点左边的相邻像素点的系数，coef1是该像素点的系数，coef2是是该像素点右边的相邻像素点的系数。对于垂直方向，dmv对应像素点在垂直方向的运动矢量差值dmv_y，coef0是该像素点上边的相邻像素点的系数，coef1是该像素点的系数，coef2是该像素点下边的相邻像素点的系数。Exemplarily, if the motion vector bias precision during filtering is 1/16 pixel precision, the maximum value is 16/16 pixels, and the minimum value is -16/16 pixels, then there are a total of 33 possible values of dmv_x or dmv_y. Based on the piecewise functions shown in Table 11 and Table 12 above, the decoder calculates and obtains the mapping table of deviation and phase according to the horizontal deviation and vertical deviation, as shown in Table 15. Among them, for the horizontal direction, the dmv corresponds to the pixel in the horizontal direction. The motion vector difference dmv_x, coef0 is the coefficient of the adjacent pixel to the left of the pixel, coef1 is the coefficient of the pixel, and coef2 is the coefficient of the adjacent pixel to the right of the pixel. For the vertical direction, the motion vector difference dmv_y of the pixel corresponding to dmv in the vertical direction, coef0 is the coefficient of the adjacent pixel above the pixel, coef1 is the coefficient of the pixel, and coef2 is the adjacent pixel below the pixel point coefficient.

进一步地，在本申请的实施例中，解码器可以确定运动矢量偏差的精度参数，并基于精度参数确定缩小参数；从而可以在利用滤波器相位进行滤波处理之后，按照缩小参数进行缩小处理和/或右移处理。Further, in the embodiment of the present application, the decoder can determine the precision parameter of the motion vector deviation, and determine the reduction parameter based on the precision parameter; thus, after performing the filtering processing with the filter phase, the reduction processing and/or reduction can be performed according to the reduction parameter. or right-shift processing.

可以理解的是，在本申请中，如果进行滤波时的运动矢量偏差精度是1/16像素精度，那么滤波器系数都放大了256倍，因此，在完成滤波处理，获得滤波结果之后，需要将滤波结果缩小256倍，或右移8位。It can be understood that, in this application, if the motion vector deviation precision during filtering is 1/16 pixel precision, then the filter coefficients are enlarged by 256 times. Therefore, after the filtering process is completed and the filtering result is obtained, it is necessary to The filtered result is scaled down by a factor of 256, or shifted right by 8 bits.

进一步地，在本申请中，缩小或右移可以在单向滤波后分别进行；也可以在水平垂直滤波都处理完后进行；其中，如果在单向滤波后分别进行缩小或右移，那么需要调整缩小的倍数或右移的位数，从而保证总的缩小倍数或右移位数一致与滤波器系数的放大倍数一致。Further, in the present application, scaling down or right shifting can be performed after unidirectional filtering; it can also be performed after both horizontal and vertical filtering are processed; wherein, if scaling down or right shifting is performed after unidirectional filtering, then it is necessary to Adjust the reduction factor or the number of right shift bits to ensure that the total reduction factor or right shift number is the same as the filter coefficient magnification.

表15Table 15

dmvdmv	coef0coef0	coef1coef1	coef2coef2
-16-16	256256	00	00
-15-15	240240	1616	00
-14-14	224224	3232	00
-13-13	208208	4848	00
-12-12	192192	6464	00
-11-11	176176	8080	00
-10-10	160160	9696	00
-9-9	144144	112112	00
-8-8	128128	128128	00
-7-7	105105	158158	-7-7
-6-6	8484	184184	-12-12
-5-5	6565	206206	-15-15
-4-4	4848	224224	-16-16
-3-3	3333	238238	-15-15
-2-2	2020	248248	-12-12
-1-1	99	254254	-7-7
00	00	256256	00
11	-7-7	254254	99
22	-12-12	248248	2020
33	-15-15	238238	3333
44	-16-16	224224	4848
55	-15-15	206206	6565
66	-12-12	184184	8484
77	-7-7	158158	105105
88	00	128128	128128
99	00	112112	144144
1010	00	9696	160160
1111	00	8080	176176
1212	00	6464	192192
1313	00	4848	208208
1414	00	3232	224224
1515	00	1616	240240
1616	00	00	256256

在本申请的实施例中，对不同的子块，滤波器可以使用不同的映射表。一个例子是，4x4的子块使用如表15所示的映射表，8x8的子块使用另外的映射表。上述不同的子块可以是不同大小的子块，也可以是不同位置的子块，如左上角的子块A，右上角的子块B，左下角的子块C和其他子块。In the embodiments of the present application, the filters may use different mapping tables for different sub-blocks. As an example, 4x4 sub-blocks use the mapping table shown in Table 15, and 8x8 sub-blocks use another mapping table. The above-mentioned different sub-blocks may be sub-blocks of different sizes or sub-blocks of different positions, such as sub-block A in the upper left corner, sub-block B in the upper-right corner, sub-block C in the lower-left corner and other sub-blocks.

可见，在本申请的实施例中，可分离滤波器的滤波参数是与像素点所对应的第一运动矢量偏差相关的。也就是说，对于不同的像素点，如果对应的第一运动矢量偏差不同，那么使用的可分离滤波器的滤波参数也是不同的。It can be seen that, in the embodiment of the present application, the filtering parameter of the separable filter is related to the deviation of the first motion vector corresponding to the pixel point. That is to say, for different pixels, if the corresponding first motion vector deviations are different, the filtering parameters of the separable filters used are also different.

可以理解的是，在本申请的实施例中，可分离滤波器用于利用与一个像素点水平和/或垂直相邻的多个相邻像素点对该像素点进行二次预测。也就是说，在本申请的中，用于进行二次预测的可分离滤波器是水平和/或垂直相邻像素点所构成的滤波器。It can be understood that, in the embodiments of the present application, the separable filter is used to perform secondary prediction on a pixel by using a plurality of adjacent pixels that are horizontally and/or vertically adjacent to the pixel. That is to say, in the present application, the separable filter used for secondary prediction is a filter formed by horizontally and/or vertically adjacent pixels.

步骤205、基于滤波参数和第一预测值，利用可分离滤波器进行滤波处理，获得像素点的预测值。 Step 205 , based on the filtering parameters and the first predicted value, perform filtering processing with a separable filter to obtain the predicted value of the pixel point.

步骤206、根据像素点的预测值，确定子块的第二预测值；其中，第二预测值用于确定子块的重建值。Step 206: Determine a second predicted value of the sub-block according to the predicted value of the pixel point; wherein, the second predicted value is used to determine the reconstructed value of the sub-block.

在本申请的实施例中，解码器在根据第一运动矢量偏差确定出可分离滤波器的滤波参数之后，便可以基于滤波参数和第一预测值，利用可分离滤波器进行滤波处理，获得像素点的预测值。进而可以根据像素点的预测值，确定出子块的第二预测值。In the embodiment of the present application, after the decoder determines the filtering parameters of the separable filter according to the first motion vector deviation, it can perform filtering processing by using the separable filter based on the filtering parameters and the first predicted value to obtain pixels point forecast. Further, the second predicted value of the sub-block may be determined according to the predicted value of the pixel point.

需要说明的是，在本申请中，解码器可以对子块中的全部像素点进行滤波处理，利用子块中的全部像素点的预测值进行子块的第二预测值的确定，也可以对子块中的部分像素点进行滤波处理，利用子块中的部分像素点的预测值进行子块的第二预测值的确定。It should be noted that in this application, the decoder may perform filtering processing on all the pixels in the sub-block, and use the predicted values of all the pixels in the sub-block to determine the second predicted value of the sub-block, or it may Filter processing is performed on some pixels in the sub-block, and the second prediction value of the sub-block is determined by using the prediction values of some pixels in the sub-block.

可以理解的是，在本申请的实施例中，解码器利用像素点所对应的第一运动矢量偏差确定出滤波参数，从而可以按照滤波参数，通过可分离滤波器实现像素点的滤波处理，进而实现对第一预测值进行修正，获得修正后的、子块的第二预测值。可见，第二预测值为基于第一预测值的修正后的预测值，同时，第二预测值用于确定子块的重建值。It can be understood that, in the embodiment of the present application, the decoder uses the first motion vector deviation corresponding to the pixel to determine the filtering parameter, so that the filtering process of the pixel can be realized by the separable filter according to the filtering parameter, and then The first predicted value is modified to obtain the modified second predicted value of the sub-block. It can be seen that the second predicted value is a modified predicted value based on the first predicted value, and at the same time, the second predicted value is used to determine the reconstructed value of the sub-block.

进一步地，在本申请的实施例中，解码器在基于滤波参数和第一预测值，利用可分离滤波器进行滤波处理，获得像素点的预测值时，可以先对滤波参数，如滤波器系数，与子块的第一预测值进行乘法运算，获得乘积结果，即像素点的预测值。然后可以在遍历子块中的全部像素点或部分像素点之后，再对子块的全部像素点或部分像素点的预测值进行加法运算，获得求和结果，最后可以对加法结果进行归一化处理，最终便可以获得子块修正后的第二预测值。Further, in the embodiment of the present application, when the decoder performs filtering processing with a separable filter based on the filtering parameters and the first predicted value, and obtains the predicted value of the pixel point, it can , and perform a multiplication operation with the first predicted value of the sub-block to obtain the multiplication result, that is, the predicted value of the pixel point. Then, after traversing all the pixels or part of the pixels in the sub-block, the predicted values of all the pixels or part of the pixels in the sub-block can be added to obtain the summation result, and finally the addition result can be normalized After processing, finally, the modified second prediction value of the sub-block can be obtained.

需要说明的是，在本申请的实施例中，在进行二次预测之前，一般情况下是将像素点所在的子块的第一预测值作为该像素点的修正前的预测值，因此，在通过可分离滤波器进行滤波时，可以将滤波参数与对应像素点的预测值，即第一预测值相乘，并对全部像素点或部分像素点对应的乘积结果进行累加，然后归一化。It should be noted that, in the embodiments of the present application, before the secondary prediction is performed, the first prediction value of the sub-block where the pixel is located is generally used as the prediction value before the correction of the pixel. Therefore, in the When filtering is performed by a separable filter, the filtering parameter can be multiplied by the predicted value of the corresponding pixel, that is, the first predicted value, and the multiplication results corresponding to all or some of the pixels can be accumulated, and then normalized.

可见，在本申请中，在获得子块内部的像素点对应的第一运动矢量偏差之后，对每一个子块，以及每一个子块中的全部像素点或部分像素点，可以根据第一运动矢量偏差，基于子块的运动补偿的第一预测值，使用可分离滤波器进行滤波，完成对子块的二次预测，得到第二预测值。It can be seen that in the present application, after obtaining the first motion vector deviation corresponding to the pixels inside the sub-block, for each sub-block and all the pixels or part of the pixels in each sub-block, according to the first motion The vector deviation, based on the first prediction value of the motion compensation of the sub-block, is filtered using a separable filter to complete the secondary prediction of the sub-block to obtain the second prediction value.

也就是说，在本申请中，解码器在基于子块的预测之后，在基于子块的预测的基础上，可以再次进行基于点的二次预测。基于点的二次预测可以使用水平垂直可分离的二维滤波器，即可分离滤波器。具体地，可分离滤波器可以分为水平方向的一维滤波器和垂直方向的一维滤波器。对一个像素点，可分离滤波器进行滤波处理后的结果为该像素点的新的预测值。That is to say, in the present application, after the sub-block-based prediction, the decoder can perform point-based secondary prediction again on the basis of the sub-block-based prediction. Point-based secondary prediction can use two-dimensional filters that are horizontally and vertically separable, i.e., separable filters. Specifically, the separable filters can be divided into one-dimensional filters in the horizontal direction and one-dimensional filters in the vertical direction. For a pixel, the result of filtering the separable filter is the new predicted value of the pixel.

进一步地，在本申请的实施例中，解码器在基于滤波参数和第一预测值，利用可分离滤波器进行滤波处理，获得像素点的预测值时，如果滤波模式为水平滤波，那么解码器可以先基于滤波参数，使用可分离滤波器对第一预测值在水平方向进行滤波处理，获得像素点的预测值；并遍历当前子块中的全部像素点或部分像素点，根据全部像素点或部分像素点对应的的预测值获得第二预测值。Further, in the embodiment of the present application, when the decoder uses a separable filter to perform filtering processing based on the filtering parameters and the first predicted value to obtain the predicted value of the pixel, if the filtering mode is horizontal filtering, the decoder Based on the filtering parameters, a separable filter can be used to filter the first predicted value in the horizontal direction to obtain the predicted value of the pixel point; The second predicted value is obtained from the predicted value corresponding to some pixel points.

进一步地，在本申请的实施例中，解码器在基于滤波参数和第一预测值，利用可分离滤波器进行滤波处理，获得像素点的预测值时，如果滤波模式为垂直滤波，那么解码器可以先基于滤波参数，使用可分离滤波器对第一预测值在垂直方向进行滤波处理，获得像素点的预测值；并遍历当前子块中的全部像素点或部分像素点，根据全部像素点对应的预测值获得第二预测值。Further, in the embodiment of the present application, when the decoder uses a separable filter to perform filtering processing based on the filtering parameters and the first predicted value to obtain the predicted value of the pixel point, if the filtering mode is vertical filtering, then the decoder Based on the filtering parameters, a separable filter can be used to filter the first predicted value in the vertical direction to obtain the predicted value of the pixel point; The predicted value of , obtains the second predicted value.

进一步地，在本申请的实施例中，解码器在基于滤波参数和第一预测值，利用可分离滤波器进行滤波处理，获得像素点的预测值时，如果滤波模式为双向分离滤波，那么解码器可以先基于滤波参数，使用可分离滤波器对第一预测值在水平方向进行滤波处理，获得水平滤波结果；然后可以基于滤波参数，对水平滤波结果在垂直方向进行滤波处理，获得像素点的预测值；最终可以遍历当前子块中的全部像素点或部分像素点，根据全部像素点或部分像素点对应的预测值获得第二预测值。Further, in the embodiment of the present application, when the decoder uses a separable filter to perform filtering processing based on the filtering parameters and the first predicted value to obtain the predicted value of the pixel point, if the filtering mode is bidirectional separation filtering, then decoding The filter can first perform filtering processing on the first predicted value in the horizontal direction based on the filtering parameters using a separable filter to obtain the horizontal filtering result; Predicted value; finally, all the pixels or part of the pixels in the current sub-block can be traversed, and the second predicted value can be obtained according to the predicted values corresponding to all the pixels or part of the pixels.

进一步地，在本申请的实施例中，解码器在基于滤波参数和第一预测值，利用可分离滤波器进行滤波处理，获得像素点的预测值时，如果滤波模式为双向分离滤波，那么解码器可以先基于滤波参数，使用可分离滤波器对第一预测值在垂直方向进行滤波处理，获得垂直滤波结果；然后可以基于滤波参数，对垂直滤波结果在水平方向进行滤波处理，获得像素点的预测值；最终可以遍历当前子块中的全部像素点或部分像素点，根据全部像素点或部分像素点对应的预测值获得第二预测值。Further, in the embodiment of the present application, when the decoder uses a separable filter to perform filtering processing based on the filtering parameters and the first predicted value to obtain the predicted value of the pixel point, if the filtering mode is bidirectional separation filtering, then decoding The filter can first perform filtering processing on the first predicted value in the vertical direction based on the filtering parameters using a separable filter to obtain the vertical filtering result; Predicted value; finally, all the pixels or part of the pixels in the current sub-block can be traversed, and the second predicted value can be obtained according to the predicted values corresponding to all the pixels or part of the pixels.

也就是说，在本申请的实施例中，解码器在使用可分离滤波器进行滤波处理时，既可以先进行水平方向的滤波处理，然后再对水平方向的滤波结果进行垂直方向的滤波处理，获得最终的滤波结果。还可以先进行垂直方向的滤波处理，然后再对垂直方向的滤波结果进行水平方向的滤波处理，获得最终的滤波结果。That is to say, in the embodiment of the present application, when the decoder uses the separable filter to perform filtering, it may first perform filtering in the horizontal direction, and then perform filtering in the vertical direction on the filtering result in the horizontal direction. Get the final filtered result. It is also possible to perform filtering processing in the vertical direction first, and then perform filtering processing in the horizontal direction on the filtering result in the vertical direction to obtain the final filtering result.

图9为滤波处理的示意图一，如图9所示，以3x3的可分离滤波器为例，水平方向是一个3抽头的滤波器，垂直方向是一个3抽头的滤波器，正方形是基于子块的预测的结果的整像素点，在使用可分离滤波器进行滤波处理时，可以先进行水平方向的滤波处理，得到3个水平方向的分像素点的值，可以表示为圆形，然后可以再对上述3个水平方向的分像素点的值进行垂直方向的滤波处理，最终得到所需的分像素点的值，即三角形。Figure 9 is a schematic diagram of filtering processing. As shown in Figure 9, taking a 3x3 separable filter as an example, the horizontal direction is a 3-tap filter, the vertical direction is a 3-tap filter, and the square is based on sub-blocks When using a separable filter for filtering, the whole pixel of the predicted result can be filtered in the horizontal direction first, and the values of three sub-pixels in the horizontal direction can be obtained, which can be expressed as a circle, and then can be re- Perform vertical filtering on the values of the three horizontal sub-pixel points, and finally obtain the required sub-pixel value, that is, a triangle.

图10为滤波处理的示意图二，如图10所示，在使用可分离滤波器进行滤波处理时，可以先进行垂直方向的滤波处理，得到3个垂直方向的分像素点的值，可以表示为圆形，然后可以再对上述3个垂直方向的分像素点的值进行水平方向的滤波处理，最终得到所需的分像素点的值，即三角形。Figure 10 is the second schematic diagram of the filtering process. As shown in Figure 10, when using the separable filter to perform the filtering process, the filtering process in the vertical direction can be performed first, and the values of the three sub-pixel points in the vertical direction can be obtained, which can be expressed as Then, the values of the three sub-pixel points in the vertical direction can be filtered in the horizontal direction to finally obtain the required value of the sub-pixel points, that is, a triangle.

进一步地，在本申请的实施例中，帧间预测可以包括单向预测和双向预测，本申请提出的帧间预测方法既可以应用到单向预测的场景中，也可以应用到双向预测的场景中。Further, in the embodiments of the present application, inter-frame prediction may include unidirectional prediction and bidirectional prediction, and the inter-frame prediction method proposed in this application can be applied to both unidirectional prediction scenarios and bidirectional prediction scenarios. middle.

具体地，在本申请中，对于双向预测场景，解码器在进行帧间预测时，可以先解析码流，确定帧间预测方向；若帧间预测方向为双向预测，则基于第一预测方向，确定子块的第四预测值和像素点的运动矢量与子块的运动矢量之间的第四运动矢量偏差；基于第二预测方向，确定子块的第五预测值和像素点的运动矢量与子块的运动矢量之间的第五运动矢量偏差；基于第四运动矢量偏差，使用可分离滤波器对第四预测值进行二次预测，获得第六预测值；基于第五运动矢量偏差，使用可分离滤波器对第五预测值进行二次预测，获得第七预测值；最后便可以根据第六预测值和第七预测值，确定子块的帧间预测值。Specifically, in this application, for a bidirectional prediction scenario, when performing inter prediction, the decoder can first parse the code stream to determine the inter prediction direction; if the inter prediction direction is bidirectional prediction, based on the first prediction direction, Determine the fourth prediction value of the sub-block and the fourth motion vector deviation between the motion vector of the pixel point and the motion vector of the sub-block; based on the second prediction direction, determine the fifth prediction value of the sub-block and the motion vector of the pixel point and The fifth motion vector deviation between the motion vectors of the sub-blocks; based on the fourth motion vector deviation, use a separable filter to perform secondary prediction on the fourth predicted value to obtain the sixth predicted value; based on the fifth motion vector deviation, use The separable filter performs secondary prediction on the fifth predicted value to obtain the seventh predicted value; finally, the inter-frame predicted value of the sub-block can be determined according to the sixth predicted value and the seventh predicted value.

其中，第一预测方向和第二预测方向可以分别为前向预测和后向预测。The first prediction direction and the second prediction direction may be forward prediction and backward prediction, respectively.

可以理解的是，在本申请的实施例中，解码器在根据第六预测值和第七预测值，确定子块的帧间预测值时，可以直接对第六预测值和第七预测值进行平均计算，将获得的平均值作为子块的帧间预测值，也可以对第六预测值和第七预测值进行加权平均计算，将获得的计算加权平均结果作为子块的帧间预测值。It can be understood that, in the embodiment of the present application, when determining the inter-frame prediction value of the sub-block according to the sixth prediction value and the seventh prediction value, the decoder can directly perform the sixth prediction value and the seventh prediction value. In the average calculation, the obtained average value is used as the inter-frame prediction value of the sub-block, and the sixth prediction value and the seventh prediction value can be weighted average calculation, and the obtained weighted average calculation result is used as the inter-frame prediction value of the sub-block.

由此可见，本申请提出的帧间预测方法可以只用于单向预测，也可以单向预测和双向预测。图11为双向预测场景中的二次预测示意图，如图11所示，在用于当前块的双向预测时，对当前块的每一个子块，基于第一预测方向和第二预测方向分别获得每一个子块的预测块A和预测块B，然后分别对预测块A0和预测块B0使用本申请的可分离滤波器，二测预测后得到每一个单向预测的二次预测的预测块A1和预测块B1，然后对两个单向预测的二次预测的预测块A1和预测块B1进行平均或加权平均得到双向预测的预测块C。其中，平均或加权平均都可以在需要进行平均或加权平均的整个预测块都得出来以后再进行平均或加权平均，也可以在需要进行平均或加权平均的子块的预测块都得出来以后再进行平均或加权平均，也可以在需要进行平均或加权平均的点的预测块都得出来以后再进行平均或加权平均，本申请不作具体限定。It can be seen that the inter-frame prediction method proposed in this application can be used only for unidirectional prediction, and can also be used for unidirectional prediction and bidirectional prediction. FIG. 11 is a schematic diagram of secondary prediction in a bidirectional prediction scenario. As shown in FIG. 11 , when it is used for bidirectional prediction of the current block, for each sub-block of the current block, based on the first prediction direction and the second prediction direction, respectively obtain Prediction block A and prediction block B of each sub-block, and then use the separable filter of the present application for prediction block A0 and prediction block B0 respectively, and obtain the prediction block A1 of secondary prediction of each unidirectional prediction after secondary prediction and prediction block B1, and then average or weighted average of two unidirectionally predicted secondary predicted prediction blocks A1 and B1 to obtain bidirectionally predicted prediction block C. Among them, the average or weighted average can be averaged or weighted average after the entire prediction block that needs to be averaged or weighted average is obtained, or it can be obtained after the prediction blocks of the sub-blocks that need to be averaged or weighted average are obtained. Average or weighted average may be performed, or average or weighted average may be performed after the prediction blocks of the points to be averaged or weighted average are obtained, which is not specifically limited in this application.

综上所述，通过步骤201至步骤206所提出的帧间预测方法，在基于子块的预测之后，针对运动矢量与子块的运动矢量有偏差的像素点，在基于子块的预测的基础上进行基于点的二次预测。基于点的二次预测使用可分离滤波器。可分离滤波器可以为横竖可分离的二维滤波器或水平垂直可分离的二维滤波器，可以由水平方向的一维滤波器和垂直方向的一维滤波器构成，最终获得的滤波结果即为像素点的新的预测值。To sum up, with the inter-frame prediction method proposed in steps 201 to 206, after the sub-block-based prediction, for the pixels whose motion vector deviates from the motion vector of the sub-block, on the basis of the sub-block-based prediction point-based secondary prediction. Point-based quadratic prediction uses separable filters. The separable filter can be a two-dimensional filter that can be separated horizontally and vertically or a two-dimensional filter that can be separated horizontally and vertically. It can be composed of a one-dimensional filter in the horizontal direction and a one-dimensional filter in the vertical direction. The final filtering result is is the new predicted value of the pixel.

本申请提出的帧间预测方法，适用于像素点的运动矢量与子块的运动矢量有偏差的情况，也就是说本申请中，解码器是在基于子块的预测后，如果像素点的运动矢量与子块的运动矢量没有偏差，那么使用本申请提出的帧间预测方法预测后的第二预测值应该和使用本申请提出的帧间预测方法之前的第一预测值相同。在本申请中，在像素点的运动矢量与子块的运动矢量没有偏差时，也可以选择对该像素点不使用本申请提出的帧间预测方法。The inter-frame prediction method proposed in this application is suitable for the situation where the motion vector of a pixel is deviated from the motion vector of a sub-block. There is no deviation between the vector and the motion vector of the sub-block, so the second prediction value after using the inter-frame prediction method proposed in this application should be the same as the first prediction value before using the inter-frame prediction method proposed in this application. In the present application, when there is no deviation between the motion vector of the pixel and the motion vector of the sub-block, it is also possible to choose not to use the inter-frame prediction method proposed in the present application for the pixel.

本申请提出的帧间预测方法可以适用于所有像素点的运动矢量与子块的运动矢量有偏差的情况，包括如仿射预测模型这种可以计算出每个像素点的运动矢量，而只进行了基于子块的预测的情况，也就是当前块所有像素点的运动矢量都可以计算出来，当前块并不是所有像素点的运动矢量都与它所在子块的运动矢量相同。还可以适用于并非所有的像素点的运动矢量都可以根据一个模型计算出来，但是当前块内部的一些像素点，如一个子块，的运动矢量发生变化而与其他位置的运动矢量不相同的时候，假设这些内容在当前帧的时间点上是连续变化的，那么这些相邻的而运动矢量不同的部分，如子块，之间的运动矢量可能是连续变化的，这时候根据计算模型等方式重新计算一些相关像素点的运动矢量后，这些运动矢量与原来的运动矢量有偏差的情况。也就是说，本申请提出的帧间预测方法，可以适用于仿射预测或者其他场景，并不对应用场景进行限定。The inter-frame prediction method proposed in this application can be applied to the situation where the motion vectors of all pixels deviate from the motion vectors of sub-blocks, including the affine prediction model that can calculate the motion vector of each pixel, and only perform In the case of sub-block-based prediction, that is, the motion vectors of all pixels in the current block can be calculated, and not all the motion vectors of pixels in the current block are the same as the motion vectors of the sub-block where it is located. It can also be applied when not all the motion vectors of pixels can be calculated according to a model, but some pixels in the current block, such as a sub-block, have their motion vectors changed and are not the same as the motion vectors of other positions. , assuming that these contents change continuously at the time point of the current frame, then the motion vectors between these adjacent parts with different motion vectors, such as sub-blocks, may change continuously. At this time, according to the calculation model and other methods After recalculating the motion vectors of some related pixels, these motion vectors deviate from the original motion vectors. That is to say, the inter-frame prediction method proposed in this application can be applied to affine prediction or other scenarios, and the application scenario is not limited.

需要说明的是，本申请提出的帧间预测方法，可以适用于任何一个图像分量上，在本实施例中示例性的对亮度分量使用二次预测方案，但是也可以用于色度分量，或者其他格式的任一分量。本申请提出的帧间预测方法也可以适用于任何一种视频格式上，包括但不限于YUV格式，包括但不限于YUV格式的亮度分量。It should be noted that the inter-frame prediction method proposed in this application can be applied to any image component. In this embodiment, a secondary prediction scheme is exemplarily used for the luminance component, but it can also be used for the chrominance component, or Any component in another format. The inter-frame prediction method proposed in this application can also be applied to any video format, including but not limited to the YUV format, including but not limited to the luminance component of the YUV format.

本实施例提供了一种帧间预测方法，解码器可以在基于子块的预测之后，针对运动矢量与子块的运动矢量有偏差的像素点，可以在基于子块的第一预测值的基础上，利用可分离滤波器进行基于点的二次预测，获得第二预测值。其中，可分离滤波器可以分离为水平方向的一维滤波器和垂直方向的一维滤波器，滤波后可以获得该像素点对应的新的预测值，最终获得子块修正后的第二预测值，本申请提出的帧间预测方法能够很好的适用于全部场景，大大提升编码性能，从而提高了编解码效率。This embodiment provides an inter-frame prediction method. After the sub-block-based prediction, the decoder may, for pixels whose motion vectors deviate from the motion vector of the sub-block, perform the sub-block-based first prediction based on the first prediction value of the sub-block. Above, a second prediction value is obtained by performing point-based secondary prediction using a separable filter. Among them, the separable filter can be separated into a one-dimensional filter in the horizontal direction and a one-dimensional filter in the vertical direction. After filtering, a new predicted value corresponding to the pixel point can be obtained, and finally a second predicted value after sub-block correction can be obtained. , the inter-frame prediction method proposed in this application can be well applied to all scenarios, greatly improving the encoding performance, thereby improving the encoding and decoding efficiency.

基于上述实施例中，在本申请的再一实施例中，解码器进行帧间预测的方法可以包括以下步骤：Based on the above embodiments, in yet another embodiment of the present application, the method for performing inter-frame prediction by a decoder may include the following steps:

步骤401、根据子块的运动矢量对子块进行预测，获得第一预测值。Step 401: Predict the sub-block according to the motion vector of the sub-block to obtain a first predicted value.

步骤402、确定子块内像素点与子块的第一运动矢量偏差。Step 402: Determine the first motion vector deviation between the pixels in the sub-block and the sub-block.

步骤403、根据像素点的第一运动矢量偏差，利用可分离滤波器对第一预测值进行滤波，得到第二预测值。Step 403: Filter the first predicted value with a separable filter according to the first motion vector deviation of the pixel point to obtain a second predicted value.

也就是说，本申请提出的帧间预测方法，可以在基于子块的预测之后，针对运动矢量与子块的运动矢量有偏差的像素点，在基于子块的预测的基础上，使用可分离滤波器进行基于点的二次预测，最终完成对第一预测值的修正，获得新的预测值，即第二预测值。That is to say, the inter-frame prediction method proposed in the present application can, after sub-block-based prediction, for pixels whose motion vector deviates from the motion vector of the sub-block, on the basis of sub-block-based prediction, use a separable The filter performs point-based secondary prediction, and finally completes the correction of the first predicted value to obtain a new predicted value, that is, the second predicted value.

具体地，基于点的二次预测使用可分离滤波器。可分离滤波器可以为二维滤波器，具体地，可分离滤波器可以为横竖可分离的二维滤波器或水平垂直可分离的二维滤波器，可以由水平方向的一维滤波器和垂直方向的一维滤波器构成。Specifically, point-based secondary prediction uses separable filters. The separable filter can be a two-dimensional filter. Specifically, the separable filter can be a two-dimensional filter that can be separated horizontally and vertically or a two-dimensional filter that can be separated horizontally and vertically. A one-dimensional filter composition of directions.

也就是说，在本申请中，如果基于像素点的二次预测使用可分离滤波器，那么该可分离滤波器可以在水平方向和垂直方向分别进行滤波处理，最终获得的滤波结果即为像素点的新的预测值。That is to say, in this application, if a separable filter is used for pixel-based secondary prediction, the separable filter can be filtered in the horizontal and vertical directions respectively, and the final filtering result is the pixel. new predicted value.

进一步地，在本申请中，对于步骤403，解码器在利用可分离滤波器对第一预测值进行滤波时，具体可以包括以下步骤：Further, in this application, for step 403, when the decoder uses a separable filter to filter the first predicted value, it may specifically include the following steps:

步骤403a、根据像素点的第一运动矢量偏差的水平分量对第一预测值进行水平方向的滤波处理，获得中间结果。Step 403a: Perform horizontal filtering processing on the first predicted value according to the horizontal component of the deviation of the first motion vector of the pixel to obtain an intermediate result.

步骤403b、根据像素点的第一运动矢量偏差的垂直分量对中间结果进行垂直方向的滤波处理，获得第二预测值。Step 403b: Perform vertical filtering processing on the intermediate result according to the vertical component of the first motion vector deviation of the pixel to obtain a second predicted value.

进一步地，在本申请中，对于步骤403，解码器在利用可分离滤波器对第一预测值进行滤波时，还可以包括以下步骤：Further, in this application, for step 403, when the decoder uses a separable filter to filter the first predicted value, it may further include the following steps:

步骤403c、根据像素点的第一运动矢量偏差的垂直分量对第一预测值进行垂直方向的滤波处理，获得中间结果。Step 403c: Perform vertical filtering processing on the first predicted value according to the vertical component of the first motion vector deviation of the pixel to obtain an intermediate result.

步骤403d、根据像素点的第一运动矢量偏差的水平分量对中间结果进行水平方向的滤波处理，获得第二预测值。Step 403d: Perform horizontal filtering processing on the intermediate result according to the horizontal component of the deviation of the first motion vector of the pixel point to obtain a second predicted value.

需要说明的是，在本申请中，对应一个待二次预测的像素点，使用可分离滤波器进行的二次预测会用到该像素点在基于子块的预测的预测块(整个块的预测块)的二次预测的起始点及其周边的几个像素点的像素值。如果不调整起始点的位置，起始点的位置为预测块中与该像素点相同的位置。It should be noted that, in this application, corresponding to a pixel to be re-predicted, the secondary prediction using the separable filter will use the pixel in the prediction block based on sub-block prediction (the prediction of the whole block). The starting point of the secondary prediction of the block) and the pixel values of several surrounding pixels. If the position of the starting point is not adjusted, the position of the starting point is the same position as the pixel in the prediction block.

进一步地，在本申请中，如果使用3x3的矩形滤波器，为了实现横竖(水平垂直)可分离的二维滤波器，水平垂直的3个抽头的滤波器分别使用的就是3x3的矩形区域。而待二次预测的像素点相邻的几个像素点包括起始点(该像素点为起始点的位置)的左边一个像素距离的像素点，右边一个像素距离的像素点，上边一个像素距离的像素点，下边一个像素距离的像素点，左边一个像素距离上边一个像素距离的像素点，右边一个像素距离上边一个像素距离的像素点，左边一个像素距离下边一个像素距离的像素点，右边一个像素距离下边一个像素距离的像素点。如果使用其他形状的滤波器，可能包括对应的其它位置。Further, in this application, if a 3x3 rectangular filter is used, in order to realize a two-dimensional filter that can be separated horizontally and vertically (horizontally and vertically), the filters of the three horizontal and vertical taps use a 3x3 rectangular area respectively. And several pixels adjacent to the pixel to be re-predicted include the pixel at a distance of one pixel to the left of the starting point (the pixel is the position of the starting point), the pixel at a distance of one pixel to the right, and the pixel at a distance of one pixel above. Pixel point, one pixel distance from the bottom, one pixel on the left, one pixel from the top, one pixel on the right, one pixel from the top, one pixel on the left, one pixel from the bottom, and one pixel on the right Pixels that are one pixel away from the bottom. If other shapes of filters are used, corresponding other positions may be included.

可以理解的是，在本申请中，解码器在确定像素点对应的第一运动矢量偏差之后，便可以根据第一运动矢量偏差进一步确定像素点对应的中心位置和第三运动矢量偏差。具体地，对于子块中的每一个像素点的像素，在进行二次预测时，如果可分离滤波器使用的默认的中心位置与子块之间的第一运动矢量偏差较大，那么获得的预测结果就可能存在较大的误差，因此，解码器在确定出每一个像素点的运动矢量与子块的运动矢量之间的第一运动矢量偏差之后，可以先对每一个像素点是否适合作为某个像素二次预测时所使用的可分离滤波器的中心位置进行确认，如果不适合，还可以重新进行中心位置的调整。It can be understood that, in the present application, after determining the first motion vector deviation corresponding to the pixel point, the decoder can further determine the center position corresponding to the pixel point and the third motion vector deviation according to the first motion vector deviation. Specifically, for the pixels of each pixel in the sub-block, when performing secondary prediction, if the default center position used by the separable filter and the first motion vector between the sub-blocks has a large deviation, then the obtained There may be a large error in the prediction result. Therefore, after determining the first motion vector deviation between the motion vector of each pixel and the motion vector of the sub-block, the decoder can first determine whether each pixel is suitable as a The center position of the separable filter used in the secondary prediction of a certain pixel is confirmed. If it is not suitable, the center position can be adjusted again.

可以理解的是，在本申请中，一个像素点对应的中心位置和第三运动矢量偏差，可以为对该像素点进行二次预测时所使用的可分离滤波器的中心位置和第一运动矢量偏差。It can be understood that, in this application, the center position and the third motion vector deviation corresponding to a pixel can be the center position and the first motion vector of the separable filter used in the secondary prediction of the pixel. deviation.

进一步地，在本申请的实施例中，解码器在根据第一运动矢量偏差确定像素点对应的中心位置和第三运动矢量偏差时，解码器可以计算第一运动矢量偏差的水平偏差的绝对值，即第一绝对值，以及垂直偏差的绝对值，即第二绝对值。然后，解码器可以根据该第一绝对值、第二绝对值以及预设偏差阈值，进一步确定出进行二次预测时所使用的中心位置和第三运动矢量偏差。Further, in the embodiment of the present application, when the decoder determines the center position corresponding to the pixel point and the third motion vector deviation according to the first motion vector deviation, the decoder can calculate the absolute value of the horizontal deviation of the first motion vector deviation. , ie the first absolute value, and the absolute value of the vertical deviation, ie the second absolute value. Then, the decoder may further determine, according to the first absolute value, the second absolute value and the preset deviation threshold, the center position and the third motion vector deviation used for the secondary prediction.

需要说明的是，在申请的实施例中，解码器可以将水平偏差的第一绝对值和垂直偏差的第二绝对值分别与预设偏差阈值进行比较，从而可以根据比较结果进行中心位置和第三运动矢量偏差的确定。其中，预设偏差阈值可以为预先设置的、用于判断是否进行中心位置和第一运动矢量偏差的调整。It should be noted that, in the embodiment of the application, the decoder can compare the first absolute value of the horizontal deviation and the second absolute value of the vertical deviation with the preset deviation threshold respectively, so that the center position and the first absolute value can be determined according to the comparison result. Determination of three motion vector deviations. The preset deviation threshold may be preset and used to determine whether to adjust the deviation between the center position and the first motion vector.

示例性的，在本申请中，预设偏差阈值可以以像素为单元，具体地，预设偏差阈值可以为k个像素；其中，k大于0.5且小于或者等于1。也就是说，可以预先将预设偏差阈值设置为二分之一个像素、四分之三个像素或者一个像素。Exemplarily, in the present application, the preset deviation threshold may be in units of pixels, and specifically, the preset deviation threshold may be k pixels; where k is greater than 0.5 and less than or equal to 1. That is, the preset deviation threshold may be set in advance to be one-half pixel, three-quarter pixel, or one pixel.

进一步地，在本申请的实施例中，解码器在根据水平偏差、垂直偏差以及预设偏差阈值，确定中心位置和第三运动矢量偏差时，如果第一绝对值和第二绝对值均小于预设偏差，便可以认为像素点的运动矢量与子块的运动矢量之间的矢量偏差较小，使用该像素点作为可分离滤波器的中心位置进行二次预测所获得的预测结果比较准确，因此，可以直接将该像素点对应的第一运动矢量偏差确定为用于进行二次预测的第三运动矢量偏差，同时，可以直接将该像素点确定为进行二次预测的可分离滤波器所使用的中心位置。Further, in the embodiment of the present application, when the decoder determines the center position and the third motion vector deviation according to the horizontal deviation, the vertical deviation and the preset deviation threshold, if the first absolute value and the second absolute value are both smaller than the predetermined deviation If the deviation is set, it can be considered that the vector deviation between the motion vector of the pixel point and the motion vector of the sub-block is small, and the prediction result obtained by using the pixel point as the center position of the separable filter for secondary prediction is more accurate, so , the first motion vector deviation corresponding to the pixel point can be directly determined as the third motion vector deviation used for secondary prediction, and at the same time, the pixel point can be directly determined to be used by the separable filter for secondary prediction the center position.

进一步地，在本申请的实施例中，解码器在根据水平偏差、垂直偏差以及预设偏差阈值，确定中心位置和第三运动矢量偏差时，如果第一绝对值大于或者等于预设偏差阈值，且第二绝对值小于预设偏差阈值，便可以认为像素点的运动矢量与子块的运动矢量之间在水平方向上的第一运动矢量偏差较大，如果使用该像素作为可分离滤波器的中心位置进行二次预测所获得的预测结果可能存在误差，因此，需要进一步按照水平偏差确定出第一调整方向，然后再根据第一调整方向进行调整，最终便可以确定出进行二次预测时所使用的中心位置和第三运动矢量偏差。其中，第一调整方向用于对第一运动矢量偏差在水平方向上进行调整，因此，第一调整方向包括左侧和右侧。Further, in the embodiment of the present application, when the decoder determines the deviation between the center position and the third motion vector according to the horizontal deviation, the vertical deviation and the preset deviation threshold, if the first absolute value is greater than or equal to the preset deviation threshold, And the second absolute value is less than the preset deviation threshold, it can be considered that the deviation of the first motion vector in the horizontal direction between the motion vector of the pixel point and the motion vector of the sub-block is relatively large. There may be errors in the prediction results obtained by secondary prediction at the center position. Therefore, it is necessary to further determine the first adjustment direction according to the horizontal deviation, and then adjust according to the first adjustment direction, and finally determine the value of the second prediction. Use the center position and the third motion vector offset. Wherein, the first adjustment direction is used to adjust the first motion vector deviation in the horizontal direction, therefore, the first adjustment direction includes the left side and the right side.

进一步地，在本申请的实施例中，解码器在根据水平偏差、垂直偏差以及预设偏差阈值，确定中心位置和第三运动矢量偏差时，如果第二绝对值大于或者等于预设偏差阈值，且第一绝对值小于预设偏差阈值，便可以认为像素点的运动矢量与子块的运动矢量之间在垂直方向上的第一运动矢量偏差较大，如果使用该像素作为可分离滤波器的中心位置进行二次预测所获得的预测结果可能存在误差，因此，需要进一步按照垂直偏差确定出第二调整方向，然后再根据第二调整方向进行调整，最终便可以确定出进行二次预测时所使用的中心位置和第三运动矢量偏差。其中，第二调整方向用于对第一运动矢量偏差在垂直方向上进行调整，因此，第二调整方向包括上侧和下侧。Further, in the embodiment of the present application, when the decoder determines the deviation between the center position and the third motion vector according to the horizontal deviation, the vertical deviation and the preset deviation threshold, if the second absolute value is greater than or equal to the preset deviation threshold, And the first absolute value is less than the preset deviation threshold, it can be considered that the deviation of the first motion vector in the vertical direction between the motion vector of the pixel point and the motion vector of the sub-block is relatively large. There may be errors in the prediction results obtained by the secondary prediction at the center position. Therefore, it is necessary to further determine the second adjustment direction according to the vertical deviation, and then adjust according to the second adjustment direction. Finally, the second adjustment direction can be determined. Use the center position and the third motion vector offset. The second adjustment direction is used to adjust the deviation of the first motion vector in the vertical direction, so the second adjustment direction includes an upper side and a lower side.

也就是说，在本申请中，解码器可以通过切换可分离滤波器使用的像素点来使得分像素点不会偏离中心较多。例如，如果水平方向的dmv_x是3/4像素，那么可以将可分离滤波器使用的像素点都向右移一个像素点，此时，对应的第一运动矢量偏差就变为-1/4像素，即第三运动矢量偏差dmv_x’为-1/4像素。That is to say, in this application, the decoder can switch the pixel points used by the separable filter so that the sub-pixel points do not deviate from the center too much. For example, if the dmv_x in the horizontal direction is 3/4 pixel, then the pixels used by the separable filter can be shifted to the right by one pixel, at this time, the corresponding first motion vector deviation becomes -1/4 pixel , that is, the third motion vector deviation dmv_x' is -1/4 pixel.

由此可见，在本申请中，在使用可分离滤波器对子块中的一个像素点进行二次预测之前，如果确定该像素点的运动矢量与子块的运动矢量之间的第一运动矢量偏差在水平方向和\或垂直方向的偏差分量的绝对值大于或者等于预设偏差阈值(如1/2像素、3/4像素或1像素)，那么便需要对二次预测所使用的起始点进行调整，即对二次预测所使用的可分离滤波器的中心位置进行调整。It can be seen that, in this application, before using a separable filter to perform secondary prediction on a pixel in a sub-block, if the first motion vector between the motion vector of the pixel and the motion vector of the sub-block is determined If the absolute value of the deviation component of the deviation in the horizontal direction and \ or vertical direction is greater than or equal to the preset deviation threshold (such as 1/2 pixel, 3/4 pixel or 1 pixel), then the starting point used for secondary prediction is required. Adjust, that is, adjust the center position of the separable filter used in the secondary prediction.

进一步地，在本申请的实施例中，在根据第一运动矢量偏差确定像素点对应的中心位置和第三运动矢量偏差之前，解码器也可以先对第一运动矢量偏差进行限制。具体地，解码器可以按照预设偏差范围对第一运动矢量偏差进行限制处理；其中，预设偏差范围包括偏差下限值和偏差上限值。Further, in the embodiment of the present application, before determining the center position corresponding to the pixel point and the third motion vector deviation according to the first motion vector deviation, the decoder may also limit the first motion vector deviation. Specifically, the decoder may limit the deviation of the first motion vector according to a preset deviation range, wherein the preset deviation range includes a deviation lower limit value and a deviation upper limit value.

进一步地，在本申请的实施例中，解码器按照预设偏差范围对第一运动矢量偏差进行限制处理时，如果水平偏差和\或垂直偏差小于偏差下限值，那么可以将水平偏差和\或垂直偏差设置为偏差下限值；如果水平偏差和\或垂直偏差大于偏差上限值，那么可以将水平偏差和\或垂直偏差设置为偏差上限值。Further, in the embodiment of the present application, when the decoder performs limiting processing on the first motion vector deviation according to the preset deviation range, if the horizontal deviation and \ or vertical deviation are less than the lower deviation limit value, then the horizontal deviation and \ Or vertical deviation is set as the lower deviation value; if the horizontal deviation and\or vertical deviation is greater than the upper deviation value, then the horizontal deviation and\or vertical deviation can be set as the upper deviation value.

进一步地，在本申请的实施例中，预设偏差范围可以由偏差下限值min和偏差上限值max构成，即预设偏差范围可以表示为(min，max)，在对第一运动矢量偏差进行限制处理时，如果第一运动矢量偏差的水平方向和\或垂直方向的偏差值小于偏差下限值min，那么可以直接将该偏差值设为min；如果第一运动矢量偏差的水平方向和\或垂直方向的偏差值大于max，那么可以直接将该偏差值设为max。Further, in the embodiment of the present application, the preset deviation range may be composed of a lower deviation limit value min and an upper deviation upper limit value max, that is, the preset deviation range may be expressed as (min, max). When the deviation is limited, if the deviation value of the horizontal direction and/or vertical direction of the first motion vector deviation is less than the deviation lower limit value min, then the deviation value can be directly set as min; if the horizontal direction of the first motion vector deviation And \ or the deviation value in the vertical direction is greater than max, then you can directly set the deviation value to max.

示例性的，在本申请中，假设上限值max为1/2像素，下限值min为-1/2像素，即限制第一运动矢量偏差为1/2像素，如果第一运动矢量偏差的水平方向和\或垂直方向的偏差值的绝对值大于1/2像素，则将第一运动矢量偏差的水平方向和\或垂直方向的偏差值设置为1/2像素；如果第一运动矢量偏差的水平方向和\或垂直方向的偏差值的绝对值小于-1/2像素，则将第一运动矢量偏差的水平方向和\或垂直方向的偏差值设置为-1/2像素。Exemplarily, in this application, it is assumed that the upper limit value max is 1/2 pixel, and the lower limit value min is -1/2 pixel, that is, the first motion vector deviation is limited to 1/2 pixel, if the first motion vector deviation The absolute value of the deviation value in the horizontal direction and \ or vertical direction is greater than 1/2 pixel, then the horizontal direction and \ or vertical direction deviation value of the first motion vector deviation is set to 1/2 pixel; if the first motion vector deviation value is set to 1/2 pixel; If the absolute value of the deviation value in the horizontal direction and \ or vertical direction of the deviation is less than -1/2 pixel, then the deviation value of the horizontal direction and \ or vertical direction of the first motion vector deviation is set to -1/2 pixel.

其中，对第一运动矢量偏差进行限制的处理，可以作为滤波的前处理，也可以在导出dmv_x和dmv_y时处理。The process of limiting the first motion vector deviation may be used as a pre-processing of filtering, or may be processed when dmv_x and dmv_y are derived.

需要说明的是，在本申请的实施例中，在对当前块的子块中的每一个像素点进行滤波时，可分离滤波器使用的对应像素点的值是上述基于子块的预测的预测值，滤波后的结果是通过本技术得到的新的预测值。具体地，可分离滤波器需要使用的、待二次预测的像素点的相邻像素点有可能超过该子块的边界。It should be noted that, in the embodiments of the present application, when filtering each pixel in the sub-block of the current block, the value of the corresponding pixel used by the separable filter is the prediction based on the above-mentioned sub-block prediction value, the filtered result is the new predicted value obtained by this technique. Specifically, the adjacent pixels of the pixels to be re-predicted that need to be used by the separable filter may exceed the boundary of the sub-block.

具体地，在本申请的实施例中，使用可分离滤波器对一个像素点进行二次预测时，需要使用与该像素点水平相邻或和\或垂直相邻的多个相邻像素点，对于该像素点和其对应的多个相邻像素点，存在均属于相同的子块和不属于相同的子块这两种情况。Specifically, in the embodiments of the present application, when using a separable filter to perform secondary prediction on a pixel, it is necessary to use a plurality of adjacent pixels that are horizontally adjacent to the pixel or and/or vertically adjacent, For the pixel point and its corresponding multiple adjacent pixel points, there are two cases that all belong to the same sub-block and that do not belong to the same sub-block.

进一步地，在本申请的实施例中，假设待二次预测的像素点所在的子块为当前子块，如果待二次预测的像素点对应的多个相邻像素点不属于当前子块，那么解码器可以根据当前子块对相邻像素点进行更新处理，从而可以获得更新后像素点。其中，解码器可以采用多种方式获取更新后像素点。Further, in the embodiment of the present application, it is assumed that the sub-block where the pixel to be re-predicted is the current sub-block, and if multiple adjacent pixels corresponding to the pixel to be re-predicted do not belong to the current sub-block, Then the decoder can update the adjacent pixels according to the current sub-block, so as to obtain the updated pixels. Among them, the decoder can obtain the updated pixels in various ways.

可以理解的是，在本申请中，解码器在根据当前子块对相邻像素点进行更新处理，获得更新后像素点时，可以先对当前子块进行扩展处理，获得扩展后子块；然后可以在扩展后子块内确定相邻像素点对应的更新后像素点。It can be understood that, in this application, when the decoder performs update processing on adjacent pixels according to the current sub-block and obtains the updated pixel points, it can first perform expansion processing on the current sub-block to obtain the expanded sub-block; then The updated pixel points corresponding to the adjacent pixel points may be determined in the extended sub-block.

也就是说，在本申请中，为了在二次预测时使所需要的像素值都使用当前子块的像素值，即限制使用的像素点均属于相同的子块，解码器可以先对当前的子块基于子块的预测的预测子块进行扩展，即先对当前子块进行扩展处理。具体地，由于二次预测时所使用的相邻像素点与待二次预测的像素点之间的距离为1个像素点，因此，即使相邻像素点不属于当前子块，也仅仅超出当前子块的边界一个像素点，可见，解码器在对当前子块进行扩展时，只需要对基于子块的预测的当前子块扩展一行或两行像素，和/或，一列或两列像素，最终便可以获得扩展后子块。That is to say, in this application, in order to use the pixel values of the current sub-block for all the required pixel values during the secondary prediction, that is, the pixels that are restricted to be used belong to the same sub-block, the decoder can The sub-block is extended based on the predicted predicted sub-block of the sub-block, that is, the current sub-block is extended first. Specifically, since the distance between the adjacent pixels used in the secondary prediction and the pixels to be re-predicted is 1 pixel, even if the adjacent pixels do not belong to the current sub-block, they only exceed the current sub-block. The boundary of the sub-block is one pixel. It can be seen that when the decoder expands the current sub-block, it only needs to expand the current sub-block based on the prediction of the sub-block by one or two rows of pixels, and/or, one or two columns of pixels, Finally, the expanded sub-block can be obtained.

示例性的，在本申请中，图12为扩展当前子块的示意图，如图12所示，以可分离滤波器为例，基于子块的预测的当前子块可以在其上下左右边界各扩展出一个像素点，一个简单的扩展方法是左边扩展的像素复制其水平方向对应的左边界的像素值，右边扩展的像素复制其水平方向对应的右边界的像素值，上边扩展的像素复制其垂直方向对应的上边界的像素值，下边扩展的像素复制其垂直方向对应的下边界的像素值，扩展的四个顶点可以复制其对应的顶点的像素值。Exemplarily, in this application, FIG. 12 is a schematic diagram of expanding the current sub-block. As shown in FIG. 12 , taking a separable filter as an example, the current sub-block based on the prediction of the sub-block can be expanded on its upper, lower, left, and right boundaries. A simple extension method is to copy the pixel value of the left border corresponding to the horizontal direction for the pixel extended on the left, copy the pixel value of the right border corresponding to the horizontal direction for the pixel extended on the right, and copy the pixel value of the vertical direction for the pixel extended on the top. The pixel value of the upper boundary corresponding to the direction, the pixel value of the lower boundary corresponding to the vertical direction is copied by the pixel extended on the lower side, and the pixel value of the corresponding vertex can be copied for the four extended vertices.

可以理解的是，在本申请的实施例中，解码器在对当前子块进行扩展处理，获得扩展后子块时，可以选择利用当前子块的全部边界位置进行扩展处理，获得扩展后子块；也可以选择利用当前子块内的、相邻像素点对应的行和\或列的边界位置进行扩展处理，获得扩展后子块。It can be understood that, in the embodiment of the present application, when the decoder performs expansion processing on the current sub-block and obtains the expanded sub-block, it can choose to use all the boundary positions of the current sub-block to perform expansion processing to obtain the expanded sub-block. ; You can also choose to use the boundary positions of the rows and/or columns corresponding to adjacent pixels in the current sub-block to perform expansion processing to obtain the expanded sub-block.

也就是说，在本申请中，在对当前子块进行扩展时，可以对当前子块的上下左右四个边界都进行扩展处理，也可以仅仅对于相邻像素点对应的一个或者两个边界进行扩展处理，例如，如果一个相邻像素点属于当前子块左边的子块，那么可以对当前子块的左侧边界进行扩展处理，而不对当前子块的其他三侧边界进行扩展处理。That is to say, in this application, when expanding the current sub-block, the expansion processing may be performed on all four boundaries of the current sub-block, the upper, lower, left, and right, or only one or two boundaries corresponding to adjacent pixels may be expanded. Expansion processing, for example, if an adjacent pixel belongs to a sub-block to the left of the current sub-block, then the left-side boundary of the current sub-block can be expanded, but the other three-side boundaries of the current sub-block are not expanded.

可以理解的是，在本申请中，解码器在根据当前子块对相邻像素点进行更新处理，获得更新后像素点时，还可以利用当前子块内的、与相邻像素点相邻的像素点，替换相邻像素点，从而获得更新后像素点。It can be understood that in this application, when the decoder updates adjacent pixels according to the current sub-block and obtains the updated pixels, it can also use the pixels in the current sub-block that are adjacent to adjacent pixels. Pixels, replace adjacent pixels to obtain updated pixels.

也就是说，在本申请中，如果二次预测的可分离滤波器的某一个位置对应的像素点超出了当前子块的边界，那么可以调整其对应的像素点为当前子块内的一个像素点。例如，当前子块的左上角像素点为(0，0)，当前子块的宽度是sub_width，当前子块的高度是sub_height，那么当前子块水平方向的范围是0～(sub_width-1)，当前子块垂直方向的范围是0～(sub_height-1)。若二次预测的可分离滤波器需要使用的像素点为(x，y)，如果x小于0，那么将x设为0。如果x大于sub_width-1，那么将x设为sub_width-1。如果y小于0，那么将y设为0，如果y大于sub_height-1，那么将y设为sub_height-1。That is to say, in this application, if a pixel corresponding to a certain position of the separable filter of the secondary prediction exceeds the boundary of the current sub-block, then the corresponding pixel can be adjusted to be a pixel in the current sub-block point. For example, if the pixel in the upper left corner of the current sub-block is (0, 0), the width of the current sub-block is sub_width, and the height of the current sub-block is sub_height, then the horizontal range of the current sub-block is 0～(sub_width-1), The vertical range of the current sub-block is 0 to (sub_height-1). If the pixel point to be used by the separable filter of the secondary prediction is (x, y), if x is less than 0, then set x to 0. If x is greater than sub_width-1, then set x to sub_width-1. If y is less than 0, set y to 0, and if y is greater than sub_height-1, set y to sub_height-1.

需要说明的是，上述对相邻像素点进行更新的方法，可以作用于整个编码单元或预测单元，即作用于当前块，也可以作用于当前块中的每一个子块，还可以作用于任意一个子块中的每一个像素点。本申请不作具体限定。It should be noted that the above method for updating adjacent pixels can be applied to the entire coding unit or prediction unit, that is, to the current block, to each sub-block in the current block, or to any each pixel in a subblock. This application does not make any specific limitations.

进一步地，在本申请的实施例中，解码器在解析获得预测模式参数之后，如果预测模式参数指示使用帧间预测模式确定当前块的帧间预测值，那么解码器可以先确定子块的扩展子块。Further, in the embodiment of the present application, after the decoder obtains the prediction mode parameter by parsing, if the prediction mode parameter indicates that the inter prediction value of the current block is determined using the inter prediction mode, the decoder may first determine the extension of the subblock. subblock.

需要说明的是，在本申请的实施例中，扩展子块为当前块中的、扩展每一个子块后所确定出的。具体地，由于二次预测时所使用的相邻像素点与待二次预测的像素点之间的距离为1个像素点，即使相邻像素点不属于当前子块，也仅仅超出当前子块的边界一个像素点，因此，解码器在基于子块进行预测时，可以先直接对当前子块对应的扩展子块进行预测，从而解决相邻像素点不属于当前子块所造成的预测效果不佳的问题。It should be noted that, in the embodiment of the present application, the extended sub-block is determined after extending each sub-block in the current block. Specifically, since the distance between the adjacent pixels used in the secondary prediction and the pixels to be re-predicted is 1 pixel, even if the adjacent pixels do not belong to the current sub-block, they only exceed the current sub-block. Therefore, when the decoder predicts based on sub-blocks, it can directly predict the extended sub-block corresponding to the current sub-block, so as to solve the problem of poor prediction effect caused by adjacent pixels not belonging to the current sub-block. good question.

进一步地，在本申请的实施例中，当前子块对应的扩展子块与当前子块相比，可以在当前子块的上侧、下侧、左侧以及右侧各增加一行(列)像素点。Further, in the embodiment of the present application, compared with the current sub-block, the extended sub-block corresponding to the current sub-block may have one row (column) of pixels added to the upper side, lower side, left side and right side of the current sub-block. point.

需要说明的是，在本申请的实施例中，基于子块尺寸参数确定出的子块的尺寸大小可以为8x8或4x4，那么，扩展子块的尺寸参数，即对应的扩展子块的尺寸大小可以为10x10或6x6。It should be noted that, in the embodiment of the present application, the size of the sub-block determined based on the sub-block size parameter may be 8×8 or 4×4. Then, the size parameter of the extended sub-block, that is, the size of the corresponding extended sub-block Can be 10x10 or 6x6.

进一步地，在本申请的实施例中，解码器在确定当前子块的扩展子块之后，便可以基于当前子块的运动矢量确定出该扩展子块的第三预测值，同时，可以确定待二次预测的像素点对应的相邻像素点。Further, in the embodiment of the present application, after determining the extended sub-block of the current sub-block, the decoder can determine the third predicted value of the extended sub-block based on the motion vector of the current sub-block, and at the same time, can determine the third prediction value of the extended sub-block to be The adjacent pixels corresponding to the secondary predicted pixels.

可以理解的是，在本申请中，由于扩展子块的中心点与对应的子块的中心点是相同的，因此扩展子块与对应的子块的运动矢量是相同的。It can be understood that, in the present application, since the center point of the extended sub-block is the same as the center point of the corresponding sub-block, the motion vectors of the extended sub-block and the corresponding sub-block are the same.

由此可见，在本申请中，在进行基于子块的预测的过程中，可以提前将二次预测所需要的像素点的像素值也同时预测出来。由于二次预测时所使用的相邻像素点与待二次预测的像素点之间的距离为1个像素点，因此在二次预测时仅仅需要使用对应的子块的上下各一行、左右各一列的像素点的像素值，按照该原则便可以确定出对应的扩展子块。例如，如果当前子块是尺寸大小为4x4的子块，那么二次预测需要使用尺寸大小为6x6的扩展子块的基于子块的预测值，如果当前子块是尺寸大小为8x8的子块，那么二次预测需要使用尺寸大小为10x10的扩展子块的基于子块的预测值。也就是说，解码器通过在基于子块的预测过程中，提前将二次预测所需要的像素点的像素值也一起预测出来，那么在后续的二次预测过程中，便不会存在待二次预测的像素点和相邻像素点不相连的问题。It can be seen that, in the present application, in the process of sub-block-based prediction, the pixel values of the pixel points required for the secondary prediction can also be predicted simultaneously in advance. Since the distance between the adjacent pixels used in the secondary prediction and the pixel to be re-predicted is 1 pixel, only the upper and lower rows and the left and right rows of the corresponding sub-blocks need to be used in the secondary prediction. For the pixel values of the pixel points in a column, the corresponding extended sub-block can be determined according to this principle. For example, if the current sub-block is a sub-block of size 4x4, then the secondary prediction needs to use the sub-block-based predictor of the extended sub-block of size 6x6, if the current sub-block is a sub-block of size 8x8, The secondary prediction then requires subblock-based predictors using extended subblocks of size 10x10. That is to say, in the sub-block-based prediction process, the decoder also predicts the pixel values of the pixels required for the secondary prediction in advance, so in the subsequent secondary prediction process, there will be no waiting for the second prediction process. The problem that the sub-predicted pixels are not connected to the adjacent pixels.

在本申请的实施例中，进一步地，解码器在***获得扩展子块的第三预测值之后，如果相邻像素点不属于当前子块，那么解码器可以基于第三预测值和相邻像素点，确定出当前子块对应的第二预测值，然后便可以将第二预测值确定为帧间预测值。In the embodiment of the present application, further, after the decoder obtains the third predicted value of the extended sub-block by predicting in advance, if the adjacent pixel does not belong to the current sub-block, the decoder may use the third predicted value and the adjacent pixel pixel point, the second prediction value corresponding to the current sub-block is determined, and then the second prediction value can be determined as the inter-frame prediction value.

可以理解的是，在本申请中，即使相邻像素点不属于当前子块，也一定属于当前子块对应的扩展子块，因此，在确定相邻像素点不属于当前子块之后，解码器便可以利用扩展子块的第三预测值和相邻像素点进一步对待二次预测的像素点进行二次预测，获得该像素点所对应的修正后预测值，遍历对应的子块中的每一个像素点，获得每一个像素点对应的修正后预测值，最终便可以确定对应的子块的第二预测值，进而可以将第二预测值确定为该子块的帧间预测值。It can be understood that in this application, even if the adjacent pixels do not belong to the current sub-block, they must belong to the extended sub-block corresponding to the current sub-block. Therefore, after determining that the adjacent pixels do not belong to the current sub-block, the decoder Then, the third predicted value of the extended sub-block and the adjacent pixels can be used to further predict the pixels to be re-predicted, obtain the corrected predicted value corresponding to the pixel, and traverse each of the corresponding sub-blocks. pixel point, obtain the corrected predicted value corresponding to each pixel point, and finally determine the second predicted value of the corresponding sub-block, and then determine the second predicted value as the inter-frame predicted value of the sub-block.

示例性的，在本申请中，如果当前块的宽度和高度分别是width和height，每个子块的宽度和高度分别是subwidth和subheight。如图4所示，当前块的亮度预测样本矩阵的左上角样本所在的子块为A，右上角样本所在的子块为B，左下角样本所在的子块为C，其他位置所在子块为其它子块。Exemplarily, in this application, if the width and height of the current block are width and height, respectively, the width and height of each sub-block are subwidth and subheight, respectively. As shown in Figure 4, the sub-block where the upper-left sample of the luminance prediction sample matrix of the current block is located is A, the sub-block where the upper-right sample is located is B, the sub-block where the lower-left sample is located is C, and the sub-blocks where other positions are located are other sub-blocks.

对当前块中的每个子块，可以将子块的运动矢量偏差矩阵记为dMv，那么：For each sub-block in the current block, the motion vector deviation matrix of the sub-block can be denoted as dMv, then:

1、如果子块是A，dMv等于dMvA；1. If the sub-block is A, dMv is equal to dMvA;

2、如果子块是B，dMv等于dMvB；2. If the sub-block is B, dMv is equal to dMvB;

3、如果子块为C，且该当前子块的控制点运动矢量组mvAffine中有3个运动矢量，dMv等于dMvC；3. If the sub-block is C, and there are 3 motion vectors in the control point motion vector group mvAffine of the current sub-block, dMv is equal to dMvC;

4、如果子块为A、B、C以外的其他子块，dMv等于dMvN。4. If the sub-block is other than A, B, and C, dMv is equal to dMvN.

进一步地，假设(x，y)是当前子块左上角位置的坐标，(i，j)是亮度子块内部像素的像素位置的坐标，i的取值范围是0～(subwidth-1)，j的取值范围是0～(subheight-1)，当前子块基于子块的的预测样本矩阵为PredMatrixTmp，PredMatrixTmp是扩展子块(subwidth+2)×(subheight+2)对应的预测样本矩阵，二次预测的预测样本矩阵为PredMatrixS，可以按照以下方法计算(x+i，y+j)的二次预测的预测样本PredMatrixS[x+i][y+j]：Further, it is assumed that (x, y) is the coordinate of the upper left corner of the current sub-block, (i, j) is the coordinate of the pixel position of the inner pixel of the luminance sub-block, and the value range of i is 0～(subwidth-1), The value range of j is 0～(subheight-1), the prediction sample matrix of the current sub-block based on the sub-block is PredMatrixTmp, and PredMatrixTmp is the prediction sample matrix corresponding to the extended sub-block (subwidth+2)×(subheight+2), The prediction sample matrix of the secondary prediction is PredMatrixS, and the prediction sample PredMatrixS[x+i][y+j] of the secondary prediction of (x+i, y+j) can be calculated as follows:

PredMatrixS[x+i][y+j]＝(PredUpTmp×UpCoef+PredCenterTmp×CenterCoefY+PredMatrixS[x+i][y+j]=(PredUpTmp×UpCoef+PredCenterTmp×CenterCoefY+

PredDownTmp×DownCoef+(1<<21))>>22PredDownTmp×DownCoef+(1<<21))>>22

PredMatrixS[x+i][y+j]＝Clip3(0，(1<<BitDepth)-1，PredMatrixS[x+i][y+j])。PredMatrixS[x+i][y+j]=Clip3(0, (1<<BitDepth)-1, PredMatrixS[x+i][y+j]).

其中，PredUpTmp＝((UPLEFT(x+i，y+j)×LeftCoef+UP(x+i，y+j)×CenterCoefX+Among them, PredUpTmp=((UPLEFT(x+i, y+j)×LeftCoef+UP(x+i, y+j)×CenterCoefX+

UPRIGHT(x+i，y+j)×RightCoef+(1<<21))>>22。UPRIGHT(x+i, y+j)×RightCoef+(1<<21))>>22.

PredCenterTmp＝((LEFT(x+i，y+j)×LeftCoef+CENTER(x+i，y+j)×CenterCoefX+PredCenterTmp=((LEFT(x+i, y+j)×LeftCoef+CENTER(x+i, y+j)×CenterCoefX+

RIGHT(x+i，y+j)×RightCoef+(1<<21))>>22。RIGHT(x+i, y+j)×RightCoef+(1<<21))>>22.

PredDownTmp＝((DOWNLEFT(x+i，y+j)×LeftCoef+DOWN(x+i，y+j)×CenterCoefX+PredDownTmp=((DOWNLEFT(x+i, y+j)×LeftCoef+DOWN(x+i, y+j)×CenterCoefX+

DOWNRIGHT(x+i，y+j)×RightCoef+(1<<21))>>22。DOWNRIGHT(x+i, y+j)×RightCoef+(1<<21))>>22.

示例性的，参考上述表9和表10，LeftCoef，RightCoef，UpCoef，DownCoef，CenterCoefX，CenterCoefY的计算如下：Exemplarily, with reference to Table 9 and Table 10 above, the calculations of LeftCoef, RightCoef, UpCoef, DownCoef, CenterCoefX, and CenterCoefY are as follows:

SquareDmvX＝dMv[i][j][0]×dMv[i][j][0]SquareDmvX=dMv[i][j][0]×dMv[i][j][0]

SquareDmvY＝dMv[i][j][1]×dMv[i][j][1]SquareDmvY=dMv[i][j][1]×dMv[i][j][1]

LeftCoef＝SquareDmvX–(dMv[i][j][0]<<10)LeftCoef=SquareDmvX–(dMv[i][j][0]<<10)

RightCoef＝SquareDmvX+(dMv[i][j][0]<<10)RightCoef=SquareDmvX+(dMv[i][j][0]<<10)

UpCoef＝SquareDmvY–(dMv[i][j][1]<<10)UpCoef=SquareDmvY–(dMv[i][j][1]<<10)

DownCoef＝SquareDmvY+(dMv[i][j][1]<<10)DownCoef=SquareDmvY+(dMv[i][j][1]<<10)

CenterCoefX＝(1<<22)-2×SquareDmvXCenterCoefX=(1<<22)-2×SquareDmvX

CenterCoefY＝(1<<22)-2×SquareDmvY；CenterCoefY=(1<<22)-2×SquareDmvY;

UPLEFT(x+i，y+j)＝PredMatrixTmp[i][j]UPLEFT(x+i, y+j)=PredMatrixTmp[i][j]

UP(x+i，y+j)＝PredMatrixTmp[i+1][j]UP(x+i, y+j)=PredMatrixTmp[i+1][j]

UPRIGHT(x+i，y+j)＝PredMatrixTmp[i+2][j]UPRIGHT(x+i, y+j)=PredMatrixTmp[i+2][j]

LEFT(x+i，y+j)＝PredMatrixTmp[i][j+1]LEFT(x+i, y+j)=PredMatrixTmp[i][j+1]

CENTER(x+i，y+j)＝PredMatrixTmp[i+1][j+1]CENTER(x+i, y+j)=PredMatrixTmp[i+1][j+1]

RIGHT(x+i，y+j)＝PredMatrixTmp[i+2][j+1]RIGHT(x+i, y+j)=PredMatrixTmp[i+2][j+1]

DOWNLEFT(x+i，y+j)＝PredMatrixTmp[i][j+2]DOWNLEFT(x+i, y+j)=PredMatrixTmp[i][j+2]

DOWN(x+i，y+j)＝PredMatrixTmp[i+1][j+2]DOWN(x+i, y+j)=PredMatrixTmp[i+1][j+2]

DOWNRIGHT(x+i，y+j)＝PredMatrixTmp[i+2][j+2]DOWNRIGHT(x+i, y+j)=PredMatrixTmp[i+2][j+2]

其中，左移和右移的位数都是根据dMv的精度确定的，dMv的精度不同，左移和右移的位数也会不同。Among them, the number of bits of left shift and right shift are determined according to the precision of dMv, and the number of bits of left shift and right shift will be different if the precision of dMv is different.

可以理解的是，在本申请中，尺寸参数为6x6或10x10的扩展子块对应的预测样本矩阵PredMatrixTmp，可以为在原有基于子块的预测的子块预测矩阵基础上增加上下各一行、左右各一列的预测值的预测矩阵。其中，PredMatrixTmp是对应于子块的矩阵，而PredMatrixS是对应于整个预测单元的矩阵，即当前块的矩阵，所以它们的索引有区别。扩展子块与当前子块相比，由于PredMatrixTmp在左边多一列且在上边多一行，因此与子块对应的位置的索引分别水平方向加1和垂直方向加1。It can be understood that, in the present application, the prediction sample matrix PredMatrixTmp corresponding to the extended sub-block whose size parameter is 6x6 or 10x10 can be a sub-block prediction matrix based on the original sub-block-based prediction. A prediction matrix with one column of predicted values. Among them, PredMatrixTmp is the matrix corresponding to the sub-block, and PredMatrixS is the matrix corresponding to the entire prediction unit, that is, the matrix of the current block, so their indices are different. Compared with the current sub-block, since the PredMatrixTmp has one more column on the left and one more row on the upper side, the index of the position corresponding to the sub-block is incremented by 1 in the horizontal direction and 1 in the vertical direction, respectively.

由于基于子块的预测需要用到插值滤波，一般情况下可以使用水平和垂直方向的8抽头滤波器进行内插。如果将插值点数由原来的4x4变成6x6，或者由原来的8x8变成10x10，那么需要增加额外的带宽，计算量也会增加。因此，需要基于预设限制条件对是否确定并使用当前子块对应的扩展子块的第三预测值。例如，如果当前子块的大小是8x8，那么便判定满足预设限制条件，则确定并使用扩展子块的第三预测值，以进行二次预测；如果当前子块的大小是4x4，那么便判定不满足预设限制条件，则不需要确定扩展子块的第三预测值，而是直接使用当前子块的第一预测值，以进行二次预测。Since the sub-block-based prediction needs to use interpolation filtering, in general, 8-tap filters in the horizontal and vertical directions can be used for interpolation. If the number of interpolation points is changed from the original 4x4 to 6x6, or the original 8x8 to 10x10, then additional bandwidth needs to be added, and the amount of calculation will also increase. Therefore, it is necessary to determine and use the third predicted value of the extended sub-block corresponding to the current sub-block based on a preset restriction condition. For example, if the size of the current sub-block is 8x8, then it is determined that the preset constraints are met, and the third prediction value of the extended sub-block is determined and used for secondary prediction; if the size of the current sub-block is 4x4, then If it is determined that the preset restriction condition is not met, the third prediction value of the extended sub-block does not need to be determined, but the first prediction value of the current sub-block is directly used for secondary prediction.

由于按现有的内插的方法增加新的预测值，如上述的上下各一行、左右各一列的预测值，会增加带宽，为了解决带宽增加的问题，还可以使用少于8抽头的滤波器进行内插，从而可以在确定额外像素点的预测值时不增加带宽，进一步地，也可以直接使用距离最近的参考像素值作为对应的额外像素点的预测值。本申请不作具体限定。Since new prediction values are added according to the existing interpolation method, such as the above-mentioned prediction values of the upper and lower rows and the left and right columns, the bandwidth will be increased. In order to solve the problem of bandwidth increase, a filter with less than 8 taps can also be used. Interpolation is performed, so that the bandwidth is not increased when the predicted value of the additional pixel is determined, and further, the nearest reference pixel value can be directly used as the predicted value of the corresponding additional pixel. This application does not make any specific limitations.

示例性的，在本申请中，图13为4x4子块的示意图，如图13所示，对一个4x4的子块使用8抽头的插值滤波器，需要使用如图所示的、该子块周围的11x11的参考像素进行插值。也就是说，对一个4x4子块来说，如果使用8抽头的插值滤波器，左侧和上侧需要多出3个参考像素、右侧和下侧需要多出4个参考像素进行插值。Exemplarily, in this application, FIG. 13 is a schematic diagram of a 4x4 sub-block. As shown in FIG. 13 , to use an 8-tap interpolation filter for a 4x4 sub-block, it is necessary to use the surrounding area of the sub-block as shown in the figure. The 11x11 reference pixels are interpolated. That is to say, for a 4x4 sub-block, if an 8-tap interpolation filter is used, 3 more reference pixels are required on the left and upper sides, and 4 more reference pixels are required on the right and lower sides for interpolation.

可以理解的是，在本申请中，可以通过对扩展子块使用较少抽头的滤波器来解决带宽增加的问题，例如，对扩展子块使用n抽头的插值滤波器，获得第三预测值；其中，n为以下值中的任一者：6，5，4，3，2。It can be understood that, in the present application, the problem of bandwidth increase can be solved by using a filter with fewer taps for the extended sub-block, for example, an interpolation filter with n taps is used for the extended sub-block to obtain the third prediction value; where n is any of the following values: 6, 5, 4, 3, 2.

示例性的，本申请提出另一种不增加带宽的方法，可以不扩展像素，而是在滤波器需要使用的像素超出了原有的像素范围时，选择使用就近的范围内的像素来代替。Exemplarily, the present application proposes another method that does not increase the bandwidth. Instead of expanding the pixels, when the pixels to be used by the filter exceed the original pixel range, the pixels in the nearest range are selected to be used instead.

可以理解的是，在本申请中，为了解决带宽增加的问题，还可以先在当前块中确定扩展子块对应的相邻的目标像素点；然后根据相邻的目标像素点的整像素值确定第三预测值。It can be understood that, in this application, in order to solve the problem of increased bandwidth, it is also possible to first determine the adjacent target pixels corresponding to the extended sub-blocks in the current block; and then determine according to the integer pixel value of the adjacent target pixels. The third predicted value.

图14为替换像素的示意图，如图14所示，为了不增加带宽，还可以直接使用最相邻的整像素的值作为需要使用的、超出该子块之外的预测值。具体地，可以直接使用方形3的像素值作为圆形2的预测值。如果水平方向的分像素MV小于等于(或小于)1/2像素，则使用分像素左边的整像素，如果水平方向的分像素MV大于(或大于等于)1/2像素，则使用分像素左边的整像素，如果垂直方向的分像素MV小于等于(或小于)1/2像素，则使用分像素上边的整像素，如果垂直方向的分像素MV大于(或大于等于)1/2像素，则使用分像素下边的整像素。FIG. 14 is a schematic diagram of replacing a pixel. As shown in FIG. 14 , in order not to increase the bandwidth, the value of the most adjacent integer pixel may be directly used as the prediction value that needs to be used beyond the sub-block. Specifically, the pixel value of square 3 can be directly used as the predicted value of circle 2. If the sub-pixel MV in the horizontal direction is less than or equal to (or less than) 1/2 pixel, the integer pixel to the left of the sub-pixel is used, and if the sub-pixel MV in the horizontal direction is greater than (or greater than or equal to) 1/2 pixel, the left sub-pixel is used. If the sub-pixel MV in the vertical direction is less than or equal to (or less than) 1/2 pixel, the integer pixel above the sub-pixel is used. If the sub-pixel MV in the vertical direction is greater than (or greater than or equal to) 1/2 pixel, then Use the whole pixel below the subpixel.

本申请实施例提供一种帧间预测方法，可以在基于子块的预测之后，针对运动矢量与子块的运动矢量有偏差的像素点，可以在基于子块的第一预测值的基础上，利用可分离滤波器进行基于点的二次预测，获得第二预测值。其中，可分离滤波器可以分离为水平方向的一维滤波器和垂直方向的一维滤波器，滤波后可以获得该像素点对应的新的预测值，最终获得子块修正后的第二预测值，本申请提出的帧间预测方法能够很好的适用于全部场景，大大提升编码性能，从而提高了编解码效率。The embodiment of the present application provides an inter-frame prediction method, which can, after sub-block-based prediction, for pixels whose motion vector deviates from the motion vector of the sub-block, on the basis of the first prediction value based on the sub-block, A second prediction value is obtained by performing point-based secondary prediction using a separable filter. Among them, the separable filter can be separated into a one-dimensional filter in the horizontal direction and a one-dimensional filter in the vertical direction. After filtering, a new predicted value corresponding to the pixel point can be obtained, and finally a second predicted value after sub-block correction can be obtained. , the inter-frame prediction method proposed in this application can be well applied to all scenarios, greatly improving the encoding performance, thereby improving the encoding and decoding efficiency.

本申请实施例提供一种帧间预测方法，该方法应用于视频编码设备，即编码器。该方法所实现的功能可以通过编码器中的第二处理器调用计算机程序来实现，当然计算机程序可以保存在第二存储器中，可见，该编码器至少包括第二处理器和第二存储器。An embodiment of the present application provides an inter-frame prediction method, which is applied to a video encoding device, that is, an encoder. The functions implemented by the method can be implemented by calling a computer program by the second processor in the encoder. Of course, the computer program can be stored in the second memory. It can be seen that the encoder includes at least a second processor and a second memory.

图15为帧间预测方法的实现流程示意图二，如图15所示，编码器进行帧间预测的方法可以包括以下步骤：FIG. 15 is a schematic diagram 2 of the implementation flow of the inter-frame prediction method. As shown in FIG. 15 , the method for performing the inter-frame prediction by the encoder may include the following steps:

步骤501、确定当前块的预测模式参数。Step 501: Determine the prediction mode parameter of the current block.

在本申请的实施例中，编码器可以先确定当前块的预测模式参数。具体地，编码器可以先确定当前块使用的预测模式，然后基于该预测模式确定对应的预测模式参数。其中，预测模式参数可以用于对当前块所使用的预测模式进行确定。In the embodiment of the present application, the encoder may first determine the prediction mode parameter of the current block. Specifically, the encoder may first determine the prediction mode used by the current block, and then determine the corresponding prediction mode parameter based on the prediction mode. The prediction mode parameter may be used to determine the prediction mode used by the current block.

需要说明的是，在本申请的实施例中，预测模式参数指示了当前块采用的预测模式以及与该预测模式相关的参数。这里，针对预测模式参数的确定，可以采用简单的决策策略，比如根据失真值的大小进行确定；也可以采用复杂的决策策略，比如根据率失真优化(Rate Distortion Optimization，RDO)的结果进行确定，本申请实施例不作任何限定。通常而言，可以采用RDO方式来确定当前块的预测模式参数。It should be noted that, in this embodiment of the present application, the prediction mode parameter indicates the prediction mode adopted by the current block and parameters related to the prediction mode. Here, for the determination of the parameters of the prediction mode, a simple decision-making strategy can be adopted, such as determining according to the size of the distortion value; or a complex decision-making strategy can be adopted, such as determining according to the result of Rate Distortion Optimization (RDO), The embodiments of the present application do not make any limitation. Generally speaking, the RDO method can be used to determine the prediction mode parameter of the current block.

具体地，在一些实施例中，编码器在确定当前块的预测模式参数时，可以先利用多种预测模式对当前块进行预编码处理，获得每一种预测模式对应的率失真代价值；然后从所获得的多个率失真代价值中选择最小率失真代价值，并根据最小率失真代价值对应的预测模式确定当前块的预测模式参数。Specifically, in some embodiments, when determining the prediction mode parameter of the current block, the encoder may first perform precoding processing on the current block by using multiple prediction modes to obtain the rate-distortion cost value corresponding to each prediction mode; then The minimum rate-distortion cost value is selected from the obtained multiple rate-distortion cost values, and the prediction mode parameter of the current block is determined according to the prediction mode corresponding to the minimum rate-distortion cost value.

也就是说，在编码器侧，针对当前块可以采用多种预测模式分别对当前块进行预编码处理。这里，多种预测模式通常包括有帧间预测模式、传统帧内预测模式和非传统帧内预测模式；其中，传统帧内预测模式可以包括有直流(Direct Current，DC)模式、平面(PLANAR)模式和角度模式等，非传统帧内预测模式可以包括有基于矩阵的帧内预测(Matrix-based Intra Prediction，MIP)模式、跨分量线性模型预测(Cross-component Linear Model Prediction，CCLM)模式、帧内块复制(Intra Block Copy，IBC)模式和PLT(Palette)模式等，而帧间预测模式可以包括有普通帧间预测模式、GPM模式和AWP模式等。That is to say, on the encoder side, multiple prediction modes may be used for the current block to perform precoding processing on the current block respectively. Here, the multiple prediction modes usually include an inter prediction mode, a traditional intra prediction mode, and a non-traditional intra prediction mode; wherein, the traditional intra prediction mode may include a direct current (Direct Current, DC) mode, a plane (PLANAR) mode mode and angle mode, etc., non-traditional intra prediction modes can include Matrix-based Intra Prediction (MIP) mode, Cross-component Linear Model Prediction (Cross-component Linear Model Prediction, CCLM) mode, frame Intra Block Copy (Intra Block Copy, IBC) mode and PLT (Palette) mode, etc., and inter-frame prediction modes may include ordinary inter-frame prediction mode, GPM mode, and AWP mode.

这样，在利用多种预测模式分别对当前块进行预编码之后，可以得到每一种预测模式对应的率失真代价值；然后从所得到的多个率失真代价值中选取最小率失真代价值，并将该最小率失真代价值对应的预测模式确定为当前块的预测模式参数。除此之外，还可以在利用多种预测模式分别对当前块进行预编码之后，得到每一种预测模式对应的失真值；然后从所得到的多个失真值中选取最小失真值，然后将该最小失真值对应的预测模式确定为当前块使用的预测模式，并根据该预测模式设置对应的预测模式参数。如此，最终使用所确定的预测模式参数对当前块进行编码，而且在这种预测模式下，可以使得预测残差较小，能够提高编码效率。In this way, after precoding the current block with multiple prediction modes, the rate-distortion cost value corresponding to each prediction mode can be obtained; then the minimum rate-distortion cost value is selected from the obtained multiple rate-distortion cost values, The prediction mode corresponding to the minimum rate-distortion cost value is determined as the prediction mode parameter of the current block. In addition, it is also possible to obtain the distortion value corresponding to each prediction mode after precoding the current block with multiple prediction modes; then select the minimum distortion value from the obtained multiple distortion values, and then use The prediction mode corresponding to the minimum distortion value is determined as the prediction mode used by the current block, and corresponding prediction mode parameters are set according to the prediction mode. In this way, the current block is finally encoded using the determined prediction mode parameters, and in this prediction mode, the prediction residual can be made smaller, and the encoding efficiency can be improved.

步骤502、当预测模式参数指示使用帧间预测模式确定当前块的帧间预测值时，确定当前块的子块的第一预测值；其中，子块包括一个或多个像素点。Step 502: When the prediction mode parameter indicates that the inter prediction value of the current block is determined using the inter prediction mode, determine the first prediction value of the sub-block of the current block, wherein the sub-block includes one or more pixels.

步骤503、对于子块内的一像素点，确定像素点的运动矢量与子块的运动矢量之间的第一运动矢量偏差。Step 503: For a pixel in the sub-block, determine the first motion vector deviation between the motion vector of the pixel and the motion vector of the sub-block.

在本申请的实，编码器在解析获得预测模式参数之后，如果解析获得的预测模式参数指示当前块使用帧间预测模式确定当前块的帧间预测值，那么编码器可以先确定出当前块的每一个子块的第一预测值，同时，还需要确定出像素点的运动矢量与子块的运动矢量之间的第一运动矢量偏差。其中，每一个子块均包括有一个或多个像素点。In the present application, after the encoder obtains the prediction mode parameter by parsing, if the prediction mode parameter obtained by parsing indicates that the current block uses the inter prediction mode to determine the inter prediction value of the current block, the encoder may first determine the current block's inter prediction value. For the first predicted value of each sub-block, at the same time, it is also necessary to determine the first motion vector deviation between the motion vector of the pixel point and the motion vector of the sub-block. Wherein, each sub-block includes one or more pixels.

示例性的，在本申请的实施例中，编码器在确定当前块的子块的第一预测值时，可以先确定当前块的子块的运动矢量，然后再基于该运动矢量确定出该子块对应的第一预测值。Exemplarily, in the embodiment of the present application, when the encoder determines the first prediction value of the sub-block of the current block, it may first determine the motion vector of the sub-block of the current block, and then determine the sub-block based on the motion vector. The first predicted value corresponding to the block.

示例性的，在本申请的实施例中，编码器在确定当前块的子块的第一预测值时，也可以先确定子块与当前块之间的第二运动矢量偏差；然后再基于第二运动矢量偏差，确定子块的第一预测值。Exemplarily, in the embodiment of the present application, when the encoder determines the first prediction value of the sub-block of the current block, it may also first determine the second motion vector deviation between the sub-block and the current block; 2. The motion vector offset determines the first predicted value of the sub-block.

也就是说，在本申请中，编码器确定当前块的子块的第一预测值可能有多种方式，既可以在计算子块的运动矢量之后，利用运动矢量进一步计算第一预测值，也可以直接计算子块的第一预测值。例如，可以针对一个子块计算获得该子块与当前块(整个编码单元或预测单元)之间的运动矢量偏差(或者水平方向和垂直方向的偏差)，然后根据这个与当前块的运动矢量偏差确定该子块的第一预测值。进一步地，在获得该子块的第一预测值之后，该子块的运动矢量可以认为是当前块的运动矢量加上与该子块的运动矢量偏差。That is to say, in this application, the encoder may determine the first predicted value of the sub-block of the current block in various ways. After calculating the motion vector of the sub-block, the first predicted value can be further calculated by using the motion vector, or The first predicted value of the sub-block can be directly calculated. For example, the motion vector deviation (or the deviation in the horizontal direction and the vertical direction) between the sub-block and the current block (the entire coding unit or prediction unit) can be calculated and obtained for a sub-block, and then the motion vector deviation from the current block can be obtained according to this motion vector deviation. A first predicted value for the sub-block is determined. Further, after the first predicted value of the sub-block is obtained, the motion vector of the sub-block can be regarded as the motion vector of the current block plus a deviation from the motion vector of the sub-block.

进一步地，在本申请中，编码器在进行像素点的运动矢量与子块的运动矢量之间的第一运动矢量偏差的确定时，相应地，既可以基于计算获得的子块的运动矢量确定第一运动矢量偏差，也可以不进行子块的运动矢量的计算而直接确定第一运动矢量偏差。也就是说，编码器可以在不确定像素点的运动矢量和子块的运动矢量的情况下直接进行第一运动矢量偏差的计算。例如，编码器可以直接根据整个当前块的运动矢量和每个子块的运动矢量偏差，导出每个像素点的运动矢量与子块的运动矢量之间的第一运动矢量偏差。Further, in the present application, when the encoder determines the first motion vector deviation between the motion vector of the pixel point and the motion vector of the sub-block, correspondingly, it can be determined based on the motion vector of the sub-block obtained by calculation. For the first motion vector offset, the first motion vector offset may be directly determined without calculating the motion vector of the sub-block. That is, the encoder can directly calculate the first motion vector deviation without determining the motion vector of the pixel point and the motion vector of the sub-block. For example, the encoder may directly derive the first motion vector deviation between the motion vector of each pixel and the motion vector of the sub-block according to the motion vector of the entire current block and the motion vector deviation of each sub-block.

需要说明的是，在本申请的实施例中，当前块为当前帧中待编码的图像块，当前帧以图像块的形式按一定顺序依次进行编码，该当前块为当前帧内按该顺序下一时刻待编码的图像块。当前块可具有多种规格尺寸，例如16×16、32×32或32×16等规格，其中数字表示当前块上像素点的行数和列数。It should be noted that, in the embodiments of the present application, the current block is the image block to be encoded in the current frame, the current frame is encoded in the form of image blocks in a certain order, and the current block is the image block in the current frame in this order. The image block to be encoded at a time. The current block may have various sizes, such as 16×16, 32×32, or 32×16, where the numbers represent the number of rows and columns of pixels on the current block.

可以理解的是，在本申请的实施例中，在编码器确定到预测模式参数指示使用帧间预测模式确定当前块的帧间预测值的情况下，就可以继续采用本申请实施例所提供的帧间预测方法。It can be understood that, in the embodiments of the present application, when the encoder determines that the prediction mode parameter indicates that the inter-frame prediction value of the current block is determined by using the inter-frame prediction mode, it can continue to use the method provided by the embodiments of the present application. Inter prediction method.

可以理解的是，本申请实施例提出的帧间预测方法对编码器执行步骤502和步骤503的顺序不进行限定，也就是说，在本申请中，编码器可以先执行步骤502，然后执行步骤503，也可以先执行步骤503，再执行步骤502，还可以同时执行步骤502和步骤503。It can be understood that the inter-frame prediction method proposed in this embodiment of the present application does not limit the order in which the encoder performs step 502 and step 503, that is, in this application, the encoder may first perform step 502, and then perform step 502. 503, step 503 may also be performed first, and then step 502 may be performed, or step 502 and step 503 may be performed simultaneously.

进一步地，在本申请的实施例中，如果编码器获得的二次预测参数指示不使用二次预测，那么，编码器可以选择直接将上述步骤502中所获得的当前块的子块的第一预测值作为子块的第二预测值，而不再进行步骤503至步骤506的处理。Further, in the embodiment of the present application, if the secondary prediction parameter obtained by the encoder indicates that secondary prediction is not used, then the encoder may choose to directly use the first sub-block of the current block obtained in step 502 above. The predicted value is used as the second predicted value of the sub-block, and the processing from step 503 to step 506 is no longer performed.

具体地，在本本申请的实施例中，如果二次预测参数指示不使用二次预测，那么编码器可以利用预测样本矩阵确定出第二预测值。其中，预测样本矩阵包括多个子块的第一预测值，编码器可以将像素点所在的子块的第一预测值确定为自身的第二预测值。Specifically, in the embodiment of the present application, if the secondary prediction parameter indicates that secondary prediction is not used, the encoder may determine the second prediction value by using the prediction sample matrix. The prediction sample matrix includes the first prediction values of multiple sub-blocks, and the encoder may determine the first prediction value of the sub-block where the pixel point is located as its own second prediction value.

步骤504、根据第一运动矢量偏差确定滤波参数。Step 504: Determine filtering parameters according to the first motion vector deviation.

在本申请的实施例中，编码器在确定子块的第一预测值和像素点的运动矢量与子块的运动矢量之间的第一运动矢量偏差之后，便可以根据第一运动矢量偏差进一步确定出可分离滤波器的滤波参数。In the embodiment of the present application, after determining the first prediction value of the sub-block and the first motion vector deviation between the motion vector of the pixel point and the motion vector of the sub-block, the encoder can further The filtering parameters of the separable filter are determined.

需要说明的是，在本申请中，目前多种常见的编编码场景已经使用了可分离滤波器，如帧间基于块的预测的插值滤波、帧间基于子块的预测的插值滤波，也包括上述仿射基于子块的预测的插值滤波等。It should be noted that, in this application, separable filters have been used in various common coding scenarios, such as interpolation filtering for inter-frame block-based prediction, interpolation filtering for inter-frame sub-block-based prediction, including The above-described affine interpolation filtering and the like are based on prediction of subblocks.

需要说明的是，在本申请的实施例中，编码器在根据第一运动矢量偏差确定滤波参数时，可以先确定滤波模式；然后再基于滤波模式，利用第一运动矢量偏差确定滤波参数；其中，第一运动矢量偏差包括水平偏差和垂直偏差。It should be noted that, in the embodiment of the present application, when the encoder determines the filtering parameters according to the first motion vector deviation, the encoder may first determine the filtering mode; then based on the filtering mode, the first motion vector deviation is used to determine the filtering parameters; wherein , the first motion vector deviation includes horizontal deviation and vertical deviation.

具体地，在本申请中，滤波模式可以包括水平滤波、垂直滤波以及双向分离滤波。也就是说，在本申请中，编码器在使用可分离滤波器对子块中的像素点进行二次预测时，可以选择使用该可分离滤波器的一个方向进行滤波处理，例如，仅仅进行水平方向的滤波，或者仅仅进行垂直方向的滤波；也可以选择使用该可分离滤波器的两个方向分别进行滤波处理，例如分别进行垂直滤波和水平滤波。Specifically, in this application, the filtering modes may include horizontal filtering, vertical filtering, and bidirectional separation filtering. That is to say, in this application, when the encoder uses the separable filter to perform secondary prediction on the pixels in the sub-block, it can choose to use one direction of the separable filter to perform filtering processing, for example, only perform horizontal filtering. filtering in the vertical direction, or only filtering in the vertical direction; you can also choose to use the two directions of the separable filter to perform filtering processing separately, for example, perform vertical filtering and horizontal filtering respectively.

可以理解的是，在本申请的实施例中，编码器在确定滤波模式时，可以先确定水平参数和垂直参数；然后再将水平参数和垂直参数分别与预设水平阈值和预设垂直阈值进行比较，获得比较结果，最终可以根据比较结果确定使用的滤波模式。It can be understood that, in the embodiment of the present application, when the encoder determines the filtering mode, it can first determine the horizontal parameter and the vertical parameter; Compare, obtain the comparison result, and finally determine the filtering mode to be used according to the comparison result.

可以理解的是，在本申请中，在将水平参数和垂直参数分别与预设水平阈值和预设垂直阈值进行比较，获得比较结果之后，如果水平参数小于预设水平阈值，且垂直参数小于预设垂直阈值，那么编码器可以认为不需要进行二次预测处理，从而可以不使用可分离滤波器进行滤波处理。It can be understood that, in the present application, after comparing the horizontal parameter and the vertical parameter with the preset horizontal threshold and the preset vertical threshold respectively, and obtaining the comparison result, if the horizontal parameter is less than the preset horizontal threshold, and the vertical parameter is less than the preset horizontal threshold If the vertical threshold is set, the encoder can consider that the secondary prediction processing is not required, so that the filtering processing can be performed without using a separable filter.

也就是说，在本申请中，基于一定的预设条件，编码器可以选择只使用一个方向的滤波器，例如只使用水平方向的滤波器或只使用垂直方向的滤波器；也可以两个方向都不使用，即跳过二次预测。其中，该预设条件可以是基于预设水平阈值和预设垂直阈值进行比较，例如，某一个或某几个值为0，或某一个或某几个值小于(或小于等于，或大于，或大于等于)某个阈值，而比较处理的作用范围可以是整个编码单元或预测单元，即当前块，也可以是每一个子块或每一个像素点。示例性的，如果比较处理的作用范围是整个编码单元或预测单元，一种可能的限制是整个编码单元或预测单元的某一个方向的最大运动矢量偏差小于一个阈值，或某一个方向的某一个参数小于一个阈值。如果整个编码单元或预测单元的水平方向的最大运动矢量偏差小于一个阈值，那么整个编码单元或预测单元只进行垂直方向的滤波；如果整个编码单元或预测单元的垂直方向的最大运动矢量偏差小于一个阈值，那么整个编码单元或预测单元只进行水平方向的滤波；如果整个编码单元或预测单元的两个方向的最大运动矢量偏差都小于一个阈值，那么整个编码单元或预测单元水平方向和垂直方向都不进行滤波。That is to say, in the present application, based on certain preset conditions, the encoder can choose to use only one filter in one direction, for example, only use the filter in the horizontal direction or only use the filter in the vertical direction; Neither are used, i.e. secondary prediction is skipped. Wherein, the preset condition may be based on the comparison between the preset horizontal threshold and the preset vertical threshold, for example, one or several values are 0, or one or several values are less than (or less than or equal to, or greater than, or greater than or equal to) a certain threshold, and the scope of action of the comparison processing may be the entire coding unit or prediction unit, that is, the current block, or may be each sub-block or each pixel. Exemplarily, if the scope of the comparison processing is the entire coding unit or the prediction unit, a possible limitation is that the maximum motion vector deviation in a certain direction of the entire coding unit or the prediction unit is less than a threshold, or a certain direction in a certain direction. The parameter is less than a threshold. If the maximum motion vector deviation in the horizontal direction of the entire coding unit or prediction unit is less than a threshold, then the entire coding unit or prediction unit is only filtered in the vertical direction; if the maximum motion vector deviation in the vertical direction of the entire coding unit or prediction unit is less than a If the maximum motion vector deviation in both directions of the entire coding unit or prediction unit is less than a threshold, then the entire coding unit or prediction unit is horizontally and vertically filtered. No filtering is performed.

可选的，在本申请中，编码器在使用运动矢量偏差进行可分离滤波器的滤波参数的确定过程中，可以对该运动矢量偏差在水平方向上的水平偏差和在垂直方向上的垂直偏差进行大小的限制。具体地，编码器可以针对水平方向和垂直方向分别设置最大值和最小值。具体地，如果水平偏差或垂直偏差大于对应方向上的最大值，那么可以将水平偏差或垂直偏差设置为对应的最大值，如果水平偏差或垂直偏差小于对应方向上的最小值，那么可以将水平偏差或垂直偏差设置为对应的最小值。其中，水平方向或垂直方向上的最大值的绝对值可以是1像素，水平方向或垂直方向上的最小值的绝对值可以是1/2像素。Optionally, in this application, in the process of determining the filtering parameters of the separable filter by using the motion vector deviation, the encoder can determine the horizontal deviation of the motion vector deviation in the horizontal direction and the vertical deviation in the vertical direction. Make size restrictions. Specifically, the encoder can set the maximum value and the minimum value for the horizontal direction and the vertical direction, respectively. Specifically, if the horizontal deviation or vertical deviation is greater than the maximum value in the corresponding direction, the horizontal deviation or vertical deviation can be set to the corresponding maximum value, and if the horizontal deviation or vertical deviation is smaller than the minimum value in the corresponding direction, then the horizontal deviation or vertical deviation can be set to the corresponding maximum value. The bias or vertical bias is set to the corresponding minimum value. The absolute value of the maximum value in the horizontal direction or the vertical direction may be 1 pixel, and the absolute value of the minimum value in the horizontal direction or the vertical direction may be 1/2 pixel.

进一步地，在本申请中，由于可分离滤波器的滤波参数可以包括多种不同类型参数，例如滤波器系数或滤波器相位，因此，编码器在进行可分离滤波器的滤波参数的确定时，可以采用多种不同的方式。Further, in this application, since the filtering parameters of the separable filter may include a variety of different types of parameters, such as filter coefficients or filter phases, when the encoder determines the filtering parameters of the separable filter, This can be done in a number of different ways.

示例性的，在本申请中，如果滤波参数为滤波器系数，那么编码器在基于滤波模式，利用第一运动矢量偏差确定滤波参数时，可以确定比例参数；然后可以根据比例参数和第一运动矢量偏差，按照滤波模式确定像素点对应的滤波器系数。Exemplarily, in the present application, if the filter parameter is a filter coefficient, then the encoder can determine the scale parameter when determining the filter parameter based on the filtering mode and the first motion vector deviation; then the scale parameter and the first motion The vector deviation determines the filter coefficient corresponding to the pixel point according to the filter mode.

可以理解的是，在本申请中，编码器在根据比例参数和第一运动矢量偏差，按照滤波模式确定像素点对应的滤波器系数时，如果滤波模式为水平滤波，那么编码器可以根据比例参数和水平偏差，按照预设计算规则计算滤波器系数。也就是说，在对像素点进行二次预测时，对于水平滤波的滤波模式，编码器可以利用像素点的水平偏差，按照预设计算规则来分别计算该像素点的滤波器系数和与该像素点水平相邻的相邻像素点的滤波器系数。It can be understood that in this application, when the encoder determines the filter coefficient corresponding to the pixel point according to the scale parameter and the first motion vector deviation according to the filter mode, if the filter mode is horizontal filtering, then the encoder can be based on the scale parameter. and the horizontal deviation, the filter coefficients are calculated according to the preset calculation rules. That is to say, when performing secondary prediction on a pixel, for the filtering mode of horizontal filtering, the encoder can use the horizontal deviation of the pixel to calculate the filter coefficient of the pixel and the correlation with the pixel according to the preset calculation rule. The filter coefficients of adjacent pixels that are horizontally adjacent to each other.

可以理解的是，在本申请中，编码器在根据比例参数和第一运动矢量偏差，按照滤波模式确定像素点对应的滤波器系数时，如果滤波模式为垂直滤波，那么编码器可以根据比例参数和垂直偏差，按照预设计算规则计算滤波器系数。也就是说，在对像素点进行二次预测时，对于垂直滤波的滤波模式，编码器可以利用像素点的垂直偏差，按照预设计算规则来分别计算该像素点的滤波器系数和与该像素点垂直相邻的相邻像素点的滤波器系数。It can be understood that in the present application, when the encoder determines the filter coefficient corresponding to the pixel point according to the filter mode according to the scale parameter and the first motion vector deviation, if the filter mode is vertical filtering, then the encoder can be based on the scale parameter. and vertical deviation, the filter coefficients are calculated according to the preset calculation rules. That is to say, when performing secondary prediction on a pixel, for the filtering mode of vertical filtering, the encoder can use the vertical deviation of the pixel to calculate the filter coefficient of the pixel and the correlation with the pixel according to the preset calculation rule. The filter coefficients of the adjacent pixels that are vertically adjacent to the point.

可以理解的是，在本申请中，编码器在根据比例参数和第一运动矢量偏差，按照滤波模式确定像素点对应的滤波器系数时，如果滤波模式为双向分离滤波，那么编码器可以根据比例参数、水平偏差以及垂直偏差，分别按照预设计算规则计算滤波器系数。也就是说，在对像素点进行二次预测时，对于双向分离滤波的滤波模式，编码器可以利用像素点的垂直偏差和水平偏差，按照预设计算规则来分别计算该像素点的滤波器系数，以及与该像素点水平相邻和垂直相邻的相邻像素点的滤波器系数。It can be understood that, in this application, when the encoder determines the filter coefficient corresponding to the pixel point according to the filter mode according to the scale parameter and the first motion vector deviation, if the filter mode is bidirectional separation filtering, then the encoder can be based on the scale. Parameters, horizontal deviation and vertical deviation, respectively calculate filter coefficients according to preset calculation rules. That is to say, when performing secondary prediction on a pixel, for the filtering mode of bidirectional separation filtering, the encoder can use the vertical deviation and horizontal deviation of the pixel to calculate the filter coefficient of the pixel according to the preset calculation rule. , and the filter coefficients of the adjacent pixels horizontally and vertically adjacent to the pixel.

进一步地，在本申请的实施例中，编码器基于像素点的第一运动矢量偏差计算获得的滤波器系数中的任意一个滤波器系数，可以为该第一运动矢量偏差的一次函数(多项式)、二次函数(多项式)或者高次函数(多项式)，本申请不作具体限定。Further, in the embodiment of the present application, any one of the filter coefficients in the filter coefficients obtained by the encoder based on the calculation of the first motion vector deviation of the pixel point may be a linear function (polynomial) of the first motion vector deviation , a quadratic function (polynomial), or a higher-order function (polynomial), which is not specifically limited in this application.

也就是说，在本申请中，编码器在按照预设计算规则中不同的计算方法计算获得多个像素点对应的多个滤波器系数中，部分滤波器系数可以为第一运动矢量偏差的一次函数(多项式)，即两者为线性关系，还可以为第一运动矢量偏差的二次函数(多项式)或高次函数(多项式)，即两者为非线性关系。That is to say, in the present application, when the encoder calculates and obtains multiple filter coefficients corresponding to multiple pixels according to different calculation methods in the preset calculation rules, some filter coefficients may be one time of the deviation of the first motion vector. The function (polynomial), that is, the two are in a linear relationship, and may also be a quadratic function (polynomial) or a higher-order function (polynomial) of the first motion vector deviation, that is, the two are in a non-linear relationship.

可见，在本申请的实施例中，在对一个像素点进行二次预测时，编码器基于比例参数和该像素点的第一运动矢量偏差，按照预设计算规则计算获得的滤波器系数是多个不同的滤波器系数，具体地，对于该像素点和与该像素点相邻的其他像素点，每个像素点对应一个滤波器系数。It can be seen that, in the embodiment of the present application, when performing secondary prediction on a pixel, the encoder calculates the filter coefficients obtained according to the preset calculation rule based on the scale parameter and the deviation of the first motion vector of the pixel. different filter coefficients, specifically, for the pixel point and other pixel points adjacent to the pixel point, each pixel point corresponds to one filter coefficient.

进一步地，在本申请的实施例中，编码器在计算滤波器系数的过程中，还可以使用预设上限值和\或预设下限值对最终的计算结果进行限制。例如，当基于第一运动矢量偏差计算获得的滤波器系数大于或者等于预设上限值时，直接将该预设上限值作为对应的滤波器系数进行导出，或者，当基于第一运动矢量偏差计算获得的滤波器系数小于或者等于预设下限值时，直接将该预设下限值作为对应的滤波器系数进行导出。Further, in the embodiments of the present application, in the process of calculating the filter coefficients, the encoder may also use a preset upper limit value and\or a preset lower limit value to limit the final calculation result. For example, when the filter coefficient calculated based on the first motion vector deviation is greater than or equal to the preset upper limit value, the preset upper limit value is directly derived as the corresponding filter coefficient, or when the first motion vector When the filter coefficient obtained by the deviation calculation is less than or equal to the preset lower limit value, the preset lower limit value is directly derived as the corresponding filter coefficient.

需要说明的是，在本申请中，对滤波器系数的大小进行限制的方式可以理解为更加具体的分段函数，因此，编码器在计算滤波器系数时，可以同时采用分段函数的方式和限制大小的方式。It should be noted that, in this application, the method of limiting the size of the filter coefficients can be understood as a more specific piecewise function. Therefore, when the encoder calculates the filter coefficients, the method of the piecewise function and Ways to limit size.

进一步地，在本申请中，如果滤波参数为滤波器相位，那么编码器在基于滤波模式，利用第一运动矢量偏差确定滤波参数时，可以先确定偏差与相位的映射表，然后再根据偏差与相位的映射表和第一运动矢量偏差，按照滤波模式确定像素点对应的滤波器相位。Further, in the present application, if the filter parameter is the filter phase, when the encoder determines the filter parameter by using the first motion vector deviation based on the filter mode, it can first determine the mapping table of the deviation and the phase, and then according to the deviation and The phase mapping table and the first motion vector deviation determine the filter phase corresponding to the pixel point according to the filter mode.

需要说明的是，在本申请中，编码器在确定偏差与相位的映射表时，既可以通过训练获得偏差与相位的映射表，也可以计算获得偏差与相位的映射表。It should be noted that, in the present application, when the encoder determines the mapping table of deviation and phase, the mapping table of deviation and phase can be obtained by training, or the mapping table of deviation and phase can be obtained by calculation.

可以理解的是，在本申请中，编码器在根据偏差与相位的映射表和第一运动矢量偏差，按照滤波模式确定像素点对应的滤波器相位时，如果滤波模式为水平滤波，那么编码器可以按照水平偏差查询偏差与相位的映射表，获得滤波器相位。也就是说，在对像素点进行二次预测时，对于水平滤波的滤波模式，编码器可以利用像素点的水平偏差，直接从偏差与相位的映射表中查询获得该像素点的滤波器相位和与该像素点水平相邻的相邻像素点的滤波器相位。It can be understood that, in this application, when the encoder determines the filter phase corresponding to the pixel point according to the filter mode according to the mapping table of deviation and phase and the first motion vector deviation, if the filter mode is horizontal filtering, then the encoder The filter phase can be obtained by querying the mapping table of deviation and phase according to the horizontal deviation. That is to say, when performing secondary prediction on a pixel, for the filtering mode of horizontal filtering, the encoder can use the horizontal deviation of the pixel to directly query the mapping table of deviation and phase to obtain the filter phase sum of the pixel. The filter phase of the adjacent pixels horizontally adjacent to this pixel.

可以理解的是，在本申请中，编码器在根据偏差与相位的映射表和第一运动矢量偏差，按照滤波模式确定像素点对应的滤波器相位时，如果滤波模式为垂直滤波，那么编码器可以按照垂直偏差查询偏差与相位的映射表，获得滤波器相位。也就是说，在对像素点进行二次预测时，对于垂直滤波的滤波模式，编码器可以利用像素点的垂直偏差，直接从偏差与相位的映射表中查询获得该像素点的滤波器相位和与该像素点垂直相邻的相邻像素点的滤波器相位。It can be understood that, in this application, when the encoder determines the filter phase corresponding to the pixel point according to the filter mode according to the mapping table of deviation and phase and the first motion vector deviation, if the filter mode is vertical filtering, then the encoder The filter phase can be obtained by querying the mapping table of deviation and phase according to the vertical deviation. That is to say, when performing secondary prediction on a pixel, for the filtering mode of vertical filtering, the encoder can use the vertical deviation of the pixel to directly query the mapping table of deviation and phase to obtain the filter phase and The filter phase of the adjacent pixels vertically adjacent to this pixel.

可以理解的是，在本申请中，编码器在根据偏差与相位的映射表和第一运动矢量偏差，按照滤波模式确定像素点对应的滤波器相位时，如果滤波模式为双向分离滤波，那么编码器可以按照水平偏差和垂直偏差，分别查询偏差与相位的映射表，获得滤波器相位。也就是说，在对像素点进行二次预测时，对于双向分离滤波的滤波模式，编码器可以利用像素点的垂直偏差和水平偏差，直接从偏差与相位的映射表中查询获得该像素点的滤波器相位、与该像素点垂直相邻和水平相邻的相邻像素点的滤波器相位。It can be understood that in the present application, when the encoder determines the filter phase corresponding to the pixel point according to the filter mode according to the mapping table of deviation and phase and the first motion vector deviation, if the filter mode is bidirectional separation filtering, then the encoding According to the horizontal deviation and vertical deviation, the filter can query the mapping table of deviation and phase respectively to obtain the filter phase. That is to say, when performing secondary prediction on a pixel, for the filtering mode of bidirectional separation filtering, the encoder can use the vertical deviation and horizontal deviation of the pixel to directly query the mapping table of deviation and phase to obtain the pixel's value. Filter phase, the filter phase of the adjacent pixels vertically and horizontally adjacent to this pixel.

具体地，在本申请中，dmv_x或dmv_y的每一个值对应一个相位，其中，编码器可以根据dmv_x或dmv_y的取值确定二次预测处理时使用的相位。如果滤波器中间的系数是常数，那么也可以用dmv_x或dmv_y的每一绝对值对应一个相位，是负数的话，与相同绝对值的正数相比把上下或左右的滤波器系数对掉即可。Specifically, in this application, each value of dmv_x or dmv_y corresponds to a phase, wherein the encoder can determine the phase used in the secondary prediction process according to the value of dmv_x or dmv_y. If the coefficient in the middle of the filter is constant, then each absolute value of dmv_x or dmv_y can also be used to correspond to a phase. If it is a negative number, compare the upper and lower or left and right filter coefficients with the positive number of the same absolute value. .

步骤505、基于滤波参数和第一预测值，利用可分离滤波器进行滤波处理，获得像素点的预测值。 Step 505 , based on the filtering parameters and the first predicted value, perform filtering processing with a separable filter to obtain the predicted value of the pixel point.

步骤506、根据像素点的预测值，确定子块的第二预测值；其中，第二预测值用于确定子块的残差。Step 506: Determine a second predicted value of the sub-block according to the predicted value of the pixel point; wherein, the second predicted value is used to determine the residual of the sub-block.

在本申请的实施例中，编码器在根据第一运动矢量偏差确定出可分离滤波器的滤波参数之后，便可以基于滤波参数和第一预测值，利用可分离滤波器进行滤波处理，获得像素点的预测值。进而可以根据像素点的预测值，确定出子块的第二预测值。In the embodiment of the present application, after the encoder determines the filtering parameters of the separable filter according to the first motion vector deviation, the encoder can perform filtering processing by using the separable filter based on the filtering parameters and the first predicted value to obtain pixels point forecast. Further, the second predicted value of the sub-block may be determined according to the predicted value of the pixel point.

需要说明的是，在本申请中，编码器可以对子块中的全部像素点进行滤波处理，利用子块中的全部像素点的预测值进行子块的第二预测值的确定，也可以对子块中的部分像素点进行滤波处理，利用子块中的部分像素点的预测值进行子块的第二预测值的确定。It should be noted that, in this application, the encoder may perform filtering processing on all the pixels in the sub-block, and use the predicted values of all the pixels in the sub-block to determine the second predicted value of the sub-block, or it may Filter processing is performed on some pixels in the sub-block, and the second prediction value of the sub-block is determined by using the prediction values of some pixels in the sub-block.

可以理解的是，在本申请的实施例中，编码器利用像素点所对应的第一运动矢量偏差确定出滤波参数，从而可以按照滤波参数，通过可分离滤波器实现像素点的滤波处理，进而实现对第一预测值进行修正，获得修正后的、子块的第二预测值。可见，第二预测值为基于第一预测值的修正后的预测值，同时，第二预测值用于确定子块的残差(residual)。It can be understood that, in the embodiment of the present application, the encoder uses the first motion vector deviation corresponding to the pixel to determine the filtering parameter, so that the filtering process of the pixel can be realized by the separable filter according to the filtering parameter, and then The first predicted value is modified to obtain the modified second predicted value of the sub-block. It can be seen that the second predicted value is a modified predicted value based on the first predicted value, and at the same time, the second predicted value is used to determine the residual of the sub-block.

进一步地，在本申请的实施例中，编码器在基于滤波参数和第一预测值，利用可分离滤波器进行滤波处理，获得像素点的预测值时，可以先对滤波参数，如滤波器系数，与子块的第一预测值进行乘法运算，获得乘积结果，即像素点的预测值。然后可以在遍历子块中的全部像素点或部分像素点之后，再对子块的全部像素点或部分像素点的预测值进行加法运算，获得求和结果，最后可以对加法结果进行归一化处理，最终便可以获得子块修正后的第二预测值。Further, in the embodiment of the present application, when the encoder uses a separable filter to perform filtering processing based on the filtering parameters and the first predicted value, and obtains the predicted value of the pixel point, it can first determine the filtering parameters, such as the filter coefficients. , and perform a multiplication operation with the first predicted value of the sub-block to obtain the multiplication result, that is, the predicted value of the pixel point. Then, after traversing all the pixels or part of the pixels in the sub-block, the predicted values of all the pixels or part of the pixels in the sub-block can be added to obtain the summation result, and finally the addition result can be normalized After processing, finally, the modified second prediction value of the sub-block can be obtained.

也就是说，在本申请中，编码器在基于子块的预测之后，在基于子块的预测的基础上，可以再次进行基于点的二次预测。基于点的二次预测可以使用水平垂直可分离的二维滤波器，即可分离滤波器。具体地，可分离滤波器可以分为水平方向的一维滤波器和垂直方向的一维滤波器。对一个像素点，可分离滤波器进行滤波处理后的结果为该像素点的新的预测值。That is to say, in the present application, the encoder can perform point-based secondary prediction again on the basis of sub-block-based prediction after sub-block-based prediction. Point-based secondary prediction can use two-dimensional filters that are horizontally and vertically separable, i.e., separable filters. Specifically, the separable filters can be divided into one-dimensional filters in the horizontal direction and one-dimensional filters in the vertical direction. For a pixel, the result of filtering the separable filter is the new predicted value of the pixel.

进一步地，在本申请的实施例中，编码器在基于滤波参数和第一预测值，利用可分离滤波器进行滤波处理，获得像素点的预测值时，如果滤波模式为水平滤波，那么编码器可以先基于滤波参数，使用可分离滤波器对第一预测值在水平方向进行滤波处理，获得像素点的预测值；并遍历当前子块中的全部像素点或部分像素点，根据全部像素点或部分像素点对应的的预测值获得第二预测值。Further, in the embodiment of the present application, when the encoder uses a separable filter to perform filtering based on the filtering parameters and the first predicted value to obtain the predicted value of the pixel, if the filtering mode is horizontal filtering, then the encoder Based on the filtering parameters, a separable filter can be used to filter the first predicted value in the horizontal direction to obtain the predicted value of the pixel point; The second predicted value is obtained from the predicted value corresponding to some pixel points.

进一步地，在本申请的实施例中，编码器在基于滤波参数和第一预测值，利用可分离滤波器进行滤波处理，获得像素点的预测值时，如果滤波模式为垂直滤波，那么编码器可以先基于滤波参数，使用可分离滤波器对第一预测值在垂直方向进行滤波处理，获得像素点的预测值；并遍历当前子块中的全部像素点或部分像素点，根据全部像素点对应的预测值获得第二预测值。Further, in the embodiment of the present application, when the encoder uses a separable filter to perform filtering processing based on the filtering parameters and the first predicted value to obtain the predicted value of the pixel point, if the filtering mode is vertical filtering, then the encoder Based on the filtering parameters, a separable filter can be used to filter the first predicted value in the vertical direction to obtain the predicted value of the pixel point; The predicted value of , obtains the second predicted value.

进一步地，在本申请的实施例中，编码器在基于滤波参数和第一预测值，利用可分离滤波器进行滤波处理，获得像素点的预测值时，如果滤波模式为双向分离滤波，那么编码器可以先基于滤波参数，使用可分离滤波器对第一预测值在水平方向进行滤波处理，获得水平滤波结果；然后可以基于滤波参数，对水平滤波结果在垂直方向进行滤波处理，获得像素点的预测值；最终可以遍历当前子块中的全部像素点或部分像素点，根据全部像素点或部分像素点对应的预测值获得第二预测值。Further, in the embodiment of the present application, when the encoder uses a separable filter to perform filtering processing based on the filtering parameter and the first predicted value to obtain the predicted value of the pixel, if the filtering mode is bidirectional separation filtering, then the encoding The filter can first perform filtering processing on the first predicted value in the horizontal direction based on the filtering parameters using a separable filter to obtain the horizontal filtering result; Predicted value; finally, all the pixels or part of the pixels in the current sub-block can be traversed, and the second predicted value can be obtained according to the predicted values corresponding to all the pixels or part of the pixels.

进一步地，在本申请的实施例中，编码器在基于滤波参数和第一预测值，利用可分离滤波器进行滤波处理，获得像素点的预测值时，如果滤波模式为双向分离滤波，那么编码器可以先基于滤波参数，使用可分离滤波器对第一预测值在垂直方向进行滤波处理，获得垂直滤波结果；然后可以基于滤波参数，对垂直滤波结果在水平方向进行滤波处理，获得像素点的预测值；最终可以遍历当前子块中的全部像素点或部分像素点，根据全部像素点或部分像素点对应的预测值获得第二预测值。Further, in the embodiment of the present application, when the encoder uses a separable filter to perform filtering processing based on the filtering parameter and the first predicted value to obtain the predicted value of the pixel, if the filtering mode is bidirectional separation filtering, then the encoding The filter can first perform filtering processing on the first predicted value in the vertical direction based on the filtering parameters using a separable filter to obtain the vertical filtering result; Predicted value; finally, all the pixels or part of the pixels in the current sub-block can be traversed, and the second predicted value can be obtained according to the predicted values corresponding to all the pixels or part of the pixels.

也就是说，在本申请的实施例中，编码器在使用可分离滤波器进行滤波处理时，既可以先进行水平方向的滤波处理，然后再对水平方向的滤波结果进行垂直方向的滤波处理，获得最终的滤波结果。还可以先进行垂直方向的滤波处理，然后再对垂直方向的滤波结果进行水平方向的滤波处理，获得最终的滤波结果。That is to say, in the embodiment of the present application, when the encoder uses a separable filter to perform filtering, it can first perform filtering in the horizontal direction, and then perform filtering in the vertical direction on the filtering result in the horizontal direction. Get the final filtered result. It is also possible to perform filtering processing in the vertical direction first, and then perform filtering processing in the horizontal direction on the filtering result in the vertical direction to obtain the final filtering result.

综上所述，通过步骤501至步骤504所提出的帧间预测方法，在基于子块的预测之后，针对运动矢量与子块的运动矢量有偏差的像素点，在基于子块的预测的基础上进行基于点的二次预测。基于点的二次预测使用可分离滤波器。可分离滤波器可以为横竖可分离的二维滤波器或水平垂直可分离的二维滤波器，可以由水平方向的一维滤波器和垂直方向的一维滤波器构成，最终获得的滤波结果即为像素点的新的预测值。To sum up, with the inter prediction method proposed in steps 501 to 504, after the sub-block-based prediction, for the pixels whose motion vector is deviated from the motion vector of the sub-block, on the basis of the sub-block-based prediction point-based secondary prediction. Point-based quadratic prediction uses separable filters. The separable filter can be a two-dimensional filter that can be separated horizontally and vertically or a two-dimensional filter that can be separated horizontally and vertically. It can be composed of a one-dimensional filter in the horizontal direction and a one-dimensional filter in the vertical direction. The final filtering result is is the new predicted value of the pixel.

本实施例提供了一种帧间预测方法，编码器可以在基于子块的预测之后，针对运动矢量与子块的运动矢量有偏差的像素点，可以在基于子块的第一预测值的基础上，利用可分离滤波器进行基于点的二次预测，获得第二预测值。其中，可分离滤波器可以分离为水平方向的一维滤波器和垂直方向的一维滤波器，滤波后可以获得该像素点对应的新的预测值，最终获得子块修正后的第二预测值，本申请提出的帧间预测方法能够很好的适用于全部场景，大大提升编码性能，从而提高了编解码效率。This embodiment provides an inter-frame prediction method. After the sub-block-based prediction, the encoder may, for pixels whose motion vectors deviate from the sub-block motion vectors, perform the sub-block-based first prediction on the basis of the sub-block-based first prediction value. Above, a second prediction value is obtained by performing point-based secondary prediction using a separable filter. Among them, the separable filter can be separated into a one-dimensional filter in the horizontal direction and a one-dimensional filter in the vertical direction. After filtering, a new predicted value corresponding to the pixel point can be obtained, and finally a second predicted value after sub-block correction can be obtained. , the inter-frame prediction method proposed in this application can be well applied to all scenarios, greatly improving the encoding performance, thereby improving the encoding and decoding efficiency.

基于上述实施例，在本申请的再一实施例中，图16为解码器的组成结构示意图一，如图16所示，本申请实施例提出的解码器300可以包括解析部分301和第一确定部分302；Based on the above embodiments, in yet another embodiment of the present application, FIG. 16 is a schematic diagram of the composition and structure of the decoder. As shown in FIG. 16 , the decoder 300 proposed in the embodiment of the present application may include a parsing part 301 and a first determination section 302;

所述解析部分301，配置为解析码流，获取当前块的预测模式参数；The parsing part 301 is configured to parse the code stream and obtain the prediction mode parameter of the current block;

所述第一确定部分302，配置为当所述预测模式参数指示使用帧间预测模式确定所述当前块的帧间预测值时，确定所述当前块的子块的第一预测值；其中，所述子块包括一个或多个像素点；对于所述子块内的一像素点，确定所述像素点的运动矢量与所述子块的运动矢量之间的第一运动矢量偏差；根据所述第一运动矢量偏差确定滤波参数；基于所述滤波参数和所述第一预测值，利用可分离滤波器进行滤波处理，获得所述像素点的预测值；根据所述像素点的预测值，确定所述子块的第二预测值；其中，所述第二预测值用于确定所述子块的重建值。The first determining part 302 is configured to determine the first prediction value of the sub-block of the current block when the prediction mode parameter indicates that the inter prediction value of the current block is determined using the inter prediction mode; wherein, The sub-block includes one or more pixel points; for a pixel point in the sub-block, determine the first motion vector deviation between the motion vector of the pixel point and the motion vector of the sub-block; according to the The first motion vector deviation determines a filter parameter; based on the filter parameter and the first predicted value, a separable filter is used to perform filtering processing to obtain the predicted value of the pixel point; according to the predicted value of the pixel point, A second prediction value of the sub-block is determined; wherein the second prediction value is used to determine the reconstruction value of the sub-block.

图17为解码器的组成结构示意图二，如图17所示，本申请实施例提出的解码器300还可以包括第一处理器303、存储有第一处理器303可执行指令的第一存储器304、第一通信接口305，和用于连接第一处理器303、第一存储器304以及第一通信接口305的第一总线306。FIG. 17 is a second schematic diagram of the composition and structure of the decoder. As shown in FIG. 17 , the decoder 300 proposed in this embodiment of the present application may further include a first processor 303 and a first memory 304 storing executable instructions of the first processor 303 , a first communication interface 305 , and a first bus 306 for connecting the first processor 303 , the first memory 304 and the first communication interface 305 .

进一步地，在本申请的实施例中，上述第一处理器303，用于解析码流，获取当前块的预测模式参数；当所述预测模式参数指示使用帧间预测模式确定所述当前块的帧间预测值时，确定所述当前块的子块的第一预测值；其中，所述子块包括一个或多个像素点；对于所述子块内的一像素点，确定所述像素点的运动矢量与所述子块的运动矢量之间的第一运动矢量偏差；根据所述第一运动矢量偏差确定滤波参数；基于所述滤波参数和所述第一预测值，利用可分离滤波器进行滤波处理，获得所述像素点的预测值；根据所述像素点的预测值，确定所述子块的第二预测值；其中，所述第二预测值用于确定所述子块的重建值。Further, in the embodiment of the present application, the above-mentioned first processor 303 is configured to parse the code stream and obtain the prediction mode parameter of the current block; when the prediction mode parameter indicates to use the inter prediction mode to determine the prediction mode parameter of the current block When the inter-frame prediction value is used, the first prediction value of the sub-block of the current block is determined; wherein, the sub-block includes one or more pixel points; for a pixel point in the sub-block, the pixel point is determined The first motion vector deviation between the motion vector of the sub-block and the motion vector of the sub-block; the filtering parameter is determined according to the first motion vector deviation; based on the filtering parameter and the first predicted value, a separable filter is used performing filtering processing to obtain the predicted value of the pixel point; determining the second predicted value of the sub-block according to the predicted value of the pixel point; wherein, the second predicted value is used to determine the reconstruction of the sub-block value.

图18为编码器的组成结构示意图一，如图18所示，本申请实施例提出的编码器400可以包括第二确定部分401；FIG. 18 is a schematic diagram 1 of the composition and structure of the encoder. As shown in FIG. 18 , the encoder 400 proposed in this embodiment of the present application may include a second determination part 401;

所述第二确定部分401，配置为确定当前块的预测模式参数；当所述预测模式参数指示使用帧间预测模式确定所述当前块的帧间预测值时，确定所述当前块的子块的第一预测值；其中，所述子块包括一个或多个像素点；对于所述子块内的一像素点，确定所述像素点的运动矢量与所述子块的运动矢量之间的第一运动矢量偏差；根据所述第一运动矢量偏差确定滤波参数；基于所述滤波参数和所述第一预测值，利用可分离滤波器进行滤波处理，获得所述像素点的预测值；根据所述像素点的预测值，确定所述子块的第二预测值；其中，所述第二预测值用于确定所述子块的残差。The second determining part 401 is configured to determine the prediction mode parameter of the current block; when the prediction mode parameter indicates that the inter prediction value of the current block is determined using the inter prediction mode, determine the sub-block of the current block where the sub-block includes one or more pixel points; for a pixel point in the sub-block, determine the difference between the motion vector of the pixel point and the motion vector of the sub-block a first motion vector deviation; determining a filtering parameter according to the first motion vector deviation; using a separable filter to perform filtering based on the filtering parameter and the first predicted value to obtain the predicted value of the pixel point; The predicted value of the pixel point determines the second predicted value of the sub-block; wherein, the second predicted value is used to determine the residual of the sub-block.

图19为编码器的组成结构示意图二，如图19所示，本申请实施例提出的编码器400还可以包括第二处理器402、存储有第二处理器402可执行指令的第二存储器403、第二通信接口404，和用于连接第二处理器402、第二存储器403以及第二通信接口404的第二总线405。FIG. 19 is a second schematic diagram of the composition and structure of the encoder. As shown in FIG. 19 , the encoder 400 proposed in this embodiment of the present application may further include a second processor 402 and a second memory 403 storing executable instructions of the second processor 402 , a second communication interface 404 , and a second bus 405 for connecting the second processor 402 , the second memory 403 and the second communication interface 404 .

进一步地，在本申请的实施例中，上述第二处理器402，配置为确定当前块的预测模式参数；当所述预测模式参数指示使用帧间预测模式确定所述当前块的帧间预测值时，确定所述当前块的子块的第一预测值；其中，所述子块包括一个或多个像素点；对于所述子块内的一像素点，确定所述像素点的运动矢量与所述子块的运动矢量之间的第一运动矢量偏差；根据所述第一运动矢量偏差确定滤波参数；基于所述滤波参数和所述第一预测值，利用可分离滤波器进行滤波处理，获得所述像素点的预测值；根据所述像素点的预测值，确定所述子块的第二预测值；其中，所述第二预测值用于确定所述子块的残差。Further, in the embodiment of the present application, the above-mentioned second processor 402 is configured to determine the prediction mode parameter of the current block; when the prediction mode parameter indicates that the inter prediction mode is used to determine the inter prediction value of the current block When , determine the first predicted value of the sub-block of the current block; wherein, the sub-block includes one or more pixel points; for a pixel in the sub-block, determine the motion vector of the pixel and a first motion vector deviation between the motion vectors of the sub-blocks; a filtering parameter is determined according to the first motion vector deviation; based on the filtering parameter and the first predicted value, a separable filter is used to perform filtering processing, Obtain the predicted value of the pixel point; determine the second predicted value of the sub-block according to the predicted value of the pixel point; wherein, the second predicted value is used to determine the residual of the sub-block.

本申请实施例提供了一种编码器和编码器，可以在基于子块的预测之后，针对运动矢量与子块的运动矢量有偏差的像素点，可以在基于子块的第一预测值的基础上，利用可分离滤波器进行基于点的二次预测，获得第二预测值。其中，可分离滤波器可以分离为水平方向的一维滤波器和垂直方向的一维滤波器，滤波后可以获得该像素点对应的新的预测值，最终获得子块修正后的第二预测值，本申请提出的帧间预测方法能够很好的适用于全部场景，大大提升编码性能，从而提高了编解码效率。Embodiments of the present application provide an encoder and an encoder that, after sub-block-based prediction, for pixels whose motion vectors are deviated from the motion vector of the sub-block, the first prediction value based on the sub-block can be used for pixel points. Above, a second prediction value is obtained by performing point-based secondary prediction using a separable filter. Among them, the separable filter can be separated into a one-dimensional filter in the horizontal direction and a one-dimensional filter in the vertical direction. After filtering, a new predicted value corresponding to the pixel point can be obtained, and finally a second predicted value after sub-block correction can be obtained. , the inter-frame prediction method proposed in this application can be well applied to all scenarios, greatly improving the encoding performance, thereby improving the encoding and decoding efficiency.

本申请实施例提供计算机可读存储介质和计算机可读存储介质，其上存储有程序，该程序被处理器执行时实现如上述实施例所述的方法。Embodiments of the present application provide a computer-readable storage medium and a computer-readable storage medium, on which a program is stored, and when the program is executed by a processor, the method described in the foregoing embodiments is implemented.

具体来讲，本实施例中的一种帧间预测方法对应的程序指令可以被存储在光盘，硬盘，U盘等存储介质上，当存储介质中的与一种帧间预测方法对应的程序指令被一电子设备读取或被执行时，包括如下步骤：Specifically, a program instruction corresponding to an inter-frame prediction method in this embodiment may be stored on a storage medium such as an optical disc, a hard disk, a U disk, etc. When the program instruction corresponding to an inter-frame prediction method in the storage medium When read or executed by an electronic device, it includes the following steps:

本领域内的技术人员应明白，本申请的实施例可提供为方法、***、或计算机程序产品。因此，本申请可采用硬件实施例、软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器和光学存储器等)上实施的计算机程序产品的形式。As will be appreciated by those skilled in the art, the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the application may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media having computer-usable program code embodied therein, including but not limited to disk storage, optical storage, and the like.

本申请是参照根据本申请实施例的方法、设备(***)、和计算机程序产品的实现流程示意图和/或方框图来描述的。应理解可由计算机程序指令实现流程示意图和/或方框图中的每一流程和/或方框、以及实现流程示意图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在实现流程示意图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。The present application is described with reference to schematic flowcharts and/or block diagrams of implementations of methods, apparatuses (systems), and computer program products according to embodiments of the present application. It will be understood that each process and/or block in the schematic flowchart illustrations and/or block diagrams, and combinations of processes and/or blocks in the schematic flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a process or processes and/or a block or blocks in the block diagrams.

这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在实现流程示意图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions An apparatus implements the functions specified in a flow or flows of the implementation flow diagram and/or a block or blocks of the block diagram.

这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在实现流程示意图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the implementing flow diagram and/or the block or blocks of the block diagram.

本申请所提供的几个方法实施例中所揭露的方法，在不冲突的情况下可以任意组合，得到新的方法实施例。本申请所提供的几个产品实施例中所揭露的特征，在不冲突的情况下可以任意组合，得到新的产品实施例。本申请所提供的几个方法或设备实施例中所揭露的特征，在不冲突的情况下可以任意组合，得到新的方法实施例或设备实施例。The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined under the condition of no conflict to obtain new method embodiments. The features disclosed in the several product embodiments provided in this application can be combined arbitrarily without conflict to obtain a new product embodiment. The features disclosed in several method or device embodiments provided in this application can be combined arbitrarily without conflict to obtain new method embodiments or device embodiments.

以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。The above are only specific embodiments of the present application, but the protection scope of the present application is not limited to this. should be covered within the scope of protection of this application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

工业实用性Industrial Applicability

Claims

一种帧间预测方法，应用于解码器，所述方法包括：An inter-frame prediction method, applied to a decoder, the method comprising:

解析码流，获取当前块的预测模式参数；Parse the code stream to obtain the prediction mode parameters of the current block;

当所述预测模式参数指示使用帧间预测模式确定所述当前块的帧间预测值时，确定所述当前块的子块的第一预测值；其中，所述子块包括一个或多个像素点；When the prediction mode parameter indicates that the inter prediction value of the current block is determined using the inter prediction mode, a first prediction value of a sub-block of the current block is determined; wherein the sub-block includes one or more pixels point;

对于所述子块内的一像素点，确定所述像素点的运动矢量与所述子块的运动矢量之间的第一运动矢量偏差；For a pixel in the sub-block, determining a first motion vector deviation between the motion vector of the pixel and the motion vector of the sub-block;

根据所述第一运动矢量偏差确定滤波参数；determining filtering parameters according to the first motion vector deviation;

基于所述滤波参数和所述第一预测值，利用可分离滤波器进行滤波处理，获得所述像素点的预测值；Based on the filtering parameters and the first predicted value, filtering is performed using a separable filter to obtain the predicted value of the pixel point;

根据所述像素点的预测值，确定所述子块的第二预测值；其中，所述第二预测值用于确定所述子块的重建值。According to the predicted value of the pixel point, a second predicted value of the sub-block is determined; wherein, the second predicted value is used to determine the reconstructed value of the sub-block.
根据权利要求1所述的方法，其中，所述根据所述第一运动矢量偏差确定滤波参数，包括：The method according to claim 1, wherein the determining a filtering parameter according to the first motion vector deviation comprises:

确定滤波模式；其中，所述滤波模式包括水平滤波、垂直滤波以及双向分离滤波；determining a filtering mode; wherein, the filtering mode includes horizontal filtering, vertical filtering and bidirectional separation filtering;

基于所述滤波模式，利用所述第一运动矢量偏差确定所述滤波参数；其中，所述第一运动矢量偏差包括水平偏差和垂直偏差。Based on the filtering mode, the filtering parameters are determined using the first motion vector offset; wherein the first motion vector offset includes a horizontal offset and a vertical offset.
根据权利要求2所述的方法，其中，所述确定滤波模式，包括：The method of claim 2, wherein said determining a filtering mode comprises:

确定水平参数和垂直参数；Determine horizontal and vertical parameters;

若所述水平参数大于或者等于预设水平阈值，且所述垂直参数小于预设垂直阈值，则确定滤波模式为所述水平滤波；If the horizontal parameter is greater than or equal to a preset horizontal threshold, and the vertical parameter is less than a preset vertical threshold, determining that the filtering mode is the horizontal filtering;

若所述水平参数小于所述预设水平阈值，且所述垂直参数大于或者等于所述预设垂直阈值，则确定滤波模式为所述垂直滤波；If the horizontal parameter is less than the preset horizontal threshold, and the vertical parameter is greater than or equal to the preset vertical threshold, determining that the filtering mode is the vertical filtering;

若所述水平参数大于或者等于所述预设水平阈值，且所述垂直参数大于或者等于所述预设垂直阈值，则确定滤波模式为所述双向分离滤波。If the horizontal parameter is greater than or equal to the preset horizontal threshold, and the vertical parameter is greater than or equal to the preset vertical threshold, the filtering mode is determined to be the bidirectional separation filtering.
根据权利要求3所述的方法，其中，The method of claim 3, wherein,

所述水平参数和所述垂直参数为所述当前块在水平方向和垂直方向上的运动信息；或者，The horizontal parameter and the vertical parameter are motion information of the current block in the horizontal direction and the vertical direction; or,

所述水平参数和所述垂直参数为所述子块在水平方向和垂直方向上的运动信息；或者，The horizontal parameter and the vertical parameter are motion information of the sub-block in the horizontal direction and the vertical direction; or,

所述水平参数和所述垂直参数为所述像素点在水平方向和垂直方向上的运动信息。The horizontal parameter and the vertical parameter are motion information of the pixel in the horizontal direction and the vertical direction.
根据权利要求4所述的方法，其中，若所述滤波参数为滤波器系数，所述基于所述滤波模式，利用所述第一运动矢量偏差确定所述滤波参数，包括：The method according to claim 4, wherein, if the filtering parameter is a filter coefficient, the determining the filtering parameter by using the first motion vector deviation based on the filtering mode comprises:

解析所述码流，获取比例参数；Parse the code stream to obtain the scale parameter;

根据所述比例参数和所述第一运动矢量偏差，按照所述滤波模式确定所述像素点对应的所述滤波器系数。The filter coefficient corresponding to the pixel point is determined according to the filter mode according to the scale parameter and the first motion vector deviation.
根据权利要求5所述的方法，其中，所述根据所述比例参数和所述第一运动矢量偏差，按照所述滤波模式确定所述像素点对应的所述滤波器系数，包括：The method according to claim 5, wherein the determining the filter coefficient corresponding to the pixel point according to the filter mode according to the scale parameter and the first motion vector deviation comprises:

若所述滤波模式为所述水平滤波，则基于所述比例参数和所述水平偏差，按照预设计算规则获得所述滤波器系数；If the filtering mode is the horizontal filtering, the filter coefficient is obtained according to a preset calculation rule based on the scale parameter and the horizontal deviation;

若所述滤波模式为所述垂直滤波，则基于所述比例参数和所述垂直偏差，按照预设计算规则获得所述滤波器系数；If the filtering mode is the vertical filtering, the filter coefficient is obtained according to a preset calculation rule based on the scale parameter and the vertical deviation;

若所述滤波模式为所述双向分离滤波，则基于所述比例参数、所述水平偏差以及所述垂直偏差，按照预设计算规则获得所述滤波器系数。If the filtering mode is the bidirectional separation filtering, the filter coefficient is obtained according to a preset calculation rule based on the scale parameter, the horizontal deviation and the vertical deviation.
根据权利要求6所述的方法，其中，所述比例参数包括至少一个比例值，所述至少一个比例值均为非零实数。The method of claim 6, wherein the scale parameter includes at least one scale value, each of which is a non-zero real number.
根据权利要求6所述的方法，其中，所述滤波器系数为所述第一运动矢量偏差的一次函数、二次函数或者高次函数。The method of claim 6, wherein the filter coefficient is a linear function, a quadratic function, or a higher-order function of the first motion vector deviation.
根据权利要求4所述的方法，其中，若所述滤波参数为滤波器相位，所述基于所述滤波模式，利用所述第一运动矢量偏差确定所述滤波参数，包括：The method according to claim 4, wherein, if the filtering parameter is a filter phase, the determining the filtering parameter by using the first motion vector deviation based on the filtering mode comprises:

确定偏差与相位的映射表；Determine the mapping table of deviation and phase;

根据所述偏差与相位的映射表和所述第一运动矢量偏差，按照所述滤波模式确定所述像素点对应的所述滤波器相位。The filter phase corresponding to the pixel point is determined according to the filtering mode according to the mapping table of the deviation and the phase and the first motion vector deviation.
根据权利要求9所述的方法，其中，所述确定偏差与相位的映射表，包括：The method according to claim 9, wherein said determining the mapping table of deviation and phase comprises:

训练获得所述偏差与相位的映射表。The training obtains the mapping table of the bias and phase.
根据权利要求9所述的方法，其中，所述确定偏差与相位的映射表，包括：The method according to claim 9, wherein said determining the mapping table of deviation and phase comprises:

计算获得所述偏差与相位的映射表。The mapping table of the deviation and the phase is obtained by calculation.
根据权利要求9所述的方法，其中，所述根据所述偏差与相位的映射表和所述第一运动矢量偏差，按照所述滤波模式确定所述像素点对应的所述滤波器相位，包括：The method according to claim 9, wherein the determining the filter phase corresponding to the pixel point according to the filtering mode according to the mapping table of the deviation and the phase and the first motion vector deviation, comprising: :

若所述滤波模式为所述水平滤波，则按照所述水平偏差查询所述偏差与相位的映射表，获得所述滤波器相位；If the filtering mode is the horizontal filtering, query the mapping table of the deviation and the phase according to the horizontal deviation to obtain the filter phase;

若所述滤波模式为所述垂直滤波，则按照所述垂直偏差查询所述偏差与相位的映射表，获得所述滤波器相位；If the filtering mode is the vertical filtering, query the mapping table of the deviation and the phase according to the vertical deviation to obtain the filter phase;

若所述滤波模式为所述双向分离滤波，则按照所述水平偏差和所述垂直偏差，分别查询所述偏差与相位的映射表，获得所述滤波器相位。If the filtering mode is the bidirectional separation filtering, according to the horizontal deviation and the vertical deviation, the mapping table of the deviation and the phase is respectively inquired to obtain the filter phase.
根据权利要求12所述的方法，其中，所述方法还包括：The method of claim 12, wherein the method further comprises:

确定所述第一运动矢量偏差的精度参数，并基于所述精度参数确定缩小参数；determining a precision parameter of the first motion vector deviation, and determining a reduction parameter based on the precision parameter;

在利用所述滤波器相位进行滤波处理之后，按照所述缩小参数进行缩小处理和/或右移处理。After the filtering process is performed using the filter phase, the reduction process and/or the right shift process is performed according to the reduction parameter.
根据权利要求5或9所述的方法，其中，所述基于所述滤波参数和所述第一预测值，利用可分离滤波器进行滤波处理，获得所述像素点的预测值，包括：The method according to claim 5 or 9, wherein, based on the filtering parameter and the first predicted value, the filtering process is performed using a separable filter to obtain the predicted value of the pixel, including:

若所述滤波模式为所述双向分离滤波，则基于所述滤波参数，使用所述可分离滤波器对所述第一预测值在水平方向进行滤波处理，获得水平滤波结果；If the filtering mode is the bidirectional separation filtering, then based on the filtering parameters, use the separable filter to perform filtering processing on the first predicted value in the horizontal direction to obtain a horizontal filtering result;

基于所述滤波参数，对所述水平滤波结果在垂直方向进行滤波处理，获得所述像素点的预测值。Based on the filtering parameters, filtering processing is performed on the horizontal filtering result in the vertical direction to obtain the predicted value of the pixel point.
根据权利要求5或9所述的方法，其中，所述基于所述滤波参数和所述第一预测值，利用可分离滤波器进行滤波处理，获得所述像素点的预测值，包括：The method according to claim 5 or 9, wherein, based on the filtering parameters and the first predicted value, performing filtering processing with a separable filter to obtain the predicted value of the pixel point comprises:

若所述滤波模式为所述双向分离滤波，则基于所述滤波参数，使用所述可分离滤波器对所述第一预测值在垂直方向进行滤波处理，获得垂直滤波结果；If the filtering mode is the two-way separation filtering, then based on the filtering parameters, use the separable filter to perform filtering processing on the first predicted value in the vertical direction to obtain a vertical filtering result;

基于所述滤波参数，对所述垂直滤波结果在水平方向进行滤波处理，获得所述像素点的预测值。Based on the filtering parameters, filtering processing is performed on the vertical filtering result in the horizontal direction to obtain the predicted value of the pixel point.
根据权利要求6所述的方法，其特征在于，所述方法还包括：The method according to claim 6, wherein the method further comprises:

对于不同的子块，使用不同的比例参数和不同的预设计算规则确定所述滤波器系数。For different sub-blocks, different scale parameters and different preset calculation rules are used to determine the filter coefficients.
根据权利要求9所述的方法，其特征在于，所述方法还包括：The method according to claim 9, wherein the method further comprises:

对于不同的子块，使用不同的偏差与相位的映射表确定所述滤波器系数。The filter coefficients are determined using different offset-to-phase mapping tables for different sub-blocks.
根据权利要求1所述的方法，其中，所述确定所述当前块的子块的第一预测值，包括：The method of claim 1, wherein the determining the first predicted value of the sub-block of the current block comprises:

确定所述当前块的子块的运动矢量；determining motion vectors for sub-blocks of the current block;

基于所述运动矢量确定所述子块的第一预测值。A first predictor for the subblock is determined based on the motion vector.
根据权利要求1所述的方法，其中，所述确定所述当前块的子块的第一预测值，包括：The method of claim 1, wherein the determining the first predicted value of the sub-block of the current block comprises:

确定所述子块与所述第二运动矢量偏差；determining that the sub-block is offset from the second motion vector;

基于所述第二运动矢量偏差，确定所述子块的第一预测值。Based on the second motion vector offset, a first predictor for the sub-block is determined.
根据权利要求1所述的方法，其中，所述方法还包括：The method of claim 1, wherein the method further comprises:

若所述像素点对应的相邻像素点不属于所述子块，则根据所述子块对所述相邻像素点进行更新处理，获得更新后像素点；If the adjacent pixel point corresponding to the pixel point does not belong to the sub-block, update the adjacent pixel point according to the sub-block to obtain the updated pixel point;

基于所述第一预测值和所述更新后像素点，确定所述子块的第二预测值，将所述第二预测值确定为所述子块的帧间预测值。Based on the first predicted value and the updated pixel points, a second predicted value of the sub-block is determined, and the second predicted value is determined as an inter-frame predicted value of the sub-block.
根据权利要求20所述的方法，其中，所述根据所述子块对所述相邻像素点进行更新处理，获得更新后像素点，包括：The method according to claim 20, wherein the updating the adjacent pixels according to the sub-blocks to obtain the updated pixels, comprising:

对所述子块进行扩展处理，获得扩展后子块；Expanding the sub-block to obtain the expanded sub-block;

在所述扩展后子块内确定所述相邻像素点对应的所述更新后像素点。The updated pixel points corresponding to the adjacent pixel points are determined in the extended sub-block.
根据权利要求21所述的方法，其中，所述对所述子块进行扩展处理，获得扩展后子块，包括：The method according to claim 21, wherein the performing expansion processing on the sub-blocks to obtain the expanded sub-blocks comprises:

利用所述子块的全部边界位置进行扩展处理，获得所述扩展后子块；或者，Perform expansion processing by using all boundary positions of the sub-block to obtain the expanded sub-block; or,

利用所述子块内的、所述相邻像素点对应的行和\或列的边界位置进行扩展处理，获得所述扩展后子块。Expansion processing is performed using the boundary positions of the rows and/or columns corresponding to the adjacent pixels in the sub-block to obtain the expanded sub-block.
根据权利要求20所述的方法，其中，所述根据所述子块对所述相邻像素点进行更新处理，获得更新后像素点，包括：The method according to claim 20, wherein the updating the adjacent pixels according to the sub-blocks to obtain the updated pixels, comprising:

利用所述子块内的、与所述相邻像素点相邻的像素点，替换所述相邻像素点，获得所述更新后像素点。The updated pixel is obtained by replacing the adjacent pixel with a pixel in the sub-block that is adjacent to the adjacent pixel.
根据权利要求19所述的方法，其中，所述方法还包括：The method of claim 19, wherein the method further comprises:

当所述预测模式参数指示使用帧间预测模式确定所述当前块的帧间预测值时，确定所述子块的扩展子块；determining an extended subblock of the subblock when the prediction mode parameter indicates that the inter prediction value of the current block is determined using an inter prediction mode;

基于所述运动矢量确定所述扩展子块的第三预测值，并确定所述像素点对应的所述相邻像素点；determining a third prediction value of the extended sub-block based on the motion vector, and determining the adjacent pixel points corresponding to the pixel points;

若所述相邻像素点不属于所述子块，基于所述第三预测值和所述相邻像素点，确定所述第二预测值，将所述第二预测值确定为所述帧间预测值。If the adjacent pixel point does not belong to the sub-block, the second prediction value is determined based on the third prediction value and the adjacent pixel point, and the second prediction value is determined as the inter frame Predictive value.
根据权利要求24所述的方法，其中，The method of claim 24, wherein,

对所述扩展子块使用n抽头的插值滤波器，获得所述第三预测值；其中，所述n为以下值中的任一者：6，5，4，3，2。The third predicted value is obtained using an n-tap interpolation filter on the extended sub-block; wherein the n is any of the following values: 6, 5, 4, 3, 2.
根据权利要求25所述的方法，其中，The method of claim 25, wherein,

对所述子块对应的参考像素进行扩展处理，获得扩展后参考像素；performing expansion processing on the reference pixels corresponding to the sub-blocks to obtain expanded reference pixels;

基于所述扩展后参考像素，使用所述子块对应的插值滤波器，获得所述第三预测值。Based on the expanded reference pixels, the third predicted value is obtained using an interpolation filter corresponding to the sub-block.
根据权利要求25所述的方法，其中，The method of claim 25, wherein,

在所述当前块中确定所述扩展子块对应的目标像素点；Determine the target pixel point corresponding to the extended sub-block in the current block;

根据所述目标像素点的整像素值确定所述第三预测值。The third predicted value is determined according to the integer pixel value of the target pixel.
根据权利要求2所述的方法，其中，所述方法还包括：The method of claim 2, wherein the method further comprises:

根据所述第一运动矢量偏差确定所述像素点对应的中心位置和第三运动矢量偏差；Determine the center position corresponding to the pixel point and the third motion vector deviation according to the first motion vector deviation;

基于所述第一预测值，按照所述中心位置和所述第三运动矢量偏差确定所述子块的第二预测值，将所述第二预测值确定为所述子块的帧间预测值。Based on the first predicted value, a second predicted value of the sub-block is determined according to the center position and the third motion vector deviation, and the second predicted value is determined as an inter-frame predicted value of the sub-block .
根据权利要求28所述的方法，其中，所述根据所述第一运动矢量偏差确定所述像素点对应的中心位置和第三运动矢量偏差，包括：The method according to claim 28, wherein the determining the center position corresponding to the pixel point and the third motion vector deviation according to the first motion vector deviation comprises:

确定所述水平偏差的第一绝对值和所述垂直偏差的第二绝对值；determining a first absolute value of the horizontal deviation and a second absolute value of the vertical deviation;

若所述第一绝对值和所述第二绝对值均小于预设偏差阈值，则将所述第一运动矢量偏差确定为所述第三运动矢量偏差，将所述像素点确定为所述中心位置；If both the first absolute value and the second absolute value are smaller than a preset deviation threshold, the first motion vector deviation is determined as the third motion vector deviation, and the pixel point is determined as the center Location;

若所述第一绝对值大于或者等于所述预设偏差阈值，且所述第二绝对值小于所述预设偏差阈值，则根据所述水平偏差确定第一调整方向，并根据所述第一调整方向，确定所述中心位置和所述第三运动矢量偏差；其中，所述第一调整方向包括左侧和右侧；If the first absolute value is greater than or equal to the preset deviation threshold, and the second absolute value is smaller than the preset deviation threshold, the first adjustment direction is determined according to the horizontal deviation, and the first adjustment direction is determined according to the first Adjusting the direction, determining the center position and the deviation of the third motion vector; wherein, the first adjustment direction includes left and right;

若所述第二绝对值大于或者等于所述预设偏差阈值，且所述第一绝对值小于所述预设偏差阈值，则根据所述垂直偏差确定第二调整方向，并根据所述第二调整方向，确定所述中心位置和所述第三运动矢量偏差；其中，所述第二调整方向包括上侧和下侧If the second absolute value is greater than or equal to the preset deviation threshold, and the first absolute value is smaller than the preset deviation threshold, a second adjustment direction is determined according to the vertical deviation, and the second adjustment direction is determined according to the second Adjusting the direction, determining the center position and the deviation of the third motion vector; wherein, the second adjustment direction includes an upper side and a lower side
根据权利要求29所述的方法，其中，所述预设偏差阈值为k个像素；其中，k大于0.5且小于或者等于1。The method according to claim 29, wherein the preset deviation threshold is k pixels; wherein, k is greater than 0.5 and less than or equal to 1.
根据权利要求2或28所述的方法，其中，所述根据所述第一运动矢量偏差确定所述像素点对应的中心位置和第三运动矢量偏差之前，所述方法还包括：The method according to claim 2 or 28, wherein before the determining the center position corresponding to the pixel point and the third motion vector deviation according to the first motion vector deviation, the method further comprises:

按照预设偏差范围对所述第一运动矢量偏差进行限制处理；其中，所述预设偏差范围包括偏差下限值和偏差上限值。The first motion vector deviation is limited according to a preset deviation range; wherein, the preset deviation range includes a deviation lower limit value and a deviation upper limit value.
根据权利要求31所述的方法，其中，所述按照预设偏差范围对所述第一运动矢量偏差进行限制处理，包括：The method according to claim 31, wherein the limiting the deviation of the first motion vector according to a preset deviation range comprises:

若所述水平偏差和\或所述垂直偏差小于所述偏差下限值，则将所述水平偏差和\或所述垂直偏差设置为所述偏差下限值；If the horizontal deviation and\or the vertical deviation are less than the deviation lower limit value, then the horizontal deviation and\or the vertical deviation are set as the deviation lower limit value;

若所述水平偏差和\或所述垂直偏差大于所述偏差上限值，则将所述水平偏差和\或所述垂直偏差设置为所述偏差上限值。If the horizontal deviation and\or the vertical deviation are greater than the deviation upper limit value, set the horizontal deviation and\or the vertical deviation as the deviation upper limit value.
根据权利要求1所述的方法，其中，所述确定所述当前块的子块的运动矢量，包括：The method of claim 1, wherein the determining a motion vector of a sub-block of the current block comprises:

解析所述码流，获取所述当前块的仿射模式参数和预测参考模式；Parse the code stream to obtain the affine mode parameter and prediction reference mode of the current block;

当所述仿射模式参数指示使用仿射模式时，确定控制点模式和子块尺寸参数；When the affine mode parameter indicates to use the affine mode, determining a control point mode and a sub-block size parameter;

根据所述预测参考模式、所述控制点模式以及所述子块尺寸参数，确定所述运动矢量。The motion vector is determined according to the prediction reference mode, the control point mode, and the subblock size parameter.
根据权利要求1所述的方法，其特征在于，所述方法还包括：The method according to claim 1, wherein the method further comprises:

解析所述码流，确定帧间预测方向；Parse the code stream to determine the inter prediction direction;

若所述帧间预测方向为双向预测，则基于第一预测方向，确定所述子块的第四预测值和所述像素点的运动矢量与所述子块的运动矢量之间的第四运动矢量偏差；基于第二预测方向，确定所述子块的第五预测值和所述像素点的运动矢量与所述子块的运动矢量之间的第五运动矢量偏差；If the inter-frame prediction direction is bidirectional prediction, determining the fourth prediction value of the sub-block and the fourth motion between the motion vector of the pixel point and the motion vector of the sub-block based on the first prediction direction vector deviation; determining the fifth prediction value of the sub-block and the fifth motion vector deviation between the motion vector of the pixel point and the motion vector of the sub-block based on the second prediction direction;

基于所述第四运动矢量偏差，使用所述可分离滤波器对所述第四预测值进行二次预测，获得第六预测值；基于所述第五运动矢量偏差，使用所述可分离滤波器对所述第五预测值进行二次预测，获得第七预测值；Based on the fourth motion vector deviation, use the separable filter to perform secondary prediction on the fourth predicted value to obtain a sixth predicted value; based on the fifth motion vector deviation, use the separable filter performing secondary prediction on the fifth predicted value to obtain a seventh predicted value;

根据所述第六预测值和所述第七预测值，确定所述子块的帧间预测值。An inter-frame prediction value of the sub-block is determined according to the sixth prediction value and the seventh prediction value.
一种帧间预测方法，应用于编码器，所述方法包括：An inter-frame prediction method, applied to an encoder, the method comprising:

确定当前块的预测模式参数；determine the prediction mode parameter of the current block;

当所述预测模式参数指示使用帧间预测模式确定所述当前块的帧间预测值时，确定所述当前块的子块的第一预测值；其中，所述子块包括一个或多个像素点；When the prediction mode parameter indicates that the inter prediction value of the current block is determined using the inter prediction mode, a first prediction value of a sub-block of the current block is determined; wherein the sub-block includes one or more pixels point;

对于所述子块内的一像素点，确定所述像素点的运动矢量与所述子块的运动矢量之间的第一运动矢量偏差；For a pixel in the sub-block, determining a first motion vector deviation between the motion vector of the pixel and the motion vector of the sub-block;

根据所述第一运动矢量偏差确定滤波参数；determining filtering parameters according to the first motion vector deviation;

基于所述滤波参数和所述第一预测值，利用可分离滤波器进行滤波处理，获得所述像素点的预测值；Based on the filtering parameters and the first predicted value, filtering is performed using a separable filter to obtain the predicted value of the pixel point;

根据所述像素点的预测值，确定所述子块的第二预测值；其中，所述第二预测值用于确定所述子块的残差。According to the predicted value of the pixel point, a second predicted value of the sub-block is determined; wherein, the second predicted value is used to determine the residual of the sub-block.
根据权利要求35所述的方法，其中，所述根据所述第一运动矢量偏差确定滤波参数，包括：The method of claim 35, wherein the determining a filtering parameter according to the first motion vector deviation comprises:

确定滤波模式；其中，所述滤波模式包括水平滤波、垂直滤波以及双向分离滤波；determining a filtering mode; wherein, the filtering mode includes horizontal filtering, vertical filtering and bidirectional separation filtering;

基于所述滤波模式，利用所述第一运动矢量偏差确定所述滤波参数；其中，所述第一运动矢量偏差包括水平偏差和垂直偏差。Based on the filtering mode, the filtering parameters are determined using the first motion vector offset; wherein the first motion vector offset includes a horizontal offset and a vertical offset.
根据权利要求36所述的方法，其中，所述确定滤波模式，包括：The method of claim 36, wherein said determining a filtering mode comprises:

确定水平参数和垂直参数；Determine horizontal and vertical parameters;

若所述水平参数大于或者等于预设水平阈值，且所述垂直参数小于预设垂直阈值，则确定滤波模式为所述水平滤波；If the horizontal parameter is greater than or equal to a preset horizontal threshold, and the vertical parameter is less than a preset vertical threshold, determining that the filtering mode is the horizontal filtering;

若所述水平参数小于所述预设水平阈值，且所述垂直参数大于或者等于所述预设垂直阈值，则确定滤波模式为所述垂直滤波；If the horizontal parameter is less than the preset horizontal threshold, and the vertical parameter is greater than or equal to the preset vertical threshold, determining that the filtering mode is the vertical filtering;

若所述水平参数大于或者等于所述预设水平阈值，且所述垂直参数大于或者等于所述预设垂直阈值，则确定滤波模式为所述双向分离滤波。If the horizontal parameter is greater than or equal to the preset horizontal threshold, and the vertical parameter is greater than or equal to the preset vertical threshold, the filtering mode is determined to be the bidirectional separation filtering.
根据权利要求37所述的方法，其中，The method of claim 37, wherein,

所述水平参数和所述垂直参数为所述当前块在水平方向和垂直方向上的运动信息；或者，The horizontal parameter and the vertical parameter are motion information of the current block in the horizontal direction and the vertical direction; or,

所述水平参数和所述垂直参数为所述子块在水平方向和垂直方向上的运动信息；或者，The horizontal parameter and the vertical parameter are motion information of the sub-block in the horizontal direction and the vertical direction; or,

所述水平参数和所述垂直参数为所述像素点在水平方向和垂直方向上的运动信息。The horizontal parameter and the vertical parameter are motion information of the pixel in the horizontal direction and the vertical direction.
根据权利要求38所述的方法，其中，若所述滤波参数为滤波器系数，所述基于所述滤波模式，利用所述第一运动矢量偏差确定所述滤波参数，包括：The method according to claim 38, wherein, if the filtering parameter is a filter coefficient, the determining the filtering parameter by using the first motion vector deviation based on the filtering mode comprises:

确定比例参数；determine the scale parameters;

根据所述比例参数和所述第一运动矢量偏差，按照所述滤波模式确定所述像素点对应的所述滤波器系数。The filter coefficient corresponding to the pixel point is determined according to the filter mode according to the scale parameter and the first motion vector deviation.
根据权利要求39所述的方法，其中，所述根据所述比例参数和所述第一运动矢量偏差，按照所述滤波模式确定所述像素点对应的所述滤波器系数，包括：The method according to claim 39, wherein the determining the filter coefficient corresponding to the pixel point according to the filter mode according to the scale parameter and the first motion vector deviation comprises:

若所述滤波模式为所述水平滤波，则基于所述比例参数和所述水平偏差，按照预设计算规则获得所述滤波器系数；If the filtering mode is the horizontal filtering, the filter coefficient is obtained according to a preset calculation rule based on the scale parameter and the horizontal deviation;

若所述滤波模式为所述垂直滤波，则基于所述比例参数和所述垂直偏差，按照预设计算规则获得所述滤波器系数；If the filtering mode is the vertical filtering, then based on the proportional parameter and the vertical deviation, obtain the filter coefficient according to a preset calculation rule;

若所述滤波模式为所述双向分离滤波，则基于所述比例参数、所述水平偏差以及所述垂直偏差，按照预设计算规则获得所述滤波器系数。If the filtering mode is the bidirectional separation filtering, the filter coefficient is obtained according to a preset calculation rule based on the scale parameter, the horizontal deviation and the vertical deviation.
根据权利要求40所述的方法，其中，所述比例参数包括至少一个比例值，所述至少一个比例值均为非零实数。41. The method of claim 40, wherein the scale parameter includes at least one scale value, each of which is a non-zero real number.
根据权利要求40所述的方法，其中，所述滤波器系数为所述第一运动矢量偏差的一次函数、二次函数或者高次函数。The method of claim 40, wherein the filter coefficient is a linear function, a quadratic function, or a higher-order function of the first motion vector deviation.
根据权利要求38所述的方法，其中，若所述滤波参数为滤波器相位，所述基于所述滤波模式，利用所述第一运动矢量偏差确定所述滤波参数，包括：The method according to claim 38, wherein, if the filtering parameter is a filter phase, the determining the filtering parameter by using the first motion vector deviation based on the filtering mode comprises:

确定偏差与相位的映射表；Determine the mapping table of deviation and phase;

根据所述偏差与相位的映射表和所述第一运动矢量偏差，按照所述滤波模式确定所述像素点对应的所述滤波器相位。The filter phase corresponding to the pixel point is determined according to the filtering mode according to the mapping table of the deviation and the phase and the first motion vector deviation.
根据权利要求43所述的方法，其中，所述根据所述偏差与相位的映射表和所述第一运动矢量偏差，按照所述滤波模式确定所述像素点对应的所述滤波器相位，包括：The method according to claim 43, wherein determining the filter phase corresponding to the pixel point according to the filtering mode according to the mapping table of the deviation and the phase and the first motion vector deviation, comprising: :

若所述滤波模式为所述水平滤波，则按照所述水平偏差查询所述偏差与相位的映射表，获得所述滤波器相位；If the filtering mode is the horizontal filtering, query the mapping table of the deviation and the phase according to the horizontal deviation to obtain the filter phase;

若所述滤波模式为所述垂直滤波，则按照所述垂直偏差查询所述偏差与相位的映射表，获得所述滤波器相位；If the filtering mode is the vertical filtering, query the mapping table of the deviation and the phase according to the vertical deviation to obtain the filter phase;

若所述滤波模式为所述双向分离滤波，则按照所述水平偏差和所述垂直偏差，分别查询所述偏差与相位的映射表，获得所述滤波器相位。If the filtering mode is the bidirectional separation filtering, according to the horizontal deviation and the vertical deviation, the mapping table of the deviation and the phase is respectively inquired to obtain the filter phase.
根据权利要求44所述的方法，其中，所述方法还包括：The method of claim 44, wherein the method further comprises:

确定所述第一运动矢量偏差的精度参数，并基于所述精度参数确定缩小参数；determining a precision parameter of the first motion vector deviation, and determining a reduction parameter based on the precision parameter;

在利用所述滤波器相位进行滤波处理之后，按照所述缩小参数进行缩小处理和/或右移处理。After the filtering process is performed using the filter phase, the reduction process and/or the right shift process is performed according to the reduction parameter.
根据权利要求39或43所述的方法，其中，所述基于所述滤波参数和所述第一预测值，利用可分离滤波器进行滤波处理，获得所述像素点的预测值，包括：The method according to claim 39 or 43, wherein, based on the filtering parameter and the first predicted value, performing filtering processing with a separable filter to obtain the predicted value of the pixel point comprises:

若所述滤波模式为所述双向分离滤波，则基于所述滤波参数，使用所述可分离滤波器对所述第一预测值在水平方向进行滤波处理，获得水平滤波结果；If the filtering mode is the bidirectional separation filtering, then based on the filtering parameters, use the separable filter to perform filtering processing on the first predicted value in the horizontal direction to obtain a horizontal filtering result;

基于所述滤波参数，对所述水平滤波结果在垂直方向进行滤波处理，获得所述像素点的预测值。Based on the filtering parameters, filtering processing is performed on the horizontal filtering result in the vertical direction to obtain the predicted value of the pixel point.
根据权利要求39或43所述的方法，其中，所述基于所述滤波参数和所述第一预测值，利用可分离滤波器进行滤波处理，获得所述像素点的预测值，包括：The method according to claim 39 or 43, wherein, based on the filtering parameter and the first predicted value, performing filtering processing with a separable filter to obtain the predicted value of the pixel point comprises:

若所述滤波模式为所述双向分离滤波，则基于所述滤波参数，使用所述可分离滤波器对所述第一预测值在垂直方向进行滤波处理，获得垂直滤波结果；If the filtering mode is the two-way separation filtering, then based on the filtering parameters, use the separable filter to perform filtering processing on the first predicted value in the vertical direction to obtain a vertical filtering result;

基于所述滤波参数，对所述垂直滤波结果在水平方向进行滤波处理，获得所述像素点的预测值。Based on the filtering parameters, filtering processing is performed on the vertical filtering result in the horizontal direction to obtain the predicted value of the pixel point.
根据权利要求40所述的方法，其特征在于，所述方法还包括：The method of claim 40, wherein the method further comprises:

对于不同的子块，使用不同的比例参数和不同的预设计算规则确定所述滤波器系数。For different sub-blocks, different scale parameters and different preset calculation rules are used to determine the filter coefficients.
根据权利要求43所述的方法，其特征在于，所述方法还包括：The method of claim 43, wherein the method further comprises:

对于不同的子块，使用不同的偏差与相位的映射表确定所述滤波器系数。The filter coefficients are determined using different offset-to-phase mapping tables for different sub-blocks.
根据权利要求35所述的方法，其中，所述确定所述当前块的子块的第一预测值，包括：36. The method of claim 35, wherein said determining a first predicted value of a sub-block of the current block comprises:

确定所述当前块的子块的运动矢量；determining motion vectors for sub-blocks of the current block;

基于所述运动矢量确定所述子块的第一预测值。A first predictor for the subblock is determined based on the motion vector.
根据权利要求35所述的方法，其中，所述确定所述当前块的子块的第一预测值，包括：36. The method of claim 35, wherein said determining a first predicted value of a sub-block of the current block comprises:

确定所述子块与所述当前块之间的第二运动矢量偏差；determining a second motion vector offset between the sub-block and the current block;

基于所述第二运动矢量偏差，确定所述子块的第一预测值。Based on the second motion vector offset, a first predictor for the sub-block is determined.
根据权利要求35所述的方法，其中，所述方法还包括：The method of claim 35, wherein the method further comprises:

若所述像素点对应的相邻像素点不属于所述子块，则根据所述子块对所述相邻像素点进行更新处理，获得更新后像素点；If the adjacent pixel point corresponding to the pixel point does not belong to the sub-block, update the adjacent pixel point according to the sub-block to obtain the updated pixel point;

基于所述第一预测值和所述更新后像素点，确定所述子块的第二预测值，将所述第二预测值确定为所述子块的帧间预测值。Based on the first predicted value and the updated pixel point, a second predicted value of the sub-block is determined, and the second predicted value is determined as an inter-frame predicted value of the sub-block.
根据权利要求52所述的方法，其中，所述根据所述子块对所述相邻像素点进行更新处理，获得更新后像素点，包括：The method according to claim 52, wherein the updating the adjacent pixels according to the sub-blocks to obtain the updated pixels, comprising:

对所述子块进行扩展处理，获得扩展后子块；Expanding the sub-block to obtain the expanded sub-block;

在所述扩展后子块内确定所述相邻像素点对应的所述更新后像素点。The updated pixel points corresponding to the adjacent pixel points are determined in the extended sub-block.
根据权利要求53所述的方法，其中，所述对所述子块进行扩展处理，获得扩展后子块，包括：The method according to claim 53, wherein the performing expansion processing on the sub-blocks to obtain the expanded sub-blocks comprises:

利用所述子块的全部边界位置进行扩展处理，获得所述扩展后子块；或者，Perform expansion processing by using all boundary positions of the sub-block to obtain the expanded sub-block; or,

利用所述子块内的、所述相邻像素点对应的行和\或列的边界位置进行扩展处理，获得所述扩展后子块。Expansion processing is performed using the boundary positions of the rows and/or columns corresponding to the adjacent pixels in the sub-block to obtain the expanded sub-block.
根据权利要求53所述的方法，其中，所述根据所述子块对所述相邻像素点进行更新处理，获得更新后像素点，包括：The method according to claim 53, wherein the performing update processing on the adjacent pixels according to the sub-blocks to obtain the updated pixels, comprising:

利用所述子块内的、与所述相邻像素点相邻的像素点，替换所述相邻像素点，获得所述更新后像素点。The updated pixel is obtained by replacing the adjacent pixel with a pixel in the sub-block that is adjacent to the adjacent pixel.
根据权利要求35所述的方法，其中，所述方法还包括：The method of claim 35, wherein the method further comprises:

当所述预测模式参数指示使用帧间预测模式确定所述当前块的帧间预测值时，确定所述子块的扩展子块；determining an extended subblock of the subblock when the prediction mode parameter indicates that the inter prediction value of the current block is determined using an inter prediction mode;

基于所述运动矢量确定所述扩展子块的第三预测值，并确定所述像素点对应的所述相邻像素点；determining a third prediction value of the extended sub-block based on the motion vector, and determining the adjacent pixel points corresponding to the pixel points;

若所述相邻像素点不属于所述子块，基于所述第三预测值和所述相邻像素点，确定所述第二预测值，将所述第二预测值确定为所述帧间预测值。If the adjacent pixel point does not belong to the sub-block, the second prediction value is determined based on the third prediction value and the adjacent pixel point, and the second prediction value is determined as the inter frame Predictive value.
根据权利要求56所述的方法，其中，The method of claim 56, wherein,

对所述扩展子块使用n抽头的插值滤波器，获得所述第三预测值；其中，所述n为以下值中的任一者：6，5，4，3，2。The third predicted value is obtained using an n-tap interpolation filter on the extended sub-block; wherein the n is any of the following values: 6, 5, 4, 3, 2.
根据权利要求57所述的方法，其中，The method of claim 57, wherein,

对所述子块对应的参考像素进行扩展处理，获得扩展后参考像素；performing expansion processing on the reference pixels corresponding to the sub-blocks to obtain expanded reference pixels;

基于所述扩展后参考像素，使用所述子块对应的插值滤波器，获得所述第三预测值。Based on the expanded reference pixels, the third predicted value is obtained using an interpolation filter corresponding to the sub-block.
根据权利要求57所述的方法，其中，The method of claim 57, wherein,

在所述当前块中确定所述扩展子块对应的目标像素点；Determine the target pixel point corresponding to the extended sub-block in the current block;

根据所述目标像素点的整像素值确定所述第三预测值。The third predicted value is determined according to the integer pixel value of the target pixel.
根据权利要求36所述的方法，其中，所述方法还包括：The method of claim 36, wherein the method further comprises:

根据所述第一运动矢量偏差确定所述像素点对应的中心位置和第三运动矢量偏差；Determine the center position corresponding to the pixel point and the third motion vector deviation according to the first motion vector deviation;

基于所述第一预测值，按照所述中心位置和所述第三运动矢量偏差确定所述子块的第二预测值，将所述第二预测值确定为所述子块的帧间预测值。Based on the first predicted value, a second predicted value of the sub-block is determined according to the center position and the third motion vector deviation, and the second predicted value is determined as an inter-frame predicted value of the sub-block .
根据权利要求60所述的方法，其中，所述根据所述第一运动矢量偏差确定所述像素点对应的中心位置和第三运动矢量偏差，包括：The method according to claim 60, wherein the determining the center position corresponding to the pixel point and the third motion vector deviation according to the first motion vector deviation comprises:

确定所述水平偏差的第一绝对值和所述垂直偏差的第二绝对值；determining a first absolute value of the horizontal deviation and a second absolute value of the vertical deviation;

若所述第一绝对值和所述第二绝对值均小于预设偏差阈值，则将所述第一运动矢量偏差确定为所述第三运动矢量偏差，将所述像素点确定为所述中心位置；If both the first absolute value and the second absolute value are smaller than a preset deviation threshold, the first motion vector deviation is determined as the third motion vector deviation, and the pixel point is determined as the center Location;

若所述第一绝对值大于或者等于所述预设偏差阈值，且所述第二绝对值小于所述预设偏差阈值，则根据所述水平偏差确定第一调整方向，并根据所述第一调整方向，确定所述中心位置和所述第三运动矢量偏差；其中，所述第一调整方向包括左侧和右侧；If the first absolute value is greater than or equal to the preset deviation threshold, and the second absolute value is less than the preset deviation threshold, the first adjustment direction is determined according to the horizontal deviation, and the first adjustment direction is determined according to the first Adjusting the direction to determine the deviation of the center position and the third motion vector; wherein, the first adjustment direction includes left and right;

若所述第二绝对值大于或者等于所述预设偏差阈值，且所述第一绝对值小于所述预设偏差阈值，则根据所述垂直偏差确定第二调整方向，并根据所述第二调整方向，确定所述中心位置和所述第三运动矢量偏差；其中，所述第二调整方向包括上侧和下侧If the second absolute value is greater than or equal to the preset deviation threshold, and the first absolute value is smaller than the preset deviation threshold, a second adjustment direction is determined according to the vertical deviation, and the second adjustment direction is determined according to the second Adjusting the direction, determining the center position and the deviation of the third motion vector; wherein, the second adjustment direction includes an upper side and a lower side
根据权利要求61所述的方法，其中，所述预设偏差阈值为k个像素；其中，k大于0.5且小于或者等于1。The method of claim 61 , wherein the preset deviation threshold is k pixels; wherein k is greater than 0.5 and less than or equal to 1.
根据权利要求36或60所述的方法，其中，所述根据所述第一运动矢量偏差确定所述像素点对应的中心位置和第三运动矢量偏差之前，所述方法还包括：The method according to claim 36 or 60, wherein before the determining the center position corresponding to the pixel point and the third motion vector deviation according to the first motion vector deviation, the method further comprises:

按照预设偏差范围对所述第一运动矢量偏差进行限制处理；其中，所述预设偏差范围包括偏差下限值和偏差上限值。The first motion vector deviation is limited according to a preset deviation range; wherein, the preset deviation range includes a deviation lower limit value and a deviation upper limit value.
根据权利要求62所述的方法，其中，所述按照预设偏差范围对所述第一运动矢量偏差进行限制处理，包括：The method according to claim 62, wherein the limiting the deviation of the first motion vector according to a preset deviation range comprises:

若所述水平偏差和\或所述垂直偏差小于所述偏差下限值，则将所述水平偏差和\或所述垂直偏差设置为所述偏差下限值；If the horizontal deviation and\or the vertical deviation are less than the deviation lower limit value, then the horizontal deviation and\or the vertical deviation are set as the deviation lower limit value;

若所述水平偏差和\或所述垂直偏差大于所述偏差上限值，则将所述水平偏差和\或所述垂直偏差设置为所述偏差上限值。If the horizontal deviation and\or the vertical deviation are greater than the deviation upper limit value, set the horizontal deviation and\or the vertical deviation as the deviation upper limit value.
根据权利要求35所述的方法，其特征在于，所述方法还包括：The method of claim 35, wherein the method further comprises:

解析所述码流，确定帧间预测方向；Parse the code stream to determine the inter prediction direction;

若所述帧间预测方向为双向预测，则基于第一预测方向，确定所述子块的第四预测值和所述像素点的运动矢量与所述子块的运动矢量之间的第四运动矢量偏差；基于第二预测方向，确定所述子块的第五预测值和所述像素点的运动矢量与所述子块的运动矢量之间的第五运动矢量偏差；If the inter-frame prediction direction is bidirectional prediction, determining the fourth prediction value of the sub-block and the fourth motion between the motion vector of the pixel point and the motion vector of the sub-block based on the first prediction direction vector deviation; determining the fifth prediction value of the sub-block and the fifth motion vector deviation between the motion vector of the pixel point and the motion vector of the sub-block based on the second prediction direction;

基于所述第四运动矢量偏差，使用所述可分离滤波器对所述第四预测值进行二次预测，获得第六预测值；基于所述第五运动矢量偏差，使用所述可分离滤波器对所述第五预测值进行二次预测，获得第七预测值；Based on the fourth motion vector deviation, use the separable filter to perform secondary prediction on the fourth predicted value to obtain a sixth predicted value; based on the fifth motion vector deviation, use the separable filter performing secondary prediction on the fifth predicted value to obtain a seventh predicted value;

根据所述第六预测值和所述第七预测值，确定所述子块的帧间预测值。An inter-frame prediction value of the sub-block is determined according to the sixth prediction value and the seventh prediction value.
一种解码器，所述解码器包括解析部分，第一确定部分；A decoder, the decoder includes a parsing part and a first determining part;

所述解析部分，配置为解析码流，获取当前块的预测模式参数；The parsing part is configured to parse the code stream to obtain the prediction mode parameter of the current block;

所述第一确定部分，配置为当所述预测模式参数指示使用帧间预测模式确定所述当前块的帧间预测值时，确定所述当前块的子块的第一预测值；其中，所述子块包括一个或多个像素点；对于所述子块内的一像素点，确定所述像素点的运动矢量与所述子块的运动矢量之间的第一运动矢量偏差；根据所述第一运动矢量偏差确定滤波参数；基于所述滤波参数和所述第一预测值，利用可分离滤波器进行滤波处理，获得所述像素点的预测值；根据所述像素点的预测值，确定所述子块的第二预测值；其中，所述第二预测值用于确定所述子块的重建值。The first determination part is configured to determine the first prediction value of the sub-block of the current block when the prediction mode parameter indicates that the inter prediction value of the current block is determined using the inter prediction mode; wherein, the the sub-block includes one or more pixel points; for a pixel point in the sub-block, determine the first motion vector deviation between the motion vector of the pixel point and the motion vector of the sub-block; according to the The first motion vector deviation determines a filter parameter; based on the filter parameter and the first predicted value, a separable filter is used to perform filtering processing to obtain the predicted value of the pixel point; according to the predicted value of the pixel point, determine the second prediction value of the sub-block; wherein the second prediction value is used to determine the reconstruction value of the sub-block.
一种解码器，所述解码器包括第一处理器、存储有所述第一处理器可执行指令的第一存储器，当所述指令被执行时，所述第一处理器执行时实现如权利要求1-34任一项所述的方法。A decoder, the decoder comprising a first processor and a first memory storing instructions executable by the first processor, when the instructions are executed, the first processor implements the right The method of any one of claims 1-34.
一种编码器，所述编码器包括第二确定部分；an encoder comprising a second determination portion;

所述第二确定部分，配置为确定当前块的预测模式参数；当所述预测模式参数指示使用帧间预测模式确定所述当前块的帧间预测值时，确定所述当前块的子块的第一预测值；其中，所述子块包括一个或多个像素点；对于所述子块内的一像素点，确定所述像素点的运动矢量与所述子块的运动矢量之间的第一运动矢量偏差；根据所述第一运动矢量偏差确定滤波参数；基于所述滤波参数和所述第一预测值，利用可分离滤波器进行滤波处理，获得所述像素点的预测值；根据所述像素点的预测值，确定所述子块的第二预测值；其中，所述第二预测值用于确定所述子块的残差。The second determination part is configured to determine the prediction mode parameter of the current block; when the prediction mode parameter indicates that the inter prediction value of the current block is determined by using the inter prediction mode, determine the sub-block of the current block. The first predicted value; wherein, the sub-block includes one or more pixel points; for a pixel point in the sub-block, determine the first prediction value between the motion vector of the pixel point and the motion vector of the sub-block a motion vector deviation; determining a filtering parameter according to the first motion vector deviation; performing filtering processing with a separable filter based on the filtering parameter and the first predicted value, to obtain the predicted value of the pixel point; The predicted value of the pixel point is used to determine the second predicted value of the sub-block; wherein, the second predicted value is used to determine the residual of the sub-block.
一种编码器，所述编码器包括第二处理器、存储有所述第二处理器可执行指令的第二存储器，当所述指令被执行时，所述第二处理器执行时实现如权利要求35-65任一项所述的方法。An encoder, the encoder comprising a second processor and a second memory storing instructions executable by the second processor, when the instructions are executed, the second processor implements the right when executed The method of any of claims 35-65.
一种计算机存储介质，所述计算机存储介质存储有计算机程序，所述计算机程序被第一处理器执行时实现如权利要求1-34任一项所述的方法，或者，被第二处理器执行时实现如权利要求35-65任一项所述的方法。A computer storage medium, the computer storage medium stores a computer program, the computer program implements the method according to any one of claims 1-34 when executed by a first processor, or is executed by a second processor The method as claimed in any one of claims 35-65 is implemented.