WO2020181471A1

WO2020181471A1 - Intra-frame prediction method and apparatus, and computer storage medium

Info

Publication number: WO2020181471A1
Application number: PCT/CN2019/077719
Authority: WO
Inventors: 周益民; 程学理
Original assignee: Oppo广东移动通信有限公司
Priority date: 2019-03-11
Filing date: 2019-03-11
Publication date: 2020-09-17
Also published as: CN113508596A

Abstract

Disclosed in the embodiments of the present application are an intra-frame prediction method and apparatus and a computer storage medium, the intra-frame prediction method comprising: when receiving a residual matrix, looking up a prediction model corresponding to a codeword identifier, the codeword identifier being an identifier corresponding to a prediction model determined on the basis of the rate-distortion cost in the encoding stage; when the prediction model is a first prediction model, determining a reconstruction decoding block adjacent to the current encoding block in a preset direction; inputting the reconstruction decoding block into the first prediction model to obtain a predicted pixel value of the current decoding block; and, on the basis of the predicted pixel value and the residual matrix, obtaining a decoding pixel value of the current decoding block.

Description

帧内预测方法、装置及计算机存储介质Intra prediction method, device and computer storage medium

技术领域Technical field

本申请实施例涉及视频编码领域，尤其涉及一种帧内预测方法、装置及计算机存储介质。The embodiments of the present application relate to the field of video coding, and in particular, to an intra-frame prediction method, device, and computer storage medium.

背景技术Background technique

在视频编码中，帧内预测过程指利用像素块的纹理信息，通过数学模型构建像素块的预测像素值，之后对预测像素值与信源像素值进行计算，得到残差矩阵，将残差矩阵进行变换量化，熵编码等一系列步骤，最终转化为二进制比特流。在视频解码时，通过相同的方式进行像素预测，加上反量化、反变换所得到的残差信息，能够形成完整的图像。In video coding, the intra-frame prediction process refers to using the texture information of the pixel block to construct the predicted pixel value of the pixel block through a mathematical model, and then to calculate the predicted pixel value and the source pixel value to obtain the residual matrix, and the residual matrix A series of steps such as transform and quantization, entropy coding, etc., are finally converted into a binary bit stream. In video decoding, pixel prediction is performed in the same way, plus residual information obtained by inverse quantization and inverse transformation, to form a complete image.

在现有的视频编解码过程中，利用与当前像素块相连的边界像素块，通过数学模型构建当前像素块的预测像素值，然而该方法仅适用于图像中的平坦区域，对于纹理较复杂的区域，边界像素块无法准确的反应当前像素块的纹理信息，导致帧内预测结果不准确。In the existing video encoding and decoding process, the boundary pixel block connected to the current pixel block is used to construct the predicted pixel value of the current pixel block through a mathematical model. However, this method is only suitable for flat areas in the image, and is more complex for textures. In regions, boundary pixel blocks cannot accurately reflect the texture information of the current pixel block, resulting in inaccurate intra-frame prediction results.

发明内容Summary of the invention

本申请实施例提供一种帧内预测方法、装置及计算机存储介质，能够提高帧内预测的准确性。The embodiments of the present application provide an intra-frame prediction method, device, and computer storage medium, which can improve the accuracy of intra-frame prediction.

本申请实施例的技术方案是这样实现的：The technical solutions of the embodiments of the present application are implemented as follows:

本实施例提供一种帧内预测方法，所述方法包括：This embodiment provides an intra-frame prediction method, which includes:

当接收到残差矩阵时，查找码字标识对应的预测模型，所述码字标识为编码阶段根据率失真代价确定出的预测模型对应的标识；When the residual matrix is received, search for the prediction model corresponding to the codeword identifier, where the codeword identifier is the identifier corresponding to the prediction model determined according to the rate-distortion cost in the encoding stage;

当所述预测模型为第一预测模型时，确定与当前编码块在预设方向相邻的重构解码块；When the prediction model is the first prediction model, determining a reconstructed decoding block adjacent to the current coding block in a preset direction;

将所述重构解码块输入所述第一预测模型中，得到所述当前解码块的预测像素值；Input the reconstructed decoded block into the first prediction model to obtain the predicted pixel value of the current decoded block;

根据所述预测像素值和所述残差矩阵，得到所述当前解码块的解码像素值。According to the predicted pixel value and the residual matrix, the decoded pixel value of the current decoded block is obtained.

在上述方法中，所述查找码字标识对应的预测模型之后，所述方法还包括：In the above method, after the search for the prediction model corresponding to the codeword identifier, the method further includes:

当所述预测模型为第二预测模型时，获取与所述当前解码块在所述预设方向相邻的边界解码块，所述边界解码块为所述重构解码块中、与所述当前解码块相邻的解码块，所述第二预测模型的率失真代价小于所述第一预测模型的率失真代价；When the prediction model is the second prediction model, obtain a boundary decoding block adjacent to the current decoding block in the preset direction, and the boundary decoding block is the reconstructed decoding block that is the same as the current decoding block. Decoding blocks adjacent to the decoding block, where the rate distortion cost of the second prediction model is less than the rate distortion cost of the first prediction model;

根据所述边界解码块和所述第二预测模型，得到所述预测像素值。Obtain the predicted pixel value according to the boundary decoding block and the second prediction model.

在上述方法中，所述将所述重构解码块输入所述第一预测模型中，得到所述当前解码块的预测像素值，包括：In the above method, the inputting the reconstructed decoded block into the first prediction model to obtain the predicted pixel value of the current decoded block includes:

确定出所述重构解码块对应的解码矩阵，其中，所述解码矩阵为将所述重构解码块中损失的解码块填充0像素值得到的矩阵；Determining a decoding matrix corresponding to the reconstructed decoded block, where the decoded matrix is a matrix obtained by filling the decoded block lost in the reconstructed decoded block with 0 pixel values;

将所述解码矩阵输入所述第一预测模型中，得到所述预测像素值。The decoding matrix is input into the first prediction model to obtain the predicted pixel value.

在上述方法中，所述将所述解码矩阵输入所述第一预测模型中，得到所述预测像素值，包括：In the above method, the inputting the decoding matrix into the first prediction model to obtain the predicted pixel value includes:

利用所述第一预测模型中的第一组卷积层，确定所述重构解码块的图像纹理特征信息；Using the first group of convolutional layers in the first prediction model to determine the image texture feature information of the reconstructed decoded block;

利用所述第一预测模型中的第二组卷积层，确定所述重构解码块的图像纹理分布信息；Determine the image texture distribution information of the reconstructed decoded block by using the second group of convolutional layers in the first prediction model;

根据所述图像纹理特征信息和所述图像纹理分布信息，得到图像拼接信息；Obtaining image mosaic information according to the image texture feature information and the image texture distribution information;

利用所述第一预测模型中的第三组卷积层，确定所述图像拼接信息的特征信息；Using the third group of convolutional layers in the first prediction model to determine the feature information of the image mosaic information;

基于所述特征信息，确定出所述当前解码块的所述预测像素值，其中，所述预测像素值为所述特征信息进行卷积计算得到的像素值。Based on the characteristic information, the predicted pixel value of the current decoded block is determined, where the predicted pixel value is a pixel value obtained by performing convolution calculation on the characteristic information.

在上述方法中，所述查找码字标识对应的预测模型之前，所述方法还包括：In the above method, before the searching the prediction model corresponding to the codeword identifier, the method further includes:

接收二进制比特流；Receive binary bit stream;

根据所述二进制比特流，确定所述残差矩阵。According to the binary bit stream, the residual matrix is determined.

在上述方法中，所述预设方向为左侧、左上侧、上侧和右侧。In the above method, the preset directions are left side, upper left side, upper side and right side.

在上述方法中，所述第二预测模型包括平面Planar模式、直流系数DC模式和多种角度模式中的任一种。In the above method, the second prediction model includes any one of a plane Planar mode, a direct current coefficient DC mode, and multiple angle modes.

本实施例提供一种帧内预测装置，所述帧内预测装置包括：This embodiment provides an intra-frame prediction device, and the intra-frame prediction device includes:

查找部分，用于当接收到残差矩阵时，查找码字标识对应的预测模型，所述码字标识为编码阶段根据率失真代价确定出的预测模型对应的标识；The search part is used to search for the prediction model corresponding to the codeword identifier when the residual matrix is received, where the codeword identifier is the identifier corresponding to the prediction model determined according to the rate-distortion cost in the encoding stage;

确定部分，用于当所述预测模型为第一预测模型时，确定与当前编码块在预设方向相邻的重构解码块；The determining part is used to determine the reconstructed decoding block adjacent to the current coding block in a preset direction when the prediction model is the first prediction model;

计算部分，用于将所述重构解码块输入所述第一预测模型中，得到所述当前解码块的预测像素值；The calculation part is used to input the reconstructed decoded block into the first prediction model to obtain the predicted pixel value of the current decoded block;

帧内预测部分，用于根据所述预测像素值和所述残差矩阵，得到所述当前解码块的解码像素值。The intra-frame prediction part is used to obtain the decoded pixel value of the current decoded block according to the predicted pixel value and the residual matrix.

在上述装置中，所述查找部分，还用于当所述预测模型为第二预测模型时，获取与所述当前解码块在所述预设方向相邻的边界解码块，所述边界解码块为所述重构解码块中、与所述当前解码块相邻的解码块，所述第二预测模型的率失真代价小于所述第一预测模型的率失真代价；In the above device, the search part is further configured to obtain a boundary decoding block adjacent to the current decoding block in the preset direction when the prediction model is the second prediction model, and the boundary decoding block For a decoding block adjacent to the current decoding block in the reconstructed decoding block, the rate distortion cost of the second prediction model is less than the rate distortion cost of the first prediction model;

所述计算部分，还用于根据所述边界解码块和所述第二预测模型，得到所述预测像素值。The calculation part is further configured to obtain the predicted pixel value according to the boundary decoding block and the second prediction model.

在上述装置中，所述确定部分，用于确定出所述重构解码块对应的解码矩阵，其中，所述解码矩阵为将所述重构解码块中损失的解码块填充0像素值得到的矩阵；In the above device, the determining part is configured to determine the decoding matrix corresponding to the reconstructed decoded block, wherein the decoded matrix is obtained by filling the decoded block lost in the reconstructed decoded block with a pixel value of 0 matrix;

所述计算部分，还用于将所述解码矩阵输入所述第一预测模型中，得到所述预测像素值。The calculation part is also used to input the decoding matrix into the first prediction model to obtain the predicted pixel value.

在上述装置中，所述确定部分，还用于利用所述第一预测模型中的第一组卷积层，提取所述重构解码块的图像纹理特征信息；利用所述第一预测模型中的第二组卷积层，确定所述重构解码块的图像纹理分布信息；根据所述图像纹理特征信息和所述图像纹理分布信息，得到图像拼接信息；利用所述第一预测模型中的第三组卷积层，确定所述图像拼接信息的特征信息；基于所述特征信息，确定出所述当前解码块的所述预测像素值，其中，所述预测像素值为所述特征信息进行卷积计算得到的像素值。In the above device, the determining part is further configured to use the first set of convolutional layers in the first prediction model to extract the image texture feature information of the reconstructed decoded block; use the first prediction model Determine the image texture distribution information of the reconstructed and decoded block; obtain image mosaic information according to the image texture feature information and the image texture distribution information; use the image texture distribution information in the first prediction model The third group of convolutional layers determines the characteristic information of the image stitching information; based on the characteristic information, the predicted pixel value of the current decoding block is determined, wherein the predicted pixel value is performed on the characteristic information Pixel value calculated by convolution.

在上述装置中，所述装置还包括：接收部分；In the above device, the device further includes: a receiving part;

所述接收部分，用于接收二进制比特流；The receiving part is used to receive a binary bit stream;

所述确定部分，还用于根据所述二进制比特流，确定所述残差矩阵。The determining part is further configured to determine the residual matrix according to the binary bit stream.

在上述装置中，所述预设方向为左侧、左上侧、上侧和右侧。In the above device, the preset directions are left side, upper left side, upper side and right side.

在上述装置中，所述第二预测模型包括Planar模式、直流系数DC模式和多种角度模式中的任一种。In the above device, the second prediction model includes any one of Planar mode, direct current coefficient DC mode and multiple angle modes.

本实施例提供一种帧内预测装置，所述帧内预测装置包括处理器、存储有所述处理器可执行指令的存储器、通信接口，和用于连接所述处理器、所述存储器以及所述通信接口的总线，当所述指令被执行时，所述处理器执行时实现如上述任一项所述的帧内预测方法。This embodiment provides an intra-frame prediction device. The intra-frame prediction device includes a processor, a memory storing executable instructions of the processor, a communication interface, and a communication interface for connecting the processor, the memory, and the processor. For the bus of the communication interface, when the instruction is executed, the processor implements the intra-frame prediction method as described in any one of the above.

本实施例提供一种计算机可读存储介质，其上存储有程序，应用于帧内预测装置中，其中，所述程序被处理器执行时实现如上述任一项所述的帧内预测方法。This embodiment provides a computer-readable storage medium with a program stored thereon and applied to an intra-frame prediction apparatus, wherein the program is executed by a processor to implement the intra-frame prediction method as described in any one of the above.

本申请实施例提供了一种帧内预测方法、装置及计算机存储介质，该帧内预测方法可以包括：当接收到残差矩阵时，查找码字标识对应的预测模型，码字标识为编码阶段根据率失真代价确定出的预测模型对应的标识；当预测模型为第一预测模型时，确定与当前编码块在预设方向相邻的重构解码块；将重构解码块输入第一预测模型中，得到当前解码块的预测像素值；根据预测像素值和残差矩阵，得到当前解码块的解码像素值。由此可见，在本申请的实施例中，帧内预测装置将与当前解码块在预设方向相邻的重构解码块输入第一预测模型中，得到当前解码块的预测像素值，由于重构解码块为完整像素块，具备准确的纹理信息，因此将重构解码块输入第一预测模型中，得到当前解码块的预测像素值，能够准确的对纹理信息进行描述，进而提高了帧内预测的准确性。The embodiments of the present application provide an intra-frame prediction method, device, and computer storage medium. The intra-frame prediction method may include: when a residual matrix is received, searching for a prediction model corresponding to a codeword identifier, and the codeword identifier is an encoding stage The identifier corresponding to the prediction model determined according to the rate-distortion cost; when the prediction model is the first prediction model, determine the reconstructed decoding block adjacent to the current coding block in the preset direction; input the reconstructed decoded block into the first prediction model , Get the predicted pixel value of the current decoded block; get the decoded pixel value of the current decoded block according to the predicted pixel value and the residual matrix. It can be seen that, in the embodiment of the present application, the intra prediction device inputs the reconstructed decoded block adjacent to the current decoded block in the preset direction into the first prediction model to obtain the predicted pixel value of the current decoded block. The constructed decoded block is a complete pixel block with accurate texture information. Therefore, the reconstructed decoded block is input into the first prediction model to obtain the predicted pixel value of the current decoded block, which can accurately describe the texture information, thereby improving the intraframe The accuracy of the forecast.

附图说明Description of the drawings

图1为现有技术的一种基于边界像素值的帧内预测模板；FIG. 1 is an intra-frame prediction template based on boundary pixel values in the prior art;

图2为本申请实施例提供的一种示例性的基于相邻重构像素块的帧内预测模板；2 is an exemplary intra prediction template based on adjacent reconstructed pixel blocks provided by an embodiment of the application;

图3为本申请实施例提供的一种帧内预测方法的流程图；FIG. 3 is a flowchart of an intra-frame prediction method provided by an embodiment of the application;

图4为本申请实施例提供的一种示例性的解码矩阵的示意图；FIG. 4 is a schematic diagram of an exemplary decoding matrix provided by an embodiment of the application;

图5为本申请实施例提供的一种示例性的第一预测模型的模型示意图；FIG. 5 is a schematic diagram of an exemplary first prediction model provided by an embodiment of the application;

图6为本申请实施例提供的一种帧内预测装置的结构示意图一；FIG. 6 is a schematic structural diagram 1 of an intra-frame prediction apparatus provided by an embodiment of this application;

图7为本申请实施例提供的一种帧内预测装置的结构示意图二。FIG. 7 is a second structural diagram of an intra-frame prediction apparatus provided by an embodiment of this application.

具体实施方式detailed description

下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述。可以理解的是，此处所描述的具体实施例仅仅用于解释相关申请，而非对该申请的限定。另外还需要说明的是，为了便于描述，附图中仅示出了与有关申请相关的部分。The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It is understandable that the specific embodiments described here are only used to explain the related application, but not to limit the application. In addition, it should be noted that, for ease of description, only the parts related to the relevant application are shown in the drawings.

常见的帧内预测技术包括Planar(在H.264标准中为Plane)模式、直流系数(DC，Direct Coefficient)模式以及多种角度模式。这些模式使用了相同的预测模板，如图1所示。在图1中，R为重构块的像素值，P为当前块的预测像素值，N为像素块的宽度。H.264标准模板仅仅使用了当前块的左侧、上侧和上右侧的边界像素块，其中，当前块左侧的边界像素块为R _0,1、R _0,2、…、R _0,N，当前块上侧的边界像素块为R _1,0、R _1,1、…、R _1,N，当前块上右侧的边界像素块为R _N+1,0、R _N+2,0、…、R _2N,0，而H.265标准在H.264的基础上新增了当前块左下侧的边界像素块，即R _0,N+1、R _0,N+2、…、R _0,2N。这2个模板均使用了当前块周围的重构块的边界像素块，并通过进行简单的线性函数计算，来获得当前块的预测像素值。由上可知，线性函数预测适用于平坦区域，但对于纹理稍复杂的区域，线性函数预测的预测准确性明显减弱，从而导致需要传输的残差信息增多，编码性能下降。对于具有复杂纹理的像素块，其边界像素无法准确反映整个像素块对的纹理信息。因此，本技术方案采用完整块预测当前编码块。如图2所示，将当前块的左上，上，右上，左方的完整重构块作为预测模板，输入预测模型中，输入当前块的预测像素值。由于完整重构块具备准确的纹理信息，通过对纹理信息的合理描述，重新构建预测模型，从而改善帧内预测性能。 Common intra-frame prediction techniques include Planar (Plane in the H.264 standard) mode, Direct Coefficient (DC, Direct Coefficient) mode, and multiple angle modes. These models use the same prediction template, as shown in Figure 1. In Figure 1, R is the pixel value of the reconstructed block, P is the predicted pixel value of the current block, and N is the width of the pixel block. The H.264 standard template only uses the boundary pixel blocks on the left, upper and right sides of the current block, where the boundary pixel blocks on the left of the current block are R _0,1 , R _0,2 ,..., R _{0 ,N} , the boundary pixel block on the upper side of the current block is R _1,0 , R _1,1 , ..., R _1,N , the boundary pixel block on the right side of the current block is R _N+1,0 , R _{N+2 ,0} ,...,R _2N,0 , and the H.265 standard adds a boundary pixel block on the lower left side of the current block based on H.264, namely R _0,N+1 , R _0,N+2 ,... , R _0,2N . Both of these two templates use the boundary pixel blocks of the reconstructed block around the current block, and obtain the predicted pixel value of the current block by performing simple linear function calculations. It can be seen from the above that linear function prediction is suitable for flat areas, but for areas with slightly more complex textures, the prediction accuracy of linear function prediction is significantly weakened, resulting in increased residual information that needs to be transmitted and reduced coding performance. For a pixel block with a complex texture, its boundary pixels cannot accurately reflect the texture information of the entire pixel block pair. Therefore, this technical solution adopts a complete block to predict the current coding block. As shown in Fig. 2, the complete reconstructed block on the upper left, upper right, upper right, and left of the current block is used as a prediction template, input into the prediction model, and input the predicted pixel value of the current block. Since the complete reconstruction block has accurate texture information, the prediction model is reconstructed through a reasonable description of the texture information, thereby improving the intra prediction performance.

下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述。The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

在一实施例中，本申请实施例提供了一种帧内预测方法，图3本申请实施例提出的一种帧内预测方法的实现流程示意图，该方法可以包括：In an embodiment, an embodiment of the present application provides an intra-frame prediction method. FIG. 3 is a schematic diagram of an implementation process of an intra-frame prediction method proposed in an embodiment of the present application. The method may include:

S101、当接收到残差矩阵时，查找码字标识对应的预测模型，码字标识为编码阶段根据率失真代价确定出的预测模型对应的标识。S101: When the residual matrix is received, search for the prediction model corresponding to the codeword identifier, where the codeword identifier is the identifier corresponding to the prediction model determined according to the rate-distortion cost in the encoding stage.

本申请实施例提供的一种帧内预测方法适用于利用与当前解码块在预设方向相邻的重构解码块对当前解码块进行帧内预测的场景下。An intra-frame prediction method provided by an embodiment of the present application is applicable to a scenario where a reconstructed decoded block adjacent to the current decoded block in a preset direction is used to perform intra-frame prediction on the current decoded block.

本申请实施例中，帧内预测装置接收到二进制比特流，之后根据二进制比特流，确定残差矩阵，具体的，帧内预测装置根据二进制比特流，确定残差矩阵的过程为：帧内预测装置将二进制比特流进行反量化和反变换，得到当前编码块的残差矩阵。In the embodiment of the present application, the intra prediction device receives the binary bit stream, and then determines the residual matrix according to the binary bit stream. Specifically, the intra prediction device determines the residual matrix according to the binary bit stream as follows: intra prediction The device performs inverse quantization and inverse transformation on the binary bit stream to obtain the residual matrix of the current coding block.

本申请实施例中，帧内预测装置在接收到残差矩阵时，帧内预测装置查找码字标识对应的预测模型，其中，码字标识为编码阶段根据率失真代价确定出的预测模型对应的标识。In the embodiment of the present application, when the intra-frame prediction device receives the residual matrix, the intra-frame prediction device searches for the prediction model corresponding to the codeword identifier, where the codeword identifier corresponds to the prediction model determined according to the rate-distortion cost in the encoding stage Logo.

本申请实施例中，在编码阶段，帧内预测装置遍历所有预测模型，对这些预测模型进行率失真代价计算，并从这些预测模型中确定出率失真代价最小的预测模型，帧内预测装置为率失真代价最小的预测模型设置码字标识，以在解码阶段根据码字标识确定出编码阶段使用的预测模型，进而保证了编解码的一致性。In the embodiments of the present application, in the encoding stage, the intra-frame prediction device traverses all prediction models, performs rate-distortion cost calculation on these prediction models, and determines the prediction model with the least rate-distortion cost from these prediction models. The intra-frame prediction device is The prediction model with the least rate-distortion cost sets a codeword identifier to determine the prediction model used in the encoding stage according to the codeword identifier in the decoding stage, thereby ensuring the consistency of the encoding and decoding.

S102、当预测模型为第一预测模型时，确定与当前编码块在预设方向相邻的重构解码块。S102: When the prediction model is the first prediction model, determine a reconstructed decoding block adjacent to the current coding block in a preset direction.

当帧内预测装置查找到码字标识对应的预测模型之后，帧内预测装置在判断出预测模型为第一预测模型时，确定与当前编码块在预设方向相邻的重构解码块。After the intra-frame prediction device finds the prediction model corresponding to the codeword identifier, the intra-frame prediction device determines the reconstructed decoding block adjacent to the current coding block in the preset direction when determining that the prediction model is the first prediction model.

本申请实施例中，预设方向包括左侧、左上侧、上侧和右侧，即帧内预测装置获取当前解码块的左侧重构解码块、左上侧重构解码块、上侧重构解码块和右侧重构解码块。In the embodiment of the present application, the preset directions include left side, upper left side, upper side, and right side, that is, the intra prediction device obtains the left side reconstructed decoded block, the upper left side reconstructed decoded block, and the upper side reconstructed of the current decoded block. The decoded block and the right reconstructed decoded block.

需要说明的是，由于选取与当前像素块在预设方向相邻的重构像素块时，将重构像素块输入第一预测模型中、计算出的当前像素块的预测像素值与当前像素块的信源像素值最接近，故，帧内预测装置采用第一预测模型进行帧内预测的率失真代价最小，因此，在解码阶段，帧内预测装置确定出预测模型为第一预测模型时，确定与当前编码块在预设方向相邻的重构解码块。It should be noted that when the reconstructed pixel block adjacent to the current pixel block in the preset direction is selected, the reconstructed pixel block is input into the first prediction model, and the calculated predicted pixel value of the current pixel block is compared with the current pixel block. The source pixel value of is the closest. Therefore, the intra prediction device adopts the first prediction model to perform intra prediction with the least rate distortion cost. Therefore, in the decoding stage, when the intra prediction device determines that the prediction model is the first prediction model, Determine the reconstructed decoding block adjacent to the current coding block in the preset direction.

进一步地，当预测模型为第二预测模型时，帧内预测装置获取与当前解码块在预设方向相邻的边界解码块，其中边界解码块为重构解码块中、与当前解码块相邻的解码块，第二预测模型的率失真代价小于第一预测模型的率失真代价；并根据边界解码块和第二预测模型，得到预测像素值，具体的，帧内预测装置根据边界解码块和第二预测模型，得到预测像素值的过程为：帧内预测装置将边界解码块输入第二预测模型中，得到预测像素值。Further, when the prediction model is the second prediction model, the intra prediction apparatus obtains a boundary decoding block adjacent to the current decoding block in a preset direction, wherein the boundary decoding block is the reconstructed decoding block and is adjacent to the current decoding block The rate-distortion cost of the second prediction model is less than the rate-distortion cost of the first prediction model; and the predicted pixel value is obtained according to the boundary decoding block and the second prediction model. Specifically, the intra prediction device decodes the block and the second prediction model according to the boundary In the second prediction model, the process of obtaining the predicted pixel value is: the intra prediction device inputs the boundary decoding block into the second prediction model to obtain the predicted pixel value.

可选的，第二预测模型包括Planar模式、DC模式和多种角度模式中的任一种。Optionally, the second prediction model includes any one of Planar mode, DC mode, and multiple angle modes.

S103、将重构解码块输入第一预测模型中，得到当前解码块的预测像素值。S103. Input the reconstructed decoded block into the first prediction model to obtain the predicted pixel value of the current decoded block.

当帧内预测装置确定与当前编码块在预设方向相邻的重构解码块之后，帧内预测装置将重构解码块输入第一预测模型中，得到当前解码块的预测像素值。After the intra prediction device determines the reconstructed decoded block adjacent to the current coding block in the preset direction, the intra prediction device inputs the reconstructed decoded block into the first prediction model to obtain the predicted pixel value of the current decoded block.

本申请实施例中，当帧内预测装置判断出存在重构解码块时，帧内预测装置确定出重构解码块对应的解码矩阵，之后，帧内预测装置将解码矩阵输入第一预测模型中，得到当前解码块的预测像素值。In the embodiment of the present application, when the intra prediction device determines that there is a reconstructed decoding block, the intra prediction device determines the decoding matrix corresponding to the reconstructed decoded block, and then the intra prediction device inputs the decoding matrix into the first prediction model , Get the predicted pixel value of the current decoded block.

具体的，帧内预测装置确定出重构解码块对应的解码矩阵的过程为：帧内预测装置将重构解码块中损失的解码块填充0像素值，得到解码矩阵。Specifically, the process of the intra prediction device determining the decoding matrix corresponding to the reconstructed decoded block is: the intra prediction device fills the decoded block lost in the reconstructed decoded block with 0 pixel values to obtain the decoding matrix.

本申请实施例中，由于4个重构解码块直接拼接形成的解码矩阵不便于直接作为预设预测模型的输入，需要对其进行扩展。如图4所示，解码块是边长为N的矩形块，第一行的3个解码块和第二行第一个解码块为重构解码块，第二行的第二个解码块和第三个解码块为缺失的解码块，将2个缺失的解码块以0像素值进行填充，再将完整的大小为2N*3N的块作为预设预测模型的输入。In the embodiment of the present application, since the decoding matrix formed by direct splicing of 4 reconstructed decoding blocks is not convenient to be directly used as the input of the preset prediction model, it needs to be extended. As shown in Figure 4, the decoded block is a rectangular block with side length N. The 3 decoded blocks in the first row and the first decoded block in the second row are reconstructed decoded blocks, and the second decoded block in the second row is The third decoded block is the missing decoded block. The two missing decoded blocks are filled with 0 pixel values, and then the complete block with a size of 2N*3N is used as the input of the preset prediction model.

本申请实施例中，第一预测模型包括不同的深度神经网络和普通数学模型，具体的根据实际情况进行选择，本申请实施例不做具体的限定。In the embodiments of the present application, the first prediction model includes different deep neural networks and ordinary mathematical models, which are specifically selected according to actual conditions, and the embodiments of the present application do not make specific limitations.

本申请实施例中，当第一预测模型为深度神经网络时，帧内预测装置将解码矩阵输入预设预测模型中，得到当前解码块的预测像素值，包括：帧内预测装置利用预设预测模型中的第一组卷积层，确定重构解码块的图像纹理特征信息；帧内预测装置利用预设预测模型中的第二组卷积层，确定重构解码块的图像纹理分布信息；之后，帧内预测装置根据图像纹理特征信息和图像纹理分布信息，得到图像拼接信息；并利用预设预测模型中的第三组卷积层，确定图像拼接信息的特征信息；最后，帧内预测装置基于特征信息，确定出当前解码块的预测像素值。In the embodiment of the present application, when the first prediction model is a deep neural network, the intra prediction device inputs the decoding matrix into the preset prediction model to obtain the predicted pixel value of the current decoding block, including: the intra prediction device uses the preset prediction The first set of convolutional layers in the model determines the image texture feature information of the reconstructed decoded block; the intra prediction device uses the second set of convolutional layers in the preset prediction model to determine the image texture distribution information of the reconstructed decoded block; After that, the intra prediction device obtains the image mosaic information according to the image texture feature information and the image texture distribution information; and uses the third group of convolutional layers in the preset prediction model to determine the feature information of the image mosaic information; finally, intra prediction Based on the characteristic information, the device determines the predicted pixel value of the current decoded block.

具体的，帧内预测装置根据图像纹理特征信息和图像纹理分布信息，得到图像拼接信息的具体过程为：帧内预测装置将图像纹理特征信息和图像纹理分布信息进行拼接，得到图像拼接信息。Specifically, the specific process for the intra prediction device to obtain image splicing information according to the image texture feature information and the image texture distribution information is: the intra prediction device splices the image texture feature information and the image texture distribution information to obtain the image splicing information.

具体的，帧内预测装置基于特征信息，确定出当前解码块的预测像素值的过程为：帧内预测装置对特征信息进行卷积计算，得到当前解码块的预测像素值。Specifically, the intra prediction device determines the predicted pixel value of the current decoded block based on the feature information: the intra prediction device performs convolution calculation on the feature information to obtain the predicted pixel value of the current decoded block.

本申请实施例中，如图5所示，每一个立方体块代表深度神经网络的一个卷积层，其中，S1-S5层采用了较小的卷积核(3*3)，为第一组卷积层，主要用于精细化的提取图像纹理特征信息；O1-O4层采用了较大的卷积核(5*5)，为第二组卷积层，主要用于粗略反应图像纹理分布信息，之后在利用拼接函数F1，将图像纹理特征信息和图像纹理分布信息进行拼接，得到图像拼接信息，并利用F2-F7(第三组卷积层)提取图像拼接信息的特征信息，最后，解码器对特征信息运用卷积深度为1的卷积运算，得到当前解码块的预测像素值。In the embodiment of this application, as shown in FIG. 5, each cube block represents a convolutional layer of the deep neural network. Among them, the S1-S5 layer uses a smaller convolution kernel (3*3), which is the first group Convolutional layer, mainly used to extract image texture feature information in a refined manner; O1-O4 layer uses a larger convolution kernel (5*5), which is the second group of convolutional layers, mainly used to roughly reflect the image texture distribution Then use the stitching function F1 to stitch the image texture feature information and the image texture distribution information to obtain the image stitching information, and use F2-F7 (the third group of convolutional layers) to extract the feature information of the image stitching information. Finally, The decoder applies a convolution operation with a convolution depth of 1 to the feature information to obtain the predicted pixel value of the current decoded block.

具体的深度神经网络的网络配置如表1所示The network configuration of the specific deep neural network is shown in Table 1.

表1深度神经网络的网络配置详情Table 1 Network configuration details of deep neural network

层名Layer name	说明Description
InputInput	原始像素矩阵2Nx3N(归一化)Original pixel matrix 2Nx3N (normalized)
O1O1	反卷积(32x5x5,stride＝2)+Leaky ReLU(alpha＝0.1)Deconvolution (32x5x5, stride＝2)+Leaky ReLU(alpha＝0.1)
O2～O4O2～O4	卷积(64x5x5,stride＝1)+Leaky ReLU(alpha＝0.1)Convolution (64x5x5, stride=1)+Leaky ReLU(alpha=0.1)
S1S1	卷积(32x3x3,stride＝1)+Leaky ReLU(alpha＝0.1)Convolution (32x3x3, stride=1)+Leaky ReLU(alpha=0.1)
S2S2	卷积(64x3x3,stride＝1)+Leaky ReLU(alpha＝0.1)Convolution (64x3x3, stride=1)+Leaky ReLU(alpha=0.1)
S3S3	卷积(128x3x3,stride＝1)+Leaky ReLU(alpha＝0.1)Convolution (128x3x3, stride=1)+Leaky ReLU(alpha=0.1)
S4S4	反卷积(128x3x3,stride＝2)+Leaky ReLU(alpha＝0.1)Deconvolution (128x3x3, stride＝2)+Leaky ReLU(alpha＝0.1)
S5S5	卷积(64x3x3,stride＝1)+Leaky ReLU(alpha＝0.1)Convolution (64x3x3, stride=1)+Leaky ReLU(alpha=0.1)
F1F1	concatenate[S5,O4]concatenate[S5,O4]
F2～F6F2～F6	卷积(64x3x3,stride＝1)+ReLUConvolution (64x3x3, stride=1)+ReLU
F7F7	卷积(32x3x3,stride＝6,4)+ReLUConvolution (32x3x3, stride=6,4)+ReLU
输出Output	卷积(1x3x3,stride＝1)+ReLU(反归一化)Convolution (1x3x3, stride=1) + ReLU (Denormalization)

其中，stride表示卷积的跨度，Leaky ReLU为一种激活函数，alpha为激活函数的参数，concatenate为拼接函数。Among them, stride represents the span of convolution, Leaky ReLU is an activation function, alpha is the parameter of the activation function, and concatenate is the splicing function.

本实施例中，深度神经网络的网络参数按照Glorot权重初始化方式进行随机初始化，对深度神经网络进行训练时，梯度下降方式选择Adam算法，训练的学习率初始值为5x10 ^-6。训练时以64组图像数据作为一个批次输入深度神经网络中，并随机打乱顺序以保证网络的泛化能力，同时定义每1000个批次为一个迭代，在一个迭代结束后进行损失计算，其中，损失函数为输出像素块与信源像素块的均方差。若10个迭代结束后的验证误差没有得到减小，则降低学习率至当前值的0.3倍。 In this embodiment, the network parameters of the deep neural network are randomly initialized according to the Glorot weight initialization method. When the deep neural network is trained, the Adam algorithm is selected as the gradient descent method, and the initial value of the training learning rate is 5×10 ^-6 . During training, 64 sets of image data are input into the deep neural network as a batch, and the order is randomly shuffled to ensure the generalization ability of the network. At the same time, every 1000 batches are defined as an iteration, and the loss calculation is performed after the end of an iteration , Where the loss function is the mean square error between the output pixel block and the source pixel block. If the verification error after 10 iterations is not reduced, the learning rate is reduced to 0.3 times the current value.

需要说明的是，由于帧内预测装置在进行解码块划分大小通常有8x8、16x16等尺寸，不同尺寸的解码块存在明显的纹理差异，亮度解码块和色度解码块也具有较大的区别。因此，本技术方案为不同尺寸的亮度解码块与色度解码块训练不同的网络参数，以保证得到更为精确的预测像素值。It should be noted that, since the decoding block size of the intra-frame prediction device is usually 8x8, 16x16, etc., there are obvious texture differences between the decoding blocks of different sizes, and the luminance decoding block and the chrominance decoding block are also quite different. Therefore, this technical solution trains different network parameters for luma decoding blocks and chroma decoding blocks of different sizes to ensure that more accurate predicted pixel values are obtained.

在实际应用中，将深度神经网络同时应用于编码器和解码器，使得编码器和解码器利用同样的预测模式获取当前块的预测像素值，进而保证编解码的一致性。In practical applications, the deep neural network is applied to the encoder and the decoder at the same time, so that the encoder and the decoder use the same prediction mode to obtain the predicted pixel value of the current block, thereby ensuring the consistency of the codec.

S104、根据预测像素值和残差矩阵，得到当前解码块的解码像素值。S104: Obtain the decoded pixel value of the current decoded block according to the predicted pixel value and the residual matrix.

当帧内预测装置将重构解码块输入第一预测模型中，得到当前解码块的预测像素值之后，帧内预测装置根据预测像素值和残差矩阵，得到当前解码块的解码像素值。After the intra prediction device inputs the reconstructed decoded block into the first prediction model to obtain the predicted pixel value of the current decoded block, the intra prediction device obtains the decoded pixel value of the current decoded block according to the predicted pixel value and the residual matrix.

本申请实施例中，帧内预测装置根据预测像素值和残差矩阵，得到当前解码块的解码像素值，并利用解码像素值重构当前解码块对应的图像信息，以完成对当前解码块的帧内预测过程。In the embodiment of this application, the intra prediction device obtains the decoded pixel value of the current decoded block according to the predicted pixel value and the residual matrix, and uses the decoded pixel value to reconstruct the image information corresponding to the current decoded block to complete the current decoded block Intra prediction process.

需要说明的是，本申请以帧内预测装置在解码时进行帧内预测的过程进行详述，帧内预测装置在编码时进行帧内预测的过程，与帧内预测装置在解码时进行帧内预测的过程相似，具体过程为：编码器遍历全部预测模型，并从全部预测模型中确定出率失真代价最小的预测模型，当预测模型为第一预测模型中，编码器和解码器为第一预测模型增加一个码字标识，以保证编解码的一致性，并将与当前编码块在预设方向相邻的重构编码块输入第一预测模型中，得到当前编码块的预测像素值，之后，利用预测像素值和信源像素值计算出残差矩阵，并将残差矩阵进行换量化，熵编码等一系列步骤，最终转化为二进制比特流。It should be noted that this application describes the process of intra-frame prediction performed by the intra-frame prediction device during decoding. The process of intra-frame prediction performed by the intra-frame prediction device during encoding is the same as that performed by the intra-frame prediction device during decoding. The prediction process is similar. The specific process is: the encoder traverses all prediction models and determines the prediction model with the least rate-distortion cost from all prediction models. When the prediction model is the first prediction model, the encoder and decoder are the first The prediction model adds a codeword identifier to ensure the consistency of coding and decoding, and input the reconstructed coding block adjacent to the current coding block in the preset direction into the first prediction model to obtain the predicted pixel value of the current coding block. , Use the predicted pixel value and the source pixel value to calculate the residual matrix, and perform a series of steps such as conversion quantization and entropy coding of the residual matrix, and finally convert it into a binary bit stream.

可以理解的是，帧内预测装置将与当前解码块在预设方向相邻的重构解码块输入第一预测模型中，得到当前解码块的预测像素值，由于重构解码块为完整像素块，具备准确的纹理信息，因此将重构解码块输入第一预测模型中，得到当前解码块的预测像素值，能够准确的对纹理信息进行描述，进而提高了帧内预测的准确性。It can be understood that the intra-frame prediction device inputs the reconstructed decoded block adjacent to the current decoded block in the preset direction into the first prediction model to obtain the predicted pixel value of the current decoded block, since the reconstructed decoded block is a complete pixel block , With accurate texture information, so the reconstructed decoded block is input into the first prediction model to obtain the predicted pixel value of the current decoded block, which can accurately describe the texture information, thereby improving the accuracy of intra prediction.

基于上述实施例，在本申请的又一实施例中，图6为本申请实施例提出的帧内预测装置的组成结构示意图一，如图6所示，本申请实施例提出的帧内预测装置1可以包括查找部分10、确定部分11、计算部分12和帧内预测部分13。Based on the above-mentioned embodiment, in another embodiment of the present application, FIG. 6 is a schematic diagram 1 of the composition structure of the intra-frame prediction apparatus proposed in this embodiment of the application. As shown in FIG. 6, the intra-frame prediction apparatus proposed in the embodiment of the present application 1 may include a search part 10, a determination part 11, a calculation part 12, and an intra prediction part 13.

所述查找部分10，用于当接收到残差矩阵时，查找码字标识对应的预测模型，所述码字标识为编码阶段根据率失真代价确定出的预测模型对应的标识；The searching part 10 is configured to search for a prediction model corresponding to a codeword identifier when the residual matrix is received, and the codeword identifier is an identifier corresponding to the prediction model determined according to the rate-distortion cost in the encoding stage;

所述确定部分11，用于当所述预测模型为第一预测模型时，确定与当前编码块在预设方向相邻的重构解码块；The determining part 11 is configured to determine a reconstructed decoding block adjacent to the current coding block in a preset direction when the prediction model is the first prediction model;

所述计算部分12，用于将所述重构解码块输入所述第一预测模型中，得到所述当前解码块的预测像素值；The calculation part 12 is configured to input the reconstructed decoded block into the first prediction model to obtain the predicted pixel value of the current decoded block;

所述帧内预测部分13，用于根据所述预测像素值和所述残差矩阵，得到所述当前解码块的解码像素值。The intra-frame prediction part 13 is configured to obtain the decoded pixel value of the current decoded block according to the predicted pixel value and the residual matrix.

进一步地，所述查找部分10，还用于当所述预测模型为第二预测模型时，获取与所述当前解码块在所述预设方向相邻的边界解码块，所述边界解码块为所述重构解码块中、与所述当前解码块相邻的解码块，所述第二预测模型的率失真代价小于所述第一预测模型的率失真代价；Further, the searching part 10 is further configured to obtain a boundary decoding block adjacent to the current decoding block in the preset direction when the prediction model is a second prediction model, and the boundary decoding block is In the reconstructed decoded block that is adjacent to the current decoded block, the rate distortion cost of the second prediction model is smaller than the rate distortion cost of the first prediction model;

所述计算部分12，还用于根据所述边界解码块输入所述第二预测模型，得到所述预测像素值。The calculation part 12 is further configured to input the second prediction model according to the boundary decoding block to obtain the predicted pixel value.

进一步地，所述确定部分11，用于确定出所述重构解码块对应的解码矩阵，其中，所述解码矩阵为将所述重构解码块中损失的解码块填充0像素值得到的矩阵；Further, the determining part 11 is configured to determine a decoding matrix corresponding to the reconstructed decoded block, wherein the decoded matrix is a matrix obtained by filling the lost decoded block in the reconstructed decoded block with 0 pixel values ；

所述计算部分12，还用于将所述解码矩阵输入所述第一预测模型中，得到所述预测像素值。The calculation part 12 is also used to input the decoding matrix into the first prediction model to obtain the predicted pixel value.

进一步地，所述确定部分11，还用于利用所述第一预测模型中的第一组卷积层，确定所述重构解码块的图像纹理特征信息；利用所述第一预测模型中的第二组卷积层，确定所述重构解码块的图像纹理分布信息；根据所述图像纹理特征信息和所述图像纹理分布信息，得到图像拼接信息；利用所述第一预测模型中的第三组卷积层，确定所述图像拼接信息的特征信息；基于所述特征信息，确定出所述当前解码块的所述预测像素值，其中，所述预测像素值为所述特征信息进行卷积计算得到的像素值。Further, the determining part 11 is further configured to use the first group of convolutional layers in the first prediction model to determine the image texture feature information of the reconstructed decoded block; use the first prediction model in the The second group of convolutional layers determines the image texture distribution information of the reconstructed decoded block; obtains image mosaic information according to the image texture feature information and the image texture distribution information; uses the first prediction model in the first prediction model Three groups of convolutional layers are used to determine the feature information of the image stitching information; based on the feature information, the predicted pixel value of the current decoded block is determined, wherein the predicted pixel value is used to roll the feature information Integrate the calculated pixel value.

进一步地，所述装置还包括：接收部分14；Further, the device further includes: a receiving part 14;

所述接收部分14，用于接收二进制比特流；The receiving part 14 is used to receive a binary bit stream;

所述确定部分11，还用于根据所述二进制比特流，确定所述残差矩阵。The determining part 11 is further configured to determine the residual matrix according to the binary bit stream.

进一步地，所述预设方向为左侧、左上侧、上侧和右侧。Further, the preset directions are left side, upper left side, upper side and right side.

进一步地，所述第二预测模型包括Planar模式、直流系数DC模式和多种角度模式中的任一种。Further, the second prediction model includes any one of Planar mode, direct current coefficient DC mode and multiple angle modes.

图7为本申请实施例提出的帧内预测装置的组成结构示意图二，如图7所示，本申请实施例提出的帧内预测装置1还可以包括处理器110、存储有处理器110可执行指令的存储器111、通信接口112，和用于连接处理器110、存储器111以及通信接口112的总线113。FIG. 7 is a second schematic diagram of the composition structure of the intra-frame prediction apparatus proposed in an embodiment of the application. As shown in FIG. The instruction memory 111, the communication interface 112, and the bus 113 for connecting the processor 110, the memory 111, and the communication interface 112.

在本申请的实施例中，上述处理器110可以为特定用途集成电路(Application Specific Integrated Circuit，ASIC)、数字信号处理器(Digital Signal Processor，DSP)、数字信号处理装置(Digital Signal Processing Device，DSPD)、可编程逻辑装置(ProgRAMmable Logic Device，PLD)、现场可编程门阵列(Field ProgRAMmable Gate Array，FPGA)、中央处理器(Central Processing Unit，CPU)、控制器、微控制器、微处理器中的至少一种。可以理解地，对于不同的设备，用于实现上述处理器功能的电子器件还可以为其它，本申请实施例不作具体限定。装置1还可以包括存储器111，该存储器111可以与处理器110连接，其中，存储器111用于存储可执行程序代码，该程序代码包括计算机操作指令，存储器111可能包含高速RAM存储器，也可能还包括非易失性存储器，例如，至少两个磁盘存储器。In the embodiment of the present application, the above-mentioned processor 110 may be an application specific integrated circuit (ASIC), a digital signal processor (Digital Signal Processor, DSP), a digital signal processing device (Digital Signal Processing Device, DSPD) ), programmable logic device (ProgRAMmable Logic Device, PLD), field programmable gate array (Field ProgRAMmable Gate Array, FPGA), central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor At least one of. It is understandable that, for different devices, the electronic devices used to implement the above-mentioned processor functions may also be other, which is not specifically limited in the embodiment of the present application. The device 1 may further include a memory 111, which may be connected to the processor 110, wherein the memory 111 is used to store executable program codes, the program codes include computer operation instructions, the memory 111 may include high-speed RAM memory, or may also include Non-volatile memory, for example, at least two disk memories.

在本申请的实施例中，总线113用于连接通信接口112、处理器110以及存储器111以及这些器件之间的相互通信。In the embodiment of the present application, the bus 113 is used to connect the communication interface 112, the processor 110 and the memory 111, and to communicate with each other among these devices.

在本申请的实施例中，存储器111，用于存储指令和数据。In the embodiment of the present application, the memory 111 is used to store instructions and data.

进一步地，在本申请的实施例中，上述处理器110，用于当接收到残差矩阵时，查找码字标识对应的预测模型，码字标识为编码阶段根据率失真代价确定出的预测模型对应的标识；当预测模型为第一预测模型时，确定与当前编码块在预设方向相邻的重构解码块；将重构解码块输入第一预测模型中，得到当前解码块的预测像素值；根据预测像素值和残差矩阵，得到当前解码块的解码像素值。Further, in the embodiment of the present application, the above-mentioned processor 110 is configured to, when the residual matrix is received, search for the prediction model corresponding to the codeword identifier, and the codeword identifier is the prediction model determined according to the rate-distortion cost at the encoding stage Corresponding identification; when the prediction model is the first prediction model, determine the reconstructed decoded block adjacent to the current coding block in the preset direction; input the reconstructed decoded block into the first prediction model to obtain the predicted pixels of the current decoded block Value; According to the predicted pixel value and the residual matrix, the decoded pixel value of the current decoded block is obtained.

在实际应用中，上述存储器111可以是易失性第一存储器(volatile memory)，例如随机存取第一存储器(Random-Access Memory，RAM)；或者非易失性第一存储器(non-volatile memory)，例如只读第一存储器(Read-Only Memory，ROM)，快闪第一存储器(flash memory)，硬盘(Hard Disk Drive，HDD)或固态硬盘(Solid-State Drive，SSD)；或者上述种类的第一存储器的组合，并向处理器110提供指令和数据。In practical applications, the aforementioned memory 111 may be a volatile memory (volatile memory), such as a random-access memory (Random-Access Memory, RAM); or a non-volatile memory (non-volatile memory). ), such as Read-Only Memory (ROM), Flash Memory (Flash Memory), Hard Disk Drive (HDD) or Solid-State Drive (SSD); or the above types And provide instructions and data to the processor 110.

另外，在本实施例中的各功能模块可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。In addition, the functional modules in this embodiment may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be realized in the form of hardware or software function module.

集成的单元如果以软件功能模块的形式实现并非作为独立的产品进行销售或使用时，可以存储在一个计算机可读取存储介质中，基于这样的理解，本实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或processor(处理器)执行本实施例方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or correct The part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal A computer, a server, or a network device, etc.) or a processor (processor) execute all or part of the steps of the method in this embodiment. The aforementioned storage media include: U disk, mobile hard disk, read only memory (Read Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes.

本申请实施例提出的一种帧内预测装置，当接收到残差矩阵时，查找码字标识对应的预测模型，码字标识为编码阶段根据率失真代价确定出的预测模型对应的标识；当预测模型为第一预测模型时，确定与当前编码块在预设方向相邻的重构解码块；将重构解码块输入第一预测模型中，得到当前解码块的预测像素值；根据预测像素值和残差矩阵，得到当前解码块的解码像素值。由此可见，在本申请的实施例中，帧内预测装置将与当前解码块在预设方向相邻的重构解码块输入第一预测模型中，得到当前解码块的预测像素值，由于重构解码块为完整像素块，具备准确的纹理信息，因此将重构解码块输入第一预测模型中，得到当前解码块的预测像素值，能够准确的对纹理信息进行描述，进而提高了帧内预测的准确性。The intra-frame prediction device proposed in the embodiment of the application searches for the prediction model corresponding to the codeword identifier when the residual matrix is received, and the codeword identifier is the identifier corresponding to the prediction model determined according to the rate-distortion cost at the encoding stage; When the prediction model is the first prediction model, determine the reconstructed decoded block adjacent to the current coding block in the preset direction; input the reconstructed decoded block into the first prediction model to obtain the predicted pixel value of the current decoded block; according to the predicted pixel Value and residual matrix to get the decoded pixel value of the current decoded block. It can be seen that, in the embodiment of the present application, the intra prediction device inputs the reconstructed decoded block adjacent to the current decoded block in the preset direction into the first prediction model to obtain the predicted pixel value of the current decoded block. The constructed decoded block is a complete pixel block with accurate texture information. Therefore, the reconstructed decoded block is input into the first prediction model to obtain the predicted pixel value of the current decoded block, which can accurately describe the texture information, thereby improving the intraframe The accuracy of the forecast.

本申请实施例提供计算机可读存储介质，其上存储有程序，该程序被处理器执行时实现如上所述的帧内预测方法。The embodiment of the present application provides a computer-readable storage medium on which a program is stored, and when the program is executed by a processor, the intra prediction method as described above is realized.

具体来讲，本实施例中的一种帧内预测方法对应的程序指令可以被存储在光盘，硬盘，U盘等存储介质上，当存储介质中的与一种帧内预测方法对应的程序指令被一电子设备读取或被执行时，实现如上述任一项所述的帧内预测方法。Specifically, the program instructions corresponding to an intra-frame prediction method in this embodiment can be stored on storage media such as optical disks, hard disks, USB flash drives, etc. When the program instructions corresponding to an intra-frame prediction method in the storage medium When read or executed by an electronic device, the intra prediction method as described in any one of the above is implemented.

本领域内的技术人员应明白，本申请的实施例可提供为方法、***、或计算机程序产品。因此，本申请可采用硬件实施例、软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器和光学存储器等)上实施的计算机程序产品的形式。Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, this application may adopt the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Moreover, this application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, optical storage, etc.) containing computer-usable program codes.

本申请是参照根据本申请实施例的方法、设备(***)、和计算机程序产品的实现流程示意图和/或方框图来描述的。应理解可由计算机程序指令实现流程示意图和/或方框图中的每一流程和/或方框、以及实现流程示意图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在实现流程示意图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。This application is described with reference to the schematic diagrams and/or block diagrams of the implementation process of the method, equipment (system), and computer program product according to the embodiments of the application. It should be understood that each process and/or block in the schematic flow diagram and/or block diagram can be implemented by computer program instructions, and a combination of processes and/or blocks in the schematic flow diagram and/or block diagram can be implemented. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing equipment to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing equipment are generated A device for realizing the functions specified in one or more processes in the schematic flow chart and/or one block or multiple blocks in the block diagram.

这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在实现流程示意图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device. The device realizes the functions specified in one or more processes in the schematic diagram and/or one block or more in the block diagram.

这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在实现流程示意图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to produce computer-implemented processing, so as to execute on the computer or other programmable equipment. The instructions provide steps for implementing functions specified in one or more processes in the schematic diagram and/or one block or more in the block diagram.

以上所述，仅为本申请的较佳实施例而已，并非用于限定本申请的保护范围。The above are only the preferred embodiments of the present application, and are not used to limit the protection scope of the present application.

工业实用性Industrial applicability

本申请实施例提供了一种帧内预测方法、装置及计算机存储介质，帧内预测装置将与当前解码块在预设方向相邻的重构解码块输入第一预测模型中，得到当前解码块的预测像素值，由于重构解码块为完整像素块，具备准确的纹理信息，因此将重构解码块输入第一预测模型中，得到当前解码块的预测像素值，能够准确的对纹理信息进行描述，进而提高了帧内预测的准确性。The embodiments of the application provide an intra-frame prediction method, device, and computer storage medium. The intra-frame prediction device inputs the reconstructed decoded block adjacent to the current decoded block in a preset direction into the first prediction model to obtain the current decoded block Because the reconstructed decoded block is a complete pixel block and has accurate texture information, the reconstructed decoded block is input into the first prediction model to obtain the predicted pixel value of the current decoded block, which can accurately perform texture information Description, thereby improving the accuracy of intra prediction.

Claims

一种帧内预测方法，所述方法包括：An intra-frame prediction method, the method includes:

当接收到残差矩阵时，查找码字标识对应的预测模型，所述码字标识为编码阶段根据率失真代价确定出的预测模型对应的标识；When the residual matrix is received, search for the prediction model corresponding to the codeword identifier, where the codeword identifier is the identifier corresponding to the prediction model determined according to the rate-distortion cost in the encoding stage;

当所述预测模型为第一预测模型时，确定与当前编码块在预设方向相邻的重构解码块；When the prediction model is the first prediction model, determine the reconstructed decoding block adjacent to the current coding block in the preset direction;

将所述重构解码块输入所述第一预测模型中，得到所述当前解码块的预测像素值；Input the reconstructed decoded block into the first prediction model to obtain the predicted pixel value of the current decoded block;

根据所述预测像素值和所述残差矩阵，得到所述当前解码块的解码像素值。According to the predicted pixel value and the residual matrix, the decoded pixel value of the current decoded block is obtained.
根据权利要求1所述的方法，其中，所述查找码字标识对应的预测模型之后，所述方法还包括：The method according to claim 1, wherein after the search for the prediction model corresponding to the codeword identifier, the method further comprises:

当所述预测模型为第二预测模型时，获取与所述当前解码块在所述预设方向相邻的边界解码块，所述边界解码块为所述重构解码块中、与所述当前解码块相邻的解码块，所述第二预测模型的率失真代价小于所述第一预测模型的率失真代价；When the prediction model is the second prediction model, obtain a boundary decoding block adjacent to the current decoding block in the preset direction, and the boundary decoding block is the reconstructed decoding block that is the same as the current decoding block. Decoding blocks adjacent to the decoding block, where the rate distortion cost of the second prediction model is less than the rate distortion cost of the first prediction model;

根据所述边界解码块和所述第二预测模型，得到所述预测像素值。Obtain the predicted pixel value according to the boundary decoding block and the second prediction model.
根据权利要求1所述的方法，其中，所述将所述重构解码块输入所述第一预测模型中，得到所述当前解码块的预测像素值，包括：The method according to claim 1, wherein the inputting the reconstructed decoded block into the first prediction model to obtain the predicted pixel value of the current decoded block comprises:

确定出所述重构解码块对应的解码矩阵，其中，所述解码矩阵为将所述重构解码块中损失的解码块填充0像素值得到的矩阵；Determining a decoding matrix corresponding to the reconstructed decoded block, where the decoded matrix is a matrix obtained by filling the decoded block lost in the reconstructed decoded block with 0 pixel values;

将所述解码矩阵输入所述第一预测模型中，得到所述预测像素值。The decoding matrix is input into the first prediction model to obtain the predicted pixel value.
根据权利要求3所述的方法，其中，所述将所述解码矩阵输入所述第一预测模型中，得到所述预测像素值，包括：The method according to claim 3, wherein said inputting said decoding matrix into said first prediction model to obtain said predicted pixel value comprises:

利用所述第一预测模型中的第一组卷积层，确定所述重构解码块的图像纹理特征信息；Using the first group of convolutional layers in the first prediction model to determine the image texture feature information of the reconstructed decoded block;

利用所述第一预测模型中的第二组卷积层，确定所述重构解码块的图像纹理分布信息；Determine the image texture distribution information of the reconstructed decoded block by using the second group of convolutional layers in the first prediction model;

根据所述图像纹理特征信息和所述图像纹理分布信息，得到图像拼接信息；Obtaining image mosaic information according to the image texture feature information and the image texture distribution information;

利用所述第一预测模型中的第三组卷积层，确定所述图像拼接信息的特征信息；Using the third group of convolutional layers in the first prediction model to determine the feature information of the image mosaic information;

基于所述特征信息，确定出所述当前解码块的所述预测像素值，其中，所述预测像素值为所述特征信息进行卷积计算得到的像素值。Based on the characteristic information, the predicted pixel value of the current decoded block is determined, where the predicted pixel value is a pixel value obtained by performing convolution calculation on the characteristic information.
根据权利要求1所述的方法，其中，所述查找码字标识对应的预测模型之前，所述方法还包括：The method according to claim 1, wherein before the searching for the prediction model corresponding to the codeword identifier, the method further comprises:

接收二进制比特流；Receive binary bit stream;

根据所述二进制比特流，确定所述残差矩阵。According to the binary bit stream, the residual matrix is determined.
根据权利要求1-4任一项所述的方法，其中，所述预设方向为左侧、左上侧、上侧和右侧。The method according to any one of claims 1-4, wherein the preset directions are left side, upper left side, upper side and right side.
根据权利要求2所述的方法，其中，所述第二预测模型包括Planar模式、直流系数DC模式和多种角度模式中的任一种。The method according to claim 2, wherein the second prediction model includes any one of Planar mode, direct current coefficient DC mode and multiple angle modes.
一种帧内预测装置，所述帧内预测装置包括：An intra-frame prediction device, which includes:

查找部分，用于当接收到残差矩阵时，查找码字标识对应的预测模型，所述码字标识为编码阶段根据率失真代价确定出的预测模型对应的标识；The search part is used to search for the prediction model corresponding to the codeword identifier when the residual matrix is received, where the codeword identifier is the identifier corresponding to the prediction model determined according to the rate-distortion cost in the encoding stage;

确定部分，用于当所述预测模型为第一预测模型时，确定与当前编码块在预设方向相邻的重构解码块；The determining part is used to determine the reconstructed decoding block adjacent to the current coding block in a preset direction when the prediction model is the first prediction model;

计算部分，用于将所述重构解码块输入所述第一预测模型中，得到所述当前解码块的预测像素值；The calculation part is used to input the reconstructed decoded block into the first prediction model to obtain the predicted pixel value of the current decoded block;

帧内预测部分，用于根据所述预测像素值和所述残差矩阵，得到所述当前解码块的解码像素值。The intra-frame prediction part is used to obtain the decoded pixel value of the current decoded block according to the predicted pixel value and the residual matrix.
根据权利要求8所述的装置，其中，The device according to claim 8, wherein:

所述查找部分，还用于当所述预测模型为第二预测模型时，获取与所述当前解码块在所述预设方向相邻的边界解码块，所述边界解码块为所述重构解码块中、与所述当前解码块相邻的解码块，所述第二预测模型的率失真代价小于所述第一预测模型的率失真代价；The searching part is further configured to obtain a boundary decoding block adjacent to the current decoding block in the preset direction when the prediction model is a second prediction model, and the boundary decoding block is the reconstruction In a decoding block adjacent to the current decoding block, the rate distortion cost of the second prediction model is less than the rate distortion cost of the first prediction model;

所述计算部分，还用于根据所述边界解码块和所述第二预测模型，得到所述预测像素值。The calculation part is further configured to obtain the predicted pixel value according to the boundary decoding block and the second prediction model.
根据权利要求8所述的装置，其中，The device according to claim 8, wherein:

所述确定部分，用于确定出所述重构解码块对应的解码矩阵，其中，所述解码矩阵为将所述重构解码块中损失的解码块填充0像素值得到的矩阵；The determining part is configured to determine a decoding matrix corresponding to the reconstructed decoded block, where the decoded matrix is a matrix obtained by filling the lost decoded block in the reconstructed decoded block with 0 pixel values;

所述计算部分，还用于将所述解码矩阵输入所述第一预测模型中，得到所述预测像素值。The calculation part is also used to input the decoding matrix into the first prediction model to obtain the predicted pixel value.
根据权利要求10所述的装置，其中，The device according to claim 10, wherein:

所述确定部分，还用于利用所述第一预测模型中的第一组卷积层，确定所述重构解码块的图像纹理特征信息；利用所述第一预测模型中的第二组卷积层，确定所述重构解码块的图像纹理分布信息；根据所述图像纹理特征信息和所述图像纹理分布信息，得到图像拼接信息；利用所述第一预测模型中的第三组卷积层，确定所述图像拼接信息的特征信息；基于所述特征信息，确定出所述当前解码块的所述预测像素值，其中，所述预测像素值为所述特征信息进行卷积计算得到的像素值。The determining part is further configured to use the first set of convolutional layers in the first prediction model to determine the image texture feature information of the reconstructed decoded block; use the second set of convolutional layers in the first prediction model Layer, determine the image texture distribution information of the reconstructed decoded block; obtain image mosaic information according to the image texture feature information and the image texture distribution information; use the third set of convolutions in the first prediction model Layer, determining the feature information of the image stitching information; based on the feature information, determining the predicted pixel value of the current decoding block, where the predicted pixel value is obtained by convolution calculation of the feature information Pixel values.
根据权利要求8所述的装置，其中，所述装置还包括：接收部分；The device according to claim 8, wherein the device further comprises: a receiving part;

所述接收部分，用于接收二进制比特流；The receiving part is used to receive a binary bit stream;

所述确定部分，还用于根据所述二进制比特流，确定所述残差矩阵。The determining part is further configured to determine the residual matrix according to the binary bit stream.
根据权利要求9-12任一项所述的装置，其中，所述预设方向为左侧、左上侧、上侧和右侧。The device according to any one of claims 9-12, wherein the preset direction is left side, upper left side, upper side and right side.
根据权利要求9所述的装置，其中，所述第二预测模型包括平面Planar模式、直流系数DC模式和多种角度模式中的任一种。The apparatus according to claim 9, wherein the second prediction model includes any one of a plane Planar mode, a direct current coefficient DC mode, and multiple angle modes.
一种帧内预测装置，其中，所述帧内预测装置包括处理器、存储有所述处理器可执行指令的存储器、通信接口，和用于连接所述处理器、所述存储器以及所述通信接口的总线，当所述指令被执行时，所述处理器执行时实现如权利要求1-7任一项所述的方法。An intra prediction device, wherein the intra prediction device includes a processor, a memory storing executable instructions of the processor, a communication interface, and a communication interface for connecting the processor, the memory, and the communication The interface bus, when the instructions are executed, the processor implements the method according to any one of claims 1-7 when executed.
一种计算机可读存储介质，其上存储有程序，应用于帧内预测装置中，其中，所述程序被处理器执行时实现如权利要求1-7任一项所述的方法。A computer-readable storage medium with a program stored thereon and applied to an intra-frame prediction device, wherein the program is executed by a processor to implement the method according to any one of claims 1-7.