CN117795956A

CN117795956A - Video encoding/decoding method and apparatus

Info

Publication number: CN117795956A
Application number: CN202280052790.0A
Authority: CN
Inventors: 朴胜煜; 许镇
Original assignee: Hyundai Motor Co; Kia Corp
Current assignee: Hyundai Motor Co; Kia Corp
Priority date: 2021-08-05
Filing date: 2022-08-05
Publication date: 2024-03-29

Abstract

Video encoding/decoding methods and apparatus are provided. The video decoding method according to the present disclosure may include the steps of: deriving a Most Probable Mode (MPM) list based on intra prediction modes of neighboring blocks neighboring the current block; applying a candidate mode in the MPM list or an intra-prediction mode of a neighboring block to a reference pixel of a first region neighboring the current block to generate a prediction pixel; calculating a residual transform absolute value sum between the predicted pixel and the reconstructed pixel of the first region; deriving a first intra prediction mode based on the residual transform absolute value sum; deriving an intra prediction mode of the current block based on the first intra prediction mode; and generating a prediction block of the current block based on the intra prediction mode of the current block.

Description

Video encoding/decoding method and apparatus

Technical Field

The present disclosure relates to a video encoding/decoding method and a video encoding/decoding apparatus, and more particularly, to a video encoding/decoding method and a video encoding/decoding apparatus that derive a best intra prediction mode for a current block based on a template adjacent to the current block.

Background

The following description merely provides background information related to the present embodiment and does not constitute prior art.

Since the amount of video data is larger than that of voice data or still image data, storing or transmitting video data without processing the video data by compression requires a large amount of hardware resources including a memory.

Thus, in storing or transmitting video data, an encoder is typically used to compress the video data for storage or transmission. Then, the decoder receives the compressed video data and decompresses and reproduces the video data. Compression techniques for this video include h.264/AVC and High Efficiency Video Coding (HEVC), and Versatile Video Coding (VVC) with coding efficiency improved by about 30% or more compared to HEVC.

However, video size, resolution, and frame rate gradually increase, and thus the amount of data to be encoded also increases. Thus, new compression techniques having better coding efficiency and higher image quality than existing compression techniques are needed.

Intra prediction is a prediction technique that allows only spatial reference, and refers to a method of predicting a current block by referring to blocks reconstructed around a block on which encoding has been currently performed. In the case of intra prediction, the intra prediction mode of the current block may be derived using a Most Probable Mode (MPM) list. In addition, the intra prediction mode of the current block may be derived using templates adjacent to the current block. In the case of deriving an intra prediction mode for a current block using a template adjacent to the current block, improvement in coding efficiency is required.

Disclosure of Invention

[ technical problem ]

It is an object of the present disclosure to provide a method and apparatus for deriving an intra prediction mode of a current block based on a template.

It is another object of the present disclosure to provide a method and apparatus for deriving an intra prediction mode of a current block using only a limited template.

It is another object of the present disclosure to provide a method and apparatus for deriving an intra prediction mode of a current block using a Most Probable Mode (MPM) list and a template.

It is another object of the present disclosure to provide a method and apparatus for deriving an intra prediction mode of a current block using an intra prediction mode of a reference block adjacent to the current block and a template.

It is another object of the present disclosure to provide a method and apparatus for improving video encoding/decoding efficiency.

It is another object of the present disclosure to provide a recording medium storing a bitstream generated by the video encoding/decoding method or the video encoding/decoding apparatus of the present disclosure.

It is another object of the present disclosure to provide a method and apparatus for transmitting a bitstream generated by the video encoding/decoding method or apparatus of the present disclosure.

[ technical solution ]

According to the present disclosure, a video decoding method includes: deriving a Most Probable Mode (MPM) list based on intra prediction modes of neighboring blocks neighboring the current block; generating a prediction pixel by applying a candidate pattern in the MPM list or an intra prediction pattern of a neighboring block to a reference pixel of a first region neighboring the current block; calculating a residual transform absolute value Sum (SATD) between the predicted pixel and the reconstructed pixel of the first region; deriving a first intra-prediction mode based on the SATD; deriving an intra prediction mode of the current block based on the first intra prediction a mode; a prediction block of the current block is generated based on an intra prediction mode of the current block.

According to the present disclosure, a video encoding method includes: determining a Most Probable Mode (MPM) list based on intra prediction modes of neighboring blocks neighboring the current block; generating a prediction pixel by applying a candidate pattern in the MPM list or an intra prediction pattern of a neighboring block to a reference pixel of a first region neighboring the current block; calculating a residual transform absolute value and SATD between the predicted pixel and the reconstructed pixel of the first region; determining a first intra-prediction mode based on the SATD; determining an intra prediction mode of the current block based on the first intra prediction a mode; a prediction block of the current block is generated based on an intra prediction mode of the current block.

Further, according to the present disclosure, a method of transmitting a bitstream generated by a video encoding method or apparatus according to the present disclosure may be provided.

In addition, according to the present disclosure, a recording medium storing a bitstream generated by the video encoding method or apparatus according to the present disclosure may be provided.

In addition, according to the present disclosure, a recording medium storing a bitstream received and decoded by a video decoding apparatus according to the present disclosure and used to reconstruct video may be provided.

[ beneficial effects ]

In accordance with the present disclosure, methods and apparatus for deriving an intra prediction mode of a current block based on a template may be provided.

In addition, according to the present disclosure, a method and apparatus for deriving an intra prediction mode of a current block using only a limited template may be provided.

In addition, according to the present disclosure, a method and apparatus for deriving an intra prediction mode of a current block using a Most Probable Mode (MPM) list and a template may be provided.

In addition, according to the present disclosure, methods and apparatuses for deriving an intra prediction mode of a current block using an intra prediction mode of a reference block adjacent to the current block and a template may be provided.

Further, according to the present disclosure, a method and apparatus for improving video encoding/decoding efficiency may be provided.

Effects that can be obtained from the present disclosure are not limited to the above-mentioned effects, and other effects that are not mentioned can be clearly understood by those of ordinary skill in the art from the following description.

Drawings

Fig. 1 is a block diagram illustrating a video encoding device in which the techniques of this disclosure may be implemented.

Fig. 2 is a diagram illustrating a method for partitioning blocks using a quadtree plus binary tree trigeminal tree (QTBTTT) structure.

Fig. 3a and 3b are diagrams illustrating a plurality of intra prediction modes including a wide-angle intra prediction mode.

Fig. 4 is a block diagram illustrating the neighboring blocks of the current block.

Fig. 5 is a block diagram illustrating a video decoding device that may implement the techniques of this disclosure.

Fig. 6 is a diagram illustrating templates and reference pixels of templates for deriving template-based intra prediction modes according to an embodiment of the present disclosure.

Fig. 7 is a diagram illustrating templates and reference pixels of templates for deriving template-based intra prediction modes according to another embodiment of the present disclosure.

Fig. 8 is a diagram illustrating a reference block for generating a Most Probable Mode (MPM) list according to an embodiment of the disclosure.

Fig. 9 is a diagram illustrating a method for generating an MPM list when all intra prediction modes of a reference block are non-directional modes according to an embodiment of the present disclosure.

Fig. 10 is a diagram illustrating a method for generating an MPM list when intra prediction modes of a reference block are a non-directional mode and a directional mode, respectively, according to an embodiment of the present disclosure.

Fig. 11 is a diagram illustrating a method for generating an MPM list when intra prediction modes of reference blocks are different directional modes according to an embodiment of the present disclosure.

Fig. 12 is a diagram illustrating neighboring blocks neighboring a current block according to an embodiment of the present disclosure.

Fig. 13 is a diagram illustrating a histogram of a pattern of a block adjacent to a current block according to an embodiment of the present disclosure.

Fig. 14 is a diagram illustrating a process of deriving a template-based intra prediction mode using a histogram of modes of blocks adjacent to a current block according to an embodiment of the present disclosure.

Fig. 15 is a diagram showing 49 and 51 modes of resolution 64 of a vertical mode according to an embodiment of the present disclosure.

Fig. 16 is a diagram illustrating syntax related to the template-based intra prediction mode derivation method of fig. 15 according to an embodiment of the present disclosure.

Fig. 17 is a diagram illustrating a method for determining a pattern based on the index of fig. 16 according to an embodiment of the present disclosure.

Fig. 18 is a diagram illustrating a video decoding process according to an embodiment of the present disclosure.

Fig. 19 is a diagram illustrating a video encoding process according to an embodiment of the present disclosure.

Detailed Description

Hereinafter, some embodiments of the present disclosure are described in detail with reference to the accompanying drawings. In the following description, like reference numerals denote like elements, although the elements are shown in different drawings. Furthermore, in the following description of some embodiments, a detailed description of related known components and functions when considered as obscuring the subject matter of the present disclosure has been omitted for the sake of clarity and conciseness.

Fig. 1 is a block diagram of a video encoding device in which the techniques of this disclosure may be implemented. Hereinafter, a video encoding apparatus and components of the apparatus are described with reference to the diagram of fig. 1.

The encoding apparatus may include a picture divider 110, a predictor 120, a subtractor 130, a transformer 140, a quantizer 145, a rearrangement unit 150, an entropy encoder 155, an inverse quantizer 160, an inverse transformer 165, an adder 170, a loop filter unit 180, and a memory 190.

Each component of the encoding apparatus may be implemented as hardware or software or as a combination of hardware and software. Further, the function of each component may be implemented as software, and a microprocessor may also be implemented to execute the function of the software corresponding to each component.

A video is made up of one or more sequences comprising a plurality of pictures. Each picture is divided into a plurality of regions, and encoding is performed on each region. For example, a picture is segmented into one or more tiles or/and slices. Herein, one or more tiles may be defined as a set of tiles. Each tile or/and slice is partitioned into one or more Coding Tree Units (CTUs). In addition, each CTU is partitioned into one or more Coding Units (CUs) by a tree structure. Information applied to each CU is encoded as a syntax of the CU, and information commonly applied to CUs included in one CTU is encoded as a syntax of the CTU. Further, information commonly applied to all blocks in one slice is encoded as a syntax of a slice header, and information applied to all blocks constituting one or more pictures is encoded as a Picture Parameter Set (PPS) or a picture header. Furthermore, information commonly referred to by a plurality of pictures is encoded to a Sequence Parameter Set (SPS). In addition, information commonly referenced by one or more SPS's is encoded to a Video Parameter Set (VPS). Further, information commonly applied to one tile or group of tiles may also be encoded as syntax of the tile or group of tiles header. The syntax included in the SPS, PPS, slice header, tile, or tile group header may be referred to as a high level syntax.

The picture divider 110 determines the size of a Coding Tree Unit (CTU). Information about the size of the CTU (CTU size) is encoded as a syntax of the SPS or PPS and transmitted to the video decoding apparatus.

The picture divider 110 divides each picture constituting a video into a plurality of Coded Tree Units (CTUs) having a predetermined size, and then recursively divides the CTUs by using a tree structure. Leaf nodes in the tree structure become Coding Units (CUs), which are the basic units of coding.

The tree structure may be a Quadtree (QT) in which a higher node (or parent node) is divided into four lower nodes (or child nodes) of the same size. The tree structure may also be a Binary Tree (BT) in which a higher node is split into two lower nodes. The tree structure may also be a Trigeminal Tree (TT), wherein the higher nodes are represented by 1:2: the ratio of 1 is divided into three lower nodes. The tree structure may also be a structure in which two or more of a QT structure, a BT structure, and a TT structure are mixed. For example, a quadtree plus binary tree (QTBT) structure may be used or a quadtree plus binary tree trigeminal tree (QTBTTT) structure may be used. Here, BTTT is added to the tree structure to be referred to as a multi-type tree (MTT).

Fig. 2 is a diagram for describing a method of dividing a block by using the QTBTTT structure.

As shown in fig. 2, CTUs may be first divided into QT structures. Quadtree partitioning may be recursive until the size of the partitioned block reaches the minimum block size (MinQTSize) of leaf nodes allowed in QT. A first flag (qt_split_flag) indicating whether each node of the QT structure is divided into four nodes of a lower layer is encoded by the entropy encoder 155 and transmitted to the video decoding apparatus. When the leaf node of QT is not greater than the maximum block size (MaxBTSize) of the root node allowed in BT, the leaf node may be further classified into at least one of BT structure or TT structure. There may be multiple directions of segmentation in the BT structure and/or the TT structure. For example, there may be two directions, i.e., a direction in which the block of the corresponding node is divided horizontally and a direction in which the block of the corresponding node is divided vertically. As shown in fig. 2, when the MTT segmentation starts, a second flag (MTT _split_flag) indicating whether the node is segmented and a flag additionally indicating a segmentation direction (vertical or horizontal) and/or a flag indicating a segmentation type (binary or trigeminal) if the node is segmented are encoded by the entropy encoder 155 and transmitted to the video decoding device.

Alternatively, a CU split flag (split_cu_flag) indicating whether a node is split may also be encoded before encoding a first flag (qt_split_flag) indicating whether each node is split into four nodes of the lower layer. When the value of the CU partition flag (split_cu_flag) indicates that each node is not partitioned, the block of the corresponding node becomes a leaf node in the partition tree structure and becomes a CU as a basic unit of encoding. When the value of the CU partition flag (split_cu_flag) indicates that each node is partitioned, the video encoding apparatus first starts encoding the first flag through the above scheme.

When QTBT is used as another embodiment of the tree structure, there may be two types, i.e., a type in which a block of a corresponding node is horizontally divided into two blocks having the same size (i.e., symmetrical horizontal division) and a type in which a block of a corresponding node is vertically divided into two blocks having the same size (i.e., symmetrical vertical division). A partition flag (split_flag) indicating whether each node of the BT structure is partitioned into lower-layer blocks and partition type information indicating a partition type are encoded by the entropy encoder 155 and transmitted to the video decoding apparatus. Meanwhile, there may additionally be a type in which a block of a corresponding node is divided into two blocks in an asymmetric form with each other. The asymmetric form may include where the blocks of the respective nodes are partitioned to have 1:3, or may also include a form in which blocks of corresponding nodes are divided in a diagonal direction.

A CU may have different sizes according to QTBT or QTBTTT partitions from CTUs. Hereinafter, a block corresponding to a CU to be encoded or decoded (i.e., a leaf node of QTBTTT) is referred to as a "current block". Since QTBTTT segmentation is employed, the shape of the current block may be rectangular in shape in addition to square shape.

The predictor 120 predicts a current block to generate a predicted block. Predictor 120 includes an intra predictor 122 and an inter predictor 124.

In general, each of the current blocks in a picture may be predictively encoded. In general, prediction of a current block may be performed by using an intra prediction technique (using data from a picture including the current block) or an inter prediction technique (using data from a picture encoded before the picture including the current block). Inter prediction includes both unidirectional prediction and bi-directional prediction.

The intra predictor 122 predicts pixels in the current block by using pixels (reference pixels) located at neighbors of the current block in the current picture including the current block. Depending on the prediction direction, there are multiple intra prediction modes. For example, as shown in fig. 3a, the plurality of intra prediction modes may include 2 non-directional modes including a planar mode and a DC mode, and may include 65 directional modes. The neighboring pixels and the arithmetic equation to be used are differently defined according to each prediction mode.

For efficient directional prediction of a current block having a rectangular shape, directional modes (# 67 to # 80), intra prediction modes # -1 to # -14) as indicated by dashed arrows in fig. 3b may be additionally used. The orientation mode may be referred to as a "wide-angle intra prediction mode". In fig. 3b, the arrows indicate the corresponding reference samples for prediction and do not represent the prediction direction. The predicted direction is opposite to the direction indicated by the arrow. When the current block has a rectangular shape, the wide-angle intra prediction mode is a mode in which prediction is performed in a direction opposite to a specific orientation mode without additional bit transmission. In this case, in the wide-angle intra prediction mode, some wide-angle intra prediction modes available for the current block may be determined by a ratio of a width and a height of the current block having a rectangular shape. For example, when the current block has a rectangular shape with a height smaller than a width, wide-angle intra prediction modes (intra prediction modes #67 to # 80) having angles smaller than 45 degrees are available. When the current block has a rectangular shape having a width greater than a height, a wide-angle intra prediction mode having an angle greater than-135 degrees may be used.

The intra predictor 122 may determine intra prediction to be used for encoding the current block. In some embodiments, intra predictor 122 may encode the current block by using a plurality of intra prediction modes, and also select an appropriate intra prediction mode to be used from among the test modes. For example, the intra predictor 122 may calculate a rate-distortion value by using a rate-distortion analysis for a plurality of tested intra prediction modes, and also select an intra prediction mode having the best rate-distortion characteristic among the tested modes.

The intra predictor 122 selects one intra prediction mode among a plurality of intra prediction modes, and predicts the current block by using neighboring pixels (reference pixels) and an arithmetic equation determined according to the selected intra prediction mode. Information about the selected intra prediction mode is encoded by the entropy encoder 155 and transmitted to the video decoding apparatus.

The inter predictor 124 generates a prediction block for the current block by using a motion compensation process. The inter predictor 124 searches for a block most similar to the current block in a reference picture encoded and decoded earlier than the current picture, and generates a prediction block for the current block by using the searched block. In addition, a Motion Vector (MV) is generated, which corresponds to a displacement between a current block in the current picture and a predicted block in the reference picture. In general, motion estimation is performed on a luminance component, and a motion vector calculated based on the luminance component is used for both the luminance component and the chrominance component. Motion information including information on a reference picture and information on a motion vector for predicting a current block is encoded by the entropy encoder 155 and transmitted to a video decoding apparatus.

The inter predictor 124 may also perform interpolation on reference pictures or reference blocks in order to increase the accuracy of prediction. In other words, sub-samples between two consecutive integer samples are interpolated by applying the filter coefficients to a plurality of consecutive integer samples comprising the two integer samples. When the process of searching for a block most similar to the current block is performed for the interpolated reference picture, a non-integer sample unit precision may be represented for the motion vector, but a decimal unit precision. The precision or resolution of the motion vector may be set differently for each target region to be encoded (e.g., units such as slices, tiles, CTUs, CUs, etc.). When such Adaptive Motion Vector Resolution (AMVR) is applied, information about the motion vector resolution to be applied to each target region should be transferred for each target region. For example, when the target area is a CU, information about the resolution of a motion vector applied to each CU is transmitted. The information on the resolution of the motion vector may be information representing the accuracy of a motion vector difference to be described below.

Meanwhile, the inter predictor 124 may perform inter prediction by using bi-directional prediction. In the case of bi-prediction, two reference pictures and two motion vectors representing block positions most similar to the current block in each reference picture are used. The inter predictor 124 selects a first reference picture and a second reference picture from the reference picture list0 (RefPicList 0) and the reference picture list1 (RefPicList 1), respectively. The inter predictor 124 also searches for a block most similar to the current block in the corresponding reference picture to generate a first reference block and a second reference block. In addition, a prediction block of the current block is generated by averaging or weighted-averaging the first reference block and the second reference block. In addition, motion information including information on two reference pictures for predicting the current block and information on two motion vectors is transmitted to the entropy encoder 155. Here, the reference picture list0 may be composed of pictures preceding the current picture in display order among the pre-reconstructed pictures, and the reference picture list1 may be composed of pictures following the current picture in display order among the pre-reconstructed pictures. However, although not particularly limited thereto, a pre-reconstructed picture following the current picture in display order may be additionally included in the reference picture list 0. Conversely, a pre-reconstructed picture preceding the current picture may be additionally included in the reference picture list 1.

In order to minimize the amount of bits consumed for encoding motion information, various methods may be used.

For example, when a reference picture and a motion vector of a current block are identical to those of a neighboring block, information capable of identifying the neighboring block is encoded to transmit motion information of the current block to a video decoding apparatus. This approach is called merge mode.

In the merge mode, the inter predictor 124 selects a predetermined number of merge candidate blocks (hereinafter, referred to as "merge candidates") from neighboring blocks of the current block.

As the neighboring blocks used to derive the merge candidates, as illustrated in fig. 4, all or some of a left block A0, a lower left block A1, an upper block B0, an upper right block B1, and an upper left block B2 adjacent to the current block in the current picture may be used. Further, a block located within a reference picture (which may be the same as or different from a reference picture used to predict the current block) that is not the current picture in which the current block is located may also be used as a merge candidate. For example, a block co-located with the current block within the reference picture or a block adjacent to the co-located block may be additionally used as a merge candidate. If the number of merging candidates selected by the method described above is smaller than the preset number, a zero vector is added to the merging candidates.

The inter predictor 124 configures a merge list including a predetermined number of merge candidates by using neighboring blocks. A merge candidate to be used as motion information of the current block is selected from among the merge candidates included in the merge list, and merge index information for identifying the selected candidate is generated. The generated merging index information is encoded by the entropy encoder 155 and transmitted to a video decoding apparatus.

The merge skip mode is a special case of the merge mode. After quantization, when all transform coefficients used for entropy encoding are close to zero, only neighboring block selection information is transmitted without transmitting a residual signal. By using the merge skip mode, relatively high encoding efficiency can be achieved for images with slight motion, still images, screen content images, and the like.

Hereinafter, the merge mode and the merge skip mode are collectively referred to as a merge/skip mode.

Another method for encoding motion information is Advanced Motion Vector Prediction (AMVP) mode.

In AMVP mode, the inter predictor 124 derives a motion vector predictor candidate for a motion vector of a current block by using neighboring blocks of the current block. As neighboring blocks used to derive the motion vector predictor candidates, all or some of a left block A0, a lower left block A1, an upper block B0, an upper right block B1, and an upper left block B2, which are neighboring to the current block in the current picture illustrated in fig. 4, may be used. Furthermore, a block located within a reference picture (which may be the same as or different from a reference picture used to predict the current block) that is not the current picture in which the current block is located may also be used as a neighboring block for deriving a motion vector predictor candidate. For example, a co-located block or a block adjacent to a co-located block of the current block within the reference picture may be used. If the number of motion vector candidates selected by the above method is less than a preset number, a zero vector is added to the motion vector candidates.

The inter predictor 124 derives a motion vector predictor candidate by using motion vectors of neighboring blocks and determines a motion vector predictor for a motion vector of the current block by using the motion vector predictor candidate. In addition, a motion vector difference is calculated by subtracting a motion vector predictor from a motion vector of the current block.

The motion vector predictor may be obtained by applying a predefined function (e.g., a center value and an average value calculation, etc.) to the motion vector predictor candidates. In this case, the video decoding device is also aware of the predefined function. Furthermore, since the neighboring block used to derive the motion vector predictor candidate is a block for which encoding and decoding have been completed, the video decoding apparatus may also already know the motion vector of the neighboring block. Thus, the video encoding apparatus does not need to encode information for identifying motion vector predictor candidates. Thus, in this case, information on the motion vector difference and information on the reference picture for predicting the current block are encoded.

Meanwhile, the motion vector predictor may also be determined by selecting a scheme of any one of the motion vector predictor candidates. In this case, information for identifying the selected motion vector predictor candidate is additionally encoded in combination with information about the motion vector difference and information about the reference picture for predicting the current block.

The subtractor 130 generates a residual block by subtracting the prediction block generated by the intra predictor 122 or the inter predictor 124 from the current block.

The transformer 140 converts a residual signal in a residual block having pixel values of a spatial domain into transform coefficients of a frequency domain. The transformer 140 may transform a residual signal in the residual block by using the total size of the residual block as a transform unit, or may also divide the residual block into a plurality of sub-blocks and may perform the transform by using the sub-blocks as transform units. Alternatively, the residual block is divided into two sub-blocks, a transform region and a non-transform region, respectively, to transform the residual signal using only the transform region sub-block as a transform unit. Here, the transform region sub-block may be 1 with a horizontal axis (or vertical axis) based: 1, one of two rectangular blocks of size ratio. In this case, a flag (cu_sbt_flag) indicates that only the sub-block is transformed, and directional (vertical/horizontal) information (cu_sbt_horizontal_flag) and/or position information (cu_sbt_pos_flag) is encoded by the entropy encoder 155 and transmitted to the video decoding apparatus. Furthermore, the transform region sub-block may have a size of 1 based on the horizontal axis (or vertical axis): 3. In this case, a flag (cu_sbt_quad_flag) indicating the corresponding division is additionally encoded by the entropy encoder 155 and transmitted to the video decoding apparatus.

Meanwhile, the transformer 140 may perform transformation on the residual block separately in the horizontal direction and the vertical direction. For the transformation, different types of transformation functions or transformation matrices may be used. For example, a pair of transform functions for horizontal transforms and vertical transforms may be defined as a Multiple Transform Set (MTS). The transformer 140 may select one transform function pair having the highest transform efficiency in the MTS and may transform the residual block in each of the horizontal and vertical directions. The information (mts_idx) of the transform function pair in the MTS is encoded by the entropy encoder 155 and transmitted to the video decoding apparatus.

The quantizer 145 quantizes the transform coefficient output from the transformer 140 using quantization parameters and outputs the quantized transform coefficient to the entropy encoder 155. The quantizer 145 may also directly quantize the relevant residual block without transformation of any block or frame. The quantizer 145 may also apply different quantization coefficients (scaling values) according to the positions of the transform coefficients in the transform block. A quantization matrix applied to quantized transform coefficients arranged in 2 dimensions may be encoded and signaled to a video decoding apparatus.

The rearrangement unit 150 may perform rearrangement of coefficient values for quantized residual values.

The rearrangement unit 150 may change the 2D coefficient array to a 1D coefficient sequence by using coefficient scanning. For example, the rearrangement unit 150 may output a 1D coefficient sequence by scanning the DC coefficient into a high frequency domain coefficient using a zig-zag scan or a diagonal scan. Instead of zig-zag scanning, vertical scanning that scans the 2D coefficient array in the column direction and horizontal scanning that scans the 2D block type coefficients in the row direction may also be used, depending on the size of the transform unit and the intra prediction mode. In other words, according to the size of the transform unit and the intra prediction mode, a scan method to be used may be determined in zig-zag scan, diagonal scan, vertical scan, and horizontal scan.

The entropy encoder 155 generates a bitstream by encoding a sequence of 1D quantized transform coefficients output from the rearrangement unit 150 using various encoding schemes including context-based adaptive binary arithmetic coding (CABAC), index golomb, and the like.

Further, the entropy encoder 155 encodes information related to block division, such as CTU size, CTU division flag, QT division flag, MTT division type, MTT division direction, etc., to allow the video decoding apparatus to divide the block equally with the video encoding apparatus. Further, the entropy encoder 155 encodes information on a prediction type indicating whether the current block is encoded by intra prediction or inter prediction. The entropy encoder 155 encodes intra prediction information (i.e., information about an intra prediction mode) or inter prediction information (a merge index in the case of a merge mode, and information about a reference picture index and a motion vector difference in the case of an AMVP mode) according to a prediction type. Further, the entropy encoder 155 encodes information related to quantization (i.e., information about quantization parameters and information about quantization matrices).

The inverse quantizer 160 dequantizes the quantized transform coefficients output from the quantizer 145 to generate transform coefficients. The inverse transformer 165 transforms the transform coefficients output from the inverse quantizer 160 from the frequency domain to the spatial domain to reconstruct the residual block.

The adder 170 adds the reconstructed residual block and the prediction block generated by the predictor 120 to reconstruct the current block. When intra prediction is performed on the next order block, pixels in the reconstructed current block may be used as reference pixels.

The loop filter unit 180 performs filtering on the reconstructed pixels in order to reduce block artifacts, ringing artifacts, blurring artifacts, etc., which occur due to block-based prediction and transform/quantization. The loop filter unit 180 as a loop filter may include all or some of a deblocking filter 182, a Sample Adaptive Offset (SAO) filter 184, and an Adaptive Loop Filter (ALF) 186.

The deblocking filter 182 filters boundaries between reconstructed blocks to remove block artifacts that occur due to block unit encoding/decoding, and the SAO filter 184 and ALF 186 perform additional filtering for the deblocked filtered video. The SAO filter 184 and ALF 186 are filters for compensating for differences between reconstructed pixels and original pixels, which occur due to lossy coding. The SAO filter 184 applies an offset as a CTU unit to enhance subjective image quality and coding efficiency. On the other hand, the ALF 186 performs block unit filtering, and compensates for distortion by applying different filters by separating boundaries of respective blocks and the degree of variation. Information about filter coefficients to be used for ALF may be encoded and signaled to a video decoding apparatus.

The reconstructed blocks filtered by the deblocking filter 182, the SAO filter 184, and the ALF 186 are stored in a memory 190. When all blocks in one picture are reconstructed, the reconstructed picture may be used as a reference picture for inter prediction of blocks within a picture to be encoded later.

Fig. 5 is a functional block diagram of a video decoding apparatus that may implement the techniques of this disclosure. Hereinafter, referring to fig. 5, a video decoding apparatus and components of the apparatus are described.

The video decoding apparatus may include an entropy decoder 510, a rearrangement unit 515, an inverse quantizer 520, an inverse transformer 530, a predictor 540, an adder 550, a loop filter unit 560, and a memory 570.

Similar to the video encoding device of fig. 1, each component of the video decoding device may be implemented as hardware or software or as a combination of hardware and software. Further, the function of each component may be implemented as software, and a microprocessor may also be implemented to execute the function of the software corresponding to each component.

The entropy decoder 510 extracts information related to block segmentation by decoding a bitstream generated by a video encoding apparatus to determine a current block to be decoded, and extracts prediction information required to restore the current block and information on a residual signal.

The entropy decoder 510 determines the size of a CTU by extracting information about the CTU size from a Sequence Parameter Set (SPS) or a Picture Parameter Set (PPS), and partitions a picture into CTUs having the determined size. In addition, the CTU is determined to be the highest layer of the tree structure, i.e., the root node, and the division information of the CTU may be extracted to divide the CTU using the tree structure.

For example, when dividing CTUs by using the QTBTTT structure, first a first flag (qt_split_flag) related to the division of QT is extracted to divide each node into four nodes of the lower layer. Further, for a node corresponding to a leaf node of QT, a second flag (MTT _split_flag), a split direction (vertical/horizontal), and/or a split type (binary/trigeminal) related to the split of the MTT are extracted to split the corresponding leaf node into an MTT structure. As a result, each node below the leaf node of QT is recursively partitioned into BT or TT structures.

As another embodiment, when the CTU is divided by using the QTBTTT structure, a CU division flag (split_cu_flag) indicating whether the CU is divided is extracted. The first flag (qt_split_flag) may also be extracted when the corresponding block is partitioned. During the segmentation process, 0 or more recursive MTT segmentations may occur after 0 or more recursive QT segmentations for each node. For example, MTT partitioning may occur immediately, or conversely, QT partitioning may occur only multiple times, relative to CTUs.

For another example, when the CTU is divided using the QTBT structure, a first flag (qt_split_flag) related to the division of QT is extracted to divide each node into four nodes of the lower layer. Further, a split flag (split_flag) indicating whether a node corresponding to a leaf node of QT is further split into BT and split direction information are extracted.

Meanwhile, when the entropy decoder 510 determines a current block to be decoded by using the partition of the tree structure, the entropy decoder 510 extracts information on a prediction type indicating whether the current block is intra-predicted or inter-predicted. When the prediction type information indicates intra prediction, the entropy decoder 510 extracts syntax elements for intra prediction information (intra prediction mode) of the current block. When the prediction type information indicates inter prediction, the entropy decoder 510 extracts information representing syntax elements of the inter prediction information (i.e., motion vectors and reference pictures to which the motion vectors refer).

Further, the entropy decoder 510 extracts quantization related information and extracts information on quantized transform coefficients of the current block as information on a residual signal.

The rearrangement unit 515 may change the sequence of the 1D quantized transform coefficients entropy-decoded by the entropy decoder 510 into a 2D coefficient array (i.e., block) again in the reverse order of the coefficient scan order performed by the video encoding apparatus.

The inverse quantizer 520 dequantizes the quantized transform coefficients, and dequantizes the quantized transform coefficients by using quantization parameters. The inverse quantizer 520 may also apply different quantized coefficients (scaling values) to the quantized transform coefficients arranged in 2D. The inverse quantizer 520 may perform inverse quantization by applying a matrix of quantized coefficients (scaled values) from the video encoding device to a 2D array of quantized transform coefficients.

The inverse transformer 530 generates a residual block for the current block by restoring a residual signal through inverse transforming the dequantized transform coefficients from the frequency domain to the spatial domain.

Further, when the inverse transformer 530 inversely transforms a partial region (sub-block) of the transform block, the inverse transformer 530 extracts a flag (cu_sbt_flag) of the transform block in which only the sub-block is transformed, direction (vertical/horizontal) information (cu_sbt_horizontal_flag) of the sub-block, and/or position information (cu_sbt_pos_flag) of the sub-block. The inverse transformer 530 also inversely transforms transform coefficients of the corresponding sub-block from the frequency domain to the spatial domain to reconstruct a residual signal and fills regions that are not inversely transformed with a value of "0" as the residual signal to generate a final residual block for the current block.

In addition, when applying MTS, the inverse transformer 530 determines a transform index or a transform matrix applied in each of the horizontal direction and the vertical direction by using MTS information (mts_idx) signaled from the video encoding apparatus. The inverse transformer 530 also performs inverse transformation on the transform coefficients in the transform block in the horizontal direction and the vertical direction by using the determined transform function.

The predictor 540 may include an intra predictor 542 and an inter predictor 544. The intra predictor 542 is activated when the prediction type of the current block is intra prediction, and the inter predictor 544 is activated when the prediction type of the current block is inter prediction.

The intra predictor 542 determines an intra prediction mode of the current block among a plurality of intra prediction modes according to a syntax element of the intra prediction mode extracted from the entropy decoder 510. The intra predictor 542 also predicts the current block by using neighboring reference pixels of the current block according to an intra prediction mode.

The inter predictor 544 determines a motion vector of the current block and a reference picture to which the motion vector refers by using syntax elements for the inter prediction mode extracted from the entropy decoder 510.

The adder 550 reconstructs the current block by adding the residual block output from the inverse transformer 530 and the prediction block output from the inter predictor 544 or the intra predictor 542. In intra prediction of a block to be decoded later, pixels within the reconstructed current block are used as reference pixels.

The loop filter unit 560, which is a loop filter, may include a deblocking filter 562, an SAO filter 564, and an ALF 566. Deblocking filter 562 performs deblocking filtering on boundaries between reconstructed blocks to remove blocking artifacts that occur due to block unit decoding. The SAO filter 564 and ALF 566 perform additional filtering on the reconstructed block after deblocking filtering to compensate for differences between the reconstructed pixels and the original pixels that occur due to lossy encoding. The filter coefficients of the ALF are determined by using information on the filter coefficients decoded from the bitstream.

The reconstructed block filtered by the deblocking filter 562, the SAO filter 564, and the ALF 566 are stored in a memory 570. When all blocks in one picture are reconstructed, the reconstructed picture may be used as a reference picture for inter prediction of blocks within a picture to be encoded later.

Fig. 6 is a diagram illustrating templates and reference pixels of templates for deriving template-based intra prediction modes according to an embodiment of the present disclosure. The intra prediction mode of the current block may be derived using a template (template) adjacent to the current block. By applying the directionality of all candidate patterns in the MPM list to the reference pixels of the template, a prediction template may be generated. Residual transform absolute value Sums (SATD) of pixels of the generated prediction template and pixels of the template that have been reconstructed may be calculated. Among the MPM candidate modes, the mode having the smallest SATD is an intra prediction mode of the current block, which is derived through a template-based intra prediction mode derivation method. The intra prediction mode of the current block, which is derived through the template-based intra prediction mode derivation method, may be used as an additional mode for a current Coding Unit (CU) block.

In the sequence parameter set, a flag indicating whether to use the template-based intra prediction derivation method may be signaled. In the case of using the template-based intra prediction derivation method, whether or not the template-based intra prediction derivation method is applied may be expressed using coding unit level flags at the coding unit level. When the current coding unit block uses the template-based intra prediction derivation method, the decoding apparatus may derive information about an intra prediction mode of the current coding unit block using the template-based intra prediction derivation method. Thus, signaling of syntax related to intra prediction modes for the remaining luma component may be omitted.

Referring to fig. 6, a template 610 used in the template-based intra prediction derivation method may be adjacent to the upper and left sides of the current block. The reference pixels 620 of the template to which the directionality of all candidate patterns in the MPM list is applied may exist adjacent to the template 610. By applying the directionality of all candidate patterns in the MPM list to the reference pixels 620 of the template, a prediction template may be generated.

Fig. 7 is a diagram illustrating templates and reference pixels of templates for deriving template-based intra prediction modes according to another embodiment of the present disclosure. Templates used in the template-based intra prediction mode derivation method may be randomly selected. Calculating the SATD between the predicted template pixel and the reconstructed template pixel generated by applying the directionality of all candidate patterns in the MPM list to the reference pixels of the template has a high complexity. The template used to derive the template-based intra-prediction mode may be variably selected depending on the directionality of the candidate modes in the MPM list. SATD between pixels of the selected template and pixels of the predicted template may be calculated. In addition to SATD, various error measurement methods, such as the sum of absolute differences or the sum of variances, may be used.

Referring to fig. 7, as an embodiment, templates used in the template-based intra prediction mode derivation method may correspond to the template (1), the template (2), and the template (3). The template (1) may be adjacent to the upper left of the current block. The template (2) may be adjacent to the upper part of the current block. The template (3) may be adjacent to the left side of the current block. The reference pixels of the template may be present adjacent to the template (1), the template (2) and the template (3). The shape and size of the templates may be randomly determined.

For example, if the candidate pattern selected from the MPM list is a vertical pattern, only the template (2) may be used. If the candidate pattern selected from the MPM list is a horizontal pattern, only the template (3) may be used. If the candidate pattern selected from the MPM list is a planar pattern or a DC pattern (which is a non-directional pattern), both the template (2) and the template (3) may be used.

For example, if the candidate pattern selected from the MPM list is a vertical pattern, only the template (3) may be used. If the candidate pattern selected from the MPM list is a horizontal pattern, only the template (2) may be used.

For example, if the candidate pattern selected from the MPM list is a vertical pattern, some pixels of the template (2) and the template (3) may be used. Here, some pixels of the template (3) may be randomly determined in consideration of the size of the current block or a randomly determined sub-sampling ratio. If the candidate pattern selected from the MPM list is a horizontal pattern, some pixels of the templates (3) and (2) may be used. Here, some pixels of the template (2) may be randomly determined in consideration of the size of the current block or a randomly determined sub-sampling ratio.

For example, if the candidate pattern selected from the MPM list is a vertical pattern, some pixels of the template (3) and the template (2) may be used. Here, some pixels of the template (2) may be randomly determined in consideration of the size of the current block or a randomly determined sub-sampling ratio. If the candidate pattern selected from the MPM list is a horizontal pattern, some pixels of the templates (2) and (3) may be used. Here, some pixels of the template (3) may be randomly determined in consideration of the size (size) of the current block or a randomly determined sub-sampling ratio.

For example, if the candidate pattern selected from the MPM list is a vertical pattern, the template (1) and the template (2) may be used. If the candidate pattern selected from the MPM list is a horizontal pattern, a template (1) and a template (3) may be used. If the candidate pattern selected from the MPM list is a planar pattern or a DC pattern (which is a non-directional pattern), the template (1), the template (2) and the template (3) may be used.

For example, if the candidate pattern selected from the MPM list is a vertical pattern, the template (1) and the template (3) may be used. If the candidate pattern selected from the MPM list is a horizontal pattern, a template (1) and a template (2) may be used.

For example, if the candidate pattern selected from the MPM list is a vertical pattern, some pixels of the template (1), the template (2), and the template (3) may be used. If the candidate pattern selected from the MPM list is a horizontal pattern, some pixels of template (1), template (3) and template (2) may be used. Here, some pixels of the template may be randomly determined in consideration of the size of the current block or a randomly determined sub-sampling ratio.

For example, if the candidate pattern selected from the MPM list is a vertical pattern, some pixels of the template (1), the template (3), and the template (2) may be used. If the candidate pattern selected from the MPM list is a horizontal pattern, some pixels of template (1), template (2) and template (3) may be used. Here, some pixels of the template may be randomly determined in consideration of the size of the current block or a randomly determined sub-sampling ratio.

In the above embodiment, the ranges of the vertical mode and the horizontal mode may be determined as specific ranges. Complexity may be reduced if a template used in the template-based intra prediction mode derivation method is variably selected according to a direction of a candidate mode in the MPM list.

Fig. 8 is a diagram illustrating a reference block for generating an MPM list according to an embodiment of the disclosure. Whether to use the template-based intra prediction mode derivation method may be determined based on the intra prediction modes of reference blocks adjacent to the current block.

Referring to fig. 8, a reference block a adjacent to an upper portion of a current block and a reference block L adjacent to a left side of the current block may be used to generate an MPM list. The intra prediction mode of the reference block a may refer to an a mode. The intra prediction mode of the reference block L may refer to an L mode. The a-mode and the L-mode may be used to determine whether to use a template-based intra prediction mode derivation method.

Referring to method 1, when the a mode and the L mode are the same, the probability that the neighboring block of the current block has one directional mode or one non-directional mode is high. In this case, the intra prediction mode of the current block may have a high probability of having the same or similar directional mode as the reference block a and the reference block L. The intra prediction mode of the current block can be efficiently encoded using the MPM list generated by the a mode and the L mode. Therefore, if the a-mode and the L-mode are the same, the template-based intra prediction mode derivation method may not be used.

Referring to method 2, the case where both the a mode and the L mode are non-directional modes includes a total of three cases: both the a mode and the L mode are planar modes; both the a mode and the L mode are DC modes; and the a mode and the L mode are a planar mode and a DC mode, respectively. If the intra prediction modes of reference blocks adjacent to the current block are all non-directional modes, there may be no directional information around the current block. If intra prediction modes of neighboring reference blocks around the current block are all non-directional modes, a specific default (default) mode may be used to configure the MPM list. Deriving intra-prediction modes based on templates may result in low coding efficiency when configuring the MPM list using default modes. Thus, when both the a-mode and the L-mode are non-directional modes, the template-based intra prediction mode derivation method may not be used.

In method 1, only when the a mode and the L mode are the same as the non-directional mode, the template-based intra prediction mode derivation method may not be used. In addition, only when the a mode and the L mode are the same as the directional mode, the template-based intra prediction mode derivation method may not be used. In addition, if the a mode and the L mode are the same without distinction between the directional mode and the non-directional mode, the template-based intra prediction mode derivation method may not be used. Method 1 and method 2 may be used independently or in combination. By applying only method 1, it can be determined whether to use the template-based intra prediction mode derivation method. By applying only method 2, it can be determined whether to use the template-based intra prediction mode derivation method. By combining method 1 and method 2, it can be determined whether to use the template-based intra prediction mode derivation method.

Fig. 9 is a diagram illustrating a method for generating an MPM list when all intra prediction modes of a reference block are non-directional modes according to an embodiment of the present disclosure. The candidate modes in the MPM list may be generated based on an intra prediction mode of a reference block adjacent to an upper portion of the current block and an intra prediction mode of a reference block adjacent to a left side of the current block. If both the intra prediction mode of the reference block adjacent to the upper portion of the current block and the intra prediction mode of the reference block adjacent to the left side of the current block are not the non-directional modes, the MPM list may be generated based on the intra prediction modes of the reference block having the direction information. In this case, the candidate mode in the MPM list may be determined as a mode similar to the intra prediction mode of the reference block. Considering the characteristics of the MPM list, an intra-prediction mode (which is a directional mode) of the reference block may be applied to reference pixels of the template to generate predicted template pixels. SATD between the predicted template pixels and the reconstructed template pixels may be calculated. It may be determined whether to apply the template-based intra prediction mode derivation method to each candidate mode in the MPM list based on the calculated SATD.

Referring to fig. 9, when intra prediction modes of reference blocks adjacent to a current block are all non-directional modes, an MPM list may be generated using a predefined default mode. The predefined default modes may include a vertical mode, a horizontal mode, a vertical-4 mode, and a horizontal +4 mode. The vertical-4 mode may represent a-4 mode based on the vertical mode. The horizontal +4 mode may represent a +4 mode based on the horizontal mode. The MPM list may include a planar mode, a DC mode, a vertical mode, a horizontal mode, a vertical-4 mode, and a horizontal +4 mode. The directionality of all candidate patterns in the MPM list may be applied to each reference pixel of the template to generate a predicted template pixel. SATD between the predicted template pixels and the reconstructed template pixels may be calculated. The candidate mode having the minimum value in the calculated SATD may be determined as an intra prediction mode of the current block.

Fig. 10 is a diagram illustrating a method for generating an MPM list when intra prediction modes of a reference block are a non-directional mode and a directional mode, respectively, according to an embodiment of the present disclosure. In fig. 8, a reference block a adjacent to an upper portion of a current block and a reference block L adjacent to a left side of the current block may be used to generate an MPM list. The intra prediction mode of the reference block a may refer to an a mode. The intra prediction mode of the reference block L may refer to an L mode. Calculating the template-based SATD of the candidate pattern may mean generating a predicted template pixel by applying directionality of the candidate pattern to a reference pixel of the template and calculating the SATD between the generated predicted template pixel and the reconstructed template pixel.

Referring to fig. 10, when the a mode and the L mode are the directional mode and the non-directional mode, respectively, or when the a mode and the L mode are the same directional mode, the MPM list may be generated based on the a mode (which is the directional mode). The MPM list may include a plane mode, an A-1 mode, an A+1 mode, an A-2 mode, and an A+2 mode. The A-1 mode, the A+1 mode, the A-2 mode, and the A+2 mode may represent a-1 mode, a +1 mode, a-2 mode, and a +2 mode, respectively, based on the A mode. Template-based SATD for a mode may be calculated. Based on the calculated values, it may be determined whether to calculate template-based SATD for the remaining candidate patterns in the MPM list. SATD _X Template-based SATD, which may represent any X-mode. Threshold value threshold_1 can beEach represents a randomly set Threshold (Threshold>0，Threshold_1>0). As an example, SATD _Planar Template-based SATD for planar modes may be represented. SATD ₅₀ Template-based SATD may represent a 50-mode.

Referring to method 1, template-based SATD for planar mode and a-mode may be calculated. The planar mode template-based SATD may be compared to the a-mode template-based SATD. If SATD _A ＜Threshold*SATD _Planar Template-based SATD for a-1 mode, a+1 mode, a-2 mode, and a+2 mode may be calculated. The mode having the minimum value in the calculated template-based SATD may be determined as an intra prediction mode of the current coding unit block.

Referring to method 2, template-based SATD of the planar mode, the A-1 mode, and the A+1 mode may be calculated. Template-based SATD for the a-1 mode may be compared to template-based SATD for the a+1 mode. If SATD _A-1 ＜Threshold*SATD _A+1 The template-based SATD for the a-2 mode may be calculated. If SATD _A-1 >Threshold*SATD _A+1 The template-based SATD for the a+2 mode may be calculated. If SATD _A-1 ＝Threshold*SATD _A+1 Template-based SATD for the a-2 mode and the a+2 mode may be calculated. The mode having the minimum value in the calculated template-based SATD may be determined as an intra prediction mode of the current coding unit block.

Referring to method 3, template-based SATD of the planar mode, the A-1 mode, and the A+1 mode may be calculated. Template-based SATD for a mode may be compared to template-based SATD for a-1 mode. Template-based SATD for a mode may be compared to template-based SATD for a+1 mode. If SATD _A-1 ＜Threshold*SATD _A The template-based SATD for the a-2 mode may be calculated. If SATD _A+1 ＜Threshold*SATD _A The template-based SATD for the a+2 mode may be calculated. The mode having the minimum value in the calculated template-based SATD may be determined as an intra prediction mode of the current coding unit block.

Referring to method 4, method 4 may correspond to a combined method 1 and method 2. Template-based SATD for planar mode and a-mode may be calculated. The planar mode template-based SATD may be compared to the a-mode template-based SATD. If SATD _A ＜Threshold*SATD _Planar Template-based SATD for the a-1 mode and the a+1 mode may be calculated. If SATD _A ＜Threshold* _SATDPlanar The a-1 mode template-based SATD may be compared to the a+1 mode template-based SATD. If SATD _A-1 ＜Threshold_1*SATD _A+1 The template-based SATD for the a-2 mode may be calculated. If SATD _A-1 >Threshold_1*SATD _A+1 The template-based SATD for the a+2 mode may be calculated. If SATD _A-1 ＝Threshold_1*SATD _A+1 Template-based SATD for the a-2 mode and the a+2 mode may be calculated. The mode having the minimum value in the calculated template-based SATD may be determined as an intra prediction mode of the current coding unit block.

Referring to method 5, method 5 may correspond to the method of combining methods 1 and 3. Template-based SATD for planar mode and a-mode may be calculated. The planar mode template-based SATD may be compared to the a-mode template-based SATD. If SATD _A ＜Threshold*SATD _Planar Template-based SATD for the a-1 mode and the a+1 mode may be calculated. If SATD _A ＜Threshold*SATD _Planar The template-based SATD for the a-mode may be compared to the template-based SATD for the a-1 mode. The a-mode template-based SATD may then be compared to the a+1-mode template-based SATD. If SATD _A-1 <Threshold_1*SATD _A The template-based SATD for the a-2 mode may be calculated. If SATD _A+1 <Threshold_1*SATD _A The template-based SATD for the a+2 mode may be calculated. The mode having the minimum value in the calculated template-based SATD may be determined as an intra prediction mode of the current coding unit block.

Fig. 11 is a diagram illustrating a method for generating an MPM list when intra prediction modes of reference blocks are different direction modes according to an embodiment of the present disclosure. In fig. 8, a reference block a adjacent to an upper portion of a current block and a reference block L adjacent to a left side of the current block may be used to generate an MPM list. The intra prediction mode of the reference block a may refer to an a mode. The intra prediction mode of the reference block L may refer to an L mode.

Referring to fig. 11, when the a mode and the L mode are different directional modes, an MPM list may be generated using each directional mode. In the MPM list, the first mode is a planar mode, the second mode is an L mode, the third mode is an a mode, the fourth mode is an l±1 mode or an a±1 mode, the fifth mode is an a±1 mode or an l±1 mode, and the sixth mode is an l±2 mode or an a±2 mode. The fourth, fifth and sixth modes are available from the a mode and the L mode. In this case, template-based SATD for the second mode (L-mode) and the third mode (a-mode) may be calculated. Based on the calculated template-based SATD, it may be determined whether to calculate the template-based SATD for a particular candidate pattern in the MPM list.

Referring to method 1, template-based SATD for planar mode, L-mode, and a-mode may be calculated. The planar mode template-based SATD may be compared to the L-mode template-based SATD. The planar mode template-based SATD may then be compared to the a-mode template-based SATD. If SATD _L ＜Threshold*SATD _Planar The sum of the template-based SATD of the patterns resulting from the L-pattern can be calculated. If SATD _A ＜Threshold*SATD _Planar Template-based SATD for the pattern resulting from the a-pattern may be calculated. The mode having the minimum value in the calculated template-based SATD may be determined as an intra prediction mode of the current coding unit block.

Referring to method 2, template-based SATD for planar mode, L-mode, and a-mode may be calculated. Template-based SATD for a mode may be compared to template-based SATD for L mode. If SATD _L ＜Threshold*SATD _A Template-based SATD for the pattern resulting from the L-pattern may be calculated. If SATD _L >Threshold*SATD _A Template-based SATD for the pattern resulting from the a-pattern may be calculated. If SATD _L ＝Threshold*SATD _A Then the modulo-based can be calculated for all remaining modes in the MPM listSATD of the plate. The mode having the minimum value in the calculated template-based SATD may be determined as an intra prediction mode of the current coding unit block.

Referring to method 3, template-based SATD for planar mode and L-mode may be calculated. The planar mode template-based SATD may be compared to the L-mode template-based SATD. If SATD _L ＜Threshold*SATD _Planar Template-based SATD for a mode may be calculated. If not SATD _L ＜Threshold*SATD _Planar Template-based SATD for the remaining candidate patterns in the MPM list may not be calculated. If SATD _A ＜Threshold* _SATDPlanar Template-based SATD for all remaining candidate patterns may be calculated. If not SATD _A ＜Threshold*SATD _Planar Template-based SATD for the remaining candidate patterns in the MPM list may not be calculated. The mode having the minimum value in the calculated template-based SATD may be determined as an intra prediction mode of the current coding unit block.

Template-based SATD for a-mode and L-mode, which are references for generating MPM lists, may be compared in different ways. Instead of calculating template-based SATD for all candidate patterns in the MPM list based on the comparison result, template-based SATD for a particular candidate pattern may be calculated. Thereby, complexity can be reduced.

Fig. 12 is a diagram illustrating neighboring blocks neighboring a current block according to an embodiment of the present disclosure. Calculating the template-based SATD of the intra prediction mode of the reference block may mean generating a prediction template by applying directionality of the intra prediction mode of the reference block to reference pixels of the template and calculating the SATD between the generated prediction template and pixels of the reconstructed template.

Referring to fig. 12, neighboring upper blocks a to H, left side blocks I to P, and upper left block Q may exist around the current block. Template-based SATD for intra prediction modes of the upper blocks a through H, left blocks I through P, and upper left block Q may be calculated. Here, the number and positions of the upper block, the left block, and the upper left block used may be randomly determined. The mode having the minimum value in the calculated template-based SATD may be determined as an intra prediction mode of the current block. Coding efficiency can be improved by using various intra prediction modes of a reference block adjacent to a current block. The template-based SATD may be calculated for only different intra prediction modes of reference blocks adjacent to the current block. In other words, the template-based SATD may be calculated for only the new intra prediction mode through redundancy check.

Referring to fig. 13, a mode histogram of a block adjacent to the current block may be configured. In the generated pattern histogram, the patterns may be ordered in descending order of the occurrence frequency of the patterns. Mode 1, mode 2, mode 3, mode 4, mode 5, mode 6, and mode 7 may represent any intra prediction mode. Mode 1, mode 2, mode 3, mode 4, mode 5, mode 6, and mode 7 may refer to intra prediction modes of blocks adjacent to the current block. If there are intra-prediction modes with the same frequency of occurrence, the intra-prediction modes may be ordered starting from the lowest number. Alternatively, the intra prediction modes may be ordered starting from the highest number.

Referring to fig. 14, SATD may be calculated from the histogram of the pattern generated in fig. 13 _mode ₁ (template-based SATD of mode 1 as the first index mode) (S1410). The index may be increased by 1 (S1420). Calculable SATD _next Which is the SATD of the next index pattern (S1430). SATD can be determined _mode _n+1 Whether or not less than Threshold SATD _mode _n (S1440). Here, SATD _mode _n+1 SATD, SATD that may correspond to mode 2 _mode _n May correspond to SATD for mode 1. SATD _mode _n+1 Can correspond to SATD _next ，SATD _next Is the SATD for the next index pattern. If not SATD _mode _n+1 <Threshold*SATD _mode _n (S1440-NO), then mouldThe equation n may be determined as an intra prediction mode of the current block (S1450). If SATD _mode _n+1 <Threshold*SATD _mode _n (S1440-yes), it is determined whether the pattern n+1 corresponds to the last index pattern (S1460). If the mode n+1 is the last index mode (S1460-Yes), the last index mode may be determined as an intra prediction mode of the current block (S1470). The last index pattern may correspond to pattern 7. If pattern n+1 is not the last index pattern (S1460-NO), the index may be incremented by 1. The SATD of the increased index pattern may be calculated.

In this way, the process of comparing template-based SATDs may be terminated when a template-based SATD for a pattern with a small index number is less than a template-based SATD for a pattern with a large index number. At this time, the mode having the small index number may be determined as an intra prediction mode for the current block. Alternatively, the process of comparing the template-based SATD may be terminated when the index number of the current comparison is the last index in the pattern histogram. By this processing, complexity can be reduced.

Mode allocation when deriving intra prediction modes of reference blocks based on template intra prediction mode derivation method Method

In case of deriving an intra prediction mode of a current block based on a template, an intra prediction mode of a reference block adjacent to the current block may be used. Here, if the intra prediction mode of the reference block is derived through the template-based intra prediction mode derivation method, the intra prediction mode of the reference block may be replaced with a specific mode. Here, the specific mode may correspond to a planar mode, a DC mode, or a specific direction mode.

Mode allocation in deriving intra prediction modes of luma blocks by template-based intra prediction mode derivation method Method

When the intra prediction mode of the chroma block is a Direct Mode (DM) mode, the intra prediction mode of the chroma block may be derived to the intra prediction mode of the corresponding luma block. Here, if the intra prediction mode of the corresponding luminance block is derived through the template-based intra prediction mode derivation method, the intra prediction mode of the corresponding luminance block may be replaced with a specific mode. Here, the specific mode may correspond to a planar mode, a DC mode, or a specific direction mode.

Self-adaptive searching method for SATD (software-as-a-service) based on template

The template-based intra prediction mode derivation method is a method of deriving an intra prediction mode of a current block based on template-based SATD for an intra prediction mode of a neighboring block of the current block or a candidate mode in an MPM list without information transmitted from an encoding apparatus. Thus, the specific directivity existing around the current block can be well reflected. In view of these characteristics, the following two methods are proposed.

Referring to method 1, an X-mode, which is a representative direction mode, can be derived by applying a Sobel (Sobel) operation (operator) to a template region. Template-based SATD for planar mode, DC mode, X-1 mode, x+1 mode, and X-2 mode may be calculated. Alternatively, template-based SATD for planar mode, DC mode, X-1 mode, X+1 mode, and X+2 mode may be calculated. Here, the mode having the minimum SATD may be determined as an intra prediction mode of the current block. To reduce the complexity of the sobel operation, the size of the template region may be variably adjusted. As the size of the template region is adjusted, the position of the reference template may also be variably adjusted.

Referring to method 2, when an intra prediction mode of a current block is determined using a template-based intra prediction mode derivation method, only an intra prediction mode of a block encoded in an intra prediction mode derived by the template-based intra prediction mode derivation method may be used among intra prediction modes of an MPM list or intra prediction modes of neighboring blocks around the current block.

Fig. 15 is a diagram illustrating 49 and 51 modes at resolution 64 of a vertical mode according to an embodiment of the present disclosure. Intra prediction modes may be derived using directional modes that are not present in existing intra prediction modes. The SATD of the candidate modes in the MPM list or the intra prediction modes of neighboring blocks around the current block may be calculated. A pattern having the minimum of the calculated values may be determined. If the determined mode is a non-directional mode, the corresponding mode may be determined as an intra prediction mode of the current block. If the corresponding pattern is a directional pattern, template-based SATD of + -1 patterns of the corresponding pattern may be calculated at a resolution different from the existing resolution 32. Based on the calculated value, the mode having the smallest value may be determined as an intra prediction mode of the current block.

Referring to fig. 15, SATD of candidate modes in the MPM list or intra prediction modes of neighboring blocks around the current block may be calculated. A pattern having the minimum of the calculated values may be determined. If the determined mode is mode 50, a template-based SATD for the 50+ -1 mode at resolution 64 may be calculated. At resolution 64, the mode of the smaller value of the template-based SATD of mode 49 and the template-based SATD of mode 51 may be determined as the intra prediction mode of the current block. The magnitude of the increase or decrease in resolution and mode may be determined randomly. Resolution 32 may represent dividing a particular segment into 32 equal segments. Modes 49 and 51 of resolution 64 do not exist in the intra prediction modes of resolution 32. At resolution 64, pattern 49 may exist between pattern 49 and pattern 50 at resolution 32. At resolution 64, pattern 51 may exist between pattern 50 and pattern 51 at resolution 32. In this way, template-based intra-prediction modes may be derived using intra-prediction modes that do not exist at resolution 32. Thus, the coding efficiency can be improved.

Referring to fig. 16, information (e.g., intra_time_flag) indicating whether to use the template-based intra prediction mode derivation method may be signaled. If the intra_time_flag is a first value (e.g., 0), the template-based intra prediction mode derivation method may not be used. If the intra_time_flag is a second value (e.g., 1), a template-based intra prediction mode derivation method may be used. If the intra_time_flag is a second value (e.g., 1) and the derived intra prediction mode is a non-directional mode, the corresponding non-directional mode may be determined as the intra prediction mode of the current block. If the intra_time_flag is a second value (e.g., 1) and the derived intra prediction mode is a directional mode, index information (e.g., intra_mode_idex) may be signaled. The intra prediction mode of the current block may be determined depending on the value of intra_mode_idex.

Fig. 17 is a diagram illustrating a method for determining a pattern based on the index of fig. 16 according to an embodiment of the present disclosure. The intra prediction mode derived at a predetermined resolution may be increased, maintained, or decreased based on the value of the index information in fig. 16.

Referring to fig. 17, if the predetermined resolution is 64, the intra prediction mode derived through the template-based intra prediction mode derivation method is mode 40, and intra_mode_idex is 0, the intra prediction mode of the current block may be mode 40 at resolution 64. If the predetermined resolution is 64, the intra prediction mode derived through the template-based intra prediction mode derivation method is mode 40, and intra_mode_idex is 10, the intra prediction mode of the current block may be mode 39 at resolution 64. If the predetermined resolution is 64, the intra prediction mode, which is derived through the template-based intra prediction mode derivation method, is mode 40, and intra_mode_idex is 11, the intra prediction mode of the current block may be mode 41 mode at resolution 64.

Referring to fig. 18, the decoding apparatus may obtain an MPM list based on intra prediction modes of neighboring blocks neighboring the current block (S1810). The decoding apparatus may generate a prediction pixel by applying a candidate mode in the MPM list or an intra prediction mode of a neighboring block to a reference pixel of a first region neighboring to the current block (S1820). The prediction pixels may be generated based on intra prediction modes of neighboring blocks. The first region may correspond to at least one of an upper region, a left region, and an upper left region of the current block. The first region may be determined based on candidate patterns in the MPM list. Generating the prediction pixels may include generating an intra-prediction mode list based on intra-prediction modes of neighboring blocks, and generating the prediction pixels based on the intra-prediction mode list. Alternatively, generating the predicted pixel may include: the second intra prediction mode is derived by applying a sobel operation to the first region, and a prediction pixel is generated based on the second intra prediction mode.

The decoding apparatus may calculate SATD between the predicted pixel and the reconstructed pixel of the first region (S1830). The decoding apparatus may derive a first intra prediction mode based on the SATD (S1840). The first intra-prediction mode may correspond to a candidate mode used to generate a predicted pixel having a minimum SATD. The first intra prediction mode may correspond to an intra prediction mode for generating a neighboring block of predicted pixels having a minimum SATD. The decoding apparatus may derive an intra prediction mode of the current block based on the first intra prediction mode (S1850). The operation of deriving the intra prediction mode of the current block based on the first intra prediction mode may correspond to an operation of deriving the first intra prediction mode as the intra prediction mode of the current block based on the first intra prediction mode being a non-directional mode. The operation of deriving the intra prediction mode of the current block based on the first intra prediction mode includes an operation of deriving a third intra prediction mode and a fourth intra prediction mode by increasing and decreasing the first intra prediction mode to a predetermined size at a predetermined resolution based on whether the first intra prediction mode is a directional mode, and an operation of deriving the intra prediction mode of the current block based on the third intra prediction mode and the fourth intra prediction mode. The decoding apparatus may generate a prediction block of the current block based on the intra prediction mode of the current block (S1860).

Referring to fig. 19, the encoding apparatus may determine an MPM list based on intra prediction modes of neighboring blocks neighboring the current block (S1910). The encoding apparatus may generate a prediction pixel by applying a candidate mode in the MPM list or an intra prediction mode of a neighboring block to a reference pixel of a first region neighboring the current block (S1920). The prediction pixels may be generated based on intra prediction modes of neighboring blocks. The first region may correspond to at least one of an upper region, a left region, and an upper left region of the current block. The first region may be determined based on candidate patterns in the MPM list. The operation of generating the predicted pixel may include: an intra prediction mode list is generated based on intra prediction modes of neighboring blocks, and a prediction pixel is generated based on the intra prediction mode list. Alternatively, the operation of generating the predicted pixel may include: a second intra-prediction mode is determined by applying a Sobel operation to the first region, and a prediction pixel is generated based on the second intra-prediction mode.

The encoding apparatus may calculate SATD between the predicted pixel and the reconstructed pixel of the first region (S1930). The encoding device may determine a first intra prediction mode based on the SATD (S1940). The first intra-prediction mode may correspond to a candidate mode used to generate a predicted pixel having a minimum SATD. The first intra prediction mode may correspond to an intra prediction mode of a neighboring block to generate a prediction pixel having a minimum SATD. The encoding apparatus may determine an intra prediction mode of the current block based on the first intra prediction mode (S1950). The operation of determining the intra prediction mode of the current block based on the first intra prediction mode may correspond to an operation of determining the first intra prediction mode as the intra prediction mode of the current block based on the first intra prediction mode being a non-directional mode. The determining of the intra prediction mode of the current block based on the first intra prediction mode may include: the method includes determining a third intra prediction mode and a fourth intra prediction mode by increasing and decreasing the first intra prediction mode to a predetermined size at a predetermined resolution based on the first intra prediction mode being a directional mode, and determining an intra prediction mode of the current block based on the third intra prediction mode and the fourth intra prediction mode. The encoding apparatus may generate a prediction block of the current block based on the intra prediction mode of the current block (S1960).

Although the steps in the various flowcharts are described as being performed sequentially, these steps merely exemplify the technical concepts of some embodiments of the present disclosure. Accordingly, one of ordinary skill in the art to which the present disclosure pertains may perform the steps by changing the order depicted in the various figures or by performing more than two steps in parallel. Therefore, the steps in the respective flowcharts are not limited to the time series order shown.

It should be understood that the above description presents illustrative embodiments that may be implemented in various other ways. The functionality described in some embodiments may be implemented by hardware, software, firmware, and/or combinations thereof. It should also be understood that the functional components described in this specification strongly emphasize their independent implementation possibilities with the "… unit".

Meanwhile, various methods or functions described in some embodiments may be implemented as instructions stored in a non-transitory recording medium that can be read and executed by one or more processors. For example, the non-transitory recording medium may include various types of recording devices in which data is stored in a form readable by a computer system. For example, the non-transitory recording medium may include a storage medium such as an erasable programmable read-only memory (EPROM), a flash memory drive, an optical disk drive, a magnetic hard disk drive, a Solid State Drive (SSD), and the like.

Although embodiments of the present disclosure have been described for illustrative purposes, those skilled in the art to which the present disclosure pertains will appreciate that various modifications, additions and substitutions are possible, without departing from the scope and spirit of the present disclosure. Accordingly, embodiments of the present disclosure have been described for brevity and clarity. The scope of the technical idea of the embodiments of the present disclosure is not limited by the drawings. Accordingly, it will be understood by those of ordinary skill in the art to which this disclosure pertains that the scope of this disclosure is not limited to the embodiments explicitly described above, but is limited by the claims and their equivalents.

Cross Reference to Related Applications

The present application claims priority from korean patent application No. 10-2021-0103366 filed on month 5 of 2021 and korean patent application No. 10-2022-0095867 filed on month 2 of 2022, the entire contents of which are incorporated herein by reference.

Claims

1. A video decoding method, comprising:

deriving a most probable mode MPM list based on intra prediction modes of neighboring blocks neighboring the current block;

generating a prediction pixel by applying a candidate pattern in an MPM list or an intra prediction pattern of the neighboring block to a reference pixel of a first region neighboring the current block;

Calculating a residual transform absolute value and SATD between the predicted and reconstructed pixels of the first region;

deriving a first intra-prediction mode based on the SATD;

deriving an intra prediction mode of the current block based on the first intra prediction mode; and

a prediction block of the current block is generated based on an intra prediction mode of the current block.

2. The video decoding method of claim 1, wherein the first region is at least one of an upper region, a left region, and an upper left region of the current block.

3. The video decoding method of claim 1, wherein the first region is determined based on candidate patterns in the MPM list.

4. The video decoding method of claim 1, wherein the first intra-prediction mode is a candidate mode for generating a prediction pixel having a minimum SATD.

5. The video decoding method of claim 1, wherein the first intra-prediction mode is an intra-prediction mode for generating a neighboring block of predicted pixels having a minimum SATD.

6. The video decoding method of claim 1, wherein the predicted pixels are generated based on the intra-prediction modes of the neighboring blocks.

7. The video decoding method of claim 1, wherein,

generating the predicted pixel includes:

generating an intra prediction mode list based on the intra prediction modes of the neighboring blocks; and

generating the predicted pixel based on the intra prediction mode list,

wherein the intra prediction mode list is generated using the frequency of occurrence of the intra prediction modes of the neighboring blocks.

8. The video decoding method of claim 1, wherein,

generating the predicted pixel includes:

deriving a second intra prediction mode by applying a sobel operation to the first region; and

the prediction pixel is generated based on the second intra prediction mode.

9. The video decoding method of claim 1, wherein,

deriving the intra-prediction mode of the current block based on the first intra-prediction mode includes:

deriving the first intra-prediction mode as the intra-prediction mode of the current block based on the first intra-prediction mode being a non-directional mode.

10. The video decoding method of claim 1, wherein,

Deriving a third intra prediction mode and a fourth intra prediction mode by increasing and decreasing the first intra prediction mode to a predetermined size at a predetermined resolution based on the first intra prediction mode being a directional mode; and

the intra prediction mode of the current block is derived based on the third intra prediction mode and the fourth intra prediction mode.

11. A video encoding method, comprising:

determining a most probable mode MPM list based on intra prediction modes of neighboring blocks neighboring the current block;

generating a prediction pixel by applying a candidate pattern in the MPM list or an intra prediction pattern of the neighboring block to a reference pixel of a first region neighboring the current block;

determining a first intra prediction mode based on the SATD;

determining an intra prediction mode of the current block based on the first intra prediction mode; and

a prediction block of the current block is generated based on the intra prediction mode of the current block.

12. The video encoding method of claim 11, wherein the first region is at least one of an upper region, a left region, and an upper left region of the current block.

13. The video encoding method of claim 11, wherein the first region is determined based on candidate patterns in the MPM list.

14. The video encoding method of claim 11, wherein the first intra-prediction mode is a candidate mode for generating a prediction pixel having a minimum SATD.

15. The video encoding method of claim 11, wherein the first intra prediction mode is an intra prediction mode for generating a neighboring block of predicted pixels having a minimum SATD.

16. The video encoding method of claim 11, wherein,

generating the predicted pixel includes:

generating the predicted pixel based on the intra prediction mode list,

17. The video encoding method of claim 11, wherein,

generating the predicted pixel includes:

the prediction pixel is generated based on the second intra prediction mode.

18. The video encoding method of claim 11, wherein,

determining the intra-prediction mode of the current block based on the first intra-prediction mode includes:

the method further includes determining the first intra-prediction mode as the intra-prediction mode of the current block based on the first intra-prediction mode being a non-directional mode.

19. The video encoding method of claim 11, wherein,

determining a third intra-prediction mode and a fourth intra-prediction mode by increasing and decreasing the first intra-prediction mode to a predetermined size at a predetermined resolution based on the first intra-prediction mode being a directional mode; and

the intra prediction mode of the current block is determined based on the third intra prediction mode and the fourth intra prediction mode.

20. A recording medium as a computer-readable recording medium storing a bit stream generated by a video encoding method, wherein the video encoding method comprises:

based on intra prediction modes of neighboring blocks neighboring the current block, deriving a most probable mode (PM list;

Generating a prediction pixel by applying a candidate mode in an MPM list or an intra prediction mode of a neighboring block to a reference pixel of a first region neighboring the current block;

deriving a first intra-prediction mode based on the SATD;