CN116671108A

CN116671108A - Method and apparatus for encoding and decoding video using selective sub-block partition information transmission

Info

Publication number: CN116671108A
Application number: CN202180076528.5A
Authority: CN
Inventors: 全炳宇; 朴智允; 金范允; 朴胜煜
Original assignee: Sungkyunkwan University School Industry Cooperation; Hyundai Motor Co; Kia Corp
Current assignee: Sungkyunkwan University School Industry Cooperation; Hyundai Motor Co; Kia Corp
Priority date: 2020-11-23
Filing date: 2021-11-23
Publication date: 2023-08-29
Also published as: KR20220071130A

Abstract

Methods and apparatus for encoding and decoding video using selective sub-block partition information transmission are disclosed. The present embodiment provides an image encoding/decoding method and apparatus that selectively encodes and decodes a partition direction of a sub-block to reduce a transmission burden of a signal using sub-block partition while efficiently performing intra prediction of a sub-block unit.

Description

Method and apparatus for encoding and decoding video using selective sub-block partition information transmission

Technical Field

The present invention relates to a video encoding/decoding method and apparatus using selective transmission of sub-block partition information.

Background

The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.

Since video data has a larger data amount than audio data or still image data, the video data requires a large amount of hardware resources (including a memory) to store or transmit the video data that is not subjected to compression processing.

Accordingly, encoders are typically used to compress and store or transmit video data. The decoder receives the compressed video data, decompresses the received compressed video data, and plays the decompressed video data. Video compression techniques include h.264/AVC, high efficiency video coding (High Efficiency Video Coding, HEVC), and multi-function video coding (Versatile Video Coding, VVC) that improves the coding efficiency of HEVC by about 30% or more.

However, as the image size, resolution, and frame rate gradually increase, the amount of data to be encoded is also increasing. Accordingly, a new compression technique providing higher coding efficiency and improved image enhancement effect compared to the existing compression technique is required.

In video (image) Coding, when an image is divided on each Coding Unit (CU) and coded on each CU, all pixels in a block to be coded are intra-predicted using one prediction mode. Since the distance between the pixel and the reference pixel may become further, a lot of energy may remain in the residual signal to be encoded. For long rectangular blocks of horizontal (or vertical) where the distance between the pixel to be predicted and the reference pixel is longer, or when the size of the block is larger, the residual energy problem in the residual signal may become more serious. The blocks may be further partitioned to solve the problem, but this causes another problem of increasing the burden for transmitting the intra prediction mode for each sub-divided block.

On the other hand, there is another solution to the problem of increasing the burden. The prior art performs prediction by dividing a block to be re-encoded into uniformly divided smaller blocks to reduce the burden while improving intra prediction efficiency, but transmits only a single prediction mode on each original block before sub-partitioning, and generally applies the single prediction mode to the small blocks of the sub-partition. The above background is referred to as Intra Sub-Partition (ISP) technology.

When the ISP is applied to the intra prediction of the current block, the video encoding and decoding apparatus may signal one intra prediction mode while predicting the block of the sub-partition using reference pixel values of the block close to the corresponding sub-partition. On the other hand, a problem that arises when the ISP technology is applied is that information indicating the sub-partition is always transmitted together with information indicating the ISP mode. Therefore, a method for efficiently encoding information indicating a sub-partition is required in terms of encoding efficiency.

Disclosure of Invention

Technical problem

The present invention in some embodiments seeks to provide video encoding/decoding methods and apparatus for selectively encoding and decoding the partitioning direction of sub-blocks. The video encoding/decoding method and apparatus alleviate the burden of signal transmission using sub-block partitioning while efficiently performing intra prediction on a per sub-block basis.

Solution method

At least one aspect of the present invention provides an intra prediction method performed by a video decoding device that applies an intra prediction mode of a current block to a sub-block obtained by dividing the current block. The method includes decoding a size of a current block and an intra prediction mode. The method further includes generating a pre-trimmed range of sub-blocks based on the size of the current block and the number of sub-blocks of the partition. Here, the pre-trimming range includes a vertical pre-trimming range and a horizontal pre-trimming range, and represents a set of intra prediction directions in which a sub-block does not use a constructed sample of a newly reconstructed neighboring sub-block when performing prediction. The method further includes setting a division direction of the sub-block according to whether the intra prediction mode is included in the vertical pre-trimming range or the horizontal pre-trimming range.

Another aspect of the present invention provides a video decoding device that applies an intra prediction mode of a current block to a sub-block obtained by dividing the current block. The apparatus includes an entropy decoder configured to decode a size of a current block and an intra prediction mode. The apparatus further includes a pre-trimming range generator configured to generate a pre-trimming range of the sub-block based on the size of the current block. Here, the pre-trimming range includes a vertical pre-trimming range and a horizontal pre-trimming range, and represents a set of intra prediction directions in which a sub-block does not use a constructed sample of a newly reconstructed neighboring sub-block when performing prediction. The apparatus further includes an intra predictor configured to set a division direction of the sub-block according to whether the intra prediction mode is included in the vertical pre-trimming range or the horizontal pre-trimming range.

Yet another aspect of the present invention provides an intra prediction method performed by a video encoding device, which applies an intra prediction mode of a current block to a sub-block obtained by dividing the current block. The method includes obtaining a size of a current block and an intra prediction mode. The method further includes generating a pre-trimmed range of sub-blocks based on the size of the current block. Here, the pre-trimming range includes a vertical pre-trimming range and a horizontal pre-trimming range, and represents a set of intra prediction directions in which a sub-block does not use a constructed sample of a newly reconstructed neighboring sub-block when performing prediction. The method further includes setting a division direction of the sub-block according to whether the intra prediction mode is included in the vertical pre-trimming range or the horizontal pre-trimming range.

Effects of the invention

As described above, the present invention provides a video encoding/decoding method and apparatus for selectively encoding and decoding a partition direction of sub-blocks. Accordingly, the video encoding/decoding method and apparatus can alleviate the burden of signal transmission using sub-block partitions while efficiently performing intra prediction on a per sub-block basis.

Drawings

Fig. 1 is a block diagram of a video encoding device in which the techniques of the present invention may be implemented.

Fig. 2 illustrates a method of partitioning a block using a quadtree plus binary tree trigeminal tree (QTBTTT) structure.

Fig. 3a and 3b illustrate a plurality of intra prediction modes including a wide-angle intra prediction mode.

Fig. 4 shows neighboring blocks of the current block.

Fig. 5 is a block diagram of a video decoding apparatus in which the techniques of the present invention may be implemented.

Fig. 6 shows a current block and sub-blocks of a sub-partition.

Fig. 7a to 7c illustrate sub-blocks not partitioned, horizontally partitioned and vertically partitioned.

Fig. 8a to 8f illustrate a method for booting pre_print_range according to an embodiment of the present invention.

Fig. 9a and 9b show pre_prunable_range_ver and pre_prunable_range_hor according to an embodiment of the present invention.

Fig. 10 is a block diagram conceptually illustrating a pre-trim range generator according to one embodiment of the present invention.

Fig. 11 is a flowchart illustrating an intra prediction method including selective encoding of a sub-block division direction according to an embodiment of the present invention.

Fig. 12 is a flowchart illustrating an intra prediction method including selective decoding of a sub-block division direction according to an embodiment of the present invention.

Detailed Description

Hereinafter, some embodiments of the present invention will be described in detail with reference to the accompanying illustrative drawings. In the following description, like reference numerals denote like elements, although the elements are shown in different drawings. Furthermore, in the following description of some embodiments, detailed descriptions of related known components and functions have been omitted for clarity and conciseness when it may be considered that the subject matter of the present invention is obscured.

Fig. 1 is a block diagram of a video encoding device in which the techniques of the present invention may be implemented. Hereinafter, a video encoding apparatus and sub-components of the apparatus are described with reference to the illustration of fig. 1.

The encoding apparatus may include: an image divider 110, a predictor 120, a subtractor 130, a transformer 140, a quantizer 145, a reordering unit 150, an entropy encoder 155, an inverse quantizer 160, an inverse transformer 165, an adder 170, a loop filtering unit 180, and a memory 190.

Each component of the encoding apparatus may be implemented as hardware or software, or as a combination of hardware and software. In addition, the function of each component may be implemented as software, and the microprocessor may also be implemented to execute the function of the software corresponding to each component.

A video is made up of one or more sequences comprising a plurality of images. Each image is divided into a plurality of regions, and encoding is performed on each region. For example, an image is segmented into one or more tiles (tiles) or/and slices (slices). Here, one or more tiles may be defined as a tile set. Each tile or/and slice is partitioned into one or more Coding Tree Units (CTUs). In addition, each CTU is partitioned into one or more Coding Units (CUs) by a tree structure. Information applied to each CU is encoded as a syntax of the CU, and information commonly applied to CUs included in one CTU is encoded as a syntax of the CTU. In addition, information commonly applied to all blocks in one slice is encoded as syntax of a slice header, and information applied to all blocks constituting one or more pictures is encoded as a picture parameter set (Picture Parameter Set, PPS) or a picture header. Furthermore, information commonly referred to by the plurality of images is encoded as a sequence parameter set (Sequence Parameter Set, SPS). In addition, information commonly referenced by the one or more SPS is encoded as a set of video parameters (Video Parameter Set, VPS). Furthermore, information commonly applied to one tile or group of tiles may also be encoded as syntax of the tile or group of tiles header. The syntax included in the SPS, PPS, slice header, tile, or tile set header may be referred to as a high level syntax.

The image divider 110 determines the size of the CTU. Information about the size of the CTU (CTU size) is encoded as a syntax of the SPS or PPS and transmitted to the video decoding apparatus.

The image divider 110 divides each image constituting a video into a plurality of CTUs having a predetermined size, and then recursively divides the CTUs by using a tree structure. Leaf nodes in the tree structure become CUs, which are the basic units of coding.

The tree structure may be a Quadtree (QT) in which a higher node (or parent node) is partitioned into four lower nodes (or child nodes) of the same size. The tree structure may also be a Binary Tree (BT) in which a higher node is split into two lower nodes. The tree structure may also be a Trigeminal Tree (TT), where the higher nodes are split into three lower nodes at a ratio of 1:2:1. The tree structure may also be a structure in which two or more of a QT structure, a BT structure, and a TT structure are mixed. For example, a quadtree plus binary tree (quadtree plus binarytree, QTBT) structure may be used, or a quadtree plus binary tree (quadtree plus binarytree ternarytree, QTBTTT) structure may be used. Here, BTTT is added to the tree structure to be called multiple-type tree (MTT).

Fig. 2 is a schematic diagram for describing a method of dividing a block by using the QTBTTT structure.

As shown in fig. 2, the CTU may be first partitioned into QT structures. Quadtree partitioning may be recursive until the size of the partitioned block reaches the minimum block size (MinQTSize) of leaf nodes allowed in QT. A first flag (qt_split_flag) indicating whether each node of the QT structure is partitioned into four lower-layer nodes is encoded by the entropy encoder 155 and signaled to the video decoding apparatus. When the leaf node of QT is not greater than the maximum block size (MaxBTSize) of the root node allowed in BT, the leaf node may be further divided into at least one of BT structure or TT structure. There may be multiple directions of segmentation in the BT structure and/or the TT structure. For example, there may be two directions, i.e., a direction of dividing the block of the corresponding node horizontally and a direction of dividing the block of the corresponding node vertically. As shown in fig. 2, when the MTT division starts, a second flag (MTT _split_flag) indicating whether a node is divided, and a flag additionally indicating a division direction (vertical or horizontal) and/or a flag indicating a division type (binary or trigeminal) in the case that a node is divided are encoded by the entropy encoder 155 and signaled to the video decoding apparatus.

Alternatively, a CU partition flag (split_cu_flag) indicating whether a node is partitioned may be further encoded before encoding a first flag (qt_split_flag) indicating whether each node is partitioned into four nodes of a lower layer. When the value of the CU partition flag (split_cu_flag) indicates that each node is not partitioned, the block of the corresponding node becomes a leaf node in the partition tree structure and becomes a CU, which is a basic unit of encoding. When the value of the CU partition flag (split_cu_flag) indicates that each node is partitioned, the video encoding apparatus first starts encoding the first flag in the above scheme.

When QTBT is used as another example of the tree structure, there may be two types, i.e., a type of horizontally dividing a block of a corresponding node into two blocks having the same size (i.e., symmetrical horizontal division) and a type of vertically dividing a block of a corresponding node into two blocks having the same size (i.e., symmetrical vertical division). A partition flag (split_flag) indicating whether each node of the BT structure is partitioned into lower-layer blocks and partition type information indicating a partition type are encoded by the entropy encoder 155 and transmitted to the video decoding apparatus. On the other hand, there may additionally be a type in which a block of a corresponding node is divided into two blocks in an asymmetric form to each other. The asymmetric form may include a form in which a block of a corresponding node is divided into two rectangular blocks having a size ratio of 1:3, or may also include a form in which a block of a corresponding node is divided in a diagonal direction.

A CU may have various sizes according to QTBT or QTBTTT divided from CTUs. Hereinafter, a block corresponding to a CU to be encoded or decoded (i.e., a leaf node of QTBTTT) is referred to as a "current block". When QTBTTT segmentation is employed, the shape of the current block may also be rectangular in shape, in addition to square shape.

The predictor 120 predicts the current block to generate a predicted block. Predictor 120 includes an intra predictor 122 and an inter predictor 124.

In general, each of the current blocks in the image may be predictively encoded. In general, prediction of a current block may be performed by using an intra prediction technique using data from an image including the current block or an inter prediction technique using data from an image encoded before the image including the current block. Inter prediction includes both unidirectional prediction and bi-directional prediction.

The intra predictor 122 predicts pixels in the current block by using pixels (reference pixels) located adjacent to the current block in the current image including the current block. Depending on the prediction direction, there are multiple intra prediction modes. For example, as shown in fig. 3a, the plurality of intra prediction modes may include 2 non-directional modes including a planar (planar) mode and a DC mode, and may include 65 directional modes. The neighboring pixels and algorithm equations to be used are defined differently according to each prediction mode.

For efficient direction prediction of a current block having a rectangular shape, direction modes (# 67 to # 80) indicated by dotted arrows in fig. 3b, intra prediction modes # -1 to # -14) may be additionally used. The direction mode may be referred to as a "wide angle intra-prediction mode". In fig. 3b, the arrows indicate the respective reference samples for prediction, rather than representing the prediction direction. The prediction direction is opposite to the direction indicated by the arrow. When the current block has a rectangular shape, the wide-angle intra prediction mode is a mode in which prediction is performed in a direction opposite to a specific direction mode without additional bit transmission. In this case, in the wide-angle intra prediction mode, some of the wide-angle intra prediction modes available for the current block may be determined by a ratio of a width to a height of the current block having a rectangular shape. For example, when the current block has a rectangular shape having a height smaller than a width, wide-angle intra prediction modes (intra prediction modes #67 to # 80) having angles smaller than 45 degrees are available. When the current block has a rectangular shape with a width greater than a height, a wide-angle intra prediction mode having an angle greater than-135 degrees is available.

The intra predictor 122 may determine intra prediction to be used for encoding the current block. In some examples, intra predictor 122 may encode the current block by utilizing a plurality of intra prediction modes, and may also select an appropriate intra prediction mode to use from among the test modes. For example, the intra predictor 122 may calculate a rate distortion value by using rate-distortion (rate-distortion) analysis of a plurality of tested intra prediction modes, and may also select an intra prediction mode having the best rate distortion characteristics among the test modes.

The intra predictor 122 selects one intra prediction mode among a plurality of intra prediction modes, and predicts the current block by using neighboring pixels (reference pixels) determined according to the selected intra prediction mode and an algorithm equation. Information about the selected intra prediction mode is encoded by the entropy encoder 155 and transmitted to a video decoding device.

The inter predictor 124 generates a prediction block of the current block by using a motion compensation process. The inter predictor 124 searches for a block most similar to the current block in a reference picture that has been encoded and decoded earlier than the current picture, and generates a predicted block of the current block by using the searched block. In addition, a Motion Vector (MV) is generated, which corresponds to a displacement (displacement) between a current block in the current image and a prediction block in the reference image. In general, motion estimation is performed on a luminance (luma) component, and a motion vector calculated based on the luminance component is used for both the luminance component and the chrominance component. Motion information including information of the reference picture and information on a motion vector for predicting the current block is encoded by the entropy encoder 155 and transmitted to a video decoding device.

The inter predictor 124 may also perform interpolation of reference pictures or reference blocks to increase the accuracy of prediction. In other words, the sub-samples are interpolated between two consecutive integer samples by applying the filter coefficients to a plurality of consecutive integer samples comprising the two integer samples. When the process of searching for a block most similar to the current block is performed on the interpolated reference image, the decimal-unit precision may be represented for the motion vector instead of the integer-sample-unit precision. The precision or resolution of the motion vector may be set differently for each target region to be encoded, e.g., a unit such as a slice, tile, CTU, CU, etc. When such adaptive motion vector resolution (adaptive motion vector resolution, AMVR) is applied, information on the motion vector resolution to be applied to each target area should be signaled for each target area. For example, when the target area is a CU, information about the resolution of a motion vector applied to each CU is signaled. The information on the resolution of the motion vector may be information representing the accuracy of a motion vector difference to be described below.

On the other hand, the inter predictor 124 may perform inter prediction by using bi-directional prediction. In the case of bi-prediction, two reference pictures and two motion vectors representing block positions most similar to the current block in each reference picture are used. The inter predictor 124 selects a first reference picture and a second reference picture from the reference picture list0 (RefPicList 0) and the reference picture list1 (RefPicList 1), respectively. The inter predictor 124 also searches for a block most similar to the current block in the corresponding reference picture to generate a first reference block and a second reference block. Further, a prediction block of the current block is generated by averaging or weighted-averaging the first reference block and the second reference block. Further, motion information including information on two reference pictures for predicting the current block and information on two motion vectors is transmitted to the entropy encoder 155. Here, the reference image list0 may be constituted by an image preceding the current image in display order among the pre-restored images, and the reference image list1 may be constituted by an image following the current image in display order among the pre-restored images. However, although not particularly limited thereto, a pre-restored image following the current image in the display order may be additionally included in the reference image list 0. Conversely, a pre-restored image preceding the current image may be additionally included in the reference image list 1.

In order to minimize the amount of bits consumed for encoding motion information, various methods may be used.

For example, when a reference image and a motion vector of a current block are identical to those of a neighboring block, information capable of identifying the neighboring block is encoded to transmit motion information of the current block to a video decoding apparatus. This method is called merge mode (merge mode).

In the merge mode, the inter predictor 124 selects a predetermined number of merge candidate blocks (hereinafter, referred to as "merge candidates") from neighboring blocks of the current block.

As the neighboring blocks used to derive the merge candidates, all or some of the left block A0, the lower left block A1, the upper block B0, the upper right block B1, and the upper left block B2 adjacent to the current block in the current image may be used, as shown in fig. 4. In addition, in addition to the current picture in which the current block is located, a block located within a reference picture (which may be the same as or different from the reference picture used to predict the current block) may also be used as a merging candidate. For example, a co-located block (co-located block) of a current block within a reference picture or a block adjacent to the co-located block may additionally be used as a merging candidate. If the number of merging candidates selected by the above method is less than a preset number, a zero vector is added to the merging candidates.

The inter predictor 124 configures a merge list including a predetermined number of merge candidates by using neighboring blocks. A merge candidate to be used as motion information of the current block is selected from among the merge candidates included in the merge list, and merge index information for identifying the selected candidate is generated. The generated merging index information is encoded by the entropy encoder 155 and transmitted to a video decoding apparatus.

The merge skip mode is a special case of the merge mode. After quantization, when all transform coefficients used for entropy coding are near zero, only neighboring block selection information is transmitted without transmitting a residual signal. By using the merge skip mode, relatively high encoding efficiency can be achieved for images with slight motion, still images, screen content images, and the like.

Hereinafter, the merge mode and the merge skip mode are collectively referred to as a merge/skip mode.

Another method for encoding motion information is advanced motion vector prediction (advanced motion vector prediction, AMVP) mode.

In the AMVP mode, the inter predictor 124 derives a motion vector prediction candidate for a motion vector of a current block by using neighboring blocks of the current block. As the neighboring blocks used to derive the motion vector prediction candidates, all or some of the left block A0, the lower left block A1, the upper side block B0, the upper right block B1, and the upper left block B2 adjacent to the current block in the current image shown in fig. 4 may be used. In addition, in addition to the current picture in which the current block is located, a block located within a reference picture (which may be the same as or different from a reference picture used to predict the current block) may also be used as a neighboring block used to derive a motion vector prediction candidate. For example, a co-located block of the current block within the reference picture or a block adjacent to the co-located block may be used. If the number of motion vector candidates selected by the above method is less than a preset number, a zero vector is added to the motion vector candidates.

The inter predictor 124 derives a motion vector prediction candidate by using the motion vector of the neighboring block, and determines a motion vector prediction of the motion vector of the current block by using the motion vector prediction candidate. In addition, a motion vector difference is calculated by subtracting a motion vector prediction from a motion vector of the current block.

Motion vector prediction may be obtained by applying a predefined function (e.g., median and average calculations, etc.) to the motion vector prediction candidates. In this case, the video decoding device is also aware of the predefined function. Further, since the neighboring block used to derive the motion vector prediction candidates is a block for which encoding and decoding have been completed, the video decoding apparatus may also already know the motion vector of the neighboring block. Therefore, the video encoding device does not need to encode information for identifying motion vector prediction candidates. Accordingly, in this case, information on a motion vector difference and information on a reference image for predicting a current block are encoded.

On the other hand, motion vector prediction may also be determined by selecting a scheme of any one of the motion vector prediction candidates. In this case, the information for identifying the selected motion vector prediction candidates is additionally encoded together with the information about the motion vector difference and the information about the reference picture for predicting the current block.

The subtractor 130 generates a residual block by subtracting the current block from the prediction block generated by the intra predictor 122 or the inter predictor 124.

The transformer 140 transforms a residual signal in a residual block having pixel values of a spatial domain into transform coefficients of a frequency domain. The transformer 140 may transform a residual signal in a residual block by using the entire size of the residual block as a transform unit, or may divide the residual block into a plurality of sub-blocks and perform the transform by using the sub-blocks as transform units. Alternatively, the residual block is divided into two sub-blocks, i.e., a transform region and a non-transform region, to transform the residual signal by using only the transform region sub-block as a transform unit. Here, the transform region sub-block may be one of two rectangular blocks having a size ratio of 1:1 based on a horizontal axis (or a vertical axis). In this case, a flag (cu_sbt_flag) indicating that only the sub-block is transformed, and direction (vertical/horizontal) information (cu_sbt_horizontal_flag) and/or position information (cu_sbt_pos_flag) are encoded by the entropy encoder 155 and signaled to the video decoding apparatus. In addition, the size of the transform region sub-block may have a size ratio of 1:3 based on the horizontal axis (or vertical axis). In this case, a flag (cu_sbt_quad_flag) dividing the corresponding division is additionally encoded by the entropy encoder 155 and signaled to the video decoding device.

On the other hand, the transformer 140 may perform transformation of the residual block separately in the horizontal direction and the vertical direction. For this transformation, various types of transformation functions or transformation matrices may be used. For example, the pair-wise transformation function for horizontal and vertical transformations may be defined as a transformation set (multiple transform set, MTS). The transformer 140 may select one transform function pair having the highest transform efficiency in the MTS and transform the residual block in each of the horizontal and vertical directions. Information (mts_idx) about the transform function pairs in the MTS is encoded by the entropy encoder 155 and signaled to the video decoding means.

The quantizer 145 quantizes the transform coefficient output from the transformer 140 using a quantization parameter, and outputs the quantized transform coefficient to the entropy encoder 155. The quantizer 145 may also immediately quantize the relevant residual block without transforming any block or frame. The quantizer 145 may also apply different quantization coefficients (scaling values) according to the positions of the transform coefficients in the transform block. A quantization matrix applied to quantized transform coefficients arranged in two dimensions may be encoded and signaled to a video decoding apparatus.

The reordering unit 150 may perform the rearrangement of the coefficient values on the quantized residual values.

The rearrangement unit 150 may change the 2D coefficient array to a 1D coefficient sequence by using coefficient scanning. For example, the rearrangement unit 150 may scan the DC coefficients to the high frequency region coefficients using zigzag scanning (zig-zag scan) or diagonal scanning (diagonal scan) to output a 1D coefficient sequence. Instead of the zig-zag scan, a vertical scan that scans the 2D coefficient array in the column direction and a horizontal scan that scans the 2D block type coefficients in the row direction may also be utilized, depending on the size of the transform unit and the intra prediction mode. In other words, the scanning method to be used may be determined in zigzag scanning, diagonal scanning, vertical scanning, and horizontal scanning according to the size of the transform unit and the intra prediction mode.

The entropy encoder 155 encodes the sequence of the 1D quantized transform coefficients output from the rearrangement unit 150 by using various encoding schemes including Context-based adaptive binary arithmetic coding (Context-based Adaptive Binary Arithmetic Code, CABAC), exponential golomb (Exponential Golomb), and the like to generate a bitstream.

Further, the entropy encoder 155 encodes information related to block division (e.g., CTU size, CTU division flag, QT division flag, MTT division type, MTT division direction, etc.) so that the video decoding apparatus can divide blocks equally to the video encoding apparatus. Further, the entropy encoder 155 encodes information on a prediction type indicating whether the current block is encoded by intra prediction or inter prediction. The entropy encoder 155 encodes intra prediction information (i.e., information about an intra prediction mode) or inter prediction information (a merge index in the case of a merge mode, and information about a reference picture index and a motion vector difference in the case of an AMVP mode) according to a prediction type. Further, the entropy encoder 155 encodes information related to quantization (i.e., information about quantization parameters and information about quantization matrices).

The inverse quantizer 160 inversely quantizes the quantized transform coefficient output from the quantizer 145 to generate a transform coefficient. The inverse transformer 165 transforms the transform coefficients output from the inverse quantizer 160 from the frequency domain to the spatial domain to restore a residual block.

The adder 170 adds the restored residual block and the prediction block generated by the predictor 120 to restore the current block. The pixels in the restored current block are used as reference pixels when intra-predicting the next block.

The loop filtering unit 180 performs filtering on the restored pixels to reduce block artifacts (blocking artifacts), ringing artifacts (ringing artifacts), blurring artifacts (blurring artifacts), etc., which occur due to block-based prediction and transform/quantization. The loop filtering unit 180 as an in-loop filter may include all or some of a deblocking filter 182, a sample adaptive offset (sample adaptive offset, SAO) filter 184, and an adaptive loop filter (adaptive loop filter, ALF) 186.

Deblocking filter 182 filters boundaries between restored blocks to remove block artifacts (blocking artifacts) that occur due to block unit encoding/decoding, and SAO filter 184 and ALF 186 additionally filter the deblock filtered video. The SAO filter 184 and ALF 186 are filters for compensating for differences between restored pixels and original pixels that occur due to lossy coding (loss coding). The SAO filter 184 applies an offset as a CTU unit to enhance subjective image quality and coding efficiency. In contrast, the ALF 186 performs block unit filtering, and applies different filters to compensate for distortion by dividing boundaries of respective blocks and the degree of variation. Information about filter coefficients to be used for ALF may be encoded and signaled to the video decoding apparatus.

The restored blocks filtered by the deblocking filter 182, the SAO filter 184, and the ALF 186 are stored in the memory 190. When all blocks in one image are restored, the restored image may be used as a reference image for inter-predicting blocks within a picture to be subsequently encoded.

Fig. 5 is a functional block diagram of a video decoding apparatus in which the techniques of the present invention may be implemented. Hereinafter, with reference to fig. 5, a video decoding apparatus and sub-components of the apparatus are described.

The video decoding apparatus may be configured to include an entropy decoder 510, a reordering unit 515, an inverse quantizer 520, an inverse transformer 530, a predictor 540, an adder 550, a loop filtering unit 560, and a memory 570.

Similar to the video encoding apparatus of fig. 1, each component of the video decoding apparatus may be implemented as hardware or software, or as a combination of hardware and software. In addition, the function of each component may be implemented as software, and the microprocessor may also be implemented to execute the function of the software corresponding to each component.

The entropy decoder 510 extracts information related to block segmentation by decoding a bitstream generated by a video encoding apparatus to determine a current block to be decoded, and extracts prediction information required to restore the current block and information on a residual signal.

The entropy decoder 510 determines the size of CTUs by extracting information about the CTU size from a Sequence Parameter Set (SPS) or a Picture Parameter Set (PPS), and partitions a picture into CTUs having the determined size. Further, the CTU is determined as the highest layer (i.e., root node) of the tree structure, and the partition information of the CTU is extracted to partition the CTU by using the tree structure.

For example, when dividing a CTU by using the QTBTTT structure, first a first flag (qt_split_flag) related to the division of QT is extracted to divide each node into four nodes of the lower layer. In addition, a second flag (mtt_split_flag), a split direction (vertical/horizontal), and/or a split type (binary/trigeminal) related to the split of the MTT are extracted with respect to a node corresponding to the leaf node of the QT to split the corresponding leaf node into an MTT structure. As a result, each node below the leaf node of QT is recursively partitioned into BT or TT structures.

As another example, when a CTU is divided by using the QTBTTT structure, a CU division flag (split_cu_flag) indicating whether to divide the CU is extracted. When the corresponding block is partitioned, a first flag (qt_split_flag) may also be extracted. During the segmentation process, recursive MTT segmentation of 0 or more times may occur after recursive QT segmentation of 0 or more times for each node. For example, for CTUs, MTT partitioning may occur immediately, or conversely, QT partitioning may occur only multiple times.

As another example, when dividing the CTU by using the QTBT structure, a first flag (qt_split_flag) related to the division of QT is extracted to divide each node into four nodes of the lower layer. In addition, a split flag (split_flag) indicating whether or not a node corresponding to a leaf node of QT is further split into BT and split direction information are extracted.

On the other hand, when the entropy decoder 510 determines the current block to be decoded by using the partition of the tree structure, the entropy decoder 510 extracts information on a prediction type indicating whether the current block is intra-predicted or inter-predicted. When the prediction type information indicates intra prediction, the entropy decoder 510 extracts syntax elements for intra prediction information (intra prediction mode) of the current block. When the prediction type information indicates inter prediction, the entropy decoder 510 extracts information representing syntax elements of the inter prediction information, i.e., a motion vector and a reference picture to which the motion vector refers.

Further, the entropy decoder 510 extracts quantization-related information and extracts information on transform coefficients of the quantized current block as information on a residual signal.

The reordering unit 515 may change the sequence of the 1D quantized transform coefficients entropy-decoded by the entropy decoder 510 into a 2D coefficient array (i.e., block) again in the reverse order of the coefficient scan order performed by the video encoding device.

The inverse quantizer 520 inversely quantizes the quantized transform coefficient and inversely quantizes the quantized transform coefficient by using a quantization parameter. The inverse quantizer 520 may also apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in 2D. The inverse quantizer 520 may perform inverse quantization by applying a matrix of quantized coefficients (scaled values) from the video encoding device to a 2D array of quantized transform coefficients.

The inverse transformer 530 restores a residual signal by inversely transforming the dequantized transform coefficients from a frequency domain to a spatial domain to generate a residual block of the current block.

Further, when the inverse transformer 530 inversely transforms a partial region (sub-block) of the transform block, the inverse transformer 530 extracts a flag (cu_sbt_flag) transforming only the sub-block of the transform block, direction (vertical/horizontal) information (cu_sbt_horizontal_flag) of the sub-block, and/or position information (cu_sbt_pos_flag) of the sub-block. The inverse transformer 530 also inversely transforms transform coefficients of the corresponding sub-block from the frequency domain to the spatial domain to restore a residual signal, and fills the region that is not inversely transformed with a value of "0" as the residual signal to generate a final residual block of the current block.

Further, when applying MTS, the inverse transformer 530 determines a transform index or a transform matrix to be applied in each of the horizontal direction and the vertical direction by using MTS information (mts_idx) signaled from the video encoding apparatus. The inverse transformer 530 also performs inverse transformation on the transform coefficients in the transform block in the horizontal direction and the vertical direction by using the determined transform function.

The predictor 540 may include an intra predictor 542 and an inter predictor 544. The intra predictor 542 is activated when the prediction type of the current block is intra prediction, and the inter predictor 544 is activated when the prediction type of the current block is inter prediction.

The intra predictor 542 determines an intra prediction mode of the current block among the plurality of intra prediction modes according to syntax elements of the intra prediction mode extracted from the entropy decoder 510. The intra predictor 542 also predicts the current block by using neighboring reference pixels of the current block according to an intra prediction mode.

The inter predictor 544 determines a motion vector of the current block and a reference picture to which the motion vector refers by using syntax elements of the inter prediction mode extracted from the entropy decoder 510.

The adder 550 restores the current block by adding the residual block output from the inverse transformer 530 to the prediction block output from the inter predictor 544 or the intra predictor 542. In intra prediction of a block to be decoded later, pixels within the restored current block are used as reference pixels.

The loop filtering unit 560, which is an in-loop filter, may include a deblocking filter 562, an SAO filter 564, and an ALF 566. Deblocking filter 562 performs deblocking filtering on boundaries between restored blocks to remove block artifacts occurring due to block unit decoding. The SAO filter 564 and ALF 566 perform additional filtering on the restored block after deblocking filtering to compensate for differences between restored pixels and original pixels that occur due to lossy encoding. The filter coefficients of the ALF are determined by using information on the filter coefficients decoded from the bitstream.

The restored blocks filtered by the deblocking filter 562, the SAO filter 564, and the ALF 566 are stored in the memory 570. When all blocks in one image are restored, the restored image may be used as a reference image for inter-predicting blocks within a picture to be subsequently encoded.

The present embodiment relates to encoding and decoding of an image (video) as described above. More specifically, the present embodiment provides a video encoding/decoding method and apparatus for selectively encoding and decoding a partition direction of sub-blocks to alleviate a burden of signal transmission using sub-block partitions while efficiently performing intra prediction on a per-sub-block basis.

The video encoding/decoding method according to the present embodiment may be performed by the intra predictor 122 of the video encoding apparatus and the intra predictor 542 of the video decoding apparatus.

Further, the aspect ratio of a block is defined as a value obtained by dividing the horizontal length of the block by the vertical length thereof.

I. Intra prediction and Intra Sub-Partition (ISP)

In the VVC technique, the intra prediction mode of the luminance block has a finely divided directional mode (i.e., 14 to 80) in addition to the non-directional mode (i.e., plane and DC), as shown in fig. 3a and 3 b. Based on the prediction mode, there are several techniques available for improving the coding efficiency of intra prediction. After sub-partitioning the current block into small blocks of the same size, ISP techniques share intra prediction modes among all sub-blocks. However, ISP techniques may apply a transform to each sub-block. Here, the sub-division of the block may be performed in a horizontal or vertical direction.

In the following description, as shown in fig. 6, a large block before being sub-partitioned is referred to as a current block, and a small block of each sub-partition is referred to as a sub-block.

ISP technology operates as follows.

The video encoding apparatus signals to the video decoding apparatus an intra_sub-areas_mode_flag indicating whether to apply the ISP and an intra_sub-areas_split_flag indicating the sub-division method. Partition types for sub-partitions according to intra_sub-partitions_mode_flag and intra_sub-partitions_split_flag are shown in table 1.

TABLE 1

IntraSubPartitionsSplitType	Name of IntraParticSplitType
		0	ISP_NO_SPLIT
1	ISP_HOR_SPLIT
		2	ISP_VER_SPLIT

The ISP technology sets the partition type intrasubpartitionsplit type as follows.

When intra_sub_distributions_mode_flag is 0, intra_sub_distributions split type is set to 0, and sub-block division is not performed, as shown in the example of fig. 7 a. In other words, no ISP is applied.

If intra_sub_modes_flag is not 0, ISP is applied. Here, the intra_sub_sub_split type is set to a value of 1+intra_sub_split_flag, and sub-block division is performed according to a division type. Horizontal sub-block division (isp_hor_split) is performed if intra-sub-partitionsplit type=1, and sub-block division (isp_ver_split) is performed in the vertical direction if intra-sub-partitionsplit type=2. In other words, intra_sub_split_flag may indicate the direction of sub-block division.

For example, when the ISP mode of sub-partitioning a block in the horizontal direction is applied to the current block, intra_sub-partition type is 1, intra_sub-partition_mode_flag is 1, and intra_sub-partition_partition_flag is 0.

In the following description, intra_sub-partitions_mode_flag is referred to as a sub-block partition application flag, intra_sub-partitions_split_flag is referred to as a sub-block partition direction flag, and intra sub-partitions split type is referred to as a sub-block partition type.

Also, isp_hor_split is used interchangeably with horizontal SPLIT, and isp_ver_split is used interchangeably with vertical SPLIT.

When the current block is divided horizontally or vertically, ISP applications may be restricted according to the size of the current block during division to prevent small blocks from being divided. In other words, when the current block size is 4×4, the ISP is not applied. A block of size 4 x 8 or 8 x 4 may be divided into two sub-blocks of the same shape and size, which is called Half Split. Other sized blocks may be partitioned into four sub-blocks of the same shape and size, referred to as quater Split.

The video encoding device sequentially encodes each sub-block. Here, each sub-block shares the same intra prediction information. In intra prediction for encoding each sub-block, the video encoding apparatus may improve compression efficiency by using reconstructed pixels in a previously encoded sub-block as predicted pixel values of subsequent sub-blocks, similar to the horizontal partition shown in fig. 7b and the vertical partition shown in fig. 7 c.

However, as described above, existing methods of dividing one block into a plurality of sub-blocks but sharing one prediction mode among the sub-blocks exhibit efficiency problems in some respects. For example, it is assumed that when intra prediction is applied to each sub-block, the current block is partitioned in a specific direction (e.g., vertical direction) according to an intra prediction mode (e.g., a mode predicted in the vertical direction). In this case, there is a case where reconstructed reference pixels in spatially neighboring sub-blocks are not employed due to a specific property of a given intra prediction direction. In this case, even if the ISP technique is used, the video decoding apparatus cannot exert the advantage of the ISP technique that employs spatially more adjacent pixels. However, if the video encoding apparatus is still signaling to apply ISP technology, the signaling indicates that horizontal partitioning is applied instead of vertical partitioning in the above example. Accordingly, in many cases, the video decoding apparatus can explicitly determine in which direction between the vertical direction and the horizontal direction the sub-block is partitioned according to the intra prediction mode. However, since the existing method always transmits the sub-block division direction flag intra_sub-division_split_flag to the video decoding apparatus to indicate the division direction, the ISP technique may not be fully optimized for intra prediction, which may limit the efficiency and advantages of the ISP technique.

Selective coding of partitioning directions of sub-blocks

In the present invention, in order to improve the efficiency of the ISP technology and maximize the benefits thereof, when the ISP technology is applied, the video encoding apparatus does not always transmit a sub-block division direction flag intra_sub-division_split_flag indicating whether it is a horizontal division or a vertical division. Instead, the intra_sub-blocks_split_flag is transmitted only when the video decoding apparatus cannot determine the sub-block division direction by itself. In other words, when the video decoding apparatus is able to determine the sub-block division direction by itself, the video encoding apparatus does not signal the sub-block division direction. In contrast, the video decoding apparatus determines the sub-block division direction by itself and decodes using the determined sub-block division direction.

In the following description, an embodiment according to the present invention is described from the perspective of an intra predictor 122 within a video encoding device.

When intra prediction is performed in units of sub-blocks of a partition, the video encoding apparatus may set a pre-trimming range pre_trimming_range. The pre-trim range pre_prune_range is a subset of the intra prediction modes and includes intra prediction directions that do not use reconstructed samples of new reconstructed neighboring sub-blocks when ISP techniques are applied to the sub-blocks. Therefore, even if the current block is divided into sub-blocks using the ISP technique, the video encoding apparatus does not refer to pixel values of previously reconstructed sub-blocks constituting the current block when prediction is performed in the intra prediction direction belonging to the pre_prediction_range. Instead, the video encoding device performs intra prediction using neighboring pixels of the current block that may be far from the current block. In other words, in this case, the advantageous features of ISP techniques that partition the current block and perform prediction with closer reference samples are not utilized. This is because the intra prediction direction prevents the utilization of the above features. For example, if the intra prediction mode indicates a vertical direction and the sub-block is vertically partitioned, it is not necessary to utilize the previously encoded sub-block at all. Here, when the sub-block is vertically divided (i.e., in the case of isp_ver_split), the intra prediction mode indicating the vertical direction belongs to pre_prediction_range.

As shown below, a pre-trimming range pre_trimming_range may be preset for each division direction of the sub-block. First, a method for generating the pre-trimming range pre_trimming_range when vertically dividing the current block will be described with reference to fig. 8a, 8b, and 8 c.

In the example of fig. 8a, θ represents the angle between the vertical line and the line passing through any pixel p (x, y) in the current block and the reference sample r (x+iidx, -1). In addition, table 2 specifies the mapping between predModeIntra and the angle parameter intrapendole.

TABLE 2

predModeIntra	-14	-13	-12	-11	-10	-9	-8	-7	-6	-5	-4	-3
													intraPredAngle	512	341	256	171	128	102	86	73	64	57	51	45
predModeIntra	-2	-1	2	3	4	5	6	7	8	9	10	11
													intraPredAngle	39	35	32	29	26	23	20	18	16	14	12	10
predModeIntra	12	13	14	15	16	17	18	19	20	21	22	23
													intraPredAngle	8	6	4	3	2	1	0	-1	-2	-3	-4	-6
predModeIntra	24	25	26	27	28	29	30	31	32	33	34	35
													intraPredAngle	-8	-10	-12	-14	-16	-18	-20	-23	-26	-29	-32	-29
predModeIntra	36	37	38	39	40	41	42	43	44	45	46	47
													intraPredAngle	-26	-23	-20	-18	-16	-14	-12	-10	-8	-6	-4	-3
predModeIntra	48	49	50	51	52	53	54	55	56	57	58	59
													intraPredAngle	-2	-1	0	1	2	3	4	6	8	10	12	14
predModeIntra	60	61	62	63	64	65	66	67	68	69	70	71
													intraPredAngle	16	18	20	23	26	29	32	35	39	45	51	57
predModeIntra	72	73	74	75	76	77	78	79	80
													intraPredAngle	64	73	86	102	128	171	256	341	512

Here, intraPredAngle is a value with 1/32 pixel precision.

Also, the iIdx shown in fig. 8a is calculated using a bit shift operation as shown in equation 1 to reduce the computational complexity.

Equation 1

iIdx＝((y+1)·intraPredAngle)＞＞5

In the example of fig. 8a, the tangent of the angle θ may be represented by equation 2.

Equation 2

Therefore, the tangent value of the angle θ can be approximated by equation 3 according to equations 1 and 2.

Equation 3

On the other hand, as shown in fig. 8b, it is assumed that the current block is divided in the vertical direction, and the size of the sub-block is pw×ph. PW and PH represent the horizontal and vertical lengths of the sub-blocks of the partition, respectively. As shown in fig. 8b, if the angle between the vertical line of the sub-block and the upper left diagonal is denoted as phi, the tangent of the angle phi can be expressed by equation 4.

Equation 4

All pixels above the upper left corner diagonal do not use the newly reconstructed reference samples. In other words, when the intra prediction direction is greater than the prediction direction corresponding to the upper left diagonal line, more than half of the pixels included in the sub-block do not use the newly reconstructed reference samples. When the intra prediction direction is the same as the upper left diagonal direction of the sub-block, a case occurs in which half of the pixels use newly reconstructed reference samples. Based on the above observation, by making Φ equal to θ, the start direction of pre_probable_range can be found. In other words, equations 5 and 6 can be derived from equations 3 and 4.

Equation 5

Equation 6

On the other hand, in equations 3, 5, and 6, N represents interpolation accuracy, where one example of the value is 32.

For example, in the example of fig. 8c, the current block is a 16×16 luma CU block. According to equation 6, the intra presadlle is calculated as-8, and referring to table 2, predModeIntra of the intra presadlle corresponding to-8 is the intra prediction mode 44. When determining whether to apply the ISP mode to the current block, since a mode greater than 44 of the isp_ver_splites indicating that the division direction of the sub-block is the vertical direction is included in the pre_available_range, the video encoding apparatus may determine the isp_hor_split regardless of the isp_ver_split at the time of encoding. Accordingly, in this case, it is not necessary to signal the sub-block division direction flag intra_sub-division_split_flag indicating the sub-block division direction.

Next, a method for generating a pre_pruned range when horizontally dividing a current block will be described with reference to fig. 8d, 8e, and 8 f.

In the example of fig. 8d, θ represents the angle between the horizontal line and the line passing through any pixel p (x, y) in the current block and the reference sample r (-1, y+iidy). Further, referring to table 2, idiy is calculated by equation 7.

Equation 7

iIdy＝((x+1)·intraPredAngle)＞＞5

In the example of fig. 8d, the tangent value of the angle θ may be represented by equation 8.

Equation 8

Thus, the tangential value of the angle θ can be approximated by equation 3 according to equations 7 and 8.

As shown in fig. 8e, if the angle between the horizontal line and the upper left diagonal line of the sub-block is expressed as phi, the tangent line of the angle phi can be expressed by equation 9.

Equation 9

On the other hand, all pixels below the upper left diagonal do not use the newly reconstructed reference samples. When the intra prediction direction is the same as the upper left diagonal direction of the sub-block, a case occurs in which half of the pixels use newly reconstructed reference samples. Based on the above observation, by making Φ equal to θ, the start direction of pre_probable_range can be found. In other words, equation 10 can be derived from equations 3 and 9.

Equation 10

As described above, N represents interpolation accuracy, where one example of this value is 32.

For example, in the example of fig. 8f, the current block is a 16×16 luma CU block. According to equation 6, the intra presadlle is calculated as-8, and referring to table 2, predModeIntra of the intra presadlle corresponding to-8 is intra prediction mode 24. When determining whether to apply the ISP mode to the current block, since a mode less than 24 out of the isp_hor_split indicating that the division direction of the sub-block is the horizontal direction is included in the pre_available_range, the video encoding apparatus may determine the isp_ver_split regardless of the isp_hor_split at the time of encoding. Accordingly, in this case, it is not necessary to signal the sub-block division direction flag intra_sub-division_split_flag indicating the sub-block division direction.

The horizontal and vertical lengths of the sub-blocks are defined as the angle between the upper left diagonal and the horizontal line or the angle between the upper left diagonal and the vertical line. Thus, pre_possible_range may be differently determined for each of the vertical SPLIT isp_ver_split and the horizontal SPLIT isp_hor_split according to the SPLIT direction. As shown in the examples of fig. 9a and 9b, pre_prunable_range may be defined as vertical pre_prunable_range_ver and horizontal pre_prunable_range_hor.

As shown in the example of fig. 9a, in the case of vertical segmentation, pre_pruneable_range_ver includes a value greater than the prediction direction corresponding to the upper left diagonal of the sub-block. On the other hand, as shown in fig. 9b, in the case of horizontal division, the pre_prediction_range_hor may include a value smaller than the prediction direction corresponding to the upper left diagonal line of the sub-block.

It is assumed that the technique according to the present embodiment is applied to VVC; in this case, pre_pending_range may be determined in consideration of the aspect ratio of the current block, the number of sub-partition blocks, and the partition direction as shown in table 3.

TABLE 3 Table 3

In table 3, X represents an intra prediction mode.

Meanwhile, based on table 3, pre_pressurized_range may be represented by equation 11.

Equation 11

In the case of ISP_HOR_SPLIT, pre_PRUNable_range_HOR: x1 is more than or equal to X2

In the case of ISP_VER_SPLIT, pre_secure_range_VER: x1 is more than X and less than or equal to X2

In equation 11, X1 and X2 represent the lower and upper limits of pre_pruneable_range, respectively.

The pre-trim range generator 1010 is included in the intra predictor 122 of the video encoding device. The pre-trim range generator 1010 may determine X1 and X2 using the horizontal and vertical lengths of the sub-blocks and the dividing direction of the sub-blocks according to table 3. If pre_pruneable_range is checked according to table 3 in case that the division direction is fixed after the aspect ratio of the current block is calculated according to the horizontal and vertical lengths of the sub-blocks, the pre_pruneable_range is the same regardless of the number of sub-blocks of the partition. For example, assume that the aspect ratio of the sub-blocks of the horizontal partition is 4. The aspect ratio of the current block is 2 if it is a Half Split (half_split) and 1 if it is a Quarter Split (quarter_split). From table 3, it can be seen that "X <24" in any case.

As another example of the present invention, the pre-trimming range generator 1010 may determine X1 and X2 using the horizontal and vertical lengths of the current block before being divided into sub-blocks, the number of sub-blocks, and the sub-block division direction according to table 3.

On the other hand, information on the horizontal and vertical lengths of the current block or the horizontal and vertical lengths of the sub-blocks may be replaced with the aspect ratio of the block. If the intra prediction mode X satisfies the corresponding condition of the horizontal division or the vertical division of equation 11, the intra predictor 122 determines that the intra prediction mode X is included in the pre_probable_range.

As described above, when the block is divided using the ISP technique, in order to prevent the block from being divided into too small blocks, the current block having a size of 4×8 or 8×4 is divided into two sub-blocks (half_split), and the current block having a size greater than the above may be divided into four sub-blocks (quart_split). In other words, according to the existing ISP technology, the number of sub-blocks may be determined according to the size of the current block. In general, in the case of quater_split, X1 and X2 determined by the pre-trim range generator 1010 according to table 3 and equation 11 are represented as shown in table 4. In the case of half_split, X1 and X2 can be represented as shown in table 5.

TABLE 4 Table 4

TABLE 5

In another embodiment of the present invention, the size of the block that prevents the block from being divided into sub-blocks may be set to different values. For example, the current block may be divided into sub-blocks such that the size of the divided blocks is not less than 8×8. As another embodiment, the current block may be divided such that the size of the sub-block is not less than 2×2 after the current block is divided into the sub-blocks.

On the other hand, although the above description shows a method for determining the pre-trimming range pre_finer_range by the intra predictor 122 within the video encoding apparatus, the method may be equally applied to the intra predictor 542 of the video decoding apparatus. In other words, to determine the pre-trimmed range pre_trimmed_range, the intra predictor 542 may further include a pre-trimmed range generator.

In the following description, an intra prediction method of selective encoding including information indicating a division direction of a sub-block performed by the intra predictor 122 within the video encoding apparatus is described.

The video encoding device obtains the size of the current block and the intra prediction mode X S1100. Here, the size of the current block may be represented by horizontal and vertical lengths.

The video encoding device generates a pre-trimming range pre_prune_range S1102 of the sub-block. Here, the pre_prunable_range includes pre_prunable_range_ver and pre_prunable_range_hor. The video encoding apparatus may generate pre_probable_range_ver and pre_probable_range_hor using the aspect ratio of the current block and the number of sub-blocks of the partition according to table 3. On the other hand, since the existing ISP technology can determine the number of sub-blocks of a partition according to the size of the current block, it is not necessary to encode the number of sub-blocks of the partition.

As another example of the present invention, when the number of sub-blocks of a partition cannot be determined according to the size of a current block, a video encoding apparatus obtains and encodes the number of sub-blocks of the partition, and transmits the encoded number of sub-blocks of the partition to a video decoding apparatus.

The video encoding apparatus checks whether the intra prediction mode X of the current block is included in the pre_probable_range_ver or the pre_probable_range_hor S1104.

If the intra prediction mode X is included in at least one of the two ranges (yes at S1004), the video encoding apparatus checks whether the intra prediction mode X of the current block is included in the pre_prediction_range_ver S1106.

When the intra prediction mode X of the current block is included in the pre_prediction_range_ver (yes at S1006), the application of the corresponding intra mode to the vertical partition means that intra prediction is performed without using newly reconstructed neighboring prediction samples. This case represents the same case where sub-block division is not performed.

The video encoding apparatus sets intra_sub_modes_flag=0, and first calculates the encoding efficiency corresponding to the setting. Further, in order to use another encoding method, the video encoding apparatus sets intra_sub_modes_flag=1, and sets the sub-block division direction flag intra_sub_split_flag to isp_hor_split, i.e., to horizontal division S1108, to calculate encoding efficiency. The video encoding apparatus can select and encode better cases between the case of using and the case of not using the ISP. Here, if the intra_sub_modes_flag=0 case yields better results, the video encoding apparatus does not have to transmit the sub-block division direction because the ISP is not applied. Further, if the intra_sub_modes_flag=1 case is more advantageous, it is apparent that the sub block division direction is isp_hor_split, i.e., horizontal division. Therefore, in this case, the video encoding apparatus may not signal the intra_sub-blocks_split_flag indicating the sub-block division direction.

In the same manner, when the intra prediction mode X of the current block is included in the pre_prediction_range_hor (no at S1006), applying the corresponding intra mode to the horizontal division means that intra prediction is performed without using newly reconstructed neighboring prediction samples. Accordingly, the video encoding apparatus sets intra_sub_split_flag to isp_ver_split to calculate encoding efficiency S1110. In this case, since it is apparent that the sub-block division direction is isp_ver_split, i.e., vertical division, the video encoding apparatus may not signal intra_sub-division_split_flag indicating the sub-block division direction.

On the other hand, when the intra prediction mode X of the current block is not included in both the pre_probable_range_ver and the pre_probable_range_hor (no in S1004), the video encoding apparatus may use a conventional method.

The video encoding device performs isp_hor_split verification and isp_ver_split verification S1112. The video encoding apparatus may determine the division direction by using a process of horizontally and vertically checking the ISP sub-block division direction.

The video encoding apparatus sets a value S1114 of intra_sub_split_flag according to the determined partition direction. As described above, when intra_sub_split_flag=0, isp_hor_split is indicated; isp_ver_split is indicated when intra_sub_alternatives_split_flag=1. After encoding the intra_sub-options_split_flag, the video encoding apparatus transmits a bitstream including the encoded intra_sub-options_split_flag to the video decoding apparatus.

In the following description, an intra prediction method including selective decoding of information indicating a sub-block division direction performed by the intra predictor 542 within the video decoding apparatus is described.

In the following description, it is assumed that the value of the intra_sub_modes_flag signaled is 1 so that the video decoding apparatus performs ISP sub-block division of the current block.

The entropy decoder 510 within the video decoding device decodes the size of the current block and the intra prediction mode X S1200. Here, the size of the current block may be represented by horizontal and vertical lengths.

The video decoding apparatus generates a pre-trimming range pre_prune_range S1202 of the sub-block. Here, the pre_prunable_range includes pre_prunable_range_ver and pre_prunable_range_hor. The video decoding apparatus may generate pre_probable_range_ver and pre_probable_range_hor using the aspect ratio of the current block and the number of sub-blocks of the partition according to table 3. On the other hand, since the video decoding apparatus can determine the number of sub-blocks of the partition according to the size of the current block using the existing ISP technology, it is not necessary to decode the number of sub-blocks of the partition.

As another example of the present invention, when the number of sub-blocks of a partition cannot be determined according to the size of the current block, the video decoding apparatus may decode the number of sub-blocks of the partition transmitted by the video encoding apparatus.

The video decoding apparatus checks whether the intra prediction mode X of the current block is included in the pre_probable_range_ver or the pre_probable_range_hor S1204.

If the intra prediction mode X is included in at least one of the two ranges (yes at S1204), the video decoding apparatus checks whether the intra prediction mode X of the current block is included in the pre_prediction_range_ver S1206.

When the intra prediction mode X of the current block is included in the pre_prediction_range_ver (yes at S1206), the video decoding apparatus sets the intra_sub-blocks_split_flag to 0 as shown in detail in the description of the video encoding apparatus without reading the value S1208 of the sub-block division direction flag intra_sub-blocks_split_flag. In other words, the video decoding apparatus may set the sub-block division direction to isp_hor_split, thereby indicating horizontal division.

In the same manner, when the intra prediction mode X of the current block is included in the pre_prediction_range_hor (no at S1206), the video decoding apparatus sets the intra_sub-blocks_split_flag to 1 without separately reading the value S1210 of the sub-block division direction flag intra_sub-blocks_split_flag. In other words, the video decoding apparatus may set the sub-block division direction to isp_ver_split, thereby indicating vertical division.

On the other hand, when the intra prediction mode X of the current block is not included in both the pre_probable_range_ver and the pre_probable_range_hor (no at S1204), the video decoding apparatus may set the division direction S1212 of the sub block by decoding a value of intra_sub-areas_split_flag indicating the division direction of the sub block.

Although steps in the respective flowcharts are described as sequentially performed, these steps merely exemplify the technical ideas of some embodiments of the present invention. Accordingly, one of ordinary skill in the relevant art may perform the steps by changing the order depicted in the various figures or by performing two or more steps in parallel. Accordingly, the steps in the various flowcharts are not limited to the order in which they occur as shown.

It should be understood that the foregoing description presents illustrative embodiments that may be implemented in various other ways. The functions described in some embodiments may be implemented by hardware, software, firmware, and/or combinations thereof. It should also be understood that the functional components described in this specification are labeled as "..units" to highlight the possibility of their independent implementation.

On the other hand, the various methods or functions described in some embodiments may be implemented as instructions stored in a non-volatile recording medium, which may be read and executed by one or more processors. Non-volatile recording media include all types of recording devices that store data in a form readable by a computer system, for example. For example, the nonvolatile recording medium may include a storage medium such as an erasable programmable read-only memory (EPROM), a flash memory drive, an optical disk drive, a magnetic hard disk drive, a Solid State Drive (SSD), and the like.

Although the exemplary embodiments of the present application have been described for illustrative purposes, those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope and spirit of the application. Accordingly, embodiments of the present application have been described for brevity and clarity. The scope of the technical idea of the embodiment of the application is not limited by the illustration. Accordingly, it will be understood by those of ordinary skill that the scope of the present application is not limited by the embodiments explicitly described above, but is instead limited by the claims and their equivalents.

(reference numerals)

122: intra-frame predictor

510: entropy decoder

542: intra-frame predictor

1010: a pre-trim range generator.

Cross Reference to Related Applications

The present application claims priority from korean patent application No.10-2020-0157793, filed 11/23 in 2020, and korean patent application No.10-2021-0161995, filed 11/23 in 2021, the entire contents of which are incorporated herein by reference.

Claims

1. An intra prediction method performed by a video decoding device for applying an intra prediction mode of a current block to a sub-block obtained by dividing the current block, the method comprising:

Decoding the size of the current block and the intra prediction mode;

generating a pre-trim range for the sub-block based on the size of the current block and the number of partitioned sub-blocks, wherein the pre-trim range includes a vertical pre-trim range and a horizontal pre-trim range and represents a set of intra-prediction directions, wherein the sub-block does not use a constructed sample of newly reconstructed neighboring sub-blocks when performing the prediction; and

the division direction of the sub-block is set according to whether the intra prediction mode is included in the vertical pre-trimming range or the horizontal pre-trimming range.

2. The method of claim 1, wherein generating the pre-trimmed range includes determining a number of sub-blocks of the partition or a number of sub-blocks using pre-decoding according to a size of the current block.

3. The method of claim 1, wherein setting the segmentation direction comprises: when the intra prediction mode is included in the vertical pre-trimming range, the division direction of the sub-block is set to the horizontal direction.

4. The method of claim 1, wherein setting the segmentation direction comprises: when the intra prediction mode is included in the horizontal pre-trimming range, the division direction of the sub-block is set to the vertical direction.

5. The method of claim 1, wherein setting the partition direction when the intra prediction mode is not included in the pre-trimming range comprises: after decoding a sub-block division direction flag indicating a division direction of a sub-block, a division direction of the sub-block is set according to the sub-block division direction flag.

6. A video decoding device that applies an intra prediction mode of a current block to a sub-block obtained by dividing the current block, the device comprising:

an entropy decoder configured to decode a size of a current block and an intra prediction mode;

a pre-trimming range generator configured to generate a pre-trimming range of the sub-block based on the size of the current block, wherein the pre-trimming range includes a vertical pre-trimming range and a horizontal pre-trimming range and represents a set of intra-prediction directions, wherein the sub-block does not use a constructed sample of a newly reconstructed neighboring sub-block when performing the prediction; and

an intra predictor configured to set a division direction of the sub-block according to whether an intra prediction mode is included in a vertical pre-trimming range or a horizontal pre-trimming range.

7. The apparatus of claim 6, wherein the pre-finishing range generator determines the number of sub-blocks of the partition or the number of sub-blocks using pre-decoding according to the size of the current block.

8. The apparatus of claim 6, wherein the vertical pre-trim range comprises a value greater than a predicted direction corresponding to an upper left diagonal of a vertically partitioned sub-block.

9. The apparatus of claim 6, wherein the horizontal pre-trim range comprises a value less than a prediction direction corresponding to an upper left diagonal of a sub-block partitioned in a horizontal direction.

10. The apparatus of claim 6, wherein the intra predictor sets a division direction of the sub-block to a horizontal direction when the intra prediction mode is included in the vertical pre-finishing range.

11. The apparatus of claim 6, wherein the intra predictor sets a division direction of the sub-block to a vertical direction when the intra prediction mode is included in the horizontal pre-finishing range.

12. The apparatus of claim 6, wherein the entropy decoder decodes a sub-block partition direction flag indicating a partition direction of the sub-block when the intra prediction mode is not included in the pre-finishing range.

13. The apparatus of claim 12, wherein the intra predictor sets a partition direction of a sub-block according to a sub-block partition direction flag.

14. An intra prediction method performed by a video encoding device that applies an intra prediction mode of a current block to a sub-block obtained by dividing the current block, the method comprising:

obtaining the size of the current block and an intra prediction mode;

generating a pre-trimmed range of sub-blocks based on the size of the current block, wherein the pre-trimmed range comprises a vertical pre-trimmed range and a horizontal pre-trimmed range and represents a set of intra-prediction directions, wherein the sub-blocks do not use the constructed samples of newly reconstructed neighboring sub-blocks when performing the prediction; and

15. The method of claim 14, wherein generating a pre-trim range comprises: the number of sub-blocks of the partition is determined according to the size of the current block or the number of previously obtained sub-blocks is used.

16. The method of claim 14, wherein setting the partition direction when the intra prediction mode is included in the vertical pre-trim range comprises: the division direction of the sub-block is set to the horizontal direction, and a sub-block division direction flag indicating the division direction of the sub-block is not generated.

17. The method of claim 14, wherein setting the partition direction when the intra prediction mode is included in the horizontal pre-finishing range comprises: the division direction of the sub-block is set to the vertical direction, and a sub-block division direction flag indicating the division direction of the sub-block is not generated.

18. The method of claim 14, wherein setting the partition direction when the intra prediction mode is not included in the pre-trimming range comprises:

generating coding efficiency according to the horizontal segmentation and the vertical segmentation of the current block; and

a sub-block division direction indicating a division direction of the sub-block is set based on the coding efficiency.