CN110024386B

CN110024386B - Method and apparatus for encoding/decoding image, recording medium for storing bit stream

Info

Publication number: CN110024386B
Application number: CN201780073904.9A
Authority: CN
Inventors: 高玄硕; 林成昶; 姜晶媛; 李镇浩; 李河贤; 全东山; 赵承眩; 金晖容; 崔振秀
Original assignee: Electronics and Telecommunications Research Institute ETRI
Current assignee: Electronics and Telecommunications Research Institute ETRI
Priority date: 2016-11-29
Filing date: 2017-11-27
Publication date: 2023-10-20
Anticipated expiration: 2037-11-27
Also published as: KR20220103685A; CN110024386A; CN117041568A; WO2018101685A1; CN117041570A; KR102617953B1; KR20240001097A; CN117041569A; CN117041567A; KR20180061025A; KR102422506B1

Abstract

The present invention relates to a method and apparatus for encoding/decoding an image. The method for decoding an image according to the present invention comprises the steps of: dividing the current block into one or more sub-blocks; deriving an intra prediction mode of each of the one or more sub-blocks using at least one of an intra prediction mode of the current block and an intra prediction mode of a neighboring block adjacent to the current block; intra prediction is performed on each of the one or more sub-blocks using the derived intra prediction mode.

Description

Method and apparatus for encoding/decoding image, recording medium for storing bit stream

Technical Field

The present invention relates to a method and apparatus for encoding/decoding an image. In particular, the present invention relates to a method and apparatus for encoding/decoding an image using intra prediction and a recording medium storing a bitstream generated by the image encoding method/apparatus of the present invention.

Background

Recently, demands for high resolution and high quality images, such as High Definition (HD) images and Ultra High Definition (UHD) images, have increased in various application fields. However, higher resolution and quality image data increases the amount of data compared to conventional image data. Accordingly, when image data is transmitted by using a medium such as a conventional wired and wireless broadband network, or when image data is stored by using a conventional storage medium, the cost of transmission and storage increases. In order to solve these problems occurring with the improvement of resolution and quality of image data, an efficient image encoding/decoding technique for higher resolution and higher quality images is required.

Image compression techniques include various techniques including: inter-prediction techniques for predicting pixel values included in a current picture from a previous picture or a subsequent picture of the current picture; an intra prediction technique of predicting pixel values included in a current picture by using pixel information in the current picture; transform and quantization techniques for compressing the energy of the residual signal; entropy coding techniques that assign short codes to values with high frequencies of occurrence and long codes to values with low frequencies of occurrence; etc. The image data can be efficiently compressed by using such an image compression technique, and can be transmitted or stored.

Disclosure of Invention

Technical problem

An object of the present invention is to provide a method and apparatus for encoding and decoding an image to improve compression efficiency and a recording medium storing a bitstream generated by the image encoding method/apparatus of the present invention.

Another object of the present invention is to provide a method and apparatus for encoding and decoding an image using intra prediction to improve compression efficiency and a recording medium storing a bitstream generated by the image encoding method/apparatus of the present invention.

Another object of the present invention is to provide a method and apparatus for encoding and decoding an image by performing intra prediction using a transform model, an isospatial model, or a bilinear interpolation model, and a recording medium storing a bitstream generated by the image encoding method/apparatus of the present invention.

Technical proposal

An image decoding method according to the present invention may include: partitioning a current block into one or more sub-blocks; deriving an intra prediction mode of each of the one or more sub-blocks by using at least one of an intra prediction mode of the current block and an intra prediction mode of a neighboring block adjacent to the current block; and performing intra prediction on each of the one or more sub-blocks by using the derived intra prediction mode.

In the image decoding method according to the present invention, the step of deriving the intra prediction mode may include: generating an Intra Prediction Direction Field (IPDF) for a current block by using at least one of an intra prediction mode of the current block and an intra prediction mode of a neighboring block; and deriving an intra prediction mode of each of the one or more sub-blocks by using the generated IPDF.

In the image decoding method according to the present invention, IPDF may be generated using a transformation model, and the transformation model may include at least one of a rigid body transformation, a similarity transformation, an affine transformation, a homography transformation, and a 3D transformation.

In the image decoding method according to the present invention, the intra prediction mode of the neighboring block used in the step of generating IPDF may be a seed point intra prediction mode (spim) of a seed block including a seed point, and the seed point may be adaptively determined based on the size or shape of the current block or sub-block.

In the image decoding method according to the present invention, a list of spims may be configured, wherein the list of spims includes intra prediction modes of neighboring blocks as spim candidates, and the spim may be selected from the list of spims according to the number of spims required to generate IPDF.

In the image decoding method according to the present invention, when there is a neighboring block in which an intra prediction mode is derived using IPDF, among neighboring blocks, IPDF of a current block may be generated based on the IPDF of the neighboring block.

In the image decoding method according to the present invention, the intra prediction mode of the sub-block located at the outermost side of the current block may be derived using the intra prediction modes of the neighboring blocks, the intra prediction mode of at least one remaining sub-block may be derived using the intra prediction mode of the sub-block located at the outermost side of the current block, and the intra prediction modes of two sub-blocks and the remaining sub-block located at the outermost side of the current block may be equally spatially derived.

In the image decoding method according to the present invention, the intra prediction mode of the sub-block located at the outermost side of the current block may be derived using the intra prediction modes of the neighboring blocks, the intra prediction mode of at least one remaining sub-block may be derived using the intra prediction mode of the sub-block located at the outermost side of the current block, and the intra prediction mode of the remaining sub-block may be derived through bilinear interpolation of the intra prediction modes of two sub-blocks located at the outermost side of the current block.

An image decoding apparatus according to the present invention may include an intra prediction unit configured to: partitioning a current block into one or more sub-blocks; deriving an intra prediction mode of each of the one or more sub-blocks by using at least one of an intra prediction mode of the current block and an intra prediction mode of a neighboring block adjacent to the current block; and performing intra prediction on each of the one or more sub-blocks by using the derived intra prediction mode.

An image encoding method according to the present invention may include: partitioning a current block into one or more sub-blocks; determining an intra prediction mode of each of the one or more sub-blocks by using at least one of an intra prediction mode of the current block and an intra prediction mode of a neighboring block adjacent to the current block; and performing intra prediction on each of the one or more sub-blocks by using the determined intra prediction mode.

In the image encoding method according to the present invention, the step of determining the intra prediction mode may include: generating an Intra Prediction Direction Field (IPDF) for a current block by using at least one of an intra prediction mode of the current block and an intra prediction mode of a neighboring block; and determining an intra prediction mode of each of the one or more sub-blocks by using the generated IPDF.

In the image encoding method according to the present invention, IPDF may be generated using a transformation model, and the transformation model may include at least one of a rigid body transformation, a similarity transformation, an affine transformation, a homography transformation, and a 3D transformation.

In the image encoding method according to the present invention, the intra prediction mode of the neighboring block used in the step of generating IPDF may be a seed point intra prediction mode (spim) of a seed block including a seed point, and the seed point may be adaptively determined based on the size or shape of the current block or sub-block.

In the image encoding method according to the present invention, a list of spims may be configured, wherein the list of spims includes intra prediction modes of neighboring blocks as spim candidates, and the spim may be selected from the list of spims according to the number of spims required to generate IPDF.

In the image encoding method according to the present invention, when there is a neighboring block in which an intra prediction mode is derived using IPDF, among neighboring blocks, IPDF of a current block may be generated based on the IPDF of the neighboring block.

In the image encoding method according to the present invention, the intra prediction mode of the sub-block located at the outermost side of the current block may be determined using the intra prediction modes of the neighboring blocks, the intra prediction mode of at least one remaining sub-block may be determined using the intra prediction mode of the sub-block located at the outermost side of the current block, and the intra prediction modes of two sub-blocks and the remaining sub-block located at the outermost side of the current block may be equally spatially determined.

In the image encoding method according to the present invention, the intra prediction mode of the sub-block located at the outermost side of the current block may be determined using the intra prediction modes of the neighboring blocks, the intra prediction mode of at least one remaining sub-block may be determined using the intra prediction mode of the sub-block located at the outermost side of the current block, and the intra prediction mode of the remaining sub-block may be determined through bilinear interpolation of the intra prediction modes of two sub-blocks located at the outermost side of the current block.

An image encoding apparatus according to the present invention may include an intra prediction unit configured to: partitioning a current block into one or more sub-blocks; determining an intra prediction mode of each of the one or more sub-blocks by using at least one of an intra prediction mode of the current block and an intra prediction mode of a neighboring block adjacent to the current block; and performing intra prediction on each of the one or more sub-blocks by using the determined intra prediction mode.

A computer readable recording medium according to the present invention may store a bitstream generated by the image encoding method of the present invention.

Advantageous effects

According to the present invention, an image encoding/decoding method and apparatus having improved compression efficiency and a recording medium storing a bitstream generated by the image encoding method/apparatus of the present invention can be provided.

And, according to the present invention, it is possible to provide an image encoding/decoding method and apparatus using intra prediction, which improves compression efficiency, and a recording medium storing a bitstream generated by the image encoding method/apparatus of the present invention.

And, according to the present invention, there can be provided an image encoding/decoding method and apparatus for performing intra prediction using a transform model, an isospatial model, or a bilinear interpolation model, and a recording medium storing a bitstream generated by the image encoding method/apparatus of the present invention.

Drawings

Fig. 1 is a block diagram showing a configuration of an encoding apparatus according to an embodiment of the present invention.

Fig. 2 is a block diagram showing a configuration of a decoding apparatus according to an embodiment of the present invention.

Fig. 3 is a diagram schematically showing a partition structure of an image when the image is encoded and decoded.

Fig. 4 is a diagram illustrating intra prediction processing.

Fig. 5 is a diagram illustrating a method of performing intra prediction on a current block according to an embodiment of the present invention.

Fig. 6 is a diagram illustrating an embodiment of deriving an intra prediction mode of a current block using spim.

Fig. 7 is a diagram showing an embodiment of constructing an spim list including two spims.

Fig. 8 is an exemplary diagram showing an embodiment of constructing an spim list including three spims.

Fig. 9 is an exemplary diagram showing an embodiment of constructing an spim list including four spims.

Fig. 10 is an exemplary diagram showing the sub-block size in the case where the size of the current block is 16×16.

Fig. 11 is a diagram illustrating an embodiment of allocating an intra prediction mode using a determined IPDF.

Fig. 12 is an exemplary diagram illustrating a reconstructed block adjacent to a current block.

Fig. 13 is a diagram illustrating an embodiment of deriving an intra prediction mode using neighboring reconstructed blocks.

Fig. 14 is a diagram illustrating an embodiment of deriving a sub-block-based intra prediction mode.

Fig. 15 is a diagram illustrating another embodiment of deriving a sub-block-based intra prediction mode.

Fig. 16 is an exemplary diagram illustrating neighboring reconstructed sample lines that may be used for intra prediction of a current block.

Fig. 17 is a diagram illustrating an embodiment of constructing a reference sample for a sub-block included in a current block.

Fig. 18 is a diagram illustrating a method of replacing an unavailable reconstructed sample with an available reconstructed sample.

Fig. 19 is an exemplary diagram illustrating intra prediction according to the shape of a current block.

Detailed Description

Mode for the invention

Many modifications may be made to the present invention and there are various embodiments of the present invention, examples of which will now be provided with reference to the accompanying drawings and described in detail. However, the present invention is not limited thereto, and the exemplary embodiments may be construed to include all modifications, equivalents, or alternatives within the technical spirit and scope of the present invention. Like reference numerals refer to the same or similar functionality in all respects. In the drawings, the shape and size of elements may be exaggerated for clarity. In the following detailed description of the invention, reference is made to the accompanying drawings that show, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the disclosure. It is to be understood that the various embodiments of the disclosure, although different, are not necessarily mutually exclusive. For example, the particular features, structures, and characteristics described herein in connection with one embodiment may be implemented in other embodiments without departing from the spirit and scope of the disclosure. Further, it is to be understood that the location or arrangement of individual elements within each disclosed embodiment may be modified without departing from the spirit and scope of the disclosure. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present disclosure is defined only by the appended claims, appropriately interpreted, along with the full range of equivalents to which the claims are entitled.

The terms "first," "second," and the like, as used in the specification, may be used to describe various components, but these components are not to be construed as limited by the terms. The term is used merely to distinguish one component from another. For example, a "first" component may be termed a "second" component, and a "second" component may be similarly termed a "first" component, without departing from the scope of the present invention. The term "and/or" includes a combination of items or any of a plurality of items.

It will be understood that in the present specification, when an element is referred to simply as being "connected" or "coupled" to another element, it can be "directly connected" or "directly coupled" to the other element or be connected or coupled to the other element with other elements interposed therebetween. In contrast, it will be understood that when an element is referred to as being "directly coupled" or "directly connected" to another element, there are no intervening elements present.

Further, constituent elements shown in the embodiments of the present invention are independently shown so as to exhibit characteristic functions different from each other. Therefore, this does not mean that each constituent element is constituted in a separate constituent unit of hardware or software. In other words, for convenience, each component includes each of the enumerated components. Thus, at least two of each component may be combined to form one component, or one component may be divided into a plurality of components to perform each function. Embodiments in which each component is combined and embodiments in which one component is divided are also included in the scope of the present invention without departing from the essence of the present invention.

The terminology used in the description presented herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The use of the expression in the singular includes the plural unless it has a distinct meaning in context. In this specification, it will be understood that terms, such as "comprises," "comprising," "has," "having," "includes," etc., are intended to specify the presence of stated features, amounts, steps, acts, elements, components, or combinations thereof disclosed in the specification, but are not intended to preclude the possibility that one or more other features, amounts, steps, acts, elements, components, or combinations thereof may be present or added. In other words, when a specific element is referred to as being "included", elements other than the corresponding element are not excluded, and additional elements may be included in the embodiments of the present invention or the scope of the present invention.

Furthermore, some constituent elements may not be indispensable constituent elements performing the necessary functions of the present invention, but may be optional constituent elements merely improving the performance thereof. The present invention may be implemented by including only essential components for implementing the essence of the present invention, and not including components for improving performance. Structures that include only the indispensable components and not only optional components for improving performance are also included in the scope of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In describing exemplary embodiments of the present invention, well-known functions or constructions will not be described in detail since they would unnecessarily obscure the present invention. The same constituent elements in the drawings are denoted by the same reference numerals, and repetitive description of the same elements will be omitted.

Further, hereinafter, an image may mean a picture constituting a video, or may mean a video itself. For example, "encoding or decoding an image or both" may mean "encoding or decoding an image or both" and "encoding or decoding an image among a plurality of images of an image" may mean "encoding or decoding an image or both. Here, the picture and the image may have the same meaning.

Description of the terms

An encoder: means a device that performs encoding.

A decoder: meaning a device that performs decoding.

And (3) block: is a sample of an mxn matrix. Here, M and N mean positive integers, and a block may mean a sample matrix in two dimensions. Blocks may represent units. The current block may mean an encoding target block that becomes a target when encoding or a decoding target block that becomes a target when decoding. Further, the current block may be at least one of a coded block, a predicted block, a residual block, and a transformed block.

Sampling points: is the basic unit constituting a block. Which can be expressed as ranging from 0 to 2 depending on the bit depth (Bd) ^Bd -1. In the present invention, a sample may be used as a meaning of a pixel.

A unit: meaning the coding and decoding unit. When encoding and decoding an image, a unit may be an area generated by partitioning a single image. Further, a unit may mean a sub-divided unit when a single image is partitioned into a plurality of sub-divided units during encoding or decoding. When encoding and decoding an image, predetermined processing for each unit may be performed. A cell may be partitioned into sub-cells that are smaller in size than the cell. By function, a unit may mean a block, a macroblock, a coding tree unit, a coding tree block, a coding unit, a coding block, a prediction unit, a prediction block, a residual unit, a residual block, a transform unit, a transform block, etc. Further, to distinguish a unit from a block, the unit may include a luma component block, a chroma component block associated with the luma component block, and a syntax element for each color component block. The cells may have various sizes and shapes, in particular, the form of the cells may be a two-dimensional geometry, such as rectangular, square, trapezoidal, triangular, pentagonal, etc. Further, the unit information may include at least one of a unit type (indicating an encoding unit, a prediction unit, a transformation unit, etc.), a unit size, a unit depth, an order in which the units are encoded and decoded, and the like.

Coding tree unit: a single coding tree block with a luminance component Y is configured and two coding tree blocks are associated with the two components Cb and Cr. Further, it may represent a syntax element including a block and each block. Each coding tree unit may be partitioned by using at least one of a quadtree partitioning method and a binary tree partitioning method to configure lower layer units such as coding units, prediction units, transform units, and the like. The coding tree unit may be used as a term for specifying a block of pixels that becomes a processing unit in decoding/encoding an image that is an input image.

Coding tree blocks: may be used as a term for specifying any one of a Y-code tree block, a Cb-code tree block, and a Cr-code tree block.

Adjacent blocks: meaning a block adjacent to the current block. The block adjacent to the current block may mean a block in contact with a boundary of the current block or a block located within a predetermined distance from the current block. Neighboring blocks may mean blocks adjacent to the vertices of the current block. Here, a block adjacent to a vertex of the current block may mean a block vertically adjacent to an adjacent block horizontally adjacent to the current block or a block horizontally adjacent to an adjacent block vertically adjacent to the current block.

Reconstructing neighboring blocks: meaning neighboring blocks that are adjacent to the current block and have been spatially/temporally encoded or decoded. Here, reconstructing a neighboring block may mean reconstructing a neighboring cell. The reconstructed spatial neighboring block may be a block that is within the current picture and has been reconstructed by encoding or decoding or both. The reconstructed temporal neighboring block is a block within the reference picture that is at the same location as the current block of the current picture or a neighboring block of the block.

Cell depth: meaning the extent to which the unit is partitioned. In the tree structure, the root node may be the highest node and the leaf node may be the lowest node. Further, when a cell is expressed as a tree structure, the level at which the cell is located may mean the cell depth.

Bit stream: meaning a bitstream comprising encoded image information.

Parameter set: corresponding to header information in the configuration of the bitstream. At least one parameter set among the video parameter set, the sequence parameter set, the picture parameter set, and the adaptive parameter set may be included in the parameter set. Further, the parameter set may include slice header and parallel block (tile) header information, etc.

Analysis: it may mean that the value of the syntax element is determined by performing entropy decoding, or it may mean that entropy decoding itself.

The symbols: it may mean at least one of syntax elements, coding parameters, and transform coefficient values of the encoding/decoding target unit. Further, the symbol may mean an entropy encoding target or an entropy decoding result.

Prediction unit: meaning a basic unit when prediction such as inter prediction, intra prediction, inter compensation, intra compensation, and motion compensation is performed. A single prediction unit may be partitioned into multiple partitions of small size, or may be partitioned into lower layer prediction units.

Prediction unit partitioning: meaning the form obtained by partitioning the prediction unit.

A conversion unit: meaning a basic unit when encoding/decoding such as transformation, inverse transformation, quantization, inverse quantization, transform coefficient encoding/decoding is performed on a residual signal. A single transform unit may be partitioned into multiple transform units having a small size.

Scaling: meaning the process of multiplying a factor by the transform coefficient level. Transform coefficients may be generated by scaling the transform coefficient levels. Scaling may also be referred to as dequantization.

Quantization parameters: may mean a value used when generating a transform coefficient level of a transform coefficient during quantization. The quantization parameter may mean a value used when generating the transform coefficient by scaling the transform coefficient level during inverse quantization. The quantization parameter may be a value mapped to a quantized step size.

Delta (Delta) quantization parameter: meaning the difference between the quantization parameter of the encoding/decoding target unit and the predicted quantization parameter.

Scanning: meaning a method of ordering coefficients within a block or matrix. For example, the operation of changing the coefficients of the two-dimensional matrix to the one-dimensional matrix may be referred to as scanning, and the operation of changing the coefficients of the one-dimensional matrix to the two-dimensional matrix may be referred to as scanning or inverse scanning.

Transform coefficients: it may mean coefficient values generated after performing a transform in an encoder. Which may mean coefficient values generated after at least one of entropy decoding and dequantization is performed in a decoder. The quantization level obtained by quantizing the transform coefficient or the residual signal or the quantized transform coefficient level may also fall within the meaning of the transform coefficient.

Quantization level: meaning a value generated by quantizing a transform coefficient or a residual signal in an encoder. Alternatively, the quantization level may mean a value as an inverse quantization target to be inverse-quantized in the decoder. Surprisingly, the quantized transform coefficient level as a result of the transform and quantization may also fall into the meaning of the quantization level.

Non-zero transform coefficients: meaning transform coefficients having values other than 0 or transform coefficient levels having values other than 0.

Quantization matrix: meaning a matrix used in quantization processing and inverse quantization processing performed to improve subject (subject) image quality or object (object) image quality. The quantization matrix may also be referred to as a scaling list.

Quantization matrix coefficients: meaning that each element within the matrix is quantized. The quantized matrix coefficients may also be referred to as matrix coefficients.

Default matrix: meaning a predetermined quantization matrix that is predefined in the encoder and decoder.

Non-default matrix: meaning quantization matrices that are not predefined in the encoder and decoder but signaled by the user.

Fig. 1 is a block diagram showing a configuration of an encoding apparatus according to an embodiment and to which the present invention is applied.

The encoding apparatus 100 may be an encoder, a video encoding apparatus, or an image encoding apparatus. The video may include at least one image. The encoding apparatus 100 may sequentially encode at least one image.

Referring to fig. 1, the encoding apparatus 100 may include a motion prediction unit 111, a motion compensation unit 112, an intra prediction unit 120, a switcher 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy encoding unit 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filter unit 180, and a reference picture buffer 190.

The encoding apparatus 100 may perform encoding on an input image by using an intra mode or an inter mode or both the intra mode and the inter mode. Further, the encoding apparatus 100 may generate a bitstream by encoding an input image, and may output the generated bitstream. The generated bit stream may be stored in a computer readable recording medium, or may be streamed over a wired/wireless transmission medium. When the intra mode is used as the prediction mode, the switcher 115 can switch to intra. Alternatively, when the inter mode is used as the prediction mode, the switcher 115 may switch to the inter mode. Here, the intra mode may mean an intra prediction mode, and the inter mode may mean an inter prediction mode. The encoding apparatus 100 may generate a prediction block of an input image. Further, after generating the prediction block, the encoding apparatus 100 may encode the residuals of the input block and the prediction block. The input image may be referred to as a current image as a current encoding target. The input block may be referred to as a current block as a current encoding target or may be referred to as an encoding target block.

When the prediction mode is an intra mode, the intra prediction unit 120 may use pixel values of blocks that have been encoded/decoded and are adjacent to the current block as reference pixels. Intra prediction unit 120 may perform spatial prediction by using reference pixels, or may generate prediction samples of an input block by performing spatial prediction. Here, intra-prediction may mean intra-prediction.

When the prediction mode is an inter mode, the motion prediction unit 111 may search for a region that best matches the input block from the reference image when performing motion prediction, and may derive a motion vector by using the searched region. The reference picture may be stored in a reference picture buffer 190.

The motion compensation unit 112 may generate a prediction block by performing motion compensation using the motion vector. Here, the inter prediction may mean inter-prediction or motion compensation.

When the value of the motion vector is not an integer, the motion prediction unit 111 and the motion compensation unit 112 may generate a prediction block by applying an interpolation filter to a partial region of the reference picture. In order to perform inter prediction or motion compensation on a coding unit, it may be determined which mode among a skip mode, a merge mode, an Advanced Motion Vector Prediction (AMVP) mode, and a current picture reference mode is used for a motion prediction and compensation method of a prediction unit included in the corresponding coding unit. Inter-picture prediction or motion compensation may then be performed differently according to the determined mode.

The subtractor 125 may generate a residual block by using the residuals of the input block and the prediction block. The residual block may be referred to as a residual signal. The residual signal may mean the difference between the original signal and the predicted signal. In addition, the residual signal may be a signal generated by transforming or quantizing or transforming and quantizing a difference between the original signal and the predicted signal. The residual block may be a residual signal of a block unit.

The transform unit 130 may generate transform coefficients by performing a transform on the residual block, and may output the generated transform coefficients. Here, the transform coefficient may be a coefficient value generated by performing a transform on the residual block. When the transform skip mode is applied, the transform unit 130 may skip the transform of the residual block.

Quantization levels may be generated by applying quantization to transform coefficients or residual signals. Hereinafter, in an embodiment, the quantized level may be referred to as a transform coefficient.

The quantization unit 140 may generate quantization levels by quantizing the transform coefficients or the residual signal according to the parameters, and may output the generated quantization levels. Here, the quantization unit 140 may quantize the transform coefficient by using a quantization matrix.

The entropy encoding unit 150 may generate a bitstream by performing entropy encoding according to probability distribution on the values calculated by the quantization unit 140 or on the encoding parameter values calculated when encoding is performed, and may output the generated bitstream. The entropy encoding unit 150 may perform entropy encoding on information for decoding an image and pixel information of the image. For example, the information for decoding the image may include a syntax element.

When entropy encoding is applied, symbols are represented by allocating a small number of bits to symbols having a high probability of generation and allocating a large number of bits to symbols having a low probability of generation, so that the size of a bit stream for the symbols to be encoded can be reduced. The entropy encoding unit 150 may use an encoding method for entropy encoding, such as exponential golomb, context Adaptive Variable Length Coding (CAVLC), context Adaptive Binary Arithmetic Coding (CABAC), and the like. For example, the entropy encoding unit 150 may perform entropy encoding by using a variable length coding/coding (VLC) table. Further, the entropy encoding unit 150 may derive a binarization method of the target symbol and a probability model of the target symbol/binary bit, and may perform arithmetic encoding by using the derived binarization method and context model.

In order to encode the transform coefficient levels, the entropy encoding unit 150 may change the two-dimensional block form coefficients into a one-dimensional vector form by using a transform coefficient scanning method.

The encoding parameters may include information (flags, indexes, etc.) such as syntax elements encoded in the encoder and signaled to the decoder and information derived when encoding or decoding is performed. The encoding parameter may mean information necessary when encoding or decoding an image. For example, the encoding parameters may include at least one value or combination of the following: unit/block size, unit/block depth, unit/block partition information, unit/block partition structure, whether to perform a quadtree-form partition, whether to perform a binary tree-form partition, binary tree-form partition direction (horizontal direction or vertical direction), binary tree-form partition form (symmetric partition or asymmetric partition), intra prediction mode/direction, reference sample filtering method, prediction block filter tap, prediction block filter coefficient, inter prediction mode, motion information, motion vector, reference picture index, inter prediction angle, inter prediction indicator, reference picture list, reference picture, motion vector predictor candidate, motion vector candidate list, whether to use merge mode, merge candidate list, whether to use skip mode, motion vector interpolation filter type, interpolation filter tap, interpolation filter coefficient, motion vector size, accuracy of motion vector representation, transform type, transform size, information whether primary (first) transform is used, information whether secondary transform is used, primary transform index, secondary transform index, information whether residual signal is present, coding block pattern, coding Block Flag (CBF), quantization parameter, quantization matrix, whether intra loop filter is applied, intra loop filter coefficient, intra loop filter tap, intra loop filter shape/form, whether deblocking filter is applied, deblocking filter coefficient, deblocking filter tap, deblocking filter strength, deblocking filter shape/form, whether adaptive sample point offset is applied, an adaptive sample offset value, an adaptive sample offset class, an adaptive sample offset type, whether an adaptive in-loop filter is applied, an adaptive in-loop filter coefficient, an adaptive in-loop filter tap, an adaptive in-loop filter shape/form, a binarization/anti-binarization method, a context model determination method, a context model update method, whether a normal mode is performed, whether a bypass mode is performed, a context binary bit, a bypass binary bit, a transform coefficient level scanning method, an image display/output order, stripe identification information, stripe type, stripe partition information, parallel block identification information, parallel block type, parallel block partition information, picture type, bit depth, and information of a luminance signal or a chrominance signal.

Here, signaling a flag or index may mean that the corresponding flag or index is entropy encoded by an encoder and included in a bitstream, and may mean that the corresponding flag or index is entropy decoded from the bitstream by a decoder.

When the encoding apparatus 100 performs encoding through inter prediction, the encoded current picture may be used as a reference picture for another image to be subsequently processed. Accordingly, the encoding apparatus 100 may reconstruct or decode the encoded current picture, or may store the reconstructed or decoded image as a reference picture.

The quantization levels may be inverse quantized in an inverse quantization unit 160 or may be inverse transformed in an inverse transformation unit 170. The inverse quantized coefficients or the inverse transformed coefficients or the inverse quantized and inverse transformed coefficients may be added to the prediction block by adder 175. The reconstructed block may be generated by adding the inverse quantized coefficients or the inverse transformed coefficients or the inverse quantized and inverse transformed coefficients to the prediction block. Here, the inverse quantized coefficient or the inverse transformed coefficient or the inverse quantized and inverse transformed coefficient may mean a coefficient on which at least one of the inverse quantization and inverse transformation is performed, and may represent a reconstructed residual block.

The reconstructed block may pass through a filter unit 180. The filter unit 180 may apply at least one of a deblocking filter, a Sample Adaptive Offset (SAO), and an Adaptive Loop Filter (ALF) to the reconstructed block or the reconstructed picture. The filter unit 180 may be referred to as an in-loop filter.

The deblocking filter may remove block distortion generated in boundaries between blocks. To determine whether to apply the deblocking filter, whether to apply the deblocking filter to the current block may be determined based on pixels included in several rows or columns in the block. When a deblocking filter is applied to a block, another filter may be applied depending on the required deblocking filter strength.

To compensate for coding errors, appropriate offset values may be added to the pixel values by using a sample adaptive offset. The sample adaptive offset may correct an offset between the deblocked image and the original image in units of pixels. The method of applying the offset in consideration of the edge information of each pixel may be used, or the following method may be used: the pixels of the image are partitioned into a predetermined number of regions, regions to which an offset is to be applied are determined, and the offset is applied to the determined regions.

The adaptive loop filter may perform filtering based on a comparison of the filtered reconstructed picture with the original picture. Pixels included in the image may be partitioned into predetermined groups, filters to be applied to each group may be determined, and different filtering may be performed for each group. The information whether to apply the ALF may be signaled through a Coding Unit (CU), and the form and coefficients of the ALF to be applied to each block may vary.

The reconstructed block or reconstructed image that passes through the filter unit 180 may be stored in a reference picture buffer 190. Fig. 2 is a block diagram showing the configuration of a decoding apparatus according to an embodiment and to which the present invention is applied.

The decoding apparatus 200 may be a decoder, a video decoding apparatus, or an image decoding apparatus.

Referring to fig. 2, the decoding apparatus 200 may include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an intra prediction unit 240, a motion compensation unit 250, an adder 255, a filter unit 260, and a reference picture buffer 270.

The decoding apparatus 200 may receive the bit stream output from the encoding apparatus 100. The decoding apparatus 200 may receive a bit stream stored in a computer readable recording medium, or may receive a bit stream transmitted through a wired/wireless transmission medium stream. The decoding apparatus 200 may decode the bitstream by using an intra mode or an inter mode. Further, the decoding apparatus 200 may generate a reconstructed image generated by decoding or may generate a decoded image, and may output the reconstructed image or the decoded image.

When the prediction mode used in decoding is an intra mode, the switcher may be switched into the intra mode. Alternatively, the switcher may be switched to the inter mode when the prediction mode used in decoding is the inter mode.

The decoding apparatus 200 may obtain a reconstructed residual block by decoding an input bitstream, and may generate a prediction block. When the reconstructed residual block and the prediction block are obtained, the decoding apparatus 200 may generate a reconstructed block that becomes a decoding target by adding the reconstructed residual block and the prediction block. The decoding target block may be referred to as a current block.

The entropy decoding unit 210 may generate symbols by performing entropy decoding on the bitstream according to the probability distribution. The generated symbols may include symbols in the form of quantization levels. Here, the entropy decoding method may be an inverse process of the above-described entropy encoding method.

In order to decode the transform coefficient level, the entropy decoding unit 210 may change the unidirectional vector form coefficient into a two-dimensional block form by using a transform coefficient scanning method.

The quantization levels may be inverse quantized in inverse quantization unit 220 or may be inverse transformed in inverse transformation unit 230. The quantization level may be the result of inverse quantization or inverse transformation or both inverse quantization and inverse transformation and may be generated as a reconstructed residual block. Here, the dequantization unit 220 may apply a quantization matrix to the quantization level.

When the intra mode is used, the intra prediction unit 240 may generate a prediction block by performing spatial prediction using pixel values of blocks that are adjacent to the decoding target block and have been decoded.

When the inter mode is used, the motion compensation unit 250 may generate a prediction block by performing motion compensation using a motion vector and a reference picture stored in the reference picture buffer 270.

The adder 255 may generate a reconstructed block by adding the reconstructed residual block and the prediction block. The filter unit 260 may apply at least one of a deblocking filter, a sample adaptive offset, and an adaptive loop filter to the reconstructed block or the reconstructed image. The filter unit 260 may output the reconstructed image. The reconstructed block or the reconstructed image may be stored in a reference picture buffer 270 and may be used when performing inter prediction.

Fig. 3 is a diagram schematically showing a partition structure of an image when the image is encoded and decoded. Fig. 3 schematically shows an example of partitioning a single unit into a plurality of lower level units.

In order to efficiently partition an image, a Coding Unit (CU) may be used when encoding and decoding. The encoding unit may be used as a basic unit when encoding/decoding an image. Further, the encoding unit may be used as a unit for distinguishing an intra mode from an inter mode when encoding/decoding an image. The coding unit may be a basic unit for prediction, transformation, quantization, inverse transformation, inverse quantization, or encoding/decoding processes of the transform coefficients.

Referring to fig. 3, an image 300 is sequentially partitioned in a Largest Coding Unit (LCU), and the LCU units are determined as a partition structure. Here, LCU may be used as the same meaning as a Coding Tree Unit (CTU). Unit partitioning may mean partitioning blocks associated with a unit. In the block partition information, information of a unit depth may be included. The depth information may represent the number of times a unit is partitioned or the degree to which the unit is partitioned, or both. The single cells may be partitioned according to a layer associated with the tree structure-based depth information. Each of the partitioned lower layer units may have depth information. The depth information may be information indicating the size of the CU, and may be stored in each CU.

The partition structure may mean a distribution of Coding Units (CUs) in LCU 310. Such a distribution may be determined based on whether a single CU is partitioned into multiple (positive integers equal to or greater than 2, including 2, 4, 8, 16, etc.) CUs. The horizontal and vertical sizes of the CUs generated by the partitioning may be half the horizontal and vertical sizes of the CUs before the partitioning, respectively, or may have sizes smaller than the horizontal and vertical sizes before the partitioning, respectively, according to the number of the partitions. A CU may be recursively partitioned into multiple CUs. Partitioning of the CU may be performed recursively until a predefined depth or a predefined size. For example, the depth of the LCU may be 0 and the depth of the Smallest Coding Unit (SCU) may be a predefined maximum depth. Here, as described above, the LCU may be a coding unit having a maximum coding unit size, and the SCU may be a coding unit having a minimum coding unit size. From the LCU 310, the CU depth increases by 1 as the horizontal or vertical size or both the horizontal and vertical sizes of the CU decrease through the partitioning operation.

Further, information indicating whether or not a CU is partitioned may be used by using partition information of the CU. The partition information may be 1-bit information. All CUs except SCU may include partition information. For example, a CU may not be partitioned when the value of partition information is 1, and a CU may be partitioned when the value of partition information is 2.

Referring to fig. 3, an LCU having a depth of 0 may be 64×64 blocks. 0 may be the minimum depth. The SCU with depth 3 may be an 8 x 8 block. 3 may be the maximum depth. CU's of 32 x 32 blocks and 16 x 16 blocks may be represented as depth 1 and depth 2, respectively.

For example, when a single coding unit is partitioned into four coding units, the horizontal and vertical sizes of the four partitioned coding units may be half the horizontal and vertical sizes of the CU before partitioning. In one embodiment, when a coding unit having a size of 32×32 is partitioned into four coding units, each of the four partitioned coding units may have a size of 16×16. When a single coding unit is partitioned into four coding units, it may be referred to that the coding units may be partitioned into quadtree forms.

For example, when a single coding unit is partitioned into two coding units, the horizontal or vertical size of the two coding units may be half of the horizontal or vertical size of the coding unit before the partitioning. For example, when the coding units having a size of 32×32 are partitioned in the vertical direction, each of the two partitioned coding units may have a size of 16×32. When a single coding unit is partitioned into two coding units, the coding units may be referred to as being partitioned into binary tree forms. LCU 320 of fig. 3 is an example of an LCU that applies both quad-tree form partitions and binary tree form partitions.

Fig. 4 is a diagram illustrating intra prediction processing.

The intra prediction mode may be a non-angular mode or an angular mode. The non-angular mode may be a DC mode or a planar mode, and the angular mode may be a prediction mode having a specific direction or angle. The intra prediction mode may be expressed by at least one of a mode number, a mode value, a mode number, and a mode angle. The number of intra prediction modes may be M including 1, and a non-directional mode and a directional mode.

The number of intra prediction modes may be fixed to N regardless of the block size. Alternatively, the number of intra prediction modes may vary depending on the block size or the color component type or both the block size and the color component type. For example, as the block size becomes larger, the number of intra prediction modes may increase. Alternatively, the number of intra prediction modes of the luminance component block may be greater than the number of intra prediction modes of the chrominance component block.

For intra prediction of the current block, a step of determining whether a sample included in the reconstructed neighboring block is available as a reference sample of the current block may be performed. When there is a sample that is not available as a reference sample of the current block, a value obtained by copying or performing interpolation or both of copying and performing interpolation on at least one sample value included in the samples in the reconstructed neighboring block may be used for an unavailable sample value of the replacement sample, whereby the replacement sample value is used as a reference sample of the current block.

When intra prediction, a filter may be applied to at least one of the reference sample and the prediction sample based on the intra prediction mode and the current block size.

In the case of the planar mode, when a prediction block of a current block is generated, a sample value of a prediction target sample point may be generated by using a weighted sum of an upper left reference sample point of the current sample point and upper right and lower left reference samples of the current sample point according to a position of the prediction target sample point within the prediction block. Further, in case of the DC mode, when generating a prediction block of the current block, an average value of upper and left reference samples of the current block may be used. Furthermore, in case of the angle mode, the prediction block may be generated by using upper, left, upper right, and/or lower left reference samples of the current block. To generate the predicted sample values, interpolation of real units may be performed.

The intra prediction mode of the current block may be entropy encoded/entropy decoded by predicting the intra prediction modes of blocks adjacent to the current block. When the intra prediction modes of the current block and the neighboring block are identical, information that the intra prediction modes of the current block and the neighboring block are identical may be signaled by using predetermined flag information. In addition, indicator information of the same intra prediction mode as that of the current block among the intra prediction modes of the plurality of neighboring blocks may be signaled. When intra prediction modes of the current block and the neighboring block are different, intra prediction mode information of the current block may be entropy encoded/entropy decoded by performing entropy encoding/entropy decoding based on the intra prediction modes of the neighboring block.

As shown in fig. 5, the intra prediction may include an intra prediction mode derivation step S510, a reference sample configuration step S520, and/or an intra prediction execution step S530.

In the intra prediction mode derivation step S510, the intra prediction mode of the current block may be derived using at least one of the following methods: a method of using an intra prediction mode of a neighboring block, a method of decoding (e.g., entropy decoding) an intra prediction mode of a current block, a method of using coding parameters of a neighboring block, a method of using an intra prediction mode of a color component, and/or a method of using an intra prediction mode using a transform model.

In the method of using intra prediction modes of neighboring blocks, the intra prediction mode of the current block may be derived by using at least one of the intra prediction modes of the neighboring blocks, a combination of one or more intra prediction modes of the neighboring blocks, and/or an intra prediction mode derived by using MPM.

In the method of using an intra prediction mode using a transformation model, at least one of a rigid body transformation, a similarity transformation, an affine transformation, and a homography transformation may be used to determine an intra prediction mode of a sub-block in a current block.

Alternatively, in the method of using the intra prediction mode using the transform model, at least one of the iso-spatial model and the bilinear filter mode may be used to determine the intra prediction mode of the sub-block in the current block.

In the reference sample configuration step S520, a reference sample selection step and/or a reference sample filtering step may be performed so that the reference samples may be configured.

In the reference sample point selecting step, in the step of configuring the reference sample points of the sub-blocks in the current block, the reference sample points may be selected differently according to the sub-block scanning method (raster scan, zig-zag scan, vertical scan, etc.).

At least one method among non-directional prediction, prediction based on location information, and/or prediction between color components may be used to perform intra prediction of the current block at the intra prediction performing step S530. In the intra prediction performing step S530, filtering for prediction samples may be performed.

Hereinafter, the intra prediction mode derivation step S510 will be described in detail.

The neighboring block of the current block may be at least one of a left lower side, a left side, an upper left side, and an upper right side neighboring block of the current block. Among the neighboring blocks, neighboring blocks that can use only the intra prediction mode may be used.

In the neighboring block of the current block, the intra prediction mode of the neighboring block at a specific position may be derived as the intra prediction mode of the current block.

Alternatively, two or more neighboring blocks are selected, and statistics of intra prediction modes of the selected neighboring blocks may be derived as intra prediction modes of the current block. The intra prediction mode may be indicated by at least one of a mode number, a mode value, and a mode angle. In the specification, the statistical value may be at least one of a minimum value, a maximum value, an average value, a weighted average value, a mode, and a median value.

The neighboring blocks at a particular location and/or selected neighboring blocks may be blocks at predefined fixed locations. Alternatively, the blocks may be specified based on information signaled by the bitstream.

When at least two intra prediction modes are used, it may be considered whether the intra prediction modes have directionality or non-directionality. For example, among two or more intra prediction modes, the directional intra prediction mode may be used to derive an intra prediction mode of the current block. Alternatively, the intra prediction mode of the current block may be derived using a non-directional intra prediction mode.

When a weighted average is used as the statistical value, a relatively high weight may be assigned to a particular intra prediction mode. The specific intra prediction mode may be, for example, at least one of a vertical mode, a horizontal mode, a diagonal mode, and a non-directional mode. Alternatively, information about a specific intra prediction mode may be signaled through a bitstream. The respective weights of the specific intra prediction modes may be the same or different from each other. Alternatively, the weights may be determined based on the sizes of the neighboring blocks. For example, relatively high weights may be assigned to intra prediction modes of relatively large neighboring blocks.

The intra prediction mode of the current block may be derived using MPM (most probable mode).

When MPM is used, the MPM list may be configured with N intra prediction modes derived using intra prediction modes of neighboring blocks. N is a positive integer and may have a value that varies according to the size and/or shape of the current block. Alternatively, the information about N may be signaled through a bitstream.

The intra prediction modes that may be included in the MPM list may be intra prediction modes of lower left, upper left, and/or upper right neighboring blocks of the current block. In addition, the nondirectional mode may be included in the MPM list. The intra prediction modes may be included in the MPM list in a predetermined order. The predetermined order may be, for example, an order of a mode of a lower left block, a mode of an upper block, a planar mode, a DC mode, a mode of a lower left block, a mode of an upper right block, and a mode of an upper left block. Alternatively, the predetermined order may be an order of a mode of the left block, a mode of the upper block, a planar mode, a DC mode, a mode of the lower left block, a mode of the upper right block, and a mode of the upper left block.

The MPM list may be configured to not include the copy mode. When the number of intra prediction modes included in the MPM list is less than N, additional intra prediction modes may be included in the MPM list. The additional intra prediction modes may be modes corresponding to +k, -k of the directional intra prediction modes included in the MPM list. Integers equal to or greater than one may be specified by k. Alternatively, at least one of a horizontal mode, a vertical mode, and a diagonal mode (45-degree angle mode, 135-degree angle mode, and 225-degree angle mode) may be included in the MPM list. Alternatively, statistics of at least one intra prediction mode of neighboring blocks may be used to derive intra prediction modes to be included in the MPM list.

There may be several MPM lists, and the several MPM lists may be configured in different ways. The intra prediction modes included in each MPM list may not be repeated.

Information (e.g., flag information) indicating whether the intra prediction mode of the current block is included in the MPM list may be signaled through a bitstream. When there are N MPM lists, N pieces of flag information may exist. The determining whether the intra prediction mode of the current block exists in the MPM list may be performed in order for the N MPM lists. Alternatively, information indicating an MPM list including an intra prediction mode of the current block among the N MPM lists may be signaled.

When the intra prediction mode of the current block is included in the MPM list, index information for specifying which mode among modes included in the MPM list may be signaled through a bitstream. Alternatively, the mode of the MPM list at a specific location (e.g., first) may be derived as the intra prediction mode of the current block.

In the configuring of the MPM list, one MPM list may be configured for a predetermined-sized block. When a predetermined size block is partitioned into a plurality of sub-blocks, each of the plurality of sub-blocks may use a configured MPM list.

Alternatively, the intra prediction mode of the current block may be derived using at least one of the intra prediction mode of the neighboring block and the intra prediction mode of the current block derived using the MPM.

For example, when the intra prediction mode of the current block derived using the MPM is pred_mpm, pred_mpm is changed to a predetermined mode by using at least one intra prediction mode of neighboring blocks so that the intra prediction mode of the current block can be derived. For example, pred_mpm may be increased or decreased by N by comparing the pred_mpm with the size of intra prediction modes of neighboring blocks. Here, N may be a predetermined integer such as +1, +2, +3, 0, -1, -2, -3, and the like.

Alternatively, when one of the modes of the pred_mpm and the neighboring block is a non-directional mode and the other is a directional mode, the non-directional mode may be derived as an intra prediction mode of the current block, or the directional mode may be derived as an intra prediction mode of the current block.

The intra prediction mode of the current block may be derived using the intra prediction mode of another color component. For example, when the current block is a chroma block, an intra prediction mode of at least one related luma block corresponding to a chroma target block may be used to derive an intra prediction mode of the chroma block. Here, the relevant luminance block may be determined based on at least one of a position, a size, a shape, or an encoding parameter of the chrominance block. Alternatively, the relevant luminance block may be determined based on at least one of a size, shape, or encoding parameters of the luminance block.

The associated luminance block may be determined using a luminance block including samples corresponding to a center position of the chrominance block, or using at least two luminance blocks each including samples corresponding to at least two positions of the chrominance block. The at least two positions may include a left hand sample position and a center sample position.

When there are several related luminance blocks, statistics of intra prediction modes of at least two related luminance blocks may be derived as intra prediction modes of a chrominance block. Alternatively, the intra prediction mode of a relatively large correlated luminance block may be derived as the intra prediction mode of a chrominance block. Alternatively, when the size of the luminance block corresponding to the predetermined position of the chrominance block is equal to or greater than the size of the chrominance block, the intra prediction mode of the chrominance block may be derived using the intra prediction mode of the associated luminance block.

For example, when the current block is partitioned into sub-blocks, at least one method of deriving an intra prediction mode of the current block may be used to derive an intra prediction mode of each of the partitioned sub-blocks.

In the deriving of the intra prediction mode, the current block may be divided into a plurality of sub-blocks smaller than the current block, and then the intra prediction mode of each sub-block may be derived. Here, the intra prediction mode may represent an intra prediction direction. The intra prediction modes may be included in a set of intra prediction modes predefined in the encoder and decoder.

For example, an Intra Prediction Direction Field (IPDF) of a current block may be generated using at least one of intra prediction modes of a block that has been encoded/decoded by using intra prediction in a reconstructed block adjacent to the current block and the intra prediction mode of the current block. A predetermined transformation model may be used to generate IPDF. After IPDF is generated, the IPDF may be used to determine an intra prediction mode for each sub-block of the current block.

Further, similar to the above example, if the current block is divided from a larger or lower depth block than the current block, the current block may be a sub-block of the larger or lower depth block. In this case, the intra prediction mode of the current block may be derived using at least one of the intra prediction modes of blocks that have been encoded/decoded through intra prediction in reconstructed blocks adjacent to a larger or lower depth block than the current block. The intra prediction mode of the current block may be derived by generating IPDF.

As the predetermined transformation model, at least one of rigid transformation, similarity transformation, affine transformation, homography transformation, 3D transformation, and other transformation may be used. The homography transform may be a perspective transform.

Since the intra prediction mode of each sub-block divided from the current block can be derived using at least one of the intra prediction modes of the block, which has been encoded/decoded by using intra prediction, among the reconstructed blocks adjacent to the current block and the intra prediction mode of the current block, the number of bits required for entropy encoding/entropy decoding the intra prediction mode of each sub-block can be reduced.

The size (granularity) of the sub-block may be smaller than or equal to the size of the current block. For example, if the size of the current block is m×n (M and N are positive integers), the size of the sub-block may be M/kxn/L. K and M may be positive integers, and L may be a positive factor of N. In addition, M/K or N/L may be a positive integer.

Furthermore, P sub-blocks may exist in the current block. P may be 0 or a positive integer. For example, 1, 2, 4, or 16 sub-blocks may exist in the current block.

Further, when the current block is divided into sub-blocks, information indicating whether the current block has been divided into sub-blocks may not be separately entropy encoded/entropy decoded. Whether the current block has been divided into sub-blocks may be determined based on information indicating whether an intra prediction mode of the current block is derived based on the sub-blocks.

In addition, since the intra prediction mode of the sub-block is inferred using at least one of the intra prediction modes of the block, which has been encoded/decoded by using intra prediction, in the reconstructed block adjacent to the current block and the intra prediction mode of the current block, the intra prediction mode of the sub-block may not be entropy encoded/entropy decoded. However, the intra prediction mode of the current block may be entropy encoded/entropy decoded. In particular, if the current block is constructed of one sub-block, the intra prediction mode of the current block may not be entropy encoded/entropy decoded, and at least one of the intra prediction modes of blocks that have been encoded/decoded by intra prediction among reconstructed blocks adjacent to the current block may be used to derive the intra prediction mode of the current block.

A block (for IPDF generation) among reconstructed blocks adjacent to the current block, which has been encoded/decoded by using intra prediction, may be referred to as a seed block (seed block). The location of the seed block may be referred to as a seed point (seed point). The seed block including the seed point may have an intra prediction mode called a seed point intra prediction mode (spim).

In the example shown in fig. 6, the size of the current block may be 16×16, and the size of each sub-block may be (16/4) × (16/4). Here, the seed block may be at least one of a plurality of neighboring blocks that have been encoded/decoded through intra prediction. The seed block or seed point may be a fixed position relative to the current block. Alternatively, at least one of the upper, left, upper left, lower left, and upper right blocks or locations may be determined as a seed block or seed point.

For example, an intra prediction mode of at least one of the upper neighboring blocks c, d, e, f and g of the current block may be used as the spim. Alternatively, the intra prediction mode of the upper right block h adjacent to the current block may be used as the spim. Alternatively, an intra prediction mode of at least one of the upper left blocks a and b adjacent to the current block may be used as the spim. Alternatively, an intra prediction mode of at least one of left blocks i, j, k, and l adjacent to the current block may be used as the spim. Alternatively, the intra prediction mode of the lower left block m adjacent to the current block may be used as the spim. For example, an intra prediction mode of the current block may also be used as the spim.

IPDF may be generated using spim of one or more seed points.

For example, if the coordinates of a plurality of seed points for determining parameters of a predetermined transformation model are (x_sp, y_sp) (sp=1, 2, 3, …), and the spim of a seed block including the seed points is mode_sp (sp=1, 2, 3, …), the coordinates (x_sp ', y_sp') of the seed points after transformation may be determined as (x_sp ', y_sp')= (x_sp+dx, y_sp+dy). Here, dx may represent displacement in the x-axis direction, and dy may represent displacement in the y-axis direction. Furthermore, dx=d_sub_cos Θ and dy=d_sub_sin Θ may be determined.

Here, Θ may be determined according to spim. For example, if the intra prediction mode is a directional mode as shown in fig. 6, each spim may have a unique direction, and a positive angle for the x-axis may be determined as Θ.

For example, for vertical intra prediction mode, Θ=270°. For example, for horizontal intra prediction mode, Θ=0°. For example, for the lower left diagonal intra prediction mode, Θ=225 °. For example, for the upper right diagonal intra prediction mode, Θ=45°. For example, for the lower right diagonal intra prediction mode, Θ=135°.

Here, for an intra prediction mode having no directivity, such as a DC mode or a plane mode, Θ may be determined as a specific value. The specific value may be, for example, 0, 90, 180 or 270 degrees.

Spim may represent directionality with respect to the seed point, and d_sub may represent the magnitude of the vector having directionality. The size of d_sub may be determined according to the size and/or shape of the seed block to which the seed point belongs. Further, d_sub may have a fixed value P for each intra prediction mode. Here, P may be 0 or an integer. For example, if the size of the current block is m×n (M and N are positive integers) and d_cur=s (S is a positive integer), d_sub (sub=1, 2, 3, …) of the k×l sub-block (K and L are positive integers) may be determined as d_sub=s (k×l)/(m×n). For example, if the size of the current block is 16×16 in fig. 6, d_cur=s and d_sub of the seed block may be determined according to at least one of the following methods.

For example, for seed block D or e, d_sub=s (4×8)/(16×16) =s/8 may be determined. For example, for seed block g or j, d_sub=s (8×8)/(16×16) =s/4 may be determined. For example, for seed block k or l, d_sub=s (8×4)/(16×16) =s/8 may be determined. For example, for seed block m or h, d_sub=s (16×16)/(16×16) =s may be determined. For example, for each seed block, D_sub may be determined as S.

Candidates may be configured by constructing a spim list for IPDF generation of the current block. The spim list may be generated using intra prediction modes of at least one of neighboring blocks of the current block. For example, the spim may be a set of one or more of an upper left side (spim_tl), an upper right side (spim_tr), a lower left side (spim_bl), and a lower right side (spim_br) spim of the current block. For a w×h current block, spim_tl may have at least one of the intra prediction modes for the upper, upper left, and left neighboring blocks of the position (0, 0) of the current block as a candidate. The spim_tr may have at least one of the intra prediction modes of the upper and upper right neighboring blocks for the position (W-1, 0) of the current block as a candidate. The spimp_bl may have at least one of the intra prediction modes of the left and lower left neighboring blocks for the position (0, h-1) of the current block as a candidate. The spimp_br may represent an intra prediction mode of a neighboring block of the current block. Or spimp_br may be used to represent an intra prediction mode of the current block.

In the example shown in fig. 6, spimp_tl may have at least one of intra prediction modes adjacent to blocks d, b, and j. Further, the spim_tr may have at least one of intra prediction modes adjacent to the blocks g and h. Further, the spimp_bl may have at least one of intra prediction modes adjacent to the blocks i and m. In addition, the spimp_br may have at least one of intra prediction modes of the current block.

Further, seed blocks or seed points may be searched in a predetermined order. For example, the spim list may be constructed from intra prediction modes of respective seed blocks or seed points by searching the seed blocks or seed points in order of left side, upper side, lower left side, upper right side, and upper side.

In order to construct the spim list, processing of excluding intra prediction modes having different directions from other modes based on the similarity between intra prediction modes may be performed for at least one of candidates of each of spim_tl, spim_tr, spim_bl, and spim_br. Here, intra Prediction Mode Difference (IPMD) may be used as a criterion for measuring similarity. Nondirectional MODEs (e.g., dc_mode and PLANAR_mode) can be excluded when constructing the SPIM list.

For example, if ipmd=abs (spim_tl_mode-spim_neighbor) > Th1 (Th 1 is a positive integer) in the comparison between each of the three candidate patterns spim_tl_mode of spim_tl and the candidate pattern spim_neighbor (neighbor=tr or BL or BR), the corresponding pattern may be excluded from the candidate set of spim_tl.

For example, if ipmd=abs (spim_tr_mode-spim_neighbor) > Th2 (Th 2 is a positive integer) in the comparison between each of the two candidate patterns spim_tr_mode of spim_tr and the candidate pattern spim_neighbor (neighbor=tl or BL or BR), the corresponding pattern may be excluded from the candidate set of spim_tr.

For example, if ipmd=abs (spim_bl_mode-spim_neighbor) > Th3 (Th 3 is a positive integer) in the comparison between each of the two candidate patterns spim_bl_mode of spim_bl and the candidate pattern spim_neighbor (neighbor=tl or TR or BR), the corresponding pattern may be excluded from the candidate set of spim_bl.

For example, if ipmd=abs (spim_br_mode-spim_neighbor) > Th4 (Th 4 is a positive integer) in the comparison between each of the candidate patterns spim_br_mode of spim_br and the candidate patterns spim_neighbor (neighbor=tl or TR or BL), the corresponding pattern may be excluded from the candidate set of spim_br.

For example, if at least one of the candidates of each of spim_tl, spim_tr, spim_bl, and spim_br is a non-directional intra prediction mode, such as a DC mode or a planar mode, the corresponding candidate may be excluded from the candidate set.

After candidates having small similarity among candidates for constructing the spim list are excluded, the number of spims required for IPDF generation may be determined according to the specific 2D transformation model used. For example, the 2D transformation model may include rigid body transformations, similarity transformations, affine transformations, and homography transformations. Further, the number of spims may be determined as a variable value of 1, 2, 3, 4, or N (N is a positive integer) according to the 2D transform model.

For example, if the transformation model for IPDF generation is a rigid body transformation, at least two spims may be required.

The rigid body transformation may have 3 degrees of freedom (3-DoF), as described in [ equation 1 ]. Here, (x, y) may be the coordinates of the seed point before transformation, and (x ', y') may be the coordinates of the seed point after transformation. Θ, tx, and ty are model parameters to be determined, which may be rotation angle, x-axis displacement, and y-axis displacement, respectively.

[ equation 1]

The pair of (x, y) - (x ', y') can be obtained using Θ determined from one spim, and two equations related to Θ, tx, and ty can be determined by substituting the pair of (x, y) - (x ', y') into [ equation 1 ]. Further, four equations related to Θ, tx, and ty may be determined from two spims, and three of the four equations may be used to determine the rigid body transformation model.

The two spims may be determined by selecting at least two of spim_tl, spim_tr, spim_bl, and spim_br. The selected spim may be added to the spim list.

As shown in FIG. 7, the SPIPM list may be populated in ascending order of the IPMD sums of the two SPIPM candidate patterns.

For example, one of the candidate patterns of spim_tl and one of the candidate patterns of spim_tr may be used as the two spims. For example, one of the candidate patterns of spim_tl and one of the candidate patterns of spim_bl may be used as the two spims. For example, one of the candidate patterns of spim_tl and one of the candidate patterns of spim_br may be used as the two spims. For example, one of the candidate patterns of spim_tr and one of the candidate patterns of spim_bl may be used as the two spims. For example, one of the candidate patterns of spim_tr and one of the candidate patterns of spim_br may be used as the two spims. For example, one of the candidate patterns of spim_bl and one of the candidate patterns of spim_br may be used as the two spims.

If two spims are not filled, then the spim list may be filled with available spims. For example, the spim list may be populated with spim ± delta. Here, delta may be any positive integer, e.g., 1, 2, 3, ….

Once the two spims (spim 1, spim 2) are determined, four equations related to Θ, tx, and ty can be generated by [ equation 1 ]. Three of the four equations may be used to determine parameters of the rigid body transformation model. The determined model may be used for IPDF generation.

For example, the rigid body transformation may be determined using at least one of two equations calculated by spim 1 and two equations calculated by spim 2. For example, the rigid body transformation may be determined using at least one of two equations calculated by spim 1 and two equations calculated by spim 2.

For example, if a similarity transformation is used as a transformation model for IPDF generation, at least two spims may be required.

The similarity transformation may have a 4-DoF as described in equation 2. Here, (x, y) may be the coordinates of the seed point before transformation, and (x ', y') may be the coordinates of the seed point after transformation. a. b, c and d may be model parameters to be determined.

[ equation 2]

The pair (x, y) - (x ', y') can be obtained using Θ determined from one spim, and two equations related to a, b, c, and d can be determined by substituting the pair (x, y) - (x ', y') into equation 2. Further, four equations related to a, b, c, and d may be determined from two spims, and may be used to obtain a similarity transformation model.

The two spims may be determined by selecting at least two of spim_tl, spim_tr, spim_bl, and spim_br. The selected spim may be added to the spim list. Here, as shown in fig. 7, the spim list may be filled in an ascending order of the sum of IPMD values of the two spim candidate patterns.

Once the two spims (spim 1, spim 2) are determined, the parameters of the similarity transformation model can be determined by generating four equations related to a, b, c, and d by equation 2. The determined model may be used for IPDF generation.

For example, if affine transformation is used as transformation model for IPDF generation, at least three spims may be required.

Affine transformations may have 6-DoF as described in equation 3. Here, (x, y) may be the coordinates of the seed point before transformation, and (x ', y') may be the coordinates of the seed point after transformation. a. b, c, d, e and f may be model parameters to be determined.

[ equation 3]

The pair (x, y) - (x ', y') can be obtained using Θ determined from one spim, and two equations related to a, b, c, d, e and f can be determined by substituting the pair (x, y) - (x ', y') into [ equation 3 ]. Further, six equations related to a, b, c, d, e and f may be determined from three spims, and the affine transformation model may be obtained using the six equations.

Three spims may be determined by selecting at least three of spim_tl, spim_tr, spim_bl, and spim_br. The selected spim may be added to the spim list.

As shown in fig. 8, the spim list may be populated with spims in ascending order of the sum of IPMD values of the three spim candidate patterns.

For example, one of the candidate patterns of spim_tl, one of the candidate patterns of spim_tr, and one of the candidate patterns of spim_bl may be used as three spims. For example, one of the candidate patterns of spim_tl, one of the candidate patterns of spim_tr, and one of the candidate patterns of spim_br may be used as three spims. For example, one of the candidate patterns of spim_tl, one of the candidate patterns of spim_bl, and one of the candidate patterns of spim_br may be used as three spims. For example, one of candidate patterns of spim_tr, one of candidate patterns of spim_bl, and one of candidate patterns of spim_br may be used as three spims.

If three spims are not filled, the spim list may be filled with available spims. For example, the spim list may be populated with spim ± delta. Here, delta may be any positive integer, e.g., 1, 2, 3, ….

Once the three spims (spim 1, spim 2, spim 3) are determined, parameters of the affine transformation model can be determined by generating six equations related to a, b, c, d, e and f by equation 3. The determined model may be used for IPDF generation.

For example, if a homography transform (or perspective transform) is used as the transform model for IPDF generation, at least four spims may be required.

Affine transformations may have 8-DoF as described in equation 4. Here, (x, y) may be the coordinates of the seed point before transformation, and (x ', y') may be the coordinates of the seed point after transformation. h1, h2, h3, h4, h5, h6, h7 and h8 may be model parameters to be determined.

[ equation 4]

The pair of (x, y) - (x ', y') can be obtained using Θ determined from one spim, and two equations related to h1, h2, h3, h4, h5, h6, h7, and h8 can be determined by substituting the pair of (x, y) - (x ', y') into [ equation 4 ]. Further, eight equations related to h1, h2, h3, h4, h5, h6, h7, and h8 may be determined from the four spims, and the homography transformation model may be obtained using the eight equations.

Four spims may be determined by selecting at least four of spim_tl, spim_tr, spim_bl, and spim_br. The selected spim may be added to the spim list.

As shown in fig. 9, the spim list may be sequentially filled with spims in an ascending order of the sum of IPMD values of the four spim candidate patterns.

For example, two of the candidate patterns of spim_tl and two of the candidate patterns of spim_tr may be used as four spims. For example, two of the candidate patterns of spim_tl and two of the candidate patterns of spim_bl may be used as four spims. For example, two of the candidate patterns of spim_tl and two of the candidate patterns of spim_br may be used as four spims. For example, two of the candidate patterns of spim_bl and two of the candidate patterns of spim_tr may be used as four spims. For example, two of the candidate patterns of spim_bl and two of the candidate patterns of spim_br may be used as four spims. For example, two of the candidate patterns of spim_tl, one of the candidate patterns of spim_tr, and one of the candidate patterns of spim_bl may be used as four spims. For example, two of the candidate patterns of spim_tl, one of the candidate patterns of spim_tr, and one of the candidate patterns of spim_br may be used as four spims. For example, two of the candidate patterns of spim_tl, one of the candidate patterns of spim_bl, and one of the candidate patterns of spim_br may be used as four spims. For example, two of the candidate patterns of spim_tr, one of the candidate patterns of spim_tl, and one of the candidate patterns of spim_bl may be used as four spims. For example, two of the candidate patterns of spim_tr, one of the candidate patterns of spim_tl, and one of the candidate patterns of spim_br may be used as four spims. For example, two of the candidate patterns of spim_tr, one of the candidate patterns of spim_tl, and one of the candidate patterns of spim_bl may be used as four spims. For example, two of the candidate patterns of spim_bl, one of the candidate patterns of spim_tl, and one of the candidate patterns of spim_tr may be used as four spims. For example, two of the candidate patterns of spim_bl, one of the candidate patterns of spim_tl, and one of the candidate patterns of spim_br may be used as four spims. For example, two of the candidate patterns of spim_bl, one of the candidate patterns of spim_tr, and one of the candidate patterns of spim_br may be used as four spims. For example, two of the candidate patterns of spim_br, one of the candidate patterns of spim_tl, and one of the candidate patterns of spim_tr may be used as four spims. For example, two of the candidate patterns of spim_br, one of the candidate patterns of spim_tl, and one of the candidate patterns of spim_bl may be used as four spims. For example, two of the candidate patterns of spim_br, one of the candidate patterns of spim_tr, and one of the candidate patterns of spim_bl may be used as four spims. For example, one of the candidate patterns of spim_tl, one of the candidate patterns of spim_tr, one of the candidate patterns of spim_bl, and one of the candidate patterns of spim_br may be used as four spims.

If four spims are not filled, the spim list may be filled with available spims. For example, the spim list may be populated with spim ± delta. Here, delta may be any positive integer, e.g., 1, 2, 3, ….

Once the four spims (spim 1, spim 2, spim 3, spim 4) are determined, the parameters of the homography transformation model can be determined by generating eight equations related to h1, h2, h3, h4, h5, h6, h7, and h8 by equation 4. The determined model may be used for IPDF generation.

After generating the IPDF using the predetermined transform model, the generated IPDF may be used to allocate an intra prediction mode to a kxl sub-block within the w×h current block. Here, the size kxl of each sub-block (K is a positive integer equal to or less than M, and L is a positive integer equal to or less than H) may be a fixed size equal to or less than the size of the current block. Or the sub-block size may be adaptively determined using the size of the current block and/or IPMD. Or the sub-block may be equal to the size of the current block.

As shown in fig. 10 (a), the size of the sub-block may be fixed to 8×8. Alternatively, as shown in fig. 10 (b), the size of the sub-block may be fixed to 4×4. Alternatively, as shown in fig. 10 (c), the size of the sub-block may be fixed to 2×2. Alternatively, as shown in (d) of fig. 10, the size of the sub-block may be fixed to 1×1. The fixed size 1×1 may be the size of the sample cell.

For example, the size of the sub-block may be determined based on the size of the current block. For example, the size of the sub-block may be determined based on at least one of the IPMD of the current block, spim_tl, spim_tr, spim_bl, and spim_br. For example, the size of the sub-block may be determined based on at least one of the IPMD of the current block, the spipm_tl, the spipm_tr, the spipm_bl, and the spipm_br, and the size of the current block.

Information about the size (granularity) of the sub-blocks may be entropy encoded/entropy decoded in the bitstream. Here, the information may be entropy encoded/entropy decoded in at least one of VPS, SPS, PPS, APS, a slice header, a parallel block header, CTU, CU, PU, TU, a block, and a sub-block.

Information about the sub-block size may be adaptively derived according to the size of the current block and/or IPMD without transmitting the information.

Further, the sub-block size may be determined based on at least one of the encoding parameters of the current block and the encoding parameters of neighboring blocks of the current block.

The intra prediction mode of the sub-block may be allocated using the determined IPDF. The intra prediction mode at a specific location in each sub-block may be implemented as a vector by substituting the coordinates of the specific location into the determined IPDF model. The particular location may be determined as the location of a pixel in the sub-block or a location adjacent to a boundary of the sub-block. For example, at least one of the upper left side, the upper right side, the lower left side, the lower right side, and the middle position of the sub-block may be determined as the specific position. If the coordinates of a specific position in the sub-block are (x, y) and the transform coefficients of the position calculated by IPDF are (x ', y'), then θ can be passed through _SB ＝atan[(y’-y)/(x’-x)]To determine the direction theta of the vector _SB 。

As shown in FIG. 11, θ can be set _SB And mapping to intra prediction modes oriented in the most similar direction among the directional modes. θ can be performed using a look-up table (LUT) _SB To intra prediction mode mapping.

In addition, when the intra prediction mode of the sub-block is allocated using IPDF, IPDF may be allocated as the intra prediction mode of the sub-block based on the nearest neighbor method. In addition, when the intra prediction mode is allocated to the sub-block using IPDF, the intra prediction mode may be allocated to the sub-block by quantizing the IPDF to an integer. In addition, when the intra prediction mode is allocated to the sub-block using IPDF, the intra prediction mode may be allocated to the sub-block by rounding the IPDF to the nearest integer.

For sub-block intra prediction based on a transform model, the information to be additionally entropy encoded/entropy decoded in the bitstream may include at least one of the following.

Information indicating whether transform model-based intra prediction per sub-block has been performed for the current block: TBIP_flag

The tbip_flag may be information indicating whether to derive an intra prediction mode of each sub-block using at least one of intra prediction modes of blocks, which have been encoded/decoded by using intra prediction, in reconstructed blocks adjacent to the current block and the intra prediction mode of the current block.

Index indicating the set of spims used: SPIPM_idx

Now, another embodiment of deriving a sub-block based intra prediction mode using a transform model will be given. For example, in case that there is a block in a neighboring reconstructed block that has been encoded/decoded by sub-block-wise intra prediction based on a transform model, an intra prediction mode may be derived for each sub-block using an IPDF model of the neighboring block, instead of directly generating the IPDF of the current block.

To determine whether there are any blocks in the neighboring reconstructed block of the current block that have been encoded/decoded by sub-block-wise intra prediction based on a transform model, a predefined scan order may be used. The scanning order may be at least one of the following orders.

For example, in fig. 12, scanning may be performed in the order of a→b→c→d→e. Alternatively, the scanning may be performed in the order of a→b→d→c→e. Alternatively, the scanning may be performed in the order of b→a→d→c→e. Alternatively, the scanning may be performed in the order of e→a→b→c→d. Alternatively, the scanning may be performed in any other order.

Alternatively, a portion of blocks A, B, C, D and C may be excluded from scanning. Alternatively, blocks other than blocks A, B, C, D and C may be scanned. The neighboring reconstructed block to be scanned may be determined based on at least one of a size, shape, and encoding parameters as described herein of at least one of the neighboring reconstructed block and the current block.

In the example shown in fig. 13, in the case of encoding/decoding a block a, which is a neighboring block to the current block, by intra prediction in sub-block units using a transform model, IPDF of the block a may be generated using at least one of spim_a_tl, spim_a_tr, spim_a_bl, and spim_a_br of the block a.

At least one of the ipmp_cur_tl, the spipm_cur_tr, the spipm_cur_bl, and the spipm_cur_br of the current block may be derived using the IPDF of the block a, and the IPDF of the current block may be generated using the derived at least one of the spipm_cur_tl, the spipm_cur_tr, the spipm_cur_bl, and the spipm_cur_br, thereby performing intra prediction per sub-block.

If at least one of the neighboring reconstructed blocks of the current block is a block encoded/decoded using IPDF, IPDF of the current block may be derived using IPDF of the corresponding neighboring reconstructed block. In addition, information of the tbip_flag may be entropy encoded/entropy decoded.

Another embodiment of deriving a sub-block based intra prediction mode using an equal spatial model will be described.

In case an isospatial model is used, at least two spims may be required. For example, as shown in (a) of fig. 14, four candidate patterns, that is, one of the candidate patterns of spim_tl, one of the candidate patterns of spim_tr, one of the candidate patterns of spim_bl, and one of the candidate patterns of spim_br may be selected in total. As shown in fig. 9, the selected four spim candidate patterns may be filled into the spim list in ascending order of the sum of IPMD values.

The intra prediction mode of the outermost sub-block of the current block may be first determined using spim_tl, spim_tr, spim_bl, and/or spim_br. Equally spatially determining the intra prediction mode may mean dividing values of the intra prediction mode using at least two intra prediction modes at equal intervals and allocating the divided intra prediction mode values to the sub-blocks.

For example, in the example shown in fig. 14 (a), if spim_tl is 24 and spim_tr is 21, the intra prediction modes of the sub-blocks A, B, C and D may be spatially determined using the values of spim_tl and spim_tr, etc. For example, as shown in (B) of fig. 14, a=24, b=23, c=22, and d=21 can be determined. For example, in the example shown in fig. 14 (a), if spim_tl is 25 and spim_bl is 38, the intra prediction modes of the sub-blocks A, E, I and M may be spatially determined using the values of spim_tl and spim_bl, etc. For example, as shown in (b) of fig. 14, a=24, e=29, i=34, and m=38 can be determined. For example, in the example shown in fig. 14 (a), if spim_bl is 38 and spim_br is 35, the intra prediction modes of the sub-blocks M, N, O and P may be spatially determined using the values of spim_bl and spim_br, etc. For example, as shown in (b) of fig. 14, m=38, n=37, o=36, and p=35 can be determined. For example, in the example shown in fig. 14 (a), if spim_tr is 21 and spim_br is 35, the intra prediction modes of the sub-blocks D, H, I and P may be spatially determined using the values of spim_tr and spim_br, etc. For example, as shown in (b) of fig. 14, d=21, h=26, i=31, and p=35 can be determined.

After all intra prediction modes of the outermost sub-block of the current block are determined, intra prediction modes of the next-to-outer sub-block may be determined. In the example shown in fig. 14, the sub-outside sub-blocks may be sub-blocks F, G, J and K. Here, the spim_tl may be reset to a value of a mode (a mode of the sub-block a in (a) of fig. 14) of an upper left position of an upper left sub-block (the sub-block F in (a) of fig. 14) of the sub-outer sub-block. Further, the spim_tr may be reset to a value of a mode (a mode of a sub-block D in (a) of fig. 14) of an upper right position of an upper right sub-block (a sub-block G in (a) of fig. 14) of the sub-outer sub-block. Further, the spimp_bl may be reset to a value of a mode (a mode of the sub-block M in (a) of fig. 14) of a lower left position of a lower left sub-block (a sub-block J in (a) of fig. 14) of the sub-outer sub-block. Further, spim_br may be reset to a value of a mode (a mode of a sub-block P in (a) of fig. 14) of a lower right position of a lower right sub-block (a sub-block K in (a) of fig. 14) of the sub-outer sub-blocks. This operation may be repeated recursively until the mode of all sub-blocks of the current block is determined.

For example, in the example shown in fig. 14 (b), if spim_tl is 24 and spim_tr is 21, intra prediction modes of the sub-blocks F and G may be spatially determined using values of spim_tl and spim_tr, etc. For example, as shown in (c) of fig. 14, f=24 and g=21 may be determined. For example, in the example shown in fig. 14 (b), if spim_bl is 38 and spim_br is 35, intra prediction modes of the sub-blocks J and K may be spatially determined using values of spim_bl and spim_br, etc. For example, as shown in (c) of fig. 14, j=38 and k=35 can be determined.

The information to be additionally entropy encoded/entropy decoded for intra prediction per sub-block based on the isospatial model may be at least one of the following.

Information indicating whether intra prediction per sub-block based on an isospatial model has been performed for the current block: ES_flag

Index indicating the set of spims used: SPIPM_idx

A description will now be given of another embodiment of deriving a sub-block-based intra prediction mode using a bilinear filter model.

In order to determine a sub-block based intra prediction mode using a bilinear filter model, at least two spims may be required. For example, as shown in (a) of fig. 15, a total of four candidate patterns may be selected by selecting one of the candidate patterns of spim_tl, one of the candidate patterns of spim_tr, one of the candidate patterns of spim_bl, and one of the candidate patterns of spim_br. As shown in fig. 9, the four spim candidate patterns selected may be sequentially filled into the spim list in ascending order of the sum of IPMD values.

The mode of the upper left sub-block (sub-block a in (a) of fig. 15) in the current block may be determined as spim_tl. Further, the mode of the upper right sub-block (sub-block D in (a) of fig. 15) in the current block may be determined as spim_tr. Further, the mode of the lower left sub-block (sub-block M in (a) of fig. 15) in the current block may be determined as spimp_bl. Further, the mode of the lower right sub-block (sub-block P in (a) of fig. 15) in the current block may be determined as spim_br. As shown in (b) of fig. 15, intra prediction modes of upper left, upper right, lower left, and lower right sub-blocks in the current block may be determined as spim_tl, spim_tr, spim_bl, and spim_br, respectively. However, the intra prediction modes of the upper left, upper right, lower left, and lower right sub-blocks in the current block are not limited to the spim_tl, spim_tr, spim_bl, and spim_br, and at least one of the intra prediction modes of the upper left, upper right, lower left, and lower right sub-blocks in the current block may be determined as at least one of the spim_tl, spim_tr, spim_bl, and spim_br.

A bilinear filter scheme may be used to determine intra prediction modes for other sub-blocks. For example, the following [ equation 5] may be used. In [ equation 5], the function () may be at least one of floor () or ceil () or round (). In the example shown in fig. 15, function () may be round (). Further, in [ equation 5], #of SubBlk in wdt may represent the number of horizontal subblocks in the current block. Similarly, # of SubBlk in hgt may represent the number of vertical sub-blocks in the current block.

[ equation 5]

For example, as shown in (c) of fig. 15, the intra prediction mode of the remaining sub-block may be determined using [ equation 5].

The information to be additionally entropy encoded/entropy decoded for intra prediction per sub-block based on the bilinear filter model may be at least one of the following.

Information indicating whether intra prediction per sub-block based on a bilinear filter model has been performed for the current block: BF_flag

Index indicating the set of spims used: SPIPM_idx

An intra prediction mode may be derived based on the sub-block using at least one of intra prediction modes of a block, which has been encoded/decoded by intra prediction, among reconstructed blocks adjacent to the current block and the intra prediction mode of the current block, and intra prediction may be performed based on the sub-block using the derived intra prediction mode. Here, samples included in a sub-block previously encoded/decoded based on the sub-block may be used as reference samples for the following intra prediction per sub-block.

The encoder may generate transform coefficients by performing at least one of a first transform, a second transform, and quantization on a residual block generated after intra prediction per sub-block. The generated transform coefficients may be entropy encoded. The first transform, the second transform, and the quantization may be performed with respect to the current block or on a sub-block basis. For example, at least one of the first transform, the second transform, and the quantization may be performed for the entire current block, or at least one of the first transform, the second transform, and the quantization may be performed for each sub-block. Here, the first transform, the second transform, and the quantization may not be performed with respect to the current block or the sub-block.

The transform coefficients may be entropy decoded in a decoder. The reconstructed residual block may be generated by performing at least one of inverse quantization, a first inverse transform, and a second inverse transform on the entropy-decoded transform coefficients. The first transform, the second transform, and the quantization may be performed with respect to the current block or on a sub-block basis. For example, at least one of the first transform, the second transform, and the quantization may be performed for the entire current block, and at least one of the first transform, the second transform, and the quantization may be performed for each sub-block. Here, the first transform, the second transform, and the quantization may not be performed with respect to the current block or the sub-block.

The intra prediction information may be entropy encoded/entropy decoded from the bitstream. Here, the intra prediction information may be signaled in at least one of VPS, SPS, PPS, APS, a slice header, and a parallel block header.

A flag indicating whether the MPMs match: for example, prev_intra_luma_pred_flag

Index indicating position in MPM list: for example mpm_idx

Intra luminance prediction mode information: for example, rem_intra_luma_pred_mode

Intra chroma prediction mode information: for example, intra_chroma_pred_mode

A flag indicating that intra prediction modes of the current block and the sub-block are derived using intra prediction modes of neighboring blocks: for example, NDIP_flag

When deriving or entropy-encoding/entropy-decoding an intra-prediction mode of a current block using N MPM lists, an indicator (MPM flag) indicating whether the same intra-prediction mode as the intra-prediction mode of the current block is included in the intra-prediction modes of the MPM lists for each of the N MPM lists: for example, MPM_FLAG_1, MPM_FLAG_2, …, MPM_FLAG_N

When the same intra prediction mode as that of the current block is included in the intra prediction mode of a specific one of the N MPM lists, index information indicating a position or order of the intra prediction mode in the MPM list: for example, MPM_IDX_1, MPM_IDX_2, …, MPM_IDX_N

Information indicating whether transform model-based intra prediction per sub-block has been performed for the current block and an index indicating the set of spims used: for example, TBIP_flag and SPIPM_idx

Information indicating whether an intra prediction mode has been derived based on sub-blocks using an isospatial model for the current block and an index indicating the set of spims used: ES_flag and SPIPM_idx

Information indicating whether an intra prediction mode has been derived based on sub-blocks using a bilinear filter model for a current block and an index indicating a set of spims used: BF_flag and SPIPM_idx

When the MPM (most probable mode) flag is 1, an intra prediction mode of a luminance component may be derived from a candidate mode including an intra mode of a neighboring unit that has been encoded/decoded by using the MPM index mpm_idx.

When the MPM (most probable mode) flag is 0, the intra prediction mode of the luminance component may be encoded/decoded by using intra prediction mode information rem_intra_luma_pred_mode regarding the luminance component.

The intra prediction mode of the chrominance component may be encoded/decoded by using intra prediction mode information intra_chroma_pred_mode and/or a corresponding intra prediction mode of the chrominance component block.

The intra-prediction information may be entropy encoded/entropy decoded from the bitstream based on at least one of the encoding parameters. For example, the ndip_flag may be encoded/decoded based on the block partition information.

For example, if at least one of the split_ flag, quadtree _flag and the binaryrate_flag is "0" and thus the block is not divided any more, the ndip_flag may be encoded/decoded. For example, if the binarytre_flag is 1, the ndip_flag may not be encoded/decoded.

At least one of the pieces of intra prediction information may not be signaled based on at least one of a size and a shape of the block.

For example, if the size of the current block is a predetermined size, at least one piece of information regarding intra prediction information of the current block may not be signaled, and at least one piece of information regarding intra prediction corresponding to the size of a previously encoded/decoded upper layer block may be used. For example, if the shape of the current block is rectangular, at least one piece of information regarding intra prediction information of the current block may not be signaled, and at least one piece of information regarding intra prediction corresponding to the size of a previously encoded/decoded upper layer block may be used.

When entropy encoding/decoding at least one piece of information in the intra prediction information, at least one of the following binarization methods may be used.

-truncated rice binarization method

K-order exponential golomb binarization method

-finite K-order exponential golomb binarization method

Fixed length binarization method

-unitary binarization method

-truncated unary binarization method

Now, a detailed description will be given of the reference point constructing step S520.

In the step of intra-predicting the current block or the sub-block having a smaller size and/or shape than the current block based on the derived intra-prediction mode, a reference sample for prediction may be constructed. The following description is given in terms of the context of the current block, and the current block may mean a sub-block. The reference samples may be constructed using one or more reconstructed samples or combinations of samples that are adjacent to the current block. Furthermore, filtering may be applied in the step of constructing the reference samples. Here, the reference sample point may be constructed using each reconstructed sample point on the plurality of reconstructed sample points as it is. Alternatively, the reference samples may be constructed after filtering between samples on the same reconstructed sample line. Alternatively, the reference samples may be constructed after filtering between samples on different reconstructed sample lines. The constructed reference samples may be represented by ref [ m, n ], and the reconstructed neighboring samples or samples obtained by filtering the reconstructed neighboring samples may be represented by rec [ m, n ]. Here, m or n may be a predetermined integer value. In the case where the size of the current block is W (horizontal) ×h (vertical), if the uppermost sample point of the current block is (0, 0), the relative position of the leftmost upper reference sample point closest to the sample point may be set to (-1, -1).

As shown in fig. 16, one or more reconstructed sample lines adjacent to the current block may be used to construct a reference sample.

For example, one of the plurality of reconstructed sample lines shown in fig. 16 may be selected, and the reference sample point may be constructed using the selected reconstructed sample line. A predetermined one of the plurality of reconstructed sample lines may be fixedly selected as the selected reconstructed sample line. Alternatively, a specific one of the plurality of reconstructed sample lines may be adaptively selected as the selected reconstructed sample line. In this case, an indicator for the selected reconstructed sample line may be signaled.

For example, one or more of the plurality of reconstructed sample lines shown in fig. 16 may be used in combination to construct a reference sample. For example, the reference samples may be constructed as a weighted sum (or weighted average) of one or more reconstructed samples. The weights for the weighted sum may be assigned based on the distance from the current block. Here, a shorter distance from the current block may be assigned a larger weight. For example, the following [ equation 6] may be used.

[ equation 6]

ref[-1,-1]＝(rec[-2,-1]+2*rec[-1,-1]+rec[-1,-2]+2)>>2

ref[x,-1]＝(rec[x,-2]+3*rec[x,-1]+2)>>2，(x＝0～W+H-1)

ref[-1,y]＝(rec[-2,y]+3*rec[-1,y]+2)>>2，(y＝0～W+H-1)

Alternatively, the reference sample may be constructed using at least one of an average value, a maximum value, a minimum value, a median value, and a most frequent value of the plurality of reconstructed samples based on at least one of a distance from the current block or an intra prediction mode.

Alternatively, the reference samples may be constructed based on changes (variations) in the values of a plurality of successive reconstructed samples. For example, the reference sample may be constructed based on at least one of whether a difference between values of two consecutive reconstructed samples is equal to or greater than a threshold value, whether the values of two consecutive reconstructed samples change continuously or discontinuously, and the like. For example, if the difference between rec < -1 > and rec < -2 > -1 > is equal to or greater than a threshold value, ref < -1 > may be determined as rec < -1 > or a value obtained by applying weighted average using a predetermined weight assigned to rec < -1 >. For example, if the values of a plurality of consecutive reconstructed samples change by n each time as the plurality of consecutive reconstructed samples are closer to the current block, the reference samples ref < -1 > -1] may be determined as rec < -1 > -n.

At least one of the number and position of reconstructed sample lines and a construction method for constructing reference sample points may be differently determined according to whether an upper side or a left side edge of the current block corresponds to a boundary of at least one of a picture, a slice, a parallel block, and a Coding Tree Block (CTB).

For example, in the step of constructing the reference sample point using the reconstructed sample lines 1 and 2, when the upper boundary of the current block corresponds to the CTB boundary, the reconstructed sample line 1 may be used for the upper side and the reconstructed sample lines 1 and 2 may be used for the left side.

For example, in the step of constructing the reference sample points using the reconstructed sample lines 1 to 4, when the upper side boundary of the current block corresponds to the CTB boundary, the reconstructed sample lines 1 and 2 may be used for the upper side and the reconstructed sample lines 1 to 4 may be used for the left side.

For example, in the step of constructing the reference sample using the reconstructed sample line 2, when the upper boundary of the current block corresponds to the CTB boundary, the reconstructed sample line 1 may be used for the upper side and the reconstructed sample line 2 may be used for the left side.

One or more reference sample lines may be constructed by the above process.

The reference sample construction method of the upper side of the current block may be different from the reference sample construction method of the left side of the current block.

Information indicating that a reference sample has been constructed using at least one of the above methods may be encoded/decoded. For example, information indicating whether or not a plurality of reconstructed sample lines are used may be encoded/decoded.

If the current block is divided into a plurality of sub-blocks, and each sub-block has an independent intra prediction mode, a reference sample may be constructed for each sub-block.

As shown in fig. 17, if the size of the current block is 16×16 and 16 4×4 sub-blocks have independent intra prediction modes, a reference sample for each sub-block may be constructed using at least one of the following methods according to a scanning scheme for predicting the sub-block.

For example, a reference sample may be constructed for each sub-block using N reconstructed sample lines adjacent to the current block. In the example shown in fig. 17, N is 1.

For example, in the case where a plurality of sub-blocks are predicted in a raster scan order of 1→2→3→ … →15→16, a reference sample for the kth sub-block may be constructed using a sample of at least one of the left side, upper right side, and lower left side sub-blocks that have been encoded/decoded.

For example, in the case where a plurality of sub-blocks are predicted in the Z scan order of 1→2→5→6→3→4→7→ … →12→15→16, a reference sample for the kth sub-block may be constructed using a sample of at least one of the left side, upper right side, and lower left side sub-blocks that have been encoded/decoded.

For example, in the case where a plurality of sub-blocks are predicted in a zigzag scanning order of 1→2→5→9→6→3→4→ … →12→15→16, a reference sample for the kth sub-block may be constructed using a sample of at least one of the left side, upper right side, and lower left side sub-blocks that have been encoded/decoded.

For example, in the case where a plurality of sub-blocks are predicted in a vertical scan order of 1→5→9→13→2→6→ … →8→12→16, a reference sample for the kth sub-block may be constructed using a sample of at least one of the left side, upper right side, and lower left side sub-blocks that have been encoded/decoded.

In case that the plurality of sub-blocks are predicted in a scan order other than the above scan order, a reference sample for a kth sub-block may be constructed using a sample of at least one of left, upper right, and lower left sub-blocks that have been encoded/decoded.

In the step of selecting the reference sample point, determination and/or population of availability of the block including the reference sample point may be performed. For example, if a block including a reference sample is available, the reference sample may be used. Meanwhile, if a block including a reference sample is not available, one or more available neighboring reference samples may be used to replace the unavailable reference sample by padding.

If the reference sample point exists outside at least one of the picture boundary, the parallel block boundary, the slice boundary, the CTB boundary, and the predetermined boundary, it may be determined that the reference sample point is not available.

In the case of encoding a current block by CIP (constrained intra prediction), if a block including a reference sample is encoded/decoded in an inter prediction mode, it may be determined that the reference sample is not available.

If it is determined that the neighboring reconstructed samples are not available, the neighboring available reconstructed samples may be used to replace the unavailable samples. For example, as shown in fig. 18, in the case where there are available samples and unavailable samples, one or more available samples may be used to replace the unavailable samples.

The sample values of the unavailable samples may be replaced with the sample values of the available samples in a predetermined order. Available samples adjacent to unavailable samples may be used to replace the unavailable samples. In the case where there are no adjacent available samples, the available sample that appears first or the nearest available sample may be used. The order of replacement of unusable samples may be from the bottom left-most side to the top right-most side. Or the replacement order of unavailable samples may be an order from the upper right most side to the lower left most side. Or the replacement order of unavailable samples may be an order from the upper left-most side to the lower right-most side and/or the lower left-most side. Or the replacement order of unavailable samples may be an order from the upper right-most side and/or the lower left-most side to the upper left-most side.

As shown in fig. 18, the unusable samples may be replaced in order from the leftmost lower sample position 0 to the rightmost upper sample. In this case, the value of the first four unavailable samples may be replaced with the value of the first occurring or closest available sample a. The value of the last available sample b may be used to replace the value of the next 13 unavailable samples.

Alternatively, combinations of available samples may be used to replace unavailable samples. For example, the unavailable spot may be replaced with the median of available spots adjacent to both ends of the unavailable spot. For example, in fig. 18, the first four unavailable samples may be filled with the value of available sample a, and the next 13 unavailable samples may be filled with the median of available sample b and available sample c. Alternatively, any value between the values of available sample b and available sample c may be used to fill the 13 unavailable samples. In this case, the unavailable samples may be replaced with different values. For example, when an unavailable sample is closer to available sample a, a value that is close to the value of available sample a may be used to replace the value of the unavailable sample. Similarly, when an unavailable sample is closer to the available sample b, a value that is close to the value of the available sample b may be used to replace the value of the unavailable sample. That is, the value of the unavailable spot may be determined based on the distance from the unavailable spot to the available spots a and/or b.

To replace unusable samples, one or more of a plurality of methods including the above method may be selectively applied. The method for replacing the unavailable samples may be signaled through information included in the bitstream, or a method predetermined by an encoder and a decoder may be used. Or the method for replacing unusable samples may be deduced through a predetermined scheme. For example, the method for replacing unavailable samples may be selected based on the difference between the values of available samples a and b and/or the number of unavailable samples. For example, a method for replacing an unavailable sample may be selected based on a comparison between a difference between values of two available samples and a threshold and/or a comparison between a number of unavailable samples and a threshold. For example, if the difference between the values of two available samples is greater than a threshold and/or if the number of unavailable samples is greater than a threshold, the value of the unavailable sample may be replaced with a different value.

For the constructed one or more reference samples, it may be determined whether to apply filtering according to at least one of an intra prediction mode, a size, and a shape of the current block. If filtering is applied, different filter types may be used according to at least one of intra prediction mode, size, and shape of the current block.

For example, for each of the plurality of reference sample lines, it may be determined differently whether to apply filtering and/or filter type. For example, filtering may be applied to a first adjacent line, while filtering may not be applied to a second adjacent line. For example, both filtered and unfiltered values may be used for the reference samples. For example, among the 3-tap filter, the 5-tap filter, and the 7-tap filter, at least one filter may be selected and applied according to at least one of an intra prediction mode, a size, and a shape of a block.

Hereinafter, the step of performing intra prediction (S530) will be described in detail.

Intra prediction may be performed on the current block or sub-block based on the derived intra prediction mode and the reference samples. In the following description, a current block may mean a sub-block.

For example, non-directional intra prediction may be performed. The non-directional intra prediction mode may be at least one of a DC mode and a planar mode.

Intra prediction of the DC mode may be performed using a median value of one or more of the constructed reference samples. Filtering may be applied to one or more prediction samples at the boundary of the current block. DC mode intra prediction may be adaptively performed according to at least one of a size and a shape of a current block.

For example, as shown in fig. 19 (a), if the shape of the current block is square, the current block may be predicted using the median of the reference samples on the upper and left sides of the current block.

For example, as shown in (b) of fig. 19, if the shape of the current block is rectangular, the current block may be predicted using the median value of reference samples adjacent to the longer side among the width and length of the current block.

For example, if the size of the current block is within a predetermined range, a predetermined sample is selected from upper or left reference samples of the current block, and prediction may be performed using a median value of the selected samples.

Planar mode intra prediction may be performed by calculating a weighted sum in consideration of distances from one or more constructed reference samples according to the location of a target intra prediction sample of the current block.

For example, the prediction block may be calculated as a weighted sum of N reference samples according to the location (x, y) of the target prediction sample. N may be a positive integer, for example, 4.

For example, directional intra prediction may be performed. The directional prediction mode may be at least one of a horizontal mode, a vertical mode, and a mode having a predetermined angle.

Horizontal/vertical mode intra prediction may be performed using one or more reference samples on horizontal/vertical lines at the location of the target intra prediction sample.

Intra prediction in a mode having a predetermined angle may be performed using one or more reference samples on a line having the predetermined angle with respect to the position of the target intra prediction sample. Here, N samples may be used. N may be a positive integer such as 2, 3, 4, 5 or 6. Further, prediction may be performed by applying an N-tap filter (such as a 2-tap, 3-tap, 4-tap, 5-tap, or 6-tap filter), for example.

For example, intra prediction may be performed based on the location information. The position information may be encoded/decoded and the reconstructed sample block at that position may be derived as an intra-predicted block for the current block. Or a block similar to the current block detected by the decoder may be derived as an intra prediction block of the current block.

For example, intra color component prediction may be performed. For example, intra prediction may be performed for a chroma component using a reconstructed luma component of a current block. Alternatively, intra prediction may be performed for another chroma component Cr using one reconstructed chroma component Cb of the current block.

Intra prediction may be performed by using one or more of the above-described various intra prediction methods in combination. For example, an intra-prediction block may be constructed for a current block by a weighted sum of a block predicted using a predetermined non-directional intra-prediction mode and a block predicted using a predetermined directional intra-prediction mode. Here, different weights may be applied according to at least one of an intra prediction mode, a block size, a shape, and/or a sample position of the current block.

The above embodiments may be performed in the same way in the encoder and decoder.

The order applied to the above embodiments may be different between the encoder and the decoder, or the order applied to the above embodiments may be the same between the encoder and the decoder.

The above embodiments may be performed for each of the luminance signal and the chrominance signal, or may be equally performed for the luminance signal and the chrominance signal.

The block form to which the above embodiments of the present invention are applied may have a square form or a non-square form.

The above embodiments of the present invention may be applied according to the size of at least one of a coded block, a predicted block, a transformed block, a current block, a coded unit, a predicted unit, a transformed unit, a unit, and a current unit. Here, the size may be defined as a minimum size or a maximum size or both of the minimum size and the maximum size so that the above embodiments are applied, or may be defined as a fixed size to which the above embodiments are applied. Further, in the above embodiments, the first embodiment may be applied to the first size, and the second embodiment may be applied to the second size. In other words, the above embodiments may be applied in combination according to the size. Further, the above embodiments may be applied when the size is equal to or greater than the minimum size and equal to or less than the maximum size. In other words, the above embodiments may be applied when the block size is included in a predetermined range.

For example, when the size of the current block is 8×8 or more, the above embodiments may be applied. For example, when the size of the current block is 4×4 or more, the above embodiments may be applied. For example, when the size of the current block is 16×16 or more, the above embodiments may be applied. For example, when the size of the current block is equal to or greater than 16×16 and equal to or less than 64×64, the above embodiments may be applied.

The above embodiments of the present invention may be applied according to a temporal layer. To identify the temporal layer to which the above embodiments may be applied, a corresponding identifier may be signaled, and the above embodiments may be applied to a specific temporal layer indicated by the corresponding identifier. Here, the identifier may be defined as a lowest layer or a highest layer or both of the lowest layer and the highest layer to which the above embodiment may be applied, or may be defined as a specific layer indicating that the embodiment is applied. Furthermore, a fixed temporal layer to which the embodiments are applied may be defined.

For example, the above embodiments may be applied when the temporal layer of the current image is the lowest layer. For example, the above embodiment may be applied when the temporal layer identifier of the current image is 1. For example, the above embodiments may be applied when the temporal layer of the current image is the highest layer.

The types of strips to which the above embodiments of the present invention are applied may be defined, and the above embodiments may be applied according to the corresponding types of strips.

In the above-described embodiment, the method is described based on a flowchart having a series of steps or units, but the present invention is not limited to the order of the steps, and some steps may be performed simultaneously with other steps or may be performed in a different order from other steps. Furthermore, it will be understood by those of ordinary skill in the art that steps in the flowcharts are not mutually exclusive, and other steps may be added to the flowcharts, or some steps may be deleted from the flowcharts without affecting the scope of the present invention.

Embodiments include examples of various aspects. It is not possible to describe every conceivable combination of aspects, but one of ordinary skill in the art will recognize that many further combinations are possible. Accordingly, the present invention may be subject to all alternatives, modifications and variations that fall within the scope of the claims.

The embodiments of the present invention can be implemented in the form of program instructions (which can be executed by various computer components and recorded in a computer-readable recording medium). The computer readable recording medium may include individual program instructions, data files, data structures, etc., or a combination of program instructions, data files, data structures, etc. The program instructions recorded in the computer-readable recording medium may be specially designed and constructed for the present invention, or they may be well known to those skilled in the art of computer software technology. Examples of the computer-readable recording medium include: magnetic recording media (such as hard disks, floppy disks, and magnetic tape); an optical data storage medium (such as a CD-ROM or DVD-ROM); magneto-optical media (such as floppy disks); and hardware devices that are specially constructed for storing and carrying out the program instructions (e.g., read Only Memory (ROM), random Access Memory (RAM), flash memory, etc.). Examples of program instructions include not only machine language code, which is formatted by a compiler, but also high-level language code that may be implemented by a computer using an interpreter. The hardware devices may be configured to be operated by one or more software modules to perform the processes according to the invention, and vice versa.

Although the present invention has been described in terms of specific terms (such as detailed elements) and limited embodiments and figures, they are merely provided to facilitate a more general understanding of the present invention, and the present invention is not limited to the above embodiments. Those skilled in the art to which the invention pertains will appreciate that various modifications and changes can be made from the above description.

The spirit of the invention should, therefore, not be limited to the above-described embodiments, and the full scope of the appended claims and equivalents thereof should be accorded the full scope of the invention.

Industrial applicability

The invention can be used for encoding/decoding video.

Claims

1. An image decoding method, comprising:

deriving an intra prediction mode of the current block;

deriving a reference sample point to be used for intra prediction of the current block;

generating a prediction block of the current block by performing intra prediction for the current block based on the intra prediction mode and the reference sample point; and

reconstructing a current block based on the prediction block,

wherein the intra prediction mode of the current block is derived using statistics of intra prediction modes of two or more neighboring blocks of the current block,

wherein the intra prediction mode of a chroma block is derived as an intra prediction mode of a corresponding luma block, which is a luma block comprising samples corresponding to a center position of the chroma block,

Wherein a reference sample point to be used for intra prediction of the current block is derived based on one reconstructed reference sample point line selected from the plurality of reconstructed reference sample point lines,

wherein, in case that the one reconstructed reference sample line is a first reconstructed reference sample line adjacent to the current block, a reference sample point to be used for intra prediction of the current block is derived by applying filtering to the one reconstructed reference sample line, and in case that the one reconstructed reference sample line is a second reconstructed reference sample line adjacent to the current block, the one reconstructed reference sample line is not filtered to derive a reference sample point to be used for intra prediction of the current block.

2. The method according to claim 1,

wherein the minimum and maximum values of the intra prediction modes of the two or more neighboring blocks are used as statistics of the intra prediction modes of the two or more neighboring blocks.

3. The method according to claim 1,

wherein the current block is divided into a plurality of sub-blocks of the same size,

performing intra prediction for each of the plurality of sub-blocks, and

the size and number of the plurality of sub-blocks are determined based on the size of the current block.

4. The method according to claim 1,

wherein the one reconstructed reference sample line is selected based on an identifier included in the bitstream.

5. The method according to claim 1,

wherein the one reconstructed reference sample line is selected based on whether an upper side edge of the current block corresponds to a boundary of a predetermined region.

6. The method according to claim 5, wherein the method comprises,

wherein the predetermined area is a coding tree block.

7. The method according to claim 1,

wherein, in case that the intra prediction mode of the current block is a DC mode, intra prediction is performed using an average value of a plurality of reference samples adjacent to a longer side of the width and height of the current block.

8. An image encoding method, comprising:

determining an intra prediction mode of the current block;

the current block is encoded based on the prediction block,

wherein the intra prediction mode of the current block is encoded using statistics of intra prediction modes of two or more neighboring blocks of the current block,

9. The method according to claim 8,

10. The method according to claim 8,

Performing intra prediction for each of the plurality of sub-blocks, and

11. The method according to claim 8,

wherein an identifier for selecting the one reconstructed reference sample line is encoded into the bitstream.

12. The method according to claim 8,

wherein the one reconstructed reference sample line is selected based on whether the upper side edge of the current block corresponds to a boundary of the coding tree block.

13. The method according to claim 8,

14. A method of transmitting data comprising a bitstream for an image, the method comprising:

obtaining the bit stream for an image; and

the data of the bit stream is transmitted and,

wherein the bit stream is generated by performing the steps of:

determining an intra prediction mode of the current block;

The current block is encoded based on the prediction block,