WO2018047669A1 - Image processing device and image processing method - Google Patents
Image processing device and image processing method Download PDFInfo
- Publication number
- WO2018047669A1 WO2018047669A1 PCT/JP2017/030860 JP2017030860W WO2018047669A1 WO 2018047669 A1 WO2018047669 A1 WO 2018047669A1 JP 2017030860 W JP2017030860 W JP 2017030860W WO 2018047669 A1 WO2018047669 A1 WO 2018047669A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- unit
- prediction
- image
- intra
- orthogonal transform
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
- H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
Definitions
- the present disclosure relates to an image processing device and an image processing method, and more particularly, to an image processing device and an image processing method capable of improving encoding efficiency when performing intra BC prediction processing.
- a coding device that performs coding using HEVC High Efficiency Video Coding performs intra prediction processing or inter prediction processing on the current block that is the encoding target block, and generates a prediction block that is a prediction image of the current block. To do. Then, the encoding device generates an encoded stream by orthogonally transforming and quantizing the prediction residual that is the difference between the prediction block and the current block.
- HEVC High Efficiency Video Coding
- the encoded stream generated in this way is inverse quantized and inverse orthogonal transformed in the decoding device. Then, the prediction residual obtained as a result is added to the prediction block, and a decoded image of the current block is generated.
- intra prediction modes called DC intra prediction, Planar intra prediction, and Angular intra prediction are adopted as the prediction modes of the intra prediction process.
- intra BC Intra block copy
- HEVC-SCC Segen Content Coding
- intra BC Intra block copy
- Prediction processing can also be used.
- Non-Patent Document 1 it has been devised to generate a prediction block by performing not only translation but also rotation on an encoded region in a screen (see, for example, Non-Patent Document 1).
- a motion vector indicating the direction and magnitude of the translation but also the rotation angle is included in the encoded stream.
- the technique described in Non-Patent Document 1 can compensate not only for the parallel movement within the screen but also for the movement in the rotational direction, but cannot compensate for a change in shape such as enlargement, reduction, or skew. Therefore, the accuracy of the prediction block cannot be sufficiently improved.
- JVET Joint Video Exploration Team
- ITU-T International Telecommunication Union Telecommunication Standardization Sector
- the present disclosure has been made in view of such a situation, and is intended to improve the encoding efficiency when performing the intra BC prediction process.
- the orthogonal transform matrix for intra BC prediction processing is used to generate the predicted image. It is an image processing apparatus provided with the conversion part which performs orthogonal transformation with respect to the residual of the image of encoding object, and the said prediction image.
- the image processing method according to the first aspect of the present disclosure corresponds to the image processing apparatus according to the first aspect of the present disclosure.
- the encoding target when a predicted image is generated by performing intra BC prediction processing on an image to be encoded, is generated using an orthogonal transform matrix for intra BC prediction processing. Orthogonal transformation is performed on the residual between the predicted image and the predicted image.
- the inverse orthogonal transformation matrix for intra BC prediction processing is used to generate the prediction image. It is an image processing apparatus provided with the inverse transformation part which performs inverse orthogonal transformation with respect to the coefficient after orthogonal transformation of the residual of the image of decoding object, and the said prediction image.
- the image processing method according to the second aspect of the present disclosure corresponds to the image processing apparatus according to the second aspect of the present disclosure.
- an inverse orthogonal transformation matrix for intra BC prediction processing is used to generate the prediction target image.
- Inverse orthogonal transformation is performed on the coefficient after the orthogonal transformation of the residual between the image and the predicted image.
- the image processing apparatuses according to the first and second aspects can be realized by causing a computer to execute a program.
- a program to be executed by a computer is provided by being transmitted via a transmission medium or by being recorded on a recording medium. Can do.
- orthogonal transformation can be performed. Moreover, according to the 1st side surface of this indication, the encoding efficiency at the time of performing an intra BC prediction process can be improved.
- inverse orthogonal transform can be performed. Moreover, according to the 2nd side surface of this indication, the encoding efficiency at the time of performing an intra BC prediction process can be improved.
- CU It is a figure explaining formation of CU. It is a block diagram which shows the structural example of an image coding apparatus. It is a figure explaining an intra BC prediction process. It is a block diagram which shows the structural example of a conversion part. It is a figure explaining the selection method of the 1st orthogonal transformation matrix. It is a flowchart explaining an image coding process. It is a flowchart explaining a flag setting process. It is a flowchart explaining a conversion process. It is a flowchart explaining optimal both conversion type determination processing. It is a block diagram which shows the structural example of an image decoding apparatus. It is a block diagram which shows the structural example of the inverse transformation part of FIG. It is a flowchart explaining an image decoding process.
- ⁇ First embodiment> (Description of CU formation)
- MPEG2 Moving Picture Experts Group 2 (ISO / IEC 13818-2)
- MPEG-4 Part10 Advanced Video Coding, hereinafter referred to as AVC
- the macro block is a block having a uniform size of 16 ⁇ 16 pixels.
- encoding processing is executed in processing units (coding units) called CU (Coding Unit).
- the CU is a block having a variable size formed by recursively dividing an LCU (Largest Coding Unit) that is a maximum coding unit.
- the maximum selectable CU size is 64x64 pixels.
- the minimum selectable CU size is 8x8 pixels.
- the minimum size CU is called a SCU (Smallest Coding Unit). Note that the maximum size of the CU is not limited to 64 ⁇ 64 pixels, and may be a larger block size such as 128 ⁇ 128 pixels or 256 ⁇ 256 pixels.
- Prediction processing for predictive coding is executed in a processing unit called PU (Prediction Unit).
- a PU is formed by dividing a CU with one of several division patterns.
- the PU is composed of processing units called PB (Prediction Block) for each luminance (Y) and color difference (Cb, Cr).
- PB Prediction Block
- Cb, Cr color difference
- the orthogonal transformation process is executed in a processing unit called TU (Transform Unit).
- a TU is formed by dividing a CU or PU to a certain depth.
- the TU is composed of processing units (transform blocks) called TB (Transform Block) for each luminance (Y) and color difference (Cb, Cr).
- block may be used as a partial area or processing unit of an image (picture) (not a block of a processing unit).
- the “block” in this case indicates an arbitrary partial area in the picture, and its size, shape, characteristics, and the like are not limited.
- the “block” in this case includes any partial area (processing unit) such as TB, TU, PB, PU, SCU, CU, LCU (CTB), sub-block, macroblock, tile, or slice. Is included.
- FIG. 1 is a diagram for explaining formation of a CU in the first embodiment.
- CU formation in the first embodiment is performed by a technique called QTBT (Quad tree plus binary tree) described in JVET-C0024, “EE2.1:“ Quad tree plus plus binary tree tree structure integration with JEM tools ”.
- QTBT Quad tree plus binary tree
- FIG. 2 is a block diagram illustrating a configuration example of the first embodiment of an image encoding device as an image processing device to which the present disclosure is applied.
- the image encoding apparatus 100 in FIG. 2 is an apparatus that encodes a prediction residual between an image and its predicted image, such as AVC and HEVC.
- the image encoding apparatus 100 is equipped with the HEVC technology or the technology proposed by JVET.
- FIG. 2 shows main components such as a processing unit and a data flow, and the ones shown in FIG. 2 are not all. That is, in the image coding apparatus 100, there may be a processing unit that is not shown as a block in FIG. 2, or there may be a process or data flow that is not shown as an arrow or the like in FIG.
- the image coding apparatus 100 performs coding for each CU on a picture that is an input frame-based moving image.
- control unit 101 of the image encoding device 100 sets a fixed value among encoding parameters (header information Hinfo, prediction information Pinfo, conversion information Tinfo, and the like) based on an external input or the like. .
- the control unit 101 supplies a fixed value of the header information Hinfo to each unit, for example, supplies a fixed value of the prediction information Pinfo, for example, to the prediction unit 119, and sets a fixed value of the conversion information Tinfo, for example, the conversion unit 112, the quantum To the quantization unit 113, the inverse quantization unit 115, and the inverse transformation unit 116.
- control unit 101 (setting unit) obtains the optimum value of the variable value from the encoding unit 112, the quantization unit 113, and the prediction unit 119, and performs encoding including the optimum value and the fixed value. Set the parameters.
- the control unit 101 supplies the set encoding parameter to the encoding unit 114, supplies the optimal value of the variable value of the prediction information Pinfo to the arithmetic unit 111, and dequantizes the optimal value of the variable value of the conversion information Tinfo.
- the unit 115 and the inverse transform unit 116 To the unit 115 and the inverse transform unit 116.
- the header information Hinfo includes, for example, a video parameter set (VPS (Video Parameter Set)), a sequence parameter set (SPS (Sequence Parameter Set)), a picture parameter set (PPS (Picture Parameter Set)), a slice header (SH), etc.
- a video parameter set VPS (Video Parameter Set)
- SPS Sequence Parameter Set
- PPS Picture Parameter Set
- SH slice header
- SPS.IntraBCflag which is a fixed value indicating whether to enable intra BC prediction processing, or a fixed value indicating whether to enable the first orthogonal transform (primary transform).
- a certain SPS.AMTflag, SPS.NSSTflag which is a fixed value indicating whether or not to enable the second orthogonal transformation (secondary transformation), and the like are included.
- the content of the header information Hinfo is arbitrary, and any information other than the above-described example may be included in the header information Hinfo.
- the prediction information Pinfo includes, for example, a split flag, which is variable information indicating the presence or absence of horizontal or vertical division in each division hierarchy at the time of PU (CU) formation.
- the prediction information Pinfo includes a variable mode indicating whether the PU prediction process is an intra prediction processing system process (intra prediction process, intra BC prediction process) or an inter prediction process for each PU.
- Information pred_mode_flag is included.
- the prediction information Pinfo includes a variable PU.IntraBCflag (PU.IntraBCmode) indicating whether or not the PU prediction processing is intra BC prediction processing. .
- PU.IntraBCflag indicates that it is an intra BC prediction process
- the prediction information Pinfo includes a variable motion vector used for the intra BC prediction process.
- the prediction information Pinfo includes variable information indicating intra prediction modes called DC intra prediction, Planar intra prediction, and Angular intra prediction.
- the prediction information Pinfo includes a variable motion vector used in the inter prediction process.
- the content of the prediction information Pinfo is arbitrary, and any information other than the example described above may be included in the prediction information Pinfo.
- the conversion information Tinfo includes TB size TBSize, which is variable information indicating the size of TB. Further, when SPS.AMTflag indicates that the first orthogonal transform is to be enabled, the conversion information Tinfo includes TU.TSflag indicating whether or not to skip the first orthogonal transform. When TU.TSflag indicates that the first orthogonal transform is not skipped, the transform information Tinfo includes TU.AMTflag indicating the first orthogonal transform mode, and the first orthogonal transform in the vertical and horizontal directions. TU.AMTIdx indicating an index specifying the first orthogonal transformation matrix to be used is included.
- the transformation information Tinfo includes TU.ROTIdx indicating an index for identifying the second orthogonal transformation matrix used for the second orthogonal transformation. included.
- the content of the conversion information Tinfo is arbitrary, and any information other than the example described above may be included in the conversion information Tinfo.
- the calculation unit 111 Based on the optimal value of split flag of the prediction information Pinfo, the calculation unit 111 sequentially sets the pictures that are moving images in units of frames as the encoding target pictures, and the encoding target pictures are the encoding target pictures.
- Set CU PU, TU.
- the calculation unit 111 subtracts the prediction image P (prediction block) of the PU supplied from the prediction unit 119 from the image I (current block) of the PU to be encoded to obtain the prediction residual D, and converts it to the conversion unit 112.
- the conversion unit 112 determines each value as a variable value candidate of the conversion information Tinfo based on the fixed value of the encoding parameter supplied from the control unit 101. For each value that is a candidate for a variable value of the conversion information Tinfo, the conversion unit 112 performs a first orthogonal transformation or a second conversion on the prediction residual D supplied from the calculation unit 111 based on the value and the fixed value. A transformation process for performing orthogonal transformation is performed, and a transformation coefficient Coeff is derived.
- the conversion unit 112 calculates the RD (Rate Distortion) cost of the CU to be encoded based on the conversion coefficient Coeff or the like for each value that is a variable value candidate of the conversion information Tinfo. Then, the conversion unit 112 determines a value as a variable value candidate of the conversion information Tinfo with the smallest RD cost as an optimum value of the variable value of the conversion information Tinfo, and supplies it to the control unit 101. The conversion unit 112 supplies a conversion coefficient Coeff corresponding to the optimum variable value of the conversion information Tinfo to the quantization unit 113.
- the quantization unit 113 determines each value that is a candidate for a variable value of the conversion information Tinfo based on the fixed value of the encoding parameter supplied from the control unit 101.
- the quantization unit 113 scales (quantizes) the transform coefficient Coeff supplied from the transform unit 112 based on the value and the fixed value for each variable value candidate of the transform information Tinfo, and performs quantization transform.
- the coefficient level level is derived.
- the quantization unit 113 calculates the RD cost of the CU to be encoded based on the quantized transform coefficient level level and the like for each value that becomes a variable value candidate of the transform information Tinfo.
- the quantization unit 113 determines a value that is a candidate for a variable value of the conversion information Tinfo with the smallest RD cost as an optimum value of the variable value of the conversion information Tinfo, and supplies it to the control unit 101.
- the quantization unit 113 supplies the quantized transform coefficient level level corresponding to the variable value optimum value of the transform information Tinfo to the encoding unit 114 and the inverse quantization unit 115.
- the encoding unit 114 encodes the quantized transform coefficient level level and the like supplied from the quantization unit 113 by a predetermined method. For example, the encoding unit 114 is supplied from the quantization unit 113 and the encoding parameters (header information Hinfo, prediction information Pinfo, conversion information Tinfo, etc.) supplied from the control unit 101 in accordance with the definition of the syntax table. The quantized transform coefficient level level is converted to the syntax value of each syntax element. Then, the encoding unit 114 encodes each syntax value (for example, arithmetic encoding) and generates a bit string obtained as a result. For example, the encoding unit 114 multiplexes the encoded bit string (encoded data) of each syntax element, and outputs the result as an encoded stream.
- the encoding unit 114 multiplexes the encoded bit string (encoded data) of each syntax element, and outputs the result as an encoded stream.
- the inverse quantization unit 115 scales (reverses) the value of the quantization transform coefficient level level supplied from the quantization unit 113 based on the fixed value and the optimum variable value supplied from the control unit 101. Quantization) to derive a transform coefficient Coeff_IQ after inverse quantization.
- the inverse quantization unit 115 supplies the transform coefficient Coeff_IQ to the inverse transform unit 116.
- the inverse quantization performed by the inverse quantization unit 115 is an inverse process of the quantization performed by the quantization unit 113.
- the inverse transform unit 116 performs a first inverse orthogonal transform on the transform coefficient Coeff_IQ supplied from the inverse quantization unit 115 based on the fixed value and the optimum variable value supplied from the control unit 101. And inverse transform processing for performing the second inverse orthogonal transform is performed to derive a prediction residual D ′.
- the inverse transform unit 116 supplies the prediction residual D ′ to the calculation unit 117.
- the inverse transform process performed by the inverse transform unit 116 is an inverse process of the transform process performed by the transform unit 112, and is the same process as the inverse transform process performed in the image decoding apparatus described later. Therefore, this inverse transform process will be described later in the description of the image decoding apparatus.
- the calculation unit 117 adds the prediction residual D ′ supplied from the inverse conversion unit 116 and the prediction image P corresponding to the prediction residual D ′ supplied from the prediction unit 119 to add a local decoded image. Derive Rec.
- the computing unit 117 supplies the local decoded image Rec to the frame memory 118.
- the frame memory 118 reconstructs a decoded image in units of pictures using the local decoded image Rec supplied from the calculation unit 117 and stores it in a buffer in the frame memory 118.
- the frame memory 118 reads the decoded image specified by the prediction unit 119 from the buffer as a reference image, and supplies the read image to the prediction unit 119.
- the frame memory 118 may store header information Hinfo, prediction information Pinfo, conversion information Tinfo, and the like related to the generation of a decoded image in a buffer in the frame memory 118.
- the prediction unit 119 determines a value that is a candidate for a variable value of the prediction information Pinfo based on the fixed value of the encoding parameter supplied from the control unit 101.
- the prediction unit 119 acquires, as a reference image, a decoded image stored in the frame memory 118 based on the value and the fixed value for each value that is a variable value candidate of the prediction information Pinfo, and uses the reference image. Intra BC prediction processing, intra prediction processing, or inter prediction processing is performed. Thereby, the predicted image P is generated.
- the prediction unit 119 sets a value that is a candidate of PU.IntraBCflag as 1 indicating intra BC prediction processing, and intra BC. It is determined as 0 which does not indicate the prediction process. And the prediction part 119 performs an intra BC prediction process based on 1 as a value which becomes a candidate of PU.IntraBCflag.
- the reference image is a decoded image obtained by locally decoding a picture including the image I.
- the reference image is a picture (pre-decoded image) decoded before the picture including the image I, and is a decoded image obtained by decoding the entire screen.
- the intra BC prediction process is a process of generating a predicted image P by translating a decoded PU-sized block existing at a position away from the PU by a motion vector in the same picture as the PU.
- the intra prediction process is a process of generating, as a predicted image P, a decoded PU size block that exists in the same picture as the PU in the direction indicated by the intra prediction mode with respect to the PU.
- the inter prediction process generates a predicted image P by translating a decoded PU size block existing at a position separated from the PU by a motion vector in a picture decoded before the picture including the PU. It is processing.
- the prediction unit 119 calculates the RD cost of the CU to be encoded based on the predicted image P or the like for each value that is a candidate for a variable value of the prediction information Pinfo. Then, the prediction unit 119 determines a value that is a candidate for a variable value of the prediction information Pinfo with the smallest RD cost as an optimum value of the variable value of the prediction information Pinfo, and supplies the optimum value to the control unit 101. The prediction unit 119 supplies the prediction image P corresponding to the optimum variable value of the prediction information Pinfo to the calculation unit 111 and the calculation unit 117.
- FIG. 3 is a diagram illustrating the intra BC prediction process performed by the prediction unit 119 of FIG.
- the upper and left regions 152 of the PU 151 have already been encoded and decoded, but the lower side of the PU 151.
- the right region 153 is not yet encoded. That is, the reference image is a decoded image in which only the region 152 is locally decoded.
- the prediction unit 119 performs intra BC prediction processing on the PU 151 using a reference image based on each value that is a candidate for a motion vector in the prediction information Pinfo.
- the prediction unit 119 calculates an RD cost based on the predicted image P corresponding to each value as a motion vector candidate generated as a result, and sets a value as a motion vector candidate that minimizes the RD cost as an optimum value. To decide.
- the motion vector 154 is determined as the optimal value of the motion vector in the intra BC prediction process. Therefore, the predicted image P corresponding to this optimum value is a decoded image of a PU-sized decoded block 155 that exists at a position separated from the PU 151 in the picture 150 by the motion vector 154.
- the prediction unit 119 is not an RD cost, but a motion vector candidate that minimizes SAD (Sum of Absolute Difference) and SATD (Sum of Absolution Transformed Difference) between the predicted image P and the image I of the PU to be encoded May be determined as an optimum value.
- FIG. 4 is a block diagram illustrating a configuration example of the conversion unit 112 in FIG.
- a selection unit 171 includes a selection unit 172, a vertical DCT2 conversion unit 173-1, a vertical DCT5 conversion unit 173-2, a vertical DCT8 conversion unit 173-3, a vertical DST1 conversion unit 173-4, and a vertical DST7.
- Conversion unit 173-5 selection unit 174, horizontal DCT2 conversion unit 175-1, horizontal DCT5 conversion unit 175-2, horizontal DCT8 conversion unit 175-3, horizontal DST1 conversion unit 175-4, horizontal DST7 conversion unit 175-5,
- the selection unit 176, NSST0 conversion unit 177-1, NSST1 conversion unit 177-2, NSST2 conversion unit 177-3, no conversion unit 177-4, and determination unit 178 are configured.
- the selection unit 171 of the conversion unit 112 is supplied with the prediction residual D from the calculation unit 111 in FIG. 2 and the header information Hinfo and the like from the control unit 101.
- the selection unit 171 determines each value as a candidate for TU.TSflag based on SPS.AMTflag in the header information Hinfo.
- the selection unit 171 supplies the prediction residual D to the selection unit 172 or the selection unit 176 based on each value that is a candidate for TU.TSflag.
- the selection unit 171 supplies the prediction residual D to the selection unit 176. Thereby, the first orthogonal transform is skipped.
- the selection unit 171 supplies the prediction residual D to the selection unit 172.
- the selection unit 172 determines each value as a candidate for TU.AMTflag and TU.AMTIdx in the vertical direction.
- the selection unit 172 performs first orthogonal transformation in the vertical direction on the basis of the values, prediction information Pinfo and the like supplied from the control unit 101, for each value that is a candidate for vertical TU.AMTflag and TU.AMTIdx Select a matrix. Then, the selection unit 172 converts the prediction residual D supplied from the selection unit 171 into the selected first orthogonal transformation matrix in the vertical direction for each value that is a candidate for the vertical TU.AMTflag and TU.AMTIdx.
- Vertical DCT2 conversion unit 173-1, vertical DCT5 conversion unit 173-2, vertical DCT8 conversion unit 173-3, vertical DST1 conversion unit 173-4, and vertical DST7 conversion unit 173-5 are: Respectively, AMT (Adaptive Multi Transform) is performed as the first orthogonal transformation in the vertical direction with respect to the prediction residual D using a predetermined first orthogonal transformation matrix.
- the first orthogonal transformation matrix used in the vertical DCT2 conversion unit 173-1, the vertical DCT5 conversion unit 173-2, the vertical DCT8 conversion unit 173-3, the vertical DST1 conversion unit 173-4, and the vertical DST7 conversion unit 173-5 is , DCT2, DCT5, DCT8, DST1, DST7, respectively.
- JVET-C1001 Joint Exploration Test Model 3
- ISO / IEC JTC1 / SC29 / WG11,26 It is described in May-1 June 2016.
- the vertical DCT2 conversion unit 173-1, the vertical DCT5 conversion unit 173-2, the vertical DCT8 conversion unit 173-3, the vertical DST1 conversion unit 173-4, and the vertical DST7 conversion unit 173-5 include a first orthogonal transform in the vertical direction.
- the conversion coefficient obtained as a result of is supplied to the selection unit 174.
- the selection unit 174 determines each value as a candidate for the horizontal TU.AMTflag and TU.AMTIdx.
- the selection unit 174 performs, for each value that is a candidate for the horizontal TU.AMTflag and TU.AMTIdx, the first orthogonal transformation in the horizontal direction based on the value, the prediction information Pinfo supplied from the control unit 101, and the like. Select a matrix.
- the selection unit 174 uses, for each value that becomes a candidate for the horizontal TU.AMTflag and TU.AMTIdx, the conversion coefficient obtained as a result of the first orthogonal transformation in the vertical direction, This is supplied to the horizontal DCT2 conversion unit 175-1, horizontal DCT5 conversion unit 175-2, horizontal DCT8 conversion unit 175-3, horizontal DST1 conversion unit 175-4, or horizontal DST7 conversion unit 175-5 corresponding to the orthogonal transformation matrix.
- Horizontal DCT2 conversion unit 175-1, horizontal DCT5 conversion unit 175-2, horizontal DCT8 conversion unit 175-3, horizontal DST1 conversion unit 175-4, and horizontal DST7 conversion unit 175-5 are:
- AMT is performed as the first orthogonal transformation in the horizontal direction on the transformation coefficient obtained as a result of the first orthogonal transformation in the vertical direction using a predetermined first orthogonal transformation matrix.
- the first orthogonal transform matrix used in the horizontal DCT2 conversion unit 175-1, horizontal DCT5 conversion unit 175-2, horizontal DCT8 conversion unit 175-3, horizontal DST1 conversion unit 175-4, and horizontal DST7 conversion unit 175-5 is , DCT2, DCT5, DCT8, DST1, DST7, respectively.
- the horizontal DCT2 conversion unit 175-1, the horizontal DCT5 conversion unit 175-2, the horizontal DCT8 conversion unit 175-3, the horizontal DST1 conversion unit 175-4, and the horizontal DST7 conversion unit 175-5 are first orthogonal transforms in the horizontal direction.
- the conversion coefficient obtained as a result of is supplied to the selection unit 176.
- the selection unit 176 determines each value as a candidate for TU.ROTIdx based on the prediction information Pinfo supplied from the control unit 101.
- the selection unit 176 selects the TU.ROTIdx candidate and Is determined to be 0 to 3.
- the selection unit 176 selects the TU.
- the candidate value of ROTIdx is determined to be 3 which is an index for specifying the non-transformed matrix as the second orthogonal transform matrix.
- the second orthogonal transform can be performed.
- the intra BC prediction process or the inter prediction process is performed, the second orthogonal transform cannot be performed.
- the characteristic of the prediction residual D depends on the intra prediction mode, so that redundancy based on the intra prediction mode remains in the transform coefficient obtained as a result of the first orthogonal transform. . Therefore, in this case, the second orthogonal transform can be further performed to remove the redundancy.
- the intra BC prediction process is a process of generating, as a predicted image, a block that exists at a position in the screen that is separated from the encoding target PU by a motion vector, and blocks around the encoding target PU are defined as a predicted image.
- the characteristics are different from the intra prediction process.
- the second orthogonal transformation matrix suitable for removing the redundancy of the prediction residual D when the intra prediction process is performed is the redundancy of the prediction residual D when the intra BC prediction process is performed. Not suitable for removing. Therefore, in the case where the intra BC prediction process is performed in the transform unit 112, the second orthogonal transform matrix candidates are only non-transformed matrices and the second orthogonal transform is not performed, as in the inter prediction process. That is, the second orthogonal transformation matrix candidate for intra BC prediction processing is the same as the second orthogonal transformation matrix candidate for inter prediction processing, unlike the second orthogonal transformation matrix candidate for intra processing. Only certain non-transformed matrices are made.
- the selection unit 176 converts the supplied transform coefficient or prediction residual D based on each value that is a candidate for TU.ROTIdx into the NSST0 conversion unit 177-1, NSST1 conversion unit 177-2, NSST2 conversion unit 177-3. , And no-conversion unit 177-4. Specifically, when the candidate values for TU.ROTIdx (TU.NSSTIdx) are 0, 1, 2, and 3, respectively, the selection unit 176 selects the conversion coefficient and the NSST0 conversion unit 177-1. , NSST1 conversion unit 177-2, NSST2 conversion unit 177-3, non-conversion unit 177-4.
- NSST0 conversion unit 177-1, NSST1 conversion unit 177-2, NSST2 conversion unit 177-3, and non-conversion unit 177-4 include a second orthogonal conversion specified by a predetermined index.
- MD-NSST Mode dependent non-separable secondary transforms
- JVET-C1001 Joint Joint Exploration Test Model 3
- JVET Standard of ITU-T Standard SG16 Standard WP3 Standard JVET Standard of ITU-T Standard SG16 Standard WP3 Standard
- ISO / IEC Standard JTC1 / SC29 / WG11 26 Standard May-1 June It is described in 2016.
- the second orthogonal transformation matrix specified by each index used in the NSST0 conversion unit 177-1, NSST1 conversion unit 177-2, and NSST2 conversion unit 177-3 differs depending on, for example, the intra prediction mode.
- NSST0 transform unit 177-1, NSST1 transform unit 177-2, NSST2 transform unit 177-3, and no transform unit 177-4 supply transform coefficients obtained as a result of the second orthogonal transform to determination unit 178.
- the second orthogonal transformation matrix used in the non-conversion unit 177-4 is a non-conversion matrix
- the transform coefficient supplied from the non-conversion unit 177-4 to the determination unit 178 is supplied from the selection unit 176. It is the conversion coefficient itself.
- the determination unit 178 determines the variable value of the conversion information Tinfo based on the conversion coefficients supplied from the NSST0 conversion unit 177-1, NSST1 conversion unit 177-2, NSST2 conversion unit 177-3, and non-conversion unit 177-4. An RD cost corresponding to each candidate value is calculated. Then, the determination unit 178 determines a value that is a candidate for a variable value of the conversion information Tinfo that minimizes the RD cost as an optimum value. The determination unit 178 supplies the optimal value of the variable value of the conversion information Tinfo to the control unit 101 in FIG. 2, and supplies the conversion coefficient corresponding to the optimal value to the quantization unit 113 as the conversion coefficient Coeff.
- the second orthogonal transform may be enabled when the intra BC prediction process is performed.
- three second orthogonal transform matrices suitable for removing redundancy based on the prediction residual D of the predicted image P generated by the intra BC prediction process are the second orthogonal transforms for the intra BC prediction process.
- each of the candidates for the second orthogonal transformation matrix for intra BC prediction processing becomes the second orthogonal transformation matrix specified by indices 0 to 3, and TU.ROTIdx
- the candidate values are determined to be 0 to 3.
- the candidate for the second orthogonal transform matrix for intra BC prediction processing is different from the candidate for the second orthogonal transform matrix for intra prediction processing. Therefore, when the intra BC prediction process is performed, the transform unit 112 can perform the second orthogonal transform suitable for the intra BC prediction process.
- FIG. 5 is a diagram for explaining a method of selecting the first orthogonal transformation matrix.
- the selection unit 172 determines 0 and 1 as candidates for the vertical PU.AMTflag. In addition, when the value that is a candidate for vertical PU.AMTflag is 0, the selection unit 172 does not determine a value that becomes a candidate for vertical TU.AMTIdx, and becomes a candidate for vertical PU.AMTflag. If the value is 1, the candidate values for TU.AMTIdx in the vertical direction are determined to be 0 and 1.
- the selection unit 172 selects the first orthogonal orthogonal transformation matrix from the candidates for the first orthogonal orthogonal transformation matrix for intra prediction processing illustrated in A of FIG.
- the selection unit 172 selects DCT2 as the first orthogonal transformation matrix in the vertical direction (Vertical) when the candidate value of TU.AMTflag is 0.
- the selection unit 172 selects the vertical direction based on the information indicating the intra prediction mode (Intramode) and the respective values that are candidates for the vertical TU.AMTIdx.
- a first orthogonal transformation matrix is selected.
- the selection unit 172 selects DST7 as the first orthogonal transformation matrix in the vertical direction.
- the selection unit 172 converts DCT5, DCT8, or DST1 into the first orthogonal orthogonal matrix in the vertical direction based on the information indicating the intra prediction mode. Choose as.
- the selection unit 172 selects candidates for the first orthogonal transform matrix in the vertical direction for inter prediction processing illustrated in B of FIG. 5.
- a first orthogonal transformation matrix in the vertical direction is selected from the inside.
- the selection unit 172 selects DCT2 as the first orthogonal transformation matrix in the vertical direction when the candidate value of TU.AMTflag is 0. On the other hand, when the value that is a candidate for TU.AMTflag is 1, the selection unit 172 selects the first orthogonal transformation matrix in the vertical direction based on each value that is a candidate for TU.AMTIdx in the vertical direction.
- the selection unit 172 selects DCT8 as the first orthogonal transformation matrix in the vertical direction.
- the selection unit 172 selects DST7 as the first orthogonal transformation matrix in the vertical direction.
- the selection unit 172 selects the first orthogonal orthogonal transformation matrix from the first orthogonal orthogonal transformation matrix for intra BC prediction processing illustrated in C of FIG.
- B in FIG. 5 is the same as C in FIG. 5, and the selection unit 172 performs the first vertical direction when the intra BC prediction process is performed as in the case where the inter prediction process is performed. Select an orthogonal transformation matrix.
- the selection unit 174 Since the selection of the first orthogonal transformation matrix in the horizontal direction by the selection unit 174 is the same as the selection of the first orthogonal matrix in the vertical direction by the selection unit 172, description thereof will be omitted.
- the first orthogonal transformation matrix in the vertical direction and the horizontal direction selected based on the respective values as candidates for TU.AMTflag and TU.AMTIdx, that is, the candidate for the first orthogonal transformation matrix is performed. It depends on the prediction process.
- the selection method of the first orthogonal transform matrix when the intra BC prediction process is performed is the same as that when the inter prediction process is performed, but is different from the case where the intra prediction process is performed. If it is, it may differ from the case where the inter prediction process is performed. That is, if the candidate for the first orthogonal transform matrix for intra BC prediction processing is different from the candidate for the first orthogonal transform matrix for intra prediction processing, the candidate for the first orthogonal transform matrix for inter prediction processing And may be different.
- the first orthogonal transformation matrix candidate for intra BC prediction processing is the first The degree of energy concentration of the transform coefficient obtained as a result of the orthogonal transform is set to be high. Further, when the first orthogonal transformation matrix candidate for intra BC prediction processing is the same as the first orthogonal transformation matrix candidate for inter prediction processing, TU.AMTflag used in the selection unit 172 and the selection unit 174 is used. And a table indicating the relationship between each candidate value of TU.AMTIdx and the first orthogonal transformation matrix (for example, the table of B of FIG. 5 or the table of C of FIG. 5) is shared during intra BC prediction processing and inter prediction processing. can do. Accordingly, the storage capacity of a memory (not shown) that stores this table can be reduced.
- FIG. 6 is a flowchart for explaining image coding processing of the image coding apparatus 100 in FIG.
- step S101 in FIG. 6 the control unit 101 sets a fixed value among the encoding parameters based on an external input or the like.
- the control unit 101 supplies a fixed value of the header information Hinfo to each unit, for example, supplies a fixed value of the prediction information Pinfo, for example, to the prediction unit 119, and sets a fixed value of the conversion information Tinfo, for example, the conversion unit 112, the quantum To the quantization unit 113, the inverse quantization unit 115, and the inverse transformation unit 116.
- the prediction unit 119 performs a prediction process of optimizing the variable value of the prediction information Pinfo and generating a predicted image P corresponding to the optimal value of the variable value of the prediction information Pinfo.
- the prediction unit 119 supplies the optimal value of the variable value of the prediction information Pinfo to the control unit 101, and supplies the prediction image P corresponding to the optimal value of the variable value of the prediction information Pinfo to the calculation unit 111 and the calculation unit 117.
- the control unit 101 supplies the calculation unit 111 with the optimum value of split flag in the prediction information Pinfo.
- step S103 the calculation unit 111 sets the CU (PU, TU) to be encoded for the picture based on the optimum value of split flag, and is supplied from the image I of the PU to be encoded and the prediction unit 119.
- the difference from the predicted image P is calculated as a predicted residual D.
- the amount of data of the prediction residual D obtained in this way is reduced compared to the original image I. Therefore, the data amount can be compressed as compared with the case where the image I is encoded as it is.
- step S104 the conversion unit 112 performs a conversion process of optimizing the variable value of the conversion information Tinfo and deriving the conversion coefficient Coeff corresponding to the optimal value of the variable value of the conversion information Tinfo. Details of this conversion processing will be described with reference to FIG.
- step S105 the quantization unit 113 quantizes the transform coefficient Coeff supplied from the transform unit 112 on the basis of each variable value candidate of the transform information Tinfo, and derives a quantized transform coefficient level level.
- the conversion unit 112 optimizes the variable value of the conversion information Tinfo, supplies the quantized conversion coefficient level level corresponding to the optimal value of the variable value of the conversion information Tinfo to the inverse quantization unit 115, and also converts the conversion information Tinfo of the conversion information Tinfo.
- the optimum variable value is supplied to the control unit 101.
- the control unit 101 sets an encoding parameter including an optimal value and a fixed value of variable values such as prediction information Pinfo and conversion information Tinfo, and supplies the encoding parameter 114 to the encoding unit 114, and reverses the optimal value of the variable value of conversion information Tinfo. This is supplied to the quantization unit 115 and the inverse transformation unit 116.
- step S106 the inverse quantization unit 115 calculates the quantization transform coefficient level level supplied from the quantization unit 113 based on the fixed value and the optimum variable value of the encoding parameter, and the quantization characteristics in step S105. Inverse quantization with characteristics corresponding to.
- the inverse quantization unit 115 supplies the transform coefficient Coeff_IQ obtained as a result to the inverse transform unit 116.
- step S107 the inverse transform unit 116 performs a step on the transform coefficient Coeff_IQ supplied from the inverse quantization unit 115 based on the fixed value and the optimum variable value supplied from the control unit 101.
- Inverse transformation processing is performed by a method corresponding to the transformation processing in S104 to derive a prediction residual D ′.
- step S108 the calculation unit 117 generates a local decoded image Rec by adding the prediction residual D 'derived by the processing in step S107 to the prediction image P supplied from the prediction unit 119.
- step S109 the frame memory 118 reconstructs a decoded image in units of pictures using the local decoded image Rec obtained by the processing in step S108, and stores it in a buffer in the frame memory 118.
- the encoding unit 114 encodes the quantized transform coefficient level level obtained by the process in step S105.
- the encoding unit 114 encodes a quantized transform coefficient level level, which is information about an image, by arithmetic coding or the like.
- the encoding unit 114 encodes the encoding parameters (header information Hinfo, prediction information Pinfo, conversion information Tinfo) supplied from the control unit 101.
- the encoding unit 114 collects the encoded data generated by such encoding and outputs the encoded data as an encoded stream to the outside of the image encoding apparatus 100. This encoded stream is transmitted to the decoding side via a transmission path or a recording medium, for example.
- step S110 ends, the image encoding process ends.
- FIG. 7 is a flowchart for explaining flag setting processing for setting SPS.IntraBCflag, SPS.AMTflag, and SPS.NSSTflag in the processing of step S101 of FIG.
- step S131 the control unit 101 determines whether or not to enable the IntraBC prediction process based on an external input or the like.
- step S131 If it is determined in step S131 that the IntraBC prediction process is enabled, the process proceeds to step S132.
- step S132 the control unit 101 sets SPS.IntraBCflag to 1, and advances the process to step S134.
- step S131 if it is determined in step S131 that the IntraBC prediction process is not enabled, the process proceeds to step S133.
- step S133 the control unit 101 sets SPS.IntraBCflag to 0, and the process proceeds to step S134.
- step S134 the control unit 101 determines whether to enable the first orthogonal transformation based on an external input or the like.
- step S134 If it is determined in step S134 that the first orthogonal transform is to be enabled, the process proceeds to step S135. In step S135, the control unit 101 sets SPS.AMTflag to 1, and advances the process to step S137.
- step S134 determines whether the first orthogonal transform is enabled. If it is determined in step S134 that the first orthogonal transform is not enabled, the process proceeds to step S136.
- step S136 the control unit 101 sets SPS.AMTflag to 0, and the process proceeds to step S137.
- step S137 the control unit 101 determines whether to enable the second orthogonal transformation based on an external input or the like.
- step S137 If it is determined in step S137 that the second orthogonal transformation is to be enabled, the process proceeds to step S138.
- step S138 the control unit 101 sets SPS.NSSTflag to 1, and ends the process.
- step S137 if it is determined in step S137 that the second orthogonal transformation is not enabled, the process proceeds to step S139.
- step S139 the control unit 101 sets SPS.NSSTflag to 0 and ends the process.
- FIG. 8 is a flowchart for explaining the conversion process in step S104 of FIG.
- the selection unit 171 determines whether the SPS.AMTflag supplied from the control unit 101 is “1”. If it is determined in step S151 that SPS.AMTflag is 1, that is, if the first orthogonal transform is valid, the process proceeds to step S152.
- step S152 the selection unit 176 determines whether or not SPS.NSSTflag supplied from the control unit 101 is 1. If it is determined in step S152 that SPS.NSSTflag is 1, that is, if the second orthogonal transform is valid, the process proceeds to step S153.
- step S153 the conversion unit 112 determines all values as candidate values for TU.TSflag, TU.AMTflag, TU.AMTIdx, and TU.ROTIdx, and generates the conversion coefficient based on these values. Both conversion type determination processes are performed. Details of the optimum both conversion type determination processing will be described with reference to FIG. After the process of step S153, the process proceeds to step S158.
- step S152 determines whether SPS.NSSTflag is not 1, the process proceeds to step S154.
- step S154 the conversion unit 112 determines all values as candidate values of TU.TSflag, TU.AMTflag, and TU.AMTIdx, and determines a value that is a candidate of TU.ROTIdx as 3,
- An optimal first orthogonal transform type determination process for generating a transform coefficient based on the value of is performed.
- the value that is a candidate for TU.ROTIdx is 3, so the second orthogonal transform is not performed.
- Transform coefficients corresponding to 3 as candidate values of TU.TSflag, TU.AMTflag, and TU.AMTId, and TU.ROTIdx candidates generated by the optimal first orthogonal transform type determination process are determined.
- the process proceeds to step S158.
- step S151 If it is determined in step S151 that SPS.AMTflag is not 1, the process proceeds to step S155.
- step S155 the selection unit 176 determines whether SPS.NSSTflag is 1. If it is determined in step S155 that SPS.NSSTflag is 1, that is, if the second orthogonal transform is valid, the process proceeds to step S156.
- step S156 the conversion unit 112 determines the value that is a candidate for TU.TSflag to be 1, determines all values as values that are candidates for TU.ROTIdx, and generates a conversion coefficient based on those values.
- the optimum second orthogonal transformation type determination process is performed. In this optimal first orthogonal transform type determination process, since TU.TSflag is 1, the first orthogonal transform is not performed.
- the transform coefficients corresponding to 1 and TU.ROTIdx candidate values generated by the optimal second orthogonal transform type determination process are supplied to the determination unit 178. Then, the process proceeds to step S158.
- the selection unit 171 determines the candidate value of TU.TSflag as 1 and supplies the prediction residual D to the selection unit 176 for selection.
- the unit 176 determines the candidate value of TU.ROTIdx as 3 and supplies the prediction residual D to the non-conversion unit 177-4.
- step S157 the non-conversion unit 177-4 performs the second orthogonal transform using the non-conversion matrix on the prediction residual D supplied from the selection unit 176, and thus uses the prediction residual D as it is. Conversion factor. Then, the non-conversion unit 177-4 supplies the conversion coefficient, which is the prediction residual D, to the determination unit 178, and the process proceeds to step S158.
- step S158 the determination unit 178 calculates an RD cost based on each supplied conversion coefficient and the like, and converts TU.TSflag, TU.AMTflag, TU.AMTIdx, TU.ROTIdx that minimizes the RD cost.
- the optimum value of the variable value of information Tinfo is determined.
- the determination unit 178 supplies the optimal value of the variable value of the conversion information Tinfo to the control unit 101, and supplies the conversion coefficient corresponding to the optimal value to the quantization unit 113 as the conversion coefficient Coeff.
- the process returns to step S104 in FIG. 6 and proceeds to step S105.
- FIG. 9 is a flowchart for explaining the optimum both conversion type determination process in step S153 of FIG.
- step S171 determines a candidate value of TU.TSflag as 0, and supplies the prediction residual D supplied from the calculation unit 111 to the selection unit 172.
- step S172 the selection unit 172 determines 0 as a candidate value for the TU.AMTflag in the vertical and horizontal directions.
- the selection unit 172 performs the first orthogonal transform in the vertical direction by the method described in FIG. 5 based on mode information pred_mode_flag, PU.IntraBCflag, information indicating the intra prediction mode, and 0 as TU.AMTflag. Select a matrix.
- the selection unit 171 includes a vertical DCT2 conversion unit 173-1, a vertical DCT5 conversion unit 173-2, a vertical DCT8 conversion unit 173-3, and a vertical DST1 conversion unit 173 corresponding to the selected first orthogonal transformation matrix in the vertical direction. 4 or the vertical residual DST7 is supplied to the vertical DST7 converter 173-5.
- step S174 the vertical DCT2 conversion unit 173-1, the vertical DCT5 conversion unit 173-2, the vertical DCT8 conversion unit 173-3, the vertical DST1 conversion unit 173-4, or the vertical DST7 conversion unit 173-5 performs the prediction residual D.
- the first orthogonal transformation is performed in the vertical direction using the first orthogonal transformation matrix selected in step S173.
- Vertical DCT2 conversion unit 173-1, vertical DCT5 conversion unit 173-2, vertical DCT8 conversion unit 173-3, vertical DST1 conversion unit 173-4, or vertical DST7 conversion unit 173-5 selects a conversion coefficient obtained as a result To the unit 174.
- the selection unit 174 performs the first orthogonal transformation in the horizontal direction by the method described in FIG. 5 based on mode information pred_mode_flag, PU.IntraBCflag, information indicating the intra prediction mode, and 0 as TU.AMTflag. Select a matrix.
- the selection unit 171 includes a horizontal DCT2 conversion unit 175-1, a horizontal DCT5 conversion unit 175-2, a horizontal DCT8 conversion unit 175-3, and a horizontal DST1 conversion unit 175 corresponding to the selected horizontal first orthogonal transformation matrix. 4 or the horizontal DST7 conversion unit 175-5 is supplied with a conversion coefficient.
- step S176 the horizontal DCT2 conversion unit 175-1, horizontal DCT5 conversion unit 175-2, horizontal DCT8 conversion unit 175-3, horizontal DST1 conversion unit 175-4, or horizontal DST7 conversion unit 175-5 performs the prediction residual D.
- the first orthogonal transformation is performed in the horizontal direction using the first orthogonal transformation matrix selected in step S175.
- the horizontal DCT2 conversion unit 175-1, horizontal DCT5 conversion unit 175-2, horizontal DCT8 conversion unit 175-3, horizontal DST1 conversion unit 175-4, or horizontal DST7 conversion unit 175-5 selects a conversion coefficient obtained as a result. To the unit 176.
- step S177 the selection unit 176 determines whether intra prediction processing has been performed based on the prediction information Pinfo.
- the selection unit 176 indicates that the mode information pred_mode_flag indicates the processing of the intra prediction processing system and that SPS.IntraBCflag does not enable the intra BC prediction processing, or PU.IntraBCflag indicates the intra BC prediction processing. If it is not, it is determined that intra prediction processing has been performed. On the other hand, when the mode information pred_mode_flag indicates processing of the intra prediction processing system, SPS.IntraBCflag indicates that the intra BC prediction processing is enabled, and PU.IntraBCflag indicates that it is intra BC prediction processing, or mode information When pred_mode_flag indicates the inter prediction process, the selection unit 176 determines that the intra prediction process is not performed.
- step S177 If it is determined in step S177 that the intra prediction process has been performed, the process proceeds to step S178.
- the selection unit 176 determines 0 as a candidate value for TU.ROTIdx.
- step S179 the selection unit 176 determines whether the candidate value for TU.ROTIdx is 3. If it is determined in step S179 that the TU.ROTIdx candidate value is not 3, the selection unit 176 selects the NSST0 conversion unit 177-1 and NSST1 conversion unit 177-2 corresponding to the TU.ROTIdx candidate value. Alternatively, the conversion coefficient is supplied to the NSST2 conversion unit 177-3.
- NSST0 transform section 177-1, NSST1 transform section 177-2, or NSST2 transform section 177-3 performs a second orthogonal transform on the transform coefficient, and determines a transform coefficient obtained as a result. To the unit 178.
- step S181 the selection unit 176 increments the value that becomes a candidate for TU.ROTIdx by 1, and the selection unit 176 repeats the processing in steps S179 to S181 until the value that becomes a candidate for TU.ROTIdx becomes 3. It is.
- step S179 if it is determined in step S179 that the candidate value for TU.ROTIdx is 3, the selection unit 176 supplies the conversion coefficient to the non-conversion unit 177-4.
- the non-transforming unit 177-4 performs the second orthogonal transform on the transform coefficient using the non-transformed matrix as the second orthogonal transform matrix, that is, does not perform the second orthogonal transform.
- the non-conversion unit 177-4 supplies the conversion coefficient supplied from the selection unit 176 as it is to the determination unit 178, and the process proceeds to step S183.
- step S177 If it is determined in step S177 that the intra prediction process is not performed, the process proceeds to step S182.
- the selection unit 176 determines 3 as a TU.ROTIdx candidate value, and supplies the conversion coefficient to the non-conversion unit 177-4.
- the non-transforming unit 177-4 performs the second orthogonal transform on the transform coefficient using the non-transformed matrix as the second orthogonal transform matrix, that is, does not perform the second orthogonal transform.
- the non-conversion unit 177-4 supplies the conversion coefficient supplied from the selection unit 176 as it is to the determination unit 178, and the process proceeds to step S183.
- step S183 the selection unit 171 determines whether the candidate value of TU.TSflag is 0. If it is determined in step S183 that the candidate value for TU.TSflag is 0, the process proceeds to step S184.
- step S184 the selection unit 172 determines whether the candidate value for the TU.AMTflag in the vertical direction and the horizontal direction is 1. If it is determined in step S184 that the candidate value of TU.AMTflag in the vertical direction and the horizontal direction is not 1, the process proceeds to step S185.
- step S185 the selection unit 172 determines the candidate value for the vertical and horizontal TU.AMTflag to be 1, and sets the candidate values for the vertical TU.AMTIdx and the horizontal TU.AMTIdx to 0. decide. And a process returns to the process of step S173 and the subsequent processes are repeated.
- the selection of the first orthogonal transformation matrix in the vertical direction is based not only on the mode information pred_mode_flag, PU.IntraBCflag, information indicating the intra prediction mode, and TU.AMTflag, but also on TU.AMTIdx in the vertical direction. Done. The same applies to the selection of the first orthogonal transformation matrix in the horizontal direction.
- step S184 if it is determined in step S184 that the candidate value for the TU.AMTflag in the vertical and horizontal directions is 1, the process proceeds to step S186.
- step S186 the selection unit 172 determines whether the value that is a candidate for the horizontal TU.AMTIdx is 1.
- step S186 If it is determined in step S186 that the horizontal TU.AMTIdx candidate value is not 1, the process proceeds to step S187.
- step S187 the selection unit 172 increments the value that becomes a candidate for the horizontal TU.AMTIdx by 1, and returns the process to step S173. Then, the subsequent processing is performed in the same manner as after step S185.
- step S186 determines whether the candidate value of TU.AMTIdx in the horizontal direction is 1.
- step S188 If it is determined in step S188 that the vertical TU.AMTIdx candidate value is not 1, the process proceeds to step S189.
- step S189 the selection unit 172 increments the vertical TU.AMTIdx candidate value by 1, determines the horizontal TU.AMTIdx candidate value to 0, and returns the process to step S173. . Then, the subsequent processing is performed in the same manner as after step S185.
- step S190 the selection unit 171 determines whether the candidate value for TU.TSflag is 1. judge. If it is determined in step S190 that the TU.TSflag candidate value is not 1, the process proceeds to step S191.
- step S191 the selection unit 171 determines 1 as a candidate value for TU.TSflag. Then, the selection unit 171 supplies the prediction residual D to the selection unit 176, and returns the process to step S177. At this time, in the process of step S180, the second orthogonal transformation is performed on the prediction residual D.
- step S183 if it is determined in step S183 that the TU.TSflag candidate value is not 0, or if it is determined in step S190 that the TU.TSflag candidate value is 1, the process is as shown in FIG. Returning to step S153, the process proceeds to step S158.
- the candidate values of TU.AMTflag in the horizontal direction and the vertical direction are the same, but they may be different.
- the candidate values for TU.AMTflag in the horizontal direction and the vertical direction are set separately.
- the optimum first orthogonal transform type determination process is performed except that the process of determining 3 as a candidate value of TU.ROTIdx is performed instead of the processes of steps S177 to S182. This is the same as the optimum both conversion type determination process of FIG. Further, in the optimum second orthogonal transformation type determination process, only the processes after step S191 in FIG. 9 are performed.
- the image coding apparatus 100 when performing the intra BC prediction process, performs the first orthogonal transform on the prediction residual D using the first orthogonal transform matrix for the intra BC prediction process. . Therefore, the first orthogonal transformation can be performed on the prediction residual D using the first orthogonal transformation matrix suitable for the intra BC prediction process. Thereby, the energy concentration degree of the transform coefficient obtained as a result of the first orthogonal transform is improved. As a result, encoding efficiency is improved. In addition, since information lost due to quantization is reduced, quantization error is reduced, and as a result, the image quality of the decoded image can be improved.
- the second orthogonal transformation is the same as the first orthogonal transformation, and the same effect can be obtained.
- FIG. 10 is a block diagram illustrating a configuration example of a first embodiment of an image decoding device as an image processing device to which the present technology is applied, which decodes an encoded stream generated by the image encoding device 100 in FIG. 2. is there.
- the image decoding device 200 in FIG. 10 decodes the encoded stream generated by the image encoding device 100 using a decoding method corresponding to the encoding method in the image encoding device 100.
- the image decoding apparatus 200 is equipped with a technique proposed for HEVC and a technique proposed for JVET.
- FIG. 10 shows the main components such as the processing unit and the flow of data, and the ones shown in FIG. 10 are not all. That is, in the image decoding apparatus 200, there may be a processing unit that is not shown as a block in FIG. 10, or there may be a process or data flow that is not shown as an arrow or the like in FIG.
- the image encoding device 100 decodes the encoded stream generated by the image encoding device 100 for each CU.
- the decoding unit 211 of the image decoding device 200 decodes the encoded stream generated by the image encoding device 100 by a predetermined decoding method corresponding to the encoding method in the encoding unit 114.
- the decoding unit 211 decodes the encoding parameters (header information Hinfo, prediction information Pinfo, conversion information Tinfo, etc.) and the quantized transform coefficient level level from the bit stream of the encoded stream according to the definition of the syntax table. .
- the decoding unit 211 divides the LCU based on the split flag included in the encoding parameter, and sequentially sets the CUs corresponding to the respective quantized transform coefficient levels level as decoding target CUs (PU, TU).
- the decoding unit 211 supplies the encoding parameters to each block. For example, the decoding unit 211 supplies the prediction information Pinfo to the prediction unit 216, supplies the transform information Tinfo and the prediction information Pinfo to the inverse quantization unit 212 and the inverse transform unit 213, and supplies the header information Hinfo to each block. In addition, the decoding unit 211 supplies the quantized transform coefficient level level to the inverse quantization unit 212.
- the inverse quantization unit 212 scales (inversely quantizes) the value of the quantized transform coefficient level level supplied from the decoding unit 211 to derive the transform coefficient Coeff_IQ. To do.
- This inverse quantization is an inverse process of quantization performed by the quantization unit 113 (FIG. 2) of the image encoding device 100. Note that the inverse quantization unit 115 (FIG. 2) performs the same inverse quantization as the inverse quantization unit 212.
- the inverse quantization unit 212 supplies the obtained transform coefficient Coeff_IQ to the inverse transform unit 213.
- the inverse transform unit 213 performs an inverse transform process on the transform coefficient Coeff_IQ supplied from the inverse quantization unit 212 based on the encoding parameter supplied from the decoding unit 211, and derives a prediction residual D '.
- This inverse conversion process is an inverse process of the conversion process performed by the conversion unit 112 (FIG. 2) of the image encoding device 100.
- the inverse transform unit 116 performs the same inverse transform as the inverse transform unit 213.
- the inverse transform unit 213 supplies the obtained prediction residual D ′ to the calculation unit 214.
- the calculation unit 214 adds the prediction residual D ′ supplied from the inverse conversion unit 213 and the prediction image P corresponding to the prediction residual D ′ to derive a local decoded image Rec.
- the calculation unit 214 reconstructs a decoded image for each picture unit using the obtained local decoded image Rec, and outputs the obtained decoded image to the outside of the image decoding apparatus 200.
- the calculation unit 214 supplies the local decoded image Rec to the frame memory 215 as well.
- the frame memory 215 reconstructs a decoded image for each picture unit using the local decoded image Rec supplied from the calculation unit 214 and stores it in a buffer in the frame memory 215.
- the frame memory 215 reads the decoded image specified by the prediction unit 216 from the buffer as a reference image, and supplies the read image to the prediction unit 216. Further, the frame memory 215 may store header information Hinfo, prediction information Pinfo, conversion information Tinfo, and the like related to generation of the decoded image in a buffer in the frame memory 215.
- the prediction unit 216 acquires the decoded image stored in the frame memory 215 as a reference image based on the prediction information Pinfo supplied from the decoding unit 211, and uses the reference image to perform an intra BC prediction process and a predetermined intra Intra prediction processing or inter prediction processing in the prediction mode is performed.
- the prediction unit 216 supplies the predicted image P generated as a result to the calculation unit 214.
- FIG. 11 is a block diagram illustrating a configuration example of the inverse transform unit 213 in FIG.
- 11 includes a selection unit 231, an NSST0 inverse conversion unit 232-1, an NSST1 inverse conversion unit 232-2, an NSST2 inverse conversion unit 232-3, a non-conversion unit 232-4, a selection unit 233, and a selection unit.
- horizontal DCT2 inverse transform unit 235-1 horizontal DCT5 inverse transform unit 235-2, horizontal DCT8 inverse transform unit 235-3, horizontal DST1 inverse transform unit 235-4, horizontal DST7 inverse transform unit 235-5, selection unit 236 , A vertical DCT2 inverse transform unit 237-1, a vertical DCT5 inverse transform unit 237-2, a vertical DCT8 inverse transform unit 237-3, a vertical DST1 inverse transform unit 237-4, and a vertical DST7 inverse transform unit 237-5.
- the selection unit 231 of the inverse conversion unit 213 is supplied with header information Hinfo, prediction information Pinfo, conversion information Tinfo, and the like from the decoding unit 211 of FIG. Further, the transform coefficient Coeff_IQ is supplied from the inverse quantization unit 212 to the selection unit 231.
- the selection unit 231 outputs the conversion coefficient Coeff_IQ based on SPS.NSSTflag in the header information Hinfo, TU.ROTIdx in the conversion information Tinfo, and the like.
- the selection unit 231 outputs the transform coefficient Coeff_IQ to the non-transform unit 232-4.
- the selection unit 231 selects TU.ROTIdx as an index of the second orthogonal transformation matrix.
- Selection section 231 outputs transform coefficient Coeff_IQ to NSST0 inverse transform section 232-1, NSST1 inverse transform section 232-2, NSST1 inverse transform section 232-2, or no transform section 232-4 corresponding to the selected index.
- the selection unit 231 converts the transform coefficient Coeff_IQ into the NSST0 inverse transform unit 232-1 and NSST1 inverse transform unit 232-2. , NSST2 inverse conversion unit 232-3 and non-conversion unit 232-4.
- NSST0 inverse transform unit 232-1, NSST1 inverse transform unit 232-2, NSST2 inverse transform unit 232-3, and non-transform unit 232-4 are identified by a predetermined index.
- a second inverse orthogonal transform is performed for every 4 ⁇ 4 pixels on the transform coefficient Coeff_IQ using a second inverse orthogonal transform matrix corresponding to 2 orthogonal transform matrices.
- the second inverse orthogonal transform matrix used in the NSST0 inverse transform unit 232-1, NSST1 inverse transform unit 232-2, and NSST2 inverse transform unit 232-3 differs depending on, for example, the intra prediction mode.
- NSST0 inverse transform unit 232-1, NSST1 inverse transform unit 232-2, NSST2 inverse transform unit 232-3, and non-transform unit 232-4 provide transform coefficients obtained as a result of the second inverse orthogonal transform to selection unit 233.
- Supply since the second inverse orthogonal transformation matrix used in the non-conversion unit 232-4 is a non-conversion matrix, the transform coefficient supplied from the non-conversion unit 232-4 to the selection unit 233 is supplied from the selection unit 231.
- the conversion coefficient Coeff_IQ itself.
- the selection unit 233 determines whether to perform the first inverse orthogonal transformation based on SPS.AMTflag in the header information Hinfo supplied from the decoding unit 211, TU.TSflag in the transformation information Tinfo, and the like.
- a transform coefficient obtained as a result of the second inverse orthogonal transform is supplied to the selection unit 234 or the calculation unit 214.
- the selection unit 233 determines that the first inverse orthogonal transform is not performed. Then, the selection unit 233 supplies the transform coefficient obtained as a result of the second inverse orthogonal transform to the calculation unit 214 as it is as the prediction residual D ′.
- the selection unit 233 determines whether the first orthogonal transformation is enabled and TU.TSflag is 0 indicating that the first orthogonal transformation is not skipped. If SPS.AMTflag is 1 indicating that the first orthogonal transformation is enabled and TU.TSflag is 0 indicating that the first orthogonal transformation is not skipped, the selection unit 233 , It is determined to perform the first inverse orthogonal transform. Then, the selection unit 233 supplies the transform coefficient obtained as a result of the second inverse orthogonal transform to the selection unit 234.
- the selection unit 234 selects the first orthogonal orthogonal transformation matrix based on the horizontal direction TU.AMTflag and TU.AMTIdx, the prediction information Pinfo, and the like in the conversion information Tinfo supplied from the decoding unit 211. Then, the selection unit 234 converts the transformation coefficient supplied from the selection unit 233 into a horizontal DCT2 inverse transformation unit 235-1 and a horizontal DCT5 inverse transformation unit 235-2 corresponding to the selected first horizontal orthogonal orthogonal transformation matrix. , To the horizontal DCT8 inverse transform unit 235-3, the horizontal DST1 inverse transform unit 235-4, or the horizontal DST7 inverse transform unit 235-5.
- Horizontal DCT2 inverse transform unit 235-1, horizontal DCT5 inverse transform unit 235-2, horizontal DCT8 inverse transform unit 235-3, horizontal DST1 inverse transform unit 235-4, and horizontal DST7 inverse transform unit 235-5 uses the first inverse orthogonal transform matrix corresponding to the predetermined first orthogonal transform matrix, respectively, and the first inverse orthogonal in the horizontal direction with respect to the transform coefficient supplied from the selection unit 234 Perform conversion.
- the inverse orthogonal transformation matrices are matrices corresponding to DCT2, DCT5, DCT8, DST1, and DST7, respectively.
- the horizontal DCT2 inverse transform unit 235-1, the horizontal DCT5 inverse transform unit 235-2, the horizontal DCT8 inverse transform unit 235-3, the horizontal DST1 inverse transform unit 235-4, and the horizontal DST7 inverse transform unit 235-5 are selected by the selection unit 234.
- the transform coefficient obtained as a result of performing the first inverse orthogonal transform in the horizontal direction on the transform coefficient supplied from is supplied to the selection unit 236.
- the selection unit 236 selects the first orthogonal orthogonal transformation matrix based on the vertical direction TU.AMTflag and TU.AMTIdx, the prediction information Pinfo, and the like in the conversion information Tinfo supplied from the decoding unit 211.
- the selection unit 236 then converts the supplied transform coefficients into a vertical DCT2 inverse transform unit 237-1, a vertical DCT5 inverse transform unit 237-2, and a vertical DCT8 inverse corresponding to the selected first orthogonal transform matrix in the vertical direction.
- the data is supplied to the conversion unit 237-3, the vertical DST1 inverse conversion unit 237-4, or the vertical DST7 inverse conversion unit 237-5.
- Vertical DCT2 inverse transform unit 237-1, vertical DCT5 inverse transform unit 237-2, vertical DCT8 inverse transform unit 237-3, vertical DST1 inverse transform unit 237-4, and vertical DST7 inverse transform unit 237-5 each perform a first inverse orthogonal transform on the supplied transform coefficients in the vertical direction by using a first inverse orthogonal transform matrix corresponding to a predetermined first orthogonal transform matrix.
- the inverse orthogonal transformation matrices are matrices corresponding to DCT2, DCT5, DCT8, DST1, and DST7, respectively.
- the vertical DCT2 inverse transform unit 237-1, the vertical DCT5 inverse transform unit 237-2, the vertical DCT8 inverse transform unit 237-3, the vertical DST1 inverse transform unit 237-4, and the vertical DST7 inverse transform unit 237-5 are supplied.
- the prediction residual D ′ obtained as a result of performing the first inverse orthogonal transform in the vertical direction on the transform coefficient is supplied to the calculation unit 214.
- the inverse transform unit 213 corresponds to the optimal value based on TU.ROTIdx, which is an index that identifies the optimal value among the candidates for the second orthogonal transform matrix for prediction processing to be performed.
- a second inverse orthogonal transform is performed on the transform coefficient using an inverse orthogonal transform matrix of 2. Therefore, when the intra prediction process is performed, the inverse transform unit 213 performs the second inverse orthogonal transform matrix corresponding to the optimum value among the candidates for the second orthogonal transform matrix for the intra BC prediction process, that is, the intra BC prediction.
- a second inverse orthogonal transform is performed on the transform coefficient using the second inverse orthogonal transform matrix for processing.
- the inverse transform unit 213 sets the optimal value based on the horizontal TU.AMTIdx that is an index for identifying the optimal value of the candidates for the first orthogonal transform matrix in the horizontal direction for prediction processing to be performed.
- a first orthogonal transform is performed in the horizontal direction on the transform coefficient using the corresponding first inverse orthogonal transform matrix. Accordingly, when the intra prediction process is performed, the inverse transform unit 213 performs the first inverse orthogonal transform matrix corresponding to the optimum value among the candidates for the first orthogonal transform matrix for the intra BC prediction process, that is, the intra BC prediction.
- a first inverse orthogonal transform is performed on the transform coefficient using the first inverse orthogonal transform matrix for processing. The same applies to the first inverse orthogonal transform in the vertical direction.
- FIG. 12 is a flowchart for explaining the image decoding process of the image decoding apparatus 200 of FIG.
- step S200 the decoding unit 211 decodes the encoded stream supplied to the image decoding apparatus 200, and obtains an encoding parameter and a quantized transform coefficient level level.
- the decoding unit 211 supplies the encoding parameter to each block.
- the decoding unit 211 supplies the quantized transform coefficient level level to the inverse quantization unit 212.
- step S201 the decoding unit 211 divides the LCU based on the split flag included in the encoding parameter, and sets a CU corresponding to each quantized transform coefficient level level to a decoding target CU (PU, TU). Steps S202 to S211 described later are performed for each CU (PU, TU) to be decoded.
- step S202 the inverse quantization unit 212 derives a transform coefficient Coeff_IQ by inversely quantizing the quantized transform coefficient level level obtained by the process in step S201.
- This inverse quantization is the inverse process of the quantization performed in step S105 (FIG. 6) of the image encoding process, and is the same process as the inverse quantization performed in step S106 (FIG. 6) of the image encoding process. is there.
- step S203 the inverse transform unit 213 performs an inverse transform process on the transform coefficient Coeff_IQ obtained by the process in step S202, and derives a prediction residual D ′.
- This inverse conversion process is an inverse process of the conversion process performed in step S104 (FIG. 6) of the image encoding process, and is the same process as the inverse conversion performed in step S107 (FIG. 6) of the image encoding process. . Details of the inverse conversion process will be described with reference to FIG.
- step S204 the prediction unit 216 determines whether the slice including the PU to be decoded is an I slice based on the encoding parameter supplied from the decoding unit 211. If it is determined in step S204 that the slice is not an I slice, that is, if the slice including the decoding target PU is a P slice or a B slice, the process proceeds to step S205.
- step S205 the prediction unit 216 extracts mode information pred_mode_flag from the encoding parameter.
- step S206 the prediction unit 216 determines whether or not the mode information pred_mode_flag indicates inter prediction processing.
- step S206 If it is determined in step S206 that the mode information pred_mode_flag indicates inter prediction processing, the processing proceeds to step S207.
- step S207 the prediction unit 216 reads a reference image from the frame memory 215 based on the prediction information Pinfo, and performs inter prediction processing using the reference image and a motion vector in the prediction information Pinfo. Then, the prediction unit 216 supplies the predicted image P generated as a result to the calculation unit 214, and the process proceeds to step S211.
- step S204 if it is determined in step S204 that the slice is an I slice, or if it is determined in step S206 that the mode information pred_mode_flag does not indicate inter prediction processing, the processing proceeds to step S208.
- step S208 the prediction unit 216 determines whether or not PU.IntraBCflag in the prediction information Pinfo is 1 indicating that it is an intra BC prediction process. If it is determined in step S208 that PU.IntraBCflag is 1, the process proceeds to step S209.
- step S209 the prediction unit 216 reads a locally decoded decoded image including the decoding target PU as a reference image from the frame memory 215, and uses the reference image and the motion vector in the prediction information Pinfo to perform intra. BC prediction processing is performed. Then, the prediction unit 216 supplies the predicted image P generated as a result to the calculation unit 214, and the process proceeds to step S211.
- step S210 the prediction unit 216 reads a locally decoded decoded image including the decoding target PU as a reference image from the frame memory 215, and uses the reference image to determine the intra prediction mode included in the prediction information Pinfo. Perform intra prediction processing. Then, the prediction unit 216 supplies the predicted image P generated as a result to the calculation unit 214, and the process proceeds to step S211.
- step S211 the calculation unit 214 adds the prediction residual D ′ supplied from the inverse conversion unit 213 and the prediction image P supplied from the prediction unit 216 to derive a local decoded image Rec.
- the calculation unit 214 reconstructs a decoded image for each picture unit using the obtained local decoded image Rec, and outputs the obtained decoded image to the outside of the image decoding apparatus 200.
- the calculation unit 214 supplies the local decoded image Rec to the frame memory 215 as well.
- step S212 the frame memory 215 reconstructs a decoded image for each picture unit using the local decoded image Rec supplied from the calculation unit 214, and stores it in a buffer in the frame memory 215. Then, the process ends.
- FIG. 13 is a flowchart illustrating a flag decoding process for decoding SPS.IntraBCflag, SPS.AMTflag, and SPS.NSSTflag in the process of step S200 of FIG.
- step S231 in FIG. 13 the decoding unit 211 decodes SPS.IntraBCflag among the encoding parameters included in the encoded stream.
- step S232 the decoding unit 211 decodes SPS.AMTflag among the encoding parameters.
- step S233 the decoding unit 211 decodes SPS.NSSTflag among the encoding parameters.
- FIG. 14 is a flowchart for explaining the inverse conversion process in step S203 of FIG.
- the inverse transform unit 213 determines whether or not pred_mode_flag in the prediction information Pinfo supplied from the decoding unit 211 indicates processing of the intra prediction processing system. If it is determined in step S251 that pred_mode_flag indicates an intra prediction processing system process, the process proceeds to step S252.
- step S252 the inverse conversion unit 213 determines whether or not SPS.IntraBCflag is 1 indicating that the intra BC prediction process is enabled. If it is determined in step S252 that SPS.IntraBCflag is 1, the process proceeds to step S253.
- step S253 the inverse transform unit 213 extracts PU.IntraBCflag from the prediction information Pinfo. Then, in step S254, it is determined whether or not PU.IntraBCflag is 1 indicating that it is an intra BC prediction process. If it is determined in step S254 that PU.IntraBCflag is 1, that is, if the intra BC prediction process is performed on the decoding target PU, the process proceeds to step S255.
- step S255 the inverse transform unit 213 selects a predetermined one from the second orthogonal transform matrix index for intra BC prediction processing and the candidates for the first orthogonal transform matrix in the horizontal and vertical directions. Perform matrix selection processing for prediction processing. Details of the matrix selection process for the intra BC prediction process will be described with reference to FIG. After the process of step S255, the process proceeds to step S258.
- step S252 if it is determined in step S252 that SPS.IntraBCflag is not 1, or if it is determined in step S254 that PU.IntraBCflag is not 1, that is, if intra prediction processing is performed on the decoding target PU.
- step S254 if it is determined in step S254 that PU.IntraBCflag is not 1, that is, if intra prediction processing is performed on the decoding target PU. The process proceeds to step S256.
- step S256 the inverse transform unit 213 selects a predetermined one from the second orthogonal transform matrix index for intra prediction processing and the first and second orthogonal transform matrix candidates in the horizontal and vertical directions. Perform the matrix selection process. Then, the process proceeds to step S258.
- step S251 when it is determined in step S251 that pred_mode_flag does not indicate processing of the intra prediction processing system, that is, when inter prediction processing is performed on the decoding target PU, the processing proceeds to step S257.
- step S257 the inverse transform unit 213 selects a predetermined one from the second orthogonal transform matrix index for inter prediction processing and the candidates for the first orthogonal transform matrix in the horizontal direction and the vertical direction. Perform the matrix selection process. Then, the process proceeds to step S258.
- step S258 the selection unit 231 determines whether the index of the second orthogonal transformation matrix selected in any of steps S255 to S257 is 3. When it is determined that the index of the second orthogonal transformation matrix selected in step S258 is not 3, the selection unit 231 selects the NSST0 inverse transformation unit 232-1, NSST1 inverse transformation unit 232-2, Alternatively, the transform coefficient Coeff_IQ supplied from the inverse quantization unit 212 is supplied to the NSST2 inverse transform unit 232-3.
- step S259 the NSST0 inverse transform unit 232-1, NSST1 inverse transform unit 232-2, or NSST2 inverse transform unit 232-3 performs the second specified by the selected index on the transform coefficient Coeff_IQ.
- a second inverse orthogonal transformation matrix is performed using a second inverse orthogonal transformation matrix corresponding to the orthogonal transformation matrix.
- the conversion coefficient obtained as a result is supplied to the selection unit 233, and the process proceeds to step S260.
- the selection unit 231 supplies the transform coefficient Coeff_IQ supplied from the inverse quantization unit 212 to the non-conversion unit 232-4.
- the non-conversion unit 232-4 performs the second inverse orthogonal transform on the transform coefficient Coeff_IQ using the no-transform matrix, that is, does not perform the second inverse orthogonal transform, and supplies the transform coefficient Coeff_IQ to the selection unit 233 as it is. To do. Then, the process proceeds to step S260.
- step S260 the selection unit 233 determines whether the first orthogonal transformation matrix in the horizontal direction and the vertical direction is selected by the processing in steps S255 to S257.
- the selection unit 233 selects the horizontal DCT2 inverse transformation unit 235-1 corresponding to the first orthogonal transformation matrix in the horizontal direction.
- the supplied conversion coefficient is supplied to the horizontal DCT5 inverse transform unit 235-2, the horizontal DCT8 inverse transform unit 235-3, the horizontal DST1 inverse transform unit 235-4, or the horizontal DST7 inverse transform unit 235-5. Then, the process proceeds to step S261.
- step S261 the horizontal DCT2 inverse transform unit 235-1, the horizontal DCT5 inverse transform unit 235-2, the horizontal DCT8 inverse transform unit 235-3, the horizontal DST1 inverse transform unit 235-4, or the horizontal DST7 inverse transform unit 235-5
- the first inverse orthogonal transform is performed in the horizontal direction on the transform coefficient supplied from the selection unit 233 using the first inverse orthogonal transform matrix corresponding to the selected first horizontal orthogonal transform matrix. Do.
- the horizontal DCT2 inverse transform unit 235-1, horizontal DCT5 inverse transform unit 235-2, horizontal DCT8 inverse transform unit 235-3, horizontal DST1 inverse transform unit 235-4, or horizontal DST7 inverse transform unit 235-5 obtains the result.
- the conversion coefficient to be obtained is supplied to the selection unit 236.
- the selection unit 236 includes a vertical DCT2 inverse transformation unit 237-1, a vertical DCT5 inverse transformation unit 237-2, a vertical DCT8 inverse transformation unit 237-3, and a vertical DST1 inverse corresponding to the selected first orthogonal transformation matrix in the vertical direction.
- the supplied conversion coefficient is supplied to the conversion unit 237-4 or the vertical DST7 inverse conversion unit 237-5.
- step S262 the vertical DCT2 inverse transform unit 237-1, the vertical DCT5 inverse transform unit 237-2, the vertical DCT8 inverse transform unit 237-3, the vertical DST1 inverse transform unit 237-4, or the vertical DST7 inverse transform unit 237-5
- the first inverse orthogonal transform is performed on the supplied transform coefficient in the vertical direction by using the first inverse orthogonal transform matrix corresponding to the selected first orthogonal transform matrix in the vertical direction.
- the horizontal DCT2 inverse transform unit 235-1, horizontal DCT5 inverse transform unit 235-2, horizontal DCT8 inverse transform unit 235-3, horizontal DST1 inverse transform unit 235-4, or horizontal DST7 inverse transform unit 235-5 obtains the result.
- the predicted residual D ′ is supplied to the calculation unit 214. And a process returns to step S203 of FIG. 12, and progresses to step S204.
- FIG. 15 is a flowchart illustrating matrix selection processing for intra BC prediction processing in FIG.
- the selection unit 231 determines whether the SPS.NSSTflag supplied from the decoding unit 211 is 1 indicating that the second orthogonal transform is valid. If it is determined in step S280 that SPS.NSSTflag is 1, the process proceeds to step S282.
- step S281 the selection unit 231 extracts TU.ROTIdx from the conversion information Tinfo supplied from the decoding unit 211.
- step S282 the selection unit 231 selects TU.ROTIdx extracted in step S281 from the second orthogonal transformation matrix index for intra BC processing as the second orthogonal transformation matrix index, and performs processing. Advances to step S284.
- step S280 determines whether SPS.NSSTflag is not 1, the process proceeds to step S283.
- step S283 the selection unit 231 selects 3 as the index of the second orthogonal transformation matrix, and the process proceeds to step S284.
- step S284 the selection unit 233 determines whether the SPS.AMTflag supplied from the decoding unit 211 is 1 indicating that the first orthogonal transformation is valid. If it is determined in step S284 that SPS.AMTflag is 1, the process proceeds to step S285.
- step S285 the selection unit 233 extracts TU.TSflag from the conversion information Tinfo supplied from the decoding unit 211.
- step S286 the selection unit 233 determines whether TU.TSflag is 1 indicating that the first orthogonal transform is skipped.
- step S286 If it is determined in step S286 that TU.TSflag is not 1, in step S287, the selection unit 233 extracts the TU.AMTflag in the horizontal direction and the vertical direction from the conversion information Tinfo. Steps S288 and S289 described later are performed in each of the horizontal direction and the vertical direction.
- step S288 the selection unit 233 determines whether TU.AMTflag is 1. If it is determined in step S288 that TU.AMTflag is 1, the process proceeds to step S289.
- step S289 the selection unit 233 extracts TU.AMTIdx from the conversion information Tinfo, and the process proceeds to step S290.
- step S288 if it is determined in step S288 that TU.AMTflag is not 1, the process proceeds to step S290.
- step S290 the selection unit 233 uses the method described in FIG. 5C, based on TU.AMTflag and TU.AMTIdx, for the first horizontal and vertical first for intra BC prediction processing in C in FIG. A first orthogonal transformation matrix in the horizontal direction and the vertical direction is selected from the candidates for the orthogonal transformation matrix. Then, the process returns to step S255 in FIG. 14 and proceeds to step S258.
- step S284 if it is determined in step S284 that SPS.AMTflag is not 1, or if TU.TSflag is determined to be 1 in step S286, the subsequent processing is skipped. That is, the first orthogonal transformation matrix in the horizontal direction and the vertical direction is not selected. Then, the process returns to step S255 in FIG. 14 and proceeds to step S258.
- the matrix selection processing for intra prediction processing is illustrated in that the first orthogonal transformation matrix in the horizontal direction and the vertical direction is selected in step S290 by the method described in FIG. 5A. This is different from the matrix selection process for 15 intra BC prediction processes.
- the matrix selection process for the inter prediction process is the intra BC prediction process of FIG. 15 in that the first orthogonal transformation matrix in the horizontal direction and the vertical direction is selected by the method described in FIG. 5B in step S290. This is different from the matrix selection process.
- the intra BC prediction process is different from the intra prediction process.
- the mode for performing the intra BC prediction process is provided as one of the intra prediction modes, and the intra BC prediction process includes: It may be performed as an intra prediction process in the intra prediction mode.
- the intra prediction mode for performing the intra prediction process is a mode from 0 to 34 defined by HEVC
- the intra prediction process of the 35th mode may be the intra BC prediction process.
- the motion information is included in the prediction information Pinfo only when the intra prediction mode is the 35th mode.
- FIG. 16 is a diagram for describing a method for selecting the first orthogonal transformation matrix when the mode 35 is provided as the intra prediction mode for performing the intra BC prediction process.
- the mode No. 35 is provided as the intra prediction mode for performing the intra BC prediction process, as shown in FIG. 16A, the first orthogonal transform matrix candidate for the intra BC prediction process of C in FIG. Instead of the first orthogonal transform matrix candidate for intra prediction processing when the intra prediction mode is the 35th mode.
- the candidates for the first orthogonal transformation matrix for inter prediction processing are the same as those in FIG. That is, B in FIG. 16 and B in FIG. 5 are the same.
- the intra BC prediction process is included in the intra prediction process, but may be included in the inter prediction process.
- one mode information pred_mode_flag indicates that the PU prediction process is an intra BC prediction process or an inter prediction process.
- the second orthogonal transform may not be performed.
- the first orthogonal transformation matrix can be made variable based on the picture to be encoded. Since the optimal first orthogonal transformation matrix depends on the characteristics of the picture to be coded, the first orthogonal transformation is performed using the optimum first orthogonal transformation matrix calculated based on the picture to be coded. By doing so, encoding efficiency can be improved.
- the image encoding device 100 and the image decoding device 200 set a default value of the first orthogonal transformation matrix in advance. Then, when the first orthogonal transformation matrix optimal for the picture to be coded is different from the default value, the image coding device 100 includes the first orthogonal transformation matrix in the SPS and transmits the SPS to the image decoding device 200. .
- the image decoding apparatus 200 is not the first inverse orthogonal transformation matrix corresponding to the default value of the first orthogonal transformation matrix, but the first first included in the SPS.
- the first inverse orthogonal transformation is performed using the first inverse orthogonal transformation matrix corresponding to the orthogonal transformation matrix. The same applies to the second orthogonal transformation matrix.
- the selection unit 231 of the image decoding device 200 supplies the transform coefficient Coeff_IQ to the non-transform unit 232-4. .
- ⁇ Second Embodiment> (Description of computer to which the present disclosure is applied)
- the series of processes described above can be executed by hardware or can be executed by software.
- a program constituting the software is installed in the computer.
- the computer includes, for example, a general-purpose personal computer capable of executing various functions by installing various programs by installing a computer incorporated in dedicated hardware.
- FIG. 17 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processing by a program.
- a CPU Central Processing Unit
- ROM Read Only Memory
- RAM Random Access Memory
- an input / output interface 810 is connected to the bus 804.
- An input unit 811, an output unit 812, a storage unit 813, a communication unit 814, and a drive 815 are connected to the input / output interface 810.
- the input unit 811 includes a keyboard, a mouse, a microphone, and the like.
- the output unit 812 includes a display, a speaker, and the like.
- the storage unit 813 includes a hard disk, a nonvolatile memory, and the like.
- the communication unit 814 includes a network interface or the like.
- the drive 815 drives a removable medium 821 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.
- the CPU 801 loads the program stored in the storage unit 813 into the RAM 803 via the input / output interface 810 and the bus 804 and executes the program, for example. A series of processing is performed.
- the program executed by the computer 800 can be provided by being recorded on a removable medium 821 as a package medium, for example.
- the program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.
- the program can be installed in the storage unit 813 via the input / output interface 810 by attaching the removable medium 821 to the drive 815.
- the program can be received by the communication unit 814 via a wired or wireless transmission medium and installed in the storage unit 813.
- the program can be installed in the ROM 802 or the storage unit 813 in advance.
- the program executed by the computer 800 may be a program that is processed in time series in the order described in this specification, or a necessary timing such as in parallel or when a call is made. It may be a program in which processing is performed.
- the division of CU, PU, and TU may be only 4 divisions.
- the present disclosure can take a cloud computing configuration in which one function is shared by a plurality of devices via a network and is jointly processed.
- each step described in the above flowchart can be executed by one device or can be shared by a plurality of devices.
- the plurality of processes included in the one step can be executed by being shared by a plurality of apparatuses in addition to being executed by one apparatus.
- this indication can also take the following structures.
- a prediction image is generated by performing intra BC prediction processing on an image to be encoded
- a residual between the image to be encoded and the prediction image is generated using an orthogonal transform matrix for intra BC prediction processing.
- An image processing apparatus including a conversion unit that performs orthogonal transform on the image.
- the orthogonal transform matrix for intra BC prediction processing is configured to be selected from candidates for orthogonal transform matrix for intra BC prediction processing.
- the candidate for the orthogonal transform matrix for the intra BC prediction process is different from the candidate for the orthogonal transform matrix for the intra prediction process.
- the image processing apparatus wherein the candidate for the orthogonal transform matrix for the intra BC prediction process is the same as the candidate for the orthogonal transform matrix for the inter prediction process.
- the orthogonal transform is composed of a first orthogonal transform and a second orthogonal transform, The converter is When the predicted image is generated by performing intra BC prediction processing on the image to be encoded, the first orthogonal transform matrix for intra BC prediction processing is used, and the first is applied to the residual.
- the image processing apparatus according to any one of (1) to (4), further comprising: a second orthogonal transform unit that performs the second orthogonal transform on a coefficient obtained as a result of the first orthogonal transform.
- the image processing device according to any one of (1) to (5), further including: a setting unit that sets information for specifying the orthogonal transform matrix for the intra BC prediction process.
- the image processing device according to any one of (1) to (5), further including: a setting unit configured to set an orthogonal transform matrix for the intra BC prediction process.
- the image processing device When a prediction image is generated by performing intra BC prediction processing on an image to be encoded, a residual between the image to be encoded and the prediction image is generated using an orthogonal transform matrix for intra BC prediction processing.
- An image processing method including a transforming step for performing orthogonal transform on. (9)
- an inverse orthogonal transformation matrix for intra BC prediction processing is used to calculate the residual between the image to be decoded and the predicted image.
- An image processing apparatus including an inverse transform unit that performs inverse orthogonal transform on a coefficient after orthogonal transform.
- the inverse orthogonal transformation matrix for intra BC prediction processing is an inverse orthogonal transformation matrix corresponding to an orthogonal transformation matrix selected from candidates for orthogonal transformation matrices for intra BC prediction processing.
- the image processing device (11) The image processing device according to (10), wherein the candidate for the orthogonal transform matrix for the intra BC prediction process is different from the candidate for the orthogonal transform matrix for the intra prediction process. (12) The image processing device according to (10), wherein the candidate for the orthogonal transform matrix for the intra BC prediction process is the same as the candidate for the orthogonal transform matrix for the inter prediction process. (13)
- the inverse orthogonal transform includes a first inverse orthogonal transform and a second inverse orthogonal transform, The inverse transformer is When the predicted image is generated by performing intra BC prediction processing on the decoding target image, the second inverse orthogonal transformation matrix for intra BC prediction processing is used, and the coefficient after orthogonal transformation of the residual is performed.
- the inverse transform unit is for the intra BC prediction process corresponding to the orthogonal transform matrix indicated by the information, based on information identifying the orthogonal transform matrix used when generating the coefficient after the orthogonal transform of the residual.
- the image processing device according to any one of (9) to (13), configured to perform inverse orthogonal transformation on a coefficient after orthogonal transformation of the residual using the inverse orthogonal transformation matrix.
- the inverse transform unit uses the inverse orthogonal transform matrix for the intra BC prediction process corresponding to the orthogonal transform matrix based on the orthogonal transform matrix used when generating the coefficient after the orthogonal transform of the residual.
- the image processing device configured to perform inverse orthogonal transform on a coefficient after orthogonal transformation of the residual.
- the image processing device When generating a predicted image by performing intra BC prediction processing on an image to be decoded, an inverse orthogonal transformation matrix for intra BC prediction processing is used to calculate the residual between the image to be decoded and the predicted image.
- An image processing method including an inverse transform step of performing inverse orthogonal transform on a coefficient after orthogonal transform.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Discrete Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
This disclosure pertains to an image processing device and image processing method which make it possible to improve encoding efficiency when performing intra BC prediction processing. To this end, a transformation unit subjects the prediction residual between the predicted image and the image to be encoded to an orthogonal transformation by using an orthogonal transformation matrix for intra BC prediction processing, when a predicted image has been generated by subjecting an image to be encoded to intra BC prediction processing. This disclosure is applicable, for example, to image encoding devices or the like which subject prediction residuals to Adaptive Multiple Transforms (AMT) or Mode-dependent non-separable secondary transforms (MD-NSST).
Description
本開示は、画像処理装置および画像処理方法に関し、特に、イントラBC予測処理を行う際の符号化効率を向上させることができるようにした画像処理装置および画像処理方法に関する。
The present disclosure relates to an image processing device and an image processing method, and more particularly, to an image processing device and an image processing method capable of improving encoding efficiency when performing intra BC prediction processing.
HEVC(High Efficiency Video Coding)で符号化を行う符号化装置は、符号化対象のブロックであるカレントブロックに対してイントラ予測処理またはインター予測処理を行い、カレントブロックの予測画像である予測ブロックを生成する。そして、符号化装置は、予測ブロックとカレントブロックの差分である予測残差を直交変換し、量子化することにより、符号化ストリームを生成する。
A coding device that performs coding using HEVC (High Efficiency Video Coding) performs intra prediction processing or inter prediction processing on the current block that is the encoding target block, and generates a prediction block that is a prediction image of the current block. To do. Then, the encoding device generates an encoded stream by orthogonally transforming and quantizing the prediction residual that is the difference between the prediction block and the current block.
このようにして生成された符号化ストリームは、復号装置において、逆量子化され、逆直交変換される。そして、その結果得られる予測残差は、予測ブロックと加算され、カレントブロックの復号画像が生成される。
The encoded stream generated in this way is inverse quantized and inverse orthogonal transformed in the decoding device. Then, the prediction residual obtained as a result is added to the prediction block, and a decoded image of the current block is generated.
HEVC(High Efficiency Video Coding) version 1では、イントラ予測処理の予測モードとして、DC intra prediction, Planar intra prediction、およびAngular intra prediction と呼ばれるイントラ予測モードが採用されている。
In HEVC (High Efficiency Video Coding) version 1, intra prediction modes called DC intra prediction, Planar intra prediction, and Angular intra prediction are adopted as the prediction modes of the intra prediction process.
また、HEVC-SCC(Screen Content Coding)では、イントラ予測処理の1つとし、画面内の符号化済みの領域をインター予測処理のように参照して予測ブロックを生成するイントラBC(Intra block copy)予測処理も使用することができる。
In HEVC-SCC (Screen Content Coding), intra BC (Intra block copy) is used as one of intra prediction processes and generates a prediction block by referring to an encoded area in the screen like an inter prediction process. Prediction processing can also be used.
しかしながら、HEVC-SCCのイントラBC予測処理では、画面内の符号化済みの領域に対して平行移動のみを行って予測ブロックを生成する。従って、予測ブロックの精度を十分に向上させることはできない。
However, in the intra BC prediction process of HEVC-SCC, a prediction block is generated by performing only parallel movement on an encoded region in the screen. Therefore, the accuracy of the prediction block cannot be sufficiently improved.
そこで、イントラBC予測処理において、画面内の符号化済みの領域に対して平行移動だけでなく回転も行って、予測ブロックを生成することが考案されている(例えば、非特許文献1参照)。この場合、平行移動の方向および大きさを表す動きベクトルだけでなく、回転角も符号化ストリームに含められる。非特許文献1に記載の技術では、画面内の平行移動だけでなく、回転方向の動きも補償することができるが、拡大や縮小、スキューといった形状の変化を補償することができない。従って、予測ブロックの精度を十分に向上させることができない。
Therefore, in intra BC prediction processing, it has been devised to generate a prediction block by performing not only translation but also rotation on an encoded region in a screen (see, for example, Non-Patent Document 1). In this case, not only a motion vector indicating the direction and magnitude of the translation but also the rotation angle is included in the encoded stream. The technique described in Non-Patent Document 1 can compensate not only for the parallel movement within the screen but also for the movement in the rotational direction, but cannot compensate for a change in shape such as enlargement, reduction, or skew. Therefore, the accuracy of the prediction block cannot be sufficiently improved.
一方、ITU-T(International Telecommunication Union Telecommunication Standardization Sector)の次世代ビデオ符号化を探索するJVET(Joint Video Exploration Team)では、アフィン変換を用いてインター予測処理を行うことが提案されている(例えば、非特許文献2参照)。これにより、インター予測処理時に、画面間の平行移動および回転方向の動き、並びに、拡大や縮小、スキューといった形状の変化を補償して予測ブロックを生成することができる。
On the other hand, JVET (Joint Video Exploration Team), which searches for the next generation video encoding of ITU-T (International Telecommunication Union Telecommunication Standardization Sector), proposes to perform inter prediction processing using affine transformation (for example, Non-patent document 2). Thereby, during the inter prediction process, it is possible to generate a prediction block by compensating for the parallel movement between the screens, the movement in the rotation direction, and the change in shape such as enlargement, reduction, and skew.
しかしながら、イントラBC予測処理を行う場合に、イントラBC予測処理に適した直交変換行列を用いて予測残差を直交変換することは考案されていない。従って、イントラBC予測処理を行う場合、予測残差の直交変換後の係数のエネルギー集中度(Energy compaction)が低下する。その結果、符号化効率が低下する。
However, when performing intra BC prediction processing, it has not been devised to orthogonally transform a prediction residual using an orthogonal transform matrix suitable for intra BC prediction processing. Therefore, when the intra BC prediction process is performed, the energy concentration degree (Energyactioncompaction) of the coefficient after orthogonal transformation of the prediction residual is lowered. As a result, the encoding efficiency decreases.
本開示は、このような状況に鑑みてなされたものであり、イントラBC予測処理を行う際の符号化効率を向上させることができるようにするものである。
The present disclosure has been made in view of such a situation, and is intended to improve the encoding efficiency when performing the intra BC prediction process.
本開示の第1の側面の画像処理装置は、符号化対象の画像に対してイントラBC予測処理を行うことにより予測画像を生成する場合、イントラBC予測処理用の直交変換行列を用いて、前記符号化対象の画像と前記予測画像との残差に対して直交変換を行う変換部を備える画像処理装置である。
When the image processing device according to the first aspect of the present disclosure generates a predicted image by performing intra BC prediction processing on an image to be encoded, the orthogonal transform matrix for intra BC prediction processing is used to generate the predicted image. It is an image processing apparatus provided with the conversion part which performs orthogonal transformation with respect to the residual of the image of encoding object, and the said prediction image.
本開示の第1の側面の画像処理方法は、本開示の第1の側面の画像処理装置に対応する。
The image processing method according to the first aspect of the present disclosure corresponds to the image processing apparatus according to the first aspect of the present disclosure.
本開示の第1の側面においては、符号化対象の画像に対してイントラBC予測処理を行うことにより予測画像を生成する場合、イントラBC予測処理用の直交変換行列を用いて、前記符号化対象の画像と前記予測画像との残差に対して直交変換が行われる。
In the first aspect of the present disclosure, when a predicted image is generated by performing intra BC prediction processing on an image to be encoded, the encoding target is generated using an orthogonal transform matrix for intra BC prediction processing. Orthogonal transformation is performed on the residual between the predicted image and the predicted image.
本開示の第2の側面の画像処理装置は、復号対象の画像に対してイントラBC予測処理を行うことにより予測画像を生成する場合、イントラBC予測処理用の逆直交変換行列を用いて、前記復号対象の画像と前記予測画像との残差の直交変換後の係数に対して逆直交変換を行う逆変換部を備える画像処理装置である。
When the image processing device according to the second aspect of the present disclosure generates a prediction image by performing intra BC prediction processing on an image to be decoded, the inverse orthogonal transformation matrix for intra BC prediction processing is used to generate the prediction image. It is an image processing apparatus provided with the inverse transformation part which performs inverse orthogonal transformation with respect to the coefficient after orthogonal transformation of the residual of the image of decoding object, and the said prediction image.
本開示の第2の側面の画像処理方法は、本開示の第2の側面の画像処理装置に対応する。
The image processing method according to the second aspect of the present disclosure corresponds to the image processing apparatus according to the second aspect of the present disclosure.
本開示の第2の側面においては、復号対象の画像に対してイントラBC予測処理を行うことにより予測画像を生成する場合、イントラBC予測処理用の逆直交変換行列を用いて、前記復号対象の画像と前記予測画像との残差の直交変換後の係数に対して逆直交変換が行われる。
In the second aspect of the present disclosure, when a predicted image is generated by performing intra BC prediction processing on an image to be decoded, an inverse orthogonal transformation matrix for intra BC prediction processing is used to generate the prediction target image. Inverse orthogonal transformation is performed on the coefficient after the orthogonal transformation of the residual between the image and the predicted image.
なお、第1の側面および第2の側面の画像処理装置は、コンピュータにプログラムを実行させることにより実現することができる。
The image processing apparatuses according to the first and second aspects can be realized by causing a computer to execute a program.
また、第1の側面および第2の側面の画像処理装置を実現するために、コンピュータに実行させるプログラムは、伝送媒体を介して伝送することにより、又は、記録媒体に記録して、提供することができる。
In order to realize the image processing apparatus according to the first aspect and the second aspect, a program to be executed by a computer is provided by being transmitted via a transmission medium or by being recorded on a recording medium. Can do.
本開示の第1の側面によれば、直交変換を行うことができる。また、本開示の第1の側面によれば、イントラBC予測処理を行う際の符号化効率を向上させることができる。
According to the first aspect of the present disclosure, orthogonal transformation can be performed. Moreover, according to the 1st side surface of this indication, the encoding efficiency at the time of performing an intra BC prediction process can be improved.
本開示の第2の側面によれば、逆直交変換を行うことができる。また、本開示の第2の側面によれば、イントラBC予測処理を行う際の符号化効率を向上させることができる。
According to the second aspect of the present disclosure, inverse orthogonal transform can be performed. Moreover, according to the 2nd side surface of this indication, the encoding efficiency at the time of performing an intra BC prediction process can be improved.
なお、ここに記載された効果は必ずしも限定されるものではなく、本開示中に記載されたいずれかの効果であってもよい。
It should be noted that the effects described here are not necessarily limited, and may be any of the effects described in the present disclosure.
以下、本開示を実施するための形態(以下、実施の形態という)について説明する。なお、説明は以下の順序で行う。
1.第1実施の形態:符号化装置および復号装置(図1乃至図16)
2.第2実施の形態:コンピュータ(図17) Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described. The description will be given in the following order.
1. First Embodiment: Encoding Device and Decoding Device (FIGS. 1 to 16)
2. Second Embodiment: Computer (FIG. 17)
1.第1実施の形態:符号化装置および復号装置(図1乃至図16)
2.第2実施の形態:コンピュータ(図17) Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described. The description will be given in the following order.
1. First Embodiment: Encoding Device and Decoding Device (FIGS. 1 to 16)
2. Second Embodiment: Computer (FIG. 17)
<第1実施の形態>
(CUの形成の説明)
MPEG2(Moving Picture Experts Group 2(ISO/IEC 13818-2))やMPEG-4 Part10(Advanced Video Coding、以下AVCと記す)などの旧来の画像符号化方式では、符号化処理は、マクロブロックと呼ばれる処理単位で実行される。マクロブロックは、16x16画素の均一なサイズを有するブロックである。これに対し、HEVCでは、符号化処理は、CU(Coding Unit)と呼ばれる処理単位(符号化単位)で実行される。CUは、最大符号化単位であるLCU(Largest Coding Unit)を再帰的に分割することにより形成される、可変的なサイズを有するブロックである。選択可能なCUの最大サイズは、64x64画素である。選択可能なCUの最小サイズは、8x8画素である。最小サイズのCUは、SCU(Smallest Coding Unit)と呼ばれる。なお、CUの最大サイズは、64x64画素に限定されず、より大きい128x128画素、256x256画素などのブロックサイズとしてもよい。 <First embodiment>
(Description of CU formation)
In conventional image coding methods such as MPEG2 (Moving Picture Experts Group 2 (ISO / IEC 13818-2)) and MPEG-4 Part10 (Advanced Video Coding, hereinafter referred to as AVC), the encoding process is called a macroblock. It is executed in units of processing. The macro block is a block having a uniform size of 16 × 16 pixels. On the other hand, in HEVC, encoding processing is executed in processing units (coding units) called CU (Coding Unit). The CU is a block having a variable size formed by recursively dividing an LCU (Largest Coding Unit) that is a maximum coding unit. The maximum selectable CU size is 64x64 pixels. The minimum selectable CU size is 8x8 pixels. The minimum size CU is called a SCU (Smallest Coding Unit). Note that the maximum size of the CU is not limited to 64 × 64 pixels, and may be a larger block size such as 128 × 128 pixels or 256 × 256 pixels.
(CUの形成の説明)
MPEG2(Moving Picture Experts Group 2(ISO/IEC 13818-2))やMPEG-4 Part10(Advanced Video Coding、以下AVCと記す)などの旧来の画像符号化方式では、符号化処理は、マクロブロックと呼ばれる処理単位で実行される。マクロブロックは、16x16画素の均一なサイズを有するブロックである。これに対し、HEVCでは、符号化処理は、CU(Coding Unit)と呼ばれる処理単位(符号化単位)で実行される。CUは、最大符号化単位であるLCU(Largest Coding Unit)を再帰的に分割することにより形成される、可変的なサイズを有するブロックである。選択可能なCUの最大サイズは、64x64画素である。選択可能なCUの最小サイズは、8x8画素である。最小サイズのCUは、SCU(Smallest Coding Unit)と呼ばれる。なお、CUの最大サイズは、64x64画素に限定されず、より大きい128x128画素、256x256画素などのブロックサイズとしてもよい。 <First embodiment>
(Description of CU formation)
In conventional image coding methods such as MPEG2 (Moving Picture Experts Group 2 (ISO / IEC 13818-2)) and MPEG-4 Part10 (Advanced Video Coding, hereinafter referred to as AVC), the encoding process is called a macroblock. It is executed in units of processing. The macro block is a block having a uniform size of 16 × 16 pixels. On the other hand, in HEVC, encoding processing is executed in processing units (coding units) called CU (Coding Unit). The CU is a block having a variable size formed by recursively dividing an LCU (Largest Coding Unit) that is a maximum coding unit. The maximum selectable CU size is 64x64 pixels. The minimum selectable CU size is 8x8 pixels. The minimum size CU is called a SCU (Smallest Coding Unit). Note that the maximum size of the CU is not limited to 64 × 64 pixels, and may be a larger block size such as 128 × 128 pixels or 256 × 256 pixels.
このように、可変的なサイズを有するCUが採用される結果、HEVCでは、画像の内容に応じて画質及び符号化効率を適応的に調整することが可能である。予測符号化のための予測処理は、PU(Prediction Unit)と呼ばれる処理単位で実行される。PUは、CUをいくつかの分割パタンのうちの1つで分割することにより形成される。また、PUは、輝度(Y)及び色差(Cb,Cr)毎のPB(Prediction Block)と呼ばれる処理単位から構成される。さらに、直交変換処理は、TU(Transform Unit)と呼ばれる処理単位で実行される。TUは、CU又はPUをある深さまで分割することにより形成される。また、TUは、輝度(Y)及び色差(Cb,Cr)毎のTB(Transform Block)と呼ばれる処理単位(変換ブロック)ら構成される。
As described above, as a result of adopting a CU having a variable size, in HEVC, it is possible to adaptively adjust the image quality and the coding efficiency according to the content of the image. Prediction processing for predictive coding is executed in a processing unit called PU (Prediction Unit). A PU is formed by dividing a CU with one of several division patterns. The PU is composed of processing units called PB (Prediction Block) for each luminance (Y) and color difference (Cb, Cr). Further, the orthogonal transformation process is executed in a processing unit called TU (Transform Unit). A TU is formed by dividing a CU or PU to a certain depth. The TU is composed of processing units (transform blocks) called TB (Transform Block) for each luminance (Y) and color difference (Cb, Cr).
以下においては、画像(ピクチャ)の部分領域や処理単位として「ブロック」を用いて説明する場合がある(処理部のブロックではない)。この場合の「ブロック」は、ピクチャ内の任意の部分領域を示し、その大きさ、形状、および特性等は限定されない。つまり、この場合の「ブロック」には、例えば、TB、TU、PB、PU、SCU、CU、LCU(CTB)、サブブロック、マクロブロック、タイル、またはスライス等、任意の部分領域(処理単位)が含まれるものとする。
In the following description, “block” may be used as a partial area or processing unit of an image (picture) (not a block of a processing unit). The “block” in this case indicates an arbitrary partial area in the picture, and its size, shape, characteristics, and the like are not limited. In other words, the “block” in this case includes any partial area (processing unit) such as TB, TU, PB, PU, SCU, CU, LCU (CTB), sub-block, macroblock, tile, or slice. Is included.
図1は、第1実施の形態におけるCUの形成について説明する図である。
FIG. 1 is a diagram for explaining formation of a CU in the first embodiment.
第1実施の形態におけるCUの形成は、JVET-C0024,“EE2.1: Quadtree plus binary tree structure integration with JEM tools”に記載されているQTBT(Quad tree plus binary tree)と呼ばれる技術で行われる。
CU formation in the first embodiment is performed by a technique called QTBT (Quad tree plus binary tree) described in JVET-C0024, “EE2.1:“ Quad tree plus plus binary tree tree structure integration with JEM tools ”.
具体的には、HEVCでは、1つのブロックを4(=2x2)個にのみ分割することができるが、第1実施の形態では、1つのブロックを4(=2x2)個だけでなく、2(=1x2,2x1)個のサブブロックにも分割することができる。即ち、第1実施の形態では、CUの形成は、1つのブロックの4個または2個のサブブロックへの分割を再帰的に繰り返すことにより行われ、結果として四分木(Quad-Tree)状、または、2分木(Binary-Tree)状のツリー構造が形成される。なお、第1実施の形態では、PUとTUは、CUと同一である。
Specifically, in HEVC, one block can be divided into only 4 (= 2x2), but in the first embodiment, one block is not only 4 (= 2x2) but 2 ( = 1 × 2, 2 × 1) sub-blocks can also be divided. That is, in the first embodiment, the CU is formed by recursively dividing one block into four or two sub-blocks, resulting in a quad-tree shape. Alternatively, a binary tree (Binary-Tree) tree structure is formed. In the first embodiment, PU and TU are the same as CU.
(画像符号化装置の構成例)
図2は、本開示を適用した画像処理装置としての画像符号化装置の第1実施の形態の構成例を示すブロック図である。図2の画像符号化装置100は、AVCやHEVCのように、画像とその予測画像との予測残差を符号化する装置である。例えば、画像符号化装置100は、HEVCの技術や、JVETにて提案された技術を実装している。 (Configuration example of image encoding device)
FIG. 2 is a block diagram illustrating a configuration example of the first embodiment of an image encoding device as an image processing device to which the present disclosure is applied. Theimage encoding apparatus 100 in FIG. 2 is an apparatus that encodes a prediction residual between an image and its predicted image, such as AVC and HEVC. For example, the image encoding apparatus 100 is equipped with the HEVC technology or the technology proposed by JVET.
図2は、本開示を適用した画像処理装置としての画像符号化装置の第1実施の形態の構成例を示すブロック図である。図2の画像符号化装置100は、AVCやHEVCのように、画像とその予測画像との予測残差を符号化する装置である。例えば、画像符号化装置100は、HEVCの技術や、JVETにて提案された技術を実装している。 (Configuration example of image encoding device)
FIG. 2 is a block diagram illustrating a configuration example of the first embodiment of an image encoding device as an image processing device to which the present disclosure is applied. The
なお、図2においては、処理部やデータの流れ等の主なものを示しており、図2に示されるものが全てとは限らない。つまり、画像符号化装置100において、図2においてブロックとして示されていない処理部が存在したり、図2において矢印等として示されていない処理やデータの流れが存在したりしてもよい。
Note that FIG. 2 shows main components such as a processing unit and a data flow, and the ones shown in FIG. 2 are not all. That is, in the image coding apparatus 100, there may be a processing unit that is not shown as a block in FIG. 2, or there may be a process or data flow that is not shown as an arrow or the like in FIG.
図2の画像符号化装置100は、制御部101、演算部111、変換部112、量子化部113、符号化部114、逆量子化部115、逆変換部116、演算部117、フレームメモリ118、および予測部119を有する。画像符号化装置100は、入力されるフレーム単位の動画像であるピクチャに対してCUごとに符号化を行う。
2 includes a control unit 101, a calculation unit 111, a conversion unit 112, a quantization unit 113, a coding unit 114, an inverse quantization unit 115, an inverse conversion unit 116, a calculation unit 117, and a frame memory 118. And a prediction unit 119. The image coding apparatus 100 performs coding for each CU on a picture that is an input frame-based moving image.
具体的には、画像符号化装置100の制御部101は、外部からの入力等に基づいて、符号化パラメータ(ヘッダ情報Hinfo、予測情報Pinfo、変換情報Tinfo等)のうちの固定値を設定する。制御部101は、ヘッダ情報Hinfoの固定値を、例えば各部に供給し、予測情報Pinfoの固定値を、例えば予測部119に供給し、変換情報Tinfoの固定値を、例えば、変換部112、量子化部113、逆量子化部115、および逆変換部116に供給する。
Specifically, the control unit 101 of the image encoding device 100 sets a fixed value among encoding parameters (header information Hinfo, prediction information Pinfo, conversion information Tinfo, and the like) based on an external input or the like. . The control unit 101 supplies a fixed value of the header information Hinfo to each unit, for example, supplies a fixed value of the prediction information Pinfo, for example, to the prediction unit 119, and sets a fixed value of the conversion information Tinfo, for example, the conversion unit 112, the quantum To the quantization unit 113, the inverse quantization unit 115, and the inverse transformation unit 116.
また、制御部101(設定部)は、変換部112、量子化部113、および予測部119から符号化パラメータのうちの可変値の最適値を取得し、その最適値と固定値を含む符号化パラメータを設定する。制御部101は、設定された符号化パラメータを符号化部114に供給し、予測情報Pinfoの可変値の最適値を演算部111に供給し、変換情報Tinfoの可変値の最適値を逆量子化部115および逆変換部116に供給する。
Further, the control unit 101 (setting unit) obtains the optimum value of the variable value from the encoding unit 112, the quantization unit 113, and the prediction unit 119, and performs encoding including the optimum value and the fixed value. Set the parameters. The control unit 101 supplies the set encoding parameter to the encoding unit 114, supplies the optimal value of the variable value of the prediction information Pinfo to the arithmetic unit 111, and dequantizes the optimal value of the variable value of the conversion information Tinfo. To the unit 115 and the inverse transform unit 116.
ヘッダ情報Hinfoは、例えば、ビデオパラメータセット(VPS(Video Parameter Set))、シーケンスパラメータセット(SPS(Sequence Parameter Set))、ピクチャパラメータセット(PPS(Picture Parameter Set))、スライスヘッダ(SH)等の情報を含む。例えば、ヘッダ情報HinfoのSPSには、イントラBC予測処理を有効にするかどうかを示す固定値であるSPS.IntraBCflag,第1の直交変換(プライマリ変換)を有効にするかどうかを示す固定値であるSPS.AMTflag、第2の直交変換(セカンダリ変換)を有効にするかどうかを示す固定値であるSPS.NSSTflagなどが含まれる。もちろん、ヘッダ情報Hinfoの内容は任意であり、上述した例以外のどのような情報がこのヘッダ情報Hinfoに含まれるようにしてもよい。
The header information Hinfo includes, for example, a video parameter set (VPS (Video Parameter Set)), a sequence parameter set (SPS (Sequence Parameter Set)), a picture parameter set (PPS (Picture Parameter Set)), a slice header (SH), etc. Contains information. For example, in the SPS of the header information Hinfo, SPS.IntraBCflag which is a fixed value indicating whether to enable intra BC prediction processing, or a fixed value indicating whether to enable the first orthogonal transform (primary transform). A certain SPS.AMTflag, SPS.NSSTflag which is a fixed value indicating whether or not to enable the second orthogonal transformation (secondary transformation), and the like are included. Of course, the content of the header information Hinfo is arbitrary, and any information other than the above-described example may be included in the header information Hinfo.
予測情報Pinfoには、例えば、PU(CU)の形成時の各分割階層における水平方向または垂直方向の分割の有無を示す可変の情報であるsplit flag等が含まれる。また、予測情報Pinfoには、PUごとに、PUの予測処理がイントラ予測処理系の処理(イントラ予測処理,イントラBC予測処理)であるか、または、インター予測処理であるかを示す可変のモード情報pred_mode_flagが含まれる。
The prediction information Pinfo includes, for example, a split flag, which is variable information indicating the presence or absence of horizontal or vertical division in each division hierarchy at the time of PU (CU) formation. The prediction information Pinfo includes a variable mode indicating whether the PU prediction process is an intra prediction processing system process (intra prediction process, intra BC prediction process) or an inter prediction process for each PU. Information pred_mode_flag is included.
また、モード情報pred_mode_flagがイントラ予測処理系の処理を示す場合、予測情報Pinfoには、PUの予測処理がイントラBC予測処理であるかどうかを示す可変のPU.IntraBCflag(PU.IntraBCmode)が含まれる。PU.IntraBCflagがイントラBC予測処理であることを示す場合、予測情報Pinfoには、イントラBC予測処理に用いられる可変の動きベクトルが含まれる。
Further, when the mode information pred_mode_flag indicates processing of an intra prediction processing system, the prediction information Pinfo includes a variable PU.IntraBCflag (PU.IntraBCmode) indicating whether or not the PU prediction processing is intra BC prediction processing. . When PU.IntraBCflag indicates that it is an intra BC prediction process, the prediction information Pinfo includes a variable motion vector used for the intra BC prediction process.
一方、PU.IntraBCflagがイントラBC予測処理ではないことを示す場合、予測情報Pinfoには、DC intra prediction, Planar intra prediction、およびAngular intra prediction と呼ばれるイントラ予測モードを示す可変の情報が含まれる。
On the other hand, when PU.IntraBCflag indicates that it is not an intra BC prediction process, the prediction information Pinfo includes variable information indicating intra prediction modes called DC intra prediction, Planar intra prediction, and Angular intra prediction.
また、モード情報pred_mode_flagがインター予測処理を示す場合、予測情報Pinfoには、インター予測処理で用いられる可変の動きベクトルなどが含まれる。もちろん、予測情報Pinfoの内容は任意であり、上述した例以外のどのような情報がこの予測情報Pinfoに含まれるようにしてもよい。
Also, when the mode information pred_mode_flag indicates an inter prediction process, the prediction information Pinfo includes a variable motion vector used in the inter prediction process. Of course, the content of the prediction information Pinfo is arbitrary, and any information other than the example described above may be included in the prediction information Pinfo.
変換情報Tinfoには、TBのサイズを示す可変の情報であるTBサイズTBSizeなどが含まれる。また、SPS.AMTflagが第1の直交変換を有効にすることを示す場合、変換情報Tinfoには、第1の直交変換をスキップするかどうかを示すTU.TSflag等が含まれる。また、TU.TSflagが第1の直交変換をスキップしないことを示す場合、変換情報Tinfoには、第1の直交変換のモードを示すTU.AMTflag、垂直方向および水平方向の第1の直交変換に用いられる第1の直交変換行列を特定するインデックスを示すTU.AMTIdx等が含まれる。さらに、SPS.NSSTflagが第2の直交変換を有効にすることを示す場合、変換情報Tinfoには、第2の直交変換に用いられる第2の直交変換行列を特定するインデックスを示すTU.ROTIdxが含まれる。もちろん、変換情報Tinfoの内容は任意であり、上述した例以外のどのような情報がこの変換情報Tinfoに含まれるようにしてもよい。
The conversion information Tinfo includes TB size TBSize, which is variable information indicating the size of TB. Further, when SPS.AMTflag indicates that the first orthogonal transform is to be enabled, the conversion information Tinfo includes TU.TSflag indicating whether or not to skip the first orthogonal transform. When TU.TSflag indicates that the first orthogonal transform is not skipped, the transform information Tinfo includes TU.AMTflag indicating the first orthogonal transform mode, and the first orthogonal transform in the vertical and horizontal directions. TU.AMTIdx indicating an index specifying the first orthogonal transformation matrix to be used is included. Furthermore, when SPS.NSSTflag indicates that the second orthogonal transformation is to be enabled, the transformation information Tinfo includes TU.ROTIdx indicating an index for identifying the second orthogonal transformation matrix used for the second orthogonal transformation. included. Of course, the content of the conversion information Tinfo is arbitrary, and any information other than the example described above may be included in the conversion information Tinfo.
演算部111は、予測情報Pinfoのsplit flagの最適値に基づいて、入力されるフレーム単位の動画像であるピクチャを順に符号化対象のピクチャとし、符号化対象のピクチャに対して符号化対象のCU(PU,TU)を設定する。演算部111は、符号化対象のPUの画像I(カレントブロック)から、予測部119から供給されたPUの予測画像P(予測ブロック)を減算して予測残差Dを求め、それを変換部112に供給する。
Based on the optimal value of split flag of the prediction information Pinfo, the calculation unit 111 sequentially sets the pictures that are moving images in units of frames as the encoding target pictures, and the encoding target pictures are the encoding target pictures. Set CU (PU, TU). The calculation unit 111 subtracts the prediction image P (prediction block) of the PU supplied from the prediction unit 119 from the image I (current block) of the PU to be encoded to obtain the prediction residual D, and converts it to the conversion unit 112.
変換部112は、制御部101から供給される符号化パラメータの固定値に基づいて、変換情報Tinfoの可変値の候補となる各値を決定する。変換部112は、変換情報Tinfoの可変値の候補となる値ごとに、その値と固定値に基づいて、演算部111から供給される予測残差Dに対して第1の直交変換や第2の直交変換を行う変換処理を行い、変換係数Coeffを導出する。
The conversion unit 112 determines each value as a variable value candidate of the conversion information Tinfo based on the fixed value of the encoding parameter supplied from the control unit 101. For each value that is a candidate for a variable value of the conversion information Tinfo, the conversion unit 112 performs a first orthogonal transformation or a second conversion on the prediction residual D supplied from the calculation unit 111 based on the value and the fixed value. A transformation process for performing orthogonal transformation is performed, and a transformation coefficient Coeff is derived.
変換部112は、変換情報Tinfoの可変値の候補となる値ごとに、変換係数Coeff等に基づいて符号化対象のCUのRD(Rate Distortion)コストを算出する。そして、変換部112は、RDコストが最も小さくなる変換情報Tinfoの可変値の候補となる値を、変換情報Tinfoの可変値の最適値に決定し、制御部101に供給する。変換部112は、変換情報Tinfoの可変値の最適値に対応する変換係数Coeffを量子化部113に供給する。
The conversion unit 112 calculates the RD (Rate Distortion) cost of the CU to be encoded based on the conversion coefficient Coeff or the like for each value that is a variable value candidate of the conversion information Tinfo. Then, the conversion unit 112 determines a value as a variable value candidate of the conversion information Tinfo with the smallest RD cost as an optimum value of the variable value of the conversion information Tinfo, and supplies it to the control unit 101. The conversion unit 112 supplies a conversion coefficient Coeff corresponding to the optimum variable value of the conversion information Tinfo to the quantization unit 113.
量子化部113は、制御部101から供給される符号化パラメータの固定値に基づいて、変換情報Tinfoの可変値の候補となる各値を決定する。量子化部113は、変換情報Tinfoの可変値の候補となる値ごとに、その値と固定値に基づいて、変換部112から供給される変換係数Coeffをスケーリング(量子化)し、量子化変換係数レベルlevelを導出する。量子化部113は、変換情報Tinfoの可変値の候補となる値ごとに、量子化変換係数レベルlevel等に基づいて符号化対象のCUのRDコストを算出する。そして、量子化部113は、RDコストが最も小さくなる変換情報Tinfoの可変値の候補となる値を、変換情報Tinfoの可変値の最適値に決定し、制御部101に供給する。量子化部113は、変換情報Tinfoの可変値の最適値に対応する量子化変換係数レベルlevelを符号化部114および逆量子化部115に供給する。
The quantization unit 113 determines each value that is a candidate for a variable value of the conversion information Tinfo based on the fixed value of the encoding parameter supplied from the control unit 101. The quantization unit 113 scales (quantizes) the transform coefficient Coeff supplied from the transform unit 112 based on the value and the fixed value for each variable value candidate of the transform information Tinfo, and performs quantization transform. The coefficient level level is derived. The quantization unit 113 calculates the RD cost of the CU to be encoded based on the quantized transform coefficient level level and the like for each value that becomes a variable value candidate of the transform information Tinfo. Then, the quantization unit 113 determines a value that is a candidate for a variable value of the conversion information Tinfo with the smallest RD cost as an optimum value of the variable value of the conversion information Tinfo, and supplies it to the control unit 101. The quantization unit 113 supplies the quantized transform coefficient level level corresponding to the variable value optimum value of the transform information Tinfo to the encoding unit 114 and the inverse quantization unit 115.
符号化部114は、量子化部113から供給される量子化変換係数レベルlevel等を所定の方法で符号化する。例えば、符号化部114は、シンタックステーブルの定義に沿って、制御部101から供給される符号化パラメータ(ヘッダ情報Hinfo、予測情報Pinfo、変換情報Tinfo等)と、量子化部113から供給される量子化変換係数レベルlevelを、各シンタックス要素のシンタックス値へ変換する。そして、符号化部114は、各シンタックス値を符号化(例えば、算術符号化)し、その結果得られるビット列を生成する。符号化部114は、例えば、符号化された各シンタックス要素のビット列(符号化データ)を多重化し、符号化ストリームとして出力する。
The encoding unit 114 encodes the quantized transform coefficient level level and the like supplied from the quantization unit 113 by a predetermined method. For example, the encoding unit 114 is supplied from the quantization unit 113 and the encoding parameters (header information Hinfo, prediction information Pinfo, conversion information Tinfo, etc.) supplied from the control unit 101 in accordance with the definition of the syntax table. The quantized transform coefficient level level is converted to the syntax value of each syntax element. Then, the encoding unit 114 encodes each syntax value (for example, arithmetic encoding) and generates a bit string obtained as a result. For example, the encoding unit 114 multiplexes the encoded bit string (encoded data) of each syntax element, and outputs the result as an encoded stream.
逆量子化部115は、制御部101から供給される符号化パラメータの固定値および可変値の最適値に基づいて、量子化部113から供給される量子化変換係数レベルlevelの値をスケーリング(逆量子化)し、逆量子化後の変換係数Coeff_IQを導出する。逆量子化部115は、その変換係数Coeff_IQを逆変換部116に供給する。この逆量子化部115により行われる逆量子化は、量子化部113により行われる量子化の逆処理である。
The inverse quantization unit 115 scales (reverses) the value of the quantization transform coefficient level level supplied from the quantization unit 113 based on the fixed value and the optimum variable value supplied from the control unit 101. Quantization) to derive a transform coefficient Coeff_IQ after inverse quantization. The inverse quantization unit 115 supplies the transform coefficient Coeff_IQ to the inverse transform unit 116. The inverse quantization performed by the inverse quantization unit 115 is an inverse process of the quantization performed by the quantization unit 113.
逆変換部116は、制御部101から供給される符号化パラメータの固定値および可変値の最適値に基づいて、逆量子化部115から供給される変換係数Coeff_IQに対して第1の逆直交変換や第2の逆直交変換を行う逆変換処理を行い、予測残差D’を導出する。逆変換部116は、その予測残差D’を演算部117に供給する。この逆変換部116により行われる逆変換処理は、変換部112により行われる変換処理の逆処理であり、後述する画像復号装置において行われる逆変換処理と同様の処理である。従って、この逆変換処理については、画像復号装置に関する説明において後述する。
The inverse transform unit 116 performs a first inverse orthogonal transform on the transform coefficient Coeff_IQ supplied from the inverse quantization unit 115 based on the fixed value and the optimum variable value supplied from the control unit 101. And inverse transform processing for performing the second inverse orthogonal transform is performed to derive a prediction residual D ′. The inverse transform unit 116 supplies the prediction residual D ′ to the calculation unit 117. The inverse transform process performed by the inverse transform unit 116 is an inverse process of the transform process performed by the transform unit 112, and is the same process as the inverse transform process performed in the image decoding apparatus described later. Therefore, this inverse transform process will be described later in the description of the image decoding apparatus.
演算部117は、逆変換部116から供給される予測残差D’と、予測部119より供給される、その予測残差D’に対応する予測画像Pとを加算して局所的な復号画像Recを導出する。演算部117は、その局所的な復号画像Recをフレームメモリ118に供給する。
The calculation unit 117 adds the prediction residual D ′ supplied from the inverse conversion unit 116 and the prediction image P corresponding to the prediction residual D ′ supplied from the prediction unit 119 to add a local decoded image. Derive Rec. The computing unit 117 supplies the local decoded image Rec to the frame memory 118.
フレームメモリ118は、演算部117より供給される局所的な復号画像Recを用いてピクチャ単位の復号画像を再構築し、フレームメモリ118内のバッファへ格納する。フレームメモリ118は、予測部119により指定される復号画像を参照画像としてバッファより読み出し、予測部119に供給する。また、フレームメモリ118は、復号画像の生成に係るヘッダ情報Hinfo、予測情報Pinfo、変換情報Tinfoなどを、フレームメモリ118内のバッファに格納するようにしても良い。
The frame memory 118 reconstructs a decoded image in units of pictures using the local decoded image Rec supplied from the calculation unit 117 and stores it in a buffer in the frame memory 118. The frame memory 118 reads the decoded image specified by the prediction unit 119 from the buffer as a reference image, and supplies the read image to the prediction unit 119. The frame memory 118 may store header information Hinfo, prediction information Pinfo, conversion information Tinfo, and the like related to the generation of a decoded image in a buffer in the frame memory 118.
予測部119は、制御部101から供給される符号化パラメータの固定値に基づいて、予測情報Pinfoの可変値の候補となる値を決定する。予測部119は、予測情報Pinfoの可変値の候補となる値ごとに、その値と固定値に基づいて、フレームメモリ118に格納された復号画像を参照画像として取得し、その参照画像を用いて、イントラBC予測処理、イントラ予測処理、またはインター予測処理を行う。これにより、予測画像Pが生成される。
The prediction unit 119 determines a value that is a candidate for a variable value of the prediction information Pinfo based on the fixed value of the encoding parameter supplied from the control unit 101. The prediction unit 119 acquires, as a reference image, a decoded image stored in the frame memory 118 based on the value and the fixed value for each value that is a variable value candidate of the prediction information Pinfo, and uses the reference image. Intra BC prediction processing, intra prediction processing, or inter prediction processing is performed. Thereby, the predicted image P is generated.
例えば、SPS.IntraBCflagが、イントラBC予測処理を有効にすることを示す1である場合、予測部119は、PU.IntraBCflagの候補となる値を、イントラBC予測処理を示す1、および、イントラBC予測処理を示さない0に決定する。そして、予測部119は、PU.IntraBCflagの候補となる値としての1に基づいて、イントラBC予測処理を行う。
For example, when SPS.IntraBCflag is 1 indicating that intra BC prediction processing is enabled, the prediction unit 119 sets a value that is a candidate of PU.IntraBCflag as 1 indicating intra BC prediction processing, and intra BC. It is determined as 0 which does not indicate the prediction process. And the prediction part 119 performs an intra BC prediction process based on 1 as a value which becomes a candidate of PU.IntraBCflag.
なお、イントラBC予測処理またはイントラ予測処理を行う場合、参照画像は、画像Iを含むピクチャが局所的に復号された復号画像である。また、インター予測処理を行う場合、参照画像は、画像Iを含むピクチャより前に復号されたピクチャ(復号済み前画像)であり、画面全体が復号された復号画像である。
In addition, when performing intra BC prediction processing or intra prediction processing, the reference image is a decoded image obtained by locally decoding a picture including the image I. When performing inter prediction processing, the reference image is a picture (pre-decoded image) decoded before the picture including the image I, and is a decoded image obtained by decoding the entire screen.
また、イントラBC予測処理とは、PUと同一ピクチャ内の、PUから動きベクトルだけ離れた位置に存在する復号済みのPUサイズのブロックを平行移動することにより予測画像Pとして生成する処理である。イントラ予測処理とは、PUと同一ピクチャ内の、PUに対してイントラ予測モードが示す方向に存在する復号済みのPUサイズのブロックを予測画像Pとして生成する処理である。インター予測処理とは、PUを含むピクチャより前に復号されたピクチャ内の、PUから動きベクトルだけ離れた位置に存在する復号済みのPUサイズのブロックを平行移動することにより予測画像Pとして生成する処理である。
Also, the intra BC prediction process is a process of generating a predicted image P by translating a decoded PU-sized block existing at a position away from the PU by a motion vector in the same picture as the PU. The intra prediction process is a process of generating, as a predicted image P, a decoded PU size block that exists in the same picture as the PU in the direction indicated by the intra prediction mode with respect to the PU. The inter prediction process generates a predicted image P by translating a decoded PU size block existing at a position separated from the PU by a motion vector in a picture decoded before the picture including the PU. It is processing.
予測部119は、予測情報Pinfoの可変値の候補となる値ごとに、予測画像P等に基づいて符号化対象のCUのRDコストを算出する。そして、予測部119は、RDコストが最も小さくなる予測情報Pinfoの可変値の候補となる値を、予測情報Pinfoの可変値の最適値に決定し、制御部101に供給する。予測部119は、予測情報Pinfoの可変値の最適値に対応する予測画像Pを演算部111や演算部117に供給する。
The prediction unit 119 calculates the RD cost of the CU to be encoded based on the predicted image P or the like for each value that is a candidate for a variable value of the prediction information Pinfo. Then, the prediction unit 119 determines a value that is a candidate for a variable value of the prediction information Pinfo with the smallest RD cost as an optimum value of the variable value of the prediction information Pinfo, and supplies the optimum value to the control unit 101. The prediction unit 119 supplies the prediction image P corresponding to the optimum variable value of the prediction information Pinfo to the calculation unit 111 and the calculation unit 117.
図3は、図2の予測部119により行われるイントラBC予測処理を説明する図である。
FIG. 3 is a diagram illustrating the intra BC prediction process performed by the prediction unit 119 of FIG.
図3に示すように、ピクチャ150の符号化対象のPU151に対してイントラBC予測処理を行う場合、PU151の上側および左側の領域152は既に符号化され、復号されているが、PU151の下側および右側の領域153はまだ符号化されていない。即ち、参照画像は、領域152のみが局部的に復号された復号画像である。
As shown in FIG. 3, when performing intra BC prediction processing on the PU 151 to be encoded in the picture 150, the upper and left regions 152 of the PU 151 have already been encoded and decoded, but the lower side of the PU 151. The right region 153 is not yet encoded. That is, the reference image is a decoded image in which only the region 152 is locally decoded.
予測部119は、予測情報Pinfoのうちの動きベクトルの候補となる各値に基づいて、参照画像を用いてPU151に対してイントラBC予測処理を行う。予測部119は、その結果生成される動きベクトルの候補となる各値に対応する予測画像P等に基づいてRDコストを算出し、RDコストが最小となる動きベクトルの候補となる値を最適値に決定する。
The prediction unit 119 performs intra BC prediction processing on the PU 151 using a reference image based on each value that is a candidate for a motion vector in the prediction information Pinfo. The prediction unit 119 calculates an RD cost based on the predicted image P corresponding to each value as a motion vector candidate generated as a result, and sets a value as a motion vector candidate that minimizes the RD cost as an optimum value. To decide.
図3の例では、動きベクトル154が、イントラBC予測処理における動きベクトルの最適値に決定されている。従って、この最適値に対応する予測画像Pは、ピクチャ150内のPU151から動きベクトル154だけ離れた位置に存在するPUサイズの復号済みのブロック155の復号画像である。
In the example of FIG. 3, the motion vector 154 is determined as the optimal value of the motion vector in the intra BC prediction process. Therefore, the predicted image P corresponding to this optimum value is a decoded image of a PU-sized decoded block 155 that exists at a position separated from the PU 151 in the picture 150 by the motion vector 154.
なお、予測部119は、RDコストではなく、予測画像Pと符号化対象のPUの画像IとのSAD(Sum of Absolute Difference)やSATD(Sum of Absolution Transformed Difference)が最小となる動きベクトルの候補となる値を最適値に決定するようにしてもよい。
Note that the prediction unit 119 is not an RD cost, but a motion vector candidate that minimizes SAD (Sum of Absolute Difference) and SATD (Sum of Absolution Transformed Difference) between the predicted image P and the image I of the PU to be encoded May be determined as an optimum value.
(変換部の構成例)
図4は、図2の変換部112の構成例を示すブロック図である。 (Configuration example of conversion unit)
FIG. 4 is a block diagram illustrating a configuration example of theconversion unit 112 in FIG.
図4は、図2の変換部112の構成例を示すブロック図である。 (Configuration example of conversion unit)
FIG. 4 is a block diagram illustrating a configuration example of the
図4の変換部112は、選択部171、選択部172、垂直DCT2変換部173-1、垂直DCT5変換部173-2、垂直DCT8変換部173-3、垂直DST1変換部173-4、垂直DST7変換部173-5、選択部174、水平DCT2変換部175-1、水平DCT5変換部175-2、水平DCT8変換部175-3、水平DST1変換部175-4、水平DST7変換部175-5、選択部176、NSST0変換部177-1、NSST1変換部177-2、NSST2変換部177-3、無変換部177-4、および決定部178により構成される。
4 includes a selection unit 171, a selection unit 172, a vertical DCT2 conversion unit 173-1, a vertical DCT5 conversion unit 173-2, a vertical DCT8 conversion unit 173-3, a vertical DST1 conversion unit 173-4, and a vertical DST7. Conversion unit 173-5, selection unit 174, horizontal DCT2 conversion unit 175-1, horizontal DCT5 conversion unit 175-2, horizontal DCT8 conversion unit 175-3, horizontal DST1 conversion unit 175-4, horizontal DST7 conversion unit 175-5, The selection unit 176, NSST0 conversion unit 177-1, NSST1 conversion unit 177-2, NSST2 conversion unit 177-3, no conversion unit 177-4, and determination unit 178 are configured.
変換部112の選択部171には、図2の演算部111から予測残差Dが供給されるとともに、制御部101からヘッダ情報Hinfo等が供給される。選択部171は、ヘッダ情報HinfoのうちのSPS.AMTflagに基づいて、TU.TSflagの候補となる各値を決定する。選択部171は、TU.TSflagの候補となる各値に基づいて、予測残差Dを選択部172または選択部176に供給する。
The selection unit 171 of the conversion unit 112 is supplied with the prediction residual D from the calculation unit 111 in FIG. 2 and the header information Hinfo and the like from the control unit 101. The selection unit 171 determines each value as a candidate for TU.TSflag based on SPS.AMTflag in the header information Hinfo. The selection unit 171 supplies the prediction residual D to the selection unit 172 or the selection unit 176 based on each value that is a candidate for TU.TSflag.
具体的には、TU.TSflagの候補となる値が、第1の直交変換をスキップすることを示す1である場合、選択部171は、予測残差Dを選択部176に供給する。これにより、第1の直交変換がスキップされる。また、TU.TSflagの候補となる値が、第1の直交変換をスキップしないことを示す0である場合、選択部171は、予測残差Dを選択部172に供給する。
Specifically, when the value that is a candidate of TU.TSflag is 1 indicating that the first orthogonal transform is skipped, the selection unit 171 supplies the prediction residual D to the selection unit 176. Thereby, the first orthogonal transform is skipped. When the value that is a candidate for TU.TSflag is 0 indicating that the first orthogonal transform is not skipped, the selection unit 171 supplies the prediction residual D to the selection unit 172.
選択部172は、垂直方向のTU.AMTflagとTU.AMTIdxの候補となる各値を決定する。選択部172は、垂直方向のTU.AMTflagとTU.AMTIdxの候補となる値ごとに、その値と、制御部101から供給される予測情報Pinfo等に基づいて、垂直方向の第1の直交変換行列を選択する。そして、選択部172は、垂直方向のTU.AMTflagとTU.AMTIdxの候補となる値ごとに、選択部171から供給される予測残差Dを、選択された垂直方向の第1の直交変換行列に対応する垂直DCT2変換部173-1、垂直DCT5変換部173-2、垂直DCT8変換部173-3、垂直DST1変換部173-4、または垂直DST7変換部173-5に供給する。
The selection unit 172 determines each value as a candidate for TU.AMTflag and TU.AMTIdx in the vertical direction. The selection unit 172 performs first orthogonal transformation in the vertical direction on the basis of the values, prediction information Pinfo and the like supplied from the control unit 101, for each value that is a candidate for vertical TU.AMTflag and TU.AMTIdx Select a matrix. Then, the selection unit 172 converts the prediction residual D supplied from the selection unit 171 into the selected first orthogonal transformation matrix in the vertical direction for each value that is a candidate for the vertical TU.AMTflag and TU.AMTIdx. To the vertical DCT2 conversion unit 173-1, vertical DCT5 conversion unit 173-2, vertical DCT8 conversion unit 173-3, vertical DST1 conversion unit 173-4, or vertical DST7 conversion unit 173-5.
垂直DCT2変換部173-1、垂直DCT5変換部173-2、垂直DCT8変換部173-3、垂直DST1変換部173-4、および垂直DST7変換部173-5(第1の直交変換部)は、それぞれ、所定の第1の直交変換行列を用いて予測残差Dに対して垂直方向にAMT(Adaptive Multiple Transform)を第1の直交変換として行う。垂直DCT2変換部173-1、垂直DCT5変換部173-2、垂直DCT8変換部173-3、垂直DST1変換部173-4、および垂直DST7変換部173-5で用いられる第1の直交変換行列は、それぞれ、DCT2,DCT5,DCT8,DST1,DST7である。
Vertical DCT2 conversion unit 173-1, vertical DCT5 conversion unit 173-2, vertical DCT8 conversion unit 173-3, vertical DST1 conversion unit 173-4, and vertical DST7 conversion unit 173-5 (first orthogonal conversion unit) are: Respectively, AMT (Adaptive Multi Transform) is performed as the first orthogonal transformation in the vertical direction with respect to the prediction residual D using a predetermined first orthogonal transformation matrix. The first orthogonal transformation matrix used in the vertical DCT2 conversion unit 173-1, the vertical DCT5 conversion unit 173-2, the vertical DCT8 conversion unit 173-3, the vertical DST1 conversion unit 173-4, and the vertical DST7 conversion unit 173-5 is , DCT2, DCT5, DCT8, DST1, DST7, respectively.
AMTおよびAMTで用いられる行列の詳細は、J.Chen等,”Algorithm Description of Joint Exploration Test Model 3(JVET-C1001)”,JVET of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11,26 May-1 June 2016に記載されている。
Details of the matrix used in AMT and AMT can be found in J. Chen et al., “Algorithm Description of Joint Exploration Test Model 3 (JVET-C1001)”, JVET of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11,26 It is described in May-1 June 2016.
垂直DCT2変換部173-1、垂直DCT5変換部173-2、垂直DCT8変換部173-3、垂直DST1変換部173-4、および垂直DST7変換部173-5は、垂直方向の第1の直交変換の結果得られる変換係数を選択部174に供給する。
The vertical DCT2 conversion unit 173-1, the vertical DCT5 conversion unit 173-2, the vertical DCT8 conversion unit 173-3, the vertical DST1 conversion unit 173-4, and the vertical DST7 conversion unit 173-5 include a first orthogonal transform in the vertical direction. The conversion coefficient obtained as a result of is supplied to the selection unit 174.
選択部174は、水平方向のTU.AMTflagとTU.AMTIdxの候補となる各値を決定する。選択部174は、水平方向のTU.AMTflagとTU.AMTIdxの候補となる値ごとに、その値と、制御部101から供給される予測情報Pinfo等に基づいて、水平方向の第1の直交変換行列を選択する。そして、選択部174は、水平方向のTU.AMTflagとTU.AMTIdxの候補となる値ごとに、垂直方向の第1の直交変換の結果得られる変換係数を、選択された水平方向の第1の直交変換行列に対応する水平DCT2変換部175-1、水平DCT5変換部175-2、水平DCT8変換部175-3、水平DST1変換部175-4、または水平DST7変換部175-5に供給する。
The selection unit 174 determines each value as a candidate for the horizontal TU.AMTflag and TU.AMTIdx. The selection unit 174 performs, for each value that is a candidate for the horizontal TU.AMTflag and TU.AMTIdx, the first orthogonal transformation in the horizontal direction based on the value, the prediction information Pinfo supplied from the control unit 101, and the like. Select a matrix. Then, the selection unit 174 uses, for each value that becomes a candidate for the horizontal TU.AMTflag and TU.AMTIdx, the conversion coefficient obtained as a result of the first orthogonal transformation in the vertical direction, This is supplied to the horizontal DCT2 conversion unit 175-1, horizontal DCT5 conversion unit 175-2, horizontal DCT8 conversion unit 175-3, horizontal DST1 conversion unit 175-4, or horizontal DST7 conversion unit 175-5 corresponding to the orthogonal transformation matrix.
水平DCT2変換部175-1、水平DCT5変換部175-2、水平DCT8変換部175-3、水平DST1変換部175-4、および水平DST7変換部175-5(第1の直交変換部)は、それぞれ、所定の第1の直交変換行列を用いて垂直方向の第1の直交変換の結果得られる変換係数に対して水平方向にAMTを第1の直交変換として行う。水平DCT2変換部175-1、水平DCT5変換部175-2、水平DCT8変換部175-3、水平DST1変換部175-4、および水平DST7変換部175-5で用いられる第1の直交変換行列は、それぞれ、DCT2,DCT5,DCT8,DST1,DST7である。
Horizontal DCT2 conversion unit 175-1, horizontal DCT5 conversion unit 175-2, horizontal DCT8 conversion unit 175-3, horizontal DST1 conversion unit 175-4, and horizontal DST7 conversion unit 175-5 (first orthogonal conversion unit) are: In each case, AMT is performed as the first orthogonal transformation in the horizontal direction on the transformation coefficient obtained as a result of the first orthogonal transformation in the vertical direction using a predetermined first orthogonal transformation matrix. The first orthogonal transform matrix used in the horizontal DCT2 conversion unit 175-1, horizontal DCT5 conversion unit 175-2, horizontal DCT8 conversion unit 175-3, horizontal DST1 conversion unit 175-4, and horizontal DST7 conversion unit 175-5 is , DCT2, DCT5, DCT8, DST1, DST7, respectively.
水平DCT2変換部175-1、水平DCT5変換部175-2、水平DCT8変換部175-3、水平DST1変換部175-4、および水平DST7変換部175-5は、水平方向の第1の直交変換の結果得られる変換係数を選択部176に供給する。
The horizontal DCT2 conversion unit 175-1, the horizontal DCT5 conversion unit 175-2, the horizontal DCT8 conversion unit 175-3, the horizontal DST1 conversion unit 175-4, and the horizontal DST7 conversion unit 175-5 are first orthogonal transforms in the horizontal direction. The conversion coefficient obtained as a result of is supplied to the selection unit 176.
選択部176は、制御部101から供給される予測情報Pinfo等に基づいて、TU.ROTIdxの候補となる各値を決定する。
The selection unit 176 determines each value as a candidate for TU.ROTIdx based on the prediction information Pinfo supplied from the control unit 101.
具体的には、予測情報Pinfoのうちのモード情報pred_mode_flagがイントラ予測処理系の処理を示し、PU.IntraBCflagがイントラBC予測処理ではないことを示す場合、選択部176は、TU.ROTIdxの候補となる値を0乃至3に決定する。
Specifically, when the mode information pred_mode_flag in the prediction information Pinfo indicates processing of the intra prediction processing system and PU.IntraBCflag indicates that it is not intra BC prediction processing, the selection unit 176 selects the TU.ROTIdx candidate and Is determined to be 0 to 3.
一方、モード情報pred_mode_flagがイントラ予測処理系の処理を示し、PU.IntraBCflagがイントラBC予測処理であることを示す場合、または、モード情報pred_mode_flagがインター予測処理を示す場合、選択部176は、TU.ROTIdxの候補となる値を、第2の直交変換行列として無変換行列を特定するインデックスである3に決定する。
On the other hand, when the mode information pred_mode_flag indicates intra prediction processing and the PU.IntraBCflag indicates intra BC prediction processing, or when the mode information pred_mode_flag indicates inter prediction processing, the selection unit 176 selects the TU. The candidate value of ROTIdx is determined to be 3 which is an index for specifying the non-transformed matrix as the second orthogonal transform matrix.
以上により、イントラ予測処理が行われる場合、第2の直交変換を行うことができるが、イントラBC予測処理またはインター予測処理が行われる場合、第2の直交変換を行うことはできない。
As described above, when the intra prediction process is performed, the second orthogonal transform can be performed. However, when the intra BC prediction process or the inter prediction process is performed, the second orthogonal transform cannot be performed.
即ち、イントラ予測処理が行われる場合、予測残差Dの特性はイントラ予測モードに依存するため、イントラ予測モードに基づいた冗長性が、第1の直交変換の結果得られる変換係数に残っている。従って、この場合、その冗長性を取り除くためにさらに第2の直交変換を行うことができるようにされる。
That is, when the intra prediction process is performed, the characteristic of the prediction residual D depends on the intra prediction mode, so that redundancy based on the intra prediction mode remains in the transform coefficient obtained as a result of the first orthogonal transform. . Therefore, in this case, the second orthogonal transform can be further performed to remove the redundancy.
しかしながら、イントラBC予測処理は、符号化対象のPUから動きベクトルだけ離れた画面内の位置に存在するブロックを予測画像として生成する処理であり、符号化対象のPUの周囲のブロックを予測画像とするイントラ予測処理とは特性が異なる。
However, the intra BC prediction process is a process of generating, as a predicted image, a block that exists at a position in the screen that is separated from the encoding target PU by a motion vector, and blocks around the encoding target PU are defined as a predicted image. The characteristics are different from the intra prediction process.
従って、イントラ予測処理が行われた場合の予測残差Dの冗長性を取り除くことに適した第2の直交変換行列は、イントラBC予測処理が行われた場合の予測残差Dの冗長性を取り除くことには適していない。よって、変換部112では、イントラBC予測処理が行われる場合、インター予測処理と同様に、第2の直交変換行列の候補が無変換行列のみとされ、第2の直交変換は行われない。即ち、イントラBC予測処理用の第2の直交変換行列の候補が、イントラ処理用の第2の直交変換行列の候補とは異なり、インター予測処理用の第2の直交変換行列の候補と同一である無変換行列のみにされる。
Therefore, the second orthogonal transformation matrix suitable for removing the redundancy of the prediction residual D when the intra prediction process is performed is the redundancy of the prediction residual D when the intra BC prediction process is performed. Not suitable for removing. Therefore, in the case where the intra BC prediction process is performed in the transform unit 112, the second orthogonal transform matrix candidates are only non-transformed matrices and the second orthogonal transform is not performed, as in the inter prediction process. That is, the second orthogonal transformation matrix candidate for intra BC prediction processing is the same as the second orthogonal transformation matrix candidate for inter prediction processing, unlike the second orthogonal transformation matrix candidate for intra processing. Only certain non-transformed matrices are made.
選択部176は、TU.ROTIdxの候補となる各値に基づいて、供給される変換係数または予測残差Dを、NSST0変換部177-1、NSST1変換部177-2、NSST2変換部177-3、および無変換部177-4の少なくとも1つに供給する。具体的には、TU.ROTIdx(TU.NSSTIdx)の候補となる値が、それぞれ、0,1,2,3である場合、選択部176は、変換係数を、それぞれ、NSST0変換部177-1、NSST1変換部177-2、NSST2変換部177-3、無変換部177-4に供給する。
The selection unit 176 converts the supplied transform coefficient or prediction residual D based on each value that is a candidate for TU.ROTIdx into the NSST0 conversion unit 177-1, NSST1 conversion unit 177-2, NSST2 conversion unit 177-3. , And no-conversion unit 177-4. Specifically, when the candidate values for TU.ROTIdx (TU.NSSTIdx) are 0, 1, 2, and 3, respectively, the selection unit 176 selects the conversion coefficient and the NSST0 conversion unit 177-1. , NSST1 conversion unit 177-2, NSST2 conversion unit 177-3, non-conversion unit 177-4.
NSST0変換部177-1、NSST1変換部177-2、NSST2変換部177-3、および無変換部177-4(第2の直交変換部)は、所定のインデックスにより特定される第2の直交変換行列を用いて、供給される変換係数または予測残差Dに対して、4(横)×4(縦)画素ごとに、MD-NSST(Mode dependent non-separable secondary transforms)を第2の直交変換として行う。MD-NSSTの詳細は、J.Chen等,”Algorithm Description of Joint Exploration Test Model 3(JVET-C1001)”,JVET of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11,26 May-1 June 2016に記載されている。
NSST0 conversion unit 177-1, NSST1 conversion unit 177-2, NSST2 conversion unit 177-3, and non-conversion unit 177-4 (second orthogonal conversion unit) include a second orthogonal conversion specified by a predetermined index. Using a matrix, MD-NSST (Mode dependent non-separable secondary transforms) is second orthogonally transformed for every 4 (horizontal) x 4 (vertical) pixels for the supplied transform coefficient or prediction residual D Do as. For details on MD-NSST, see J. Chen et al., “Algorithm Description Description of Joint Joint Exploration Test Test Model 3 (JVET-C1001)”, JVET Standard of ITU-T Standard SG16 Standard WP3 Standard and ISO / IEC Standard JTC1 / SC29 / WG11, 26 Standard May-1 June It is described in 2016.
NSST0変換部177-1、NSST1変換部177-2、およびNSST2変換部177-3で用いられる各インデックスにより特定される第2の直交変換行列は、例えば、イントラ予測モードに応じて異なる。
The second orthogonal transformation matrix specified by each index used in the NSST0 conversion unit 177-1, NSST1 conversion unit 177-2, and NSST2 conversion unit 177-3 differs depending on, for example, the intra prediction mode.
NSST0変換部177-1、NSST1変換部177-2、NSST2変換部177-3、および無変換部177-4は、第2の直交変換の結果得られる変換係数を決定部178に供給する。なお、無変換部177-4で用いられる第2の直交変換行列は無変換行列であるため、無変換部177-4から決定部178に供給される変換係数は、選択部176から供給される変換係数そのものである。
NSST0 transform unit 177-1, NSST1 transform unit 177-2, NSST2 transform unit 177-3, and no transform unit 177-4 supply transform coefficients obtained as a result of the second orthogonal transform to determination unit 178. Note that since the second orthogonal transformation matrix used in the non-conversion unit 177-4 is a non-conversion matrix, the transform coefficient supplied from the non-conversion unit 177-4 to the determination unit 178 is supplied from the selection unit 176. It is the conversion coefficient itself.
決定部178は、NSST0変換部177-1、NSST1変換部177-2、NSST2変換部177-3、および無変換部177-4から供給される変換係数に基づいて、変換情報Tinfoの可変値の候補となる各値に対応するRDコストを算出する。そして、決定部178は、RDコストが最小となる変換情報Tinfoの可変値の候補となる値を最適値に決定する。決定部178は、変換情報Tinfoの可変値の最適値を図2の制御部101に供給し、その最適値に対応する変換係数を変換係数Coeffとして量子化部113に供給する。
The determination unit 178 determines the variable value of the conversion information Tinfo based on the conversion coefficients supplied from the NSST0 conversion unit 177-1, NSST1 conversion unit 177-2, NSST2 conversion unit 177-3, and non-conversion unit 177-4. An RD cost corresponding to each candidate value is calculated. Then, the determination unit 178 determines a value that is a candidate for a variable value of the conversion information Tinfo that minimizes the RD cost as an optimum value. The determination unit 178 supplies the optimal value of the variable value of the conversion information Tinfo to the control unit 101 in FIG. 2, and supplies the conversion coefficient corresponding to the optimal value to the quantization unit 113 as the conversion coefficient Coeff.
なお、イントラBC予測処理が行われる場合に第2の直交変換を可能にするようにしてもよい。この場合、イントラBC予測処理により生成された予測画像Pの予測残差Dに基づく冗長性を取り除くことに適した3つの第2の直交変換行列が、イントラBC予測処理用の第2の直交変換行列の候補として用意される。そして、イントラBC予測処理が行われる場合、このイントラBC予測処理用の第2の直交変換行列の候補のそれぞれが、0乃至3のインデックスにより特定される第2の直交変換行列となり、TU.ROTIdxの候補となる値が0乃至3に決定される。
Note that the second orthogonal transform may be enabled when the intra BC prediction process is performed. In this case, three second orthogonal transform matrices suitable for removing redundancy based on the prediction residual D of the predicted image P generated by the intra BC prediction process are the second orthogonal transforms for the intra BC prediction process. Prepared as a matrix candidate. When intra BC prediction processing is performed, each of the candidates for the second orthogonal transformation matrix for intra BC prediction processing becomes the second orthogonal transformation matrix specified by indices 0 to 3, and TU.ROTIdx The candidate values are determined to be 0 to 3.
以上のように、変換部112では、イントラBC予測処理用の第2の直交変換行列の候補が、イントラ予測処理用の第2の直交変換行列の候補とは異なる。従って、変換部112は、イントラBC予測処理が行われる場合、イントラBC予測処理に適した第2の直交変換を行うことができる。
As described above, in the transform unit 112, the candidate for the second orthogonal transform matrix for intra BC prediction processing is different from the candidate for the second orthogonal transform matrix for intra prediction processing. Therefore, when the intra BC prediction process is performed, the transform unit 112 can perform the second orthogonal transform suitable for the intra BC prediction process.
(第1の直交変換行列の選択方法の説明)
図5は、第1の直交変換行列の選択方法を説明する図である。 (Description of selection method of first orthogonal transformation matrix)
FIG. 5 is a diagram for explaining a method of selecting the first orthogonal transformation matrix.
図5は、第1の直交変換行列の選択方法を説明する図である。 (Description of selection method of first orthogonal transformation matrix)
FIG. 5 is a diagram for explaining a method of selecting the first orthogonal transformation matrix.
選択部172は、SPS.AMTflagが第1の直交変換を有効にすることを示す場合、垂直方向のPU.AMTflagの候補となる値を0と1に決定する。また、選択部172は、垂直方向のPU.AMTflagの候補となる値が0である場合、垂直方向のTU.AMTIdxの候補となる値を決定せず、垂直方向のPU.AMTflagの候補となる値が1である場合、垂直方向のTU.AMTIdxの候補となる値を0と1に決定する。
When the SPS.AMTflag indicates that the first orthogonal transform is valid, the selection unit 172 determines 0 and 1 as candidates for the vertical PU.AMTflag. In addition, when the value that is a candidate for vertical PU.AMTflag is 0, the selection unit 172 does not determine a value that becomes a candidate for vertical TU.AMTIdx, and becomes a candidate for vertical PU.AMTflag. If the value is 1, the candidate values for TU.AMTIdx in the vertical direction are determined to be 0 and 1.
モード情報pred_mode_flagがイントラ予測処理系の処理を示し、SPS.IntraBCflagがイントラBC予測処理が有効ではないことを示すか、または、PU.IntraBCflagがイントラBC予測処理ではないことを示す場合、即ちイントラ予測処理が行われる場合、選択部172は、図5のAに示すイントラ予測処理用の垂直方向の第1の直交変換行列の候補の中から、垂直方向の第1の直交変換行列を選択する。
When the mode information pred_mode_flag indicates the processing of the intra prediction processing system and SPS.IntraBCflag indicates that the intra BC prediction processing is not valid, or indicates that the PU.IntraBCflag is not the intra BC prediction processing, that is, intra prediction When the process is performed, the selection unit 172 selects the first orthogonal orthogonal transformation matrix from the candidates for the first orthogonal orthogonal transformation matrix for intra prediction processing illustrated in A of FIG.
具体的には、選択部172は、TU.AMTflagの候補となる値が0であるとき、垂直方向(Vertical)の第1の直交変換行列としてDCT2を選択する。一方、TU.AMTflagの候補となる値が1であるとき、選択部172は、イントラ予測モード(Intramode)を示す情報と垂直方向のTU.AMTIdxの候補となる各値に基づいて、垂直方向の第1の直交変換行列を選択する。
Specifically, the selection unit 172 selects DCT2 as the first orthogonal transformation matrix in the vertical direction (Vertical) when the candidate value of TU.AMTflag is 0. On the other hand, when the candidate value of TU.AMTflag is 1, the selection unit 172 selects the vertical direction based on the information indicating the intra prediction mode (Intramode) and the respective values that are candidates for the vertical TU.AMTIdx. A first orthogonal transformation matrix is selected.
例えば、垂直方向のTU.AMTIdxの候補となる値が0である場合、選択部172は、DST7を垂直方向の第1の直交変換行列として選択する。また、垂直方向のTU.AMTIdxの候補となる値が1である場合、選択部172は、イントラ予測モードを示す情報に基づいて、DCT5,DCT8、またはDST1を垂直方向の第1の直交変換行列として選択する。
For example, when the candidate value of TU.AMTIdx in the vertical direction is 0, the selection unit 172 selects DST7 as the first orthogonal transformation matrix in the vertical direction. When the candidate value of TU.AMTIdx in the vertical direction is 1, the selection unit 172 converts DCT5, DCT8, or DST1 into the first orthogonal orthogonal matrix in the vertical direction based on the information indicating the intra prediction mode. Choose as.
また、モード情報pred_mode_flagがインター予測処理を示す場合、即ちインター予測処理が行われる場合、選択部172は、図5のBに示すインター予測処理用の垂直方向の第1の直交変換行列の候補の中から、垂直方向の第1の直交変換行列を選択する。
Further, when the mode information pred_mode_flag indicates inter prediction processing, that is, when inter prediction processing is performed, the selection unit 172 selects candidates for the first orthogonal transform matrix in the vertical direction for inter prediction processing illustrated in B of FIG. 5. A first orthogonal transformation matrix in the vertical direction is selected from the inside.
具体的には、選択部172は、TU.AMTflagの候補となる値が0であるとき、垂直方向の第1の直交変換行列としてDCT2を選択する。一方、TU.AMTflagの候補となる値が1であるとき、選択部172は、垂直方向のTU.AMTIdxの候補となる各値に基づいて、垂直方向の第1の直交変換行列を選択する。
Specifically, the selection unit 172 selects DCT2 as the first orthogonal transformation matrix in the vertical direction when the candidate value of TU.AMTflag is 0. On the other hand, when the value that is a candidate for TU.AMTflag is 1, the selection unit 172 selects the first orthogonal transformation matrix in the vertical direction based on each value that is a candidate for TU.AMTIdx in the vertical direction.
例えば、垂直方向のTU.AMTIdxの候補となる値が0である場合、選択部172は、DCT8を垂直方向の第1の直交変換行列として選択する。また、垂直方向のTU.AMTIdxの候補となる値が1である場合、選択部172は、DST7を垂直方向の第1の直交変換行列として選択する。
For example, when the candidate value of TU.AMTIdx in the vertical direction is 0, the selection unit 172 selects DCT8 as the first orthogonal transformation matrix in the vertical direction. When the candidate value for TU.AMTIdx in the vertical direction is 1, the selection unit 172 selects DST7 as the first orthogonal transformation matrix in the vertical direction.
さらに、モード情報pred_mode_flagがイントラ予測処理系の処理を示し、SPS.IntraBCflagがイントラBC予測処理が有効であることを示し、PU.IntraBCflagがイントラBC予測処理であることを示す場合、即ちイントラBC予測処理が行われる場合、選択部172は、図5のCに示すイントラBC予測処理用の垂直方向の第1の直交変換行列の中から、垂直方向の第1の直交変換行列を選択する。
Further, when the mode information pred_mode_flag indicates processing of the intra prediction processing system, SPS.IntraBCflag indicates that the intra BC prediction processing is valid, and PU.IntraBCflag indicates that it is intra BC prediction processing, that is, intra BC prediction. When the process is performed, the selection unit 172 selects the first orthogonal orthogonal transformation matrix from the first orthogonal orthogonal transformation matrix for intra BC prediction processing illustrated in C of FIG.
具体的には、図5のBは図5のCと同一であり、選択部172は、イントラBC予測処理が行われる場合、インター予測処理が行われる場合と同様に、垂直方向の第1の直交変換行列を選択する。
Specifically, B in FIG. 5 is the same as C in FIG. 5, and the selection unit 172 performs the first vertical direction when the intra BC prediction process is performed as in the case where the inter prediction process is performed. Select an orthogonal transformation matrix.
選択部174による水平方向(Horizontal)の第1の直交変換行列の選択は、選択部172による垂直方向の第1の直交行列の選択と同様であるので、説明は省略する。
Since the selection of the first orthogonal transformation matrix in the horizontal direction by the selection unit 174 is the same as the selection of the first orthogonal matrix in the vertical direction by the selection unit 172, description thereof will be omitted.
以上のように、TU.AMTflagとTU.AMTIdxの候補となる各値に基づいて選択される垂直方向および水平方向の第1の直交変換行列、即ち第1の直交変換行列の候補は、行われる予測処理によって異なっている。
As described above, the first orthogonal transformation matrix in the vertical direction and the horizontal direction selected based on the respective values as candidates for TU.AMTflag and TU.AMTIdx, that is, the candidate for the first orthogonal transformation matrix is performed. It depends on the prediction process.
なお、図5の例では、イントラBC予測処理が行われる場合の第1の直交変換行列の選択方法は、インター予測処理が行われる場合と同様であるが、イントラ予測処理が行われる場合と異なっていれば、インター予測処理が行われる場合と異なっていてもよい。即ち、イントラBC予測処理用の第1の直交変換行列の候補は、イントラ予測処理用の第1の直交変換行列の候補と異なっていれば、インター予測処理用の第1の直交変換行列の候補と異なっていてもよい。
In the example of FIG. 5, the selection method of the first orthogonal transform matrix when the intra BC prediction process is performed is the same as that when the inter prediction process is performed, but is different from the case where the intra prediction process is performed. If it is, it may differ from the case where the inter prediction process is performed. That is, if the candidate for the first orthogonal transform matrix for intra BC prediction processing is different from the candidate for the first orthogonal transform matrix for intra prediction processing, the candidate for the first orthogonal transform matrix for inter prediction processing And may be different.
イントラBC予測処理用の第1の直交変換行列の候補がインター予測処理用の第1の直交変換行列の候補と異なる場合、イントラBC予測処理用の第1の直交変換行列の候補は、第1の直交変換の結果得られる変換係数のエネルギー集中度が高くなるように設定される。また、イントラBC予測処理用の第1の直交変換行列の候補がインター予測処理用の第1の直交変換行列の候補と同一である場合、選択部172および選択部174において用いられる、TU.AMTflagとTU.AMTIdxの候補となる各値と第1の直交変換行列の関係を示すテーブル(例えば、図5のBや図5のCのテーブル)をイントラBC予測処理時とインター予測処理時で共有することができる。従って、このテーブルを記憶する図示せぬメモリの記憶容量を削減することができる。
When the first orthogonal transformation matrix candidate for intra BC prediction processing is different from the first orthogonal transformation matrix candidate for inter prediction processing, the first orthogonal transformation matrix candidate for intra BC prediction processing is the first The degree of energy concentration of the transform coefficient obtained as a result of the orthogonal transform is set to be high. Further, when the first orthogonal transformation matrix candidate for intra BC prediction processing is the same as the first orthogonal transformation matrix candidate for inter prediction processing, TU.AMTflag used in the selection unit 172 and the selection unit 174 is used. And a table indicating the relationship between each candidate value of TU.AMTIdx and the first orthogonal transformation matrix (for example, the table of B of FIG. 5 or the table of C of FIG. 5) is shared during intra BC prediction processing and inter prediction processing. can do. Accordingly, the storage capacity of a memory (not shown) that stores this table can be reduced.
(画像符号化装置の処理の説明)
図6は、図1の画像符号化装置100の画像符号化処理を説明するフローチャートである。 (Description of processing of image encoding device)
FIG. 6 is a flowchart for explaining image coding processing of theimage coding apparatus 100 in FIG.
図6は、図1の画像符号化装置100の画像符号化処理を説明するフローチャートである。 (Description of processing of image encoding device)
FIG. 6 is a flowchart for explaining image coding processing of the
図6のステップS101において、制御部101は、外部からの入力等に基づいて、符号化パラメータのうちの固定値を設定する。制御部101は、ヘッダ情報Hinfoの固定値を、例えば各部に供給し、予測情報Pinfoの固定値を、例えば予測部119に供給し、変換情報Tinfoの固定値を、例えば、変換部112、量子化部113、逆量子化部115、および逆変換部116に供給する。
In step S101 in FIG. 6, the control unit 101 sets a fixed value among the encoding parameters based on an external input or the like. The control unit 101 supplies a fixed value of the header information Hinfo to each unit, for example, supplies a fixed value of the prediction information Pinfo, for example, to the prediction unit 119, and sets a fixed value of the conversion information Tinfo, for example, the conversion unit 112, the quantum To the quantization unit 113, the inverse quantization unit 115, and the inverse transformation unit 116.
ステップS102において、予測部119は、予測情報Pinfoの可変値を最適化し、予測情報Pinfoの可変値の最適値に対応する予測画像Pを生成する予測処理を行う。予測部119は、予測情報Pinfoの可変値の最適値を制御部101に供給し、予測情報Pinfoの可変値の最適値に対応する予測画像Pを演算部111と演算部117に供給する。制御部101は、予測情報Pinfoのうちのsplit flagの最適値等を演算部111に供給する。
In step S102, the prediction unit 119 performs a prediction process of optimizing the variable value of the prediction information Pinfo and generating a predicted image P corresponding to the optimal value of the variable value of the prediction information Pinfo. The prediction unit 119 supplies the optimal value of the variable value of the prediction information Pinfo to the control unit 101, and supplies the prediction image P corresponding to the optimal value of the variable value of the prediction information Pinfo to the calculation unit 111 and the calculation unit 117. The control unit 101 supplies the calculation unit 111 with the optimum value of split flag in the prediction information Pinfo.
ステップS103において、演算部111は、split flagの最適値に基づいてピクチャに対して符号化対象のCU(PU,TU)を設定し、符号化対象のPUの画像Iと予測部119から供給される予測画像Pとの差分を予測残差Dとして演算する。このようにして求められた予測残差Dは、元の画像Iに比べてデータ量が低減される。したがって、画像Iをそのまま符号化する場合に比べて、データ量を圧縮することができる。
In step S103, the calculation unit 111 sets the CU (PU, TU) to be encoded for the picture based on the optimum value of split flag, and is supplied from the image I of the PU to be encoded and the prediction unit 119. The difference from the predicted image P is calculated as a predicted residual D. The amount of data of the prediction residual D obtained in this way is reduced compared to the original image I. Therefore, the data amount can be compressed as compared with the case where the image I is encoded as it is.
ステップS104において、変換部112は、変換情報Tinfoの可変値を最適化し、変換情報Tinfoの可変値の最適値に対応する変換係数Coeffを導出する変換処理を行う。この変換処理の詳細は、後述する図8を参照して説明する。
In step S104, the conversion unit 112 performs a conversion process of optimizing the variable value of the conversion information Tinfo and deriving the conversion coefficient Coeff corresponding to the optimal value of the variable value of the conversion information Tinfo. Details of this conversion processing will be described with reference to FIG.
ステップS105において、量子化部113は、変換情報Tinfoの可変値の候補となる各値に基づいて、変換部112から供給される変換係数Coeffを量子化し、量子化変換係数レベルlevelを導出する。変換部112は、変換情報Tinfoの可変値の最適化を行い、変換情報Tinfoの可変値の最適値に対応する量子化変換係数レベルlevelを逆量子化部115に供給するとともに、変換情報Tinfoの可変値の最適値を制御部101に供給する。制御部101は、予測情報Pinfo、変換情報Tinfo等の可変値の最適値と固定値を含む符号化パラメータを設定して符号化部114に供給し、変換情報Tinfoの可変値の最適値を逆量子化部115および逆変換部116に供給する。
In step S105, the quantization unit 113 quantizes the transform coefficient Coeff supplied from the transform unit 112 on the basis of each variable value candidate of the transform information Tinfo, and derives a quantized transform coefficient level level. The conversion unit 112 optimizes the variable value of the conversion information Tinfo, supplies the quantized conversion coefficient level level corresponding to the optimal value of the variable value of the conversion information Tinfo to the inverse quantization unit 115, and also converts the conversion information Tinfo of the conversion information Tinfo. The optimum variable value is supplied to the control unit 101. The control unit 101 sets an encoding parameter including an optimal value and a fixed value of variable values such as prediction information Pinfo and conversion information Tinfo, and supplies the encoding parameter 114 to the encoding unit 114, and reverses the optimal value of the variable value of conversion information Tinfo. This is supplied to the quantization unit 115 and the inverse transformation unit 116.
ステップS106において、逆量子化部115は、符号化パラメータの固定値および可変値の最適値に基づいて、量子化部113から供給される量子化変換係数レベルlevelを、ステップS105の量子化の特性に対応する特性で逆量子化する。逆量子化部115は、その結果得られる変換係数Coeff_IQを逆変換部116に供給する。
In step S106, the inverse quantization unit 115 calculates the quantization transform coefficient level level supplied from the quantization unit 113 based on the fixed value and the optimum variable value of the encoding parameter, and the quantization characteristics in step S105. Inverse quantization with characteristics corresponding to. The inverse quantization unit 115 supplies the transform coefficient Coeff_IQ obtained as a result to the inverse transform unit 116.
ステップS107において、逆変換部116は、制御部101から供給される符号化パラメータの固定値および可変値の最適値に基づいて、逆量子化部115から供給される変換係数Coeff_IQに対して、ステップS104の変換処理に対応する方法で逆変換処理を行い、予測残差D’を導出する。
In step S107, the inverse transform unit 116 performs a step on the transform coefficient Coeff_IQ supplied from the inverse quantization unit 115 based on the fixed value and the optimum variable value supplied from the control unit 101. Inverse transformation processing is performed by a method corresponding to the transformation processing in S104 to derive a prediction residual D ′.
ステップS108において、演算部117は、ステップS107の処理により導出された予測残差D’を、予測部119から供給される予測画像Pと加算することにより、局所的な復号画像Recを生成する。
In step S108, the calculation unit 117 generates a local decoded image Rec by adding the prediction residual D 'derived by the processing in step S107 to the prediction image P supplied from the prediction unit 119.
ステップS109において、フレームメモリ118は、ステップS108の処理により得られた、局所的な復号画像Recを用いてピクチャ単位の復号画像を再構築し、フレームメモリ118内のバッファへ記憶する。
In step S109, the frame memory 118 reconstructs a decoded image in units of pictures using the local decoded image Rec obtained by the processing in step S108, and stores it in a buffer in the frame memory 118.
ステップS110において、符号化部114は、ステップS105の処理により得られた量子化変換係数レベルlevelを符号化する。例えば、符号化部114は、画像に関する情報である量子化変換係数レベルlevelを、算術符号化等により符号化する。また、このとき、符号化部114は、制御部101から供給される符号化パラメータ(ヘッダ情報Hinfo、予測情報Pinfo、変換情報Tinfo)を符号化する。符号化部114は、このような符号化により生成された符号化データをまとめて、符号化ストリームとして画像符号化装置100の外部に出力する。この符号化ストリームは、例えば、伝送路や記録媒体を介して復号側に伝送される。
In step S110, the encoding unit 114 encodes the quantized transform coefficient level level obtained by the process in step S105. For example, the encoding unit 114 encodes a quantized transform coefficient level level, which is information about an image, by arithmetic coding or the like. At this time, the encoding unit 114 encodes the encoding parameters (header information Hinfo, prediction information Pinfo, conversion information Tinfo) supplied from the control unit 101. The encoding unit 114 collects the encoded data generated by such encoding and outputs the encoded data as an encoded stream to the outside of the image encoding apparatus 100. This encoded stream is transmitted to the decoding side via a transmission path or a recording medium, for example.
ステップS110の処理が終了すると、画像符号化処理が終了する。
When the process of step S110 ends, the image encoding process ends.
図7は、図6のステップS101の処理のうちの、SPS.IntraBCflag,SPS.AMTflag、およびSPS.NSSTflagを設定するフラグ設定処理を説明するフローチャートである。
FIG. 7 is a flowchart for explaining flag setting processing for setting SPS.IntraBCflag, SPS.AMTflag, and SPS.NSSTflag in the processing of step S101 of FIG.
図7のステップS131において、制御部101は、外部からの入力等に基づいて、IntraBC予測処理を有効にするかどうかを判定する。
7, in step S131, the control unit 101 determines whether or not to enable the IntraBC prediction process based on an external input or the like.
ステップS131でIntraBC予測処理を有効にすると判定された場合、処理はステップS132に進む。ステップS132において、制御部101は、SPS.IntraBCflagを1に設定し、処理をステップS134に進める。
If it is determined in step S131 that the IntraBC prediction process is enabled, the process proceeds to step S132. In step S132, the control unit 101 sets SPS.IntraBCflag to 1, and advances the process to step S134.
一方、ステップS131でIntraBC予測処理を有効にしないと判定された場合、処理はステップS133に進む。ステップS133において、制御部101は、SPS.IntraBCflagを0に設定し、処理をステップS134に進める。
On the other hand, if it is determined in step S131 that the IntraBC prediction process is not enabled, the process proceeds to step S133. In step S133, the control unit 101 sets SPS.IntraBCflag to 0, and the process proceeds to step S134.
ステップS134において、制御部101は、外部からの入力等に基づいて、第1の直交変換を有効にするかどうかを判定する。
In step S134, the control unit 101 determines whether to enable the first orthogonal transformation based on an external input or the like.
ステップS134で第1の直交変換を有効にすると判定された場合、処理はステップS135に進む。ステップS135において、制御部101は、SPS.AMTflagを1に設定し、処理をステップS137に進める。
If it is determined in step S134 that the first orthogonal transform is to be enabled, the process proceeds to step S135. In step S135, the control unit 101 sets SPS.AMTflag to 1, and advances the process to step S137.
一方、ステップS134で第1の直交変換を有効にしないと判定された場合、処理はステップS136に進む。ステップS136において、制御部101は、SPS.AMTflagを0に設定し、処理をステップS137に進める。
On the other hand, if it is determined in step S134 that the first orthogonal transform is not enabled, the process proceeds to step S136. In step S136, the control unit 101 sets SPS.AMTflag to 0, and the process proceeds to step S137.
ステップS137において、制御部101は、外部からの入力等に基づいて、第2の直交変換を有効にするかどうかを判定する。
In step S137, the control unit 101 determines whether to enable the second orthogonal transformation based on an external input or the like.
ステップS137で第2の直交変換を有効にすると判定された場合、処理はステップS138に進む。ステップS138において、制御部101は、SPS.NSSTflagを1に設定し、処理を終了する。
If it is determined in step S137 that the second orthogonal transformation is to be enabled, the process proceeds to step S138. In step S138, the control unit 101 sets SPS.NSSTflag to 1, and ends the process.
一方、ステップS137で第2の直交変換を有効にしないと判定された場合、処理はステップS139に進む。ステップS139において、制御部101は、SPS.NSSTflagを0に設定し、処理を終了する。
On the other hand, if it is determined in step S137 that the second orthogonal transformation is not enabled, the process proceeds to step S139. In step S139, the control unit 101 sets SPS.NSSTflag to 0 and ends the process.
図8は、図6のステップS104の変換処理を説明するフローチャートである。
FIG. 8 is a flowchart for explaining the conversion process in step S104 of FIG.
図8のステップS151において、選択部171は、制御部101から供給されるSPS.AMTflagが1であるかどうかを判定する。ステップS151でSPS.AMTflagが1であると判定された場合、即ち第1の直交変換が有効である場合、処理はステップS152に進む。
8, the selection unit 171 determines whether the SPS.AMTflag supplied from the control unit 101 is “1”. If it is determined in step S151 that SPS.AMTflag is 1, that is, if the first orthogonal transform is valid, the process proceeds to step S152.
ステップS152において、選択部176は、制御部101から供給されるSPS.NSSTflagが1であるかどうかを判定する。ステップS152でSPS.NSSTflagが1であると判定された場合、即ち第2の直交変換が有効である場合、処理はステップS153に進む。
In step S152, the selection unit 176 determines whether or not SPS.NSSTflag supplied from the control unit 101 is 1. If it is determined in step S152 that SPS.NSSTflag is 1, that is, if the second orthogonal transform is valid, the process proceeds to step S153.
ステップS153において、変換部112は、TU.TSflag, TU.AMTflag, TU.AMTIdx、およびTU.ROTIdxの候補となる値として全ての値を決定し、それらの値に基づいて変換係数を生成する最適両変換タイプ決定処理を行う。この最適両変換タイプ決定処理の詳細は、後述する図9を参照して説明する。ステップS153の処理後、処理はステップS158に進む。
In step S153, the conversion unit 112 determines all values as candidate values for TU.TSflag, TU.AMTflag, TU.AMTIdx, and TU.ROTIdx, and generates the conversion coefficient based on these values. Both conversion type determination processes are performed. Details of the optimum both conversion type determination processing will be described with reference to FIG. After the process of step S153, the process proceeds to step S158.
一方、ステップS152でSPS.NSSTflagが1ではないと判定された場合、処理はステップS154に進む。
On the other hand, if it is determined in step S152 that SPS.NSSTflag is not 1, the process proceeds to step S154.
ステップS154において、変換部112は、TU.TSflag, TU.AMTflag、およびTU.AMTIdxの候補となる値として全ての値を決定するとともに、TU.ROTIdxの候補となる値を3に決定し、それらの値に基づいて変換係数を生成する最適第1直交変換タイプ決定処理を行う。この最適第1直交変換タイプ決定処理では、TU.ROTIdxの候補となる値が3であるため、第2の直交変換は行われない。最適第1直交変換タイプ決定処理により生成された、TU.TSflag, TU.AMTflag、およびTU.AMTIdの候補となる各値並びにTU.ROTIdxの候補となる値としての3に対応する変換係数は決定部178に供給される。そして、処理はステップS158に進む。
In step S154, the conversion unit 112 determines all values as candidate values of TU.TSflag, TU.AMTflag, and TU.AMTIdx, and determines a value that is a candidate of TU.ROTIdx as 3, An optimal first orthogonal transform type determination process for generating a transform coefficient based on the value of is performed. In this optimal first orthogonal transform type determination process, the value that is a candidate for TU.ROTIdx is 3, so the second orthogonal transform is not performed. Transform coefficients corresponding to 3 as candidate values of TU.TSflag, TU.AMTflag, and TU.AMTId, and TU.ROTIdx candidates generated by the optimal first orthogonal transform type determination process are determined. To the unit 178. Then, the process proceeds to step S158.
また、ステップS151でSPS.AMTflagが1ではないと判定された場合、処理はステップS155に進む。ステップS155において、選択部176は、SPS.NSSTflagが1であるかどうかを判定する。ステップS155でSPS.NSSTflagが1であると判定された場合、即ち第2の直交変換が有効である場合、処理はステップS156に進む。
If it is determined in step S151 that SPS.AMTflag is not 1, the process proceeds to step S155. In step S155, the selection unit 176 determines whether SPS.NSSTflag is 1. If it is determined in step S155 that SPS.NSSTflag is 1, that is, if the second orthogonal transform is valid, the process proceeds to step S156.
ステップS156において、変換部112は、TU.TSflagの候補となる値を1に決定するとともに、TU.ROTIdxの候補となる値として全ての値を決定し、それらの値に基づいて変換係数を生成する最適第2直交変換タイプ決定処理を行う。この最適第1直交変換タイプ決定処理では、TU.TSflagが1であるため、第1の直交変換は行われない。最適第2直交変換タイプ決定処理により生成された、TU.TSflagの候補となる値としての1およびTU.ROTIdxの候補となる各値に対応する変換係数は決定部178に供給される。そして、処理はステップS158に進む。
In step S156, the conversion unit 112 determines the value that is a candidate for TU.TSflag to be 1, determines all values as values that are candidates for TU.ROTIdx, and generates a conversion coefficient based on those values. The optimum second orthogonal transformation type determination process is performed. In this optimal first orthogonal transform type determination process, since TU.TSflag is 1, the first orthogonal transform is not performed. The transform coefficients corresponding to 1 and TU.ROTIdx candidate values generated by the optimal second orthogonal transform type determination process are supplied to the determination unit 178. Then, the process proceeds to step S158.
一方、ステップS155でSPS.NSSTflagが0であると判定された場合、選択部171は、TU.TSflagの候補となる値を1に決定して予測残差Dを選択部176に供給し、選択部176は、TU.ROTIdxの候補となる値を3に決定して予測残差Dを無変換部177-4に供給する。
On the other hand, when it is determined in step S155 that SPS.NSSTflag is 0, the selection unit 171 determines the candidate value of TU.TSflag as 1 and supplies the prediction residual D to the selection unit 176 for selection. The unit 176 determines the candidate value of TU.ROTIdx as 3 and supplies the prediction residual D to the non-conversion unit 177-4.
そして、ステップS157において、無変換部177-4は、選択部176から供給される予測残差Dに対して無変換行列を用いて第2の直交変換を行うことにより、予測残差Dをそのまま変換係数とする。そして、無変換部177-4は、予測残差Dである変換係数を決定部178に供給し、処理をステップS158に進める。
In step S157, the non-conversion unit 177-4 performs the second orthogonal transform using the non-conversion matrix on the prediction residual D supplied from the selection unit 176, and thus uses the prediction residual D as it is. Conversion factor. Then, the non-conversion unit 177-4 supplies the conversion coefficient, which is the prediction residual D, to the determination unit 178, and the process proceeds to step S158.
ステップS158において、決定部178は、供給される各変換係数等に基づいてRDコストを算出し、そのRDコストが最小となるTU.TSflag, TU.AMTflag, TU.AMTIdx,TU.ROTIdxを、変換情報Tinfoの可変値の最適値に決定する。そして、決定部178は、変換情報Tinfoの可変値の最適値を制御部101に供給し、その最適値に対応する変換係数を変換係数Coeffとして量子化部113に供給する。ステップS158の処理後、処理は図6のステップS104に戻り、ステップS105に進む。
In step S158, the determination unit 178 calculates an RD cost based on each supplied conversion coefficient and the like, and converts TU.TSflag, TU.AMTflag, TU.AMTIdx, TU.ROTIdx that minimizes the RD cost. The optimum value of the variable value of information Tinfo is determined. Then, the determination unit 178 supplies the optimal value of the variable value of the conversion information Tinfo to the control unit 101, and supplies the conversion coefficient corresponding to the optimal value to the quantization unit 113 as the conversion coefficient Coeff. After the process of step S158, the process returns to step S104 in FIG. 6 and proceeds to step S105.
図9は、図8のステップS153の最適両変換タイプ決定処理を説明するフローチャートである。
FIG. 9 is a flowchart for explaining the optimum both conversion type determination process in step S153 of FIG.
図9のステップS171において、選択部171は、TU.TSflagの候補となる値を0に決定し、演算部111から供給される予測残差Dを選択部172に供給する。ステップS172において、選択部172は、垂直方向および水平方向のTU.AMTflagの候補となる値を0に決定する。
9, in step S171, the selection unit 171 determines a candidate value of TU.TSflag as 0, and supplies the prediction residual D supplied from the calculation unit 111 to the selection unit 172. In step S172, the selection unit 172 determines 0 as a candidate value for the TU.AMTflag in the vertical and horizontal directions.
ステップS173において、選択部172は、モード情報pred_mode_flag、PU.IntraBCflag、イントラ予測モードを示す情報、およびTU.AMTflagとしての0に基づいて、図5で説明した方法で垂直方向の第1の直交変換行列を選択する。選択部171は、選択された垂直方向の第1の直交変換行列に対応する垂直DCT2変換部173-1、垂直DCT5変換部173-2、垂直DCT8変換部173-3、垂直DST1変換部173-4、または垂直DST7変換部173-5に予測残差Dを供給する。
In step S173, the selection unit 172 performs the first orthogonal transform in the vertical direction by the method described in FIG. 5 based on mode information pred_mode_flag, PU.IntraBCflag, information indicating the intra prediction mode, and 0 as TU.AMTflag. Select a matrix. The selection unit 171 includes a vertical DCT2 conversion unit 173-1, a vertical DCT5 conversion unit 173-2, a vertical DCT8 conversion unit 173-3, and a vertical DST1 conversion unit 173 corresponding to the selected first orthogonal transformation matrix in the vertical direction. 4 or the vertical residual DST7 is supplied to the vertical DST7 converter 173-5.
ステップS174において、垂直DCT2変換部173-1、垂直DCT5変換部173-2、垂直DCT8変換部173-3、垂直DST1変換部173-4、または垂直DST7変換部173-5は、予測残差Dに対して、ステップS173で選択された第1の直交変換行列を用いて垂直方向に第1の直交変換を行う。垂直DCT2変換部173-1、垂直DCT5変換部173-2、垂直DCT8変換部173-3、垂直DST1変換部173-4、または垂直DST7変換部173-5は、その結果得られる変換係数を選択部174に供給する。
In step S174, the vertical DCT2 conversion unit 173-1, the vertical DCT5 conversion unit 173-2, the vertical DCT8 conversion unit 173-3, the vertical DST1 conversion unit 173-4, or the vertical DST7 conversion unit 173-5 performs the prediction residual D. On the other hand, the first orthogonal transformation is performed in the vertical direction using the first orthogonal transformation matrix selected in step S173. Vertical DCT2 conversion unit 173-1, vertical DCT5 conversion unit 173-2, vertical DCT8 conversion unit 173-3, vertical DST1 conversion unit 173-4, or vertical DST7 conversion unit 173-5 selects a conversion coefficient obtained as a result To the unit 174.
ステップS175において、選択部174は、モード情報pred_mode_flag、PU.IntraBCflag、イントラ予測モードを示す情報、およびTU.AMTflagとしての0に基づいて、図5で説明した方法で水平方向の第1の直交変換行列を選択する。選択部171は、選択された水平方向の第1の直交変換行列に対応する水平DCT2変換部175-1、水平DCT5変換部175-2、水平DCT8変換部175-3、水平DST1変換部175-4、または水平DST7変換部175-5に変換係数を供給する。
In step S175, the selection unit 174 performs the first orthogonal transformation in the horizontal direction by the method described in FIG. 5 based on mode information pred_mode_flag, PU.IntraBCflag, information indicating the intra prediction mode, and 0 as TU.AMTflag. Select a matrix. The selection unit 171 includes a horizontal DCT2 conversion unit 175-1, a horizontal DCT5 conversion unit 175-2, a horizontal DCT8 conversion unit 175-3, and a horizontal DST1 conversion unit 175 corresponding to the selected horizontal first orthogonal transformation matrix. 4 or the horizontal DST7 conversion unit 175-5 is supplied with a conversion coefficient.
ステップS176において、水平DCT2変換部175-1、水平DCT5変換部175-2、水平DCT8変換部175-3、水平DST1変換部175-4、または水平DST7変換部175-5は、予測残差Dに対して、ステップS175で選択された第1の直交変換行列を用いて水平方向に第1の直交変換を行う。水平DCT2変換部175-1、水平DCT5変換部175-2、水平DCT8変換部175-3、水平DST1変換部175-4、または水平DST7変換部175-5は、その結果得られる変換係数を選択部176に供給する。
In step S176, the horizontal DCT2 conversion unit 175-1, horizontal DCT5 conversion unit 175-2, horizontal DCT8 conversion unit 175-3, horizontal DST1 conversion unit 175-4, or horizontal DST7 conversion unit 175-5 performs the prediction residual D. On the other hand, the first orthogonal transformation is performed in the horizontal direction using the first orthogonal transformation matrix selected in step S175. The horizontal DCT2 conversion unit 175-1, horizontal DCT5 conversion unit 175-2, horizontal DCT8 conversion unit 175-3, horizontal DST1 conversion unit 175-4, or horizontal DST7 conversion unit 175-5 selects a conversion coefficient obtained as a result. To the unit 176.
ステップS177において、選択部176は、予測情報Pinfoに基づいてイントラ予測処理が行われたかどうかを判定する。
In step S177, the selection unit 176 determines whether intra prediction processing has been performed based on the prediction information Pinfo.
具体的には、選択部176は、モード情報pred_mode_flagがイントラ予測処理系の処理を示し、SPS.IntraBCflagがイントラBC予測処理を有効にしないことを示すか、または、PU.IntraBCflagがイントラBC予測処理ではないことを示す場合、イントラ予測処理が行われたと判定する。一方、モード情報pred_mode_flagがイントラ予測処理系の処理を示し、SPS.IntraBCflagがイントラBC予測処理を有効にすることを示し、PU.IntraBCflagがイントラBC予測処理であることを示す場合、または、モード情報pred_mode_flagがインター予測処理を示す場合、選択部176は、イントラ予測処理が行われていないと判定する。
Specifically, the selection unit 176 indicates that the mode information pred_mode_flag indicates the processing of the intra prediction processing system and that SPS.IntraBCflag does not enable the intra BC prediction processing, or PU.IntraBCflag indicates the intra BC prediction processing. If it is not, it is determined that intra prediction processing has been performed. On the other hand, when the mode information pred_mode_flag indicates processing of the intra prediction processing system, SPS.IntraBCflag indicates that the intra BC prediction processing is enabled, and PU.IntraBCflag indicates that it is intra BC prediction processing, or mode information When pred_mode_flag indicates the inter prediction process, the selection unit 176 determines that the intra prediction process is not performed.
ステップS177でイントラ予測処理が行われたと判定された場合、処理はステップS178に進む。ステップS178において、選択部176は、TU.ROTIdxの候補となる値を0に決定する。
If it is determined in step S177 that the intra prediction process has been performed, the process proceeds to step S178. In step S178, the selection unit 176 determines 0 as a candidate value for TU.ROTIdx.
ステップS179において、選択部176はTU.ROTIdxの候補となる値が3であるかどうかを判定する。ステップS179でTU.ROTIdxの候補となる値が3ではないと判定された場合、選択部176は、TU.ROTIdxの候補となる値に対応するNSST0変換部177-1、NSST1変換部177-2、またはNSST2変換部177-3に変換係数を供給する。
In step S179, the selection unit 176 determines whether the candidate value for TU.ROTIdx is 3. If it is determined in step S179 that the TU.ROTIdx candidate value is not 3, the selection unit 176 selects the NSST0 conversion unit 177-1 and NSST1 conversion unit 177-2 corresponding to the TU.ROTIdx candidate value. Alternatively, the conversion coefficient is supplied to the NSST2 conversion unit 177-3.
そして、ステップS180において、NSST0変換部177-1、NSST1変換部177-2、またはNSST2変換部177-3は、変換係数に対して第2の直交変換を行い、その結果得られる変換係数を決定部178に供給する。
In step S180, NSST0 transform section 177-1, NSST1 transform section 177-2, or NSST2 transform section 177-3 performs a second orthogonal transform on the transform coefficient, and determines a transform coefficient obtained as a result. To the unit 178.
ステップS181において、選択部176は、TU.ROTIdxの候補となる値を1だけインクリメントし、選択部176は、TU.ROTIdxの候補となる値が3になるまで、ステップS179乃至S181の処理が繰り返される。
In step S181, the selection unit 176 increments the value that becomes a candidate for TU.ROTIdx by 1, and the selection unit 176 repeats the processing in steps S179 to S181 until the value that becomes a candidate for TU.ROTIdx becomes 3. It is.
一方、ステップS179でTU.ROTIdxの候補となる値が3であると判定された場合、選択部176は、変換係数を無変換部177-4に供給する。無変換部177-4は、変換係数に対して、無変換行列を第2の直交変換行列として用いて第2の直交変換を行う、即ち第2の直交変換を行わない。無変換部177-4は、その結果、選択部176から供給される変換係数をそのまま決定部178に供給し、処理をステップS183に進める。
On the other hand, if it is determined in step S179 that the candidate value for TU.ROTIdx is 3, the selection unit 176 supplies the conversion coefficient to the non-conversion unit 177-4. The non-transforming unit 177-4 performs the second orthogonal transform on the transform coefficient using the non-transformed matrix as the second orthogonal transform matrix, that is, does not perform the second orthogonal transform. As a result, the non-conversion unit 177-4 supplies the conversion coefficient supplied from the selection unit 176 as it is to the determination unit 178, and the process proceeds to step S183.
また、ステップS177でイントラ予測処理が行われないと判定された場合、処理はステップS182に進む。ステップS182において、選択部176は、TU.ROTIdxの候補となる値を3に決定し、変換係数を無変換部177-4に供給する。無変換部177-4は、変換係数に対して、無変換行列を第2の直交変換行列として用いて第2の直交変換を行う、即ち第2の直交変換を行わない。無変換部177-4は、その結果、選択部176から供給される変換係数をそのまま決定部178に供給し、処理をステップS183に進める。
If it is determined in step S177 that the intra prediction process is not performed, the process proceeds to step S182. In step S182, the selection unit 176 determines 3 as a TU.ROTIdx candidate value, and supplies the conversion coefficient to the non-conversion unit 177-4. The non-transforming unit 177-4 performs the second orthogonal transform on the transform coefficient using the non-transformed matrix as the second orthogonal transform matrix, that is, does not perform the second orthogonal transform. As a result, the non-conversion unit 177-4 supplies the conversion coefficient supplied from the selection unit 176 as it is to the determination unit 178, and the process proceeds to step S183.
ステップS183において、選択部171は、TU.TSflagの候補となる値が0であるかどうかを判定する。ステップS183でTU.TSflagの候補となる値が0であると判定された場合、処理はステップS184に進む。
In step S183, the selection unit 171 determines whether the candidate value of TU.TSflag is 0. If it is determined in step S183 that the candidate value for TU.TSflag is 0, the process proceeds to step S184.
ステップS184において、選択部172は、垂直方向および水平方向のTU.AMTflagの候補となる値が1であるかどうかを判定する。ステップS184で垂直方向および水平方向のTU.AMTflagの候補となる値が1ではないと判定された場合、処理はステップS185に進む。
In step S184, the selection unit 172 determines whether the candidate value for the TU.AMTflag in the vertical direction and the horizontal direction is 1. If it is determined in step S184 that the candidate value of TU.AMTflag in the vertical direction and the horizontal direction is not 1, the process proceeds to step S185.
ステップS185において、選択部172は、垂直方向および水平方向のTU.AMTflagの候補となる値を1に決定し、垂直方向のTU.AMTIdxと水平方向のTU.AMTIdxの候補となる値を0に決定する。そして、処理はステップS173の処理に戻り、以降の処理が繰り返される。なお、このとき、垂直方向の第1の直交変換行列の選択は、モード情報pred_mode_flag、PU.IntraBCflag、イントラ予測モードを示す情報、およびTU.AMTflagだけでなく、垂直方向のTU.AMTIdxにも基づいて行われる。水平方向の第1の直交変換行列の選択も同様である。
In step S185, the selection unit 172 determines the candidate value for the vertical and horizontal TU.AMTflag to be 1, and sets the candidate values for the vertical TU.AMTIdx and the horizontal TU.AMTIdx to 0. decide. And a process returns to the process of step S173 and the subsequent processes are repeated. At this time, the selection of the first orthogonal transformation matrix in the vertical direction is based not only on the mode information pred_mode_flag, PU.IntraBCflag, information indicating the intra prediction mode, and TU.AMTflag, but also on TU.AMTIdx in the vertical direction. Done. The same applies to the selection of the first orthogonal transformation matrix in the horizontal direction.
一方、ステップS184で垂直方向および水平方向のTU.AMTflagの候補となる値が1であると判定された場合、処理はステップS186に進む。ステップS186において、選択部172は、水平方向のTU.AMTIdxの候補となる値が1であるかどうかを判定する。
On the other hand, if it is determined in step S184 that the candidate value for the TU.AMTflag in the vertical and horizontal directions is 1, the process proceeds to step S186. In step S186, the selection unit 172 determines whether the value that is a candidate for the horizontal TU.AMTIdx is 1.
ステップS186で水平方向のTU.AMTIdxの候補となる値が1ではないと判定された場合、処理はステップS187に進む。ステップS187において、選択部172は、水平方向のTU.AMTIdxの候補となる値を1だけインクリメントして、処理をステップS173に戻す。そして、以降の処理が、ステップS185の処理後と同様に行われる。
If it is determined in step S186 that the horizontal TU.AMTIdx candidate value is not 1, the process proceeds to step S187. In step S187, the selection unit 172 increments the value that becomes a candidate for the horizontal TU.AMTIdx by 1, and returns the process to step S173. Then, the subsequent processing is performed in the same manner as after step S185.
一方、ステップS186で水平方向のTU.AMTIdxの候補となる値が1であると判定された場合、処理はステップS188に進む。ステップS188において、選択部172は、垂直方向のTU.AMTIdxの候補となる値が1であるかどうかを判定する。
On the other hand, if it is determined in step S186 that the candidate value of TU.AMTIdx in the horizontal direction is 1, the process proceeds to step S188. In step S188, the selection unit 172 determines whether the value that is a candidate for TU.AMTIdx in the vertical direction is 1.
ステップS188で垂直方向のTU.AMTIdxの候補となる値が1ではないと判定された場合、処理はステップS189に進む。ステップS189において、選択部172は、垂直方向のTU.AMTIdxの候補となる値を1だけインクリメントし、水平方向のTU.AMTIdxの候補となる値を0に決定して、処理をステップS173に戻す。そして、以降の処理が、ステップS185の処理後と同様に行われる。
If it is determined in step S188 that the vertical TU.AMTIdx candidate value is not 1, the process proceeds to step S189. In step S189, the selection unit 172 increments the vertical TU.AMTIdx candidate value by 1, determines the horizontal TU.AMTIdx candidate value to 0, and returns the process to step S173. . Then, the subsequent processing is performed in the same manner as after step S185.
一方、ステップS188で垂直方向のTU.AMTIdxの候補となる値が1であると判定された場合、ステップS190において、選択部171は、TU.TSflagの候補となる値が1であるかどうかを判定する。ステップS190でTU.TSflagの候補となる値が1ではないと判定された場合、処理はステップS191に進む。
On the other hand, when it is determined in step S188 that the candidate value for TU.AMTIdx in the vertical direction is 1, in step S190, the selection unit 171 determines whether the candidate value for TU.TSflag is 1. judge. If it is determined in step S190 that the TU.TSflag candidate value is not 1, the process proceeds to step S191.
ステップS191において、選択部171は、TU.TSflagの候補となる値を1に決定する。そして、選択部171は、予測残差Dを選択部176に供給し、処理をステップS177に戻す。なお、このとき、ステップS180の処理では、予測残差Dに対して第2の直交変換が行われる。
In step S191, the selection unit 171 determines 1 as a candidate value for TU.TSflag. Then, the selection unit 171 supplies the prediction residual D to the selection unit 176, and returns the process to step S177. At this time, in the process of step S180, the second orthogonal transformation is performed on the prediction residual D.
一方、ステップS183でTU.TSflagの候補となる値が0ではないと判定された場合、または、ステップS190でTU.TSflagの候補となる値が1であると判定された場合、処理は図8のステップS153に戻り、ステップS158に進む。
On the other hand, if it is determined in step S183 that the TU.TSflag candidate value is not 0, or if it is determined in step S190 that the TU.TSflag candidate value is 1, the process is as shown in FIG. Returning to step S153, the process proceeds to step S158.
なお、図9では、水平方向と垂直方向のTU.AMTflagの候補となる値が同一であるようにしたが、異なるようにしてもよい。この場合、水平方向と垂直方向のTU.AMTflagの候補となる値が別々に設定される。
In FIG. 9, the candidate values of TU.AMTflag in the horizontal direction and the vertical direction are the same, but they may be different. In this case, the candidate values for TU.AMTflag in the horizontal direction and the vertical direction are set separately.
また、図示は省略するが、最適第1直交変換タイプ決定処理は、ステップS177乃至S182の処理の代わりに、TU.ROTIdxの候補となる値として3が決定される処理が行われる点を除いて、図9の最適両変換タイプ決定処理と同様である。また、最適第2直交変換タイプ決定処理では、図9のテップS191以降の処理のみが行われる。
Although not shown, the optimum first orthogonal transform type determination process is performed except that the process of determining 3 as a candidate value of TU.ROTIdx is performed instead of the processes of steps S177 to S182. This is the same as the optimum both conversion type determination process of FIG. Further, in the optimum second orthogonal transformation type determination process, only the processes after step S191 in FIG. 9 are performed.
以上のように、画像符号化装置100は、イントラBC予測処理を行う場合、イントラBC予測処理用の第1の直交変換行列を用いて、予測残差Dに対して第1の直交変換を行う。従って、イントラBC予測処理に適した第1の直交変換行列を用いて予測残差Dに対して第1の直交変換を行うことができる。これにより、第1の直交変換の結果得られる変換係数のエネルギー集中度が向上する。その結果、符号化効率が向上する。また、量子化により失われる情報が低減するため、量子化誤差が低減し、その結果、復号画像の画質を向上させることができる。第2の直交変換についても、第1の直交変換と同様であり、同様の効果を奏することができる。
As described above, when performing the intra BC prediction process, the image coding apparatus 100 performs the first orthogonal transform on the prediction residual D using the first orthogonal transform matrix for the intra BC prediction process. . Therefore, the first orthogonal transformation can be performed on the prediction residual D using the first orthogonal transformation matrix suitable for the intra BC prediction process. Thereby, the energy concentration degree of the transform coefficient obtained as a result of the first orthogonal transform is improved. As a result, encoding efficiency is improved. In addition, since information lost due to quantization is reduced, quantization error is reduced, and as a result, the image quality of the decoded image can be improved. The second orthogonal transformation is the same as the first orthogonal transformation, and the same effect can be obtained.
(画像復号装置の構成例)
図10は、図2の画像符号化装置100により生成された符号化ストリームを復号する、本技術を適用した画像処理装置としての画像復号装置の第1実施の形態の構成例を示すブロック図である。図10の画像復号装置200は、画像符号化装置100により生成された符号化ストリームを、画像符号化装置100における符号化方法に対応する復号方法で復号する。例えば、画像復号装置200は、HEVCに提案された技術や、JVETにて提案された技術を実装している。 (Configuration example of image decoding device)
FIG. 10 is a block diagram illustrating a configuration example of a first embodiment of an image decoding device as an image processing device to which the present technology is applied, which decodes an encoded stream generated by theimage encoding device 100 in FIG. 2. is there. The image decoding device 200 in FIG. 10 decodes the encoded stream generated by the image encoding device 100 using a decoding method corresponding to the encoding method in the image encoding device 100. For example, the image decoding apparatus 200 is equipped with a technique proposed for HEVC and a technique proposed for JVET.
図10は、図2の画像符号化装置100により生成された符号化ストリームを復号する、本技術を適用した画像処理装置としての画像復号装置の第1実施の形態の構成例を示すブロック図である。図10の画像復号装置200は、画像符号化装置100により生成された符号化ストリームを、画像符号化装置100における符号化方法に対応する復号方法で復号する。例えば、画像復号装置200は、HEVCに提案された技術や、JVETにて提案された技術を実装している。 (Configuration example of image decoding device)
FIG. 10 is a block diagram illustrating a configuration example of a first embodiment of an image decoding device as an image processing device to which the present technology is applied, which decodes an encoded stream generated by the
なお、図10においては、処理部やデータの流れ等の主なものを示しており、図10に示されるものが全てとは限らない。つまり、画像復号装置200において、図10においてブロックとして示されていない処理部が存在したり、図10において矢印等として示されていない処理やデータの流れが存在したりしてもよい。
Note that FIG. 10 shows the main components such as the processing unit and the flow of data, and the ones shown in FIG. 10 are not all. That is, in the image decoding apparatus 200, there may be a processing unit that is not shown as a block in FIG. 10, or there may be a process or data flow that is not shown as an arrow or the like in FIG.
図10の画像復号装置200は、復号部211、逆量子化部212、逆変換部213、演算部214、フレームメモリ215、および予測部216を有する。画像符号化装置100は、画像符号化装置100により生成された符号化ストリームに対してCUごとに復号を行う。
10 includes a decoding unit 211, an inverse quantization unit 212, an inverse conversion unit 213, a calculation unit 214, a frame memory 215, and a prediction unit 216. The image encoding device 100 decodes the encoded stream generated by the image encoding device 100 for each CU.
具体的には、画像復号装置200の復号部211は、画像符号化装置100により生成された符号化ストリームを、符号化部114における符号化方法に対応する所定の復号方法で復号する。例えば、復号部211は、シンタックステーブルの定義に沿って、符号化ストリームのビット列から、符号化パラメータ(ヘッダ情報Hinfo、予測情報Pinfo、変換情報Tinfo等)と量子化変換係数レベルlevelを復号する。復号部211は、符号化パラメータに含まれるsplit flagに基づいてLCUを分割し、各量子化変換係数レベルlevelに対応するCUを順に復号対象のCU(PU,TU)に設定する。
Specifically, the decoding unit 211 of the image decoding device 200 decodes the encoded stream generated by the image encoding device 100 by a predetermined decoding method corresponding to the encoding method in the encoding unit 114. For example, the decoding unit 211 decodes the encoding parameters (header information Hinfo, prediction information Pinfo, conversion information Tinfo, etc.) and the quantized transform coefficient level level from the bit stream of the encoded stream according to the definition of the syntax table. . The decoding unit 211 divides the LCU based on the split flag included in the encoding parameter, and sequentially sets the CUs corresponding to the respective quantized transform coefficient levels level as decoding target CUs (PU, TU).
復号部211は、符号化パラメータを各ブロックへ供給する。例えば、復号部211は、予測情報Pinfoを予測部216に供給し、変換情報Tinfoと予測情報Pinfoを逆量子化部212と逆変換部213に供給し、ヘッダ情報Hinfoを各ブロックに供給する。また、復号部211は、量子化変換係数レベルlevelを逆量子化部212に供給する。
The decoding unit 211 supplies the encoding parameters to each block. For example, the decoding unit 211 supplies the prediction information Pinfo to the prediction unit 216, supplies the transform information Tinfo and the prediction information Pinfo to the inverse quantization unit 212 and the inverse transform unit 213, and supplies the header information Hinfo to each block. In addition, the decoding unit 211 supplies the quantized transform coefficient level level to the inverse quantization unit 212.
逆量子化部212は、復号部211から供給される変換情報Tinfoに基づいて、復号部211から供給される量子化変換係数レベルlevelの値をスケーリング(逆量子化)し、変換係数Coeff_IQを導出する。この逆量子化は、画像符号化装置100の量子化部113(図2)により行われる量子化の逆処理である。なお、逆量子化部115(図2)は、この逆量子化部212と同様の逆量子化を行う。逆量子化部212は、得られた変換係数Coeff_IQを逆変換部213に供給する。
Based on the transform information Tinfo supplied from the decoding unit 211, the inverse quantization unit 212 scales (inversely quantizes) the value of the quantized transform coefficient level level supplied from the decoding unit 211 to derive the transform coefficient Coeff_IQ. To do. This inverse quantization is an inverse process of quantization performed by the quantization unit 113 (FIG. 2) of the image encoding device 100. Note that the inverse quantization unit 115 (FIG. 2) performs the same inverse quantization as the inverse quantization unit 212. The inverse quantization unit 212 supplies the obtained transform coefficient Coeff_IQ to the inverse transform unit 213.
逆変換部213は、復号部211から供給される符号化パラメータに基づいて、逆量子化部212より供給される変換係数Coeff_IQに対して逆変換処理を行い、予測残差D’を導出する。この逆変換処理は、画像符号化装置100の変換部112(図2)により行われる変換処理の逆処理である。なお、逆変換部116は、この逆変換部213と同様の逆変換を行う。逆変換部213は、得られた予測残差D’を演算部214に供給する。
The inverse transform unit 213 performs an inverse transform process on the transform coefficient Coeff_IQ supplied from the inverse quantization unit 212 based on the encoding parameter supplied from the decoding unit 211, and derives a prediction residual D '. This inverse conversion process is an inverse process of the conversion process performed by the conversion unit 112 (FIG. 2) of the image encoding device 100. Note that the inverse transform unit 116 performs the same inverse transform as the inverse transform unit 213. The inverse transform unit 213 supplies the obtained prediction residual D ′ to the calculation unit 214.
演算部214は、逆変換部213から供給される予測残差D’とその予測残差D’に対応する予測画像Pとを加算し、局所的な復号画像Recを導出する。演算部214は、得られた局所的な復号画像Recを用いてピクチャ単位毎の復号画像を再構築し、得られた復号画像を画像復号装置200の外部に出力する。また、演算部214は、その局所的な復号画像Recをフレームメモリ215にも供給する。
The calculation unit 214 adds the prediction residual D ′ supplied from the inverse conversion unit 213 and the prediction image P corresponding to the prediction residual D ′ to derive a local decoded image Rec. The calculation unit 214 reconstructs a decoded image for each picture unit using the obtained local decoded image Rec, and outputs the obtained decoded image to the outside of the image decoding apparatus 200. In addition, the calculation unit 214 supplies the local decoded image Rec to the frame memory 215 as well.
フレームメモリ215は、演算部214より供給される局所的な復号画像Recを用いてピクチャ単位毎の復号画像を再構築し、フレームメモリ215内のバッファに格納する。フレームメモリ215は、予測部216により指定される復号画像を参照画像としてバッファより読み出し、予測部216に供給する。また、フレームメモリ215は、その復号画像の生成に係るヘッダ情報Hinfo、予測情報Pinfo、変換情報Tinfoなどをフレームメモリ215内のバッファに格納するようにしても良い。
The frame memory 215 reconstructs a decoded image for each picture unit using the local decoded image Rec supplied from the calculation unit 214 and stores it in a buffer in the frame memory 215. The frame memory 215 reads the decoded image specified by the prediction unit 216 from the buffer as a reference image, and supplies the read image to the prediction unit 216. Further, the frame memory 215 may store header information Hinfo, prediction information Pinfo, conversion information Tinfo, and the like related to generation of the decoded image in a buffer in the frame memory 215.
予測部216は、復号部211から供給される予測情報Pinfo等に基づいて、フレームメモリ215に格納された復号画像を参照画像として取得し、その参照画像を用いてイントラBC予測処理、所定のイントラ予測モードのイントラ予測処理、またはインター予測処理を行う。予測部216は、その結果生成される予測画像Pを演算部214に供給する。
The prediction unit 216 acquires the decoded image stored in the frame memory 215 as a reference image based on the prediction information Pinfo supplied from the decoding unit 211, and uses the reference image to perform an intra BC prediction process and a predetermined intra Intra prediction processing or inter prediction processing in the prediction mode is performed. The prediction unit 216 supplies the predicted image P generated as a result to the calculation unit 214.
(逆変換部の構成例)
図11は、図10の逆変換部213の構成例を示すブロック図である。 (Configuration example of inverse conversion unit)
FIG. 11 is a block diagram illustrating a configuration example of theinverse transform unit 213 in FIG.
図11は、図10の逆変換部213の構成例を示すブロック図である。 (Configuration example of inverse conversion unit)
FIG. 11 is a block diagram illustrating a configuration example of the
図11の逆変換部213は、選択部231、NSST0逆変換部232-1、NSST1逆変換部232-2、NSST2逆変換部232-3、無変換部232-4、選択部233、選択部234、水平DCT2逆変換部235-1、水平DCT5逆変換部235-2、水平DCT8逆変換部235-3、水平DST1逆変換部235-4、水平DST7逆変換部235-5、選択部236、垂直DCT2逆変換部237-1、垂直DCT5逆変換部237-2、垂直DCT8逆変換部237-3、垂直DST1逆変換部237-4、および垂直DST7逆変換部237-5により構成される。
11 includes a selection unit 231, an NSST0 inverse conversion unit 232-1, an NSST1 inverse conversion unit 232-2, an NSST2 inverse conversion unit 232-3, a non-conversion unit 232-4, a selection unit 233, and a selection unit. 234, horizontal DCT2 inverse transform unit 235-1, horizontal DCT5 inverse transform unit 235-2, horizontal DCT8 inverse transform unit 235-3, horizontal DST1 inverse transform unit 235-4, horizontal DST7 inverse transform unit 235-5, selection unit 236 , A vertical DCT2 inverse transform unit 237-1, a vertical DCT5 inverse transform unit 237-2, a vertical DCT8 inverse transform unit 237-3, a vertical DST1 inverse transform unit 237-4, and a vertical DST7 inverse transform unit 237-5. .
逆変換部213の選択部231には、図10の復号部211からヘッダ情報Hinfo、予測情報Pinfo、変換情報Tinfo等が供給される。また、選択部231には、逆量子化部212から変換係数Coeff_IQが供給される。
The selection unit 231 of the inverse conversion unit 213 is supplied with header information Hinfo, prediction information Pinfo, conversion information Tinfo, and the like from the decoding unit 211 of FIG. Further, the transform coefficient Coeff_IQ is supplied from the inverse quantization unit 212 to the selection unit 231.
選択部231は、ヘッダ情報HinfoのうちのSPS.NSSTflag、変換情報TinfoのうちのTU.ROTIdx等に基づいて、変換係数Coeff_IQを出力する。
The selection unit 231 outputs the conversion coefficient Coeff_IQ based on SPS.NSSTflag in the header information Hinfo, TU.ROTIdx in the conversion information Tinfo, and the like.
具体的には、SPS.NSSTflagが、第2の直交変換を有効にしないことを示す0である場合、選択部231は、変換係数Coeff_IQを無変換部232-4に出力する。
Specifically, when SPS.NSSTflag is 0 indicating that the second orthogonal transform is not enabled, the selection unit 231 outputs the transform coefficient Coeff_IQ to the non-transform unit 232-4.
一方、SPS.NSSTflagが、第2の直交変換を有効にすることを示す1である場合、選択部231は、TU.ROTIdxを第2の直交変換行列のインデックスとして選択する。選択部231は、選択されたインデックスに対応するNSST0逆変換部232-1、NSST1逆変換部232-2、NSST1逆変換部232-2、または無変換部232-4に変換係数Coeff_IQを出力する。具体的には、TU.ROTIdxの候補となる値が0,1,2,3である場合、選択部231は、変換係数Coeff_IQを、NSST0逆変換部232-1、NSST1逆変換部232-2、NSST2逆変換部232-3、無変換部232-4に供給する。
On the other hand, when SPS.NSSTflag is 1 indicating that the second orthogonal transformation is enabled, the selection unit 231 selects TU.ROTIdx as an index of the second orthogonal transformation matrix. Selection section 231 outputs transform coefficient Coeff_IQ to NSST0 inverse transform section 232-1, NSST1 inverse transform section 232-2, NSST1 inverse transform section 232-2, or no transform section 232-4 corresponding to the selected index. . Specifically, when the candidate values for TU.ROTIdx are 0, 1, 2, 3, the selection unit 231 converts the transform coefficient Coeff_IQ into the NSST0 inverse transform unit 232-1 and NSST1 inverse transform unit 232-2. , NSST2 inverse conversion unit 232-3 and non-conversion unit 232-4.
NSST0逆変換部232-1、NSST1逆変換部232-2、NSST2逆変換部232-3、および無変換部232-4(第2の逆直交変換部)は、所定のインデックスにより特定される第2の直交変換行列に対応する第2の逆直交変換行列を用いて、変換係数Coeff_IQに対して、4×4画素ごとに第2の逆直交変換を行う。NSST0逆変換部232-1、NSST1逆変換部232-2、およびNSST2逆変換部232-3で用いられる第2の逆直交変換行列は、例えば、イントラ予測モードに応じて異なる。
NSST0 inverse transform unit 232-1, NSST1 inverse transform unit 232-2, NSST2 inverse transform unit 232-3, and non-transform unit 232-4 (second inverse orthogonal transform unit) are identified by a predetermined index. A second inverse orthogonal transform is performed for every 4 × 4 pixels on the transform coefficient Coeff_IQ using a second inverse orthogonal transform matrix corresponding to 2 orthogonal transform matrices. The second inverse orthogonal transform matrix used in the NSST0 inverse transform unit 232-1, NSST1 inverse transform unit 232-2, and NSST2 inverse transform unit 232-3 differs depending on, for example, the intra prediction mode.
NSST0逆変換部232-1、NSST1逆変換部232-2、NSST2逆変換部232-3、および無変換部232-4は、第2の逆直交変換の結果得られる変換係数を選択部233に供給する。なお、無変換部232-4で用いられる第2の逆直交変換行列は無変換行列であるため、無変換部232-4から選択部233に供給される変換係数は、選択部231から供給される変換係数Coeff_IQそのものである。
NSST0 inverse transform unit 232-1, NSST1 inverse transform unit 232-2, NSST2 inverse transform unit 232-3, and non-transform unit 232-4 provide transform coefficients obtained as a result of the second inverse orthogonal transform to selection unit 233. Supply. Note that since the second inverse orthogonal transformation matrix used in the non-conversion unit 232-4 is a non-conversion matrix, the transform coefficient supplied from the non-conversion unit 232-4 to the selection unit 233 is supplied from the selection unit 231. The conversion coefficient Coeff_IQ itself.
選択部233は、復号部211から供給されるヘッダ情報HinfoのうちのSPS.AMTflag、変換情報TinfoのうちのTU.TSflag等に基づいて、第1の逆直交変換を行うかどうかを判定し、第2の逆直交変換の結果得られる変換係数を選択部234または演算部214に供給する。
The selection unit 233 determines whether to perform the first inverse orthogonal transformation based on SPS.AMTflag in the header information Hinfo supplied from the decoding unit 211, TU.TSflag in the transformation information Tinfo, and the like. A transform coefficient obtained as a result of the second inverse orthogonal transform is supplied to the selection unit 234 or the calculation unit 214.
具体的には、SPS.AMTflagが、第1の直交変換を有効にしないことを示す0である場合、または、TU.TSflagが、第1の直交変換をスキップすることを示す1である場合、選択部233は、第1の逆直交変換を行わないと判定する。そして、選択部233は、第2の逆直交変換の結果得られる変換係数を予測残差D´としてそのまま演算部214に供給する。
Specifically, when SPS.AMTflag is 0 indicating that the first orthogonal transform is not enabled, or when TU.TSflag is 1 indicating that the first orthogonal transform is skipped, The selection unit 233 determines that the first inverse orthogonal transform is not performed. Then, the selection unit 233 supplies the transform coefficient obtained as a result of the second inverse orthogonal transform to the calculation unit 214 as it is as the prediction residual D ′.
一方、SPS.AMTflagが、第1の直交変換を有効にすることを示す1であり、かつ、TU.TSflagが、第1の直交変換をスキップしないことを示す0である場合、選択部233は、第1の逆直交変換を行うと判定する。そして、選択部233は、第2の逆直交変換の結果得られる変換係数を選択部234に供給する。
On the other hand, when SPS.AMTflag is 1 indicating that the first orthogonal transformation is enabled and TU.TSflag is 0 indicating that the first orthogonal transformation is not skipped, the selection unit 233 , It is determined to perform the first inverse orthogonal transform. Then, the selection unit 233 supplies the transform coefficient obtained as a result of the second inverse orthogonal transform to the selection unit 234.
選択部234は、復号部211から供給される変換情報Tinfoのうちの水平方向のTU.AMTflagとTU.AMTIdx、予測情報Pinfo等に基づいて、水平方向の第1の直交変換行列を選択する。そして、選択部234は、選択部233から供給される変換係数を、選択された水平方向の第1の直交変換行列に対応する水平DCT2逆変換部235-1、水平DCT5逆変換部235-2、水平DCT8逆変換部235-3、水平DST1逆変換部235-4、または水平DST7逆変換部235-5に供給する。
The selection unit 234 selects the first orthogonal orthogonal transformation matrix based on the horizontal direction TU.AMTflag and TU.AMTIdx, the prediction information Pinfo, and the like in the conversion information Tinfo supplied from the decoding unit 211. Then, the selection unit 234 converts the transformation coefficient supplied from the selection unit 233 into a horizontal DCT2 inverse transformation unit 235-1 and a horizontal DCT5 inverse transformation unit 235-2 corresponding to the selected first horizontal orthogonal orthogonal transformation matrix. , To the horizontal DCT8 inverse transform unit 235-3, the horizontal DST1 inverse transform unit 235-4, or the horizontal DST7 inverse transform unit 235-5.
水平DCT2逆変換部235-1、水平DCT5逆変換部235-2、水平DCT8逆変換部235-3、水平DST1逆変換部235-4、および水平DST7逆変換部235-5(第1の逆直交変換部)は、それぞれ、所定の第1の直交変換行列に対応する第1の逆直交変換行列を用いて、選択部234から供給される変換係数に対して水平方向に第1の逆直交変換を行う。水平DCT2逆変換部235-1、水平DCT5逆変換部235-2、水平DCT8逆変換部235-3、水平DST1逆変換部235-4、および水平DST7逆変換部235-5で用いられる第1の逆直交変換行列は、それぞれ、DCT2,DCT5,DCT8,DST1,DST7に対応する行列である。
Horizontal DCT2 inverse transform unit 235-1, horizontal DCT5 inverse transform unit 235-2, horizontal DCT8 inverse transform unit 235-3, horizontal DST1 inverse transform unit 235-4, and horizontal DST7 inverse transform unit 235-5 (first inverse (Orthogonal transform unit) uses the first inverse orthogonal transform matrix corresponding to the predetermined first orthogonal transform matrix, respectively, and the first inverse orthogonal in the horizontal direction with respect to the transform coefficient supplied from the selection unit 234 Perform conversion. First used in horizontal DCT2 inverse transform unit 235-1, horizontal DCT5 inverse transform unit 235-2, horizontal DCT8 inverse transform unit 235-3, horizontal DST1 inverse transform unit 235-4, and horizontal DST7 inverse transform unit 235-5 The inverse orthogonal transformation matrices are matrices corresponding to DCT2, DCT5, DCT8, DST1, and DST7, respectively.
水平DCT2逆変換部235-1、水平DCT5逆変換部235-2、水平DCT8逆変換部235-3、水平DST1逆変換部235-4、および水平DST7逆変換部235-5は、選択部234から供給される変換係数に対して水平方向に第1の逆直交変換を行った結果得られる変換係数を選択部236に供給する。
The horizontal DCT2 inverse transform unit 235-1, the horizontal DCT5 inverse transform unit 235-2, the horizontal DCT8 inverse transform unit 235-3, the horizontal DST1 inverse transform unit 235-4, and the horizontal DST7 inverse transform unit 235-5 are selected by the selection unit 234. The transform coefficient obtained as a result of performing the first inverse orthogonal transform in the horizontal direction on the transform coefficient supplied from is supplied to the selection unit 236.
選択部236は、復号部211から供給される変換情報Tinfoのうちの垂直方向のTU.AMTflagとTU.AMTIdx、予測情報Pinfo等に基づいて、垂直方向の第1の直交変換行列を選択する。そして、選択部236は、供給される変換係数を、選択された垂直方向の第1の直交変換行列に対応する垂直DCT2逆変換部237-1、垂直DCT5逆変換部237-2、垂直DCT8逆変換部237-3、垂直DST1逆変換部237-4、または垂直DST7逆変換部237-5に供給する。
The selection unit 236 selects the first orthogonal orthogonal transformation matrix based on the vertical direction TU.AMTflag and TU.AMTIdx, the prediction information Pinfo, and the like in the conversion information Tinfo supplied from the decoding unit 211. The selection unit 236 then converts the supplied transform coefficients into a vertical DCT2 inverse transform unit 237-1, a vertical DCT5 inverse transform unit 237-2, and a vertical DCT8 inverse corresponding to the selected first orthogonal transform matrix in the vertical direction. The data is supplied to the conversion unit 237-3, the vertical DST1 inverse conversion unit 237-4, or the vertical DST7 inverse conversion unit 237-5.
垂直DCT2逆変換部237-1、垂直DCT5逆変換部237-2、垂直DCT8逆変換部237-3、垂直DST1逆変換部237-4、および垂直DST7逆変換部237-5(第1の逆直交変換部)は、それぞれ、所定の第1の直交変換行列に対応する第1の逆直交変換行列を用いて、供給される変換係数に対して垂直方向に第1の逆直交変換を行う。垂直DCT2逆変換部237-1、垂直DCT5逆変換部237-2、垂直DCT8逆変換部237-3、垂直DST1逆変換部237-4、および垂直DST7逆変換部237-5で用いられる第1の逆直交変換行列は、それぞれ、DCT2,DCT5,DCT8,DST1,DST7に対応する行列である。
Vertical DCT2 inverse transform unit 237-1, vertical DCT5 inverse transform unit 237-2, vertical DCT8 inverse transform unit 237-3, vertical DST1 inverse transform unit 237-4, and vertical DST7 inverse transform unit 237-5 (first inverse (Orthogonal transform units) each perform a first inverse orthogonal transform on the supplied transform coefficients in the vertical direction by using a first inverse orthogonal transform matrix corresponding to a predetermined first orthogonal transform matrix. First used in the vertical DCT2 inverse transform unit 237-1, the vertical DCT5 inverse transform unit 237-2, the vertical DCT8 inverse transform unit 237-3, the vertical DST1 inverse transform unit 237-4, and the vertical DST7 inverse transform unit 237-5 The inverse orthogonal transformation matrices are matrices corresponding to DCT2, DCT5, DCT8, DST1, and DST7, respectively.
垂直DCT2逆変換部237-1、垂直DCT5逆変換部237-2、垂直DCT8逆変換部237-3、垂直DST1逆変換部237-4、および垂直DST7逆変換部237-5は、供給される変換係数に対して垂直方向に第1の逆直交変換を行った結果得られる予測残差D´を演算部214に供給する。
The vertical DCT2 inverse transform unit 237-1, the vertical DCT5 inverse transform unit 237-2, the vertical DCT8 inverse transform unit 237-3, the vertical DST1 inverse transform unit 237-4, and the vertical DST7 inverse transform unit 237-5 are supplied. The prediction residual D ′ obtained as a result of performing the first inverse orthogonal transform in the vertical direction on the transform coefficient is supplied to the calculation unit 214.
以上のように、逆変換部213は、行われる予測処理用の第2の直交変換行列の候補のうちの最適値を特定するインデックスであるTU.ROTIdxに基づいて、その最適値に対応する第2の逆直交変換行列を用いて、変換係数に対して第2の逆直交変換を行う。従って、イントラ予測処理が行われる場合、逆変換部213は、イントラBC予測処理用の第2の直交変換行列の候補のうちの最適値に対応する第2の逆直交変換行列、即ちイントラBC予測処理用の第2の逆直交変換行列を用いて、変換係数に対して第2の逆直交変換を行う。
As described above, the inverse transform unit 213 corresponds to the optimal value based on TU.ROTIdx, which is an index that identifies the optimal value among the candidates for the second orthogonal transform matrix for prediction processing to be performed. A second inverse orthogonal transform is performed on the transform coefficient using an inverse orthogonal transform matrix of 2. Therefore, when the intra prediction process is performed, the inverse transform unit 213 performs the second inverse orthogonal transform matrix corresponding to the optimum value among the candidates for the second orthogonal transform matrix for the intra BC prediction process, that is, the intra BC prediction. A second inverse orthogonal transform is performed on the transform coefficient using the second inverse orthogonal transform matrix for processing.
また、逆変換部213は、行われる予測処理用の水平方向の第1の直交変換行列の候補のうちの最適値を特定するインデックスである水平方向のTU.AMTIdxに基づいて、その最適値に対応する第1の逆直交変換行列を用いて、変換係数に対して水平方向に第1の直交変換を行う。従って、イントラ予測処理が行われる場合、逆変換部213は、イントラBC予測処理用の第1の直交変換行列の候補のうちの最適値に対応する第1の逆直交変換行列、即ちイントラBC予測処理用の第1の逆直交変換行列を用いて、変換係数に対して第1の逆直交変換を行う。垂直方向の第1の逆直交変換についても同様である。
Further, the inverse transform unit 213 sets the optimal value based on the horizontal TU.AMTIdx that is an index for identifying the optimal value of the candidates for the first orthogonal transform matrix in the horizontal direction for prediction processing to be performed. A first orthogonal transform is performed in the horizontal direction on the transform coefficient using the corresponding first inverse orthogonal transform matrix. Accordingly, when the intra prediction process is performed, the inverse transform unit 213 performs the first inverse orthogonal transform matrix corresponding to the optimum value among the candidates for the first orthogonal transform matrix for the intra BC prediction process, that is, the intra BC prediction. A first inverse orthogonal transform is performed on the transform coefficient using the first inverse orthogonal transform matrix for processing. The same applies to the first inverse orthogonal transform in the vertical direction.
(画像復号装置の処理の説明)
図12は、図10の画像復号装置200の画像復号処理を説明するフローチャートである。 (Description of processing of image decoding apparatus)
FIG. 12 is a flowchart for explaining the image decoding process of theimage decoding apparatus 200 of FIG.
図12は、図10の画像復号装置200の画像復号処理を説明するフローチャートである。 (Description of processing of image decoding apparatus)
FIG. 12 is a flowchart for explaining the image decoding process of the
ステップS200において、復号部211は、画像復号装置200に供給される符号化ストリームを復号し、符号化パラメータと量子化変換係数レベルlevelを得る。復号部211は、符号化パラメータを各ブロックへ供給する。また、復号部211は、量子化変換係数レベルlevelを逆量子化部212に供給する。
In step S200, the decoding unit 211 decodes the encoded stream supplied to the image decoding apparatus 200, and obtains an encoding parameter and a quantized transform coefficient level level. The decoding unit 211 supplies the encoding parameter to each block. In addition, the decoding unit 211 supplies the quantized transform coefficient level level to the inverse quantization unit 212.
ステップS201において、復号部211は、符号化パラメータに含まれるsplit flagに基づいてLCUを分割し、各量子化変換係数レベルlevelに対応するCUを復号対象のCU(PU,TU)に設定する。後述するステップS202乃至S211の処理は、復号対象のCU(PU,TU)ごとに行われる。
In step S201, the decoding unit 211 divides the LCU based on the split flag included in the encoding parameter, and sets a CU corresponding to each quantized transform coefficient level level to a decoding target CU (PU, TU). Steps S202 to S211 described later are performed for each CU (PU, TU) to be decoded.
ステップS202において、逆量子化部212は、ステップS201の処理により得られた量子化変換係数レベルlevelを逆量子化して変換係数Coeff_IQを導出する。この逆量子化は、画像符号化処理のステップS105(図6)において行われる量子化の逆処理であり、画像符号化処理のステップS106(図6)において行われる逆量子化と同様の処理である。
In step S202, the inverse quantization unit 212 derives a transform coefficient Coeff_IQ by inversely quantizing the quantized transform coefficient level level obtained by the process in step S201. This inverse quantization is the inverse process of the quantization performed in step S105 (FIG. 6) of the image encoding process, and is the same process as the inverse quantization performed in step S106 (FIG. 6) of the image encoding process. is there.
ステップS203において、逆変換部213は、ステップS202の処理により得られた変換係数Coeff_IQに対して逆変換処理を行い、予測残差D'を導出する。この逆変換処理は、画像符号化処理のステップS104(図6)において行われる変換処理の逆処理であり、画像符号化処理のステップS107(図6)において行われる逆変換と同様の処理である。この逆変換処理の詳細は、後述する図14を参照して説明する。
In step S203, the inverse transform unit 213 performs an inverse transform process on the transform coefficient Coeff_IQ obtained by the process in step S202, and derives a prediction residual D ′. This inverse conversion process is an inverse process of the conversion process performed in step S104 (FIG. 6) of the image encoding process, and is the same process as the inverse conversion performed in step S107 (FIG. 6) of the image encoding process. . Details of the inverse conversion process will be described with reference to FIG.
ステップS204において、予測部216は、復号部211から供給される符号化パラメータに基づいて、復号対象のPUを含むスライスはIスライスであるかどうかを判定する。ステップS204でIスライスではないと判定された場合、即ち、復号対象のPUを含むスライスがPスライスまたはBスライスである場合、処理はステップS205に進む。
In step S204, the prediction unit 216 determines whether the slice including the PU to be decoded is an I slice based on the encoding parameter supplied from the decoding unit 211. If it is determined in step S204 that the slice is not an I slice, that is, if the slice including the decoding target PU is a P slice or a B slice, the process proceeds to step S205.
ステップS205において、予測部216は、符号化パラメータからモード情報pred_mode_flagを抽出する。ステップS206において、予測部216は、モード情報pred_mode_flagがインター予測処理を示すかどうかを判定する。
In step S205, the prediction unit 216 extracts mode information pred_mode_flag from the encoding parameter. In step S206, the prediction unit 216 determines whether or not the mode information pred_mode_flag indicates inter prediction processing.
ステップS206でモード情報pred_mode_flagがインター予測処理を示すと判定された場合、処理はステップS207に進む。ステップS207において、予測部216は、予測情報Pinfoに基づいてフレームメモリ215から参照画像を読み出し、その参照画像と予測情報Pinfoのうちの動きベクトルとを用いてインター予測処理を行う。そして、予測部216は、その結果生成される予測画像Pを演算部214に供給し、処理をステップS211に進める。
If it is determined in step S206 that the mode information pred_mode_flag indicates inter prediction processing, the processing proceeds to step S207. In step S207, the prediction unit 216 reads a reference image from the frame memory 215 based on the prediction information Pinfo, and performs inter prediction processing using the reference image and a motion vector in the prediction information Pinfo. Then, the prediction unit 216 supplies the predicted image P generated as a result to the calculation unit 214, and the process proceeds to step S211.
一方、ステップS204でIスライスであると判定された場合、または、ステップS206でモード情報pred_mode_flagがインター予測処理を示さないと判定された場合、処理はステップS208に進む。
On the other hand, if it is determined in step S204 that the slice is an I slice, or if it is determined in step S206 that the mode information pred_mode_flag does not indicate inter prediction processing, the processing proceeds to step S208.
ステップS208において、予測部216は、予測情報PinfoのうちのPU.IntraBCflagが、イントラBC予測処理であることを示す1であるかどうかを判定する。ステップS208でPU.IntraBCflagが1であると判定された場合、処理はステップS209に進む。
In step S208, the prediction unit 216 determines whether or not PU.IntraBCflag in the prediction information Pinfo is 1 indicating that it is an intra BC prediction process. If it is determined in step S208 that PU.IntraBCflag is 1, the process proceeds to step S209.
ステップS209において、予測部216は、復号対象のPUを含む局所的に復号された復号画像をフレームメモリ215から参照画像として読み出し、その参照画像と予測情報Pinfoのうちの動きベクトルとを用いてイントラBC予測処理を行う。そして、予測部216は、その結果生成される予測画像Pを演算部214に供給し、処理をステップS211に進める。
In step S209, the prediction unit 216 reads a locally decoded decoded image including the decoding target PU as a reference image from the frame memory 215, and uses the reference image and the motion vector in the prediction information Pinfo to perform intra. BC prediction processing is performed. Then, the prediction unit 216 supplies the predicted image P generated as a result to the calculation unit 214, and the process proceeds to step S211.
ステップS208でPU.IntraBCflagが1ではないと判定された場合、処理はステップS210に進む。ステップS210において、予測部216は、復号対象のPUを含む局所的に復号された復号画像をフレームメモリ215から参照画像として読み出し、その参照画像を用いて、予測情報Pinfoに含まれるイントラ予測モードのイントラ予測処理を行う。そして、予測部216は、その結果生成される予測画像Pを演算部214に供給し、処理をステップS211に進める。
If it is determined in step S208 that PU.IntraBCflag is not 1, the process proceeds to step S210. In step S210, the prediction unit 216 reads a locally decoded decoded image including the decoding target PU as a reference image from the frame memory 215, and uses the reference image to determine the intra prediction mode included in the prediction information Pinfo. Perform intra prediction processing. Then, the prediction unit 216 supplies the predicted image P generated as a result to the calculation unit 214, and the process proceeds to step S211.
ステップS211において、演算部214は、逆変換部213から供給される予測残差D’を、予測部216から供給される予測画像Pと加算し、局所的な復号画像Recを導出する。演算部214は、得られた局所的な復号画像Recを用いてピクチャ単位毎の復号画像を再構築し、得られた復号画像を画像復号装置200の外部に出力する。また、演算部214は、その局所的な復号画像Recをフレームメモリ215にも供給する。
In step S211, the calculation unit 214 adds the prediction residual D ′ supplied from the inverse conversion unit 213 and the prediction image P supplied from the prediction unit 216 to derive a local decoded image Rec. The calculation unit 214 reconstructs a decoded image for each picture unit using the obtained local decoded image Rec, and outputs the obtained decoded image to the outside of the image decoding apparatus 200. In addition, the calculation unit 214 supplies the local decoded image Rec to the frame memory 215 as well.
ステップS212において、フレームメモリ215は、演算部214より供給される局所的な復号画像Recを用いてピクチャ単位毎の復号画像を再構築し、フレームメモリ215内のバッファに記憶する。そして、処理は終了する。
In step S212, the frame memory 215 reconstructs a decoded image for each picture unit using the local decoded image Rec supplied from the calculation unit 214, and stores it in a buffer in the frame memory 215. Then, the process ends.
図13は、図12のステップS200の処理のうちの、SPS.IntraBCflag,SPS.AMTflag、およびSPS.NSSTflagを復号するフラグ復号処理を説明するフローチャートである。
FIG. 13 is a flowchart illustrating a flag decoding process for decoding SPS.IntraBCflag, SPS.AMTflag, and SPS.NSSTflag in the process of step S200 of FIG.
図13のステップS231において、復号部211は、符号化ストリームに含まれる符号化パラメータのうちのSPS.IntraBCflagを復号する。ステップS232において、復号部211は、符号化パラメータのうちのSPS.AMTflagを復号する。ステップS233において、復号部211は、符号化パラメータのうちのSPS.NSSTflagを復号する。
In step S231 in FIG. 13, the decoding unit 211 decodes SPS.IntraBCflag among the encoding parameters included in the encoded stream. In step S232, the decoding unit 211 decodes SPS.AMTflag among the encoding parameters. In step S233, the decoding unit 211 decodes SPS.NSSTflag among the encoding parameters.
図14は、図12のステップS203の逆変換処理を説明するフローチャートである。
FIG. 14 is a flowchart for explaining the inverse conversion process in step S203 of FIG.
図14のステップS251において、逆変換部213は、復号部211から供給される予測情報Pinfoのうちのpred_mode_flagがイントラ予測処理系の処理を示すかどうかを判定する。ステップS251でpred_mode_flagがイントラ予測処理系の処理を示すと判定された場合、処理はステップS252に進む。
14, the inverse transform unit 213 determines whether or not pred_mode_flag in the prediction information Pinfo supplied from the decoding unit 211 indicates processing of the intra prediction processing system. If it is determined in step S251 that pred_mode_flag indicates an intra prediction processing system process, the process proceeds to step S252.
ステップS252において、逆変換部213は、SPS.IntraBCflagが、イントラBC予測処理を有効にすることを示す1であるかどうかを判定する。ステップS252でSPS.IntraBCflagが1であると判定された場合、処理はステップS253に進む。
In step S252, the inverse conversion unit 213 determines whether or not SPS.IntraBCflag is 1 indicating that the intra BC prediction process is enabled. If it is determined in step S252 that SPS.IntraBCflag is 1, the process proceeds to step S253.
ステップS253において、逆変換部213は、予測情報PinfoからPU.IntraBCflagを抽出する。そして、ステップS254において、PU.IntraBCflagが、イントラBC予測処理であることを示す1であるかどうかを判定する。ステップS254でPU.IntraBCflagが1であると判定された場合、即ち復号対象のPUに対してイントラBC予測処理が行われる場合、処理はステップS255に進む。
In step S253, the inverse transform unit 213 extracts PU.IntraBCflag from the prediction information Pinfo. Then, in step S254, it is determined whether or not PU.IntraBCflag is 1 indicating that it is an intra BC prediction process. If it is determined in step S254 that PU.IntraBCflag is 1, that is, if the intra BC prediction process is performed on the decoding target PU, the process proceeds to step S255.
ステップS255において、逆変換部213は、イントラBC予測処理用の第2の直交変換行列のインデックスや水平方向および垂直方向の第1の直交変換行列の候補の中から所定のものを選択するイントラBC予測処理用の行列選択処理を行う。このイントラBC予測処理用の行列選択処理の詳細は、後述する図15を参照して説明する。ステップS255の処理後、処理はステップS258に進む。
In step S255, the inverse transform unit 213 selects a predetermined one from the second orthogonal transform matrix index for intra BC prediction processing and the candidates for the first orthogonal transform matrix in the horizontal and vertical directions. Perform matrix selection processing for prediction processing. Details of the matrix selection process for the intra BC prediction process will be described with reference to FIG. After the process of step S255, the process proceeds to step S258.
一方、ステップS252でSPS.IntraBCflagが1ではないと判定された場合、またはステップS254でPU.IntraBCflagが1ではないと判定された場合、即ち復号対象のPUに対してイントラ予測処理が行われる場合、処理はステップS256に進む。
On the other hand, if it is determined in step S252 that SPS.IntraBCflag is not 1, or if it is determined in step S254 that PU.IntraBCflag is not 1, that is, if intra prediction processing is performed on the decoding target PU. The process proceeds to step S256.
ステップS256において、逆変換部213は、イントラ予測処理用の第2の直交変換行列のインデックスや水平方向および垂直方向の第1の直交変換行列の候補の中から所定のものを選択するイントラ予測処理用の行列選択処理を行う。そして、処理はステップS258に進む。
In step S256, the inverse transform unit 213 selects a predetermined one from the second orthogonal transform matrix index for intra prediction processing and the first and second orthogonal transform matrix candidates in the horizontal and vertical directions. Perform the matrix selection process. Then, the process proceeds to step S258.
また、ステップS251でpred_mode_flagがイントラ予測処理系の処理を示さないと判定された場合、即ち復号対象のPUに対してインター予測処理が行われる場合、処理はステップS257に進む。
In addition, when it is determined in step S251 that pred_mode_flag does not indicate processing of the intra prediction processing system, that is, when inter prediction processing is performed on the decoding target PU, the processing proceeds to step S257.
ステップS257において、逆変換部213は、インター予測処理用の第2の直交変換行列のインデックスや水平方向および垂直方向の第1の直交変換行列の候補の中から所定のものを選択するインター予測処理用の行列選択処理を行う。そして、処理はステップS258に進む。
In step S257, the inverse transform unit 213 selects a predetermined one from the second orthogonal transform matrix index for inter prediction processing and the candidates for the first orthogonal transform matrix in the horizontal direction and the vertical direction. Perform the matrix selection process. Then, the process proceeds to step S258.
ステップS258において、選択部231は、ステップS255乃至S257のいずれかで選択された第2の直交変換行列のインデックスが3であるかどうかを判定する。ステップS258で選択された第2の直交変換行列のインデックスが3ではないと判定された場合、選択部231は、そのインデックスに対応するNSST0逆変換部232-1、NSST1逆変換部232-2、またはNSST2逆変換部232-3に、逆量子化部212から供給される変換係数Coeff_IQを供給する。
In step S258, the selection unit 231 determines whether the index of the second orthogonal transformation matrix selected in any of steps S255 to S257 is 3. When it is determined that the index of the second orthogonal transformation matrix selected in step S258 is not 3, the selection unit 231 selects the NSST0 inverse transformation unit 232-1, NSST1 inverse transformation unit 232-2, Alternatively, the transform coefficient Coeff_IQ supplied from the inverse quantization unit 212 is supplied to the NSST2 inverse transform unit 232-3.
そして、ステップS259において、NSST0逆変換部232-1、NSST1逆変換部232-2、またはNSST2逆変換部232-3は、変換係数Coeff_IQに対して、選択されたインデックスにより特定される第2の直交変換行列に対応する第2の逆直交変換行列を用いて第2の逆直交変換行列を行う。その結果得られる変換係数は選択部233に供給され、処理はステップS260に進む。
In step S259, the NSST0 inverse transform unit 232-1, NSST1 inverse transform unit 232-2, or NSST2 inverse transform unit 232-3 performs the second specified by the selected index on the transform coefficient Coeff_IQ. A second inverse orthogonal transformation matrix is performed using a second inverse orthogonal transformation matrix corresponding to the orthogonal transformation matrix. The conversion coefficient obtained as a result is supplied to the selection unit 233, and the process proceeds to step S260.
一方、ステップS258で第2の直交変換行列のインデックスが3であると判定された場合、選択部231は、逆量子化部212から供給される変換係数Coeff_IQを無変換部232-4に供給する。無変換部232-4は、変換係数Coeff_IQに対して無変換行列を用いて第2の逆直交変換を行い、即ち第2の逆直交変換を行わず、変換係数Coeff_IQをそのまま選択部233に供給する。そして、処理はステップS260に進む。
On the other hand, when it is determined in step S258 that the index of the second orthogonal transformation matrix is 3, the selection unit 231 supplies the transform coefficient Coeff_IQ supplied from the inverse quantization unit 212 to the non-conversion unit 232-4. . The non-conversion unit 232-4 performs the second inverse orthogonal transform on the transform coefficient Coeff_IQ using the no-transform matrix, that is, does not perform the second inverse orthogonal transform, and supplies the transform coefficient Coeff_IQ to the selection unit 233 as it is. To do. Then, the process proceeds to step S260.
ステップS260において、選択部233は、ステップS255乃至S257の処理により水平方向および垂直方向の第1の直交変換行列が選択されたかどうかを判定する。ステップS260で水平方向および垂直方向の第1の直交変換行列が選択されたと判定された場合、選択部233は、その水平方向の第1の直交変換行列に対応する水平DCT2逆変換部235-1、水平DCT5逆変換部235-2、水平DCT8逆変換部235-3、水平DST1逆変換部235-4、または水平DST7逆変換部235-5に、供給された変換係数を供給する。そして、処理はステップS261に進む。
In step S260, the selection unit 233 determines whether the first orthogonal transformation matrix in the horizontal direction and the vertical direction is selected by the processing in steps S255 to S257. When it is determined in step S260 that the first orthogonal transformation matrix in the horizontal direction and the vertical direction has been selected, the selection unit 233 selects the horizontal DCT2 inverse transformation unit 235-1 corresponding to the first orthogonal transformation matrix in the horizontal direction. The supplied conversion coefficient is supplied to the horizontal DCT5 inverse transform unit 235-2, the horizontal DCT8 inverse transform unit 235-3, the horizontal DST1 inverse transform unit 235-4, or the horizontal DST7 inverse transform unit 235-5. Then, the process proceeds to step S261.
ステップS261において、水平DCT2逆変換部235-1、水平DCT5逆変換部235-2、水平DCT8逆変換部235-3、水平DST1逆変換部235-4、または水平DST7逆変換部235-5は、選択部233から供給される変換係数に対して、選択された水平方向の第1の直交変換行列に対応する第1の逆直交変換行列を用いて、水平方向に第1の逆直交変換を行う。水平DCT2逆変換部235-1、水平DCT5逆変換部235-2、水平DCT8逆変換部235-3、水平DST1逆変換部235-4、または水平DST7逆変換部235-5は、その結果得られる変換係数を選択部236に供給する。選択部236は、選択された垂直方向の第1の直交変換行列に対応する垂直DCT2逆変換部237-1、垂直DCT5逆変換部237-2、垂直DCT8逆変換部237-3、垂直DST1逆変換部237-4、または垂直DST7逆変換部237-5に、供給された変換係数を供給する。
In step S261, the horizontal DCT2 inverse transform unit 235-1, the horizontal DCT5 inverse transform unit 235-2, the horizontal DCT8 inverse transform unit 235-3, the horizontal DST1 inverse transform unit 235-4, or the horizontal DST7 inverse transform unit 235-5 The first inverse orthogonal transform is performed in the horizontal direction on the transform coefficient supplied from the selection unit 233 using the first inverse orthogonal transform matrix corresponding to the selected first horizontal orthogonal transform matrix. Do. The horizontal DCT2 inverse transform unit 235-1, horizontal DCT5 inverse transform unit 235-2, horizontal DCT8 inverse transform unit 235-3, horizontal DST1 inverse transform unit 235-4, or horizontal DST7 inverse transform unit 235-5 obtains the result. The conversion coefficient to be obtained is supplied to the selection unit 236. The selection unit 236 includes a vertical DCT2 inverse transformation unit 237-1, a vertical DCT5 inverse transformation unit 237-2, a vertical DCT8 inverse transformation unit 237-3, and a vertical DST1 inverse corresponding to the selected first orthogonal transformation matrix in the vertical direction. The supplied conversion coefficient is supplied to the conversion unit 237-4 or the vertical DST7 inverse conversion unit 237-5.
ステップS262において、垂直DCT2逆変換部237-1、垂直DCT5逆変換部237-2、垂直DCT8逆変換部237-3、垂直DST1逆変換部237-4、または垂直DST7逆変換部237-5は、供給された変換係数に対して、選択された垂直方向の第1の直交変換行列に対応する第1の逆直交変換行列を用いて、垂直方向に第1の逆直交変換を行う。水平DCT2逆変換部235-1、水平DCT5逆変換部235-2、水平DCT8逆変換部235-3、水平DST1逆変換部235-4、または水平DST7逆変換部235-5は、その結果得られる予測残差D´を演算部214に供給する。そして、処理は図12のステップS203に戻り、ステップS204に進む。
In step S262, the vertical DCT2 inverse transform unit 237-1, the vertical DCT5 inverse transform unit 237-2, the vertical DCT8 inverse transform unit 237-3, the vertical DST1 inverse transform unit 237-4, or the vertical DST7 inverse transform unit 237-5 The first inverse orthogonal transform is performed on the supplied transform coefficient in the vertical direction by using the first inverse orthogonal transform matrix corresponding to the selected first orthogonal transform matrix in the vertical direction. The horizontal DCT2 inverse transform unit 235-1, horizontal DCT5 inverse transform unit 235-2, horizontal DCT8 inverse transform unit 235-3, horizontal DST1 inverse transform unit 235-4, or horizontal DST7 inverse transform unit 235-5 obtains the result. The predicted residual D ′ is supplied to the calculation unit 214. And a process returns to step S203 of FIG. 12, and progresses to step S204.
図15は、図14のイントラBC予測処理用の行列選択処理を説明するフローチャートである。
FIG. 15 is a flowchart illustrating matrix selection processing for intra BC prediction processing in FIG.
図15のステップS280において、選択部231(図11)は、復号部211から供給されるSPS.NSSTflagが、第2の直交変換を有効にすることを示す1であるかどうかを判定する。ステップS280でSPS.NSSTflagが1であると判定された場合、処理はステップS282に進む。
15, the selection unit 231 (FIG. 11) determines whether the SPS.NSSTflag supplied from the decoding unit 211 is 1 indicating that the second orthogonal transform is valid. If it is determined in step S280 that SPS.NSSTflag is 1, the process proceeds to step S282.
ステップS281において、選択部231は、復号部211から供給される変換情報TinfoからTU.ROTIdxを抽出する。
In step S281, the selection unit 231 extracts TU.ROTIdx from the conversion information Tinfo supplied from the decoding unit 211.
ステップS282において、選択部231は、イントラBC処理用の第2の直交変換行列のインデックスの中から、ステップS281で抽出されたTU.ROTIdxを、第2の直交変換行列のインデックスとして選択し、処理をステップS284に進める。
In step S282, the selection unit 231 selects TU.ROTIdx extracted in step S281 from the second orthogonal transformation matrix index for intra BC processing as the second orthogonal transformation matrix index, and performs processing. Advances to step S284.
一方、ステップS280でSPS.NSSTflagが1ではないと判定された場合、処理はステップS283に進む。ステップS283において、選択部231は、第2の直交変換行列のインデックスとして3を選択し、処理をステップS284に進める。
On the other hand, if it is determined in step S280 that SPS.NSSTflag is not 1, the process proceeds to step S283. In step S283, the selection unit 231 selects 3 as the index of the second orthogonal transformation matrix, and the process proceeds to step S284.
ステップS284において、選択部233は、復号部211から供給されるSPS.AMTflagが第1の直交変換を有効にすることを示す1であるかどうかを判定する。ステップS284でSPS.AMTflagが1であると判定された場合、処理はステップS285に進む。
In step S284, the selection unit 233 determines whether the SPS.AMTflag supplied from the decoding unit 211 is 1 indicating that the first orthogonal transformation is valid. If it is determined in step S284 that SPS.AMTflag is 1, the process proceeds to step S285.
ステップS285において、選択部233は、復号部211から供給される変換情報TinfoからTU.TSflagを抽出する。ステップS286において、選択部233は、TU.TSflagが、第1の直交変換をスキップすることを示す1であるかどうかを判定する。
In step S285, the selection unit 233 extracts TU.TSflag from the conversion information Tinfo supplied from the decoding unit 211. In step S286, the selection unit 233 determines whether TU.TSflag is 1 indicating that the first orthogonal transform is skipped.
ステップS286でTU.TSflagが1ではないと判定された場合、ステップS287において、選択部233は、変換情報Tinfoから水平方向および垂直方向のTU.AMTflagを抽出する。後述するステップS288およびS289の処理は、水平方向および垂直方向のそれぞれについて行われる。
If it is determined in step S286 that TU.TSflag is not 1, in step S287, the selection unit 233 extracts the TU.AMTflag in the horizontal direction and the vertical direction from the conversion information Tinfo. Steps S288 and S289 described later are performed in each of the horizontal direction and the vertical direction.
ステップS288において、選択部233は、TU.AMTflagが1であるかどうかを判定する。ステップS288でTU.AMTflagが1であると判定された場合、処理はステップS289に進む。
In step S288, the selection unit 233 determines whether TU.AMTflag is 1. If it is determined in step S288 that TU.AMTflag is 1, the process proceeds to step S289.
ステップS289において、選択部233は、変換情報TinfoからTU.AMTIdxを抽出し、処理をステップS290に進む。
In step S289, the selection unit 233 extracts TU.AMTIdx from the conversion information Tinfo, and the process proceeds to step S290.
一方、ステップS288でTU.AMTflagが1ではないと判定された場合、処理はステップS290に進む。
On the other hand, if it is determined in step S288 that TU.AMTflag is not 1, the process proceeds to step S290.
ステップS290において、選択部233は、図5のCで説明した方法で、TU.AMTflagとTU.AMTIdxに基づいて、図5のCのイントラBC予測処理用の水平方向および垂直方向の第1の直交変換行列の候補の中から、水平方向および垂直方向の第1の直交変換行列を選択する。そして、処理は図14のステップS255に戻り、ステップS258に進む。
In step S290, the selection unit 233 uses the method described in FIG. 5C, based on TU.AMTflag and TU.AMTIdx, for the first horizontal and vertical first for intra BC prediction processing in C in FIG. A first orthogonal transformation matrix in the horizontal direction and the vertical direction is selected from the candidates for the orthogonal transformation matrix. Then, the process returns to step S255 in FIG. 14 and proceeds to step S258.
また、ステップS284でSPS.AMTflagが1ではないと判定された場合、または、ステップS286でTU.TSflagが1であると判定された場合、以降の処理はスキップされる。即ち水平方向および垂直方向の第1の直交変換行列は選択されない。そして、処理は図14のステップS255に戻り、ステップS258に進む。
Further, if it is determined in step S284 that SPS.AMTflag is not 1, or if TU.TSflag is determined to be 1 in step S286, the subsequent processing is skipped. That is, the first orthogonal transformation matrix in the horizontal direction and the vertical direction is not selected. Then, the process returns to step S255 in FIG. 14 and proceeds to step S258.
なお、図示は省略するが、イントラ予測処理用の行列選択処理は、ステップS290において図5のAで説明した方法で水平方向および垂直方向の第1の直交変換行列が選択される点が、図15のイントラBC予測処理用の行列選択処理と異なっている。また、インター予測処理用の行列選択処理は、ステップS290において図5のBで説明した方法で水平方向および垂直方向の第1の直交変換行列が選択される点が、図15のイントラBC予測処理用の行列選択処理と異なっている。
Although not shown in the figure, the matrix selection processing for intra prediction processing is illustrated in that the first orthogonal transformation matrix in the horizontal direction and the vertical direction is selected in step S290 by the method described in FIG. 5A. This is different from the matrix selection process for 15 intra BC prediction processes. Further, the matrix selection process for the inter prediction process is the intra BC prediction process of FIG. 15 in that the first orthogonal transformation matrix in the horizontal direction and the vertical direction is selected by the method described in FIG. 5B in step S290. This is different from the matrix selection process.
また、上述した説明では、イントラBC予測処理がイントラ予測処理とは異なる予測処理であるようにしたが、イントラBC予測処理を行うモードをイントラ予測モードの1つとして設け、イントラBC予測処理は、そのイントラ予測モードのイントラ予測処理として行われるようにしてもよい。例えば、イントラ予測処理を行うイントラ予測モードが、HEVCで定義される0から34番までのモードである場合、35番のモードのイントラ予測処理がイントラBC予測処理であるようにすることができる。この場合、イントラ予測モードが35番のモードであるときにのみ予測情報Pinfoに動きベクトルが含まれる。
In the above description, the intra BC prediction process is different from the intra prediction process. However, the mode for performing the intra BC prediction process is provided as one of the intra prediction modes, and the intra BC prediction process includes: It may be performed as an intra prediction process in the intra prediction mode. For example, when the intra prediction mode for performing the intra prediction process is a mode from 0 to 34 defined by HEVC, the intra prediction process of the 35th mode may be the intra BC prediction process. In this case, the motion information is included in the prediction information Pinfo only when the intra prediction mode is the 35th mode.
(第1の直交変換行列の他の選択方法の説明)
図16は、イントラBC予測処理を行うイントラ予測モードとして35番のモードが設けられた場合の第1の直交変換行列の選択方法を説明する図である。 (Description of another selection method of the first orthogonal transformation matrix)
FIG. 16 is a diagram for describing a method for selecting the first orthogonal transformation matrix when themode 35 is provided as the intra prediction mode for performing the intra BC prediction process.
図16は、イントラBC予測処理を行うイントラ予測モードとして35番のモードが設けられた場合の第1の直交変換行列の選択方法を説明する図である。 (Description of another selection method of the first orthogonal transformation matrix)
FIG. 16 is a diagram for describing a method for selecting the first orthogonal transformation matrix when the
イントラBC予測処理を行うイントラ予測モードとして35番のモードが設けられた場合、図16のAに示すように、図5のCのイントラBC予測処理用の第1の直交変換行列の候補が、イントラ予測モードが35番のモードである場合のイントラ予測処理用の第1の直交変換行列の候補に代わる。
When the mode No. 35 is provided as the intra prediction mode for performing the intra BC prediction process, as shown in FIG. 16A, the first orthogonal transform matrix candidate for the intra BC prediction process of C in FIG. Instead of the first orthogonal transform matrix candidate for intra prediction processing when the intra prediction mode is the 35th mode.
図16のBに示すように、インター予測処理用の第1の直交変換行列の候補は、図5の場合と同様である。即ち、図16のBと図5のBは同一である。
As shown in FIG. 16B, the candidates for the first orthogonal transformation matrix for inter prediction processing are the same as those in FIG. That is, B in FIG. 16 and B in FIG. 5 are the same.
なお、第1実施の形態では、イントラBC予測処理は、イントラ予測処理系の処理に含められたが、インター予測処理系の処理に含められるようにしてもよい。この場合、PUの予測処理がイントラBC予測処理またはインター予測処理であることが、1つのモード情報pred_mode_flagで示される。また、第2の直交変換は行われないようにしてもよい。
In the first embodiment, the intra BC prediction process is included in the intra prediction process, but may be included in the inter prediction process. In this case, one mode information pred_mode_flag indicates that the PU prediction process is an intra BC prediction process or an inter prediction process. Further, the second orthogonal transform may not be performed.
さらに、リアルタイム性が要求されない画像符号化装置100では、符号化対象のピクチャに基づいて第1の直交変換行列を可変にすることも可能である。最適な第1の直交変換行列は、符号化対象のピクチャの特性に依存するため、符号化対象のピクチャに基づいて算出された最適な第1の直交変換行列を用いて第1の直交変換を行うことにより、符号化効率を向上させることができる。
Furthermore, in the image encoding device 100 that does not require real-time properties, the first orthogonal transformation matrix can be made variable based on the picture to be encoded. Since the optimal first orthogonal transformation matrix depends on the characteristics of the picture to be coded, the first orthogonal transformation is performed using the optimum first orthogonal transformation matrix calculated based on the picture to be coded. By doing so, encoding efficiency can be improved.
この場合、例えば、画像符号化装置100と画像復号装置200は、予め第1の直交変換行列のデフォルト値を設定しておく。そして、画像符号化装置100は、符号化対象のピクチャに最適な第1の直交変換行列がデフォルト値とは異なる場合、その第1の直交変換行列をSPSに含めて画像復号装置200に送信する。画像復号装置200は、SPSに第1の直交変換行列が含まれている場合、第1の直交変換行列のデフォルト値に対応する第1の逆直交変換行列ではなく、SPSに含まれる第1の直交変換行列に対応する第1の逆直交変換行列を用いて第1の逆直交変換を行う。第2の直交変換行列についても同様である。
In this case, for example, the image encoding device 100 and the image decoding device 200 set a default value of the first orthogonal transformation matrix in advance. Then, when the first orthogonal transformation matrix optimal for the picture to be coded is different from the default value, the image coding device 100 includes the first orthogonal transformation matrix in the SPS and transmits the SPS to the image decoding device 200. . When the first orthogonal transformation matrix is included in the SPS, the image decoding apparatus 200 is not the first inverse orthogonal transformation matrix corresponding to the default value of the first orthogonal transformation matrix, but the first first included in the SPS. The first inverse orthogonal transformation is performed using the first inverse orthogonal transformation matrix corresponding to the orthogonal transformation matrix. The same applies to the second orthogonal transformation matrix.
また、第1実施の形態では、インター予測処理またはイントラBC予測処理が行われる場合、TU.ROTIdxの最適値としての3が符号化パラメータに含まれたが、TU.ROTIdxの最適値としての3が符号化パラメータに含まれないようにしてもよい。この場合、画像復号装置200の選択部231は、モード情報pred_mode_flagおよびPU.IntraBCflagに基づいて、インター予測処理またはイントラBC予測処理が行われる場合、変換係数Coeff_IQを無変換部232-4に供給する。
In the first embodiment, when inter prediction processing or intra BC prediction processing is performed, 3 as the optimal value of TU.ROTIdx is included in the encoding parameter, but 3 as the optimal value of TU.ROTIdx. May not be included in the encoding parameter. In this case, when the inter prediction process or the intra BC prediction process is performed based on the mode information pred_mode_flag and PU.IntraBCflag, the selection unit 231 of the image decoding device 200 supplies the transform coefficient Coeff_IQ to the non-transform unit 232-4. .
<第2実施の形態>
(本開示を適用したコンピュータの説明)
上述した一連の処理は、ハードウエアにより実行することもできるし、ソフトウエアにより実行することもできる。一連の処理をソフトウエアにより実行する場合には、そのソフトウエアを構成するプログラムが、コンピュータにインストールされる。ここで、コンピュータには、専用のハードウエアに組み込まれているコンピュータや、各種のプログラムをインストールすることで、各種の機能を実行することが可能な、例えば汎用のパーソナルコンピュータなどが含まれる。 <Second Embodiment>
(Description of computer to which the present disclosure is applied)
The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software is installed in the computer. Here, the computer includes, for example, a general-purpose personal computer capable of executing various functions by installing various programs by installing a computer incorporated in dedicated hardware.
(本開示を適用したコンピュータの説明)
上述した一連の処理は、ハードウエアにより実行することもできるし、ソフトウエアにより実行することもできる。一連の処理をソフトウエアにより実行する場合には、そのソフトウエアを構成するプログラムが、コンピュータにインストールされる。ここで、コンピュータには、専用のハードウエアに組み込まれているコンピュータや、各種のプログラムをインストールすることで、各種の機能を実行することが可能な、例えば汎用のパーソナルコンピュータなどが含まれる。 <Second Embodiment>
(Description of computer to which the present disclosure is applied)
The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software is installed in the computer. Here, the computer includes, for example, a general-purpose personal computer capable of executing various functions by installing various programs by installing a computer incorporated in dedicated hardware.
図17は、上述した一連の処理をプログラムにより実行するコンピュータのハードウエアの構成例を示すブロック図である。
FIG. 17 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processing by a program.
コンピュータ800において、CPU(Central Processing Unit)801,ROM(Read Only Memory)802,RAM(Random Access Memory)803は、バス804により相互に接続されている。
In the computer 800, a CPU (Central Processing Unit) 801, a ROM (Read Only Memory) 802, and a RAM (Random Access Memory) 803 are connected to each other by a bus 804.
バス804には、さらに、入出力インタフェース810が接続されている。入出力インタフェース810には、入力部811、出力部812、記憶部813、通信部814、及びドライブ815が接続されている。
Further, an input / output interface 810 is connected to the bus 804. An input unit 811, an output unit 812, a storage unit 813, a communication unit 814, and a drive 815 are connected to the input / output interface 810.
入力部811は、キーボード、マウス、マイクロホンなどよりなる。出力部812は、ディスプレイ、スピーカなどよりなる。記憶部813は、ハードディスクや不揮発性のメモリなどよりなる。通信部814は、ネットワークインタフェースなどよりなる。ドライブ815は、磁気ディスク、光ディスク、光磁気ディスク、又は半導体メモリなどのリムーバブルメディア821を駆動する。
The input unit 811 includes a keyboard, a mouse, a microphone, and the like. The output unit 812 includes a display, a speaker, and the like. The storage unit 813 includes a hard disk, a nonvolatile memory, and the like. The communication unit 814 includes a network interface or the like. The drive 815 drives a removable medium 821 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.
以上のように構成されるコンピュータ800では、CPU801が、例えば、記憶部813に記憶されているプログラムを、入出力インタフェース810及びバス804を介して、RAM803にロードして実行することにより、上述した一連の処理が行われる。
In the computer 800 configured as described above, the CPU 801 loads the program stored in the storage unit 813 into the RAM 803 via the input / output interface 810 and the bus 804 and executes the program, for example. A series of processing is performed.
コンピュータ800(CPU801)が実行するプログラムは、例えば、パッケージメディア等としてのリムーバブルメディア821に記録して提供することができる。また、プログラムは、ローカルエリアネットワーク、インターネット、デジタル衛星放送といった、有線または無線の伝送媒体を介して提供することができる。
The program executed by the computer 800 (CPU 801) can be provided by being recorded on a removable medium 821 as a package medium, for example. The program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.
コンピュータ800では、プログラムは、リムーバブルメディア821をドライブ815に装着することにより、入出力インタフェース810を介して、記憶部813にインストールすることができる。また、プログラムは、有線または無線の伝送媒体を介して、通信部814で受信し、記憶部813にインストールすることができる。その他、プログラムは、ROM802や記憶部813に、あらかじめインストールしておくことができる。
In the computer 800, the program can be installed in the storage unit 813 via the input / output interface 810 by attaching the removable medium 821 to the drive 815. The program can be received by the communication unit 814 via a wired or wireless transmission medium and installed in the storage unit 813. In addition, the program can be installed in the ROM 802 or the storage unit 813 in advance.
なお、コンピュータ800が実行するプログラムは、本明細書で説明する順序に沿って時系列に処理が行われるプログラムであっても良いし、並列に、あるいは呼び出しが行われたとき等の必要なタイミングで処理が行われるプログラムであっても良い。
Note that the program executed by the computer 800 may be a program that is processed in time series in the order described in this specification, or a necessary timing such as in parallel or when a call is made. It may be a program in which processing is performed.
本明細書に記載された効果はあくまで例示であって限定されるものではなく、他の効果があってもよい。
The effects described in this specification are merely examples and are not limited, and may have other effects.
また、本開示の実施の形態は、上述した実施の形態に限定されるものではなく、本開示の要旨を逸脱しない範囲において種々の変更が可能である。
The embodiments of the present disclosure are not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present disclosure.
例えば、CU,PU,TUの分割は4分割のみであってもよい。
For example, the division of CU, PU, and TU may be only 4 divisions.
また、本開示は、1つの機能をネットワークを介して複数の装置で分担、共同して処理するクラウドコンピューティングの構成をとることができる。
In addition, the present disclosure can take a cloud computing configuration in which one function is shared by a plurality of devices via a network and is jointly processed.
また、上述のフローチャートで説明した各ステップは、1つの装置で実行する他、複数の装置で分担して実行することができる。
Further, each step described in the above flowchart can be executed by one device or can be shared by a plurality of devices.
さらに、1つのステップに複数の処理が含まれる場合には、その1つのステップに含まれる複数の処理は、1つの装置で実行する他、複数の装置で分担して実行することができる。
Further, when a plurality of processes are included in one step, the plurality of processes included in the one step can be executed by being shared by a plurality of apparatuses in addition to being executed by one apparatus.
なお、本開示は、以下のような構成もとることができる。
In addition, this indication can also take the following structures.
(1)
符号化対象の画像に対してイントラBC予測処理を行うことにより予測画像を生成する場合、イントラBC予測処理用の直交変換行列を用いて、前記符号化対象の画像と前記予測画像との残差に対して直交変換を行う変換部
を備える画像処理装置。
(2)
前記イントラBC予測処理用の直交変換行列は、イントラBC予測処理用の直交変換行列の候補から選択される
ように構成された
前記(1)に記載の画像処理装置。
(3)
前記イントラBC予測処理用の直交変換行列の候補は、イントラ予測処理用の直交変換行列の候補とは異なる
ように構成された
前記(2)に記載の画像処理装置。
(4)
前記イントラBC予測処理用の直交変換行列の候補は、インター予測処理用の直交変換行列の候補と同一である
ように構成された
前記(2)に記載の画像処理装置。
(5)
前記直交変換は、第1の直交変換と第2の直交変換により構成され、
前記変換部は、
前記符号化対象の画像に対してイントラBC予測処理を行うことにより前記予測画像を生成する場合、イントラBC予測処理用の第1の直交変換行列を用いて、前記残差に対して前記第1の直交変換を行う第1の直交変換部と、
前記符号化対象の画像に対してイントラBC予測処理を行うことにより前記予測画像を生成する場合、イントラBC予測処理用の第2の直交変換行列を用いて、前記第1の直交変換部による前記第1の直交変換の結果得られる係数に対して前記第2の直交変換を行う第2の直交変換部と
を備える
前記(1)乃至(4)のいずれかに記載の画像処理装置。
(6)
前記イントラBC予測処理用の直交変換行列を特定する情報を設定する設定部
をさらに備える
前記(1)乃至(5)のいずれかに記載の画像処理装置。
(7)
前記イントラBC予測処理用の直交変換行列を設定する設定部
をさらに備える
前記(1)乃至(5)のいずれかに記載の画像処理装置。
(8)
画像処理装置が、
符号化対象の画像に対してイントラBC予測処理を行うことにより予測画像を生成する場合、イントラBC予測処理用の直交変換行列を用いて、前記符号化対象の画像と前記予測画像との残差に対して直交変換を行う変換ステップ
を含む画像処理方法。
(9)
復号対象の画像に対してイントラBC予測処理を行うことにより予測画像を生成する場合、イントラBC予測処理用の逆直交変換行列を用いて、前記復号対象の画像と前記予測画像との残差の直交変換後の係数に対して逆直交変換を行う逆変換部
を備える画像処理装置。
(10)
前記イントラBC予測処理用の逆直交変換行列は、イントラBC予測処理用の直交変換行列の候補から選択された直交変換行列に対応する逆直交変換行列である。
ように構成された
前記(9)に記載の画像処理装置。
(11)
前記イントラBC予測処理用の直交変換行列の候補は、イントラ予測処理用の直交変換行列の候補とは異なる
ように構成された
前記(10)に記載の画像処理装置。
(12)
前記イントラBC予測処理用の直交変換行列の候補は、インター予測処理用の直交変換行列の候補と同一である
ように構成された
前記(10)に記載の画像処理装置。
(13)
前記逆直交変換は、第1の逆直交変換と第2の逆直交変換により構成され、
前記逆変換部は、
前記復号対象の画像に対してイントラBC予測処理を行うことにより前記予測画像を生成する場合、イントラBC予測処理用の第2の逆直交変換行列を用いて、前記残差の直交変換後の係数に対して前記第2の逆直交変換を行う第2の逆直交変換部と、
前記復号対象の画像に対してイントラBC予測処理を行うことにより前記予測画像を生成する場合、イントラBC予測処理用の第1の逆直交変換行列を用いて、前記第2の逆直交変換部による前記第2の逆直交変換の結果得られる係数に対して前記第1の逆直交変換を行う第1の逆直交変換部と
を備える
前記(9)乃至(12)のいずれかに記載の画像処理装置。
(14)
前記逆変換部は、前記残差の直交変換後の係数を生成する際に用いられた直交変換行列を特定する情報に基づいて、前記情報が示す直交変換行列に対応する前記イントラBC予測処理用の逆直交変換行列を用いて、前記残差の直交変換後の係数に対して逆直交変換を行う
ように構成された
前記(9)乃至(13)のいずれかに記載の画像処理装置。
(15)
前記逆変換部は、前記残差の直交変換後の係数を生成する際に用いられた直交変換行列に基づいて、前記直交変換行列に対応する前記イントラBC予測処理用の逆直交変換行列を用いて、前記残差の直交変換後の係数に対して逆直交変換を行う
ように構成された
前記(9)乃至(13)のいずれかに記載の画像処理装置。
(16)
画像処理装置が、
復号対象の画像に対してイントラBC予測処理を行うことにより予測画像を生成する場合、イントラBC予測処理用の逆直交変換行列を用いて、前記復号対象の画像と前記予測画像との残差の直交変換後の係数に対して逆直交変換を行う逆変換ステップ
を含む画像処理方法。 (1)
When a prediction image is generated by performing intra BC prediction processing on an image to be encoded, a residual between the image to be encoded and the prediction image is generated using an orthogonal transform matrix for intra BC prediction processing. An image processing apparatus including a conversion unit that performs orthogonal transform on the image.
(2)
The image processing device according to (1), wherein the orthogonal transform matrix for intra BC prediction processing is configured to be selected from candidates for orthogonal transform matrix for intra BC prediction processing.
(3)
The image processing device according to (2), wherein the candidate for the orthogonal transform matrix for the intra BC prediction process is different from the candidate for the orthogonal transform matrix for the intra prediction process.
(4)
The image processing apparatus according to (2), wherein the candidate for the orthogonal transform matrix for the intra BC prediction process is the same as the candidate for the orthogonal transform matrix for the inter prediction process.
(5)
The orthogonal transform is composed of a first orthogonal transform and a second orthogonal transform,
The converter is
When the predicted image is generated by performing intra BC prediction processing on the image to be encoded, the first orthogonal transform matrix for intra BC prediction processing is used, and the first is applied to the residual. A first orthogonal transform unit that performs orthogonal transform of:
When generating the predicted image by performing intra BC prediction processing on the image to be encoded, the first orthogonal transform unit uses the second orthogonal transform matrix for intra BC prediction processing. The image processing apparatus according to any one of (1) to (4), further comprising: a second orthogonal transform unit that performs the second orthogonal transform on a coefficient obtained as a result of the first orthogonal transform.
(6)
The image processing device according to any one of (1) to (5), further including: a setting unit that sets information for specifying the orthogonal transform matrix for the intra BC prediction process.
(7)
The image processing device according to any one of (1) to (5), further including: a setting unit configured to set an orthogonal transform matrix for the intra BC prediction process.
(8)
The image processing device
When a prediction image is generated by performing intra BC prediction processing on an image to be encoded, a residual between the image to be encoded and the prediction image is generated using an orthogonal transform matrix for intra BC prediction processing. An image processing method including a transforming step for performing orthogonal transform on.
(9)
When generating a predicted image by performing intra BC prediction processing on an image to be decoded, an inverse orthogonal transformation matrix for intra BC prediction processing is used to calculate the residual between the image to be decoded and the predicted image. An image processing apparatus including an inverse transform unit that performs inverse orthogonal transform on a coefficient after orthogonal transform.
(10)
The inverse orthogonal transformation matrix for intra BC prediction processing is an inverse orthogonal transformation matrix corresponding to an orthogonal transformation matrix selected from candidates for orthogonal transformation matrices for intra BC prediction processing.
The image processing apparatus according to (9), configured as described above.
(11)
The image processing device according to (10), wherein the candidate for the orthogonal transform matrix for the intra BC prediction process is different from the candidate for the orthogonal transform matrix for the intra prediction process.
(12)
The image processing device according to (10), wherein the candidate for the orthogonal transform matrix for the intra BC prediction process is the same as the candidate for the orthogonal transform matrix for the inter prediction process.
(13)
The inverse orthogonal transform includes a first inverse orthogonal transform and a second inverse orthogonal transform,
The inverse transformer is
When the predicted image is generated by performing intra BC prediction processing on the decoding target image, the second inverse orthogonal transformation matrix for intra BC prediction processing is used, and the coefficient after orthogonal transformation of the residual is performed. A second inverse orthogonal transform unit for performing the second inverse orthogonal transform on
When the predicted image is generated by performing intra BC prediction processing on the decoding target image, the second inverse orthogonal transform unit uses the first inverse orthogonal transform matrix for intra BC prediction processing. The image processing according to any one of (9) to (12), further comprising: a first inverse orthogonal transform unit that performs the first inverse orthogonal transform on a coefficient obtained as a result of the second inverse orthogonal transform. apparatus.
(14)
The inverse transform unit is for the intra BC prediction process corresponding to the orthogonal transform matrix indicated by the information, based on information identifying the orthogonal transform matrix used when generating the coefficient after the orthogonal transform of the residual The image processing device according to any one of (9) to (13), configured to perform inverse orthogonal transformation on a coefficient after orthogonal transformation of the residual using the inverse orthogonal transformation matrix.
(15)
The inverse transform unit uses the inverse orthogonal transform matrix for the intra BC prediction process corresponding to the orthogonal transform matrix based on the orthogonal transform matrix used when generating the coefficient after the orthogonal transform of the residual. The image processing device according to any one of (9) to (13), configured to perform inverse orthogonal transform on a coefficient after orthogonal transformation of the residual.
(16)
The image processing device
When generating a predicted image by performing intra BC prediction processing on an image to be decoded, an inverse orthogonal transformation matrix for intra BC prediction processing is used to calculate the residual between the image to be decoded and the predicted image. An image processing method including an inverse transform step of performing inverse orthogonal transform on a coefficient after orthogonal transform.
符号化対象の画像に対してイントラBC予測処理を行うことにより予測画像を生成する場合、イントラBC予測処理用の直交変換行列を用いて、前記符号化対象の画像と前記予測画像との残差に対して直交変換を行う変換部
を備える画像処理装置。
(2)
前記イントラBC予測処理用の直交変換行列は、イントラBC予測処理用の直交変換行列の候補から選択される
ように構成された
前記(1)に記載の画像処理装置。
(3)
前記イントラBC予測処理用の直交変換行列の候補は、イントラ予測処理用の直交変換行列の候補とは異なる
ように構成された
前記(2)に記載の画像処理装置。
(4)
前記イントラBC予測処理用の直交変換行列の候補は、インター予測処理用の直交変換行列の候補と同一である
ように構成された
前記(2)に記載の画像処理装置。
(5)
前記直交変換は、第1の直交変換と第2の直交変換により構成され、
前記変換部は、
前記符号化対象の画像に対してイントラBC予測処理を行うことにより前記予測画像を生成する場合、イントラBC予測処理用の第1の直交変換行列を用いて、前記残差に対して前記第1の直交変換を行う第1の直交変換部と、
前記符号化対象の画像に対してイントラBC予測処理を行うことにより前記予測画像を生成する場合、イントラBC予測処理用の第2の直交変換行列を用いて、前記第1の直交変換部による前記第1の直交変換の結果得られる係数に対して前記第2の直交変換を行う第2の直交変換部と
を備える
前記(1)乃至(4)のいずれかに記載の画像処理装置。
(6)
前記イントラBC予測処理用の直交変換行列を特定する情報を設定する設定部
をさらに備える
前記(1)乃至(5)のいずれかに記載の画像処理装置。
(7)
前記イントラBC予測処理用の直交変換行列を設定する設定部
をさらに備える
前記(1)乃至(5)のいずれかに記載の画像処理装置。
(8)
画像処理装置が、
符号化対象の画像に対してイントラBC予測処理を行うことにより予測画像を生成する場合、イントラBC予測処理用の直交変換行列を用いて、前記符号化対象の画像と前記予測画像との残差に対して直交変換を行う変換ステップ
を含む画像処理方法。
(9)
復号対象の画像に対してイントラBC予測処理を行うことにより予測画像を生成する場合、イントラBC予測処理用の逆直交変換行列を用いて、前記復号対象の画像と前記予測画像との残差の直交変換後の係数に対して逆直交変換を行う逆変換部
を備える画像処理装置。
(10)
前記イントラBC予測処理用の逆直交変換行列は、イントラBC予測処理用の直交変換行列の候補から選択された直交変換行列に対応する逆直交変換行列である。
ように構成された
前記(9)に記載の画像処理装置。
(11)
前記イントラBC予測処理用の直交変換行列の候補は、イントラ予測処理用の直交変換行列の候補とは異なる
ように構成された
前記(10)に記載の画像処理装置。
(12)
前記イントラBC予測処理用の直交変換行列の候補は、インター予測処理用の直交変換行列の候補と同一である
ように構成された
前記(10)に記載の画像処理装置。
(13)
前記逆直交変換は、第1の逆直交変換と第2の逆直交変換により構成され、
前記逆変換部は、
前記復号対象の画像に対してイントラBC予測処理を行うことにより前記予測画像を生成する場合、イントラBC予測処理用の第2の逆直交変換行列を用いて、前記残差の直交変換後の係数に対して前記第2の逆直交変換を行う第2の逆直交変換部と、
前記復号対象の画像に対してイントラBC予測処理を行うことにより前記予測画像を生成する場合、イントラBC予測処理用の第1の逆直交変換行列を用いて、前記第2の逆直交変換部による前記第2の逆直交変換の結果得られる係数に対して前記第1の逆直交変換を行う第1の逆直交変換部と
を備える
前記(9)乃至(12)のいずれかに記載の画像処理装置。
(14)
前記逆変換部は、前記残差の直交変換後の係数を生成する際に用いられた直交変換行列を特定する情報に基づいて、前記情報が示す直交変換行列に対応する前記イントラBC予測処理用の逆直交変換行列を用いて、前記残差の直交変換後の係数に対して逆直交変換を行う
ように構成された
前記(9)乃至(13)のいずれかに記載の画像処理装置。
(15)
前記逆変換部は、前記残差の直交変換後の係数を生成する際に用いられた直交変換行列に基づいて、前記直交変換行列に対応する前記イントラBC予測処理用の逆直交変換行列を用いて、前記残差の直交変換後の係数に対して逆直交変換を行う
ように構成された
前記(9)乃至(13)のいずれかに記載の画像処理装置。
(16)
画像処理装置が、
復号対象の画像に対してイントラBC予測処理を行うことにより予測画像を生成する場合、イントラBC予測処理用の逆直交変換行列を用いて、前記復号対象の画像と前記予測画像との残差の直交変換後の係数に対して逆直交変換を行う逆変換ステップ
を含む画像処理方法。 (1)
When a prediction image is generated by performing intra BC prediction processing on an image to be encoded, a residual between the image to be encoded and the prediction image is generated using an orthogonal transform matrix for intra BC prediction processing. An image processing apparatus including a conversion unit that performs orthogonal transform on the image.
(2)
The image processing device according to (1), wherein the orthogonal transform matrix for intra BC prediction processing is configured to be selected from candidates for orthogonal transform matrix for intra BC prediction processing.
(3)
The image processing device according to (2), wherein the candidate for the orthogonal transform matrix for the intra BC prediction process is different from the candidate for the orthogonal transform matrix for the intra prediction process.
(4)
The image processing apparatus according to (2), wherein the candidate for the orthogonal transform matrix for the intra BC prediction process is the same as the candidate for the orthogonal transform matrix for the inter prediction process.
(5)
The orthogonal transform is composed of a first orthogonal transform and a second orthogonal transform,
The converter is
When the predicted image is generated by performing intra BC prediction processing on the image to be encoded, the first orthogonal transform matrix for intra BC prediction processing is used, and the first is applied to the residual. A first orthogonal transform unit that performs orthogonal transform of:
When generating the predicted image by performing intra BC prediction processing on the image to be encoded, the first orthogonal transform unit uses the second orthogonal transform matrix for intra BC prediction processing. The image processing apparatus according to any one of (1) to (4), further comprising: a second orthogonal transform unit that performs the second orthogonal transform on a coefficient obtained as a result of the first orthogonal transform.
(6)
The image processing device according to any one of (1) to (5), further including: a setting unit that sets information for specifying the orthogonal transform matrix for the intra BC prediction process.
(7)
The image processing device according to any one of (1) to (5), further including: a setting unit configured to set an orthogonal transform matrix for the intra BC prediction process.
(8)
The image processing device
When a prediction image is generated by performing intra BC prediction processing on an image to be encoded, a residual between the image to be encoded and the prediction image is generated using an orthogonal transform matrix for intra BC prediction processing. An image processing method including a transforming step for performing orthogonal transform on.
(9)
When generating a predicted image by performing intra BC prediction processing on an image to be decoded, an inverse orthogonal transformation matrix for intra BC prediction processing is used to calculate the residual between the image to be decoded and the predicted image. An image processing apparatus including an inverse transform unit that performs inverse orthogonal transform on a coefficient after orthogonal transform.
(10)
The inverse orthogonal transformation matrix for intra BC prediction processing is an inverse orthogonal transformation matrix corresponding to an orthogonal transformation matrix selected from candidates for orthogonal transformation matrices for intra BC prediction processing.
The image processing apparatus according to (9), configured as described above.
(11)
The image processing device according to (10), wherein the candidate for the orthogonal transform matrix for the intra BC prediction process is different from the candidate for the orthogonal transform matrix for the intra prediction process.
(12)
The image processing device according to (10), wherein the candidate for the orthogonal transform matrix for the intra BC prediction process is the same as the candidate for the orthogonal transform matrix for the inter prediction process.
(13)
The inverse orthogonal transform includes a first inverse orthogonal transform and a second inverse orthogonal transform,
The inverse transformer is
When the predicted image is generated by performing intra BC prediction processing on the decoding target image, the second inverse orthogonal transformation matrix for intra BC prediction processing is used, and the coefficient after orthogonal transformation of the residual is performed. A second inverse orthogonal transform unit for performing the second inverse orthogonal transform on
When the predicted image is generated by performing intra BC prediction processing on the decoding target image, the second inverse orthogonal transform unit uses the first inverse orthogonal transform matrix for intra BC prediction processing. The image processing according to any one of (9) to (12), further comprising: a first inverse orthogonal transform unit that performs the first inverse orthogonal transform on a coefficient obtained as a result of the second inverse orthogonal transform. apparatus.
(14)
The inverse transform unit is for the intra BC prediction process corresponding to the orthogonal transform matrix indicated by the information, based on information identifying the orthogonal transform matrix used when generating the coefficient after the orthogonal transform of the residual The image processing device according to any one of (9) to (13), configured to perform inverse orthogonal transformation on a coefficient after orthogonal transformation of the residual using the inverse orthogonal transformation matrix.
(15)
The inverse transform unit uses the inverse orthogonal transform matrix for the intra BC prediction process corresponding to the orthogonal transform matrix based on the orthogonal transform matrix used when generating the coefficient after the orthogonal transform of the residual. The image processing device according to any one of (9) to (13), configured to perform inverse orthogonal transform on a coefficient after orthogonal transformation of the residual.
(16)
The image processing device
When generating a predicted image by performing intra BC prediction processing on an image to be decoded, an inverse orthogonal transformation matrix for intra BC prediction processing is used to calculate the residual between the image to be decoded and the predicted image. An image processing method including an inverse transform step of performing inverse orthogonal transform on a coefficient after orthogonal transform.
100 画像符号化装置, 101 制御部, 112 変換部, 116 逆変換部, 173-1 垂直DCT2変換部, 173-2 垂直DCT5変換部, 173-3 垂直DCT8変換部, 173-4 垂直DST1変換部, 173-5 垂直DST7変換部, 175-1 水平DCT2変換部, 175-2 水平DCT5変換部, 175-3 水平DCT8変換部, 175-4 水平DST1変換部, 175-5 水平DST7変換部, 177-1 NSST0変換部, 177-2 NSST1変換部, 177-3 NSST2変換部, 177-4 無変換部, 200 画像復号装置, 213 逆変換部, 232-1 NSST0逆変換部, 232-2 NSST1逆変換部, 232-3 NSST2逆変換部, 232-4 無変換部, 235-1 水平DCT2逆変換部, 235-2 水平DCT5逆変換部, 235-3 水平DCT8逆変換部, 235-4 水平DST1逆変換部, 235-5 水平DST7逆変換部, 237-1 垂直DCT2逆変換部, 237-2 垂直DCT5逆変換部, 237-3 垂直DCT8逆変換部, 237-4 垂直DST1逆変換部, 237-5 垂直DST7逆変換部
100 image encoding device, 101 control unit, 112 conversion unit, 116 inverse conversion unit, 173-1 vertical DCT2 conversion unit, 173-2 vertical DCT5 conversion unit, 173-3 vertical DCT8 conversion unit, 173-4 vertical DST1 conversion unit , 173-5 Vertical DST7 conversion unit, 175-1 Horizontal DCT2 conversion unit, 175-2 Horizontal DCT5 conversion unit, 175-3 Horizontal DCT8 conversion unit, 175-4 Horizontal DST1 conversion unit, 175-5 Horizontal DST7 conversion unit, 177 -1 NSST0 conversion unit, 177-2 NSST1 conversion unit, 177-3 NSST2 conversion unit, 177-4 no conversion unit, 200 image decoding device, 213 reverse conversion unit, 232-1 NSST0 reverse conversion unit, 232-2 NSST1 reverse Conversion unit, 233-3 NSST2 reverse conversion unit, 232-4 No conversion unit, 235-1 Horizontal DCT2 reverse conversion unit, 235-2 Horizontal DCT5 reverse conversion unit, 235-3 Horizontal DCT8 inverse transform unit, 235-4 Horizontal DST1 inverse transform unit, 235-5 Horizontal DST7 inverse transform unit, 237-1 Vertical DCT2 inverse transform unit, 237-2 Vertical DCT5 inverse transform unit, 237-3 Vertical DCT8 inverse transform unit , 237-4 Vertical DST1 inverse transform unit, 237-5 Vertical DST7 inverse transform unit
Claims (16)
- 符号化対象の画像に対してイントラBC予測処理を行うことにより予測画像を生成する場合、イントラBC予測処理用の直交変換行列を用いて、前記符号化対象の画像と前記予測画像との残差に対して直交変換を行う変換部
を備える画像処理装置。 When a prediction image is generated by performing intra BC prediction processing on an image to be encoded, a residual between the image to be encoded and the prediction image is generated using an orthogonal transform matrix for intra BC prediction processing. An image processing apparatus including a conversion unit that performs orthogonal transform on the image. - 前記イントラBC予測処理用の直交変換行列は、イントラBC予測処理用の直交変換行列の候補から選択される
ように構成された
請求項1に記載の画像処理装置。 The image processing device according to claim 1, wherein the orthogonal transform matrix for intra BC prediction processing is selected from candidates for orthogonal transform matrix for intra BC prediction processing. - 前記イントラBC予測処理用の直交変換行列の候補は、イントラ予測処理用の直交変換行列の候補とは異なる
ように構成された
請求項2に記載の画像処理装置。 The image processing device according to claim 2, wherein the candidate for the orthogonal transform matrix for intra BC prediction processing is configured to be different from the candidate for the orthogonal transform matrix for intra prediction processing. - 前記イントラBC予測処理用の直交変換行列の候補は、インター予測処理用の直交変換行列の候補と同一である
ように構成された
請求項2に記載の画像処理装置。 The image processing device according to claim 2, wherein the candidate for the orthogonal transform matrix for the intra BC prediction process is the same as the candidate for the orthogonal transform matrix for the inter prediction process. - 前記直交変換は、第1の直交変換と第2の直交変換により構成され、
前記変換部は、
前記符号化対象の画像に対してイントラBC予測処理を行うことにより前記予測画像を生成する場合、イントラBC予測処理用の第1の直交変換行列を用いて、前記残差に対して前記第1の直交変換を行う第1の直交変換部と、
前記符号化対象の画像に対してイントラBC予測処理を行うことにより前記予測画像を生成する場合、イントラBC予測処理用の第2の直交変換行列を用いて、前記第1の直交変換部による前記第1の直交変換の結果得られる係数に対して前記第2の直交変換を行う第2の直交変換部と
を備える
請求項1に記載の画像処理装置。 The orthogonal transform is composed of a first orthogonal transform and a second orthogonal transform,
The converter is
When the predicted image is generated by performing intra BC prediction processing on the image to be encoded, the first orthogonal transform matrix for intra BC prediction processing is used, and the first is applied to the residual. A first orthogonal transform unit that performs orthogonal transform of:
When generating the predicted image by performing intra BC prediction processing on the image to be encoded, the first orthogonal transform unit uses the second orthogonal transform matrix for intra BC prediction processing. The image processing apparatus according to claim 1, further comprising: a second orthogonal transform unit that performs the second orthogonal transform on a coefficient obtained as a result of the first orthogonal transform. - 前記イントラBC予測処理用の直交変換行列を特定する情報を設定する設定部
をさらに備える
請求項1に記載の画像処理装置。 The image processing apparatus according to claim 1, further comprising: a setting unit configured to set information for specifying an orthogonal transform matrix for the intra BC prediction process. - 前記イントラBC予測処理用の直交変換行列を設定する設定部
をさらに備える
請求項1に記載の画像処理装置。 The image processing apparatus according to claim 1, further comprising: a setting unit that sets an orthogonal transformation matrix for the intra BC prediction process. - 画像処理装置が、
符号化対象の画像に対してイントラBC予測処理を行うことにより予測画像を生成する場合、イントラBC予測処理用の直交変換行列を用いて、前記符号化対象の画像と前記予測画像との残差に対して直交変換を行う変換ステップ
を含む画像処理方法。 The image processing device
When a prediction image is generated by performing intra BC prediction processing on an image to be encoded, a residual between the image to be encoded and the prediction image is generated using an orthogonal transform matrix for intra BC prediction processing. An image processing method including a transforming step for performing orthogonal transform on. - 復号対象の画像に対してイントラBC予測処理を行うことにより予測画像を生成する場合、イントラBC予測処理用の逆直交変換行列を用いて、前記復号対象の画像と前記予測画像との残差の直交変換後の係数に対して逆直交変換を行う逆変換部
を備える画像処理装置。 When generating a predicted image by performing intra BC prediction processing on an image to be decoded, an inverse orthogonal transformation matrix for intra BC prediction processing is used to calculate the residual between the image to be decoded and the predicted image. An image processing apparatus including an inverse transform unit that performs inverse orthogonal transform on a coefficient after orthogonal transform. - 前記イントラBC予測処理用の逆直交変換行列は、イントラBC予測処理用の直交変換行列の候補から選択された直交変換行列に対応する逆直交変換行列である。
ように構成された
請求項9に記載の画像処理装置。 The inverse orthogonal transformation matrix for intra BC prediction processing is an inverse orthogonal transformation matrix corresponding to an orthogonal transformation matrix selected from candidates for orthogonal transformation matrices for intra BC prediction processing.
The image processing apparatus according to claim 9 configured as described above. - 前記イントラBC予測処理用の直交変換行列の候補は、イントラ予測処理用の直交変換行列の候補とは異なる
ように構成された
請求項10に記載の画像処理装置。 The image processing apparatus according to claim 10, wherein the candidate for the orthogonal transform matrix for intra BC prediction processing is configured to be different from the candidate for the orthogonal transform matrix for intra prediction processing. - 前記イントラBC予測処理用の直交変換行列の候補は、インター予測処理用の直交変換行列の候補と同一である
ように構成された
請求項10に記載の画像処理装置。 The image processing device according to claim 10, wherein the candidate for the orthogonal transform matrix for intra BC prediction processing is configured to be the same as the candidate for the orthogonal transform matrix for inter prediction processing. - 前記逆直交変換は、第1の逆直交変換と第2の逆直交変換により構成され、
前記逆変換部は、
前記復号対象の画像に対してイントラBC予測処理を行うことにより前記予測画像を生成する場合、イントラBC予測処理用の第2の逆直交変換行列を用いて、前記残差の直交変換後の係数に対して前記第2の逆直交変換を行う第2の逆直交変換部と、
前記復号対象の画像に対してイントラBC予測処理を行うことにより前記予測画像を生成する場合、イントラBC予測処理用の第1の逆直交変換行列を用いて、前記第2の逆直交変換部による前記第2の逆直交変換の結果得られる係数に対して前記第1の逆直交変換を行う第1の逆直交変換部と
を備える
請求項9に記載の画像処理装置。 The inverse orthogonal transform includes a first inverse orthogonal transform and a second inverse orthogonal transform,
The inverse transformer is
When the predicted image is generated by performing intra BC prediction processing on the decoding target image, the second inverse orthogonal transformation matrix for intra BC prediction processing is used, and the coefficient after orthogonal transformation of the residual is performed. A second inverse orthogonal transform unit for performing the second inverse orthogonal transform on
When the predicted image is generated by performing intra BC prediction processing on the decoding target image, the second inverse orthogonal transform unit uses the first inverse orthogonal transform matrix for intra BC prediction processing. The image processing apparatus according to claim 9, further comprising: a first inverse orthogonal transform unit that performs the first inverse orthogonal transform on a coefficient obtained as a result of the second inverse orthogonal transform. - 前記逆変換部は、前記残差の直交変換後の係数を生成する際に用いられた直交変換行列を特定する情報に基づいて、前記情報が示す直交変換行列に対応する前記イントラBC予測処理用の逆直交変換行列を用いて、前記残差の直交変換後の係数に対して逆直交変換を行う
ように構成された
請求項9に記載の画像処理装置。 The inverse transform unit is for the intra BC prediction process corresponding to the orthogonal transform matrix indicated by the information, based on information identifying the orthogonal transform matrix used when generating the coefficient after the orthogonal transform of the residual The image processing apparatus according to claim 9, wherein the inverse orthogonal transform matrix is used to perform inverse orthogonal transform on the coefficient after the orthogonal transform of the residual. - 前記逆変換部は、前記残差の直交変換後の係数を生成する際に用いられた直交変換行列に基づいて、前記直交変換行列に対応する前記イントラBC予測処理用の逆直交変換行列を用いて、前記残差の直交変換後の係数に対して逆直交変換を行う
ように構成された
請求項9に記載の画像処理装置。 The inverse transform unit uses the inverse orthogonal transform matrix for the intra BC prediction process corresponding to the orthogonal transform matrix based on the orthogonal transform matrix used when generating the coefficient after the orthogonal transform of the residual. The image processing apparatus according to claim 9, configured to perform inverse orthogonal transform on a coefficient after orthogonal transformation of the residual. - 画像処理装置が、
復号対象の画像に対してイントラBC予測処理を行うことにより予測画像を生成する場合、イントラBC予測処理用の逆直交変換行列を用いて、前記復号対象の画像と前記予測画像との残差の直交変換後の係数に対して逆直交変換を行う逆変換ステップ
を含む画像処理方法。 The image processing device
When generating a predicted image by performing intra BC prediction processing on an image to be decoded, an inverse orthogonal transformation matrix for intra BC prediction processing is used to calculate the residual between the image to be decoded and the predicted image. An image processing method including an inverse transform step of performing inverse orthogonal transform on a coefficient after orthogonal transform.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2016-177338 | 2016-09-12 | ||
JP2016177338 | 2016-09-12 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2018047669A1 true WO2018047669A1 (en) | 2018-03-15 |
Family
ID=61562293
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2017/030860 WO2018047669A1 (en) | 2016-09-12 | 2017-08-29 | Image processing device and image processing method |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2018047669A1 (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110570369A (en) * | 2019-08-23 | 2019-12-13 | 温州医科大学 | thyroid nodule ultrasonic image denoising method |
WO2020008909A1 (en) * | 2018-07-06 | 2020-01-09 | ソニー株式会社 | Image processing device and method |
WO2020171671A1 (en) * | 2019-02-24 | 2020-08-27 | 엘지전자 주식회사 | Image coding method based on quadratic transform, and apparatus therefor |
CN112567743A (en) * | 2018-08-15 | 2021-03-26 | 日本放送协会 | Image encoding device, image decoding device, and program |
-
2017
- 2017-08-29 WO PCT/JP2017/030860 patent/WO2018047669A1/en active Application Filing
Non-Patent Citations (2)
Title |
---|
CHEN, JIANLE ET AL.: "Algorithm Description of Joint Exploration Test Model 3", JOINT VIDEO EXPLORATION TEAM (JVET) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11 3RD MEETING, 6 July 2016 (2016-07-06), Geneva, CH, pages 22 - 25 * |
ZHANG, XINGYU ET AL.: "trType for Intra Block Copy for screen content coding", JOINT COLLABORATIVE TEAM ON VIDEO CODING (JCT-VC) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11 18TH MEETING, 30 June 2014 (2014-06-30), Sapporo, JP * |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2020008909A1 (en) * | 2018-07-06 | 2020-01-09 | ソニー株式会社 | Image processing device and method |
CN112567743A (en) * | 2018-08-15 | 2021-03-26 | 日本放送协会 | Image encoding device, image decoding device, and program |
WO2020171671A1 (en) * | 2019-02-24 | 2020-08-27 | 엘지전자 주식회사 | Image coding method based on quadratic transform, and apparatus therefor |
US11483589B2 (en) | 2019-02-24 | 2022-10-25 | Lg Electronics Inc. | Image coding method based on quadratic transform, and apparatus therefor |
CN110570369A (en) * | 2019-08-23 | 2019-12-13 | 温州医科大学 | thyroid nodule ultrasonic image denoising method |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11070826B2 (en) | Constrained position dependent intra prediction combination (PDPC) | |
CN114630132B (en) | Model selection in neural network based in-loop filters for video codec | |
JP6462119B2 (en) | Computing device | |
KR102369117B1 (en) | Intra block copy prediction with asymmetric partitions and encoder-side search patterns, search ranges and approaches to partitioning | |
TWI666918B (en) | Determining palette size, palette entries and filtering of palette coded blocks in video coding | |
TW201841503A (en) | Intra filtering flag in video coding | |
TW201830964A (en) | Deriving bilateral filter information based on a prediction mode in video coding | |
US11663476B2 (en) | Method and device for providing compression and transmission of training parameters in distributed processing environment | |
TW201743619A (en) | Confusion of multiple filters in adaptive loop filtering in video coding | |
JP7343669B2 (en) | Method and apparatus for color conversion in VVC | |
WO2018047669A1 (en) | Image processing device and image processing method | |
WO2015139165A1 (en) | Encoder-side decisions for screen content encoding | |
JP7297918B2 (en) | Color conversion for video coding | |
JP7412356B2 (en) | Multi-type tree depth expansion for picture boundary processing | |
CN111355951A (en) | Video decoding method, device and decoding equipment | |
JP2023012484A (en) | Video decoding method and video decoder | |
WO2014087861A1 (en) | Image processing device, image processing method, and program | |
JP2023505270A (en) | Method and apparatus for video coding | |
CN111385572A (en) | Prediction mode determining method and device, coding equipment and decoding equipment | |
CN115314711A (en) | Neural network based filtering for image/video coding and decoding | |
KR102513756B1 (en) | Image encoding method, decoding method, encoder, decoder and storage medium | |
JP2023038229A (en) | Picture partitioning method and apparatus | |
CN117280693A (en) | Unified neural network filter model | |
JP2024028598A (en) | Content adaptive segmentation prediction | |
CN114913249A (en) | Encoding method, decoding method and related devices |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 17848610 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 17848610 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: JP |