WO2019187396A1 - Image decoding device, image coding device, image processing system, image decoding method, and program - Google Patents

Image decoding device, image coding device, image processing system, image decoding method, and program Download PDF

Info

Publication number
WO2019187396A1
WO2019187396A1 PCT/JP2018/046410 JP2018046410W WO2019187396A1 WO 2019187396 A1 WO2019187396 A1 WO 2019187396A1 JP 2018046410 W JP2018046410 W JP 2018046410W WO 2019187396 A1 WO2019187396 A1 WO 2019187396A1
Authority
WO
WIPO (PCT)
Prior art keywords
conversion
transform
size
unit
inverse
Prior art date
Application number
PCT/JP2018/046410
Other languages
French (fr)
Japanese (ja)
Inventor
圭 河村
内藤　整
Original Assignee
Kddi株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Kddi株式会社 filed Critical Kddi株式会社
Publication of WO2019187396A1 publication Critical patent/WO2019187396A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
    • H04N19/122—Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock

Definitions

  • the present invention relates to an image decoding device, an image encoding device, an image processing system, an image decoding method, and a program.
  • a prediction residual signal that is a difference between a prediction signal generated by intra prediction (intraframe prediction) or inter prediction (interframe prediction) and an input image signal is generated, and conversion processing and quantization of the prediction residual signal are performed.
  • Techniques for performing processing for example, HEVC; High Efficiency Video Coding have been proposed (for example, Non-Patent Document 1).
  • the conversion process is a process of converting a prediction residual signal into a frequency component signal using a conversion unit having a conversion unit size as one unit.
  • JEM7 Joint Exploration Test Model 7
  • a first feature is an image decoding device that performs decoding processing of encoded data encoded by an image encoding device and obtains a coefficient level value subjected to conversion processing by the encoding device And an inverse transformation processing unit that obtains a prediction residual signal and at least performs an inverse transformation process of the coefficient level value, and the inverse transformation processing unit has a transform unit size as the inverse transformation process.
  • 1 inverse transform processing and second inverse transform processing are performed with a transform size smaller than the transform unit size.
  • the transform unit size corresponding to the coefficient level value is a threshold value.
  • the second inverse transformation process is a process applied to a case where the conversion unit size is larger than the threshold value.
  • a second feature is the first feature, wherein the image decoding apparatus includes an enlargement processing unit that performs an enlargement process for aligning the transform size corresponding to the prediction residual signal with the transform unit size, and the enlargement process includes:
  • the gist of the present invention is processing applied to a case where the conversion unit size is larger than the threshold value.
  • the transform size is determined such that the number of horizontal pixels of the transform unit size is represented by a power of 2 of the number of horizontal pixels of the transform size.
  • the number of vertical pixels of the conversion unit size is determined to be represented by a power of 2 of the number of vertical pixels of the conversion size, or the number of horizontal pixels of the conversion unit size is equal to the number of horizontal pixels of the conversion size.
  • the number of vertical pixels of the conversion unit size is determined to be expressed by the power of 2 of the number of vertical pixels of the conversion size.
  • the fourth feature is any one of the first feature to the third feature, wherein the decoding unit obtains the conversion size by the decoding process.
  • the fifth feature is any one of the first feature to the third feature, wherein the conversion size is associated with the conversion unit size in advance.
  • a sixth feature is any one of the first feature to the fifth feature, wherein the decoding unit has the coefficient level having a predetermined coefficient level position when the transform unit size is larger than the threshold. Get the value as zero.
  • a seventh feature is an image encoding device, a generation unit that generates a prediction residual signal that is a difference between a prediction signal generated by intra prediction or inter prediction and an input image signal, and the prediction residual signal
  • a conversion processing unit that obtains a coefficient level value, and the conversion processing unit performs a first conversion process using the conversion unit size as the conversion process, and the conversion unit size is larger than the conversion unit size.
  • the second conversion process is performed with a small conversion size
  • the first conversion process is a process applied to a case where a conversion unit size corresponding to the coefficient level value is equal to or smaller than a threshold value.
  • the gist of the 2-conversion process is a process applied to a case where the conversion unit size is larger than the threshold value.
  • An eighth feature is an image processing system including an image encoding device and an image decoding device, and the image encoding device is a difference between a prediction signal generated by intra prediction or inter prediction and an input image signal.
  • a generation unit that generates a prediction residual signal; and a conversion processing unit that performs conversion processing of the prediction residual signal and obtains a coefficient level value.
  • the image decoding device performs encoding by the image encoding device.
  • a decoding unit that performs decoding processing of the encoded data, and obtains the coefficient level value; and at least performs inverse transformation processing of the coefficient level value, and an inverse transformation processing unit that obtains the prediction residual signal
  • the conversion processing unit performs a first conversion process with the conversion unit size as the conversion process, and a second conversion size smaller than the conversion unit size.
  • the inverse conversion processing unit is configured to perform a conversion process, and the inverse conversion processing unit includes a first inverse conversion process with the conversion unit size and a second inverse conversion with a conversion size smaller than the conversion unit size.
  • the first conversion process and the first inverse conversion process are processes applied to a case where a conversion unit size corresponding to the coefficient level value is equal to or smaller than a threshold value.
  • the gist of the 2 conversion process and the second inverse conversion process is a process applied to a case where the conversion unit size is larger than the threshold value.
  • a ninth feature is an image decoding method, which performs a decoding process of encoded data encoded by an image encoding apparatus and obtains a coefficient level value subjected to a conversion process by the encoding apparatus A and at least performing the inverse transformation process of the coefficient level value and obtaining a prediction residual signal, and the step B includes a first inverse transformation process with the transformation unit size as the inverse transformation process. And performing a second inverse transform process with a transform size smaller than the transform unit size, and the first inverse transform process is applied to a case where the transform unit size corresponding to the coefficient level value is equal to or smaller than a threshold value.
  • the second inverse transform process is a process applied to a case where the transform unit size is larger than the threshold value. .
  • a tenth feature is a program used in an image decoding device, which performs decoding processing of encoded data encoded by the image encoding device, and calculates coefficient level values subjected to conversion processing by the encoding device.
  • Step A to obtain and at least perform inverse conversion processing of the coefficient level value, and execute Step B to acquire a prediction residual signal.
  • the conversion unit size is set as the inverse conversion processing. 1 inverse transform process and a step of performing a second inverse transform process with a transform size smaller than the transform unit size, wherein the transform unit size corresponding to the coefficient level value is equal to or smaller than a threshold value.
  • the second inverse transform process is applied to a case where the transform unit size is larger than the threshold value.
  • the gist that it is a sense.
  • FIG. 1 is a diagram illustrating an image processing system 10 according to the embodiment.
  • FIG. 2 is a diagram illustrating the image encoding device 100 according to the embodiment.
  • FIG. 3 is a diagram illustrating the image decoding device 200 according to the embodiment.
  • FIG. 4 is a diagram for explaining the reduction process and the enlargement process according to the embodiment.
  • FIG. 5 is a diagram for explaining the conversion process and the inverse conversion process according to the embodiment.
  • FIG. 6 is a diagram for explaining the conversion process and the inverse conversion process according to the embodiment.
  • FIG. 7 is a diagram for explaining coefficient level values according to the third modification.
  • an image decoding device an image encoding device, an image processing system, which can suppress an increase in load necessary for the conversion processing or the inverse conversion processing, even when the conversion unit size is enlarged.
  • An image decoding apparatus performs a decoding process of encoded data encoded by an image encoding apparatus, obtains a coefficient level value subjected to a conversion process by the encoding apparatus, and a coefficient An inverse transformation processing unit that performs at least level value inverse transformation processing and obtains a prediction residual signal.
  • the inverse conversion processing unit is configured to perform a first inverse conversion process with a conversion unit size and a second inverse conversion process with a conversion size smaller than the conversion unit size as the inverse conversion process.
  • the first conversion process is a process applied to a case where the conversion unit size corresponding to the coefficient level value is equal to or smaller than a threshold value.
  • the second conversion process is a process applied to a case where the conversion unit size is larger than a threshold value.
  • the second inverse conversion process is applied as the inverse conversion process with a conversion size smaller than the conversion unit size. According to such a configuration, even when the conversion unit size is increased, an increase in load necessary for the inverse conversion process can be suppressed.
  • An image encoding device includes a generation unit that generates a prediction residual signal that is a difference between a prediction signal generated by intra prediction or inter prediction and an input image signal, and a conversion process of the prediction residual signal. And at least a conversion processing unit that acquires a coefficient level value.
  • the conversion processing unit is configured to perform a first conversion process with a conversion unit size and a second conversion process with a conversion size smaller than the conversion unit size as the conversion process.
  • the first conversion process is a process applied to a case where the conversion unit size corresponding to the coefficient level value is equal to or smaller than a threshold value.
  • the second conversion process is a process applied to a case where the conversion unit size is larger than a threshold value.
  • the second conversion process is applied as a conversion process with a conversion size smaller than the conversion unit size. According to such a configuration, even if the conversion unit size is increased, an increase in load necessary for the conversion process can be suppressed.
  • an image decoding method related to the operation of the above-described image decoding device may be provided, or an image encoding method related to the operation of the above-described image encoding device may be provided.
  • an image processing system including the above-described image decoding device and image encoding device may be provided.
  • a program related to the operation of the above-described image decoding device may be provided, or a program related to the operation of the above-described image encoding device may be provided.
  • FIG. 1 is a diagram illustrating an image processing system 10 according to an embodiment.
  • the image processing system 10 includes an image encoding device 100 and an image decoding device 200.
  • the image encoding device 100 generates encoded data by encoding an input image signal.
  • the image decoding device 200 generates an output image signal by decoding the encoded data.
  • the encoded data may be transmitted from the image encoding device 100 to the image decoding device 200 via a transmission path.
  • the encoded data may be provided from the image encoding device 100 to the image decoding device 200 after being stored in a storage medium.
  • FIG. 2 is a diagram illustrating the image encoding device 100 according to the embodiment.
  • the image encoding device 100 includes an inter prediction unit 111, an intra prediction unit 112, a subtractor 121, an adder 122, a reduction processing unit 131, an enlargement processing unit 132, A quantization unit 141, an inverse transform / inverse quantization unit 142, a determination unit 150, a switch 151, a switch 152, an encoding unit 160, an in-loop filter 170, and a frame buffer 180 are included.
  • the inter prediction unit 111 generates a prediction signal by inter prediction (interframe prediction). Specifically, the inter prediction unit 111 identifies and identifies a reference unit included in a reference frame by comparing a frame to be encoded (hereinafter referred to as a target frame) with a reference frame stored in the frame buffer 180. Determine a predicted motion vector for the given reference unit. The inter prediction unit 111 generates a prediction signal for each prediction unit based on the prediction unit and the prediction motion vector. The inter prediction unit 111 outputs the prediction signal to the subtracter 121 and the adder 122.
  • the reference frame is a frame different from the target frame.
  • the intra prediction unit 112 generates a prediction signal by intra prediction (intraframe prediction). Specifically, the intra prediction unit 112 specifies a reference unit included in the target frame, and generates a prediction signal for each prediction unit based on the specified reference unit. The intra prediction unit 112 outputs the prediction signal to the subtracter 121 and the adder 122.
  • the reference unit is a unit adjacent to an encoding target unit (hereinafter referred to as a target unit).
  • the subtractor 121 subtracts the prediction signal from the input image signal and outputs the prediction residual signal to the transform / quantization unit 141.
  • the subtractor 121 constitutes a generation unit that generates a prediction residual signal that is a difference between a prediction signal generated by intra prediction or inter prediction and an input image signal.
  • the adder 122 adds the prediction signal to the prediction residual signal output from the inverse transform / inverse quantization unit 142 or the expansion processing unit 132, and outputs the pre-filter decoded signal to the intra prediction unit 112 and the in-loop filter 170.
  • the pre-filter decoded signal constitutes a reference unit used in the intra prediction unit 112. The source of the prediction residual signal is switched by the switch 152.
  • the reduction processing unit 131 performs a reduction process for aligning the conversion unit size corresponding to the prediction error signal to the conversion size.
  • the reduction process is a process applied to a case where the conversion unit size is larger than a threshold value.
  • the conversion unit size is the size of the conversion unit to which the conversion process is to be applied. Details of the reduction processing will be described later (see FIG. 4).
  • the conversion unit size may be defined by N (number of horizontal pixels) ⁇ M (number of vertical pixels).
  • N number of horizontal pixels
  • M number of vertical pixels
  • N and M may be the same number, such as 4 ⁇ 4, 8 ⁇ 8, 16 ⁇ 16, 32 ⁇ 32, 64 ⁇ 64, 128 ⁇ 128, etc., and 32 ⁇ 64, 32 ⁇ 128, 64 ⁇ It may be a different number such as 128.
  • the transform size is the size of a prediction residual signal unit to which a second transform process or a second inverse transform process to be described later is to be applied.
  • the conversion size may be considered to be synonymous with the size of the base pattern (conversion matrix) used in the second conversion process or the second inverse conversion process.
  • the enlargement processing unit 132 performs an enlargement process for aligning the conversion size corresponding to the prediction residual signal with the conversion unit size.
  • the enlargement process is a process applied to a case where the conversion unit size is larger than a threshold value. Details of the enlargement process will be described later (see FIG. 4).
  • the transform / quantization unit 141 performs at least conversion processing of the prediction residual signal and obtains a coefficient level value. Further, the transform / quantization unit 141 may perform quantization of the coefficient level value.
  • the conversion process is a process of converting the prediction residual signal into a frequency component signal. In the conversion process, a base pattern (transformation matrix) corresponding to discrete cosine transform (DCT; Discrete Cosine Transform) may be used, and a base pattern (transformation matrix) corresponding to discrete sine transform (DST; Discrete Sine Transform) May be used.
  • DCT discrete cosine transform
  • DST discrete Sine Transform
  • the transform / quantization unit 141 is configured to perform the first transform process with a transform unit size and perform the second transform process with a transform size smaller than the transform unit size as the transform process.
  • the first conversion process is a process applied to a case where the conversion unit size corresponding to the coefficient level value is equal to or smaller than a threshold value.
  • the second conversion process is a process applied to a case where the conversion unit size is larger than a threshold value.
  • the transform / quantization unit 141 performs the first transform process using the prediction residual signal output from the subtractor 121.
  • the transform / quantization unit 141 performs the second transform process using the prediction residual signal output from the reduction processing unit 131.
  • the inverse transform / inverse quantization unit 142 performs an inverse transform process on the coefficient level value output from the switch 151.
  • the inverse transform / inverse quantization unit 142 may perform inverse quantization of the coefficient level value prior to the inverse transform process.
  • the inverse transformation process and the inverse quantization are performed in the reverse procedure of the transformation process and the quantization performed by the transformation / quantization unit 141.
  • the inverse transform / inverse quantization unit 142 is configured to perform the first inverse transform process with the transform unit size and the second inverse transform process with the transform size smaller than the transform unit size as the inverse transform process. ing.
  • the first conversion process is a process applied to a case where the conversion unit size corresponding to the coefficient level value is equal to or smaller than a threshold value.
  • the second conversion process is a process applied to a case where the conversion unit size is larger than a threshold value.
  • the determination unit 150 determines whether to apply the first conversion process or the second conversion process. Specifically, the determination unit 150 determines to apply the first conversion process when the conversion unit size is equal to or smaller than the threshold value. On the other hand, the determination unit 150 determines to apply the second conversion process when the conversion unit size is larger than the threshold.
  • the threshold may be 64.
  • the determination unit 150 may determine to apply the second conversion process when the number of horizontal pixels of the conversion unit size is larger than the threshold value.
  • the determination unit 150 may determine to apply the second conversion process when the number of vertical pixels of the conversion unit size is larger than the threshold value.
  • the determination unit 150 may determine to apply the second conversion process when both the number of horizontal pixels and the number of vertical pixels of the conversion unit size are larger than the threshold value.
  • the determination unit 150 controls the switch 151 and the switch 152 based on the determination result. Specifically, when it is determined that the first conversion process is applied, the determination unit 150 controls the switch 151 so that the coefficient level value subjected to the first conversion process is output, and performs inverse conversion. The switch 152 is controlled so that the prediction residual signal output from the inverse quantization unit 142 is output. On the other hand, when it is determined that the second conversion process is applied, the determination unit 150 controls the switch 151 so that the coefficient level value subjected to the second conversion process is output, and the enlargement processing unit 132. The switch 152 is controlled so that the prediction residual signal output from is output.
  • the encoding unit 160 encodes the coefficient level value output from the transform / quantization unit 141 and outputs encoded data.
  • encoding is entropy encoding in which codes having different lengths are assigned based on the occurrence probability of coefficient level values.
  • the encoding unit 160 encodes control data used in the decoding process in addition to the coefficient level value.
  • the control data may include size data such as an encoding unit size, a prediction unit size, and a transform unit size.
  • the control data may include data (including a flag) indicating a prediction residual signal generation method, a base pattern type used in conversion processing, and the like.
  • the in-loop filter 170 performs a filtering process on the pre-filter decoded signal output from the adder 122 and outputs the post-filter decoded signal to the frame buffer 180.
  • the filtering process is a deblocking filtering process that reduces distortion generated at a boundary portion of a block (a prediction unit or a transform unit).
  • the frame buffer 180 stores reference frames used in the inter prediction unit 111.
  • the post-filtered decoded signal constitutes a reference frame used in the inter prediction unit 111.
  • FIG. 3 is a diagram illustrating the image decoding device 200 according to the embodiment.
  • the image decoding device 200 includes a decoding unit 210, an inverse transform / inverse quantization unit 220, a determination unit 230, a switch 231, an expansion processing unit 240, an adder 250, and inter prediction.
  • the decoding unit 210 decodes the encoded data generated by the image encoding device 100 and decodes the coefficient level value.
  • the decoding is entropy decoding in a procedure opposite to the entropy encoding performed by the encoding unit 160.
  • the decoding unit 210 may acquire control data by decoding the encoded data.
  • the control data may include size data such as an encoding unit size, a prediction unit size, and a transform unit size.
  • the control data may include data (including a flag) indicating a prediction residual signal generation method, a base pattern type used in the inverse transformation process, and the like.
  • the inverse transform / inverse quantization unit 220 performs an inverse transform process on the coefficient level value output from the decoding unit 210.
  • the inverse transform / inverse quantization unit 220 may perform inverse quantization of the coefficient level value prior to the inverse transform process.
  • the inverse transformation process and the inverse quantization are performed in the reverse procedure of the transformation process and the quantization performed by the transformation / quantization unit 141.
  • the determination unit 230 determines whether to apply the first inverse transformation process or the second inverse transformation process in the same procedure as the determination unit 150. Specifically, the determination unit 230 determines to apply the first inverse conversion process when the conversion unit size is equal to or smaller than the threshold value. On the other hand, the determination unit 230 determines to apply the second inverse conversion process when the conversion unit size is larger than the threshold value.
  • the determination unit 230 controls the switch 231 based on the determination result. Specifically, the determination unit 230 switches the switch 231 so that the prediction residual signal output from the inverse transform / inverse quantization unit 220 is output when it is determined that the first inverse transform process is applied. To control. On the other hand, the determination unit 230 controls the switch 231 so that the prediction residual signal output from the enlargement processing unit 240 is output when it is determined that the second inverse transform process is applied.
  • the enlargement processing unit 240 performs the enlargement processing for aligning the conversion size corresponding to the prediction residual signal to the conversion unit size, similarly to the expansion processing unit 132.
  • the enlargement process is a process applied to a case where the conversion unit size is larger than a threshold value. Details of the enlargement process will be described later (see FIG. 4).
  • the adder 250 adds the prediction signal to the prediction residual signal output from the inverse transform / inverse quantization unit 220 or the expansion processing unit 240, and outputs the pre-filter decoded signal to the intra prediction unit 262 and the in-loop filter 270. .
  • the pre-filter decoded signal constitutes a reference unit used in the intra prediction unit 262.
  • the source of the prediction residual signal is switched by the switch 231.
  • the inter prediction unit 261 generates a prediction signal by inter prediction (interframe prediction) in the same manner as the inter prediction unit 111. Specifically, the inter prediction unit 261 generates a prediction signal for each prediction unit based on the motion vector and the reference frame decoded from the encoded data. The inter prediction unit 261 outputs the prediction signal to the adder 250.
  • inter prediction interframe prediction
  • the intra prediction unit 262 In the same manner as the intra prediction unit 112, the intra prediction unit 262 generates a prediction signal by intra prediction (intraframe prediction). Specifically, the intra prediction unit 262 specifies a reference unit included in the target frame, and generates a prediction signal for each prediction unit based on the specified reference unit. The intra prediction unit 262 outputs the prediction signal to the adder 250.
  • intra prediction intra prediction
  • the in-loop filter 270 performs a filtering process on the pre-filter decoded signal output from the adder 250 and outputs the post-filter decoded signal to the frame buffer 280.
  • the filtering process is a deblocking filtering process that reduces distortion generated at a boundary portion of a block (a prediction unit or a transform unit).
  • the frame buffer 280 stores the reference frame used in the inter prediction unit 261, similarly to the frame buffer 180.
  • the post-filter decoded signal constitutes a reference frame used in the inter prediction unit 261.
  • FIG. 4 is a diagram for explaining the reduction process and the enlargement process according to the embodiment.
  • the reduction process and the enlargement process are processes performed when the second conversion process and the second inverse conversion process are applied.
  • FIG. 4 illustrates a case where the conversion unit size is defined by N (number of horizontal pixels) ⁇ M (number of vertical pixels).
  • the reduction process may be a process of reducing the conversion unit size N to the conversion size N / x in the horizontal direction.
  • the enlargement process may be a process of enlarging the conversion size N / x to the conversion unit size N in the horizontal direction.
  • x may be a power of 2. That is, the conversion size may be determined so that the number of horizontal pixels of the conversion unit size is represented by a power of 2 of the number of horizontal pixels of the conversion size.
  • the reduction process may be a process of reducing the conversion unit size M to the conversion size M / y in the vertical direction.
  • the enlargement process may be a process of enlarging the conversion size M / y to the conversion unit size M in the vertical direction.
  • y may be a power of 2. That is, the conversion size may be determined such that the number of vertical pixels of the conversion unit size is represented by a power of 2 of the number of vertical pixels of the conversion size.
  • the reduction process reduces the conversion unit size N to the conversion size N / x in the horizontal direction and reduces the conversion unit size M to the conversion size M / y in the vertical direction. It may be a process to do.
  • the enlargement process may be a process of enlarging the conversion size N / x to the conversion unit size N in the horizontal direction and enlarging the conversion size M / y to the conversion unit size M in the vertical direction.
  • x and y may be powers of 2.
  • x may be the same value as y or a value different from y.
  • the conversion size is expressed by the number of horizontal pixels of the conversion unit size being a power of 2 of the number of horizontal pixels of the conversion size, and the number of vertical pixels of the conversion unit size being expressed by the power of 2 of the number of vertical pixels of the conversion size. It may be determined to be.
  • the reduction process may be a process of thinning out pixels.
  • the enlargement process may be a process of interpolating pixels.
  • the pixel interpolation may be a process of copying one adjacent pixel, or a process of generating a pixel from two or more adjacent pixels by a weighted average or the like.
  • x or y is a power of 2
  • the load of the reduction process and the enlargement process can be reduced.
  • conversion processing and inverse conversion processing are diagrams for explaining the conversion process and the inverse conversion process according to the embodiment.
  • the conversion process includes a first conversion process and a second conversion process
  • the inverse conversion process includes a first inverse conversion process and a second inverse conversion process.
  • FIG. 5 illustrates a case where the conversion unit size is represented by M 1 ⁇ N 1 . Since the first conversion process is a process that does not involve a reduction process, the size of the prediction error signal is the conversion unit size, as shown in FIG. Therefore, the prediction residual signal is converted into a frequency component using a conversion matrix (DCT or DST) having a conversion unit size. Since the first inverse conversion process is the reverse of the first conversion process, the description thereof is omitted.
  • DCT or DST conversion matrix
  • FIG. 5 illustrates a case where the conversion unit size is represented by M 2 ⁇ N 2 .
  • the size of the prediction error signal is a conversion size (M 2 / x ⁇ N 2 / y) as shown in FIG. Therefore, the prediction residual signal is converted into a frequency component using a conversion matrix (DCT or DST) having a conversion size. Since the second inverse conversion process is the reverse of the second conversion process, the description thereof is omitted.
  • the transform unit size is a power of 2 of the transform size
  • a part of the transform matrix (DCT or DST) having the transform unit size is used as the transform matrix (DCT or DST) having the transform size. Is possible.
  • the second inverse transform process is applied as the inverse transform process with a transform size smaller than the transform unit size. According to such a configuration, even when the conversion unit size is increased, an increase in load necessary for the inverse conversion process can be suppressed.
  • the second transform process is applied as a transform process with a transform size smaller than the transform unit size. According to such a configuration, even if the conversion unit size is increased, an increase in load necessary for the conversion process can be suppressed.
  • the conversion size may be determined such that the number of horizontal pixels or / and the number of vertical pixels of the conversion unit size is represented by a power of 2 of the number of horizontal pixels or / and the number of vertical pixels of the conversion size.
  • the base pattern (conversion matrix) used in the second conversion process or the second inverse conversion process is shared with the base pattern (conversion matrix) used in the first conversion process or the first inverse conversion process. Therefore, an increase in circuit scale necessary for the conversion process and the inverse conversion process can be suppressed.
  • the image encoding device 100 encodes data (including a flag) indicating the conversion size as control data.
  • the image decoding device 200 acquires the transform size by decoding the encoded data. According to such a configuration, although the amount of encoded data increases, the degree of freedom for setting the relationship between the conversion unit size and the conversion size increases.
  • the conversion size is associated with the conversion unit size in advance.
  • the image decoding device 200 identifies the conversion size based on the conversion unit size. For example, when the conversion unit size is 128 ⁇ 128, the conversion size may be predetermined to be 64 ⁇ 64 (or 32 ⁇ 32). As described above, the conversion size is determined in advance such that the number of horizontal pixels or / and the number of vertical pixels of the conversion unit size is represented by the power of 2 of the number of horizontal pixels or / and the number of vertical pixels of the conversion size. Also good. According to such a configuration, although the degree of freedom for setting the relationship between the transform unit size and the transform size is reduced, an increase in the amount of encoded data can be suppressed.
  • the coefficient level value having a predetermined coefficient level position is replaced with zero.
  • the coefficient level value is defined in a space defined by a horizontal frequency and a vertical frequency, as shown in FIG.
  • the predetermined coefficient level position may be a position where the horizontal frequency is higher than the first frequency, a position where the vertical frequency is higher than the second frequency, or The horizontal frequency may be higher than the first frequency, and both the vertical frequency may be higher than the second frequency.
  • the first frequency may be the same value as the second frequency, or may be a value different from the second frequency.
  • the image encoding device 100 determines a predetermined coefficient level position (high-frequency component) when the conversion unit size is larger than the threshold, that is, when the second conversion process is applied. The coefficient level value having is obtained as zero.
  • the image decoding device 200 determines a predetermined coefficient level position (high frequency component). The coefficient level value having is obtained as zero.
  • the load on the image decoding apparatus 200 associated with the inverse transform process can be reduced by replacing the high frequency component with zero.
  • the image encoding device 100 may omit encoding of coefficient level values acquired with zero. According to such a configuration, when encoded data is transmitted from the image encoding apparatus 100 to the image decoding apparatus 200, the signaling amount of the encoded data can be reduced.
  • a determination unit 150 is provided separately from the transform / quantization unit 141 and the inverse transform / inverse quantization unit 142.
  • the transform / quantization unit 141 may determine whether to apply the first transform process or to apply the second transform process. In such a case, the transform / quantization unit 141 performs the first transform process when the transform unit size is equal to or smaller than the threshold value. On the other hand, the transform / quantization unit 141 performs the second transform process when the transform unit size is larger than the threshold value.
  • the inverse transform / inverse quantization unit 142 may determine whether to apply the first inverse transform process or the second inverse transform process.
  • the inverse transform / inverse quantization unit 142 performs the first inverse transform process when the transform unit size is equal to or smaller than the threshold value.
  • the inverse transform / inverse quantization unit 142 performs the second inverse transform process when the transform unit size is larger than the threshold.
  • a determination unit 230 is provided separately from the inverse transform / inverse quantization unit 220.
  • the inverse transform / inverse quantization unit 220 may determine whether to apply the first inverse transform process or the second inverse transform process. Specifically, the inverse transform / inverse quantization unit 220 performs the first inverse transform process when the transform unit size is equal to or smaller than the threshold value. On the other hand, the inverse transform / inverse quantization unit 220 performs the second inverse transform process when the transform unit size is larger than the threshold.
  • the image encoding device 100 determines to apply the first conversion process when the predetermined condition is not satisfied, and performs the second conversion process when the predetermined condition is satisfied. May be determined to apply.
  • the predetermined condition may include a condition other than the condition that the conversion unit size is larger than the threshold value.
  • the predetermined condition may include a condition that a specific intra prediction mode is applied, and the motion information (predicted motion vector or the like) of the conversion target block is the same as the motion information of an adjacent block of the conversion target block. Conditions may be included.
  • the image decoding apparatus 200 determines to apply the first inverse transform process when the predetermined condition is not satisfied, and performs the second inverse transform when the predetermined condition is satisfied. You may determine with applying a process.
  • the predetermined condition may include a condition other than the condition that the conversion unit size is larger than the threshold value.
  • the predetermined condition may include a condition that a specific intra prediction mode is applied, and motion information (predicted motion vector, etc.) of the inverse transform target block is the same as motion information of an adjacent block of the inverse transform target block. It may include the condition that it exists.
  • the image encoding device 100 may generate data indicating whether or not the second inverse transform process should be applied as control data.
  • the image decoding apparatus 200 may determine whether to apply the second inverse transformation process based on data indicating whether the second inverse transformation process should be applied. For example, the data indicating whether or not the second inverse transformation process should be applied is “1” indicating that the second inverse transformation process should be applied and “2” indicating that the second inverse transformation process should not be applied. It may be a flag that can take 0 ′′.
  • the second conversion process and the second inverse conversion process are not applied to the color difference prediction residual signal but applied to the luminance prediction residual signal when the conversion unit size is larger than the threshold. May be.
  • the second conversion process and the second inverse conversion process may be applied to the color difference prediction residual signal instead of being applied to the luminance prediction residual signal when the conversion unit size is larger than the threshold.
  • the second conversion process and the second inverse conversion process may be applied to both the luminance prediction residual signal and the color difference prediction residual signal when the conversion unit size is larger than the threshold value.
  • a program for causing a computer to execute each process performed by the image encoding device 100 and the image decoding device 200 may be provided.
  • the program may be recorded on a computer readable medium. If a computer-readable medium is used, a program can be installed in the computer.
  • the computer-readable medium on which the program is recorded may be a non-transitory recording medium.
  • the non-transitory recording medium is not particularly limited, but may be a recording medium such as a CD-ROM or a DVD-ROM.
  • a chip including a memory that stores a program for executing each process performed by the image encoding device 100 and the image decoding device 200 and a processor that executes the program stored in the memory may be provided.
  • an image decoding device even when the transform unit size is increased, an image decoding device, an image coding device, and an image processing system that can suppress an increase in load necessary for the transform process or the inverse transform process.
  • An image decoding method and program can be provided.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Discrete Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

This image decoding device is provided with: a decoding unit which performs decoding processing on coded data coded by an image coding device, and acquires a coefficient level value that has been subjected to transformation processing by the coding device; and an inverse transformation processing unit which performs at least inverse transformation processing on the coefficient level value, and acquires a prediction residual signal. The inverse transformation processing unit is configured to perform, as the inverse transformation processing, first inverse transformation processing with a transformation unit size, and second inverse transformation processing with a transformation size smaller than the transformation unit size. The first transformation processing is processing that is applied to a case where the transformation unit size corresponding to the coefficient level value is less than or equal to a threshold value. The second transformation processing is processing that is applied to a case where the transformation unit size is greater than the threshold value.

Description

画像復号装置、画像符号化装置、画像処理システム、画像復号方法及びプログラムImage decoding apparatus, image encoding apparatus, image processing system, image decoding method, and program
　本発明は、画像復号装置、画像符号化装置、画像処理システム、画像復号方法及びプログラムに関する。 The present invention relates to an image decoding device, an image encoding device, an image processing system, an image decoding method, and a program.
　従来、イントラ予測（フレーム内予測）又はインター予測（フレーム間予測）によって生成される予測信号と入力画像信号との差分である予測残差信号を生成し、予測残差信号の変換処理及び量子化処理を行う技術（例えば、ＨＥＶＣ；Ｈｉｇｈ　Ｅｆｆｉｃｉｅｎｃｙ　Ｖｉｄｅｏ　Ｃｏｄｉｎｇ）が提案されている（例えば、非特許文献１）。変換処理は、変換ユニットサイズを有する変換ユニットを１つの単位として、予測残差信号を周波数成分信号に変換する処理である。 Conventionally, a prediction residual signal that is a difference between a prediction signal generated by intra prediction (intraframe prediction) or inter prediction (interframe prediction) and an input image signal is generated, and conversion processing and quantization of the prediction residual signal are performed. Techniques for performing processing (for example, HEVC; High Efficiency Video Coding) have been proposed (for example, Non-Patent Document 1). The conversion process is a process of converting a prediction residual signal into a frequency component signal using a conversion unit having a conversion unit size as one unit.
　さらに、上述したＨＥＶＣに対する次世代方式（ＦＶＣ；Ｆｕｔｕｒｅ　Ｖｉｄｅｏ　Ｃｏｄｉｎｇ）では、上述した変換ユニットサイズとして、ＨＥＶＣよりも大きな変換ユニットサイズを用いることが検討されている（例えば、非特許文献２）。 Further, in the next generation method (FVC; Future Video Coding) for HEVC described above, it has been studied to use a conversion unit size larger than HEVC as the conversion unit size described above (for example, Non-Patent Document 2).
　第１の特徴は、画像復号装置であって、画像符号化装置によって符号化された符号化データの復号処理を行うとともに、前記符号化装置によって変換処理が施された係数レベル値を取得する復号部と、前記係数レベル値の逆変換処理を少なくとも行うとともに、予測残差信号を取得する逆変換処理部とを備え、前記逆変換処理部は、前記逆変換処理として、前記変換ユニットサイズで第１逆変換処理と、前記変換ユニットサイズよりも小さい変換サイズで第２逆変換処理を行うように構成されており、前記第１逆変換処理は、前記係数レベル値に対応する変換ユニットサイズが閾値以下であるケースに適用される処理であり、前記第２逆変換処理は、前記変換ユニットサイズが前記閾値よりも大きいケースに適用される処理であることを要旨とする。 A first feature is an image decoding device that performs decoding processing of encoded data encoded by an image encoding device and obtains a coefficient level value subjected to conversion processing by the encoding device And an inverse transformation processing unit that obtains a prediction residual signal and at least performs an inverse transformation process of the coefficient level value, and the inverse transformation processing unit has a transform unit size as the inverse transformation process. 1 inverse transform processing and second inverse transform processing are performed with a transform size smaller than the transform unit size. In the first inverse transform processing, the transform unit size corresponding to the coefficient level value is a threshold value. The second inverse transformation process is a process applied to a case where the conversion unit size is larger than the threshold value. To.
　第２の特徴は、第１の特徴において、画像復号装置が、前記予測残差信号に対応する前記変換サイズを前記変換ユニットサイズに揃える拡大処理を行う拡大処理部を備え、前記拡大処理は、前記変換ユニットサイズが前記閾値よりも大きいケースに適用される処理であることを要旨とする。 A second feature is the first feature, wherein the image decoding apparatus includes an enlargement processing unit that performs an enlargement process for aligning the transform size corresponding to the prediction residual signal with the transform unit size, and the enlargement process includes: The gist of the present invention is processing applied to a case where the conversion unit size is larger than the threshold value.
　第３の特徴は、第１の特徴又は第２の特徴において前記変換サイズは、前記変換ユニットサイズの水平画素数が前記変換サイズの水平画素数の２のべき乗で表されるように定められ、或いは、前記変換ユニットサイズの垂直画素数が前記変換サイズの垂直画素数の２のべき乗で表されるように定められ、或いは、前記変換ユニットサイズの水平画素数が前記変換サイズの水平画素数の２のべき乗で表され、かつ、前記変換ユニットサイズの垂直画素数が前記変換サイズの垂直画素数の２のべき乗で表されるように定められる。 According to a third feature, in the first feature or the second feature, the transform size is determined such that the number of horizontal pixels of the transform unit size is represented by a power of 2 of the number of horizontal pixels of the transform size, Alternatively, the number of vertical pixels of the conversion unit size is determined to be represented by a power of 2 of the number of vertical pixels of the conversion size, or the number of horizontal pixels of the conversion unit size is equal to the number of horizontal pixels of the conversion size. The number of vertical pixels of the conversion unit size is determined to be expressed by the power of 2 of the number of vertical pixels of the conversion size.
　第４の特徴は、第１の特徴乃至第３の特徴のいずれかにおいて、前記復号部は、前記復号処理によって前記変換サイズを取得する。 The fourth feature is any one of the first feature to the third feature, wherein the decoding unit obtains the conversion size by the decoding process.
　第５の特徴は、第１の特徴乃至第３の特徴のいずれかにおいて、前記変換サイズは、前記変換ユニットサイズと予め対応付けられている。 The fifth feature is any one of the first feature to the third feature, wherein the conversion size is associated with the conversion unit size in advance.
　第６の特徴は、第１の特徴乃至第５の特徴のいずれかにおいて、前記復号部は、前記変換ユニットサイズが前記閾値よりも大きい場合において、予め定められた係数レベル位置を有する前記係数レベル値をゼロとして取得する。 A sixth feature is any one of the first feature to the fifth feature, wherein the decoding unit has the coefficient level having a predetermined coefficient level position when the transform unit size is larger than the threshold. Get the value as zero.
　第７の特徴は、画像符号化装置であって、イントラ予測又はインター予測によって生成される予測信号と入力画像信号との差分である予測残差信号を生成する生成部と、前記予測残差信号の変換処理を少なくとも行うとともに、係数レベル値を取得する変換処理部とを備え、前記変換処理部は、前記変換処理として、前記変換ユニットサイズで第１変換処理を行い、前記変換ユニットサイズよりも小さい変換サイズで第２変換処理を行うように構成されており、前記第１変換処理は、前記係数レベル値に対応する変換ユニットサイズが閾値以下であるケースに適用される処理であり、前記第２変換処理は、前記変換ユニットサイズが前記閾値よりも大きいケースに適用される処理であることを要旨とする。 A seventh feature is an image encoding device, a generation unit that generates a prediction residual signal that is a difference between a prediction signal generated by intra prediction or inter prediction and an input image signal, and the prediction residual signal A conversion processing unit that obtains a coefficient level value, and the conversion processing unit performs a first conversion process using the conversion unit size as the conversion process, and the conversion unit size is larger than the conversion unit size. The second conversion process is performed with a small conversion size, and the first conversion process is a process applied to a case where a conversion unit size corresponding to the coefficient level value is equal to or smaller than a threshold value. The gist of the 2-conversion process is a process applied to a case where the conversion unit size is larger than the threshold value.
　第８の特徴は、画像符号化装置及び画像復号装置を備える画像処理システムであって、前記画像符号化装置は、イントラ予測又はインター予測によって生成される予測信号と入力画像信号との差分である予測残差信号を生成する生成部と、前記予測残差信号の変換処理を行うとともに、係数レベル値を取得する変換処理部とを備え、前記画像復号装置は、前記画像符号化装置によって符号化された符号化データの復号処理を行うとともに、前記係数レベル値を取得する復号部と、前記係数レベル値の逆変換処理を少なくとも行うとともに、前記予測残差信号を取得する逆変換処理部とを備え、前記変換処理部は、前記変換処理として、前記変換ユニットサイズで第１変換処理を行い、前記変換ユニットサイズよりも小さい変換サイズで第２変換処理を行うように構成されており、前記逆変換処理部は、前記逆変換処理として、前記変換ユニットサイズで第１逆変換処理と、前記変換ユニットサイズよりも小さい変換サイズで第２逆変換処理を行うように構成されており、前記第１変換処理及び前記第１逆変換処理は、前記係数レベル値に対応する変換ユニットサイズが閾値以下であるケースに適用される処理であり、前記第２変換処理及び前記第２逆変換処理は、前記変換ユニットサイズが前記閾値よりも大きいケースに適用される処理であることを要旨とする。 An eighth feature is an image processing system including an image encoding device and an image decoding device, and the image encoding device is a difference between a prediction signal generated by intra prediction or inter prediction and an input image signal. A generation unit that generates a prediction residual signal; and a conversion processing unit that performs conversion processing of the prediction residual signal and obtains a coefficient level value. The image decoding device performs encoding by the image encoding device. A decoding unit that performs decoding processing of the encoded data, and obtains the coefficient level value; and at least performs inverse transformation processing of the coefficient level value, and an inverse transformation processing unit that obtains the prediction residual signal The conversion processing unit performs a first conversion process with the conversion unit size as the conversion process, and a second conversion size smaller than the conversion unit size. The inverse conversion processing unit is configured to perform a conversion process, and the inverse conversion processing unit includes a first inverse conversion process with the conversion unit size and a second inverse conversion with a conversion size smaller than the conversion unit size. The first conversion process and the first inverse conversion process are processes applied to a case where a conversion unit size corresponding to the coefficient level value is equal to or smaller than a threshold value. The gist of the 2 conversion process and the second inverse conversion process is a process applied to a case where the conversion unit size is larger than the threshold value.
　第９の特徴は、画像復号方法であって、画像符号化装置によって符号化された符号化データの復号処理を行うとともに、前記符号化装置によって変換処理が施された係数レベル値を取得するステップＡと、前記係数レベル値の逆変換処理を少なくとも行うとともに、予測残差信号を取得するステップＢとを備え、前記ステップＢは、前記逆変換処理として、前記変換ユニットサイズで第１逆変換処理と、前記変換ユニットサイズよりも小さい変換サイズで第２逆変換処理を行うステップを含み、前記第１逆変換処理は、前記係数レベル値に対応する変換ユニットサイズが閾値以下であるケースに適用される処理であり、前記第２逆変換処理は、前記変換ユニットサイズが前記閾値よりも大きいケースに適用される処理であることを要旨とする。 A ninth feature is an image decoding method, which performs a decoding process of encoded data encoded by an image encoding apparatus and obtains a coefficient level value subjected to a conversion process by the encoding apparatus A and at least performing the inverse transformation process of the coefficient level value and obtaining a prediction residual signal, and the step B includes a first inverse transformation process with the transformation unit size as the inverse transformation process. And performing a second inverse transform process with a transform size smaller than the transform unit size, and the first inverse transform process is applied to a case where the transform unit size corresponding to the coefficient level value is equal to or smaller than a threshold value. The second inverse transform process is a process applied to a case where the transform unit size is larger than the threshold value. .
　第１０の特徴は、画像復号装置で用いるプログラムであって、画像符号化装置によって符号化された符号化データの復号処理を行うとともに、前記符号化装置によって変換処理が施された係数レベル値を取得するステップＡと、前記係数レベル値の逆変換処理を少なくとも行うとともに、予測残差信号を取得するステップＢとを実行させ、前記ステップＢは、前記逆変換処理として、前記変換ユニットサイズで第１逆変換処理と、前記変換ユニットサイズよりも小さい変換サイズで第２逆変換処理を行うステップを含み、前記第１逆変換処理は、前記係数レベル値に対応する変換ユニットサイズが閾値以下であるケースに適用される処理であり、前記第２逆変換処理は、前記変換ユニットサイズが前記閾値よりも大きいケースに適用される処理であることを要旨とする。 A tenth feature is a program used in an image decoding device, which performs decoding processing of encoded data encoded by the image encoding device, and calculates coefficient level values subjected to conversion processing by the encoding device. Step A to obtain and at least perform inverse conversion processing of the coefficient level value, and execute Step B to acquire a prediction residual signal. In Step B, the conversion unit size is set as the inverse conversion processing. 1 inverse transform process and a step of performing a second inverse transform process with a transform size smaller than the transform unit size, wherein the transform unit size corresponding to the coefficient level value is equal to or smaller than a threshold value. The second inverse transform process is applied to a case where the transform unit size is larger than the threshold value. The gist that it is a sense.
図１は、実施形態に係る画像処理システム１０を示す図である。FIG. 1 is a diagram illustrating an image processing system 10 according to the embodiment. 図２は、実施形態に係る画像符号化装置１００を示す図である。FIG. 2 is a diagram illustrating the image encoding device 100 according to the embodiment. 図３は、実施形態に係る画像復号装置２００を示す図である。FIG. 3 is a diagram illustrating the image decoding device 200 according to the embodiment. 図４は、実施形態に係る縮小処理及び拡大処理を説明するための図である。FIG. 4 is a diagram for explaining the reduction process and the enlargement process according to the embodiment. 図５は、実施形態に係る変換処理及び逆変換処理を説明するための図である。FIG. 5 is a diagram for explaining the conversion process and the inverse conversion process according to the embodiment. 図６は、実施形態に係る変換処理及び逆変換処理を説明するための図である。FIG. 6 is a diagram for explaining the conversion process and the inverse conversion process according to the embodiment. 図７は、変更例３に係る係数レベル値を説明するための図である。FIG. 7 is a diagram for explaining coefficient level values according to the third modification.
　以下において、実施形態について図面を参照しながら説明する。なお、以下の図面の記載において、同一又は類似の部分には、同一又は類似の符号を付している。 Hereinafter, embodiments will be described with reference to the drawings. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.
　但し、図面は模式的なものであり、各寸法の比率などは現実のものとは異なる場合があることに留意すべきである。従って、具体的な寸法などは以下の説明を参酌して判断すべきである。また、図面相互間においても互いの寸法の関係又は比率が異なる部分が含まれている場合があることは勿論である。 However, it should be noted that the drawings are schematic and ratios of dimensions may differ from actual ones. Therefore, specific dimensions and the like should be determined in consideration of the following description. Of course, the drawings may include portions having different dimensional relationships or ratios.
　［開示の概要］
　ところで、上述した変換ユニットサイズの拡大に伴って、変換処理又は逆変換処理に必要な負荷（演算時間及び回路規模など）の増大が見込まれる。 [Outline of Disclosure]
By the way, with the increase in the conversion unit size described above, an increase in load (such as calculation time and circuit scale) necessary for the conversion process or the inverse conversion process is expected.
　そこで、開示では、変換ユニットサイズが拡大した場合であっても、変換処理又は逆変換処理に必要な負荷の増大を抑制することを可能とする画像復号装置、画像符号化装置、画像処理システム、画像復号方法及びプログラムについて説明する。 Therefore, in the disclosure, an image decoding device, an image encoding device, an image processing system, which can suppress an increase in load necessary for the conversion processing or the inverse conversion processing, even when the conversion unit size is enlarged, An image decoding method and program will be described.
　開示の概要に係る画像復号装置は、画像符号化装置によって符号化された符号化データの復号処理を行うとともに、符号化装置によって変換処理が施された係数レベル値を取得する復号部と、係数レベル値の逆変換処理を少なくとも行うとともに、予測残差信号を取得する逆変換処理部とを備える。逆変換処理部は、逆変換処理として、変換ユニットサイズで第１逆変換処理と、変換ユニットサイズよりも小さい変換サイズで第２逆変換処理を行うように構成されている。第１変換処理は、係数レベル値に対応する変換ユニットサイズが閾値以下であるケースに適用される処理である。第２変換処理は、変換ユニットサイズが閾値よりも大きいケースに適用される処理である。 An image decoding apparatus according to an overview of the disclosure performs a decoding process of encoded data encoded by an image encoding apparatus, obtains a coefficient level value subjected to a conversion process by the encoding apparatus, and a coefficient An inverse transformation processing unit that performs at least level value inverse transformation processing and obtains a prediction residual signal. The inverse conversion processing unit is configured to perform a first inverse conversion process with a conversion unit size and a second inverse conversion process with a conversion size smaller than the conversion unit size as the inverse conversion process. The first conversion process is a process applied to a case where the conversion unit size corresponding to the coefficient level value is equal to or smaller than a threshold value. The second conversion process is a process applied to a case where the conversion unit size is larger than a threshold value.
　開示の概要に係る画像復号装置では、変換ユニットサイズが閾値よりも大きい場合に、逆変換処理として、変換ユニットサイズよりも小さい変換サイズで第２逆変換処理が適用される。このような構成によれば、変換ユニットサイズが拡大した場合であっても、逆変換処理に必要な負荷の増大を抑制することができる。 In the image decoding device according to the outline of the disclosure, when the conversion unit size is larger than the threshold, the second inverse conversion process is applied as the inverse conversion process with a conversion size smaller than the conversion unit size. According to such a configuration, even when the conversion unit size is increased, an increase in load necessary for the inverse conversion process can be suppressed.
　開示の概要に係る画像符号化装置は、イントラ予測又はインター予測によって生成される予測信号と入力画像信号との差分である予測残差信号を生成する生成部と、予測残差信号の変換処理を少なくとも行うとともに、係数レベル値を取得する変換処理部とを備える。変換処理部は、変換処理として、変換ユニットサイズで第１変換処理を行い、変換ユニットサイズよりも小さい変換サイズで第２変換処理を行うように構成されている。第１変換処理は、係数レベル値に対応する変換ユニットサイズが閾値以下であるケースに適用される処理である。第２変換処理は、変換ユニットサイズが閾値よりも大きいケースに適用される処理である。 An image encoding device according to an outline of the disclosure includes a generation unit that generates a prediction residual signal that is a difference between a prediction signal generated by intra prediction or inter prediction and an input image signal, and a conversion process of the prediction residual signal. And at least a conversion processing unit that acquires a coefficient level value. The conversion processing unit is configured to perform a first conversion process with a conversion unit size and a second conversion process with a conversion size smaller than the conversion unit size as the conversion process. The first conversion process is a process applied to a case where the conversion unit size corresponding to the coefficient level value is equal to or smaller than a threshold value. The second conversion process is a process applied to a case where the conversion unit size is larger than a threshold value.
　開示の概要に係る画像符号化装置では、変換ユニットサイズが閾値よりも大きい場合に、変換処理として、変換ユニットサイズよりも小さい変換サイズで第２変換処理が適用される。このような構成によれば、変換ユニットサイズが拡大した場合であっても、変換処理に必要な負荷の増大を抑制することができる。 In the image encoding device according to the outline of the disclosure, when the conversion unit size is larger than the threshold, the second conversion process is applied as a conversion process with a conversion size smaller than the conversion unit size. According to such a configuration, even if the conversion unit size is increased, an increase in load necessary for the conversion process can be suppressed.
　開示の概要としては、上述した画像復号装置の動作に係る画像復号方法が提供されてもよく、上述した画像符号化装置の動作に係る画像符号化方法が提供されてもよい。開示の概要としては、上述した画像復号装置及び画像符号化装置を有する画像処理システムが提供されてもよい。開示の概要としては、上述した画像復号装置の動作に係るプログラムが提供されてもよく、上述した画像符号化装置の動作に係るプログラムが提供されてもよい。 As an outline of the disclosure, an image decoding method related to the operation of the above-described image decoding device may be provided, or an image encoding method related to the operation of the above-described image encoding device may be provided. As an overview of the disclosure, an image processing system including the above-described image decoding device and image encoding device may be provided. As an overview of the disclosure, a program related to the operation of the above-described image decoding device may be provided, or a program related to the operation of the above-described image encoding device may be provided.
　［実施形態］
　（画像処理システム）
　以下において、実施形態に係る画像処理システムについて説明する。図１は、実施形態に係る実施形態に係る画像処理システム１０を示す図である。 [Embodiment]
(Image processing system)
Hereinafter, an image processing system according to the embodiment will be described. FIG. 1 is a diagram illustrating an image processing system 10 according to an embodiment.
　図１に示すように、画像処理システム１０は、画像符号化装置１００及び画像復号装置２００を有する。画像符号化装置１００は、入力画像信号を符号化することによって符号化データを生成する。画像復号装置２００は、符号化データを復号することによって出力画像信号を生成する。符号化データは、画像符号化装置１００から画像復号装置２００に対して伝送路を介して送信されてもよい。符号化データは、記憶媒体に格納された上で、画像符号化装置１００から画像復号装置２００に提供されてもよい。 As shown in FIG. 1, the image processing system 10 includes an image encoding device 100 and an image decoding device 200. The image encoding device 100 generates encoded data by encoding an input image signal. The image decoding device 200 generates an output image signal by decoding the encoded data. The encoded data may be transmitted from the image encoding device 100 to the image decoding device 200 via a transmission path. The encoded data may be provided from the image encoding device 100 to the image decoding device 200 after being stored in a storage medium.
　（画像符号化装置）
　以下において、実施形態に係る画像符号化装置について説明する。図２は、実施形態に係る画像符号化装置１００を示す図である。 (Image coding device)
Hereinafter, an image encoding device according to the embodiment will be described. FIG. 2 is a diagram illustrating the image encoding device 100 according to the embodiment.
　図２に示すように、画像符号化装置１００は、インター予測部１１１と、イントラ予測部１１２と、減算器１２１と、加算器１２２と、縮小処理部１３１と、拡大処理部１３２と、変換・量子化部１４１と、逆変換・逆量子化部１４２と、判定部１５０と、スイッチ１５１と、スイッチ１５２と、符号化部１６０と、インループフィルタ１７０と、フレームバッファ１８０とを有する。 As illustrated in FIG. 2, the image encoding device 100 includes an inter prediction unit 111, an intra prediction unit 112, a subtractor 121, an adder 122, a reduction processing unit 131, an enlargement processing unit 132, A quantization unit 141, an inverse transform / inverse quantization unit 142, a determination unit 150, a switch 151, a switch 152, an encoding unit 160, an in-loop filter 170, and a frame buffer 180 are included.
　インター予測部１１１は、インター予測（フレーム間予測）によって予測信号を生成する。具体的には、インター予測部１１１は、符号化対象のフレーム（以下、対象フレーム）とフレームバッファ１８０に格納される参照フレームとの比較によって、参照フレームに含まれる参照ユニットを特定し、特定された参照ユニットに対する予測動きベクトルを決定する。インター予測部１１１は、予測ユニット及び予測動きベクトルに基づいて予測信号を予測ユニット毎に生成する。インター予測部１１１は、予測信号を減算器１２１及び加算器１２２に出力する。参照フレームは、対象フレームとは異なるフレームである。 The inter prediction unit 111 generates a prediction signal by inter prediction (interframe prediction). Specifically, the inter prediction unit 111 identifies and identifies a reference unit included in a reference frame by comparing a frame to be encoded (hereinafter referred to as a target frame) with a reference frame stored in the frame buffer 180. Determine a predicted motion vector for the given reference unit. The inter prediction unit 111 generates a prediction signal for each prediction unit based on the prediction unit and the prediction motion vector. The inter prediction unit 111 outputs the prediction signal to the subtracter 121 and the adder 122. The reference frame is a frame different from the target frame.
　イントラ予測部１１２は、イントラ予測（フレーム内予測）によって予測信号を生成する。具体的には、イントラ予測部１１２は、対象フレームに含まれる参照ユニットを特定し、特定された参照ユニットに基づいて予測信号を予測ユニット毎に生成する。イントラ予測部１１２は、予測信号を減算器１２１及び加算器１２２に出力する。例えば、参照ユニットは、符号化対象のユニット（以下、対象ユニット）に隣接するユニットである。 The intra prediction unit 112 generates a prediction signal by intra prediction (intraframe prediction). Specifically, the intra prediction unit 112 specifies a reference unit included in the target frame, and generates a prediction signal for each prediction unit based on the specified reference unit. The intra prediction unit 112 outputs the prediction signal to the subtracter 121 and the adder 122. For example, the reference unit is a unit adjacent to an encoding target unit (hereinafter referred to as a target unit).
　減算器１２１は、入力画像信号から予測信号を減算し、予測残差信号を変換・量子化部１４１に出力する。ここで、減算器１２１は、イントラ予測又はインター予測によって生成される予測信号と入力画像信号との差分である予測残差信号を生成する生成部を構成する。 The subtractor 121 subtracts the prediction signal from the input image signal and outputs the prediction residual signal to the transform / quantization unit 141. Here, the subtractor 121 constitutes a generation unit that generates a prediction residual signal that is a difference between a prediction signal generated by intra prediction or inter prediction and an input image signal.
　加算器１２２は、逆変換・逆量子化部１４２又は拡大処理部１３２から出力される予測残差信号に予測信号を加算し、フィルタ前復号信号をイントラ予測部１１２及びインループフィルタ１７０に出力する。フィルタ前復号信号は、イントラ予測部１１２で用いる参照ユニットを構成する。予測残差信号のソースは、スイッチ１５２によって切り替えられる。 The adder 122 adds the prediction signal to the prediction residual signal output from the inverse transform / inverse quantization unit 142 or the expansion processing unit 132, and outputs the pre-filter decoded signal to the intra prediction unit 112 and the in-loop filter 170. . The pre-filter decoded signal constitutes a reference unit used in the intra prediction unit 112. The source of the prediction residual signal is switched by the switch 152.
　縮小処理部１３１は、予測誤差信号に対応する変換ユニットサイズを変換サイズに揃える縮小処理を行う。縮小処理は、変換ユニットサイズが閾値よりも大きいケースに適用される処理である。変換ユニットサイズは、変換処理を適用すべき変換ユニットのサイズである。縮小処理の詳細については後述する（図４を参照）。 The reduction processing unit 131 performs a reduction process for aligning the conversion unit size corresponding to the prediction error signal to the conversion size. The reduction process is a process applied to a case where the conversion unit size is larger than a threshold value. The conversion unit size is the size of the conversion unit to which the conversion process is to be applied. Details of the reduction processing will be described later (see FIG. 4).
　ここで、変換ユニットサイズは、Ｎ（水平画素数）×Ｍ（垂直画素数）で定義されてもよい。例えば、Ｎの最大数は１２８であってもよく、Ｍの最大数は１２８であってもよい。Ｎ及びＭは、４×４、８×８、１６×１６、３２×３２、６４×６４、１２８×１２８などのように同じ数であってもよく、３２×６４、３２×１２８、６４×１２８などのように異なる数であってもよい。変換サイズは、後述する第２変換処理又は第２逆変換処理を適用すべき予測残差信号のユニットのサイズである。変換サイズは、第２変換処理又は第２逆変換処理で用いる基底パターン（変換行列）のサイズと同義であると考えてもよい。 Here, the conversion unit size may be defined by N (number of horizontal pixels) × M (number of vertical pixels). For example, the maximum number of N may be 128, and the maximum number of M may be 128. N and M may be the same number, such as 4 × 4, 8 × 8, 16 × 16, 32 × 32, 64 × 64, 128 × 128, etc., and 32 × 64, 32 × 128, 64 × It may be a different number such as 128. The transform size is the size of a prediction residual signal unit to which a second transform process or a second inverse transform process to be described later is to be applied. The conversion size may be considered to be synonymous with the size of the base pattern (conversion matrix) used in the second conversion process or the second inverse conversion process.
　拡大処理部１３２は、予測残差信号に対応する変換サイズを変換ユニットサイズに揃える拡大処理を行う。拡大処理は、変換ユニットサイズが閾値よりも大きいケースに適用される処理である。拡大処理の詳細については後述する（図４を参照）。 The enlargement processing unit 132 performs an enlargement process for aligning the conversion size corresponding to the prediction residual signal with the conversion unit size. The enlargement process is a process applied to a case where the conversion unit size is larger than a threshold value. Details of the enlargement process will be described later (see FIG. 4).
　変換・量子化部１４１は、予測残差信号の変換処理を少なくも行うとともに、係数レベル値を取得する。さらに、変換・量子化部１４１は、係数レベル値の量子化を行ってもよい。変換処理は、予測残差信号を周波数成分信号に変換する処理である。変換処理では、離散コサイン変換（ＤＣＴ；Ｄｉｓｃｒｅｔｅ　Ｃｏｓｉｎｅ　Ｔｒａｎｓｆｏｒｍ）に対応する基底パターン（変換行列）が用いられてもよく、離散サイン変換（ＤＳＴ；Ｄｉｓｃｒｅｔｅ　Ｓｉｎｅ　Ｔｒａｎｓｆｏｒｍ）に対応する基底パターン（変換行列）が用いられてもよい。 The transform / quantization unit 141 performs at least conversion processing of the prediction residual signal and obtains a coefficient level value. Further, the transform / quantization unit 141 may perform quantization of the coefficient level value. The conversion process is a process of converting the prediction residual signal into a frequency component signal. In the conversion process, a base pattern (transformation matrix) corresponding to discrete cosine transform (DCT; Discrete Cosine Transform) may be used, and a base pattern (transformation matrix) corresponding to discrete sine transform (DST; Discrete Sine Transform) May be used.
　実施形態では、変換・量子化部１４１は、変換処理として、変換ユニットサイズで第１変換処理を行い、変換ユニットサイズよりも小さい変換サイズで第２変換処理を行うように構成されている。第１変換処理は、係数レベル値に対応する変換ユニットサイズが閾値以下であるケースに適用される処理である。第２変換処理は、変換ユニットサイズが閾値よりも大きいケースに適用される処理である。具体的には、変換・量子化部１４１は、減算器１２１から出力される予測残差信号を用いて第１変換処理を行う。変換・量子化部１４１は、縮小処理部１３１から出力される予測残差信号を用いて第２変換処理を行う。 In the embodiment, the transform / quantization unit 141 is configured to perform the first transform process with a transform unit size and perform the second transform process with a transform size smaller than the transform unit size as the transform process. The first conversion process is a process applied to a case where the conversion unit size corresponding to the coefficient level value is equal to or smaller than a threshold value. The second conversion process is a process applied to a case where the conversion unit size is larger than a threshold value. Specifically, the transform / quantization unit 141 performs the first transform process using the prediction residual signal output from the subtractor 121. The transform / quantization unit 141 performs the second transform process using the prediction residual signal output from the reduction processing unit 131.
　逆変換・逆量子化部１４２は、スイッチ１５１から出力される係数レベル値の逆変換処理を行う。ここで、逆変換・逆量子化部１４２は、逆変換処理に先立って、係数レベル値の逆量子化を行ってもよい。逆変換処理及び逆量子化は、変換・量子化部１４１で行われる変換処理及び量子化とは逆の手順で行われる。 The inverse transform / inverse quantization unit 142 performs an inverse transform process on the coefficient level value output from the switch 151. Here, the inverse transform / inverse quantization unit 142 may perform inverse quantization of the coefficient level value prior to the inverse transform process. The inverse transformation process and the inverse quantization are performed in the reverse procedure of the transformation process and the quantization performed by the transformation / quantization unit 141.
　実施形態では、逆変換・逆量子化部１４２は、逆変換処理として、変換ユニットサイズで第１逆変換処理と、変換ユニットサイズよりも小さい変換サイズで第２逆変換処理を行うように構成されている。第１変換処理は、係数レベル値に対応する変換ユニットサイズが閾値以下であるケースに適用される処理である。第２変換処理は、変換ユニットサイズが閾値よりも大きいケースに適用される処理である。 In the embodiment, the inverse transform / inverse quantization unit 142 is configured to perform the first inverse transform process with the transform unit size and the second inverse transform process with the transform size smaller than the transform unit size as the inverse transform process. ing. The first conversion process is a process applied to a case where the conversion unit size corresponding to the coefficient level value is equal to or smaller than a threshold value. The second conversion process is a process applied to a case where the conversion unit size is larger than a threshold value.
　判定部１５０は、第１変換処理を適用すべきか、第２変換処理を適用すべきかを判定する。具体的には、判定部１５０は、変換ユニットサイズが閾値以下である場合に、第１変換処理を適用すると判定する。一方で、判定部１５０は、変換ユニットサイズが閾値よりも大きい場合に、第２変換処理を適用すると判定する。特に限定されるものではないが、閾値は６４であってもよい。 The determination unit 150 determines whether to apply the first conversion process or the second conversion process. Specifically, the determination unit 150 determines to apply the first conversion process when the conversion unit size is equal to or smaller than the threshold value. On the other hand, the determination unit 150 determines to apply the second conversion process when the conversion unit size is larger than the threshold. Although not particularly limited, the threshold may be 64.
　ここで、判定部１５０は、変換ユニットサイズの水平画素数が閾値よりも大きい場合に、第２変換処理を適用すると判定してもよい。判定部１５０は、変換ユニットサイズの垂直画素数が閾値よりも大きい場合に、第２変換処理を適用すると判定してもよい。判定部１５０は、変換ユニットサイズの水平画素数及び垂直画素数の双方が閾値よりも大きい場合に、第２変換処理を適用すると判定してもよい。 Here, the determination unit 150 may determine to apply the second conversion process when the number of horizontal pixels of the conversion unit size is larger than the threshold value. The determination unit 150 may determine to apply the second conversion process when the number of vertical pixels of the conversion unit size is larger than the threshold value. The determination unit 150 may determine to apply the second conversion process when both the number of horizontal pixels and the number of vertical pixels of the conversion unit size are larger than the threshold value.
　判定部１５０は、判定結果に基づいて、スイッチ１５１及びスイッチ１５２を制御する。具体的には、判定部１５０は、第１変換処理が適用されると判定された場合に、第１変換処理が施された係数レベル値が出力されるようにスイッチ１５１を制御し、逆変換・逆量子化部１４２から出力される予測残差信号が出力されるようにスイッチ１５２を制御する。一方で、判定部１５０は、第２変換処理が適用されると判定された場合に、第２変換処理が施された係数レベル値が出力されるようにスイッチ１５１を制御し、拡大処理部１３２から出力される予測残差信号が出力されるようにスイッチ１５２を制御する。 The determination unit 150 controls the switch 151 and the switch 152 based on the determination result. Specifically, when it is determined that the first conversion process is applied, the determination unit 150 controls the switch 151 so that the coefficient level value subjected to the first conversion process is output, and performs inverse conversion. The switch 152 is controlled so that the prediction residual signal output from the inverse quantization unit 142 is output. On the other hand, when it is determined that the second conversion process is applied, the determination unit 150 controls the switch 151 so that the coefficient level value subjected to the second conversion process is output, and the enlargement processing unit 132. The switch 152 is controlled so that the prediction residual signal output from is output.
　符号化部１６０は、変換・量子化部１４１から出力された係数レベル値を符号化し、符号化データを出力する。例えば、符号化は、係数レベル値の発生確率に基づいて異なる長さの符号を割り当てるエントロピー符号化である。 The encoding unit 160 encodes the coefficient level value output from the transform / quantization unit 141 and outputs encoded data. For example, encoding is entropy encoding in which codes having different lengths are assigned based on the occurrence probability of coefficient level values.
　符号化部１６０は、係数レベル値に加えて、復号処理で用いる制御データを符号化する。制御データは、符号化ユニットサイズ、予測ユニットサイズ、変換ユニットサイズなどのサイズデータを含んでもよい。制御データは、予測残差信号の生成方法や変換処理で用いる基底パターンの種類などを示すデータ（フラグを含む）を含んでもよい。 The encoding unit 160 encodes control data used in the decoding process in addition to the coefficient level value. The control data may include size data such as an encoding unit size, a prediction unit size, and a transform unit size. The control data may include data (including a flag) indicating a prediction residual signal generation method, a base pattern type used in conversion processing, and the like.
　インループフィルタ１７０は、加算器１２２から出力されるフィルタ前復号信号に対してフィルタ処理を行うとともに、フィルタ後復号信号をフレームバッファ１８０に出力する。例えば、フィルタ処理は、ブロック（予測ユニット又は変換ユニット）の境界部分で生じる歪みを減少するデブロッキングフィルタ処理である。 The in-loop filter 170 performs a filtering process on the pre-filter decoded signal output from the adder 122 and outputs the post-filter decoded signal to the frame buffer 180. For example, the filtering process is a deblocking filtering process that reduces distortion generated at a boundary portion of a block (a prediction unit or a transform unit).
　フレームバッファ１８０は、インター予測部１１１で用いる参照フレームを蓄積する。フィルタ後復号信号は、インター予測部１１１で用いる参照フレームを構成する。 The frame buffer 180 stores reference frames used in the inter prediction unit 111. The post-filtered decoded signal constitutes a reference frame used in the inter prediction unit 111.
　（画像復号装置）
　以下において、実施形態に係る画像復号装置について説明する。図３は、実施形態に係る画像復号装置２００を示す図である。 (Image decoding device)
Hereinafter, an image decoding apparatus according to the embodiment will be described. FIG. 3 is a diagram illustrating the image decoding device 200 according to the embodiment.
　図３に示すように、画像復号装置２００は、復号部２１０と、逆変換・逆量子化部２２０と、判定部２３０と、スイッチ２３１と、拡大処理部２４０と、加算器２５０と、インター予測部２６１と、イントラ予測部２６２と、インループフィルタ２７０と、フレームバッファ２８０とを有する。 As illustrated in FIG. 3, the image decoding device 200 includes a decoding unit 210, an inverse transform / inverse quantization unit 220, a determination unit 230, a switch 231, an expansion processing unit 240, an adder 250, and inter prediction. Unit 261, intra prediction unit 262, in-loop filter 270, and frame buffer 280.
　復号部２１０は、画像符号化装置１００によって生成される符号化データを復号し、係数レベル値を復号する。例えば、復号は、符号化部１６０で行われるエントロピー符号化とは逆の手順のエントロピー復号である。 The decoding unit 210 decodes the encoded data generated by the image encoding device 100 and decodes the coefficient level value. For example, the decoding is entropy decoding in a procedure opposite to the entropy encoding performed by the encoding unit 160.
　復号部２１０は、符号化データの復号処理によって制御データを取得してもよい。上述したように、制御データは、符号化ユニットサイズ、予測ユニットサイズ、変換ユニットサイズなどのサイズデータを含んでもよい。制御データは、予測残差信号の生成方法や逆変換処理で用いる基底パターンの種類などを示すデータ（フラグを含む）を含んでもよい。 The decoding unit 210 may acquire control data by decoding the encoded data. As described above, the control data may include size data such as an encoding unit size, a prediction unit size, and a transform unit size. The control data may include data (including a flag) indicating a prediction residual signal generation method, a base pattern type used in the inverse transformation process, and the like.
　逆変換・逆量子化部２２０は、復号部２１０から出力される係数レベル値の逆変換処理を行う。ここで、逆変換・逆量子化部２２０は、逆変換処理に先立って、係数レベル値の逆量子化を行ってもよい。逆変換処理及び逆量子化は、変換・量子化部１４１で行われる変換処理及び量子化とは逆の手順で行われる。 The inverse transform / inverse quantization unit 220 performs an inverse transform process on the coefficient level value output from the decoding unit 210. Here, the inverse transform / inverse quantization unit 220 may perform inverse quantization of the coefficient level value prior to the inverse transform process. The inverse transformation process and the inverse quantization are performed in the reverse procedure of the transformation process and the quantization performed by the transformation / quantization unit 141.
　判定部２３０は、判定部１５０と同様の手順で、第１逆変換処理を適用すべきか、第２逆変換処理を適用すべきかを判定する。具体的には、判定部２３０は、変換ユニットサイズが閾値以下である場合に、第１逆変換処理を適用すると判定する。一方で、判定部２３０は、変換ユニットサイズが閾値よりも大きい場合に、第２逆変換処理を適用すると判定する。 The determination unit 230 determines whether to apply the first inverse transformation process or the second inverse transformation process in the same procedure as the determination unit 150. Specifically, the determination unit 230 determines to apply the first inverse conversion process when the conversion unit size is equal to or smaller than the threshold value. On the other hand, the determination unit 230 determines to apply the second inverse conversion process when the conversion unit size is larger than the threshold value.
　判定部２３０は、判定結果に基づいて、スイッチ２３１を制御する。具体的には、判定部２３０は、第１逆変換処理が適用されると判定された場合に、逆変換・逆量子化部２２０から出力される予測残差信号が出力されるようにスイッチ２３１を制御する。一方で、判定部２３０は、第２逆変換処理が適用されると判定された場合に、拡大処理部２４０から出力される予測残差信号が出力されるようにスイッチ２３１を制御する。 The determination unit 230 controls the switch 231 based on the determination result. Specifically, the determination unit 230 switches the switch 231 so that the prediction residual signal output from the inverse transform / inverse quantization unit 220 is output when it is determined that the first inverse transform process is applied. To control. On the other hand, the determination unit 230 controls the switch 231 so that the prediction residual signal output from the enlargement processing unit 240 is output when it is determined that the second inverse transform process is applied.
　拡大処理部２４０は、拡大処理部１３２と同様に、予測残差信号に対応する変換サイズを変換ユニットサイズに揃える拡大処理を行う。拡大処理は、変換ユニットサイズが閾値よりも大きいケースに適用される処理である。拡大処理の詳細については後述する（図４を参照）。 The enlargement processing unit 240 performs the enlargement processing for aligning the conversion size corresponding to the prediction residual signal to the conversion unit size, similarly to the expansion processing unit 132. The enlargement process is a process applied to a case where the conversion unit size is larger than a threshold value. Details of the enlargement process will be described later (see FIG. 4).
　加算器２５０は、逆変換・逆量子化部２２０又は拡大処理部２４０から出力される予測残差信号に予測信号を加算し、フィルタ前復号信号をイントラ予測部２６２及びインループフィルタ２７０に出力する。フィルタ前復号信号は、イントラ予測部２６２で用いる参照ユニットを構成する。予測残差信号のソースは、スイッチ２３１によって切り替えられる。 The adder 250 adds the prediction signal to the prediction residual signal output from the inverse transform / inverse quantization unit 220 or the expansion processing unit 240, and outputs the pre-filter decoded signal to the intra prediction unit 262 and the in-loop filter 270. . The pre-filter decoded signal constitutes a reference unit used in the intra prediction unit 262. The source of the prediction residual signal is switched by the switch 231.
　インター予測部２６１は、インター予測部１１１と同様に、インター予測（フレーム間予測）によって予測信号を生成する。具体的には、インター予測部２６１は、符号化データから復号した動きベクトル及び参照フレームに基づいて予測信号を予測ユニット毎に生成する。インター予測部２６１は、予測信号を加算器２５０に出力する。 The inter prediction unit 261 generates a prediction signal by inter prediction (interframe prediction) in the same manner as the inter prediction unit 111. Specifically, the inter prediction unit 261 generates a prediction signal for each prediction unit based on the motion vector and the reference frame decoded from the encoded data. The inter prediction unit 261 outputs the prediction signal to the adder 250.
　イントラ予測部２６２は、イントラ予測部１１２と同様に、イントラ予測（フレーム内予測）によって予測信号を生成する。具体的には、イントラ予測部２６２は、対象フレームに含まれる参照ユニットを特定し、特定された参照ユニットに基づいて予測信号を予測ユニット毎に生成する。イントラ予測部２６２は、予測信号を加算器２５０に出力する。 In the same manner as the intra prediction unit 112, the intra prediction unit 262 generates a prediction signal by intra prediction (intraframe prediction). Specifically, the intra prediction unit 262 specifies a reference unit included in the target frame, and generates a prediction signal for each prediction unit based on the specified reference unit. The intra prediction unit 262 outputs the prediction signal to the adder 250.
　インループフィルタ２７０は、インループフィルタ１７０と同様に、加算器２５０から出力されるフィルタ前復号信号に対してフィルタ処理を行うとともに、フィルタ後復号信号をフレームバッファ２８０に出力する。例えば、フィルタ処理は、ブロック（予測ユニット又は変換ユニット）の境界部分で生じる歪みを減少するデブロッキングフィルタ処理である。 Similarly to the in-loop filter 170, the in-loop filter 270 performs a filtering process on the pre-filter decoded signal output from the adder 250 and outputs the post-filter decoded signal to the frame buffer 280. For example, the filtering process is a deblocking filtering process that reduces distortion generated at a boundary portion of a block (a prediction unit or a transform unit).
　フレームバッファ２８０は、フレームバッファ１８０と同様に、インター予測部２６１で用いる参照フレームを蓄積する。フィルタ後復号信号は、インター予測部２６１で用いる参照フレームを構成する。 The frame buffer 280 stores the reference frame used in the inter prediction unit 261, similarly to the frame buffer 180. The post-filter decoded signal constitutes a reference frame used in the inter prediction unit 261.
　（縮小処理及び拡大処理）
　以下において、実施形態に係る縮小処理及び拡大処理について説明する。図４は、実施形態に係る縮小処理及び拡大処理を説明するための図である。上述したように、縮小処理及び拡大処理は、第２変換処理及び第２逆変換処理が適用される場合に行われる処理である。図４では、変換ユニットサイズは、Ｎ（水平画素数）×Ｍ（垂直画素数）で定義されるケースを例示する。 (Reduction processing and enlargement processing)
Hereinafter, the reduction process and the enlargement process according to the embodiment will be described. FIG. 4 is a diagram for explaining the reduction process and the enlargement process according to the embodiment. As described above, the reduction process and the enlargement process are processes performed when the second conversion process and the second inverse conversion process are applied. FIG. 4 illustrates a case where the conversion unit size is defined by N (number of horizontal pixels) × M (number of vertical pixels).
　第１に、図４に示すように、縮小処理は、水平方向において、変換ユニットサイズＮを変換サイズＮ／ｘに縮小する処理であってもよい。拡大処理は、水平方向において、変換サイズＮ／ｘを変換ユニットサイズＮに拡大する処理であってもよい。特に限定されるものではないが、ｘは２のべき乗であってもよい。すなわち、変換サイズは、変換ユニットサイズの水平画素数が変換サイズの水平画素数の２のべき乗で表されるように定められてもよい。 First, as shown in FIG. 4, the reduction process may be a process of reducing the conversion unit size N to the conversion size N / x in the horizontal direction. The enlargement process may be a process of enlarging the conversion size N / x to the conversion unit size N in the horizontal direction. Although not particularly limited, x may be a power of 2. That is, the conversion size may be determined so that the number of horizontal pixels of the conversion unit size is represented by a power of 2 of the number of horizontal pixels of the conversion size.
　第２に、図４に示すように、縮小処理は、垂直方向において、変換ユニットサイズＭを変換サイズＭ／ｙに縮小する処理であってもよい。拡大処理は、垂直方向において、変換サイズＭ／ｙを変換ユニットサイズＭに拡大する処理であってもよい。特に限定されるものではないが、ｙは２のべき乗であってもよい。すなわち、変換サイズは、変換ユニットサイズの垂直画素数が変換サイズの垂直画素数の２のべき乗で表されるように定められてもよい。 Second, as shown in FIG. 4, the reduction process may be a process of reducing the conversion unit size M to the conversion size M / y in the vertical direction. The enlargement process may be a process of enlarging the conversion size M / y to the conversion unit size M in the vertical direction. Although not particularly limited, y may be a power of 2. That is, the conversion size may be determined such that the number of vertical pixels of the conversion unit size is represented by a power of 2 of the number of vertical pixels of the conversion size.
　第３に、図４に示すように、縮小処理は、水平方向において、変換ユニットサイズＮを変換サイズＮ／ｘに縮小するとともに、垂直方向において、変換ユニットサイズＭを変換サイズＭ／ｙに縮小する処理であってもよい。拡大処理は、水平方向において、変換サイズＮ／ｘを変換ユニットサイズＮに拡大するとともに、垂直方向において、変換サイズＭ／ｙを変換ユニットサイズＭに拡大する処理であってもよい。特に限定されるものではないが、ｘ及びｙは２のべき乗であってもよい。ｘは、ｙと同じ値であってもよく、ｙと異なる値であってもよい。すなわち、変換サイズは、変換ユニットサイズの水平画素数が変換サイズの水平画素数の２のべき乗で表され、かつ、変換ユニットサイズの垂直画素数が変換サイズの垂直画素数の２のべき乗で表されるように定められてもよい。 Thirdly, as shown in FIG. 4, the reduction process reduces the conversion unit size N to the conversion size N / x in the horizontal direction and reduces the conversion unit size M to the conversion size M / y in the vertical direction. It may be a process to do. The enlargement process may be a process of enlarging the conversion size N / x to the conversion unit size N in the horizontal direction and enlarging the conversion size M / y to the conversion unit size M in the vertical direction. Although not particularly limited, x and y may be powers of 2. x may be the same value as y or a value different from y. That is, the conversion size is expressed by the number of horizontal pixels of the conversion unit size being a power of 2 of the number of horizontal pixels of the conversion size, and the number of vertical pixels of the conversion unit size being expressed by the power of 2 of the number of vertical pixels of the conversion size. It may be determined to be.
　縮小処理は、画素を間引く処理であってもよい。拡大処理は、画素を補間する処理であってもよい。画素の補間は、１つの隣接画素をコピーする処理であってもよく、２以上の隣接画素から加重平均などによって画素を生成する処理であってもよい。ｘ又はｙが２のべき乗であることによって、縮小処理及び拡大処理の負荷を軽減することができる。 The reduction process may be a process of thinning out pixels. The enlargement process may be a process of interpolating pixels. The pixel interpolation may be a process of copying one adjacent pixel, or a process of generating a pixel from two or more adjacent pixels by a weighted average or the like. When x or y is a power of 2, the load of the reduction process and the enlargement process can be reduced.
　（変換処理及び逆変換処理）
　以下において、実施形態に係る変換処理及び逆変換処理について説明する。図５及び図６は、実施形態に係る変換処理及び逆変換処理を説明するための図である。上述したように、変換処理は、第１変換処理及び第２変換処理を含み、逆変換処理は、第１逆変換処理及び第２逆変換処理を含む。 (Conversion processing and inverse conversion processing)
Hereinafter, the conversion process and the inverse conversion process according to the embodiment will be described. 5 and 6 are diagrams for explaining the conversion process and the inverse conversion process according to the embodiment. As described above, the conversion process includes a first conversion process and a second conversion process, and the inverse conversion process includes a first inverse conversion process and a second inverse conversion process.
　第１に、第１変換処理について図５を参照しながら説明する。図５では、変換ユニットサイズがＭ１×Ｎ１で表されるケースを例示する。第１変換処理は縮小処理を伴わない処理であるため、図５に示すように、予測誤差信号のサイズは変換ユニットサイズである。従って、変換ユニットサイズを有する変換行列（ＤＣＴ又はＤＳＴ）を用いて、予測残差信号が周波数成分に変換される。第１逆変換処理は、第１変換処理と逆の手順であるため、その説明については省略する。 First, the first conversion process will be described with reference to FIG. FIG. 5 illustrates a case where the conversion unit size is represented by M 1 × N 1 . Since the first conversion process is a process that does not involve a reduction process, the size of the prediction error signal is the conversion unit size, as shown in FIG. Therefore, the prediction residual signal is converted into a frequency component using a conversion matrix (DCT or DST) having a conversion unit size. Since the first inverse conversion process is the reverse of the first conversion process, the description thereof is omitted.
　第２に、第２変換処理について図６を参照しながら説明する。図５では、変換ユニットサイズがＭ２×Ｎ２で表されるケースを例示する。第２変換処理は縮小処理を伴う処理であるため、図６に示すように、予測誤差信号のサイズは変換サイズ（Ｍ２／ｘ×Ｎ２／ｙ）である。従って、変換サイズを有する変換行列（ＤＣＴ又はＤＳＴ）を用いて、予測残差信号が周波数成分に変換される。第２逆変換処理は、第２変換処理と逆の手順であるため、その説明については省略する。 Second, the second conversion process will be described with reference to FIG. FIG. 5 illustrates a case where the conversion unit size is represented by M 2 × N 2 . Since the second conversion process is a process involving a reduction process, the size of the prediction error signal is a conversion size (M 2 / x × N 2 / y) as shown in FIG. Therefore, the prediction residual signal is converted into a frequency component using a conversion matrix (DCT or DST) having a conversion size. Since the second inverse conversion process is the reverse of the second conversion process, the description thereof is omitted.
　ここで、変換ユニットサイズが変換サイズの２のべき乗である場合には、変換サイズを有する変換行列（ＤＣＴ又はＤＳＴ）として、変換ユニットサイズを有する変換行列（ＤＣＴ又はＤＳＴ）の一部を用いることが可能である。 Here, when the transform unit size is a power of 2 of the transform size, a part of the transform matrix (DCT or DST) having the transform unit size is used as the transform matrix (DCT or DST) having the transform size. Is possible.
　（作用及び効果）
　実施形態に係る画像復号装置２００では、変換ユニットサイズが閾値よりも大きい場合に、逆変換処理として、変換ユニットサイズよりも小さい変換サイズで第２逆変換処理が適用される。このような構成によれば、変換ユニットサイズが拡大した場合であっても、逆変換処理に必要な負荷の増大を抑制することができる。 (Function and effect)
In the image decoding apparatus 200 according to the embodiment, when the transform unit size is larger than the threshold, the second inverse transform process is applied as the inverse transform process with a transform size smaller than the transform unit size. According to such a configuration, even when the conversion unit size is increased, an increase in load necessary for the inverse conversion process can be suppressed.
　実施形態に係る画像符号化装置１００では、変換ユニットサイズが閾値よりも大きい場合に、変換処理として、変換ユニットサイズよりも小さい変換サイズで第２変換処理が適用される。このような構成によれば、変換ユニットサイズが拡大した場合であっても、変換処理に必要な負荷の増大を抑制することができる。 In the image encoding device 100 according to the embodiment, when the transform unit size is larger than the threshold, the second transform process is applied as a transform process with a transform size smaller than the transform unit size. According to such a configuration, even if the conversion unit size is increased, an increase in load necessary for the conversion process can be suppressed.
　実施形態では、変換サイズは、変換ユニットサイズの水平画素数又は／及び垂直画素数が変換サイズの水平画素数又は／及び垂直画素数の２のべき乗で表されるように定められてもよい。このような構成によれば、第２変換処理又は第２逆変換処理で用いる基底パターン（変換行列）を、第１変換処理又は第１逆変換処理で用いる基底パターン（変換行列）と共通化することができ、変換処理及び逆変換処理に必要な回路規模の増大を抑制することができる。 In the embodiment, the conversion size may be determined such that the number of horizontal pixels or / and the number of vertical pixels of the conversion unit size is represented by a power of 2 of the number of horizontal pixels or / and the number of vertical pixels of the conversion size. According to such a configuration, the base pattern (conversion matrix) used in the second conversion process or the second inverse conversion process is shared with the base pattern (conversion matrix) used in the first conversion process or the first inverse conversion process. Therefore, an increase in circuit scale necessary for the conversion process and the inverse conversion process can be suppressed.
　［変更例１］
　以下において、実施形態の変更例１について説明する。以下においては、実施形態に対する相違点について主として説明する。 [Modification 1]
Hereinafter, Modification Example 1 of the embodiment will be described. In the following, differences from the embodiment will be mainly described.
　変更例１において、画像符号化装置１００は、変換サイズを示すデータ（フラグを含む）を制御データとして符号化する。画像復号装置２００は、符号化データの復号処理によって変換サイズを取得する。このような構成によれば、符号化データの量が増大するものの、変換ユニットサイズと変換サイズとの関係を設定する自由度が増大する。 In the first modification, the image encoding device 100 encodes data (including a flag) indicating the conversion size as control data. The image decoding device 200 acquires the transform size by decoding the encoded data. According to such a configuration, although the amount of encoded data increases, the degree of freedom for setting the relationship between the conversion unit size and the conversion size increases.
　［変更例２］
　以下において、実施形態の変更例２について説明する。以下においては、実施形態に対する相違点について主として説明する。 [Modification 2]
Hereinafter, a second modification of the embodiment will be described. In the following, differences from the embodiment will be mainly described.
　変更例２において、変換サイズは、変換ユニットサイズと予め対応付けられている。画像復号装置２００は、変換ユニットサイズに基づいて変換サイズを特定する。例えば、変換ユニットサイズが１２８×１２８である場合に、変換サイズは６４×６４（又は、３２×３２）となるように予め定められていてもよい。上述したように、変換サイズは、変換ユニットサイズの水平画素数又は／及び垂直画素数が変換サイズの水平画素数又は／及び垂直画素数の２のべき乗で表されるように予め定められていてもよい。このような構成によれば、変換ユニットサイズと変換サイズとの関係を設定する自由度が減少するものの、符号化データの量の増大を抑制することができる。 In change example 2, the conversion size is associated with the conversion unit size in advance. The image decoding device 200 identifies the conversion size based on the conversion unit size. For example, when the conversion unit size is 128 × 128, the conversion size may be predetermined to be 64 × 64 (or 32 × 32). As described above, the conversion size is determined in advance such that the number of horizontal pixels or / and the number of vertical pixels of the conversion unit size is represented by the power of 2 of the number of horizontal pixels or / and the number of vertical pixels of the conversion size. Also good. According to such a configuration, although the degree of freedom for setting the relationship between the transform unit size and the transform size is reduced, an increase in the amount of encoded data can be suppressed.
　［変更例３］
　以下において、実施形態の変更例３について説明する。以下においては、実施形態に対する相違点について主として説明する。 [Modification 3]
Hereinafter, Modification Example 3 of the embodiment will be described. In the following, differences from the embodiment will be mainly described.
　変更例３において、変換ユニットサイズが閾値よりも大きい場合において、予め定められた係数レベル位置を有する係数レベル値がゼロと置換される。具体的には、係数レベル値は、図７に示すように、水平方向周波数及び垂直方向周波数によって定義される空間で定義される。このようなケースにおいて、予め定められた係数レベル位置は、水平方向周波数が第１周波数よりも高い位置であってもよく、垂直方向周波数が第２周波数よりも高い位置であってもよく、或いは、水平方向周波数が第１周波数よりも高く、かつ、垂直方向周波数の双方が第２周波数よりも高い位置であってもよい。第１周波数は、第２周波数と同じ値であってもよく、第２周波数と異なる値であってもよい。 In Modification 3, when the conversion unit size is larger than the threshold value, the coefficient level value having a predetermined coefficient level position is replaced with zero. Specifically, the coefficient level value is defined in a space defined by a horizontal frequency and a vertical frequency, as shown in FIG. In such a case, the predetermined coefficient level position may be a position where the horizontal frequency is higher than the first frequency, a position where the vertical frequency is higher than the second frequency, or The horizontal frequency may be higher than the first frequency, and both the vertical frequency may be higher than the second frequency. The first frequency may be the same value as the second frequency, or may be a value different from the second frequency.
　すなわち、画像符号化装置１００（符号化部１６０）は、変換ユニットサイズが閾値よりも大きい場合において、すなわち、第２変換処理が適用される場合において、予め定められた係数レベル位置（高周波成分）を有する係数レベル値をゼロとして取得する。同様に、画像復号装置２００（復号部２１０）は、変換ユニットサイズが閾値よりも大きい場合において、すなわち、第２逆変換処理が適用される場合において、予め定められた係数レベル位置（高周波成分）を有する係数レベル値をゼロとして取得する。 That is, the image encoding device 100 (encoding unit 160) determines a predetermined coefficient level position (high-frequency component) when the conversion unit size is larger than the threshold, that is, when the second conversion process is applied. The coefficient level value having is obtained as zero. Similarly, when the transform unit size is larger than the threshold, that is, when the second inverse transform process is applied, the image decoding device 200 (decoding unit 210) determines a predetermined coefficient level position (high frequency component). The coefficient level value having is obtained as zero.
　ここで、変換ユニットサイズが閾値よりも大きい場合には、変換ユニットに含まれる高周波成分が少ない可能性が高い。従って、変更例３に示す構成では、高周波成分をゼロに置換することによって、逆変換処理に伴う画像復号装置２００の負荷を軽減することができる。 Here, when the conversion unit size is larger than the threshold value, there is a high possibility that the high-frequency component contained in the conversion unit is small. Therefore, in the configuration shown in the modification example 3, the load on the image decoding apparatus 200 associated with the inverse transform process can be reduced by replacing the high frequency component with zero.
　さらに、画像符号化装置１００は、ゼロで取得される係数レベル値の符号化を省略してもよい。このような構成によれば、画像符号化装置１００から画像復号装置２００に符号化データが送信される場合に、符号化データのシグナリング量を低減することができる。 Furthermore, the image encoding device 100 may omit encoding of coefficient level values acquired with zero. According to such a configuration, when encoded data is transmitted from the image encoding apparatus 100 to the image decoding apparatus 200, the signaling amount of the encoded data can be reduced.
　［その他の実施形態］
　本発明は上述した実施形態によって説明したが、この開示の一部をなす論述及び図面は、この発明を限定するものであると理解すべきではない。この開示から当業者には様々な代替実施形態、実施例及び運用技術が明らかとなろう。 [Other Embodiments]
Although the present invention has been described with reference to the above-described embodiments, it should not be understood that the descriptions and drawings constituting a part of this disclosure limit the present invention. From this disclosure, various alternative embodiments, examples and operational techniques will be apparent to those skilled in the art.
　実施形態では、変換・量子化部１４１及び逆変換・逆量子化部１４２とは別に判定部１５０が設けられる。しかしながら、実施形態はこれに限定されるものではない。変換・量子化部１４１は、自ら第１変換処理を適用すべきか、第２変換処理を適用すべきかを判定してもよい。このようなケースにおいて、変換・量子化部１４１は、変換ユニットサイズが閾値以下である場合に、第１変換処理を行う。一方で、変換・量子化部１４１は、変換ユニットサイズが閾値よりも大きい場合に、第２変換処理を行う。同様に、逆変換・逆量子化部１４２は、自ら第１逆変換処理を適用すべきか、第２逆変換処理を適用すべきかを判定してもよい。具体的には、逆変換・逆量子化部１４２は、変換ユニットサイズが閾値以下である場合に、第１逆変換処理を行う。一方で、逆変換・逆量子化部１４２は、変換ユニットサイズが閾値よりも大きい場合に、第２逆変換処理を行う。 In the embodiment, a determination unit 150 is provided separately from the transform / quantization unit 141 and the inverse transform / inverse quantization unit 142. However, the embodiment is not limited to this. The transform / quantization unit 141 may determine whether to apply the first transform process or to apply the second transform process. In such a case, the transform / quantization unit 141 performs the first transform process when the transform unit size is equal to or smaller than the threshold value. On the other hand, the transform / quantization unit 141 performs the second transform process when the transform unit size is larger than the threshold value. Similarly, the inverse transform / inverse quantization unit 142 may determine whether to apply the first inverse transform process or the second inverse transform process. Specifically, the inverse transform / inverse quantization unit 142 performs the first inverse transform process when the transform unit size is equal to or smaller than the threshold value. On the other hand, the inverse transform / inverse quantization unit 142 performs the second inverse transform process when the transform unit size is larger than the threshold.
　実施形態では、逆変換・逆量子化部２２０とは別に判定部２３０が設けられる。しかしながら、実施形態はこれに限定されるものではない。逆変換・逆量子化部２２０は、自ら第１逆変換処理を適用すべきか、第２逆変換処理を適用すべきかを判定してもよい。具体的には、逆変換・逆量子化部２２０は、変換ユニットサイズが閾値以下である場合に、第１逆変換処理を行う。一方で、逆変換・逆量子化部２２０は、変換ユニットサイズが閾値よりも大きい場合に、第２逆変換処理を行う。 In the embodiment, a determination unit 230 is provided separately from the inverse transform / inverse quantization unit 220. However, the embodiment is not limited to this. The inverse transform / inverse quantization unit 220 may determine whether to apply the first inverse transform process or the second inverse transform process. Specifically, the inverse transform / inverse quantization unit 220 performs the first inverse transform process when the transform unit size is equal to or smaller than the threshold value. On the other hand, the inverse transform / inverse quantization unit 220 performs the second inverse transform process when the transform unit size is larger than the threshold.
　実施形態では、画像符号化装置１００（判定部１５０）は、所定条件が満たされていない場合に、第１変換処理を適用すると判定し、所定条件が満たされている場合に、第２変換処理を適用すると判定してもよい。所定条件は、変換ユニットサイズが閾値よりも大きい条件以外の条件を含んでもよい。例えば、所定条件は、特定のイントラ予測モードが適用されているという条件を含んでもよく、変換対象ブロックの動き情報（予測動きベクトルなど）が変換対象ブロックの隣接ブロックの動き情報と同じであるという条件を含んでもよい。 In the embodiment, the image encoding device 100 (determination unit 150) determines to apply the first conversion process when the predetermined condition is not satisfied, and performs the second conversion process when the predetermined condition is satisfied. May be determined to apply. The predetermined condition may include a condition other than the condition that the conversion unit size is larger than the threshold value. For example, the predetermined condition may include a condition that a specific intra prediction mode is applied, and the motion information (predicted motion vector or the like) of the conversion target block is the same as the motion information of an adjacent block of the conversion target block. Conditions may be included.
　実施形態において、画像復号装置２００（判定部２３０）は、所定条件が満たされていない場合に、第１逆変換処理を適用すると判定し、所定条件が満たされている場合に、第２逆変換処理を適用すると判定してもよい。所定条件は、変換ユニットサイズが閾値よりも大きい条件以外の条件を含んでもよい。例えば、所定条件は、特定のイントラ予測モードが適用されているという条件を含んでもよく、逆変換対象ブロックの動き情報（予測動きベクトルなど）が逆変換対象ブロックの隣接ブロックの動き情報と同じであるという条件を含んでもよい。 In the embodiment, the image decoding apparatus 200 (determination unit 230) determines to apply the first inverse transform process when the predetermined condition is not satisfied, and performs the second inverse transform when the predetermined condition is satisfied. You may determine with applying a process. The predetermined condition may include a condition other than the condition that the conversion unit size is larger than the threshold value. For example, the predetermined condition may include a condition that a specific intra prediction mode is applied, and motion information (predicted motion vector, etc.) of the inverse transform target block is the same as motion information of an adjacent block of the inverse transform target block. It may include the condition that it exists.
　実施形態では特に触れていないが、画像符号化装置１００は、第２逆変換処理を適用すべきか否かを示すデータを制御データとして生成してもよい。このようなケースにおいて、画像復号装置２００は、第２逆変換処理を適用すべきか否かを示すデータに基づいて、第２逆変換処理を適用するか否かを判定してもよい。例えば、第２逆変換処理を適用すべきか否かを示すデータは、第２逆変換処理を適用すべきである旨を示す“１”及び第２逆変換処理を適用すべきでない旨を示す“０”を取り得るフラグであってもよい。 Although not particularly mentioned in the embodiment, the image encoding device 100 may generate data indicating whether or not the second inverse transform process should be applied as control data. In such a case, the image decoding apparatus 200 may determine whether to apply the second inverse transformation process based on data indicating whether the second inverse transformation process should be applied. For example, the data indicating whether or not the second inverse transformation process should be applied is “1” indicating that the second inverse transformation process should be applied and “2” indicating that the second inverse transformation process should not be applied. It may be a flag that can take 0 ″.
　実施形態では特にふれていないが、第２変換処理及び第２逆変換処理は、変換ユニットサイズが閾値よりも大きい場合において、色差予測残差信号には適用されず、輝度予測残差信号に適用されてもよい。第２変換処理及び第２逆変換処理は、変換ユニットサイズが閾値よりも大きい場合において、輝度予測残差信号には適用されず、色差予測残差信号に適用されてもよい。第２変換処理及び第２逆変換処理は、変換ユニットサイズが閾値よりも大きい場合において、輝度予測残差信号及び色差予測残差信号の双方に適用されてもよい。 Although not particularly mentioned in the embodiment, the second conversion process and the second inverse conversion process are not applied to the color difference prediction residual signal but applied to the luminance prediction residual signal when the conversion unit size is larger than the threshold. May be. The second conversion process and the second inverse conversion process may be applied to the color difference prediction residual signal instead of being applied to the luminance prediction residual signal when the conversion unit size is larger than the threshold. The second conversion process and the second inverse conversion process may be applied to both the luminance prediction residual signal and the color difference prediction residual signal when the conversion unit size is larger than the threshold value.
　実施形態では特に触れていないが、画像符号化装置１００及び画像復号装置２００が行う各処理をコンピュータに実行させるプログラムが提供されてもよい。また、プログラムは、コンピュータ読取り可能媒体に記録されていてもよい。コンピュータ読取り可能媒体を用いれば、コンピュータにプログラムをインストールすることが可能である。ここで、プログラムが記録されたコンピュータ読取り可能媒体は、非一過性の記録媒体であってもよい。非一過性の記録媒体は、特に限定されるものではないが、例えば、ＣＤ－ＲＯＭやＤＶＤ－ＲＯＭ等の記録媒体であってもよい。 Although not particularly mentioned in the embodiment, a program for causing a computer to execute each process performed by the image encoding device 100 and the image decoding device 200 may be provided. The program may be recorded on a computer readable medium. If a computer-readable medium is used, a program can be installed in the computer. Here, the computer-readable medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, but may be a recording medium such as a CD-ROM or a DVD-ROM.
　或いは、画像符号化装置１００及び画像復号装置２００が行う各処理を実行するためのプログラムを記憶するメモリ及びメモリに記憶されたプログラムを実行するプロセッサによって構成されるチップが提供されてもよい。 Alternatively, a chip including a memory that stores a program for executing each process performed by the image encoding device 100 and the image decoding device 200 and a processor that executes the program stored in the memory may be provided.
　なお、日本国特許出願第２０１８－６５３４７号（２０１８年３月２９日出願）の全内容が、参照により、本願明細書に組み込まれている。 Note that the entire contents of Japanese Patent Application No. 2018-65347 (filed on Mar. 29, 2018) are incorporated herein by reference.
　一態様によれば、変換ユニットサイズが拡大した場合であっても、変換処理又は逆変換処理に必要な負荷の増大を抑制することを可能とする画像復号装置、画像符号化装置、画像処理システム、画像復号方法及びプログラムを提供することができる。 According to one aspect, even when the transform unit size is increased, an image decoding device, an image coding device, and an image processing system that can suppress an increase in load necessary for the transform process or the inverse transform process. An image decoding method and program can be provided.

Claims (10)

  1. 　画像符号化装置によって符号化された符号化データの復号処理を行うとともに、前記符号化装置によって変換処理が施された係数レベル値を取得する復号部と、
    　前記係数レベル値の逆変換処理を少なくとも行うとともに、予測残差信号を取得する逆変換処理部とを備え、
    　前記逆変換処理部は、前記逆変換処理として、前記変換ユニットサイズで第１逆変換処理と、前記変換ユニットサイズよりも小さい変換サイズで第２逆変換処理を行うように構成されており、
    　前記第１逆変換処理は、前記係数レベル値に対応する変換ユニットサイズが閾値以下であるケースに適用される処理であり、
    　前記第２逆変換処理は、前記変換ユニットサイズが前記閾値よりも大きいケースに適用される処理である、画像復号装置。     A decoding unit that performs decoding processing of encoded data encoded by the image encoding device, and obtains a coefficient level value that has been subjected to conversion processing by the encoding device;
    And at least performing an inverse transformation process of the coefficient level value, and an inverse transformation processing unit for obtaining a prediction residual signal,
    The inverse transform processing unit is configured to perform a first inverse transform process with the transform unit size and a second inverse transform process with a transform size smaller than the transform unit size as the inverse transform process.
    The first inverse transform process is a process applied to a case where a transform unit size corresponding to the coefficient level value is a threshold value or less,
    The second inverse transform process is an image decoding apparatus that is a process applied to a case where the transform unit size is larger than the threshold value.
  2. 　前記予測残差信号に対応する前記変換サイズを前記変換ユニットサイズに揃える拡大処理を行う拡大処理部を備え、
    　前記拡大処理は、前記変換ユニットサイズが前記閾値よりも大きいケースに適用される処理である、請求項１に記載の画像復号装置。     An enlargement processing unit that performs an enlargement process for aligning the transform size corresponding to the prediction residual signal to the transform unit size;
    The image decoding apparatus according to claim 1, wherein the enlargement process is a process applied to a case where the conversion unit size is larger than the threshold value.
  3. 　前記変換サイズは、前記変換ユニットサイズの水平画素数が前記変換サイズの水平画素数の２のべき乗で表されるように定められ、或いは、前記変換ユニットサイズの垂直画素数が前記変換サイズの垂直画素数の２のべき乗で表されるように定められ、或いは、前記変換ユニットサイズの水平画素数が前記変換サイズの水平画素数の２のべき乗で表され、かつ、前記変換ユニットサイズの垂直画素数が前記変換サイズの垂直画素数の２のべき乗で表されるように定められる、請求項１又は請求項２に記載の画像復号装置。 The transform size is determined such that the number of horizontal pixels of the transform unit size is represented by a power of 2 of the number of horizontal pixels of the transform size, or the number of vertical pixels of the transform unit size is vertical of the transform size The number of pixels is determined to be expressed by a power of 2, or the number of horizontal pixels of the conversion unit size is expressed by a power of 2 of the number of horizontal pixels of the conversion size, and the vertical pixel of the conversion unit size The image decoding device according to claim 1 or 2, wherein the number is determined to be expressed by a power of 2 of the number of vertical pixels of the transform size.
  4. 　前記復号部は、前記復号処理によって前記変換サイズを取得する、請求項１乃至請求項３のいずれかに記載の画像復号装置。 The image decoding device according to any one of claims 1 to 3, wherein the decoding unit acquires the transform size by the decoding process.
  5. 　前記変換サイズは、前記変換ユニットサイズと予め対応付けられている、請求項１乃至請求項３のいずれかに記載の画像復号装置。 4. The image decoding device according to claim 1, wherein the conversion size is associated with the conversion unit size in advance.
  6. 　前記復号部は、前記変換ユニットサイズが前記閾値よりも大きい場合において、予め定められた係数レベル位置を有する前記係数レベル値をゼロとして取得する、請求項１乃至請求項５のいずれかに記載の画像復号装置。 6. The decoding unit according to claim 1, wherein the decoding unit acquires the coefficient level value having a predetermined coefficient level position as zero when the transform unit size is larger than the threshold value. Image decoding device.
  7. 　イントラ予測又はインター予測によって生成される予測信号と入力画像信号との差分である予測残差信号を生成する生成部と、
    　前記予測残差信号の変換処理を少なくとも行うとともに、係数レベル値を取得する変換処理部とを備え、
    　前記変換処理部は、前記変換処理として、前記変換ユニットサイズで第１変換処理を行い、前記変換ユニットサイズよりも小さい変換サイズで第２変換処理を行うように構成されており、
    　前記第１変換処理は、前記係数レベル値に対応する変換ユニットサイズが閾値以下であるケースに適用される処理であり、
    　前記第２変換処理は、前記変換ユニットサイズが前記閾値よりも大きいケースに適用される処理である、画像符号化装置。     A generation unit that generates a prediction residual signal that is a difference between a prediction signal generated by intra prediction or inter prediction and an input image signal;
    A conversion processing unit that performs at least conversion processing of the prediction residual signal and obtains a coefficient level value;
    The conversion processing unit is configured to perform a first conversion process with the conversion unit size and a second conversion process with a conversion size smaller than the conversion unit size as the conversion process,
    The first conversion process is a process applied to a case where a conversion unit size corresponding to the coefficient level value is equal to or smaller than a threshold value.
    The image conversion apparatus, wherein the second conversion process is a process applied to a case where the conversion unit size is larger than the threshold value.
  8. 　画像符号化装置及び画像復号装置を備える画像処理システムであって、
    　前記画像符号化装置は、
    　　イントラ予測又はインター予測によって生成される予測信号と入力画像信号との差分である予測残差信号を生成する生成部と、
    　　前記予測残差信号の変換処理を行うとともに、係数レベル値を取得する変換処理部とを備え、
    　前記画像復号装置は、
    　　前記画像符号化装置によって符号化された符号化データの復号処理を行うとともに、前記係数レベル値を取得する復号部と、
    　　前記係数レベル値の逆変換処理を少なくとも行うとともに、前記予測残差信号を取得する逆変換処理部とを備え、
    　前記変換処理部は、前記変換処理として、前記変換ユニットサイズで第１変換処理を行い、前記変換ユニットサイズよりも小さい変換サイズで第２変換処理を行うように構成されており、
    　前記逆変換処理部は、前記逆変換処理として、前記変換ユニットサイズで第１逆変換処理と、前記変換ユニットサイズよりも小さい変換サイズで第２逆変換処理を行うように構成されており、
    　前記第１変換処理及び前記第１逆変換処理は、前記係数レベル値に対応する変換ユニットサイズが閾値以下であるケースに適用される処理であり、
    　前記第２変換処理及び前記第２逆変換処理は、前記変換ユニットサイズが前記閾値よりも大きいケースに適用される処理である、画像処理システム。     An image processing system comprising an image encoding device and an image decoding device,
    The image encoding device includes:
    A generation unit that generates a prediction residual signal that is a difference between a prediction signal generated by intra prediction or inter prediction and an input image signal;
    A conversion processing unit that performs conversion processing of the prediction residual signal and obtains a coefficient level value;
    The image decoding device includes:
    A decoding unit that performs decoding processing of the encoded data encoded by the image encoding device and acquires the coefficient level value;
    And at least performing an inverse transformation process of the coefficient level value, and comprising an inverse transformation processing unit for obtaining the prediction residual signal,
    The conversion processing unit is configured to perform a first conversion process with the conversion unit size and a second conversion process with a conversion size smaller than the conversion unit size as the conversion process,
    The inverse transform processing unit is configured to perform a first inverse transform process with the transform unit size and a second inverse transform process with a transform size smaller than the transform unit size as the inverse transform process.
    The first conversion process and the first inverse conversion process are processes applied to a case where a conversion unit size corresponding to the coefficient level value is equal to or smaller than a threshold value.
    The image processing system, wherein the second conversion process and the second inverse conversion process are processes applied to a case where the conversion unit size is larger than the threshold value.
  9. 　画像符号化装置によって符号化された符号化データの復号処理を行うとともに、前記符号化装置によって変換処理が施された係数レベル値を取得するステップＡと、
    　前記係数レベル値の逆変換処理を少なくとも行うとともに、予測残差信号を取得するステップＢとを備え、
    　前記ステップＢは、前記逆変換処理として、前記変換ユニットサイズで第１逆変換処理と、前記変換ユニットサイズよりも小さい変換サイズで第２逆変換処理を行うステップを含み、
    　前記第１逆変換処理は、前記係数レベル値に対応する変換ユニットサイズが閾値以下であるケースに適用される処理であり、
    　前記第２逆変換処理は、前記変換ユニットサイズが前記閾値よりも大きいケースに適用される処理である、画像復号方法。     Step A of performing decoding processing of the encoded data encoded by the image encoding device and acquiring a coefficient level value subjected to conversion processing by the encoding device;
    And performing at least the inverse conversion processing of the coefficient level value, and obtaining a prediction residual signal,
    The step B includes a step of performing a first inverse transform process with the transform unit size and a second inverse transform process with a transform size smaller than the transform unit size as the inverse transform process,
    The first inverse transform process is a process applied to a case where a transform unit size corresponding to the coefficient level value is a threshold value or less,
    The image decoding method, wherein the second inverse transform process is a process applied to a case where the transform unit size is larger than the threshold value.
  10. 　画像復号装置で用いるプログラムであって、
    　画像符号化装置によって符号化された符号化データの復号処理を行うとともに、前記符号化装置によって変換処理が施された係数レベル値を取得するステップＡと、
    　前記係数レベル値の逆変換処理を少なくとも行うとともに、予測残差信号を取得するステップＢとを実行させ、
    　前記ステップＢは、前記逆変換処理として、前記変換ユニットサイズで第１逆変換処理と、前記変換ユニットサイズよりも小さい変換サイズで第２逆変換処理を行うステップを含み、
    　前記第１逆変換処理は、前記係数レベル値に対応する変換ユニットサイズが閾値以下であるケースに適用される処理であり、
    　前記第２逆変換処理は、前記変換ユニットサイズが前記閾値よりも大きいケースに適用される処理である、プログラム。     A program used in an image decoding device,
    Step A of performing decoding processing of the encoded data encoded by the image encoding device and acquiring a coefficient level value subjected to conversion processing by the encoding device;
    Performing at least the inverse conversion process of the coefficient level value, and performing a step B of obtaining a prediction residual signal,
    The step B includes a step of performing a first inverse transform process with the transform unit size and a second inverse transform process with a transform size smaller than the transform unit size as the inverse transform process,
    The first inverse transform process is a process applied to a case where a transform unit size corresponding to the coefficient level value is a threshold value or less,
    The second inverse transform process is a program applied to a case where the transform unit size is larger than the threshold value.
PCT/JP2018/046410 2018-03-29 2018-12-17 Image decoding device, image coding device, image processing system, image decoding method, and program WO2019187396A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2018065347A JP7109961B2 (en) 2018-03-29 2018-03-29 Image decoding device, image encoding device, image processing system, image decoding method and program
JP2018-065347 2018-03-29

Publications (1)

Publication Number Publication Date
WO2019187396A1 true WO2019187396A1 (en) 2019-10-03

Family

ID=68058667

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2018/046410 WO2019187396A1 (en) 2018-03-29 2018-12-17 Image decoding device, image coding device, image processing system, image decoding method, and program

Country Status (2)

Country Link
JP (1) JP7109961B2 (en)
WO (1) WO2019187396A1 (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0284895A (en) * 1988-09-20 1990-03-26 Nippon Telegr & Teleph Corp <Ntt> Picture encoding system
US20110134999A1 (en) * 2009-12-09 2011-06-09 Samsung Electronics Co., Ltd. Method and apparatus for encoding video, and method and apparatus for decoding video
JP2014532377A (en) * 2011-10-14 2014-12-04 アドバンスト・マイクロ・ディバイシズ・インコーポレイテッドＡｄｖａｎｃｅｄ Ｍｉｃｒｏ Ｄｅｖｉｃｅｓ Ｉｎｃｏｒｐｏｒａｔｅｄ Region-based image compression
WO2016203981A1 (en) * 2015-06-16 2016-12-22 シャープ株式会社 Image decoding device and image encoding device

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9432696B2 (en) 2014-03-17 2016-08-30 Qualcomm Incorporated Systems and methods for low complexity forward transforms using zeroed-out coefficients

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0284895A (en) * 1988-09-20 1990-03-26 Nippon Telegr & Teleph Corp <Ntt> Picture encoding system
US20110134999A1 (en) * 2009-12-09 2011-06-09 Samsung Electronics Co., Ltd. Method and apparatus for encoding video, and method and apparatus for decoding video
JP2014532377A (en) * 2011-10-14 2014-12-04 アドバンスト・マイクロ・ディバイシズ・インコーポレイテッドＡｄｖａｎｃｅｄ Ｍｉｃｒｏ Ｄｅｖｉｃｅｓ Ｉｎｃｏｒｐｏｒａｔｅｄ Region-based image compression
WO2016203981A1 (en) * 2015-06-16 2016-12-22 シャープ株式会社 Image decoding device and image encoding device

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
"SERIES H: AUDIOVISUAL AND MULTIMEDIA SYSTEMS Coding of moving video", ITU-T, IMPLEMENTORS’ GUIDE FOR H.263: VIDEO CODING FOR LOW BIT RATE COMMUNICATION, 24 June 2005 (2005-06-24), pages 130 - 142, XP055227901 *
ALEXIS MICHAEL TOURAPIS: "Reduced Resolution Update Mode for Enhanced Compression", JOINT COLLABORATIVE TEAM ON VIDEO CODING (JCT-VC) OF ITU-T SG 16 WP3, 20 January 2012 (2012-01-20), San Jose, CA , USA, pages 1 - 4, XP030111474 *
KEI KAWAMURA ET AL.: "Description of SDR video coding technology proposal by KDDI", JOINT VIDEO EXPERTS TEAM (JVET) OF ITU-T, SG16 WP3, 10 April 2018 (2018-04-10), San Diego, US, pages i - iii, 1-23, XP030151176 *

Also Published As

Publication number Publication date
JP2019176421A (en) 2019-10-10
JP7109961B2 (en) 2022-08-01

Similar Documents

Publication Publication Date Title
JP6660074B2 (en) Video decoding method and apparatus
KR102140331B1 (en) Method for intra prediction and apparatus thereof
JP2006114979A (en) Image coding apparatus and method thereof
US20220030250A1 (en) Encoding device, decoding device, and program
WO2020184348A1 (en) Image decoding device, image decoding method, and program
US11871021B2 (en) Moving picture decoding device, moving picture decoding method, and program obtaining chrominance values from corresponding luminance values
JP2007013298A (en) Image coding apparatus
JP5213964B2 (en) Video encoding apparatus and video decoding apparatus
JP7229413B2 (en) Image encoding device, image decoding device, and program
JP2006270851A (en) Image coding device and image decoding device
WO2019187396A1 (en) Image decoding device, image coding device, image processing system, image decoding method, and program
JP6590412B2 (en) Video encoding apparatus, method and program, and video decoding apparatus, method and program
EP3993415A1 (en) Encoding device, decoding device, and program
WO2020059341A1 (en) Image decoding device, image encoding device, image processing system, and program
KR20130049736A (en) Method and apparatus for inter prediction
JP2010283739A (en) Moving image encoding device
JP2020053725A (en) Predictive image correction device, image encoding device, image decoding device, and program
JP7034363B2 (en) Image decoder, image decoding method and program
US20230007311A1 (en) Image encoding device, image encoding method and storage medium, image decoding device, and image decoding method and storage medium
JP7083971B1 (en) Image decoder, image decoding method and program
JP7444599B2 (en) Intra prediction device, image encoding device, image decoding device, and program
WO2020184262A1 (en) Image decoding device, image decoding method, and program
JP2012124851A (en) Motion compensation encoder, motion compensation encoding method and program
WO2020044985A1 (en) Image decoding device, image encoding device, image processing system, and program
JP6480790B2 (en) Image determination apparatus, encoding apparatus, and program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18913016

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18913016

Country of ref document: EP

Kind code of ref document: A1