WO2011074357A1 - 画像フィルタ、符号化装置、復号装置、および、データ構造 - Google Patents
画像フィルタ、符号化装置、復号装置、および、データ構造 Download PDFInfo
- Publication number
- WO2011074357A1 WO2011074357A1 PCT/JP2010/070053 JP2010070053W WO2011074357A1 WO 2011074357 A1 WO2011074357 A1 WO 2011074357A1 JP 2010070053 W JP2010070053 W JP 2010070053W WO 2011074357 A1 WO2011074357 A1 WO 2011074357A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- filter
- image
- coefficient
- vector
- filter coefficient
- Prior art date
Links
- 239000013598 vector Substances 0.000 claims abstract description 378
- 230000000694 effects Effects 0.000 claims description 150
- 230000003044 adaptive effect Effects 0.000 abstract description 146
- 238000001914 filtration Methods 0.000 description 173
- 238000000034 method Methods 0.000 description 73
- 238000004364 calculation method Methods 0.000 description 56
- 230000008569 process Effects 0.000 description 51
- 230000008859 change Effects 0.000 description 37
- 238000013139 quantization Methods 0.000 description 27
- 238000007619 statistical method Methods 0.000 description 23
- 230000006870 function Effects 0.000 description 22
- 238000010586 diagram Methods 0.000 description 16
- 230000002457 bidirectional effect Effects 0.000 description 6
- 238000009795 derivation Methods 0.000 description 5
- 230000000630 rising effect Effects 0.000 description 5
- 239000011159 matrix material Substances 0.000 description 4
- 230000009467 reduction Effects 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 3
- 230000007423 decrease Effects 0.000 description 3
- 230000004069 differentiation Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- DNTFEAHNXKUSKQ-RFZPGFLSSA-N (1r,2r)-2-aminocyclopentane-1-sulfonic acid Chemical compound N[C@@H]1CCC[C@H]1S(O)(=O)=O DNTFEAHNXKUSKQ-RFZPGFLSSA-N 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000005562 fading Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/117—Filters, e.g. for pre-processing or post-processing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
- H04N19/82—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
- H04N19/86—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving reduction of coding artifacts, e.g. of blockiness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20004—Adaptive image processing
- G06T2207/20012—Locally adaptive
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20172—Image enhancement details
- G06T2207/20192—Edge enhancement; Edge preservation
Definitions
- the present invention relates to an image filter that performs image filtering.
- the present invention also relates to an encoding device and a decoding device provided with such an image filter.
- the present invention also relates to a data structure of encoded data generated by such an encoding device and referenced by the decoding device.
- a moving image encoding device In order to efficiently transmit or record moving images, a moving image encoding device is used.
- a specific moving picture encoding method for example, H.264 is used.
- H.264 / MPEG-4 AVC and KTA software for example, H.264 / MPEG-4 AVC and KTA software.
- an image (picture) constituting a moving image is obtained by dividing a slice obtained by dividing an image, a macroblock obtained by dividing the slice, and a macroblock. It is managed by a hierarchical structure consisting of blocks to be encoded, and is usually encoded for each block.
- Non-Patent Document 1 discloses a filter called ALF (Adaptive Loop Filter) that performs filtering on a decoded image.
- ALF Adaptive Loop Filter
- the ALF uses a slice on the decoded image as a unit region, and for each unit region, determines a filter coefficient that minimizes an error between the decoded image to which the ALF is applied and the original image. Filtering based on a coefficient is performed.
- the present invention has been made in view of the above problems, and the object thereof is to use a filter coefficient determined for each unit region, and more appropriately, even when image characteristics are not uniform within the unit region.
- An object is to realize an image filter capable of performing filtering.
- an image filter is an image filter that calculates a pixel value of a target pixel in an output image from a pixel value of each pixel belonging to the target region in the input image and a filter coefficient vector.
- a filter coefficient vector changing unit that changes the filter coefficient vector in accordance with at least one of a position of the target region in the input image and a position of the target pixel in the output image. It is a feature.
- filter coefficient vector changing means for changing the filter coefficient vector according to at least one of the position of the target region in the input image and the position of the target pixel in the output image. Therefore, there is an effect that appropriate filtering can be performed for each target area according to at least one of the position of the target area and the position of the target pixel in the output image.
- the image filter according to the present invention is an image filter that calculates the pixel value of the target pixel in the output image from the pixel value of each pixel belonging to the target region in the input image and the filter coefficient vector.
- Filter coefficient vector changing means for changing the filter coefficient vector according to at least one of the position of the target region in the input image and the position of the target pixel in the output image is provided. Therefore, appropriate filtering can be performed for each target region according to at least one of the position of the target region and the position of the target pixel in the output image.
- (A) shows a first example of filter coefficient information
- (b) shows a second example of filter coefficient information
- (c) shows a third example of filter coefficient information.
- the inter prediction image generation part provided with the adaptive filter which concerns on embodiment.
- the inter prediction image generation part provided with the adaptive filter which concerns on embodiment.
- the inter prediction image generation part provided with the adaptive filter which concerns on embodiment.
- the inter prediction image generation part provided with the adaptive filter which concerns on embodiment.
- FIG. 1 is a block diagram showing the configuration of the adaptive filter 100. As shown in FIG. 1, the adaptive filter 100 includes a filter unit 110 and a control variable determination unit 120.
- the adaptive filter 100 is an image filter that can be used for filtering image data.
- the adaptive filter 100 is adapted for each region (filter target region T) obtained by dividing the unit region when filtering is performed on a unit region on the image indicated by the image data and which is a filtering target. Filtering can be performed automatically.
- the adaptive filter 100 can be suitably used for, for example, a moving image encoding device that encodes moving image data, or a moving image decoding device that decodes encoded moving image data.
- the adaptive filter 100 generates the output image data # 110a by taking the weighted linear sum based on the filter coefficient of the pixel values included in a certain region for each pixel value of the input image data # 100, Output.
- the teacher data # 200 is input to the adaptive filter 100.
- the teacher data # 200 is image data indicating a teacher image referred to when determining the filter coefficient, as will be described later.
- input image data input to an image encoding device including the adaptive filter 100 can be cited.
- the adaptive filter 100 outputs filter coefficient information # 110b, which is information about the filter coefficient used for filtering, together with the output image data # 110a.
- the adaptive filter 100 takes the weighted linear sum while changing the filter coefficient for each of the certain regions based on the image data # 100 and the side information # 100a of the image data # 100.
- the side information # 100a of the image data # 100 refers to information obtained from encoded data necessary to generate the image data # 100. That is, the transform coefficient necessary for generating the prediction residual, the frequency transform size, the quantization parameter information, the prediction mode, the motion compensation size, the motion vector, the reference frame, etc. necessary for generating the predicted image. All of the information, the weight information of weighted prediction, the information regarding the position of the block and the block boundary, and the like.
- the filter coefficient information # 110b is information including a filter coefficient and information necessary for generating the filter coefficient. Specific examples will be described later.
- region T is made into the rectangle of MTxNT here, the filter object area
- the shape of the filter target region T is not limited to a rectangle, but may be a rhombus, a circle, a vertical / horizontal line segment, an oblique line segment, or any other shape.
- the size of the filter target region T does not need to be fixed in the unit region and may be variable. For example, when a variable block size is adopted for conversion or motion compensation, it is appropriate to set the variable block as the region T.
- the control variable determination unit 120 receives the image data # 100 and the side information # 100a of the image data # 100, and supplies the control variable # 120a and the control base # 120b to the filter unit 110.
- the filter unit 110 takes the output image data # 110a by adding a weighted linear sum based on the filter coefficient and an offset value of the pixel values included in a certain region. calculate.
- the filter unit 110 calculates the pixel value SO (x ′, y ′) at the coordinates (x ′, y ′) of the output image data # 110a by a weighted linear sum represented by the equation (1). To do.
- SI (x, y) represents a pixel value at the coordinates (x, y) of the image data # 100
- h (i, j) is a filter multiplied by the pixel value SI (x + i, y + j). It represents a coefficient.
- R represents an area (hereinafter referred to as a filter reference area) constituted by pixels that are the objects of the weighted linear sum. More specifically, R represents a set of relative coordinates of the pixels to be subjected to the weighted linear sum.
- Hoffset represents an offset value to be added to the pixel value.
- the coordinates (x ′, y ′) and the coordinates (x, y) may be the same coordinates or different coordinates as long as they have a one-to-one correspondence. Further, the specific correspondence between the coordinates (x ′, y ′) and the coordinates (x, y) does not limit the present invention, and the specific configuration of the image processing apparatus in which the adaptive filter 100 is mounted. Determined by.
- M ⁇ N tap filtering can be characterized by a filter coefficient matrix H that is an M ⁇ N matrix having filter coefficients h (i, j) as components and the offset hoffset.
- the filter reference area is a rectangular area of M ⁇ N taps
- the present invention is not limited to this, and a rhombus, a circle, a vertical /
- the present invention can also be applied to a filter reference region R having a horizontal line segment, a diagonal line segment, or any other shape.
- the filter coefficient and the offset will be expressed using a vector notation in which each component of the filter coefficient matrix H and the offset hoffset are mapped to an M ⁇ N + 1-dimensional vector.
- the 1st to M ⁇ Nth components of the filter coefficient vector V represent the filter coefficient h (i, j), and the last component of the filter coefficient vector V (hereinafter referred to as an offset component). Represents an offset hoffset.
- the filter coefficient h (i, j) and the offset hoffset can be expressed uniformly.
- each component of the filter coefficient vector V is represented as v (K) using a one-dimensional subscript K.
- v (1) h (1,1)
- v (2) h (1,2)
- V (M ⁇ N) v (M, N)
- v (M ⁇ N + 1) hoffset.
- % represents a surplus. That is, x% y represents a remainder obtained by dividing x by y.
- X / y represents the result of performing an operation of truncating the fractional part (that is, an operation of calculating a floor function) on the quotient obtained by dividing x by y.
- equation (1) can be expressed by the following equation (4).
- SO (X ') and SI (X + K) are the same one-dimensional notation as described above corresponding to SO (x', y ') and SI (x + i, y + j), respectively.
- the sum related to K is assumed to add an offset component.
- the sum related to the region R ′ includes the sum related to the offset component in addition to the sum related to the filter reference region R expressed by the one-dimensional notation.
- the 1st to M ⁇ Nth components of the filter coefficient vector V may or may not all be independent of each other.
- the filter coefficient vector V is limited to left-right symmetry, up-down symmetry, and point symmetry.
- the filter coefficient vector V can be expressed as follows.
- w1, w2, and w3 are elements of the filter coefficient vector W.
- the fourth-order filter coefficient vector V is expressed by a third-order filter coefficient vector W, and the degree of freedom is reduced by one.
- vectors relating to w1, w2, and w3 are called bases.
- W V.
- filter parameters the number of parameters for expressing the filter.
- each component of the filter coefficient vector W decreases.
- the amount of codes can be reduced, and conventionally point symmetry and the like are often used.
- a function for obtaining the filter coefficient vector V from the filter parameters is referred to as a filter calculation function F.
- Expression (5) is an example of the filter calculation function F.
- the base is not limited to the symmetric base as described above, and may be another base.
- the filtering in the adaptive filter 100 according to the present invention is not limited to the weighted linear sum, but may be a weighted nonlinear sum using each component of the filter coefficient vector V.
- the adaptive filter 100 uses the filter coefficient vector V as the side information # 100a, each pixel value included in the image data # 100, the position of each pixel included in the image data # 100, or a function thereof. It can be switched accordingly.
- ⁇ Example of Filtering Process by Adaptive Filter 100> (Example of switching filter coefficients depending on whether skip mode is used)
- the adaptive filter 100 determines that the pixel at the coordinates (x, y) of the image data # 100 is H.264.
- a case will be described as an example where the filter coefficient vector is switched according to whether or not it belongs to a macroblock to which the skip mode in the H.246 / AVC standard is applied.
- Step S1 the control variable determination unit 120 sets the control variable v_skip that takes 0 or 1 depending on whether or not the pixel at the coordinates (x, y) of the image data # 100 belongs to the macroblock to which the skip mode is applied. The value is set to 1 and the control variable v_skip is output to the filter unit 110.
- control variable determination unit 120 associates the pixel at the coordinates (x, y) of the image data # 100 with information on whether or not the pixel belongs to a macroblock to which the skip mode is applied (hereinafter referred to as skip information).
- skip information information on whether or not the pixel belongs to a macroblock to which the skip mode is applied.
- the filter basis b_skip is a basis vector composed of basis components corresponding to each component of the filter coefficient vector V. That is, the filter base b_skip is an M ⁇ N + 1 dimensional vector for M ⁇ N tap filtering.
- an edge enhancement / blurring filter base can be preferably used as the filter base b_skip.
- the edge enhancement / blurring filter base is a filter base having an effect of enhancing the edge or blurring the edge, and includes, for example, a base of a twice-differential filter.
- the edge enhancement / blurring filter base is simply referred to as an edge enhancement filter base, but it is emphasized that it also has a blurring effect.
- edge enhancement filter bases include, for example, 3 ⁇ 3 taps, horizontal edge enhancement bases (0, 0, 0, ⁇ 1, 2, ⁇ 1, 0, 0, 0, 0), vertical edges Enhanced basis (0, -1, 0, 2, 0, 0, -1, 0, 0), standard Laplacian basis (-1, -1, -1, -1, 8, -1, -1, ⁇ 1, ⁇ 1, 0), modified Laplacian basis ( ⁇ 1, ⁇ 2, ⁇ 1, ⁇ 2, 12, ⁇ 2, ⁇ 1, ⁇ 2, ⁇ 1, 0), first oblique Laplacian The basis (-1, -2, 0, -2, 6, 0, -1, 0, 0, 0) and the second diagonal Laplacian basis (0, 0, -1, 0, 6, -2) , 0, -2, -1, 0) and the like.
- the control variable determination unit 120 outputs the edge enhancement filter base or a linear sum thereof as the filter base b_skip to the filter unit 110.
- Step S2 The filter unit 110 determines a filter coefficient vector V for each slice or each frame.
- w (K) is each component of the filter coefficient vector W (fixed coefficient vector)
- b_skip (K) is each component of the filter base (fixed vector) b_skip described above.
- A_skip is a coefficient (fixed coefficient) associated with the control variable v_skip.
- variable in a fixed vector and a fixed coefficient means that it does not change depending on the position of the pixel, and “variable” of the variable coefficient means that it changes depending on the position of the pixel.
- Equation (6) is an example of the filter calculation function F using control variables, and the filter parameters of the function F are a fixed coefficient vector, a fixed coefficient, and a control variable.
- the filter unit 110 determines, for each unit region such as a slice or a frame, a filter parameter that is fixed regardless of the region (referred to as a fixed filter parameter).
- a filter parameter that is fixed regardless of the region.
- each component w (K) of the fixed coefficient vector W and the fixed coefficient a_skip are determined.
- Equation (6) when the coefficient applied to the edge enhancement basis b_skip (K) is positive (in this case, corresponding to the case where a_skip is positive), an edge enhancement effect can be obtained in a region where the control variable v_skip is 1. On the contrary, when the coefficient applied to b_skip (K) is negative (in this case, corresponding to the case where a_skip is negative), a blurring effect can be obtained in a region where v_skip is 1.
- the filter unit 110 can determine the fixed filter parameter by, for example, a statistical method.
- S (X) represents a pixel value at the coordinates (x, y) of the teacher data # 200.
- the sum for X is the sum for all pixels included in the unit region. That is, when the fixed filter parameter is obtained for each slice, the sum is obtained for all the pixels included in the slice.
- all the pixels included in the unit area are used.
- some pixels may be used.
- the filter unit 110 is used as an adaptive deblocking filter, it is appropriate to limit the processing region to only pixels whose distance from the block boundary is a certain distance or less and determine the fixed filter parameter only from the processing region. It is.
- the fixed filter parameter is determined from the pixel in the filtering area. Is appropriate.
- the filter parameter V in Expression (7) is converted to a filter parameter function using the filter calculation function F.
- an equation generally called a normal equation can be obtained by setting an equation obtained by partial differentiation of the obtained equation with a fixed parameter to 0.
- a fixed parameter is calculated by solving a normal equation by Gaussian elimination or the like.
- the value is substituted into S, SI, and a control variable (here, v_skip) determined for each pixel or each region.
- the introduction of the control variable v_skip can be expressed as M ⁇ N + 2 taps as the introduction of the offset is expressed as an M ⁇ N + 1 tap filter.
- the dimension of SI can be expanded as follows. If such an expression is used, there is a problem of obtaining M ⁇ N + 2 values of v that minimize Equation (7).
- a filter coefficient vector V that minimizes an absolute value error obtained by replacing the square calculation in the square error E with an absolute value calculation.
- requiring is mentioned.
- a function obtained by adding a value obtained by multiplying the square error E to the square of each term of the filter coefficient vector by a predetermined constant may be used as an objective function (so-called ridge regression).
- the square error E can be reduced while keeping the filter coefficient vector at a small value.
- the square error E is slightly larger than the minimum value, but the code amount can be suppressed because the filter coefficient vector is a small value. Note that the method of determining the filter coefficient vector V by the above statistical method does not limit the present invention.
- the adaptive filter 100 can determine the filter coefficient vector V when the value of the filter coefficient vector is not changed by the control variable by using the above statistical method with the control variable always set to 0.
- control variable v_skip set in this way is supplied to the filter unit 110 for each pixel included in the image data # 100.
- Step S4 The filter unit 110 takes a weighted average of the pixel values included in the image data # 100 based on each component of the filter coefficient vector V using the value of the control variable v_skip supplied from the control variable determination unit 120. Each pixel value of the output image data # 110a is calculated. Specifically, the filter unit 110 calculates the pixel value SO (X ′) by using each component v (K) of the filter coefficient vector V in the equation (4).
- the adaptive filter 100 performs filtering having an edge enhancement effect when the pixel at the coordinates (x, y) of the image data # 100 belongs to the macroblock to which the skip mode is applied, and otherwise performs the edge Perform filtering without emphasis effect.
- a macroblock image to which the skip mode is applied tends to have a clearer edge than a macroblock to which the skip mode is not applied.
- a_skip is obtained as a negative value, and optimal blurring processing can be applied.
- the adaptive filter 100 can switch the filter coefficient vector V by setting the value of the control variable v_skip to 1 or 0. That is, since the adaptive filter 100 can switch the filter coefficient for each macroblock without calculating the filter coefficient for each macroblock, effective filtering can be performed while reducing the amount of calculation necessary for calculating the filter coefficient. It can be performed.
- skip information allocated to any one of the plurality of macroblocks may be used.
- the image filter (adaptive filter 100) has the pixel value of each pixel belonging to the target region (filter reference region R) in the input image (image data # 100) and the filter coefficient vector (filter coefficient).
- An image filter for calculating a pixel value of a target pixel in an output image (output image data # 110a) from the vector V), the position of the target region in the input image, and the target pixel in the output image A filter coefficient vector changing unit (control variable determining unit 120) that changes the filter coefficient vector according to at least one of the positions is provided.
- filter coefficient vector changing means for changing the filter coefficient vector according to at least one of the position of the target region in the input image and the position of the target pixel in the output image. Therefore, appropriate filtering can be performed for each target region according to at least one of the position of the target region and the position of the target pixel in the output image.
- the filter coefficient vector (filter coefficient vector V) is a fixed filter coefficient vector (fixed coefficient) determined in advance for each unit area constituting the output image.
- the filter coefficient vector calculated from the vector W), the position of the target area (filter target area T) in the input image (image data # 100) by the filter coefficient vector changing means (control variable determining unit 120), and , And can be decomposed into a sum with a variable filter coefficient vector that is changed according to at least one of the positions of the target pixels in the output image.
- the filter coefficient vector can be decomposed into a sum of a fixed filter coefficient vector and a variable filter coefficient vector determined in advance for each unit area constituting the output image, and the filter coefficient vector
- the changing unit can change the variable filter coefficient vector according to at least one of a position of the target region in the input image and a position of the target pixel in the output image.
- the unit area may be a slice, a macro block, or a block.
- the fixed filter coefficient vector and the variable filter coefficient vector may be defined so as to include an offset component.
- variable filter coefficient vector further includes a fixed coefficient (for example, a_skip) determined for each unit area constituting the output image, and the filter.
- a variable coefficient control variable, for example, v_skip
- v_skip control variable that is changed according to the position of the target region in the input image or the position of the target pixel in the output image by the coefficient vector changing unit, and a predetermined fixed value It can be decomposed into a vector (filter basis, for example, b_skip).
- variable filter coefficient vector can be further decomposed into a fixed coefficient determined for each unit region constituting the output image, the variable coefficient, and the fixed vector. Further, the fixed vector is determined in advance, and the filter coefficient vector changing means sets the variable coefficient according to the position of the target region in the input image or the position of the target pixel in the output image. Since it can be changed, more appropriate filtering can be performed according to the characteristics of the image in the target region.
- a desired type of filtering adapted to each target region is provided by providing the image filter with the fixed coefficient vector and the fixed coefficient determined for each unit region constituting the output image. It can be performed.
- FIG. 2 is a diagram illustrating a case where Q (Q ⁇ 2) target regions PR1 to PRQ exist on the unit region UR.
- the number of components of the fixed coefficient vector W is P (P ⁇ 2). If the number of fixed coefficients is one, adaptive filtering can be performed on all Q target regions PR1 to PRQ using a total of P + 1 filter coefficients.
- a type of filtering adapted to each target region is performed with a smaller number of filter coefficients than when the filter coefficient vector is calculated individually for each target region. be able to.
- the Q target regions PR1 to PRQ are drawn so as not to overlap each other, in general, the Q target regions PR1 to PRQ include regions that overlap each other.
- the feature of the present invention is described using the vector expression of the filter coefficient.
- the present invention can also be expressed as an action on each component of the filter coefficient as follows.
- the fourth component of the filter coefficient vector V is The six components depend on the control variable v_skip. Further, the rate of change of each of the fourth to sixth components with respect to the control variable v_skip is determined by the filter base b_skip.
- a predetermined component of the filter coefficient vector V ( The fourth component to the sixth component) change in accordance with the value of the control variable v_skip, and components other than the predetermined component (the first component to the third component and the seventh component to the tenth component) are controlled. It is determined regardless of the value of the variable v_skip.
- a component that changes according to the control variable and a component that does not change according to the control variable among the components of the filter coefficient vector V are determined according to the filter base output by the control variable determination unit 120.
- the rate of change of the component that changes according to the control variable with respect to the control variable is determined.
- the filter coefficient derivation process of Equation (6) can be obtained separately for components that change according to control variables and components that do not change.
- the case of 3 ⁇ 3 tap filtering is performed again, and the case where the horizontal edge enhancement base (0, 0, 0, ⁇ 1, 2, ⁇ 1, 0, 0, 0, 0) is used as b_skip is described below.
- the filter coefficient derivation process by the adaptive filter 100 will be described with reference to FIG.
- FIG. 3 is a diagram illustrating an example of an operation flow for performing the filtering process.
- the adaptive filter 100 calculates and holds w ′ (i) in advance using a formula (6) for a filter coefficient vector that changes according to a control variable.
- Step S102 The value of the control variable v_skip in the filter target region T is calculated.
- Step S103 Loop 2: Processing from S104 to S108 is performed from i to 1 using loop variable i.
- Step S104 If the element v (i) of the filter coefficient vector corresponding to the loop variable i is not changed by the control variable, the process proceeds to S105. That is, if i is 1, 2, 3, 7, 8, 9, 10, the process proceeds to S105. Otherwise, the process proceeds to S106.
- Step S106 If the control variable v_skip is 0, the process proceeds to S105. If the control variable v_skip is 1, the process proceeds to S107.
- Step S108 Loop 2 termination (Step S109) Using the derived filter coefficient vector V, the filtering shown in Expression (4) is performed.
- Step S110 End of Loop 1
- the derivation of the filter coefficient vector V is performed by branching without using the sum with the component that varies depending on the control variable.
- the sum processing and the branch processing appear to be separate as specific processing, but are completely the same as the processing result, and the present invention includes both.
- the control variable used for branching is not limited to the above example, and may be any example including the following example.
- FIG. 4 is a diagram illustrating another example of an operation flow for performing filtering.
- FIG. 4 shows a case where the calculation of the filter coefficient vector V uses a sum with a component that varies depending on the control variable.
- the processing from S104 to S107 in FIG. 3 is replaced with the following step S154 using Equation (6).
- Step S154 The element v (i) of the filter coefficient vector is calculated from the control variable v_skip according to the equation (6).
- the values of filter coefficients corresponding to the values of the components and control variables are stored in a two-dimensional table v2 (i, v) determined in advance by the positions of the components of the filter coefficients and the values of the control variables. It is also preferable to draw the two-dimensional table v2 (i, v) at the time of filtering.
- i means the position of the filter coefficient component
- v means the value of the control variable.
- Expression (6) is an example of the filter calculation function F.
- the filter calculation function F has a structure that can be decomposed into a fixed filter coefficient vector of the first term and a variable filter coefficient vector of the second term.
- F1 is a function of a fixed coefficient vector
- F2 is a function of a control variable and a fixed coefficient.
- the number of control variables and fixed coefficients may be plural, and when there are a plurality of control variables, the number of fixed coefficients may be changed depending on the control variable.
- b_wi is Nw bases for wi, and unit bases, symmetric bases, and the like are used.
- a unit basis is used for b_wi, but is not limited thereto.
- b_vkj is Nak bases prepared for each control variable vk, and an edge enhancement filter base or the like is used.
- a symmetrical basis is used for b_wi, a basis having the same symmetry may be used for b_vkj.
- the filter coefficient vector on the left side of Equation (9) can be decomposed into a fixed filter coefficient vector of the first term and a variable filter coefficient vector of the second term, as shown on the right side of the equation.
- the fixed filter coefficient vector can be decomposed into a fixed coefficient vector (element wi) and a base (b_wi), and the variable filter coefficient vector includes a fixed coefficient (akj), a control variable (vk), and a fixed vector ( b_vkj).
- the number of fixed filter coefficients is 3 (w1, w2, w3)
- the number of fixed coefficients corresponding to the control variable v_skip is 2 (a_skip1 and a_skip2)
- the number of fixed coefficients for the control variable v_flat is 1 (a_flat). It is. The meaning of the control variable v_flat will be described later.
- control variable determination unit 120 may be configured to output a plurality of filter bases having different effects to the filter unit 110 as filter bases associated with the skip information.
- the filter unit 110 has filter coefficients having each component w (K) expressed by the following equation (11). Each coefficient of the vector V is determined with reference to the teacher data # 200.
- v (K) w (K) + a_skip ⁇ v_skip ⁇ b_skip (K) + a_dc ⁇ v_dc ⁇ b_dc (K) (11)
- a_dc is a DC offset coefficient (fixed coefficient) corresponding to the DC offset base.
- the filter unit 110 determines each component w (K) of the fixed coefficient vector W, the edge enhancement coefficient a_skip, and the DC offset coefficient a_dc by, for example, the statistical method described above for each slice or for each frame.
- the control variable determination unit 120 outputs the control variable v_skip having a value of 1 to the filter unit 110 for the pixels belonging to the macroblock to which the skip mode is applied, and the value is 0 otherwise.
- the control variable v_skip is output to the filter unit 110.
- the filter unit 110 performs filtering based on the filter coefficient vector V using the control variable v_skip supplied from the control variable determination unit 120.
- the adaptive filter 100 performs filtering having an edge enhancement effect on a macroblock to which the skip mode is applied, and filtering having an effect of changing luminance by the DC offset. It can be performed.
- a central element base b_c (0, 0, 0, 0, 1, 0, 0, 0, 0, 0) is introduced so that only the central element is 1.
- control variable determination unit 120 may output another edge enhancement base b_skip1 together with a certain edge enhancement base b_skip0 to the filter unit 110 as a filter base associated with the skip information.
- b_skip0 is a horizontal edge enhancement basis (0, 0, 0, -1, 2, -1, 0, 0, 0, 0)
- b_skip1 (0, -1, 0, 0, 2, 0, -1, 0, 0, 0).
- edge enhancement / blur bases horizontal blur bases (0, 0, 0, 1, 2, 1, 0, 0, 0), vertical blur bases (0, 1, 0, 0, 2, 0, 0, 1, 0, 0) or omnidirectional blur bases (1, 1, 1, 1, 2, 1, 1, 1, 1, 0) It may be used.
- the filter unit 110 uses the filter coefficient vector V having each component v (K) expressed by the following equation (12). Each coefficient is determined with reference to the teacher data # 200.
- v (K) w (K) + a_skip0 * v_skip * b_skip0 (K) + a_skip1 * v_skip * b_skip1 (K) (12)
- a_skip0 is an edge enhancement coefficient (fixed coefficient) corresponding to the edge enhancement base b_skip0 (K)
- a_skip1 is an edge enhancement coefficient (fixed coefficient) corresponding to b_skip1 (K).
- the filter unit 110 determines each component w (K) of the fixed coefficient vector, the edge enhancement coefficients a_skip0, and a_skip1 for each slice or each frame, for example, by the statistical method described above.
- the control variable determination unit 120 outputs the control variable v_skip having a value of 1 to the filter unit 110 for the pixels belonging to the macroblock to which the skip mode is applied, and the value is 0 otherwise.
- the control variable v_skip is output to the filter unit 110.
- the filter unit 110 performs filtering based on the filter coefficient vector V using the control variable v_skip supplied from the control variable determination unit 120.
- the adaptive filter 100 can perform filtering based on the filter coefficient having the edge enhancement effect and the blurring effect on the macroblock to which the skip mode is applied, For a macroblock to which the skip mode is not applied, filtering based on a filter coefficient that does not have an edge enhancement effect and a blurring effect can be performed.
- the adaptive filter 100 switches the filter coefficient vector V depending on whether each pixel included in the image data # 100 belongs to a macro block to which the skip mode is applied.
- the present invention is not limited to this.
- the flatness of the image in the filter target region T is an index that is determined according to the variation in the pixel value of each pixel included in the filter target region T.
- the case where the variation in the pixel value of each pixel included in the image in the filter target region T is smaller is the case where the flatness of the image in the region is larger, and each pixel included in the image in the filter target region T
- the case where the variation of the pixel value is larger is the case where the flatness of the image in the region is smaller.
- the flatness may be obtained using a rectangular area MV ⁇ NV smaller than the area T.
- the control variable determining unit 120 sets the value of the control variable v_flat that takes 0 or 1 according to the flatness of the image in the filter target region T to 1 and outputs the control variable v_flat to the filter unit 110.
- control variable determination unit 120 outputs the filter base b_flat associated with the flatness of the image in the filter target region T to the filter unit 110.
- the filter base b_flat for example, the above-described edge enhancement filter base, Laplacian base expanded to 5 ⁇ 5 taps (1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 0, ⁇ 16, 0, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0) or their linear sum (or weighted linear sum) may be used.
- the filter unit 110 determines each coefficient of the filter coefficient vector V having each component v (K) expressed by the following equation (13) with reference to the teacher data # 200.
- v (K) w (K) + a_flat ⁇ v_flat ⁇ b_flat (K) (13)
- a_flat is a coefficient (fixed coefficient) corresponding to the filter base b_flat.
- the filter unit 110 determines each component w (K) and coefficient a_flat of the filter coefficient vector for each slice or frame, for example, by the statistical method described above.
- the control variable determination unit 120 calculates the variation of the pixel values of the pixels included in the filter target region T for each pixel included in the image data # 100. Specifically, for example, the control variable determination unit 120 calculates the variance of the pixel values of the pixels included in the filter target region T for each pixel included in the image data # 100.
- control variable determination unit 120 refers to the pixel value for each pixel included in the image data # 100 and calculates the variation in the pixel value of each pixel included in the filter target region T. Specifically, the control variable determining unit 120 refers to the pixel value of each pixel included in the image data # 100, and calculates the variance of the pixel value of each pixel included in the filter target region T.
- the control variable determining unit 120 sets the control variable v_flat to 0 when the variance of the pixel values is equal to or greater than a predetermined threshold, and controls the control variable when the variance of the pixel values is less than the predetermined threshold.
- v_flat is set to 1, and the control variable v_flat is output to the filter unit 110 for each pixel included in the image data # 100.
- control variable determination unit 120 may output the control variable v_flat to the filter unit 110 for each filter target region T.
- the filter unit 110 performs filtering based on the filter coefficient vector V using the control variable v_flat supplied from the control variable determination unit 120.
- the adaptive filter 100 performs filtering having a blurring effect on the filter target region T having a high flatness, and performs a blurring effect on the filter target region T having a low flatness. Therefore, effective filtering can be performed on slices or frames including regions having different flatnesses.
- the filter coefficient is switched according to the flatness of the image, the effect of the luminance change due to the DC offset can be obtained by using the filter base corresponding to the DC offset as described above and the filter coefficient. It is also possible to perform filtering.
- the flatness of the image in the filter target region T is determined according to the magnitude of the AC component associated with the block to which the pixel at the coordinates (x, y) of the image data # 100 belongs. It is good also as a structure.
- the control variable determination unit 120 sets the value of the following control variable to 1 and outputs it to the filter unit 110.
- a control variable v_edge_0 that takes 0 or 1 depending on whether or not a vertical edge is included in the filter target region T
- a control variable v_edge_90 that takes 0 or 1 depending on whether or not the filter target region T includes a horizontal edge.
- a control variable v_edge_45 that takes 0 or 1 depending on whether or not the filter target region T includes an obliquely upward rising edge.
- a control variable v_edge_135 that takes 0 or 1 depending on whether or not the filter target region T includes a diagonally downward-sloping edge. Further, the control variable determination unit 120 outputs the following filter base to the filter unit 12.
- a filter base b_edge_0 associated with information about whether or not the edge to be filtered includes a vertical edge
- B_edge_0 (0, -1, 0, 0, 2, 0, 0, -1, 0, 0)
- B_edge_90 (0, 0, 0, -1, 2, -1, 0, 0, 0, 0)
- B_edge_45 (0, 0, -1, 0, 2, 0, -1, 0, 0, 0)
- B_edge_135 (-1, 0, 0, 0, 2, 0, 0, 0, -1, 0)
- the filter unit 110 determines each coefficient of the filter coefficient vector V having each component v (K) expressed by the following equation (14) with reference to the teacher data # 200.
- v (K) w (K) + ⁇ a_edge_n ⁇ v_edge_n ⁇ b_edge_n (K) (14)
- the adaptive filter 100 can perform filtering using different filter coefficients depending on the angle of the edge included in the target region T. Therefore, the slice or frame including the edges of various angles Also, effective filtering can be performed.
- the control variable determination unit 120 may be configured to output the following filter base to the filter unit 110.
- the first filter base b_edge_0_0 b_edge_0 associated with the information indicating whether or not the filter target region T includes an edge in the vertical direction
- the second filter base b_edge_0_1 b_edge_90 associated with the information on whether or not the vertical edge is included in the filter target region T
- a first filter base b_edge_90_0 b_edge_90 associated with information on whether or not the filter target region T includes a horizontal edge.
- the second filter base b_edge_90_1 b_edge_0 associated with the information on whether or not the filter target region T includes an edge in the horizontal direction
- a first filter base b_edge_45_0 b_edge_45 associated with information indicating whether or not the filter target region T includes a diagonally upward rising edge
- the second filter base b_edge_45_1 b_edge_135 associated with the information indicating whether or not the filter target region T includes an obliquely upward rising edge
- the first filter base b_edge_135_0 b_edge_135 associated with the information indicating whether or not the filter target region T includes an obliquely downward-sloping edge
- the first filter base b_edge_135_1 b_edge_45 associated with the information indicating whether or not the filter target region T includes a diagonally lower right edge
- the filter unit 110 determines each coefficient of the filter coefficient vector V having each component v (K) expressed by the following equation (15) with reference to the teacher data #
- v (K) w (K) + ⁇ a_edge_n_0 ⁇ v_edge_n ⁇ b_edge_n_0 (K) + ⁇ a_edge_n_1 ⁇ v_edge_n ⁇ b_edge_n_1 (K) (15)
- the adaptive filter 100 includes edges of various angles because the adaptive filter 100 can perform filtering using different filter coefficients depending on the angle of the edge included in the target region T. Effective filtering can also be performed on slices or frames.
- w in the above (preliminary step) of ⁇ filter coefficient calculation processing>'(I) may be calculated by the following equation and held.
- control variable v_skip may be read as the control variable v_edge_90.
- the control variable v_edge_90 may be read as the control variable v_edge_90.
- the filter coefficient is switched according to the average luminance of the image in the filter target region T of the image data # 100.
- the average luminance of the image in the filter target region T is the average of the luminance values of the pixels included in the filter target region T.
- the luminance value of each pixel can generally be obtained as a function of the pixel value of the pixel.
- the control variable determination unit 120 sets one of 0, 1, and 2 as the control variable v_lum according to the average luminance of the image in the filter target region T, and sets the control variable v_lum to the filter unit 110. Output.
- control variable determination unit 120 outputs the filter base b_lum associated with the average brightness of the image in the filter target region T to the filter unit 110.
- the filter base b_lum for example, the above-described edge enhancement filter base or a linear sum (or weighted linear sum) thereof may be used.
- the filter unit 110 determines each coefficient of the filter coefficient vector V having each component v (K) expressed by the following equation (16) with reference to the teacher data # 200.
- v (K) w (K) + a_lum ⁇ v_lum ⁇ b_lum (K) (16)
- a_lum is a coefficient (fixed coefficient) corresponding to the filter base b_lum.
- the filter unit 110 determines each component w (K) and coefficient a_lum of the fixed coefficient vector for each slice or frame, for example, using the statistical method described above.
- the control variable determination unit 120 calculates, for each filter target region T, an average value of luminance values associated with pixels included in the region. For example, when the average value is less than 48, the control variable determination unit 120 sets the value of the control variable v_lum to 2, and when the average value is 48 or more and less than 96, the value of the control variable v_lum Is set to 1, and when the average value is 96 or more, the value of the control variable v_lum is set to 0.
- the average value takes values from 0 to 255.
- control variable determining unit 120 outputs the control variable v_lum to the filter unit 110 for each filter target region T.
- the filter unit 110 performs filtering for each filter target region T based on the filter coefficient vector V using the control variable v_lum supplied from the control variable determination unit 120.
- the filtering base having the effect of the luminance change due to the DC offset can be performed by using the filter base corresponding to the DC offset as described above and the filter coefficient. It is also possible to do this.
- Y indicates the size of the luminance component among the color components displayed by the pixel
- U indicates the size of the color component defined by a constant multiple of the difference between the blue component and the luminance component
- V indicates the size of the color component defined by a constant multiple of the difference between the red component and the luminance component.
- Y, U, and V each take a value from 0 to 255.
- the control variable determination unit 120 sets the value of the control variable v_skin that takes 0 or 1 to 1 according to whether or not the pixel at the coordinates (x, y) of the image data # 100 indicates skin color, and performs the control
- the variable v_skin is output to the filter unit 110.
- control variable determination unit 120 outputs the filter base b_skin associated with whether or not the pixel at the coordinates (x, y) of the image data # 100 indicates the skin color to the filter unit 110.
- the filter base b_skin for example, the above-described edge enhancement filter base or a linear sum (or weighted linear sum) thereof may be used.
- the filter unit 110 determines each coefficient of the filter coefficient vector V having each component v (K) expressed by the following equation (17) with reference to the teacher data # 200.
- v (K) w (K) + a_skin ⁇ v_skin ⁇ b_skin (K) (17)
- a_skin is a coefficient (fixed coefficient) corresponding to the filter base b_skin.
- the filter unit 110 determines each component v (K) and coefficient a_skin of the filter coefficient vector for each slice or frame by, for example, the statistical method described above.
- the control variable determination unit 120 sets the value of the control variable v_skin to 1 when the color indicated by the pixel at the coordinates (x, y) of the image data # 100 is within the above range, and the filter unit 110 Output for. Further, when the color indicated by the pixel at the coordinates (x, y) of the image data # 100 is not within the above range, the control variable determining unit 120 sets the value of the control variable v_skin to 0, and the filter unit 110. Output for.
- the filter unit 110 performs filtering for each filter target region T based on the filter coefficient vector V using the control variable v_skin supplied from the control variable determination unit 120.
- the filter base corresponding to the DC offset as described above and the filter coefficient are used to perform the filtering having the effect of the luminance change due to the DC offset. It is also possible to do this.
- the above block is, for example, H.264. It is a unit of conversion or motion compensation in the H.264 / AVC standard, and generally, there is a tendency that block noise is likely to occur at a block boundary.
- the control variable determining unit 120 sets the value of the control variable v_block that takes 0 or 1 to 1 depending on whether or not the filter target region T includes a block boundary, and sets the control variable v_block to the filter unit 110. Output.
- control variable determination unit 120 outputs a filter base b_block associated with whether or not the filter target region T includes a block boundary to the filter unit 110.
- the filter base b_block for example, the above-described edge enhancement filter bases or their linear sum (or weighted linear sum) may be used.
- the filter unit 110 determines each coefficient of the filter coefficient vector V having each component v (K) expressed by the following equation (18) with reference to the teacher data # 200.
- v (K) w (K) + a_block ⁇ v_block ⁇ b_block (K) (18)
- a_block is a coefficient (fixed coefficient) corresponding to the filter base b_block.
- the filter unit 110 determines each component w (K) and coefficient a_block of the fixed coefficient vector for each slice or frame, for example, using the statistical method described above.
- the control variable determining unit 120 determines whether or not the filter target region T includes a block boundary. When the filter target region T includes a block boundary, the control variable v_block is set to 1. When the filter target region T does not include the block boundary, the value of the control variable v_block is set to 0. Further, the control variable determining unit 120 outputs the control variable v_block to the filter unit 110.
- the filter unit 110 performs filtering for each filter target region T based on the filter coefficient vector V using the control variable v_block supplied from the control variable determination unit 120.
- the filter base corresponding to the DC offset as described above and the filter coefficient are used. It is also possible to perform filtering having the effect of luminance change due to DC offset.
- the filter coefficient is switched according to the side information associated with the block to which the pixel at the coordinates (x, y) of the image data # 100 belongs.
- the block to which the pixel at the coordinates (x, y) of the image data # 100 belongs is, for example, H.264.
- the filter coefficient is switched according to whether the block is generated based on unidirectional prediction in the H.264 / AVC standard or the block generated based on bidirectional prediction will be described.
- the unidirectional prediction in the H.264 / AVC standard is a case where the block is predicted using one frame encoded / decoded before the frame to which the block belongs. This is a case where the block is predicted using two frames that are encoded / decoded before the frame to which the block belongs. In general, one or two frames are used for prediction, but more frames may be used.
- the control variable determination unit 120 is a block to which a pixel at the coordinates (x, y) of the image data # 100 belongs is a block generated based on unidirectional prediction or generated based on bidirectional prediction Depending on whether it is a block, the value of the control variable v_pred that takes 0 or 1 is set to 1, and the control variable v_pred is output to the filter unit 110.
- control variable determining unit 120 determines whether the block to which the pixel at the coordinates (x, y) of the image data # 100 belongs is a block generated based on unidirectional prediction, or is generated based on bidirectional prediction.
- the filter base b_pred associated with the block is output to the filter unit 110.
- the filter base b_pred for example, the above-described edge enhancement filter base or a linear sum (or weighted linear sum) thereof may be used.
- the filter unit 110 determines each coefficient of the filter coefficient vector V having each component v (K) expressed by the following equation (19) with reference to the teacher data # 200.
- v (K) w (K) + a_pred ⁇ v_pred ⁇ b_pred (K) (19)
- a_pred is a coefficient (fixed coefficient) corresponding to the filter base b_pred.
- the filter unit 110 determines each component w (K) and coefficient a_pred of the fixed coefficient vector for each slice or each frame, for example, by the statistical method described above.
- the control variable determining unit 120 sets the value of the control variable v_pred to 0 when the block to which the pixel at the coordinates (x, y) of the image data # 100 belongs is a block generated based on unidirectional prediction. If the block is a block generated based on bidirectional prediction, the value of the control variable v_pred is set to 1. In addition, the control variable determination unit 120 outputs the control variable v_pred to the filter unit 110.
- the filter unit 110 performs filtering for each filter target region T based on the filter coefficient vector V using the control variable v_pred supplied from the control variable determination unit 120.
- the block to which the pixel at the coordinates (x, y) of the image data # 100 belongs is a block generated based on unidirectional prediction, it has an edge or blur enhancement effect. If the block to which the pixel belongs is a block generated based on bidirectional prediction, filtering having edge enhancement or blurring effect can be performed.
- the block to which the pixel at the coordinates (x, y) of the image data # 100 belongs is H.264.
- a case will be described in which the filter coefficient is switched according to whether the block is an intra-predicted block or an inter-predicted block in the H.264 / AVC standard.
- the control variable determination unit 120 determines whether the block to which the pixel at the coordinates (x, y) of the image data # 100 belongs is an intra-predicted block or an inter-predicted block.
- the value of the control variable v_int taking 1 is set to 1, and the control variable v_int is output to the filter unit 110.
- control variable determination unit 120 is associated with whether the block to which the pixel at the coordinates (x, y) of the image data # 100 belongs is an intra-predicted block or an inter-predicted block.
- the filter base b_int is output to the filter unit 110.
- the filter base b_int for example, the above-described edge enhancement filter base or a linear sum (or weighted linear sum) thereof may be used.
- the filter unit 110 determines each coefficient of the filter coefficient vector V having each component v (K) expressed by the following equation (20) with reference to the teacher data # 200.
- v (K) w (K) + a_int ⁇ v_int ⁇ b_int (K) (20)
- a_int is a coefficient (fixed coefficient) corresponding to the filter base b_int.
- the filter unit 110 determines each component w (K) and coefficient a_int of the fixed coefficient vector for each slice or frame, for example, by the statistical method described above.
- the control variable determining unit 120 sets the value of the control variable v_int to 1 and is inter-predicted. If it is a block, the value of the control variable v_int is set to 0. Further, the control variable determination unit 120 outputs the control variable v_int to the filter unit 110.
- the filter unit 110 performs filtering for each filter target region T based on the filter coefficient vector V using the control variable v_pred supplied from the control variable determination unit 120.
- the filter base corresponding to the DC offset as described above and the filter coefficient are used even when the filter coefficient is switched depending on whether the prediction is inter prediction or intra prediction. Accordingly, it is possible to perform filtering having an effect of luminance change due to DC offset.
- the control variable determination unit 120 sets the value of the control variable v_dist that takes 0 or 1 to 1 according to the distance between the pixel at the coordinates (x, y) of the image data # 100 and the block boundary of the block to which the pixel belongs. Then, the control variable v_dist is output to the filter unit 110.
- control variable determination unit 120 outputs the filter base b_dist associated with the distance between the pixel at the coordinates (x, y) of the image data # 100 and the block boundary of the block to which the pixel belongs to the filter unit 110.
- the filter base b_dist for example, the above-described edge enhancement filter bases or their linear sum (or weighted linear sum) may be used.
- the filter unit 110 determines each coefficient of the filter coefficient vector V having each component v (K) expressed by the following equation (21) with reference to the teacher data # 200.
- v (K) w (K) + a_dist ⁇ v_dist ⁇ b_dist (K) (21)
- a_dist is a coefficient (fixed coefficient) corresponding to the filter base b_dist.
- the filter unit 110 determines each component w (K) and coefficient a_dist of the fixed coefficient vector for each slice or each frame by, for example, the statistical method described above.
- the control variable determination unit 120 calculates the distance between the pixel at the coordinates (x, y) of the image data # 100 and the block boundary of the block to which the pixel belongs, and the distance is based on a predetermined first threshold value. Is set to 2, and the control variable v_dist is set to 2 when the distance is equal to or larger than the first threshold and smaller than the second threshold greater than the first threshold. The value is set to 1 and the value of the control variable v_dist is set to 0 when the distance is greater than or equal to the second threshold. Further, the control variable determination unit 120 outputs the control variable v_dist to the filter unit 110.
- the filter unit 110 performs filtering for each filter target region T based on the filter coefficient vector V using the control variable v_dist supplied from the control variable determination unit 120.
- three-stage filtering with different edge enhancement effects is performed according to the distance between the pixel at the coordinates (x, y) of the image data # 100 and the block boundary of the block to which the pixel belongs. It can be carried out. That is, when the distance between the pixel and the block boundary is smaller than the first threshold, filtering with the strongest edge enhancement effect is performed among the three stages, and the distance is equal to or greater than the first threshold. If the distance is smaller than the second threshold value, the second strongest edge enhancement effect is filtered out of the three steps, and the edge enhancement effect is obtained when the distance is equal to or greater than the second threshold value. No filtering can be done.
- the filter base may be changed depending on whether the block boundary is a vertical boundary or a horizontal boundary. That is, when the upper block boundary is a vertical boundary, the filter coefficient b_dist takes the above-described lateral edge enhancement basis, and when the upper block boundary is a horizontal boundary, the filter coefficient b_dist A configuration may be adopted in which a vertical edge emphasis basis is taken.
- the control variable determination unit 120 is configured to output a control variable according to the distance from the vertical block boundary and a control variable according to the distance from the horizontal block boundary. It is also possible to perform filtering with different edge enhancement effects according to the distance and the distance from the horizontal block boundary.
- control variable v_skip may be read as the control variable v_dist. The same applies when another base is used as the filter base b_dist.
- the control variable determination unit 120 sets the value of the control variable v_xy taking 0 or 1 to 1 according to the position in the frame of the pixel at the coordinates (x, y) of the image data # 100, and sets the control variable v_xy to 1 Output to the filter unit 110.
- control variable determination unit 120 outputs the filter base b_xy associated with the position in the frame of the pixel at the coordinates (x, y) of the image data # 100 to the filter unit 110.
- the filter base b_xy for example, the above-described edge enhancement base or the above-described DC offset base may be used.
- the filter unit 110 determines each coefficient of the filter coefficient vector V having each component v (K) expressed by the following equation (22) with reference to the teacher data # 200.
- v (K) w (K) + a_xy ⁇ v_xy ⁇ b_xy (K) (22)
- a_xy is a coefficient (fixed coefficient) corresponding to the filter base b_xy.
- the filter unit 110 determines each component w (K) and coefficient a_xy of the fixed coefficient vector for each slice or each frame by, for example, the statistical method described above.
- control variable determination unit 120 sets the value of the control variable v_xy to 1 and the pixel is When located on the lower half, the value of the control variable v_xy is set to zero. Further, the control variable determining unit 120 outputs the control variable v_xy to the filter unit 110.
- the filter unit 110 performs filtering for each filter target region T based on the filter coefficient vector V using the control variable v_xy supplied from the control variable determination unit 120.
- the filter base b_xy has been described as an example.
- the present invention is not limited to this, and the above-described edge enhancement base is used as the filter base b_xy. Is also possible.
- the quantization parameter QP is, for example, MPEG-2 and H.264.
- the H.264 / AVC standard it refers to a parameter associated with a quantization step and allocated to each block.
- the control variable determining unit 120 sets the value of the control variable v_qp that takes 0 or 1 to 1 according to the size of the quantization parameter associated with the block to which the pixel at the coordinates (x, y) of the image data # 100 belongs.
- the control variable v_qp is set and output to the filter unit 110.
- control variable determination unit 120 provides the filter unit 110 with the filter base b_qp associated with the magnitude of the quantization parameter QP associated with the block to which the pixel at the coordinates (x, y) of the image data # 100 belongs. Output.
- the filter base b_qp for example, the edge enhancement base described above may be used.
- the filter unit 110 determines each coefficient of the filter coefficient vector V having each component v (K) expressed by the following equation (23) with reference to the teacher data # 200.
- v (K) w (K) + a_qp ⁇ v_qp ⁇ b_qp (K) (23)
- a_qp is a coefficient (fixed coefficient) corresponding to the filter base b_qp.
- the filter unit 110 determines each component w (K) and coefficient a_qp of the fixed coefficient vector for each slice or each frame by, for example, the statistical method described above.
- the control variable determination unit 120 sets the control variable v_qp.
- the value of the control variable v_qp is set to 0. Further, the control variable determining unit 120 outputs the control variable v_qp to the filter unit 110.
- the filter unit 110 performs filtering for each filter target region T based on the filter coefficient vector V using the control variable v_qp supplied from the control variable determination unit 120.
- a block associated with a larger quantization parameter QP tends to be more blurred.
- filtering having a blurring effect is performed when the quantization parameter QP is equal to or greater than the threshold value, and filtering without a blurring effect is performed when the quantization parameter QP is smaller than the threshold value. It can be carried out.
- the DC component is, for example, H.264.
- H.264 / AVC standard it is a coefficient corresponding to a DC component among coefficients after frequency conversion is performed on pixel values included in the block.
- the control variable determining unit 120 sets the value of the control variable v_dc ′ that takes 0 or 1 to 1 according to the magnitude of the DC component associated with the block to which the pixel at the coordinates (x, y) of the image data # 100 belongs. Then, the control variable v_dc ′ is output to the filter unit 110.
- control variable determination unit 120 outputs the filter base b_dc ′ associated with the magnitude of the DC component associated with the block to which the pixel at the coordinates (x, y) of the image data # 100 belongs to the filter unit 110.
- the filter base b_dc ′ for example, the edge enhancement base described above may be used.
- the filter unit 110 determines each coefficient of the filter coefficient vector V having each component v (K) expressed by the following equation (24) with reference to the teacher data # 200.
- v (K) w (K) + a_dc ′ ⁇ v_dc ′ ⁇ b_dc ′ (K) (24)
- a_dc ′ is a coefficient (fixed coefficient) corresponding to the filter base b_dc ′.
- the filter unit 110 determines each component w (K) and coefficient a_dc ′ of the fixed coefficient vector for each slice or each frame by, for example, the statistical method described above.
- the control variable determination unit 120 determines the value of the control variable v_dc ′ when the value of the DC component associated with the block to which the pixel at the coordinates (x, y) of the image data # 100 belongs is equal to or greater than a predetermined threshold. Is set to 0, and the value of the control variable v_dc ′ is set to 1 when the value of the DC component is smaller than the threshold value. Further, the control variable determination unit 120 outputs the control variable v_dc ′ to the filter unit 110.
- the filter unit 110 performs filtering for each filter target region T based on the filter coefficient vector V using the control variable v_dc ′ supplied from the control variable determination unit 120.
- filtering having a blur effect is performed when the value of the DC component is smaller than the threshold value, and filtering without the blur effect is performed when the value of the DC component is equal to or greater than the threshold value. It can be performed.
- W ′ (i) in step) may be calculated by the following formula and held.
- the motion vector is, for example, H.264.
- H.264 / AVC standard it indicates a motion vector used for inter-screen prediction.
- the control variable determination unit 120 sets the value of the control variable v_mv that takes 0 or 1 to 1 according to the magnitude of the motion vector associated with the block to which the pixel at the coordinates (x, y) of the image data # 100 belongs.
- the control variable v_mv is output to the filter unit 110.
- control variable determination unit 120 outputs the filter base b_mv associated with the magnitude of the motion vector associated with the block to which the pixel at the coordinates (x, y) of the image data # 100 belongs to the filter unit 110.
- the filter base b_mv for example, the edge enhancement base described above may be used.
- the filter unit 110 determines each coefficient of the filter coefficient vector V having each component v (K) expressed by the following equation (25) with reference to the teacher data # 200.
- v (K) w (K) + a_mv ⁇ v_mv ⁇ b_mv (K) (25)
- a_mv is a coefficient (fixed coefficient) corresponding to the filter base b_mv.
- the filter unit 110 determines each component w (K) and coefficient a_mv of the fixed coefficient vector for each slice or each frame, for example, by the statistical method described above.
- the control variable determination unit 120 determines the value of the control variable v_mv when the magnitude of the motion vector associated with the block to which the pixel belongs at the coordinates (x, y) of the image data # 100 is equal to or greater than a predetermined threshold. Is set to 1 and the value of the control variable v_mv is set to 0 when the magnitude of the motion vector is smaller than the threshold value. Further, the control variable determination unit 120 outputs the control variable v_mv to the filter unit 110.
- the filter unit 110 performs filtering for each filter target region T based on the filter coefficient vector V using the control variable v_mv supplied from the control variable determination unit 120.
- filtering having a blur effect is performed when the magnitude of the motion vector is equal to or greater than the threshold value, and no blur effect is provided when the magnitude of the motion vector is smaller than the threshold value. Filtering can be performed.
- the DC offset can be obtained by using the filter base corresponding to the DC offset as described above and the filter coefficient. It is also possible to perform filtering having the effect of luminance change due to.
- the filter coefficient may be switched according to the magnitude of the difference motion vector that is the difference between the motion vector and the prediction vector.
- the present invention includes a case where filter coefficients are switched by appropriately combining the above examples. That is, the control variable determination unit 120 may output a plurality of control bases and a plurality of control bases corresponding to the control variables and control bases described in the above example. By adopting such a configuration, more effective filtering can be performed.
- control variable determination unit 120 determines a plurality of control variables that have a small contribution to filtering among the plurality of control variables, and the contribution The filter coefficient may be switched by setting a new control variable and switching the value of the new control variable by linear sum of a plurality of control variables having a small value.
- the filter coefficient may be switched by switching the value of the control variable v_02.
- control variable having a small contribution to filtering does not limit the present invention.
- filtering is performed using different values for each control variable, and pixel values before filtering are determined.
- the control variable having a small difference from the pixel value after filtering may be identified as the control variable having the small contribution.
- the configuration in which the coefficient for the filter base is determined by the above-described statistical method for the filter base output from the control variable determination unit 120 is described as an example. It is not limited to.
- control variable determination unit 120 may be configured to determine an optimal filter base for the coefficient determined by the statistical method described above.
- the optimum filter coefficient and the filter base for the image data # 100 are obtained by repeating the steps of calculating the coefficient with the filter base fixed and determining the optimum filter coefficient with the coefficient fixed. It is also possible.
- the moving image encoding apparatus 1 includes H.264 as a part thereof. This is a moving picture encoding apparatus using the technology adopted in the H.264 / AVC standard and KTA software.
- FIG. 5 is a block diagram showing a configuration of the moving picture encoding apparatus 1.
- the moving image encoding apparatus 1 includes a transform / quantization unit 11, a variable length encoding unit 12, an inverse quantization / inverse transform unit 13, a buffer memory 14, an intra-predicted image generation unit 15, A prediction image generation unit 16, a prediction method control unit 18, a motion vector redundancy reduction unit 19, an adder 21, a subtracter 22, a deblocking filter 50, and an adaptive filter 100 are provided.
- the moving image encoding apparatus 1 receives an input image # 1 divided into block images (hereinafter referred to as “macroblocks”) composed of a plurality of adjacent pixels.
- macroblocks an input image # 1 divided into block images
- the moving image encoding apparatus 1 performs an encoding process on the input image # 1 and outputs encoded data # 2.
- the transform / quantization unit 11 performs DCT (Discrete Cosine Transform) transform on a difference image # 22 between an input image # 1 divided into macroblocks and a prediction image # 18a output from the prediction method control unit 18 described later.
- DCT Discrete Cosine Transform
- the frequency component is quantized to generate quantized prediction residual data # 11.
- the quantization is an operation for associating the frequency component with an integer value.
- the DCT transform and quantization are performed in units of blocks obtained by dividing a macroblock.
- a macro block to be processed is referred to as a “target macro block”, and a block to be processed is referred to as a “target block”.
- the inverse quantization / inverse transform unit 13 decodes the quantized prediction residual data # 11 and generates a prediction residual # 13. Specifically, the inverse quantization / inverse transform unit 13 performs inverse quantization of the quantized prediction residual data # 11, that is, associates integer values constituting the quantized prediction residual data # 11 with frequency components. Then, inverse DCT transform of the frequency component, that is, inverse transform to the pixel component of the target macroblock based on the frequency component is performed to generate prediction residual # 13.
- the adder 21 adds the prediction residual # 13 and the prediction image # 18a to generate a decoded image # 21.
- the generated decoded image # 21 is supplied to the deblocking filter 50.
- the deblocking filter 50 determines the block boundary in the decoded image # 21 when the difference between the pixel values of pixels adjacent to each other via the block boundary in the decoded image # 21 or the macroblock boundary is smaller than a predetermined threshold. Alternatively, deblocking processing is performed on the macroblock boundary. The image data subjected to the deblocking process is output as a deblocked image # 50.
- the adaptive filter 100 performs the filtering described in ⁇ Example of filtering by the adaptive filter 100> on the deblocked image # 50, and outputs the output image data # 110a to the buffer memory 14.
- the deblock image # 50 corresponds to the image data # 100 described above.
- the side information associated with the input image # 1, the decoded image # 21, and the decoded image # 21 is input to the adaptive filter 100.
- the input image # 1 corresponds to the teacher data # 200 described above
- the side information corresponds to the side information # 100a described above.
- the adaptive filter 100 outputs filter coefficient information # 101 that is information on the filter coefficient used for filtering.
- the filter coefficient information # 101 corresponds to the filter coefficient information # 110b described above. That is, the filter coefficient information # 101 includes each component w (K) of the fixed coefficient vector W and the fixed coefficient used for filtering.
- the filter coefficient information # 101 includes each component w ( K) and a fixed coefficient a_skip. The same applies to other operation examples.
- the adaptive filter 100 in this use example substantially corresponds to KLF's ALF (Adaptive Loop Filter). That is, the adaptive filter 100 in this usage example receives the pixel value of a pixel with integer coordinates (hereinafter referred to as an integer pixel) and outputs the pixel value of the integer pixel. This corresponds to taking all of x, y, x ', and y' in the formula (1).
- KLF's ALF Adaptive Loop Filter
- the moving picture encoding apparatus 1 may be configured not to include the deblocking filter 50.
- the adaptive filter 100 performs direct filtering on the decoded image # 21, not the deblocked image # 50.
- the adaptive filter 100 can have the property of removing block noise as with the deblocking filter. In such a case, an effect of reducing the processing amount can be obtained by omitting the deblocking filter 50 process.
- an adaptive filter specialized for the deblocking filter as the deblocking filter 50 a higher effect than that of the conventional deblocking filter can be obtained (for example, a filter that changes the blurring effect depending on the luminance value can be applied). ).
- the moving picture encoding apparatus 1 may have any of a configuration including a deblocking filter and an adaptive filter, a configuration including only an adaptive filter without including a deblocking filter, and a configuration including both an adaptive filter for deblocking filter and an adaptive filter.
- the intra-predicted image generation unit 15 extracts the local decoded image # 14a (the decoded area of the same frame as the target macroblock) from the decoded image # 21 stored in the buffer memory 14, and the frame based on the local decoded image # 14a Intra prediction is performed to generate an intra prediction image # 15.
- the inter prediction image generation unit 16 calculates and assigns a motion vector # 17 to the target block on the input image # 1 by using the reference image # 14b in which the entire frame has already been decoded and stored in the buffer memory 14. .
- the calculated motion vector # 17 is output to the predicted image generation unit 16 and the motion vector redundancy reduction unit 19, and is stored in the buffer memory 14.
- the inter predicted image generation unit 16 performs motion compensation based on the motion vector # 17 for each block with respect to the reference image # 14b, and generates an inter predicted image # 16.
- the prediction method control unit 18 compares the intra prediction image # 15, the inter prediction image # 16, and the input image # 1 in units of macro blocks, and the intra prediction image # 15 or the inter prediction image # 16. Any one of them is selected and output as a predicted image # 18a. Moreover, the prediction method control part 18 outputs prediction mode # 18b which is the information showing which was selected among intra prediction image # 15 or inter prediction image # 16. The predicted image # 18a is input to the subtracter 22.
- the prediction mode # 18b is stored in the buffer memory 14 and input to the variable length encoding unit 12.
- the motion vector redundancy reduction unit 19 assigns the motion vector # 17 to the target block in the inter prediction image generation unit 16, and then assigns the motion vector group # 14c to the other block and stored in the buffer memory 14. Based on this, a prediction vector is calculated. In addition, the motion vector redundancy reduction unit 19 calculates a difference between the prediction vector and the motion vector # 17, and generates a difference motion vector # 19. The generated difference motion vector # 19 is output to the variable length coding unit 12.
- variable length coding unit 12 performs variable length coding on the filter coefficients included in the quantized prediction residual data # 11, the difference motion vector # 19, the prediction mode # 18b, and the filter coefficient information # 101, Coded data # 2 is generated.
- the subtracter 22 takes the difference between the input image # 1 and the predicted image # 18a for the target macroblock, and outputs the difference image # 22.
- Filter coefficient information # 101 is information necessary for calculating a filter coefficient or a filter coefficient.
- the filter coefficient calculation formula F is the formula (9)
- the fixed coefficient vector wi and the fixed coefficient akj are obtained.
- the filter coefficient calculation formula F is the formula (6)
- w (i), i 1 to 10
- a_skip are the filter coefficient information # 101.
- the encoded data of the filter coefficient information # 101 is composed of fixed filter parameters.
- Each value of the filter coefficient information # 101 has a high appearance probability of a value close to 0 except for the central element. Therefore, each value of the filter coefficient information # 101 is divided into a code and an absolute value. As the value is closer to 0, a code that becomes a shorter code word is assigned and encoded.
- the encoding method is not limited to this, and an arithmetic code or the like may be applied. Utilizing that the central element is a value close to 1.
- the prediction value of the filter coefficient information # 101 It is also appropriate to calculate the prediction value of the filter coefficient information # 101 and encode the difference (prediction difference) between the filter coefficient information # 101 and the prediction value.
- the predicted value the central value is close to 1 and the other elements are close to 0, so that the predicted value of w (5) corresponding to the center is 0.8 and the predicted value of w (i) around it is set. What is necessary is just to set it as 0.1.
- the prediction method is not limited to this.
- the filter coefficient vector can be divided into a component that changes depending on the control variable and a component that does not change. It is also appropriate to encode using this characteristic. 6B and 6C show two examples of this.
- the example of FIG. 6B is (1) a filter coefficient for each component that does not change with the control variable, and (2) a filter coefficient for each component that changes with the control variable, and each value of the control variable. And a filter coefficient corresponding to.
- the example of (c) of FIG. 6 includes (1) a filter coefficient (referred to as a basic coefficient) relating to each component that does not change depending on the control variable, and (2) a filter coefficient and a basic coefficient relating to each component that changes depending on the control variable. And a difference between the two.
- the filter coefficient v (i) corresponding to the case where the control variable v_skip is 0 and the filter coefficient v ′ (i) corresponding to the case where the control variable v_skip is 1.
- the differences dv ′ (i), i 4, 5, 6 are encoded.
- the component that changes depending on the filter coefficient and the component that does not change are encoded for each component.
- the number of filter coefficient information to be encoded is smaller.
- the DC offset base b_dc is used as the base corresponding to the control variable
- the component that changes depending on the filter coefficient and the component that does not change are encoded for each component.
- the number of filter coefficient information to be encoded is equal between the method of encoding and the method of encoding only the filter coefficient information of the part related to the control variable.
- the encoding device (moving image encoding device 1) according to the present invention includes the image filter (adaptive filter 100), and is characterized in that the image filter acts on a decoded image.
- the encoding device configured as described above can perform filtering by the image filter on a decoded image that has been encoded and re-decoded by the encoding device. The same effect is produced.
- the encoding apparatus configured as described above includes each component of the fixed filter coefficient vector (filter coefficient vector calculated from the fixed coefficient vector W) determined for each unit region, and the fixed coefficient. Therefore, filtering can be performed using a smaller number of filter coefficient vector components than when using filter coefficient vectors having different components for each target region. It can be performed.
- the code amount of the filter coefficient vector can be reduced as compared with the case where the filter coefficient vector having a different component for each target region is used.
- the adaptive filter 100 according to the present invention can also be used for the inter predicted image generation unit 16 in the moving image encoding device 1.
- FIG. 7 is a block diagram illustrating a configuration of the inter prediction image generation unit 16 using the adaptive filter 100.
- the application filter 100 in this usage example is referred to as an application filter 100 ′.
- the inter prediction image generation unit 16 includes a prediction image generation unit 16a, a motion vector estimation unit 17, and an adaptive filter 100 '.
- the adaptive filter 100 ′ in this use example substantially corresponds to an AIF (Adaptive Interpolation Filter) in KTA.
- Step 101 The reference image # 14b stored in the buffer memory 14 is input to the adaptive filter 100 ′.
- the reference signal # 14b is referred to as a first reference signal # 14b.
- the adaptive filter 100 performs filtering on the first reference image # 14b based on a predetermined filter coefficient, and outputs first output image data # 110a'. That is, the adaptive filter 100 ′ in this step operates as a fixed filter that performs filtering based on a predetermined filter coefficient.
- the predetermined filter coefficient is, for example, H.264.
- a filter coefficient used in H.264 / AVC may be used.
- an image signal with pixel accuracy equal to or smaller than the integer pixel is generated from the pixel value of the integer pixel included in the first reference image # 14 b by interpolation. This corresponds to taking x ′ and y ′ as numbers other than integers in Equation (1). Therefore, the first output image data 100 ′ is image data including a pixel value of an integer pixel and an interpolation signal with pixel accuracy equal to or less than the integer pixel.
- the motion vector estimation unit 17 generates a first motion vector # 17 ′ based on the first output image data # 110a ′ and the input image # 1.
- the first motion vector # 17 ′ is performed for one slice or one frame.
- the adaptive filter 100 ′ obtains a filter coefficient for the first reference image # 14b by the process described in ⁇ Example of filtering by the adaptive filter 100>.
- An image generated by filtering by the adaptive filter 100 ′ is referred to as second output image data # 110a ′′.
- the first reference image # 14b corresponds to the above-described image data # 100
- the second output image data # 110a '' corresponds to the above-described output image data # 110a.
- the input image # 1 is input to the adaptive filter 100 ′ as the teacher data # 200 described above.
- the adaptive filter 100 ′ generates an image signal with pixel accuracy equal to or less than an integer pixel by interpolation from pixel values of a plurality of pixels having integer coordinates (referred to as integer pixels) included in the first reference image # 14 b. This corresponds to taking x ′ and y ′ as numbers other than integers in Equation (1). Therefore, the second output image data # 110a 'is image data including a pixel value of an integer pixel and an interpolation signal having a pixel accuracy equal to or less than the integer pixel.
- the motion vector estimation unit 17 generates a second motion vector # 17 ′′ based on the second output image data # 110a ′′ and the input image # 1.
- Step 105 The adaptive filter 100 ′ performs filtering using the filter coefficient obtained for the first reference image # 14b, and generates a motion compensated image based on the second motion vector # 17 ′′, whereby the inter predicted image # 16 is generated and output.
- the applied filter 100 ′ sets a control variable and a filter base according to the position of the pixel included in the output image data # 110a ′ when generating the second output image data # 110a ′. It is good also as such a structure.
- the moving image decoding apparatus 2 includes H.264 as a part thereof. This is a moving picture decoding apparatus using the technology adopted in the H.264 / AVC standard and KTA software.
- FIG. 8 is a block diagram showing the configuration of the video decoding device 2.
- the video decoding device 2 includes a variable length code decoding unit 23, a motion vector restoration unit 24, a buffer memory 25, an inter prediction image generation unit 26, an intra prediction image generation unit 27, and a prediction method determination unit 28. , An inverse quantization / inverse transform unit 29, an adder 30, a deblocking filter 50, and an adaptive filter 100.
- the video decoding device 2 receives the encoded data # 2 and outputs an output image # 3.
- variable length code decoding unit 23 performs variable length decoding on the encoded data # 2, and outputs a differential motion vector # 23a, side information # 23b, quantized prediction residual data # 23c, and filter coefficient information # 23d.
- the motion vector restoration unit 24 decodes the motion vector # 24 of the target partition from the difference motion vector # 23a and the motion vector # 25a that has already been decoded and stored in the buffer memory 25.
- the buffer memory 25 stores output image data # 110a output from the adaptive filter 100, motion vector # 24, and side information # 23b.
- the inter prediction image generation unit 26 is decoded by the motion vector restoration unit 24, and based on the motion vector # 24 that passes through the buffer memory 25 and the reference image # 25d stored in the buffer memory 25, the inter prediction image # 26. Is generated.
- the motion vector # 25c includes the same motion vector as the motion vector # 24.
- the reference image # 25d corresponds to output image data # 110a output from the application filter 100 described later.
- the intra-predicted image generation unit 27 generates an intra-predicted image # 27 from the locally decoded image # 25b in the same image as the target macroblock stored in the buffer memory 25.
- the prediction method determination unit 28 selects either the intra prediction image # 27 or the inter prediction image # 26 based on the prediction mode information included in the side information # 23b, and outputs it as the prediction image # 28. To do.
- the inverse quantization / inverse transform unit 29 performs inverse quantization and inverse DCT transform on the quantized prediction residual data # 23c, and outputs a prediction residual # 29.
- Adder 30 adds prediction residual # 29 and prediction image # 28, and outputs the result as decoded image # 3.
- the output decoded image # 3 is supplied to the deblocking filter 50.
- the deblocking filter 50 determines the block boundary in the decoded image # 3 when the difference between the pixel values of the pixels adjacent to each other via the block boundary in the decoded image # 3 or the macroblock boundary is smaller than a predetermined threshold. Alternatively, deblocking processing is performed on the macroblock boundary. The image data subjected to the deblocking process is output as a deblocked image # 50.
- the adaptive filter 100 performs the filtering described in ⁇ Example of filtering by the adaptive filter 100> on the deblocked image # 50.
- the deblock image # 50 corresponds to the image data # 100 described above.
- the adaptive filter 100 in this usage example calculates a filter coefficient based on the filter coefficient information # 23d decoded from the encoded data # 2, and performs filtering using the filter coefficient.
- the adaptive filter 100 in this usage example uses the filter coefficient information # 23d decoded from the encoded data # 2 instead of determining the filter coefficient by the statistical method described in ⁇ Example of filtering by the adaptive filter 100>. Based on this, a filter coefficient is calculated, and filtering using the filter coefficient is performed. The switching of the filter coefficient in the adaptive filter 100 is as described in ⁇ Example of filtering by the adaptive filter 100>.
- the adaptive filter 100 outputs the output image data # 110a generated by filtering the deblocked image # 50 to the buffer memory 25.
- the side information # 23b is input to the adaptive filter 100.
- the side information # 23b is information indicating whether the target block is a bidirectionally predicted block or a unidirectionally predicted block, and whether or not the skip mode is applied to the target macroblock.
- Information, prediction mode information indicating whether the target macroblock is an intra-predicted macroblock or an inter-predicted macroblock, and a quantization parameter QP associated with the target block are included.
- the side information # 23b corresponds to the side information # 100a described above.
- the adaptive filter 100 in this use example substantially corresponds to KLF's ALF (Adaptive Loop Filter). That is, the adaptive filter 100 in this usage example receives the pixel value of the integer pixel and outputs the pixel value of the integer pixel. This corresponds to taking all of x, y, x ', and y' in the formula (1).
- KLF's ALF Adaptive Loop Filter
- the moving picture decoding apparatus 2 may be configured not to include the deblocking filter 50.
- the adaptive filter 100 performs direct filtering on the decoded image # 3, not the deblocked image # 50. Similar to the moving picture encoding apparatus 1, the effect of reducing the processing amount can be obtained.
- the video decoding device 2 includes a configuration including a deblocking filter and an adaptive filter, a configuration including only an adaptive filter without including a deblocking filter, an adaptive filter for the deblocking filter, and an adaptive filter. Any of the configurations including both of them may be used.
- variable length code decoding unit 23 decodes the fixed filter parameter as the filter coefficient information # 23d.
- w (1) to w (M ⁇ N + 1) and a_skip are decoded.
- the adaptive filter 100 inputs a fixed filter parameter and a control variable to a filter calculation function F whose general form is represented by Expression (8), Expression (9), etc., and derives a filter coefficient.
- w (1) to w (M ⁇ N + 1) decoded in Expression (6) and a_skip are input, and a filter coefficient vector V is derived.
- variable length code decoding unit 23 decodes the filter coefficient as filter coefficient information # 23d.
- one filter coefficient is encoded for a component that does not change with the control variable, and a plurality of filter coefficients are encoded for each value of the control variable for the component that changes with the control variable.
- the variable-length code decoding unit 23 can derive the filter coefficient by branching the process depending on whether or not the variable variable code decoding unit changes depending on the control variable.
- the variable length code decoding unit 23 decodes the basic filter coefficient as the filter coefficient information # 23d, and then decodes the difference from the basic filter coefficient.
- the filter coefficient v ′ (i) corresponding to the case where the control variable v_skip is 1 is derived by the following using the basic filter coefficient v (i) in the case of a component that does not change depending on the control variable.
- the difference dV (i) between the basic filter coefficient v (i) and the basic filter coefficient is derived as follows.
- the filter coefficient can be derived by performing branching for each component depending on whether or not it varies depending on the control variable.
- the adaptive filter 100 performs the operation as already described with reference to FIGS.
- the flow is as shown in FIG. 3, and when the filter coefficient vector is derived by the sum as shown in Expression (9), the flow is as shown in FIG.
- the filter coefficient vector may be processed by the flow of FIG. 3 using a preliminary step, or a table determined in advance by the position of the filter coefficient component and the value of the control variable is derived. You may process by table drawing.
- the decoding device (moving image decoding device 2) according to the present invention includes the image filter (adaptive filter 100), and is characterized in that the image filter acts on the decoded image.
- the decoding device configured as described above has the same effect as the image filter. Further, the decoding device configured as described above is configured to calculate each component of the fixed filter coefficient vector (filter coefficient vector calculated from the fixed coefficient vector W) and the fixed coefficient determined for each unit region. Since it is possible to perform appropriate filtering for each target region (filter target region T), a smaller number of filter coefficient vectors are used than when filter coefficient vectors having different components are used for each target region. Filtering can be performed using the components.
- the code amount of the filter coefficient vector can be reduced as compared with the case where a filter coefficient vector having a different component for each target region is used.
- the adaptive filter 100 according to the present invention can also be used for the inter predicted image generation unit 26 in the video decoding device 2.
- FIG. 9 is a block diagram illustrating a configuration of the inter prediction image generation unit 26 using the adaptive filter 100.
- the application filter 100 in this usage example is referred to as an application filter 100 ′′.
- the inter predicted image generation unit 26 includes a predicted image generation unit 26 a and an adaptive filter 100 ′′.
- the adaptive filter 100 ′′ performs filtering on the reference image # 25 d stored in the buffer memory 25.
- the reference image # 25d corresponds to the image data # 100 described above.
- the adaptive filter 100 ′′ in this usage example performs filtering based on the filter coefficient information # 23 d decoded from the encoded data # 2.
- the adaptive filter 100 ′′ in this usage example uses the filter coefficient information ## decoded from the encoded data # 2 instead of determining the filter coefficient by the statistical method described in ⁇ Example of filtering by the adaptive filter 100>. Filtering is performed based on 23d. The switching of the filter coefficient in the adaptive filter 100 is as described in ⁇ Example of filtering by the adaptive filter 100>.
- the adaptive filter 100 '' outputs output image data # 110a '' '' generated by filtering the reference image # 25d to the predicted image generation unit 26a.
- the side information # 23b is input to the adaptive filter 100 ''.
- the side information # 23b is information indicating whether the target block is a bidirectionally predicted block or a unidirectionally predicted block, and whether or not the skip mode is applied to the target macroblock.
- Information, prediction mode information indicating whether the target macroblock is an intra-predicted macroblock or an inter-predicted macroblock, and a quantization parameter QP associated with the target block are included.
- the side information # 23b corresponds to the side information # 100a described above.
- the adaptive filter 100 ′′ in this use example substantially corresponds to an AIF (Adaptive Interpolation Filter) in KTA. That is, the adaptive filter 100 ′′ in this usage example receives the pixel value of an integer pixel and outputs a pixel value with pixel accuracy equal to or less than the integer pixel. This corresponds to taking x and y as integers and taking x 'and y' as integers or decimals in equation (1).
- AIF Adaptive Interpolation Filter
- the predicted image generation unit 26a generates and outputs an inter predicted image # 26 by performing motion compensation on the output image data # 110a "" using the motion vector # 25c.
- FIG. 10 is a diagram showing a bit stream #BS for each slice of encoded data # 2 that is generated using the video encoding device 1 and is referred to by the video decoding device 2.
- the bitstream #BS includes filter coefficient information FC and macroblock information MB1 to MBN.
- FIG. 11 is a diagram showing an example of the configuration of the filter coefficient information FC.
- the filter coefficient information FC is information including the filter coefficient information # 23d generated by the adaptive filter 100, and is the same as the filter coefficient information # 101 already described with reference to FIG.
- the moving picture decoding apparatus 2 may decode information indicating the type of control variable to be used in addition to the filter coefficient information # 23d. For example, when decoding 8-bit information from 0 to 255 and using flatness as the control variable, 1 when using the angle of the edge, 2 when using the angle of the edge, 4 when using the luminance, and when using the color 8 when using whether to include an edge, 32 when using a prediction direction, 64 when using skip / non-skip, and 128 when using inter / intra. Depending on whether or not, the type of control variable to be used may be determined. Note that it is possible to select a plurality of control variables in any combination by encoding such encoded data.
- the control variable that can be selected may be any control variable described in the present specification, or may be a control variable that is not described.
- the base type used for the control variable may be decoded. For example, 8-bit information from 0 to 255 is decoded, the horizontal edge emphasis base is 1, the vertical edge emphasis base is 2, the standard Laplacian base is 4, the first diagonal Laplacian base is 8, and the second The diagonal Laplacian base is 16, the DC offset base is 32,..., And the type of base to be used may be determined depending on whether the bit of each value is set.
- the video decoding device 2 can select the base type to be used for each of the plurality of control variables by decoding the base type for each control variable.
- FIG. 11 shows an example in which each base is designated for the case where two control variables # 1 and # 2 are used.
- the base that can be selected may be any base described in this specification, or a base that is not described.
- the moving picture decoding apparatus 2 is configured to provide information on the number of filter taps, for example, 0 for a 3 ⁇ 3 filter, 1 for a 5 ⁇ 5 filter, 2, 2 for a 7 ⁇ 7 filter, and 9 ⁇ 9 filters. In some cases, 3 or the like may be decoded. In this way, the number of filter coefficient vectors can be changed according to the filter effect, and the coding efficiency is improved.
- the video decoding device 2 may change the number of base taps according to the number of decoded filter taps, or may not change the number of base taps according to the number of filter taps. In this case, even if the number of filter taps is 5 ⁇ 5, the base may be 3 ⁇ 3 or the like.
- the value of the filter coefficient information # 23d at a position outside the base range is decoded as the value of the filter coefficient vector itself.
- the value of the filter coefficient vector at a position outside the base range is a component that does not change depending on the control variable.
- the value of the filter coefficient vector itself is determined as a component that varies depending on the control variable even outside the range of the number of taps of the filter. Only filter coefficient vectors corresponding to the number of taps are used.
- the moving picture decoding apparatus 2 is information indicating the format of the encoded data of the filter, for example, 0 in the case of the format of FIG. 6A, 1 in the format of FIG. In the case of (c), 2 may be decoded. Even in an encoding apparatus in which it is difficult to obtain the filter coefficient information # 23d in the form of FIG. 6A due to the algorithm for obtaining the filter coefficient vector, the coding is performed by changing the filter coefficient vector to be used according to the control variable. Efficiency can be improved.
- the video decoding device 2 may decode information specifying the bit depth of the filter coefficient information. For example, it is decoded whether the bit depth is 8, 10, or 12 bits. The bit depth is equivalent to a roughness for quantizing the filter coefficient information.
- the moving image decoding apparatus 2 may decode the range of the value of the control variable, for example, 0, 1 or 0, 1, 2. Also, information indicating a threshold used when determining a control variable from the image data # 100 and the side information # 100a of the image data # 100, for example, threshold information for determining the value of the control variable from the brightness and flatness May be decrypted. Further, the filtering unit may be decoded. The filtering unit may be a pixel, a block, a plurality of blocks, or another unit. In addition, information for specifying the presence or absence of a filter region may be included by using a method of providing a region where filtering is not performed in a unit region where filtering is performed.
- filter coefficient information control information These pieces of information are called filter coefficient information control information.
- the filter coefficient information control information such as the control variable, the base, and the number of taps
- the optimum filter coefficient vector can be switched according to the sequence, the picture, and the slice, and the coding efficiency is improved.
- the degree of freedom of the encoding device is improved. For example, if the capacity of the encoding device is large, it is possible to increase the encoding efficiency by increasing the number of control variables or the number of bases. Conversely, when the coding efficiency is low, the number of control variables and bases can be minimized or not used.
- control variable or basis used according to the characteristics of the installed software or hardware, such as using flatness as the control variable can also be changed.
- Such a degree of freedom is beneficial not only in the encoding device but also in the decoding device. That is, when it is assumed that the capability of the decoding apparatus is small, encoding can be performed with the number of control variables and the number of bases to be used as a minimum.
- filter coefficient information control information information (filter coefficient information format control information) indicating which information is included may be decoded from the encoded data.
- the control variable information may be included, but the tap information may not be included.
- the filter coefficient information 14b, the filter coefficient information control information, and the filter coefficient information format control information may be encoded at different positions on the encoded data.
- the filter coefficient information control information is encoded with a slice header, but the filter coefficient information format control information may be encoded with a picture header, and the filter coefficient information format control information may be encoded with a sequence header.
- H It may be encoded with encoded data corresponding to a picture parameter set and a sequence parameter set in H.264 / AVC.
- Macro block information MB1 to MBN is information related to macro blocks included in the slice, and includes macro block prediction mode # 18b, block division information, and the like.
- N represents the number of macroblocks included in the slice.
- the data structure of the encoded data according to the present invention is a data structure of encoded data that is referred to by a decoding device that includes the image filter and applies the image filter to a decoded image, and includes the fixed coefficient vector.
- a decoding device that includes the image filter and applies the image filter to a decoded image, and includes the fixed coefficient vector.
- each component of the fixed filter coefficient vector calculated from the fixed coefficient vector, and the fixed coefficient or prediction difference data thereof are included.
- each component of the fixed coefficient vector included in the encoded data in the decoding device that decodes an image based on the encoded data, each component of the fixed coefficient vector included in the encoded data, and By using the fixed coefficient, it is possible to perform more appropriate filtering for each target area while using a smaller number of filter coefficients than in the case where different filter coefficients are included for each target area.
- the image filter according to the present invention is an image filter that calculates the pixel value of the target pixel in the output image from the pixel value of each pixel belonging to the target region in the input image and the filter coefficient vector.
- a filter coefficient vector changing unit that changes the filter coefficient vector according to at least one of the position of the target area in the input image and the position of the target pixel in the output image is provided.
- filter coefficient vector changing means for changing the filter coefficient vector according to at least one of the position of the target region in the input image and the position of the target pixel in the output image. Therefore, there is an effect that appropriate filtering can be performed for each target area according to at least one of the position of the target area and the position of the target pixel in the output image.
- the filter coefficient vector is determined by the fixed filter coefficient vector predetermined for each unit area constituting the output image and the filter coefficient vector changing means, and It is preferable that the image can be decomposed into a sum of a position of the target region and a variable filter coefficient vector that is changed according to at least one of the position of the target pixel in the output image.
- the filter coefficient vector can be decomposed into a sum of a fixed filter coefficient vector and a variable filter coefficient vector determined in advance for each unit area constituting the output image, and the filter coefficient vector
- the changing unit can change the variable filter coefficient vector according to at least one of a position of the target region in the input image and a position of the target pixel in the output image.
- the unit area may be a slice, a macro block, or a block.
- the fixed filter coefficient vector and the variable filter coefficient vector may be defined so as to include an offset component.
- the fixed filter coefficient vector can be decomposed into a fixed coefficient vector predetermined for each unit area constituting the output image and a base.
- variable filter coefficient vector further includes the fixed coefficient determined for each unit region constituting the output image, and the filter coefficient vector changing unit, and the variable coefficient coefficient vector in the input image. It is preferable that the image can be decomposed into a variable coefficient that is changed according to the position of the target region or the position of the target pixel in the output image, and a predetermined fixed vector.
- variable filter coefficient vector can be further decomposed into a fixed coefficient determined for each unit region constituting the output image, the variable coefficient, and the fixed vector. Further, the fixed vector is determined in advance, and the filter coefficient vector changing means sets the variable coefficient according to the position of the target region in the input image or the position of the target pixel in the output image. Since it can be changed, there is a further effect that more appropriate filtering can be performed according to the characteristics of the image in the target region.
- the fixed coefficient vector (or fixed filter coefficient vector) and the fixed coefficient determined for each unit region constituting the output image and the fixed coefficient are applied to the image filter, thereby adapting to each target region.
- the desired type of filtering can be performed.
- the number of target regions in the unit region is 10
- the number of components of the fixed coefficient vector (or fixed filter coefficient vector) is P
- the number of fixed coefficients is 1.
- a desired type of filtering adapted to 10 target regions can be performed by supplying a total of P + 1 coefficient data to the image filter.
- the fixed coefficient vector is calculated individually for each of the 10 target regions, 10 ⁇ P coefficient data is required. Therefore, according to the configuration described above, the type of filtering adapted to each target region is performed with a smaller number of coefficient data than when a fixed coefficient vector (or filter coefficient vector) is calculated for each target region. be able to.
- variable coefficient is changed according to the flatness of the image on the target region by the filter coefficient vector changing unit, and the fixed vector has an edge enhancement effect and a blur effect. It is preferable that the filter base has at least one effect.
- the fixed vector is a filter base having at least one of an edge enhancement effect and a blurring effect
- the filter coefficient changing unit determines the variable coefficient according to the position of the target region. It can be changed according to the flatness of the image on the target area. Therefore, according to the above configuration, more appropriate filtering can be performed by changing at least one of the edge enhancement effect and the blurring effect according to the flatness of the image on the target region. There is a further effect.
- the filter base having at least one of the edge enhancement effect and the blurring effect has an effect of blurring the edge instead of the effect of enhancing the edge according to the sign of the fixed coefficient (the same applies hereinafter). Therefore, according to said structure, there exists the further effect that more appropriate filtering can be performed by changing the edge blurring effect according to the flatness of the image on the said target area.
- variable coefficient is changed according to the angle of the edge of the image on the target region by the filter coefficient vector changing unit, and the fixed vector is used for edge enhancement effect and blurring. It is preferable that the filter base has at least one of the effects.
- the fixed vector is a filter base having at least one of an edge enhancement effect and a blurring effect
- the filter coefficient changing unit determines the variable coefficient according to the position of the target region. It can be changed according to the angle of the edge of the image on the target area. Therefore, according to the above configuration, the variable coefficient can be filtered more appropriately by changing at least one of the edge enhancement effect and the blurring effect according to the angle of the edge of the image on the target region. There is a further effect that it can be performed.
- variable coefficient is changed by the filter coefficient vector changing unit according to the luminance of the image on the target area, and the fixed vector has an edge enhancement effect and a blur effect.
- a filter base having at least one effect is preferable.
- the fixed vector is a filter base having at least one of an edge enhancement effect and a blurring effect
- the filter coefficient changing unit determines the variable coefficient according to the position of the target region. It can be changed according to the brightness of the image on the target area. Therefore, according to the above configuration, more appropriate filtering can be performed by changing at least one of the edge enhancement effect and the blur effect according to the luminance of the image on the target region. The effect which becomes.
- variable coefficient is changed according to the color of the image on the target area by the filter coefficient vector changing unit, and the fixed vector has an edge enhancement effect and a blur effect.
- a filter base having at least one effect is preferable.
- the fixed vector is a filter base having at least one of an edge enhancement effect and a blurring effect
- the filter coefficient changing unit determines the variable coefficient according to the position of the target region. It can be changed according to the color of the image on the target area. Therefore, according to the above configuration, more appropriate filtering can be performed by changing at least one of the edge enhancement effect and the blur effect according to the color of the image on the target region. The effect which becomes.
- variable coefficient is changed by the filter coefficient vector changing unit according to whether the target region includes a block boundary, and the fixed vector has an edge enhancement effect and It is preferable that the filter base has at least one of the blurring effects.
- the fixed vector is a filter base having at least one of an edge enhancement effect and a blurring effect
- the filter coefficient changing unit is configured to change the variable coefficient
- the target region has a block boundary. It can be changed depending on whether or not it is included. Therefore, according to the above configuration, more appropriate filtering can be performed by changing at least one of the edge enhancement effect and the blurring effect depending on whether the target region includes a block boundary. There is a further effect of being able to.
- variable coefficient is a block in which the block including the target region is unidirectionally predicted or bidirectionally predicted by the filter coefficient vector changing unit.
- the fixed vector is preferably a filter base having at least one of an edge enhancement effect and a blurring effect.
- the fixed vector is a filter base having at least one of an edge enhancement effect and a blurring effect
- the filter coefficient changing unit includes the variable coefficient as a block including the target region. It can be changed depending on whether it is a unidirectionally predicted block or a bidirectionally predicted block. Therefore, according to the above configuration, at least one of the edge enhancement effect and the blurring effect depends on whether the block including the target region is a unidirectionally predicted block or a bidirectionally predicted block. By changing the effect, there is a further effect that more appropriate filtering can be performed.
- variable coefficient is changed by the filter coefficient vector changing unit depending on whether the block including the target area is a block to which a skip mode is applied.
- the fixed vector is preferably a filter base having at least one of an edge enhancement effect and a blurring effect.
- the fixed vector is a filter base having at least one of an edge enhancement effect and a blurring effect
- the filter coefficient changing unit includes the variable coefficient as a block including the target region.
- the block can be changed depending on whether the block is applied with the skip mode. Therefore, according to the above configuration, by changing at least one of the edge enhancement effect and the blur effect according to whether or not the block including the target region is a block to which the skip mode is applied, There is a further effect that more appropriate filtering can be performed.
- variable coefficient is determined depending on whether the block including the target region is an inter predicted block or an intra predicted block by the filter coefficient vector changing unit.
- the fixed vector is preferably a filter base having at least one of an edge enhancement effect and a blurring effect.
- the fixed vector is a filter base having at least one of an edge enhancement effect and a blurring effect
- the filter coefficient changing unit includes the variable coefficient as a block including the target region. It can be changed depending on whether it is an inter predicted block or an intra predicted block. Therefore, according to the above configuration, at least one of the edge enhancement effect and the blurring effect is obtained depending on whether the block including the target region is an inter predicted block or an intra predicted block. By changing it, there is a further effect that more appropriate filtering can be performed.
- each component of the fixed filter coefficient vector and the fixed coefficient are a specific image and a decoded image obtained by encoding / decoding the specific image as an input image. It is preferable that the difference between the output image and the output image is set to be minimized.
- each component of the fixed filter coefficient vector and the fixed coefficient are obtained when a specific image and a decoded image obtained by encoding / decoding the specific image are input images. Since it is determined to minimize the difference from the output image, each component of the fixed filter coefficient vector optimized for the entire unit area, and the variable coefficient vector calculated using the fixed coefficient There is a further effect that filtering can be performed using.
- the fixed filter coefficient vector and the variable coefficient vector are orthogonal to each other.
- the fixed filter coefficient vector and the variable coefficient vector are orthogonal to each other, compared to a case where the fixed filter coefficient vector and the variable coefficient vector are not orthogonal to each other, The degree of freedom of the fixed filter coefficient vector can be reduced. Therefore, according to said structure, there exists an effect that suitable filtering can be performed for every said object area
- the fixed filter coefficient vector and the variable coefficient vector may not be orthogonal to each other.
- a decoding device is characterized by including the image filter, and causing the image filter to act on the decoded image.
- the decoding device configured as described above has the same effect as the image filter.
- the decoding device configured as described above uses each component of the fixed coefficient vector (or fixed coefficient vector) and the fixed coefficient determined for each unit area, and uses the fixed coefficient for each target area. Appropriate filtering can be performed, so that filter coefficient vectors can be derived using a smaller number of coefficient data than when using fixed coefficient vectors (or fixed coefficient vectors) having different components for each target region. There is an effect.
- the coefficient data (filter coefficient vector) can be compared with the case of using a fixed coefficient vector (or fixed filter coefficient vector) having a different component for each target region. There is an effect that the amount of codes can be reduced.
- an encoding device is characterized by including the above-described image filter, and causing the image filter to act on a decoded image.
- the encoding device configured as described above can perform filtering by the image filter on a decoded image that has been encoded and re-decoded by the encoding device. The same effect is produced.
- the encoding apparatus configured as described above performs more appropriate filtering for each target region using each component of the fixed coefficient vector and the fixed coefficient that are determined for each unit region. Therefore, the filter coefficient vector can be derived using a small number of coefficient data, compared to the case where a fixed coefficient vector (or a fixed filter coefficient vector) having a different component for each target region is used.
- the coefficient data (filter coefficient vector) can be compared with the case of using a fixed coefficient vector (fixed filter coefficient vector) having a different component for each target region. There is an effect that the amount of codes can be reduced.
- a data structure of encoded data according to the present invention is a data structure of encoded data that is referred to by a decoding device that includes the image filter and applies the image filter to a decoded image. It includes a fixed coefficient vector that represents each component or the fixed filter coefficient vector, and the fixed coefficient, or the fixed filter coefficient vector, the fixed coefficient vector, and the prediction difference data of the fixed coefficient. Yes.
- each component of the fixed filter coefficient vector decoded from the encoded data in the decoding device that decodes an image based on the encoded data, each component of the fixed filter coefficient vector decoded from the encoded data, And, by using the fixed coefficient, it is possible to perform more appropriate filtering for each target area while using a smaller number of filter coefficients than when including different filter coefficients for each target area. Play.
- the present invention can be suitably used for an image filter for filtering image data.
- the present invention also relates to a broadcast receiving terminal that decodes encoded image data, an HDD recorder that encodes image data and records it on a recording medium, and decodes image data recorded on the recording medium. It can be used suitably.
- Adaptive filter image filter
- Filter unit 120 Control variable determining unit (filter coefficient vector changing means) 1 Video encoding device (encoding device) 2 Video decoding device (decoding device) 16, 26 Inter prediction image generation unit R Filter reference area T Filter target area
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Image Processing (AREA)
- Compression Of Band Width Or Redundancy In Fax (AREA)
Abstract
Description
(スキップモードであるか否かに応じてフィルタ係数の切り替えを行う例)
以下では、上述した記法を用いて、適応フィルタ100によるフィルタ係数ベクトルの算出、および、切り替え方法の例について説明する。特に、適応フィルタ100が、画像データ#100の座標(x、y)における画素が、H.246/AVC規格におけるスキップモードが適用されたマクロブロックに属しているか否かに応じて、フィルタ係数ベクトルを切り替える場合を例に挙げ説明する。
まず、制御変数決定部120は、画像データ#100の座標(x、y)における画素が、スキップモードが適用されたマクロブロックに属しているか否かに応じて0または1をとる制御変数v_skipの値を1に設定し、当該制御変数v_skipをフィルタ部110に対して出力する。
フィルタ部110は、スライスごと、または、フレームごとに、フィルタ係数ベクトルVを決定する。ここで、フィルタ係数ベクトルVの各成分v(k)は、
v(K)=w(K)+a_skip×v_skip×b_skip(K) …(6)
によって表される。ここで、w(K)は、フィルタ係数ベクトルW(固定係数ベクトル)の各成分であり、b_skip(K)は、上述したフィルタ基底(固定ベクトル)b_skipの各成分である。また、v_skipは上述した制御変数(可変係数)であり、ステップS1において、v_skip=1に設定された制御変数である。また、a_skipは、制御変数v_skipに関連付けられた係数(固定係数)である。
v(M×N+2)=a_skip
としてVの次元をM×N+2次元に拡張し、SIのM×N+2番目の項を
SI(M×N+2)=Σv_skip×b_skip(K)×SI(K)
としてSIの次元を拡張すればよい。このような表現を用いれば、式(7)を最小化するM×N+2個のvの値を求める問題となる。
制御変数決定部120は、画像データ#100の座標(x、y)における画素が、スキップモードが適用されたマクロブロックに属しているか否かに応じて、制御変数v_skipの値を設定する。具体的には、制御変数決定部120は、画像データ#100の座標(x、y)における画素が、スキップモードが適用されたマクロブロックに属しているときに、v_skip=1に設定し、そうでないときに、v_skip=0に設定する。
フィルタ部110は、制御変数決定部120から供給される制御変数v_skipの値を用いたフィルタ係数ベクトルVの各成分に基づいて、画像データ#100に含まれる画素値の加重平均をとることによって、出力画像データ#110aの各画素値を算出する。具体的には、フィルタ部110は、フィルタ係数ベクトルVの各成分v(K)を式(4)に用いることによって、画素値SO(X’)を算出する。
(w(1)、w(2)、w(3)、w(4)-a_skip×v_skip、w(5)+2×a_skip×v_skip、w(6)-a_skip×v_skip、w(7)、w(8)、w(9)、w(10))
上式から明らかなように、b_skipとして横方向エッジ強調基底(0、0、0、-1、2、-1、0、0、0、0)をとった場合には、フィルタ係数ベクトルVの第1成分から第3成分、および、第7成分から第10成分は、制御変数v_skipに依存しない。すなわち、フィルタ係数ベクトルVの第1成分から第3成分、および、第7成分から第10成分の各々は、スキップモードであるか否かに関わらず、上記統計的手法によって定められた一定の値をとる。
また、式(6)のフィルタ係数導出処理は、制御変数によって変化する成分と、変化しない成分とに場合を分けて求めることが可能である。以下では再度、3×3タップのフィルタリングの場合であって、b_skipとして横方向エッジ強調基底(0、0、0、-1、2、-1、0、0、0、0)を用いる場合を例に挙げ、図3を参照して、適応フィルタ100によるフィルタ係数導出処理について説明する。
まず、適応フィルタ100は、制御変数によって変化するフィルタ係数ベクトルに対し、式(6)を用いて予めw´(i)を算出し保持しておく。
w´(4)=w(4)+a_skip×v_skip×b_skip(4)
=w(4)+a_skip×v_skip×-1
w´(5)=w(5)+a_skip×v_skip×2
w´(6)=w(6)+a_skip×v_skip×-1
(本ステップ)
(ステップS101)ループ1:フィルタ対象領域T毎に、出力画像を構成する単位領域全体に対して、S102からS109の処理を行う。
(ステップS109)導出したフィルタ係数ベクトルVを用いて、式(4)に示すフィルタリングを行う。
上記では、フィルタ係数ベクトルVの導出を、制御変数によって変化する成分との和を用いることなく分岐で行っている。この和の処理と、分岐の処理は、具体的な処理としては別に見えるが、処理の結果としては全く同一であり、本発明はどちらも含むものとする。分岐に用いる制御変数については上記例に限らず、下記に示す例を含む何れの例でも構わない。
式(6)は、フィルタ算出関数Fの一例である。固定係数ベクトルwi(1≦i≦Nw、Nwは固定係数ベクトルの次元数)、制御変数vk(1≦k≦Nv、Nvは制御変数の総数)、固定係数akj(kは上記のとおり、1≦j≦Nak、Nakは制御変数Vkに関する固定係数の総数)、とした場合、関数Fは一般に、以下の式(8)のように表現される。
…(8)
ここで、フィルタ算出関数Fは、第一項の固定フィルタ係数ベクトルと、第二項の可変フィルタ係数ベクトルに分解可能である構造をもつ。なお、F1は、固定係数ベクトルの関数、F2は、制御変数と固定係数との関数である。ここで、制御変数及び固定係数の数も複数でよいし、さらに、制御変数が複数の場合には、制御変数によって固定係数の数を変更しても構わない。
既に説明した通り、式(7)のフィルタパラメータVを、フィルタ算出関数Fを用いてフィルタパラメータの関数にした上で、得られた式を偏微分して得られる値を0と置いて得られる正規方程式を解く。
v(K)=w(K)+a_skip×v_skip×b_skip(K)+a_dc×v_dc×b_dc(K) …(11)
ここで、a_dcは、DCオフセット基底に対応したDCオフセット係数(固定係数)である。
v(K)=w(K)+a_skip0×v_skip×b_skip0(K)+a_skip1×v_skip×b_skip1(K) …(12)
ここで、a_skip0は、エッジ強調基底b_skip0(K)に対応したエッジ強調係数(固定係数)であり、a_skip1は、b_skip1(K)に対応したエッジ強調係数(固定係数)である。
まず、画像データ#100のフィルタ対象領域Tにおける画像の平坦度に応じて、フィルタ係数を切り替える場合について説明する。ここで、フィルタ対象領域Tにおける画像の平坦度とは、フィルタ対象領域Tに含まれる各画素の画素値のばらつきに応じて定められる指標のことである。フィルタ対象領域Tにおける画像に含まれる各画素の画素値のばらつきがより小さい場合とは、当該領域における画像の平坦度がより大きい場合のことであり、フィルタ対象領域Tにおける画像に含まれる各画素の画素値のばらつきがより大きい場合とは、当該領域における画像の平坦度がより小さい場合のことである。なお、フィルタ対象領域Tの制御変数を求める際に、フィルタ対象領域Tの全画素を用いる必要はない(以下同様)。ここでは領域Tよりも小さい矩形領域MV×NVを用いて平坦度を求めても良い。
v(K)=w(K)+a_flat×v_flat×b_flat(K) …(13)
ここで、a_flatは、フィルタ基底b_flatに対応した係数(固定係数)である。
w´(4)=w(4)+a_flat×v_flat×b_flat(4)
=w(4)+a_flat×v_flat×-1
w´(5)=w(5)+a_flat×v_flat×2
w´(6)=w(6)+a_flat×v_flat×-1
また、上記<フィルタ係数算出処理>における他のステップにおいては、制御変数v_skipを制御変数v_flatと読み替えればよい。
次に、画像データ#100のフィルタ対象領域Tにおける画像に含まれるエッジの角度に応じて、フィルタ係数を切り替える場合について説明する。
・フィルタ対象領域Tに垂直方向のエッジが含まれているか否かに応じて0または1をとる制御変数v_edge_0
・フィルタ対象領域Tに水平方向のエッジが含まれているか否かに応じて0または1をとる制御変数v_edge_90
・フィルタ対象領域Tに斜め右上がりのエッジが含まれているか否かに応じて0または1をとる制御変数v_edge_45
・フィルタ対象領域Tに斜め右下がりのエッジが含まれているか否かに応じて0または1をとる制御変数v_edge_135
また、制御変数決定部120は、以下のフィルタ基底をフィルタ部12に対して出力する。
・フィルタ対象領域Tに垂直方向のエッジが含まれているか否かの情報に関連付けられたフィルタ基底b_edge_0
・フィルタ対象領域Tに水平方向のエッジが含まれているか否かの情報に関連付けられたフィルタ基底b_edge_90
・フィルタ対象領域Tに斜め右上がりのエッジが含まれているか否かの情報に関連付けられたフィルタ基底b_edge_45
・フィルタ対象領域Tに斜め右下がりのエッジが含まれているか否かの情報に関連付けられたフィルタ基底b_edge_135
なお、フィルタ基底b_edge_n(n=0、45、90、135)は、例えば、3×3タップのフィルタリングを行う場合、以下のようにとればよい。
・b_edge_0=(0、-1、0、0、2、0、0、-1、0、0)
・b_edge_90=(0、0、0、-1、2、-1、0、0、0、0)
・b_edge_45=(0、0、-1、0、2、0、-1、0、0、0)
・b_edge_135=(-1、0、0、0、2、0、0、0、-1、0)
フィルタ部110は、以下の式(14)によって表現される各成分v(K)を有するフィルタ係数ベクトルVの各係数を教師データ#200を参照して決定する。
v(K)=w(K)+Σa_edge_n×v_edge_n×b_edge_n(K) …(14)
ここで、記号Σは、エッジの角度を表す添え字n(n=0、45、90、135)に関する和を表している。また、a_edge_n(n=0、45、90、135)は、それぞれ、フィルタ基底b_edge_n(n=0、45、90、135)に対応した係数である。
・フィルタ対象領域Tに垂直方向のエッジが含まれているときにv_edge_0=1、そうでないときにv_edge_0=0
・フィルタ対象領域Tに水平方向のエッジが含まれているときにv_edge_90=1、そうでないときにv_edge_90=0
・フィルタ対象領域Tに斜め右上がりのエッジが含まれているときにv_edge_45=1、そうでないときにv_edge_45=0
・フィルタ対象領域Tに斜め右下がりのエッジが含まれているときにv_edge_135=1、そうでないときにv_edge_45=0
フィルタ部110は、制御変数決定部120から供給される制御変数v_edge_n(n=0、45、90、135)を用いたフィルタ係数ベクトルVに基づいて、フィルタリングを行う。
・フィルタ対象領域Tに垂直方向のエッジが含まれているか否かの情報に関連付けられた第1のフィルタ基底b_edge_0_0=b_edge_0
・フィルタ対象領域Tに垂直方向のエッジが含まれているか否かの情報に関連付けられた第2のフィルタ基底b_edge_0_1=b_edge_90
・フィルタ対象領域Tに水平方向のエッジが含まれているか否かの情報に関連付けられた第1のフィルタ基底b_edge_90_0=b_edge_90
・フィルタ対象領域Tに水平方向のエッジが含まれているか否かの情報に関連付けられた第2のフィルタ基底b_edge_90_1=b_edge_0
・フィルタ対象領域Tに斜め右上がりのエッジが含まれているか否かの情報に関連付けられた第1のフィルタ基底b_edge_45_0=b_edge_45
・フィルタ対象領域Tに斜め右上がりのエッジが含まれているか否かの情報に関連付けられた第2のフィルタ基底b_edge_45_1=b_edge_135
・フィルタ対象領域Tに斜め右下がりのエッジが含まれているか否かの情報に関連付けられた第1のフィルタ基底b_edge_135_0=b_edge_135
・フィルタ対象領域Tに斜め右下がりのエッジが含まれているか否かの情報に関連付けられた第1のフィルタ基底b_edge_135_1=b_edge_45
この場合、フィルタ部110は、以下の式(15)によって表現される各成分v(K)を有するフィルタ係数ベクトルVの各係数を教師データ#200を参照して決定する。
v(K)=w(K)+Σa_edge_n_0×v_edge_n×b_edge_n_0(K)+Σa_edge_n_1×v_edge_n×b_edge_n_1(K) …(15)
ここで、記号Σは、エッジの角度を表す添え字n(n=0、45、90、135)に関する和を表している。また、a_edge_n_0(n=0、45、90、135)、および、a_edge_n_1(n=0、45、90、135)は、それぞれ、フィルタ基底b_edge_n_0(n=0、45、90、135)、および、フィルタ基底b_edge_n_1(n=0、45、90、135)に対応した係数(固定係数)である。
w´(4)=w(4)+a_edge_90×v_edge_90×b_edge_90(4)=w(4)+a_edge_90×v_edge_90×-1
w´(5)=w(5)+a_edge_90×v_edge_90×2
w´(6)=w(6)+a_edge_90×v_edge_90×-1
また、上記<フィルタ係数算出処理>における他のステップにおいては、制御変数v_skipを制御変数v_edge_90と読み替えればよい。また、フィルタ基底b_edge_nとして他の基底をとった場合も同様である。また、式(14)において、nについての和をとる場合に対しても容易に一般化できる。
次に、画像データ#100のフィルタ対象領域Tにおける画像の平均輝度に応じて、フィルタ係数を切り替える場合について説明する。ここで、フィルタ対象領域Tにおける画像の平均輝度とは、フィルタ対象領域Tに含まれる各画素の輝度値の平均のことである。また、各画素の輝度値は、一般に、当該画素の画素値の関数として求めることができる。
v(K)=w(K)+a_lum×v_lum×b_lum(K) …(16)
ここで、a_lumは、フィルタ基底b_lumに対応した係数(固定係数)である。
w´(4)=w(4)+a_lum×v_lum×b_lum(4)
=w(4)+a_lum×v_lum×-1
w´(5)=w(5)+a_lum×v_lum×2
w´(6)=w(6)+a_lum×v_lum×-1
また、上記<フィルタ係数算出処理>における他のステップにおいては、制御変数v_skipを制御変数v_lumと読み替えればよい。また、フィルタ基底b_lumとして他の基底をとった場合も同様である。
次に、画像データ#100のフィルタ対象領域Tにおける画像の色に応じて、フィルタ係数を切り替える場合について説明する。特に、フィルタ対象領域Tにおける画素が肌色を示しているか否かに応じて、フィルタ係数を切り替える場合について説明する。ここで、フィルタ対象領域Tにおける画素が肌色を示している場合とは、例えば、フィルタ対象領域Tに含まれる画素が表す色の平均が、YUV成分で示して、133≦V≦177、かつ、77≦U≦127、かつ、60≦Y≦145の範囲内である場合である。ここで、Yは、当該画素の表示する色成分のうち、輝度成分の大きさを示しており、Uは、青色成分と輝度成分との差の定数倍によって定義される色成分の大きさを示しており、Vは、赤色成分と輝度成分との差の定数倍によって定義される色成分の大きさを示している。なお、上記の例では、Y、U、Vのそれぞれは、0から255までの値をとるものとしている。
v(K)=w(K)+a_skin×v_skin×b_skin(K) …(17)
ここで、a_skinは、フィルタ基底b_skinに対応した係数(固定係数)である。
w´(4)=w(4)+a_skin×v_skin×b_skin(4)
=w(4)+a_skin×v_skin×-1
w´(5)=w(5)+a_skin×v_skin×2
w´(6)=w(6)+a_skin×v_skin×-1
また、上記<フィルタ係数算出処理>における他のステップにおいては、制御変数v_skipを制御変数v_skinと読み替えればよい。また、フィルタ基底b_skinとして他の基底をとった場合も同様である。
次に、フィルタ対象領域Tがブロックの境界を含んでいるか否かに応じてフィルタ係数を切り替える場合について説明する。
v(K)=w(K)+a_block×v_block×b_block(K) …(18)
ここで、a_blockは、フィルタ基底b_blockに対応した係数(固定係数)である。
w´(4)=w(4)+a_block×v_block×b_block(4)
=w(4)+a_block×v_block×-1
w´(5)=w(5)+a_block×v_block×2
w´(6)=w(6)+a_block×v_block×-1
また、上記<フィルタ係数算出処理>における他のステップにおいては、制御変数v_skipを制御変数v_blockと読み替えればよい。また、フィルタ基底b_blockとして他の基底をとった場合も同様である。
次に、画像データ#100の座標(x、y)における画素が属するブロックに関連付けられたサイド情報に応じて、フィルタ係数を切り替える場合について説明する。特に、画像データ#100の座標(x、y)における画素が属するブロックが、例えば、H.264/AVC規格における単方向予測に基づいて生成されたブロックであるか、または、双方向予測に基づいて生成されたブロックであるかに応じて、フィルタ係数を切り替える場合について説明する。
v(K)=w(K)+a_pred×v_pred×b_pred(K) …(19)
ここで、a_predは、フィルタ基底b_predに対応した係数(固定係数)である。
w´(4)=w(4)+a_pred×v_pred×b_pred(4)
=w(4)+a_pred×v_pred×-1
w´(5)=w(5)+a_pred×v_pred×2
w´(6)=w(6)+a_pred×v_pred×-1
また、上記<フィルタ係数算出処理>における他のステップにおいては、制御変数v_skipを制御変数v_predと読み替えればよい。また、フィルタ基底b_predとして他の基底をとった場合も同様である。
次に、画像データ#100の座標(x、y)における画素が属するブロックが、H.264/AVC規格におけるイントラ予測されたブロックであるか、インター予測されたブロックであるかに応じてフィルタ係数を切り替える場合について説明する。
v(K)=w(K)+a_int×v_int×b_int(K) …(20)
ここで、a_intは、フィルタ基底b_intに対応した係数(固定係数)である。
w´(4)=w(4)+a_int×v_int×b_int(4)
=w(4)+a_int×v_int×-1
w´(5)=w(5)+a_int×v_int×2
w´(6)=w(6)+a_int×v_int×-1
また、上記<フィルタ係数算出処理>における他のステップにおいては、制御変数v_skipを制御変数v_intと読み替えればよい。また、フィルタ基底b_intとして他の基底をとった場合も同様である。
次に、画像データ#100の座標(x、y)における画素と、当該画素が属するブロックのブロック境界との距離に応じて、フィルタ係数を切り替える場合について説明する。
v(K)=w(K)+a_dist×v_dist×b_dist(K) …(21)
ここで、a_distは、フィルタ基底b_distに対応した係数(固定係数)である。
w´(4)=w(4)+a_dist×v_dist×b_dist(4)
=w(4)+a_dist×v_dist×-1
w´(5)=w(5)+a_dist×v_dist×2
w´(6)=w(6)+a_dist×v_dist×-1
また、上記<フィルタ係数算出処理>における他のステップにおいては、制御変数v_skipを制御変数v_distと読み替えればよい。また、フィルタ基底b_distとして他の基底をとった場合も同様である。
次に、画像データ#100の座標(x、y)における画素のフレーム内における位置に応じて、フィルタ係数を切り替える場合について説明する。
v(K)=w(K)+a_xy×v_xy×b_xy(K) …(22)
ここで、a_xyは、フィルタ基底b_xyに対応した係数(固定係数)である。
w´(4)=w(4)+a_xy×v_xy×b_xy(4)
=w(4)+a_xy×v_xy×-1
w´(5)=w(5)+a_xy×v_xy×2
w´(6)=w(6)+a_xy×v_xy×-1
また、上記<フィルタ係数算出処理>における他のステップにおいては、制御変数v_skipを制御変数v_xyと読み替えればよい。また、フィルタ基底b_xyとして他の基底をとった場合も同様である。
次に、画像データ#100の座標(x、y)における画素が属するブロックに関連付けられた量子化パラメータQPに応じて、フィルタ係数を切り替える場合について説明する。
v(K)=w(K)+a_qp×v_qp×b_qp(K) …(23)
ここで、a_qpは、フィルタ基底b_qpに対応した係数(固定係数)である。
w´(4)=w(4)+a_qp×v_qp×b_qp(4)
=w(4)+a_qp×v_qp×-1
w´(5)=w(5)+a_qp×v_qp×2
w´(6)=w(6)+a_qp×v_qp×-1
また、上記<フィルタ係数算出処理>における他のステップにおいては、制御変数v_skipを制御変数v_qpと読み替えればよい。また、フィルタ基底b_qpとして他の基底をとった場合も同様である。
次に、画像データ#100の座標(x、y)における画素が属するブロックに関連付けられたDC成分の大きさに応じて、フィルタ係数を切り替える場合について説明する。
v(K)=w(K)+a_dc’×v_dc’×b_dc’(K) …(24)
ここで、a_dc’は、フィルタ基底b_dc’に対応した係数(固定係数)である。
w´(4)=w(4)+a_dc’×v_dc’×b_dc’(4)
=w(4)+a_dc’×v_dc’×-1
w´(5)=w(5)+a_dc’×v_dc’×2
w´(6)=w(6)+a_dc’×v_dc’×-1
また、上記<フィルタ係数算出処理>における他のステップにおいては、制御変数v_skipを制御変数v_dc’と読み替えればよい。また、フィルタ基底b_dc’として他の基底をとった場合も同様である。
次に、画像データ#100の座標(x、y)における画素が属するブロックに関連付けられた動きベクトルの大きさに応じて、フィルタ係数を切り替える場合について説明する。
v(K)=w(K)+a_mv×v_mv×b_mv(K) …(25)
ここで、a_mvは、フィルタ基底b_mvに対応した係数(固定係数)である。
w´(4)=w(4)+a_mv×v_mv×b_mv(4)
=w(4)+a_mv×v_mv×-1
w´(5)=w(5)+a_mv×v_mv×2
w´(6)=w(6)+a_mv×v_mv×-1
また、上記<フィルタ係数算出処理>における他のステップにおいては、制御変数v_skipを制御変数v_mvと読み替えればよい。また、フィルタ基底b_mvとして他の基底をとった場合も同様である。
まず、適応フィルタ100を備えている動画像符号化装置1について図4を参照して説明する。
ここで、フィルタ係数情報#101の符号化について、図6の(a)~(c)を参照して具体的に説明する。図6の(a)~(c)は、それぞれ、フィルタ係数情報#101の符号化データの構成を示す図である。
また、本発明に係る適応フィルタ100は、動画像符号化装置1におけるインター予測画像生成部16に対して用いることもできる。
バッファメモリ14に格納された参照画像#14bが、適応フィルタ100’に入力される。なお、以下では、参照信号#14bを、第1の参照信号#14bと呼ぶことにする。
動きベクトル推定部17は、第1の出力画像データ#110a’と、入力画像#1とに基づいて、第1の動きベクトル#17’を生成する。第1の動きベクトル#17’は1スライスもしくは1フレーム分行われる。
適応フィルタ100’は、第1の参照画像#14bに対して、<適応フィルタ100によるフィルタリングの例>において説明した処理によってフィルタ係数を求める。適応フィルタ100’によるフィルタリングで生成される画像を、第2の出力画像データ#110a’’と呼ぶ。ここで、第1の参照画像#14bは、上述した画像データ#100に対応しており、第2の出力画像データ#110a’’は、上述した出力画像データ#110aに対応している。また、入力画像#1は、上述した教師データ#200として、適応フィルタ100’に入力される。
動きベクトル推定部17は、第2の出力画像データ#110a’’と、入力画像#1とに基づいて、第2の動きベクトル#17’’を生成する。
適応フィルタ100’は、第1の参照画像#14bに対して求めたフィルタ係数によるフィルタリングを行い、第2の動きベクトル#17’’に基づいた動き補償画像を生成することにより、インター予測画像#16を生成し出力する。
以下では、適応フィルタ100を備えている動画像復号装置2について図8を参照して説明する。
<図6の(a)の例に代表される場合>
固定フィルタパラメータもしくはその予測差分データが符号化されている場合には、可変長符号復号部23は、フィルタ係数情報#23dとして、固定フィルタパラメータを復号する。図6の(a)ではw(1)~w(M×N+1)とa_skipを復号する。
フィルタ係数もしくはその予測差分データが符号化されている場合には、可変長符号復号部23はフィルタ係数情報#23dとして、フィルタ係数を復号する。なお、図6の(b)の例では、制御変数によって変化しない成分については1つのフィルタ係数、制御変数によって変化する成分は、制御変数の値毎に複数のフィルタ係数が符号化されている。
可変長符号復号部23は、制御変数によって変化するかどうかによって処理を分岐することでフィルタ係数を導出することができる。図6の(b)の例では、制御変数によって変化しないフィルタ係数ベクトルの要素、v(i)、i=1、2、3、7、8、9、10をまず復号する。次いで、制御変数によって変化するフィルタ係数ベクトルの要素について、制御変数v_skipが0の場合に対応するフィルタ係数v(i)、i=4、5、6と、制御変数v_skipが1の場合に対応するフィルタ係数v´(i)、i=4、5、6を復号する。さらに、制御変数によって変化しないフィルタ係数ベクトルの要素についてはv´(i)=v(i)とする。
基本のフィルタ係数および基本のフィルタ係数との差分、もしくは、それらの予測差分データが符号化されている。可変長符号復号部23はフィルタ係数情報#23dとして、基本のフィルタ係数を復号し、その後、基本のフィルタ係数との差分を復号する。図6の(c)の例では、基本のフィルタ係数としてv(i)、i=1~10を復号する。これらは、制御変数v_skipが0の場合に対応するフィルタ係数ベクトルVとなる。次いで、制御変数によって変化するフィルタ係数ベクトルの要素について、基本の差分との差分であるdV(i)、i=4、5、6を復号する。制御変数v_skipが1の場合に対応するフィルタ係数v´(i)は、制御変数によって変化しない成分の場合は、基本のフィルタ係数v(i)を用いて、以下により導出する。
制御変数によって変化する成分の場合は、基本のフィルタ係数v(i)と、基本のフィルタ係数との差分dV(i)を用いて、以下により導出する。
このように、制御変数によって変化するかどうかで成分毎に分岐を行うことでフィルタ係数を導出することができる。
動画像復号装置2においても、適応フィルタ100は図3、図4で既に説明した通りの動作を行う。フィルタ係数ベクトルを分岐により導出する場合には、図3に示すフローになり、フィルタ係数ベクトルを式(9)のような和により導出する場合には、図4に示すフローになる。なお、フィルタ係数ベクトルを和により導出する場合にも予備ステップを用いて、図3のフローで処理しても良いし、予めフィルタ係数の成分の位置と制御変数の値によって定まるテーブルを導出し、テーブル引きにより処理しても構わない。
上記では、復号するフィルタ係数情報#23dとして、フィルタ係数、もしくは、フィルタ係数の差分、もしくは、フィルタ係数の増加度を復号していたが、それらの予測値を導出し、予測値との差分を復号する構成としても良い。
また、本発明に係る適応フィルタ100は、動画像復号装置2におけるインター予測画像生成部26に対して用いることもできる。
以下では、動画像符号化装置1を用いて生成した符号化データ#2の構成について、図10および図11を参照して説明する。
以上のように、本発明に係る画像フィルタは、入力画像における対象領域に属する各画素の画素値とフィルタ係数ベクトルとから、出力画像における対象画素の画素値を算出する画像フィルタであって、上記入力画像における上記対象領域の位置、及び、上記出力画像における上記対象画素の位置の少なくとも何れかに応じて上記フィルタ係数ベクトルを変更するフィルタ係数ベクトル変更手段を備えている、ことを特徴としている。
110 フィルタ部
120 制御変数決定部(フィルタ係数ベクトル変更手段)
1 動画像符号化装置(符号化装置)
2 動画像復号装置(復号装置)
16、26 インター予測画像生成部
R フィルタ参照領域
T フィルタ対象領域
Claims (13)
- 入力画像における対象領域に属する各画素の画素値とフィルタ係数ベクトルとから、出力画像における対象画素の画素値を算出する画像フィルタであって、
上記入力画像における上記対象領域の位置、及び、上記出力画像における上記対象画素の位置の少なくとも何れかに応じて上記フィルタ係数ベクトルを変更するフィルタ係数ベクトル変更手段を備えている、
ことを特徴とする画像フィルタ。 - 上記フィルタ係数ベクトルが、上記出力画像を構成する単位領域毎に予め定められた固定フィルタ係数ベクトルと、上記フィルタ係数ベクトル変更手段によって、上記入力画像における上記対象領域の位置、及び、上記出力画像における上記対象画素の位置の少なくとも何れかに応じて変更される可変フィルタ係数ベクトルとの和に分解可能である、
ことを特徴とする請求項1に記載の画像フィルタ。 - 上記可変フィルタ係数ベクトルは、更に、
上記出力画像を構成する単位領域毎に定められた固定係数と、上記フィルタ係数ベクトル変更手段によって、上記入力画像における上記対象領域の位置、又は、上記出力画像における上記対象画素の位置に応じて変更される可変係数と、予め定められた固定ベクトルと、に分解可能である、
ことを特徴とする請求項2に記載の画像フィルタ。 - 上記可変係数は、上記フィルタ係数ベクトル変更手段によって、上記対象領域上の画像の平坦度に応じて変更され、
上記固定ベクトルは、エッジ強調効果およびぼかし効果のうち少なくとも1つの効果を有するフィルタ基底である、
ことを特徴とする請求項3に記載の画像フィルタ。 - 上記可変係数は、上記フィルタ係数ベクトル変更手段によって、上記対象領域上の画像のエッジの角度に応じて変更され、
上記固定ベクトルは、エッジ強調効果およびぼかし効果のうち少なくとも1つの効果を有するフィルタ基底である、
ことを特徴とする請求項3または4に記載の画像フィルタ。 - 上記可変係数は、上記フィルタ係数ベクトル変更手段によって、上記対象領域上の画像の輝度に応じて変更され、
上記固定ベクトルは、エッジ強調効果およびぼかし効果のうち少なくとも1つの効果を有するフィルタ基底である、
ことを特徴とする請求項3から5の何れか1項に記載の画像フィルタ。 - 上記可変係数は、上記フィルタ係数ベクトル変更手段によって、上記対象領域を含むブロックがスキップモードが適用されたブロックであるか否かに応じて変更され、
上記固定ベクトルは、エッジ強調効果およびぼかし効果のうち少なくとも1つの効果を有するフィルタ基底である、
ことを特徴とする請求項3から6の何れか1項に記載の画像フィルタ。 - 上記可変係数は、上記フィルタ係数ベクトル変更手段によって、上記対象領域を含むブロックがインター予測されたブロックであるかまたはイントラ予測されたブロックであるかに応じて変更され、
上記固定ベクトルは、エッジ強調効果およびぼかし効果のうち少なくとも1つの効果を有するフィルタ基底である、
ことを特徴とする請求項3から7の何れか1項に記載の画像フィルタ。 - 上記固定フィルタ係数ベクトルの各成分、及び、上記固定係数は、特定の画像と、該特定の画像を符号化/復号化して得られる復号画像を入力画像としたときの出力画像との相違を最小化するように定められている、
ことを特徴とする請求項3から8の何れか1項に記載の画像フィルタ。 - 上記固定フィルタ係数ベクトルと上記可変フィルタ係数ベクトルとが互いに直交している、ことを特徴とする請求項2から9の何れか1項に記載の画像フィルタ。
- 請求項1から10の何れか1項に記載の画像フィルタを備え、復号画像に上記画像フィルタを作用させる、ことを特徴とする復号装置。
- 請求項1から10の何れか1項に記載の画像フィルタを備え、局所復号画像に上記画像フィルタを作用させる、ことを特徴とする符号化装置。
- 入力画像における対象領域に属する各画素の画素値とフィルタ係数ベクトルとから、出力画像における対象画素の画素値を算出する画像フィルタを備え、復号画像に当該画像フィルタを作用させる復号装置が参照する符号化データのデータ構造において、
上記入力画像における上記対象領域の位置、又は、上記出力画像における上記対象画素の位置に応じて変更される可変係数と、予め定められた固定ベクトルとの積に上記復号装置が乗じる固定係数、および、
上記可変係数と上記固定ベクトルと上記固定係数との積に上記復号装置が加算する固定フィルタ係数ベクトルであって、上記出力画像を構成する単位領域毎に予め定められた固定フィルタ係数ベクトルの各成分、
を含んでいる、ことを特徴とする符号化データのデータ構造。
Priority Applications (13)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR1020197007933A KR102060586B1 (ko) | 2009-12-18 | 2010-11-10 | 복호 장치 |
JP2011546040A JPWO2011074357A1 (ja) | 2009-12-18 | 2010-11-10 | 画像フィルタ、符号化装置、復号装置、および、データ構造 |
CN201080056671.XA CN102656888B (zh) | 2009-12-18 | 2010-11-10 | 图像滤波器、编码装置、解码装置以及数据结构 |
US13/515,957 US20120251012A1 (en) | 2009-12-18 | 2010-11-10 | Image filter, encoding device, decoding device, and data structure |
KR1020187009830A KR20180038076A (ko) | 2009-12-18 | 2010-11-10 | 복호 장치 |
CA2784291A CA2784291A1 (en) | 2009-12-18 | 2010-11-10 | Image filter, encoding device, decoding device, and data structure |
KR1020177010031A KR101849891B1 (ko) | 2009-12-18 | 2010-11-10 | 화상 필터 및 복호 장치 |
KR1020167022206A KR101729903B1 (ko) | 2009-12-18 | 2010-11-10 | 복호 장치 |
BR112012014685A BR112012014685A2 (pt) | 2009-12-18 | 2010-11-10 | filtro de imagem, dispositivo de codificação, dispositivo de decodificação e estrututa de dados. |
EP10837386.1A EP2515541A4 (en) | 2009-12-18 | 2010-11-10 | BILDFILTER, CODING DEVICE, DECODING DEVICE AND DATA STRUCTURE |
AU2010331511A AU2010331511B2 (en) | 2009-12-18 | 2010-11-10 | Image filter, encoding device, decoding device, and data structure |
US14/697,400 US9514519B2 (en) | 2009-12-18 | 2015-04-27 | Image filter |
US15/334,048 US9641865B2 (en) | 2009-12-18 | 2016-10-25 | Method for decoding moving images |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2009-288448 | 2009-12-18 | ||
JP2009288448 | 2009-12-18 |
Related Child Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/515,957 A-371-Of-International US20120251012A1 (en) | 2009-12-18 | 2010-11-10 | Image filter, encoding device, decoding device, and data structure |
US14/697,400 Continuation US9514519B2 (en) | 2009-12-18 | 2015-04-27 | Image filter |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2011074357A1 true WO2011074357A1 (ja) | 2011-06-23 |
Family
ID=44167122
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2010/070053 WO2011074357A1 (ja) | 2009-12-18 | 2010-11-10 | 画像フィルタ、符号化装置、復号装置、および、データ構造 |
Country Status (9)
Country | Link |
---|---|
US (3) | US20120251012A1 (ja) |
EP (2) | EP2515541A4 (ja) |
JP (5) | JPWO2011074357A1 (ja) |
KR (5) | KR20120105509A (ja) |
CN (1) | CN107071480A (ja) |
AU (1) | AU2010331511B2 (ja) |
BR (1) | BR112012014685A2 (ja) |
CA (2) | CA2784291A1 (ja) |
WO (1) | WO2011074357A1 (ja) |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2013090253A (ja) * | 2011-10-20 | 2013-05-13 | Nippon Hoso Kyokai <Nhk> | 画像処理装置及びプログラム |
JPWO2011086836A1 (ja) * | 2010-01-12 | 2013-05-16 | シャープ株式会社 | 符号化装置、復号装置、および、データ構造 |
JP2014179792A (ja) * | 2013-03-14 | 2014-09-25 | Nippon Telegr & Teleph Corp <Ntt> | 信号処理装置、及び信号処理方法 |
US9497455B2 (en) | 2011-06-23 | 2016-11-15 | Huawei Technologies Co., Ltd. | Offset decoding device, offset coding device, image filtering device, and data structure |
JP2019201256A (ja) * | 2018-05-14 | 2019-11-21 | シャープ株式会社 | 画像フィルタ装置 |
JP2020522170A (ja) * | 2017-05-31 | 2020-07-27 | インターデジタル ヴイシー ホールディングス, インコーポレイテッド | ピクチャの符号化および復号の方法およびデバイス |
JP2022105577A (ja) * | 2018-05-14 | 2022-07-14 | シャープ株式会社 | 画像フィルタ装置 |
Families Citing this family (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107707922A (zh) * | 2012-04-16 | 2018-02-16 | 寰发股份有限公司 | 具有分隔的符号和幅值的采样自适应偏移编码的方法和装置 |
JP6055391B2 (ja) * | 2012-11-05 | 2016-12-27 | 株式会社デンソーアイティーラボラトリ | 関連性判定装置、関連性判定プログラム、及び関連性判定方法 |
US9906790B2 (en) * | 2014-03-14 | 2018-02-27 | Qualcomm Incorporated | Deblock filtering using pixel distance |
GB2545601A (en) | 2014-09-03 | 2017-06-21 | Aira Tech Corp | Media streaming methods, apparatus and systems |
US9716871B2 (en) * | 2014-09-30 | 2017-07-25 | Apple Inc. | YUV signal generation for high dynamic range video content |
KR102525033B1 (ko) | 2015-11-11 | 2023-04-24 | 삼성전자주식회사 | 비디오 복호화 방법 및 그 장치 및 비디오 부호화 방법 및 그 장치 |
WO2018012851A1 (ko) * | 2016-07-12 | 2018-01-18 | 한국전자통신연구원 | 영상 부호화/복호화 방법 및 이를 위한 기록 매체 |
TWI604731B (zh) * | 2016-08-05 | 2017-11-01 | 瑞昱半導體股份有限公司 | 影像濾波方法及其影像濾波裝置 |
CN106846270B (zh) * | 2017-01-05 | 2020-02-14 | 浙江大华技术股份有限公司 | 一种图像边缘增强方法及装置 |
WO2019107182A1 (ja) * | 2017-12-01 | 2019-06-06 | ソニー株式会社 | 符号化装置、符号化方法、復号装置、及び、復号方法 |
KR102028908B1 (ko) | 2018-03-15 | 2019-11-08 | 써멀마스터 주식회사 | 전기히터 내장 구조의 열교환기 |
WO2019189346A1 (ja) * | 2018-03-30 | 2019-10-03 | パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ | 符号化装置、復号装置、符号化方法及び復号方法 |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2007143178A (ja) * | 2001-11-29 | 2007-06-07 | Matsushita Electric Ind Co Ltd | 符号化歪除去方法 |
JP2008500775A (ja) * | 2004-05-27 | 2008-01-10 | ソニー エレクトロニクス インク | 符号化方法、符号化装置、フィルタ生成方法及びフィルタ生成装置 |
Family Cites Families (33)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3288811B2 (ja) * | 1993-07-26 | 2002-06-04 | 日本無線株式会社 | ビデオコーデックにおける後処理フィルタ制御方式及び後処理フィルタ制御回路 |
JPH08163561A (ja) * | 1994-12-06 | 1996-06-21 | Matsushita Electric Ind Co Ltd | 画像データ圧縮装置 |
JPH0998417A (ja) * | 1995-09-29 | 1997-04-08 | Matsushita Electric Ind Co Ltd | 画像通信装置及び画像通信方法 |
KR100244290B1 (ko) | 1997-09-09 | 2000-02-01 | 구자홍 | 저속 전송에서의 동영상을 위한 디블록킹 필터링 방법 |
JP3943333B2 (ja) * | 1998-03-05 | 2007-07-11 | 松下電器産業株式会社 | 画像符号化方法、画像符号化復号化方法、画像符号化装置、及び画像記録再生装置 |
JP2001275110A (ja) * | 2000-03-24 | 2001-10-05 | Matsushita Electric Ind Co Ltd | 動的なループ及びポストフィルタリングのための方法及び装置 |
US7054500B1 (en) * | 2000-12-06 | 2006-05-30 | Realnetworks, Inc. | Video compression and decompression system with postfilter to filter coding artifacts |
EP1978748B1 (en) | 2001-11-29 | 2014-04-16 | Panasonic Corporation | Coding distortion removal method |
EP2320661B8 (en) | 2001-11-29 | 2015-09-02 | Godo Kaisha IP Bridge 1 | Coding distortion removal method |
MXPA04006814A (es) * | 2002-01-14 | 2004-12-06 | Nokia Corp | Filtros dinamicos de codificacion. |
US20060007239A1 (en) * | 2004-07-06 | 2006-01-12 | Harrison Charles F | Color correction system |
US7773158B2 (en) * | 2005-10-12 | 2010-08-10 | Panasonic Corporation | Visual processing device, display device, and integrated circuit |
EP2001239B1 (en) * | 2006-03-27 | 2017-09-13 | Sun Patent Trust | Picture coding apparatus and picture decoding apparatus |
JP4784386B2 (ja) * | 2006-05-01 | 2011-10-05 | 富士ゼロックス株式会社 | 復号化装置、逆量子化方法及びプログラム |
EP2061233B1 (en) * | 2006-09-14 | 2013-06-26 | Mitsubishi Electric Corporation | Image processing device and image processing method |
JP4785690B2 (ja) * | 2006-09-19 | 2011-10-05 | キヤノン株式会社 | 画像読取装置及び画像読取装置の制御方法 |
JP4872862B2 (ja) * | 2006-09-28 | 2012-02-08 | ソニー株式会社 | 画像データ演算装置および方法、プログラム、並びに記録媒体 |
KR100879536B1 (ko) * | 2006-10-30 | 2009-01-22 | 삼성전자주식회사 | 영상의 화질 개선을 위한 방법 및 시스템 |
KR100922275B1 (ko) | 2006-12-15 | 2009-10-15 | 경희대학교 산학협력단 | 경계 필터링 강도의 결정 방법 및 이를 이용한 디블록킹필터링 방법과 장치 |
US8509316B2 (en) * | 2007-01-09 | 2013-08-13 | Core Wireless Licensing, S.a.r.l. | Adaptive interpolation filters for video coding |
US8942505B2 (en) * | 2007-01-09 | 2015-01-27 | Telefonaktiebolaget L M Ericsson (Publ) | Adaptive filter representation |
EP1944974A1 (en) * | 2007-01-09 | 2008-07-16 | Matsushita Electric Industrial Co., Ltd. | Position dependent post-filter hints |
JP4978402B2 (ja) * | 2007-09-28 | 2012-07-18 | 富士通セミコンダクター株式会社 | 画像処理フィルタ、画像処理フィルタの画像処理方法及び画像処理フィルタを備える画像処理装置の画像処理回路 |
MX2010009194A (es) * | 2008-03-07 | 2010-09-10 | Toshiba Kk | Metodo y dispositivo de codificacion/decodificacion dinamica de imagen. |
US8195001B2 (en) * | 2008-04-09 | 2012-06-05 | Intel Corporation | In-loop adaptive wiener filter for video coding and decoding |
AU2009240883A1 (en) * | 2008-04-30 | 2009-11-05 | Kabushiki Kaisha Toshiba | Time-varying image encoding and decoding device |
WO2009133844A1 (ja) * | 2008-04-30 | 2009-11-05 | 株式会社 東芝 | エッジを考慮したフィルタリング機能を備えた動画像符号化/復号化方法及び装置 |
EP2457196A4 (en) * | 2009-07-21 | 2013-02-06 | Qualcomm Inc | METHOD AND SYSTEM FOR DETECTION AND ENHANCEMENT OF VIDEO IMAGES |
US8154632B2 (en) * | 2009-08-24 | 2012-04-10 | Lifesize Communications, Inc. | Detection of defective pixels in an image sensor |
JP5321439B2 (ja) * | 2009-12-15 | 2013-10-23 | 株式会社Jvcケンウッド | 画像符号化装置、画像復号化装置、画像符号化方法、及び、画像復号化方法 |
US9237355B2 (en) * | 2010-02-19 | 2016-01-12 | Qualcomm Incorporated | Adaptive motion resolution for video coding |
CN103109530A (zh) * | 2010-07-09 | 2013-05-15 | 三星电子株式会社 | 使用可调整环路滤波对视频编码的方法和设备以及使用可调整环路滤波对视频解码的方法和设备 |
SG11201500846TA (en) * | 2012-09-26 | 2015-05-28 | Panasonic Ip Corp America | Image coding method, image decoding method, image coding apparatus, image decoding apparatus, and image coding and decoding apparatus |
-
2010
- 2010-11-10 CA CA2784291A patent/CA2784291A1/en not_active Abandoned
- 2010-11-10 US US13/515,957 patent/US20120251012A1/en not_active Abandoned
- 2010-11-10 JP JP2011546040A patent/JPWO2011074357A1/ja active Pending
- 2010-11-10 EP EP10837386.1A patent/EP2515541A4/en not_active Withdrawn
- 2010-11-10 AU AU2010331511A patent/AU2010331511B2/en not_active Ceased
- 2010-11-10 CN CN201611019063.7A patent/CN107071480A/zh active Pending
- 2010-11-10 BR BR112012014685A patent/BR112012014685A2/pt not_active Application Discontinuation
- 2010-11-10 KR KR1020127018360A patent/KR20120105509A/ko active Application Filing
- 2010-11-10 KR KR1020197007933A patent/KR102060586B1/ko active IP Right Grant
- 2010-11-10 KR KR1020177010031A patent/KR101849891B1/ko active IP Right Grant
- 2010-11-10 CA CA3038111A patent/CA3038111C/en active Active
- 2010-11-10 KR KR1020187009830A patent/KR20180038076A/ko active Application Filing
- 2010-11-10 EP EP17201827.7A patent/EP3301932A1/en not_active Withdrawn
- 2010-11-10 WO PCT/JP2010/070053 patent/WO2011074357A1/ja active Application Filing
- 2010-11-10 KR KR1020167022206A patent/KR101729903B1/ko active IP Right Grant
-
2014
- 2014-06-13 JP JP2014122316A patent/JP5834112B2/ja not_active Expired - Fee Related
-
2015
- 2015-04-27 US US14/697,400 patent/US9514519B2/en active Active
- 2015-11-02 JP JP2015215633A patent/JP6159376B2/ja not_active Expired - Fee Related
-
2016
- 2016-10-25 US US15/334,048 patent/US9641865B2/en active Active
-
2017
- 2017-06-01 JP JP2017109226A patent/JP6335365B2/ja not_active Expired - Fee Related
-
2018
- 2018-04-20 JP JP2018081383A patent/JP2018142978A/ja not_active Ceased
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2007143178A (ja) * | 2001-11-29 | 2007-06-07 | Matsushita Electric Ind Co Ltd | 符号化歪除去方法 |
JP2008500775A (ja) * | 2004-05-27 | 2008-01-10 | ソニー エレクトロニクス インク | 符号化方法、符号化装置、フィルタ生成方法及びフィルタ生成装置 |
Non-Patent Citations (4)
Title |
---|
ITU-T TELECOMMUNICATION STANDARDIZATION SECTOR VCEG-AI18, July 2008 (2008-07-01) |
See also references of EP2515541A4 |
TAKESHI CHUJOH ET AL.: "Improvement of Block-based Adaptive Loop Filter", ITU - TELECOMMUNICATIONS STANDARDIZATION SECTOR VCEG-AJ13, 4 October 2008 (2008-10-04), XP030003635 * |
WATANABE, T. ET AL.: "In-loop filter using block-based filter control for video coding, Image Processing (ICIP)", IMAGE PROCESSING (ICIP), 2009 16TH IEEE INTERNATIONAL CONFERENCE ON, - 10 October 2009 (2009-10-10), pages 1013 - 1016, XP008153604 * |
Cited By (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPWO2011086836A1 (ja) * | 2010-01-12 | 2013-05-16 | シャープ株式会社 | 符号化装置、復号装置、および、データ構造 |
US10477209B2 (en) | 2011-06-23 | 2019-11-12 | Huawei Technologies Co., Ltd. | Offset decoding device, offset coding device, image filtering device |
US20180098067A1 (en) | 2011-06-23 | 2018-04-05 | Huawei Technologies Co., Ltd. | Offset decoding device, offset coding device, image filtering device |
US11765350B2 (en) | 2011-06-23 | 2023-09-19 | Huawei Technologies Co., Ltd. | Offset decoding device, offset coding device, image filtering device |
US9866833B2 (en) | 2011-06-23 | 2018-01-09 | Huawei Technologies Co., Ltd. | Offset decoding device, offset coding device, image filtering device |
US11089302B2 (en) | 2011-06-23 | 2021-08-10 | Huawei Technologies Co., Ltd. | Offset decoding device, offset coding device, image filtering device |
US10390012B2 (en) | 2011-06-23 | 2019-08-20 | Huawei Technologies Co., Ltd. | Offset decoding device, offset coding device, image filtering device |
US10440362B2 (en) | 2011-06-23 | 2019-10-08 | Huawei Technologies Co., Ltd. | Offset decoding device, offset coding device, image filtering device |
US10764580B2 (en) | 2011-06-23 | 2020-09-01 | Huawei Technologies Co., Ltd. | Offset decoding device, offset coding device, image filtering device |
US9497455B2 (en) | 2011-06-23 | 2016-11-15 | Huawei Technologies Co., Ltd. | Offset decoding device, offset coding device, image filtering device, and data structure |
JP2013090253A (ja) * | 2011-10-20 | 2013-05-13 | Nippon Hoso Kyokai <Nhk> | 画像処理装置及びプログラム |
JP2014179792A (ja) * | 2013-03-14 | 2014-09-25 | Nippon Telegr & Teleph Corp <Ntt> | 信号処理装置、及び信号処理方法 |
JP2020522170A (ja) * | 2017-05-31 | 2020-07-27 | インターデジタル ヴイシー ホールディングス, インコーポレイテッド | ピクチャの符号化および復号の方法およびデバイス |
JP7179023B2 (ja) | 2017-05-31 | 2022-11-28 | インターデジタル マディソン パテント ホールディングス, エスアーエス | ピクチャの符号化および復号の方法およびデバイス |
JP7073186B2 (ja) | 2018-05-14 | 2022-05-23 | シャープ株式会社 | 画像フィルタ装置 |
JP2022105577A (ja) * | 2018-05-14 | 2022-07-14 | シャープ株式会社 | 画像フィルタ装置 |
JP7332753B2 (ja) | 2018-05-14 | 2023-08-23 | シャープ株式会社 | 画像フィルタ装置 |
JP2019201256A (ja) * | 2018-05-14 | 2019-11-21 | シャープ株式会社 | 画像フィルタ装置 |
Also Published As
Publication number | Publication date |
---|---|
JPWO2011074357A1 (ja) | 2013-04-25 |
EP3301932A1 (en) | 2018-04-04 |
KR101849891B1 (ko) | 2018-04-17 |
AU2010331511A1 (en) | 2012-07-26 |
JP6335365B2 (ja) | 2018-05-30 |
JP2018142978A (ja) | 2018-09-13 |
KR20170045361A (ko) | 2017-04-26 |
US20170041636A1 (en) | 2017-02-09 |
KR20180038076A (ko) | 2018-04-13 |
US20120251012A1 (en) | 2012-10-04 |
US20150242999A1 (en) | 2015-08-27 |
AU2010331511B2 (en) | 2015-01-15 |
CA3038111C (en) | 2022-08-09 |
EP2515541A4 (en) | 2015-08-26 |
JP2017200198A (ja) | 2017-11-02 |
JP6159376B2 (ja) | 2017-07-05 |
JP2016036169A (ja) | 2016-03-17 |
BR112012014685A2 (pt) | 2016-04-05 |
JP5834112B2 (ja) | 2015-12-16 |
JP2014209760A (ja) | 2014-11-06 |
CN107071480A (zh) | 2017-08-18 |
CN102656888A (zh) | 2012-09-05 |
KR20160102085A (ko) | 2016-08-26 |
US9514519B2 (en) | 2016-12-06 |
CA3038111A1 (en) | 2011-06-23 |
CA2784291A1 (en) | 2011-06-23 |
KR101729903B1 (ko) | 2017-04-24 |
EP2515541A1 (en) | 2012-10-24 |
KR102060586B1 (ko) | 2019-12-30 |
KR20190031601A (ko) | 2019-03-26 |
KR20120105509A (ko) | 2012-09-25 |
US9641865B2 (en) | 2017-05-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6335365B2 (ja) | 復号装置 | |
JP6660074B2 (ja) | 映像復号化方法及び装置 | |
JP6242438B2 (ja) | ビデオ符号化方法、ビデオ復号方法、ビデオエンコーダ、及びビデオデコーダ | |
JP2008167456A (ja) | 画像符号化装置及び画像復号装置 | |
WO2011086836A1 (ja) | 符号化装置、復号装置、および、データ構造 | |
CN114640845A (zh) | 编解码方法、装置及其设备 | |
WO2011105231A1 (ja) | フィルタ係数符号化装置、フィルタ係数復号装置、動画像符号化装置、動画像復号装置、および、データ構造 | |
JP2022105653A (ja) | デブロッキングフィルタ制御装置及びプログラム | |
JP7460384B2 (ja) | 予測装置、符号化装置、復号装置、及びプログラム | |
WO2020059341A1 (ja) | 画像復号装置、画像符号化装置、画像処理システム及びプログラム | |
WO2011105230A1 (ja) | フィルタ係数符号化装置、フィルタ係数復号装置、動画像符号化装置、動画像復号装置、および、データ構造 | |
JP7483725B2 (ja) | 符号化装置、復号装置、及びプログラム | |
JP2021052249A (ja) | 符号化装置、復号装置、及びプログラム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 201080056671.X Country of ref document: CN |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 10837386 Country of ref document: EP Kind code of ref document: A1 |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 10837386 Country of ref document: EP Kind code of ref document: A1 |
|
DPE1 | Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101) | ||
WWE | Wipo information: entry into national phase |
Ref document number: 2011546040 Country of ref document: JP |
|
ENP | Entry into the national phase |
Ref document number: 2784291 Country of ref document: CA |
|
WWE | Wipo information: entry into national phase |
Ref document number: 13515957 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2010837386 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 5823/CHENP/2012 Country of ref document: IN |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2010331511 Country of ref document: AU |
|
ENP | Entry into the national phase |
Ref document number: 20127018360 Country of ref document: KR Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 2010331511 Country of ref document: AU Date of ref document: 20101110 Kind code of ref document: A |
|
REG | Reference to national code |
Ref country code: BR Ref legal event code: B01A Ref document number: 112012014685 Country of ref document: BR |
|
ENP | Entry into the national phase |
Ref document number: 112012014685 Country of ref document: BR Kind code of ref document: A2 Effective date: 20120615 |