IE20020423A1 - Deblocking block-based video data - Google Patents

Abstract

Block-based video data is deblocked by use of a strong, non-linear filter process or a weak, adaptive smoothing filter process. In the non-linear filter three pixels on each side of a block boundary are updated. In the adaptive smoothing filter only one pixel on each side is updated. A smoothness check with reference to configurable threshold values is performed and this is used to dynamically select a relevant filter process. Thus, processing capacity is optimised. <Figure 1>

Description

Deblocking block-based video data INTRODUCTION Field of the Invention The invention relates to deblocking block-based video data generated according to a standard such as MPEG-4 [1] or H.263.[2].

Prior Art Discussion These standards specify use of an 8x8 pixel block DCT (Discrete Cosine Transform) for packing information into a small number of coefficients by utilising the spatial correlation property of images. This block-based coding introduces blocking artefacts between block boundaries, as the transform does not take into account the correlation between block boundaries and because the block is independently coded. The blocking artefacts are the grid noise along the block boundaries. Such blocking artefacts mainly arise due to the quantisation of DC and AC coefficients. The other artefacts due to truncation of high frequency coefficients by quantisation are ringing effect around edges (due to Gibb’s phenomenon), “mosquito noise”, and corner outliers.

It is required to remove such artefacts while distorting the image details as little as possible. The processing complexity is also required to be low. At low bit-rate encoding it is found that blocking artefacts are visually more perceptible and undesirable.

Several post-processing techniques have been described, including two-dimensional signal adaptive filtering [3], iterative image-recovery using the theory of Projection Onto Convex Sets (POCS) [4], spatio-temporal adaptive filtering [5], use of Markov QFW TO WC IWSPECneN -2random fields [6], and a MAP based algorithm [7], Also, an adaptive post processor for block encoded images is described in reference [8], in which a space-variant lowpass filter is used to smooth the pixels at block boundaries. Reference [9] describes use of an adaptive smoothing filter as a stand-alone process for post5 processing. The MPEG-4 standard[l] suggests a post-processing algorithm which is very computations intensive and hence unsuitable for mobile applications requiring ^very low complexity. A non-linear filter suggested by Μ. Y. Shen et al [10] provides a simple algorithm but at the cost of degraded performance.

While all of these approaches improve video quality, they either suffer from being very computation-intensive or delivering low quality performance at cost of reduced complexity.

The invention is therefore directed towards providing a method and system to remove such artefacts with less computational complexity and comparable quality performance.

References [1] “Information Technology - Generic Coding of Audio-Visual Objects - Part 2:Visual,” MPEG-4 standard, ISO/IEC/ JTC 1/SC29/WG 11 N 2688, Seoul, March 1999. [2] “EL263+: Video Coding at Low Bit Rates,” IEEE Transactions on Circuits and Systems for Video Coding, pg. 849-856, Vol. 8, No. 7, Nov. 1998. [3] Y. L. Lee, H. C. Kim, and H. W. Park, “Blocking Effect Reduction of JPEG Images by Signal Adaptive Filtering,” IEEE Trans, on Image Processing, Vol. 7, pp. 229-234, Feb. 1998. -3[4] Y. Yang, Ν. Galatsanos, and A. Katsaggelos, “Projection-based Spatially Adaptive Reconstruction of Block Transform Compressed Images,” IEEE Trans on Image Processing, Vol. 4, pp. 896-908, July 1995. [5] T. S. Liu and N. S. Jayant, “Adaptive Postprocessing Algorithm for Low Bit5 Rate Video Signals,” IEEE Trans, on Image Processing, Vol. 4, pp. 1032-1035, July 1995. [6] Thomas Meier, King N. Ngan, and Gregory Crebbin, “Reduction of Blocking Artifacts in Image and Video Coding,” IEEE Trans, on Circuits and Systems for Video Technology, Vol. 9, No. 3, pp. 490-500, April 1999. [7] Thomas P. O’Rourke and Robert L. Stevenson, “Improved Image Decompression for Reduced Transform Coding Artifacts,” IEEE Trans, on Circuits and Systems for Video Technology, Vol. 5, pp. 490-499, Dec. 1995. [8] Chung J Kuo et al: “Adaptive postprocessor for block encoded images” IEEE Transactions on Circuits and Systems for video technology, IEEE Inc. New York, US vol. 5 no. 4, 1 August 1995 (1999-08-01) pages 298 - 304. [9] Hyun Wook Park and Yung Lyul Lee, “A Postprocessing Method for Reducing Quantisation Effects in Low Bit-Rate Moving Picture Coding,” IEEE Trans, on Circuits and Systems for Video Technology, Vol. 9, pp. 161-171, Feb. 1999.

[10] Mei-Yin Shen, JongWon Kim and C.-C. Jay Kuo, Nonlinear filtering based fast compression artefact reduction technique for H.263, ITU-T/SG15 Documentation Q15-F-13, Seoul, Korea, November 1998 SUMMARY OF THE INVENTION According to the invention, there is provided a method of deblocking block-based video data comprising the step of applying a filter process to the video data to change -4pixels at block boundaries, characterised in that, the.method comprises the further steps of: making a plurality of filter processes available, and 5 dynamically selecting one of said filter processes according to block boundary video data.

In one embodiment, a filter process is a non-linear filter process applied to a plurality 10 of pixels on each side of a block boundary.

In another embodiment, said non-linear filter process selects a coefficient (d,) for a pixel value, said coefficient being chosen according to the difference between a pair of pixels, one on each side of the block boundary.

In a further embodiment, the coefficient is determined by a table look-up using said difference.

In one embodiment, said non-linear filter does not result in a pixel change if the 20 difference between the boundary pixels is below a threshold.

In another embodiment, said threshold value is THR3 *QuantScale, where “QuantScale” is the quantisation parameter for the macroblock containing the boundary pixel outside the block, and “ THR3” is a constant integer.

In a further embodiment, THR3 has a value of 3.

In one embodiment, a filter process is an adaptive smoothing filter.

In another embodiment, said adaptive smoothing filter updates only one pixel on each side of a block boundary.

In a further embodiment, pixels are updated using value d which is calculated based on frequency components derived from the boundary pixels.

In one embodiment, the .value d is subtracted or added to the boundary pixel value.

In another embodiment, the frequency components are evaluated from a simple inner product of an approximated DCT kernel with pixel vectors.

In a further embodiment, the filter is selected according to a smoothness check on pixel conditions around a block boundary, in which the non-linear filter is selected if the area around the boundary is homogenous and the smoothing filter is chosen otherwise.

According to another aspect, the invention provides a video data processor comprising means for performing any method as defined above.

DETAILED DESCRIPTION OF THE INVENTION Brief Description of the Drawings The invention will be more clearly understood from the following description of some embodiments thereof, given by way of example only with reference to the accompanying drawings in which:Fig. 1 is a diagram illustrating blocks of pixels and pixels to be filtered; and Fig. 2 is a diagram illustrating a non-linear filter. -6Description of the Embodiments Referring to Fig. 1,8x8 pixels are in each block. A deblocking method processes pixels in a band around each block boundary. In a direction perpendicular to each block boundary there is a line of pixels v0 to v4 on one side and a line of pixels vs to v9 on the other side. Alternatively, as shown in Fig. 2 a shorter line of six pixels with pixels Vj and v4 at the boundary may be used.

The method uses one of two stored filter processes, namely a non-linear filter process as set out in Fig. 2 using three pixels on each side of the v3/v4 block boundary, and adaptive smoothing filtering.

Non-linear filtering Non-linear filtering updates three pixels on either side of the edge as shown in Fig. 2. The filtering process involves calculation of edge pixel difference |v3-v4| and selection of the d(s according to the following table. The effect of this non-linear filter is seen to be visually comparable to DC offset filtering [1], however the complexity of processing is much lower.

The deblocking filter process involves calculation of | v3-v41 followed by table look up to identify the d(s. The pixel filtering then is done by, Fz'= Fz'+ szgzz (%-PJ * 4; Vi=1...3 and ViVi + sign (V3-Vd* di Vi=4...6 Where, sign(x) ~lif(x>=0) and sigffx) = -1 iff x<0).

The values K, are the pixels on the edge before filtering and Vt' are updated edge pixels, dj‘s are selected form the Table below. -Ί - d=IV3-V4l d2 d3 d4 d5 de under 3 0 0 0 0 0 0 3 0 0 -1 1 0 0 4,.5 0 -1 -2 2 1 0 6,7,8 -1 -2 -3 3 2 1 9,10 -1 -2 -4 4 2 1 11,12,13 -1 -3 -5. 5 3 1 14,15,16 -2 -4 -6 6 4 2 17,18 -2 -4 -7 7 4 2 19,20 -2 -5 -8 8 5 2 21,22 -2 -5 -9 9 5 2 23 -3 -6 -10 10 6 3 >23 -cZ/8 -d/4 -d/2 d/2 d/4 d/8 A non-linear filter operation as described above is only applied if the initial condition, fl V3-V4\ < THR3 *QuantScale), is met. QuantScale means the qunatisation parameter for the macroblock containing the pixel V3 as in MPEG-4 standard[l]. In this embodiment THR3 has a value of 3.

If a pixel value is changed by the previous filtering operation, the updated pixel value is used for the next filtering step.

Adaptive Smoothing Filter This is a weak filter which updates one pixel on each side of the block boundary.

This filter uses a signal adaptive smoothing technique which differentiates image details at the block discontinuities using the frequency information of neighbour pixel arrays So, Si, and S2 (shown in Fig. 1). -810 In this method the boundary pixel values v4 and v5 are replaced with v4' and v5' as follows: v4' = v4-d, vi - v5+d, And d = CLIP (5-(a3i0'-a3iQ)//8, 0, (v4-v5)/2) δ(\α3,01 < QuantScale) Where a3,0'- SIGN(a3i0) ·ΜΙΝ(\α3,ο\, \a3I\, la3,2]).

Here CLIP (x, p, q) clips x to a value between p and q; QuantScale means the qunatisation parameter for the macroblock containing the pixel v5 as in MPEG-4 standard[l], b(Condition)=l if the condition is true and 0 otherwise. The operation “//” means integer division with rounding to nearest integer. Half-integer values are rounded away from zero. For example 3//2 is rounded to 2, and -3//2 is rounded to -2.

Frequency components c3,o, ¢/3.1, and a3,2 can be evaluated from the simple inner product of the approximated DCT kernel [2 -5 5 -2] with the pixel vectors, i.e., a3,0 - ([2-5 5-2] · [v3 v4 v5 v6]T) // 8, a3,1 = ([2 -5 5 -2] · [vl v2 v3 v4]T)//8, a3,2 = ([2 -5 5 -2] · [v5 v6 v7 v8]T) // 8.

Filter Choice Decision The required filter is chosen by performing a smoothness check on pixel conditions around a block boundary. If the area around the boundary is homogenous the non25 linear (first) filter is used, whereas the second filter is used otherwise by default.

The following procedure is used to determine which filter is to be applied. The procedure marks the block boundary homogeneous depending upon the number of adjacent pixels pairs having pixel difference more than, a threshold. -9eq_cnt = φ(νθ-ν1) + φ(ν1-ν2) + φ(ν2-ν3) + φ(ν3-ν4) + φ(ν4-ν5) + φ(ν5-νό) + φ(ν6-ν7) + φ(ν7-ν8) + φ(ν8-ν9), where, φ(γ) - 1 if |χ | If(eq_cnt > THR2) strong filter mode (non-linear) is applied, else weak filter mode (default mode) is applied.

In this embodiment the values of THR1 and THR2 are 2 and 6 respectively.

The above filter operations are applied for all the block boundaries, first along the horizontal edges followed by the vertical edges. The following are advantageous aspects of each of the filter processes:15 1. Strong filter mode (non-linear): The computational complexity is very low as compared to traditional linear filtering methods, while maintaining the quality comparable at low bit-rates. The computations are further reduced by using a table lookup. 2. Weak filter mode (default mode): The filter achieves better filtering by avoiding excessive blurring in case of nonhomogenous pixel regions.

The invention achieves optimum use of processing resources by dynamically choosing the appropriate filter. Furthermore, where the smoothness check results in the non-linear filter process being chosen, this may not actually be used according to the sub-decision involving use of the threshold THR3.

The invention is not limited to the embodiments described but may be varied in construction and detail.

Claims (15)

Claims

1. A method of deblocking block-based video data comprising the step of applying a filter process to the video data to change pixels at block boundaries, characterised in that, the method comprises the further steps of: making a plurality of filter processes available, and dynamically selecting one of said filter processes according to block boundary video data.

2. A method as claimed in claim 1, wherein a filter process is a non-linear filter process applied to a plurality of pixels on each side of a block boundary.

3. A method as claimed in claim 2, wherein said non-linear filter process selects a coefficient (d.) for a pixel value, said coefficient being chosen according to the difference between a pair of pixels, one on each side of the block boundary.

4. A method as claimed in claim 3, wherein the coefficient is determined by a table look-up using said difference.

5. A method as claimed in claim 3 or 4, wherein said non-linear filter does not result in a pixel change if the difference between the boundary pixels is below a threshold.

6. A method as claimed in claim 5, wherein said threshold value is THR3 ★QuantScale, where “QuantScale” is the quantisation parameter for the macroblock containing the boundary pixel outside the block, and “ THR3” is a constant integer. -11

7. A method as claimed in claim 6, wherein THR3 has a value of 3.

8. A method as claimed in any preceding claim, wherein a filter process is an 5 adaptive smoothing filter.

9. A method as claimed in claims 8, wherein said adaptive smoothing filter updates only one pixel on each side of a block boundary.

10. 10. A method as claimed in claims 8 or 9, wherein pixels are updated using value d which is calculated based on frequency components derived from the boundary pixels.

11. A method as claimed in claim 10, wherein the value d is subtracted or added 15 to the boundary pixel value.

12. A method as claimed in claim 10 or 11, wherein the frequency components are evaluated from a simple inner product of an approximated DCT kernel with pixel vectors.

13. A method as claimed in any preceding claim, wherein the filter is selected according to a smoothness check on pixel conditions around a block boundary, in which the non-linear filter is selected if the area around the boundary is homogenous and the smoothing filter is chosen otherwise.

14. A video data processor comprising means for performing a method as claimed in any preceding claim.

15. A computer program product comprising software code for performing a method as claimed in any of claims 1 to 13 when executing on a digital computer.