GB2501534A

GB2501534A - Control of transform processing order in high efficeency video codecs

Info

Publication number: GB2501534A
Application number: GB1207458.9A
Authority: GB
Inventors: Karl James Sharman; James Alexander Gamei; Nicholas Ian Saunders; Paul James Silcock
Original assignee: Sony Corp
Current assignee: Sony Corp
Priority date: 2012-04-26
Filing date: 2012-04-26
Publication date: 2013-10-30
Also published as: GB201207458D0

Abstract

A method for high efficiency video coding comprises the steps of selecting an intra-prediction mode for a high efficiency video coding prediction unit, and responsive to the selected intraÂ prediction mode, selecting one of a plurality of predetermined orders of transform unit processing, and processing respective transform units in the selected predetermined order. The intra prediction mode may be selected from a Z-scan order, a vertically inverted Z-scan order or vertically and horizontally inverted Z-scan order dependent on the angle corresponding to the prediction mode selected.

Description

METHOD AND APPARATUS FOR

HIGH EFFICIENCY VIDEO CODECS

The present invention relates to a method and apparatus for high efficiency video codecs.

Currcnt video codecs such as H.264/MPEG-4 Advanced Video Coding (AVC) achieve data compression primarily by only encoding the differences between successive video frames.

Thesc codccs usc a rcgular array of so-callcd macroblocks, and thc image rcgion within thc macroblock is thcn cncoded according to thc degrcc of motion found between the corrcsponding current and previous macroblocks in the video sequence, or between neighbouring macroblocks within a single frame of the video sequence.

1-11gb Efficiency Video Coding (HEVC), also known as 1-1.265 or MPEG-H Part 2, is a proposed succcssor to H.264!MPEG-4 AVC. It is intcndcd for HEVC to improve video quality is and double the data compression ratio compared to H.264, and for it to be scalable from 128 x 96 to 7680 x 4320 pixels resolution, roughly equivalent to bit rates ranging from l28kbit/s to LU 800Mbit/s.

O HEVC replaces the maeroblocks found in existing H.264 and MPEG standards with a LI") more flexible scheme based upon coding units (CU5), which are variable size structures.

C\i 20 Consequently, when encoding the image data of video frames, the CU sizes can be selected responsive to the apparent image complexity or detected motion levels, instead of using uniformly distributed macroblocks. Consequently far greater compression can be achieved in regions with little motion between frames and with little variation within a frame, whilst better image quality can be preserved in areas of high inter-frame motion or image complexity.

The compression scheme works by predicting the content of a sample block based upon values in a preceding frame (inter-prediction) or the current frame (intra-prediction), and then encoding and compressing the residual error between the prediction and the actual image.

Consequently, the bit rate (or the compression level), is dependent upon the average size or energy of the residual error, and hence in turn upon the accuracy of the predictions.

Each CU contains one or more variable-block-sized prediction units (PUs) of either intra-picture or inter-picture prediction type, and one or more transform units (TU5) which contain coefficients for spatial block transform and quantization.

It is desirable to efficiently process these prediction and transform units so as to obtain good quality prediction results, and thereby reduce thc eventual bit rate and!or comprcssion levels in the bit stream.

The present invention seeks to address, mitigate or alleviate this need.

In a first aspect, a method for high efficiency video coding is provided in accordance with claim 1.

In another aspect, a high efficiency video coding encoder is provided in accordance with claim 7.

In another aspect, a high efficiency video coding decoder is provided in accordance with claim 12.

Further respective aspects and features of the invention arc defined in the appendcd claims.

Embodiments of the present invention will now be described by way of example with LU reference to the accompanying drawings, in which: C Figure 1 is a schematic diagram of directions associated with intra-prediction modes of a Lf) high efficiency video coding scheme.

C'J 20 Figure 2 is a schematic diagram of a processing order for transform units in a high efficiency video coding scheme.

Figures 3A to 3D are schematic diagrams illustrating the processing of transform units in a high efficiency video coding scheme according to the processing order of Figure 2 for a first intra prediction mode (mode 18).

Figures 4A to 4D arc schematic diagrams illustrating the processing of transform units in a high efficiency video coding scheme according to the processing order of Figure 2 for a second intra prediction mode (mode 34).

Figure 5 is a schematic diagram of three alternative processing orders for transform units in a high efficiency video coding scheme, in accordance with an embodiment of the present invention.

Figures 6A to D are schematic diagrams illustrating the processing of transform units in a high efficiency video coding scheme according to a processing order shown in Figure 5, in accordance with an embodiment of the present invention.

Figure 7 is a schematic diagram of directions associated with intra-predietion modes of a for high efficicncy video coding scheme, illustrating corresponding processing orders for transform units in a high efficiency video coding scheme, in accordance with an embodiment of the present invention.

Figure 8 is a flow diagram of a method for high efficiency video coding in accordance with an embodiment of the present invention.

Figure 9 is a schematic diagram of a high efficiency video coding encoder in accordance with an embodiment of the present invention.

A method and apparatus for high efficiency video codecs are disclosed. In the following description, a number of specific details are presented in order to provide a thorough understanding of the embodiments of the present invention. It will be apparent, however, to a person skilled in the art that these specific details need not be employed to practice the present invention. Conversely, specific details known to the person skilled in the art are omitted for the purposes of clarity where appropriate.

LI') In HEVC, intra-prediction within an image comprises identifying a neighbouring region C of an image that approximates a current sample region, and using that neighbouring region as the LI") basis for predicting the content of the sample region. The residual error between this prediction C'J 20 and the actual image at that sample region then requires fewer bits to encode than the original image in that sample region, thereby greatly reducing the required bit rate whilst retaining high image quality. However, as noted above, the bit rate is thus indirectly dependent upon the quality of the prediction.

In order to obtain a good quality prediction, it is useful to be able to point to a region of the image that provides the best neighbouring approximation of the current sample region.

Referring now to Figure 1, consequently HEVC uses 35 different intra-prcdiction modes, 33 of which specify directions from the position of a sample point 100 to be predicted, to a nearby reference pixel in the image, in the range 45 to 225 degrees. This gives fine control over what neighbouring part of the image can be selected for prediction.

Notably however, this scheme means that when encoding a sample point, the system must constrain its selection to neighbouring image data that will have already been decoded by a decoder when it is predicting the sample point in the image, so that decoded image data is available to act as a basis for prediction.

The intra-prediction modes and hence directions are selected on a block-by-block basis.

Currently, they are specified at the Prediction Unit (PU) block level, and evaluated at the Transform Unit (TV) block level.

Figure 2 shows a typical configuration of four 8x8 pixel TUs associated with a 16x16 pixel PU, and illustrates the order in which they are processed. The predicted samples are calculated for each TU separately, using the same prediction mode as that set for the PU. The TUs are processed in a Z-scan order for TUs 0, 1, 2 and 3.

Referring to Figure 3A, for prediction modes 10 to 26 inclusive (i.e. left horizontal to positive vertical), this processing order is efficient. Figure 3A illustrates prediction mode 18 as a representative example. Notably, the neighbouring reference pixels 110 (pointed to by the arrows) are immediately adjacent to the edge of the TV 0, and therefore are as near as possible to the samples that they are being used to predict.

This near positioning is beneficial, since in most images the most similar image data to C%J 15 the sample image data will be that which is closest to it. As a result the Z-scan processing order means that the residual error from the prediction will be smaller, so reducing the amount of data LU to encode.

C As can be seen in Figures 3B, 3C and 3D, the Z-scan processing order means that the LI") same near positioning occurs when processing each successive TV 1, 2, and 3. i20

However, this processing order is less effective for other prediction modes.

Figure 4A illustrates the processing for TU 0 when the PU has set prediction mode 34 (which is orthogonal to previously illustrated mode 18). In this case, the reference pixels for TV 0 must be taken from the PU above because the TU immediately to the right (TU 1) will not have been decoded yet when decoding the image, and so reference pixels will not be available.

Referring back also to Figure 3A, it can be seen that the distance between predicted samples and reference pixels for corresponding (mirror image) cases 112, 112' are consequently much longer for the case illustrated in Figure 4A. In fact in the case shown in Figure 4A, almost half the samples in TU 0 are predicted from reference pixels that are a greater distance away than in the corresponding (mirror image) case of Figure 3A. To put it another way, the sum length of the arrows in Figure 4A is much greater than the sum length of arrows in the equivalent case of Figure 3A, showing that overall the predictions in Figure 4A are from reference pixels further from the positions of the predicted samples.

Problematically, as noted above the increased distance between the predicted samples and their reference pixels is likely to reduce the accuracy of the prediction. As a result, prediction quality will drop for some prediction modes. This means that on average, the magnitude or energy of the residual data values will correspondingly rise for some prediction modes. Again as noted previously this is a problem because it requires more bits to represent larger residual data values, potentially necessitating greater compression of these residual data values and hence a reduction in image quality, and/or greater bit rates.

This problem is propagated throughout the remaining TUs 1, 2 and 3, as seen in Figures 4B-4D.

1-lence referring now to Figure 5, in an embodiment of the present invention the processing order of the TU blocks is made prediction-mode dependent. j15

Figure 5 shows three processing orders, illustrated as non-limiting examples. The first is L() the Z-scan mentioned previously, so-called because it is ordered in the same way that one C conventionally writes the letter Z', resulting in the processing order top left, top right, bottom LCD left, bottom right. Two variants are firstly the horizontally inverted Z-scan, or HIZ-scan, which C'J 20 starts in the top-right ILT rather than the top left TLT, and the vertically inverted Z-scan or VIZ-scan, which starts at the bottom left TU.

Whilst as will be explained below, these scanning orders are suitable for the prediction modes of Figure 1 and the HEVC PU processing order, other scanning modes may be considered as required, such as the vertically and horizontally inverted scan or VHIZ-scan, which starts at the bottom right TU. Similarly these four scans can also be rotated through 90 degrees to generate corresponding N'-scans.

Hence referring now to Figures 6A-6D, for the example of prediction mode 34 previously used in Figures 4A-4D, it is better to select the HIZ-scan processing order.

It can be seen that in this case, compared to the Z-scan illustrated in Figures 4A and 4C, now the two left-most TUs 1 and 3 (as numbered in the HIZ-scan) are able to make use of nearby reference samples in a similar manner to that illustrated in Figures 3A and 3C for prediction mode 18. For example, the distance between predicted samples and reference pixels for the corresponding particular highlighted cases 112 in Figure 3A and 112" in Figure 6B are now the same, and both arc shorter than 112' in Figure 4A.

In other words, instead of all four TUs having to use some reference samples that were from a TV that was not the closest ideal TV (as in Figures 4A-4D), now only two TUs need to do this (in Figures 6A and 6C), resulting in an improved overall level of prediction accuracy across the TUs and hence an improved average magnitude or energy for the residual data values. This will then translate into fewer bits to encode and hence better image quality in the compressed video image for the same bit-rate.

Referring now also to Figure 7, for the HEVC intra-prediction modes 2 to 9 (or more generally for prediction directions in a third quadrant between 180 and 270 degrees, being to the left and below) then a VIZ-scan results in the most TV blocks being able to use the nearest reference samples for prediction.

Meanwhile for HEVC intra-prediction modes 10 to 26 (or more generally for prediction directions in a second quadrant between 90 and 180 degrees, being between horizontal and vertical to the left) then as noted previously a Z-scan results in the most TV blocks being able to use the nearest reference samples for prediction.

O For HEYC intra-prediction modes 27-34 (or more generally for prediction directions in a Lt') first quadrant, between 0 and 90 degrees, being between to the right and above) then a HIZ-scan C'J 20 results in the most TU blocks being able to use the nearest reference samples for prediction.

Finally HEYC intra-prediction modes 0 and 1 (which are planar and DC modes) use reference samples above and to the left of the predicted TU, so the Z-scan is also most appropriate in these cases.

It will also be appreciated that if HEVC or another scheme decoded images from the bottom-right comcr rather than the top left comcr, then the prediction mode directions would be likely to be a complement of the present modes (i.e. a reflection about the 45 degree line of current modes 2 and 34).

Consequently for prediction modes corresponding to directions in a fourth quadrant between 270 and 0 degrees, being to the right and below, a VHIZ-scan results in the most TU blocks being able to use the nearest reference samples for prediction.

It will be appreciated that the above quadrants nominally have shared boundaries at 0, 90, and 270 degrees. Therefore for intra-prediction modes coinciding with these angles, in principle the scan orders associated with either quadrant may be used. 1-lowever, in an embodiment of the present invention, for modes 10 (180 degrees) and 26 (90 degrees), a Z-scan may preferably be used for HEVC. Thus also in an embodiment of the present invention, the first quadrant may be interpreted as being inclusive of 0, and exclusive of 90 degrees, the second quadrant may be interpreted as being inclusive of 90 and inclusive of 180 degrees, the third quadrant may be interpreted as being exclusive of 180 and inclusive of 270 degrees, and the fourth quadrant may be interpreted as being exclusive of 270 and exclusive of 360 (i.e. zero) degrees.

Hence in light of the foregoing, referring now to Figure 8 a method for high efficiency video coding comprises: in a first step slO, selecting an intra-prediction mode for a high efficiency video coding prediction unit; and responsive to the selected intra-prediction mode, is in a second step s20, selecting one of a plurality of predetermined orders of transform unit processing; and LI') in a third step s30, processing respective transform units in the selected predetermined C order.

LU

C'J 20 It will be apparent to a person skifled in the art that variations in the above method corresponding to operation of the various embodiments of the apparatus as described and claimed herein are considered within the scope of the present invention, including but not limited to: -selecting an intra-prediction mode corresponding to an angle greater than or equal to 0 and less than 90 degrees with respect to the prediction unit (i.e. in a first quadrant), and selecting a horizontally inverted Z-scan processing order; -selecting an intra-prediction mode corresponding to an angle greater than or equal to 90 and less than or equal to 180 degrees with rcspcct to the prediction unit (i.e. in a second quadrant), and selecting a Z-scan processing order; -selecting an intra-prediction mode corresponding to an angle greater than 180 and less than or equal to 270 degrees with respect to the prediction unit (i.e. in a third quadrant), and selecting a vertically inverted Z-scan processing order; and -selecting an intra-prediction mode corresponding to an greater than 270 and less than 360 (or 0) dcgrccs with rcspcct to thc prcdiction unit (i.c. in a fourth quadrant), and sclccting a vertically and horizontally inverted Z-scan processing order.

It will be appreciated that the methods disclosed herein may be carried out on conventional hardware suitably adapted as applicable by sofiware instruction or by the inclusion or substitution of dedicated hardware.

Thus the required adaptation to existing parts of a conventional equivalent device may bc implemented in the form of a non-transitory computer program product or similar object of manufacture comprising processor implementable instructions stored on a data carrier such as a floppy disk, optical disk, hard disk, PROM, RAM, flash memory or any combination of these or other storage media, or in the form of a transmission via data signals on a network such as an Ethernet, a wireless network, the Internet, or any combination of these of other networks, or realised in hardware as an ASIC (application specific intcgratcd circuit) or an FPGA (field programmable gate array) or other configurable circuit suitable to use in adapting the conventional equivalent device.

LU

C Referring now to Figure 9, in an embodiment of the present invention such hardware may Lt') be an HEVC encoder suitable for implementing the above methods. In an embodiment of the C'.J 20 present invention, the HEVC encoder comprises an intra-frame modc selector 110 and an intra-frame mode predictor 120; a motion compensation frame predictor 130, a motion estimator 140 and a sub-pixel interpolation filter (e.g. a 1⁄4 subpixel filter); frame stores 150; an adaptive loop filter 170 and an ALF coefficient generator 175; a sample adaptive offset unit 180 and SAO coefficient generator 185; a deblocking filter 190 and a deblocking filter encoding decision unit 195; a transform unit 200 and inverse transform unit 205; a quantisation unit 210 and inversc quantisation unit 215; and an entropy encoder 220.

In the above encoder, the intra-frame mode selector 110 is operable to select the intra-prediction mode, and rcsponsivc to that selection, thc intra-frame modc prcdictor 120 is opcrablc to select one of a plurality of predetermined orders of transform unit processing. It will be appreciated however that intra-frame mode selector may also make the processing order selection.

S

A HEYC decoder (not shown) corresponding to the above l-IEVC encoder will be readily understood by a person skilled in the art to similarly comprise an intra-frame mode selector operable to select an intra-prediction mode, and an intra-frame mode predictor which, responsive to that selection, is operable to select one of a plurality of predetermined orders of transform unit processing, so as to correspond with the encoding process for that data (otherwise the transmitted residual errors would not correspond to the errors in prediction at decoding). Hence such an HEVC decoder may also implement the methods described herein.

LU

Claims

CLAIMS1. A method for high efficiency video coding, comprising the steps of selecting an intra-prediction mode for a high efficiency video coding prediction unit; and responsive to the selected intra-prediction mode, selecting one of a plurality of predetermined orders of transform unit processing; and processing respective transform units in the selected predetermined order.
2. Themethodofclaiml,inwhich an intra-prediction mode is selected corresponding to an angle greater than or equal to 0 and less than 90 degrees with respect to the prediction unit, and in which the step of selecting one of a plurality of predetermined orders of transform unit processing comprises: selecting a horizontally inverted Z-scan processing ordcr. j15
3. The method of claim 1, in which LI') an intra-prediction mode is selected corresponding to an angle greater than or equal to 90 C and less than or equal to 180 degrees with respect to the prediction unit, and in which LI") the step of selecting one of a plurality of predetermined orders of transform unit processing comprises: selecting a Z-scan processing order.
4. The method of claim 1, in which an intra-prediction mode is selected corresponding to an angle greater than 180 and less than or equal to 270 degrees with respect to the prediction unit, and in which the step of selecting one of a plurality of predetermined orders of transform unit processing comprises: selecting a vertically inverted Z-scan processing order.
5. The method of claim 1, in which an intra-predietion mode is selected corresponding to an angle greater than 270 and less than 360 or 0 degrees with respect to the prediction unit, and in which the step of selecting one of a plurality of predetermined orders of transform unit processing compriscs: selecting a vertically and horizontally inverted Z-scan processing order.
6. A computer program for implementing the steps of any preceding method claim.
7. A high efficiency video coding encoder, comprising: first selection means operable to select an intra-prediction mode for a high efficiency video coding prediction unit; and responsive to the selected intra-prediction mode, second selection means operable to select one of a plurality of predetermined orders of transform unit processing; and processing means operable to process a set of transform units in the selected predetermined ordcr.c.'115
8. The encoderofclaim 7,in which Lç) if the first selection means selects an intra-prediction mode associated with a direction in C a first quadrant, LI") the second selection means is operable to select a horizontally inverted Z-sean processing order.
9. The encoder of claim?, in which if the first selection means selects an intra-prediction mode associated with a direction in a second quadrant, the second selection means is operable to select a Z-scan processing order.
10. The encoder of claim?, in which if the first selection means selects an intra-prediction mode associated with a direction in a third quadrant, the second selection means is operable to select a vertically inverted Z-scan processing order.
11. The encoder of claim 7, in which if the first selection means selects an intra-prediction mode associated with a direction in a fourth quadrant, the second selection means is operable to select a vertically and horizontally inverted Z-scan processing order.
12. A high efficiency video coding decoder, comprising: first selection means operable to select an intra-prediction mode for a high efficiency video coding prediction unit; and responsive to the selected intra-predietion mode, second selection means operable to select one of a plurality of predetermined orders of transform unit processing; and processing means operable to process a set of transform units in the selected predetermined order.is predetermined order.U")
13. The decoder of claim 12, in which C if the first selection means selects an intra-prediction mode associated with a direction in LI") a first quadrant, C\,i 20 the second selection means is operable to select a horizontally inverted Z-scan processing order.
14. The decoder of claim 12, in which if the first selection means selects an intra-prediction mode associated with a direction in a second quadrant, the second selection means is operable to select a Z-scan processing order.
15. The decoder of claim 12, in which if the first selection means selects an intra-prediction mode associated with a direction in a third quadrant, the second selection means is operable to select a vertically inverted Z-scan processing order.
16. The decoder of claim 12, in which if thc first sciection mcans sclccts an intra-prcdiction modc associatcd with a dircction in a fourth quadrant, the second selection means is operable to select a vertically and horizontally inverted Z-scan processing order. c'J U) U, c'J