KR20170052143A

KR20170052143A - Loop filter based on memory applied in video decoder

Info

Publication number: KR20170052143A
Application number: KR1020150154229A
Authority: KR
Inventors: 김동순; 이태희; 장영종
Original assignee: 전자부품연구원
Priority date: 2015-11-04
Filing date: 2015-11-04
Publication date: 2017-05-12

Abstract

The present invention relates to a memory-based loop filter applied to a motion picture decoder.
A load data controller for loading loop filtering parameters and pixel data necessary for a loop filtering operation and storing the loop filtering parameters and the pixel data in an internal memory; horizontal edge filtering for pixel data stored in the internal memory; A single core filter for performing vertical edge filtering, and an output data controller for controlling the output of the horizontal border filtering and the output data on which the vertical border filtering is performed.
According to the present invention, since the data of one super block is sequentially stored in the double buffered internal memory, the capacity of the required internal memory can be reduced, and the processes of the horizontal border filtering and the vertical border filtering are all processed in one core filter Thus, it is possible to reduce the hardware resources and minimize the time loss of the peripheral device of the VP9 decoder by using a double buffered internal memory and a single core filter to divide the operation steps of the loop filter into a pipeline structure.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a memory-based loop filter applied to a video decoder,

The present invention relates to a memory based loop filter applied to a motion picture decoder. More particularly, the present invention relates to a hardware structure of a memory-based loop filter for a 4K class VP9 decoder, in which data of one super block is sequentially stored in a double buffered internal memory, and horizontal and vertical boundary filtering Based loop filter that processes all of the data in one core filter to reduce hardware resources.

VPx is a video compression codec product developed by On2 Technologies of USA. If the VP8 codec released in previous generation was a codec that compete with H.264, VP9 is a low-capacity, high-quality video Compression codec.

There is a growing demand for high-definition content such as full-HD content such as smartphones and tablets that support resolutions greater than full HD and PC monitors for 4K. However, it is also true that as the resolution increases, the file size becomes larger and the data usage becomes more difficult. With VP9, bandwidth is reduced to less than half that of existing video compression codecs, enabling smooth playback of high-definition video.

If the movie is streaming in the format used by the camera, use 18,000 Mbps, or 18 Gbps bandwidth for 4K picture quality. For this reason, video compression codec is used to reduce bandwidth usage as much as possible so that high-quality video can be streamed and played back. VP9 is a codec that connects VP8, but the basic principle is the same, but the bitrate can drop to half of the VP8, while maintaining the higher quality of VP9.

According to VP9, since the discontinuity of the pixels existing at the block boundary increases in the image processing of the block unit, a loop filtering operation is performed to remove the discontinuity at the boundary between the block and the block.

In the VP9 video, the super block is loop-filtered in units of prediction blocks. One super block consists of 64 pixels in the horizontal direction and 64 pixels in the vertical direction, totaling 4,096 pixels. The prediction block is 8 pixels in the horizontal direction, 8 pixels, and a total of 64 pixels. If the super-block is stored in the memory and the loop filtering operation is performed, the internal memory becomes large.

In addition, since the entire super block is stored in the internal memory, the peripheral device of the VP9 decoder waits until both the horizontal boundary filtering and the vertical boundary filtering are performed, thereby causing a time loss.

In addition, since a structure using a core filter for processing horizontal border filtering and vertical border filtering is used, there is a problem that a lot of hardware resources are used.

Korean Patent Publication No. 10-2011-0053245 (Published Date: May 19, 2011, entitled: Video Encoding System and Method Using Adaptive Loop Filter) Korean Patent Laid-Open Publication No. 10-2007-0058839 (public date: June 11, 2007, name: deblocking filter)

The present invention provides a memory-based loop filter capable of reducing a required internal memory capacity by sequentially storing data of one super block in a double buffered internal memory.

It is another object of the present invention to provide a memory-based loop filter capable of reducing hardware resources by processing both horizontal and vertical boundary filtering in one core filter.

The present invention also provides a memory-based loop filter capable of minimizing the time loss of a peripheral device of the VP9 decoder by using a double buffered internal memory and a single core filter to divide the operation steps of the loop filter into a pipeline structure And to provide a method of manufacturing the same.

The memory-based loop filter applied to the moving picture decoder according to the present invention includes a load data controller for loading loop filtering parameters and pixel data necessary for loop filtering operations and storing the loop filtering parameters and pixel data in an internal memory, A single core filter for performing horizontal edge filtering and vertical edge filtering on pixel data and an output data controller for controlling the output of the output data on which the horizontal border filtering and the vertical border filtering are performed .

In the memory-based loop filter applied to the moving picture decoder according to the present invention, the load data controller divides the pixel data into four horizontal and four vertical directions in the first and second internal memories constituting the internal memory, Buffered in units of prediction blocks and stored.

In the memory-based loop filter applied to the moving picture decoder according to the present invention, the super block including the pixel data has a size of 64 * 64 pixels, the prediction block has a size of 8 * 8 pixels, And processing is performed in units of the prediction blocks.

In the memory-based loop filter applied to the moving picture decoder according to the present invention, the moving picture decoder is a 4K class VP9 decoder.

According to the present invention, a memory-based loop filter capable of reducing the required internal memory capacity by sequentially storing data of one super block in a double buffered internal memory is provided.

In addition, a memory-based loop filter capable of reducing hardware resources is provided by processing both the horizontal boundary filtering and the vertical boundary filtering in one core filter.

In addition, a memory-based loop filter is provided that can minimize the time loss of the peripheral device of the VP9 decoder by using a double buffered internal memory and a single core filter to divide the operation steps of the loop filter into a pipeline structure It is effective.

1 is a block diagram of a memory-based loop filter according to an embodiment of the present invention.
2 is a diagram for explaining the operation of a memory-based loop filter according to an embodiment of the present invention.

It is to be understood that the specific structural or functional description of embodiments of the present invention disclosed herein is for illustrative purposes only and is not intended to limit the scope of the inventive concept But may be embodied in many different forms and is not limited to the embodiments set forth herein.

The embodiments according to the concept of the present invention can make various changes and can take various forms, so that the embodiments are illustrated in the drawings and described in detail herein. It is not intended to be exhaustive or to limit the invention to the particular forms disclosed, but on the contrary, is intended to cover all modifications, equivalents, or alternatives falling within the spirit and scope of the invention.

The terms first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The terms may be named for the purpose of distinguishing one element from another, for example, without departing from the scope of the right according to the concept of the present invention, the first element may be referred to as a second element, The component may also be referred to as a first component.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between. Other expressions that describe the relationship between components, such as "between" and "between" or "neighboring to" and "directly adjacent to" should be interpreted as well.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, the terms "comprises" or "having" and the like are used to specify that there are features, numbers, steps, operations, elements, parts or combinations thereof described herein, But do not preclude the presence or addition of one or more other features, integers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the meaning of the context in the relevant art and, unless explicitly defined herein, are to be interpreted as ideal or overly formal Do not.

Video compression is usually to reduce the amount of data representing a video signal by permanently eliminating certain redundant information in the video signal. For such compression techniques, there are various standards including MPEG, H.264, VPx, and the like.

Block-based video compression is performed on a pixel group basis with a certain size. The names of these pixel groups are various. For example, the H.264 standard specifies a 16 * 16 pixel group of pixels, referred to as a macroblock. In addition, the VPx standard specifies a group of pixels of 16 * 16 or 64 * 64 pixels, which is referred to as a super block. As such, each video frame can be divided into pixel groups, each group of pixels consisting of a plurality of smaller sized blocks. The pixel group and the blocks within the pixel group are compared with data found in the current frame or data found in the other frame to produce the predictive data and the error signal.

For example, the error signal for each block may be transformed using a discrete cosine transform (DCT), the transform coefficients obtained for each block may be quantized, and the quantized coefficients may be coded using context adaptive binary arithmetic coding ), Or the like.

The quantization level, which indicates the degree to which the transform coefficients are quantized, affects the number of bits used to represent the image data as well as the quality of the resulting decoded image. In general, when the quantization level is set to a high value, more coefficients are set to 0, resulting in a higher compression ratio, but the quality of the image is degraded. Conversely, when the quantization level is set to a low value, fewer coefficients are set to zero, resulting in better image quality, but the compression rate is poor.

Since the visual quality of the video may depend on how the quantization level is distributed over the frames in the video sequence, it is useful to control the quantization level over each frame. Some current techniques apply a uniform quantization level over each frame. However, these techniques do not consider applying low quantization levels to a portion of a visually more important frame (e.g., a human face, etc.). Similarly, these uniform quantization level techniques also do not consider applying a high quantization level to a portion of a visually less important frame (e.g., a background region, etc.).

It would be desirable to efficiently select the quantization level for each pixel group without increasing the overhead and computational complexity at the encoder / decoder.

Block artifacts appear as discontinuities between adjacent blocks. This discontinuity degrades the visual quality and degrades the reliability of the reconstructed frame for prediction of subsequent frames.

To remove this discontinuity, loop filtering in the reconstruction path can be applied to the reconstructed frame, and the invention to be described below is for this loop filtering. The choice of loop filter and the strength of the loop filter can have a significant impact on image quality. Too strong a filter may cause blurring and loss of detail, and a too weak filter may not adequately suppress discontinuity between adjacent blocks.

Although the description of the present invention has been described below with reference to the VP9 video coding format, the present invention may be implemented in connection with other video coding algorithms.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a memory-based loop filter according to an embodiment of the present invention.

1, a memory-based loop filter 1 according to an embodiment of the present invention includes a load data controller 10, a single core filter 20, an output data controller 30, a column data buffer memory, A data buffer memory, an output data buffer memory, an RDMA engine, a WDMA engine, and a loop filter main controller. For example, in one embodiment of the present invention, the motion picture decoder may be a 4K class VP9 decoder, and this case will be described below as an example. According to VP9, the discontinuity of pixels existing at a block boundary increases in the image processing of a block unit, and a loop filtering operation is performed to remove this discontinuity at the boundary portion between the block and the block.

The load data controller 10 loads the loop filtering parameters and pixel data necessary for the loop filtering operation and stores them in an internal memory.

For example, a super block including pixel data may have a size of 64 * 64 pixels, a prediction block may have a size of 8 * 8 pixels, and a super block may be configured to be processed in a prediction block unit.

For example, the load data controller 10 can store pixel data in the first internal memory 12 and the second internal memory 14 constituting the internal memory in the horizontal direction of four, For example. More specifically, the super block is loop-filtered in units of prediction blocks. One super block consists of 64 pixels in the horizontal direction and 64 pixels in the vertical direction, totaling 4,096 pixels. The prediction block is 8 pixels in the horizontal direction and 8 pixels in the vertical direction Pixel, a total of 64 pixels. If the super-block is stored in the memory and the loop filtering operation is performed, the internal memory becomes large. However, since the load data controller 10 of the present invention processes the super block in units of prediction blocks, it is possible to reduce the internal memory capacity increase. Also, if the entire super block is stored in the internal memory, the peripherals of the VP9 decoder wait until both the horizontal edge filtering and the vertical edge filtering described below are performed There is a problem that a temporal loss occurs. However, since the load data controller 10 of the present invention processes the super block in units of prediction blocks, the latency time of the peripheral device can be reduced and the overall time loss can be reduced.

The single core filter 20 performs horizontal and vertical boundary filtering on the pixel data stored in the internal memory. That is, according to the present invention, a single core filter 20 performs not only a structure in which a core filter for performing horizontal border filtering and a core filter for performing vertical border filtering are provided, but also a horizontal border filtering and a vertical border filtering do. If a structure using a core filter that processes horizontal boundary filtering and vertical boundary filtering is used, there is a problem that it occupies a large amount of hardware resources. On the other hand, according to the present invention, since the horizontal border filtering and the vertical border filtering are all performed in one single core filter 20, hardware resources can be reduced.

The output data controller 30 controls the output of the output data on which the horizontal boundary filtering and the vertical boundary filtering have been performed.

The column data buffer memory 40 buffers and temporarily stores the column data. The row data buffer memory 50 performs a function of buffering and temporarily storing row data, and the output data buffer memory 60 And buffer the output data for temporary storage.

The RDMA (Read Direct Memory Access) engine 70 reads data stored in the external memory device and directly transfers the read data to the internal memory of the load data controller 10. [

The WDMA (Write Direct Memory Access) engine 80 is a means for directly transferring data stored in an internal memory of the load data controller 10 to an external memory device.

The loop filter main controller 90 includes a load data controller 10, a single core filter 20 and an output data controller 30 to control the overall operation of the components constituting the loop filter 1 do.

2 is a diagram for explaining the operation of a memory-based loop filter according to an embodiment of the present invention.

2, in step S10, a process of loading the loop filtering parameters and the pixel data necessary for the loop filtering operation is performed by the load data controller 10, and in step S20, the load data controller 10 The pixel data is stored in the first internal memory 12 and the second internal memory 14 constituting the internal memory by double buffering the pixel data in units of four prediction units in the horizontal direction and one prediction unit in the vertical direction.

With reference to steps S10 and S20, a super block including pixel data has a size of 64 * 64 pixels, a prediction block has a size of 8 * 8 pixels, and a super block is processed in units of prediction blocks. The super block is loop-filtered in units of prediction blocks. One super block has 64 pixels in the horizontal direction and 64 pixels in the vertical direction, and a total of 4,096 pixels. The prediction block has 8 pixels in the horizontal direction and 8 pixels in the vertical direction. Pixels. If the super-block is stored in the memory and the loop filtering operation is performed, the internal memory becomes large. However, since the load data controller 10 of the present invention processes the super block in units of prediction blocks, it is possible to reduce the internal memory capacity increase. Also, if the entire super block is stored in the internal memory, the peripherals of the VP9 decoder wait until both the horizontal edge filtering and the vertical edge filtering described below are performed There is a problem that a temporal loss occurs. However, since the load data controller 10 of the present invention processes the super block in units of prediction blocks, the latency time of the peripheral device can be reduced and the overall time loss can be reduced.

In step S30, the single core filter 20 performs horizontal edge filtering and vertical edge filtering on the pixel data stored in the internal memory. If a structure using a core filter that processes horizontal boundary filtering and vertical boundary filtering is used, there is a problem that it occupies a large amount of hardware resources. On the other hand, according to the present invention, since the horizontal border filtering and the vertical border filtering are all performed in one single core filter 20, hardware resources can be reduced.

In step S40, a process of controlling the output data controller 30 to output the output data performed in the horizontal boundary filtering and the vertical boundary filtering is performed.

As described above in detail, according to the present invention, a memory-based loop filter is provided which can reduce the required internal memory capacity by sequentially storing data of one super block in a double buffered internal memory .

1: loop filter
10: load data controller
12: first internal memory
14: Second internal memory
20: Single core filter
30: Output data controller
40: column data buffer memory
50: row data buffer memory
60: Output data controller
70: Read Direct Memory Access (RDMA) engine
80: Write Direct Memory Access (WDMA) engine
90: Loop filter main controller

Claims

In a memory-based loop filter applied to a video decoder,
A load data controller for loading loop filtering parameters and pixel data necessary for loop filtering operations and storing the same in an internal memory;
A single core filter for performing horizontal edge filtering and vertical edge filtering on pixel data stored in the internal memory; And
And an output data controller for controlling output of the output data in which the horizontal boundary filtering and the vertical boundary filtering have been performed.

The method according to claim 1,
Wherein the load data controller stores the pixel data by double buffering the pixel data in a first internal memory and a second internal memory constituting the internal memory in units of four horizontally and vertically one prediction block units, A memory-based loop filter applied to

3. The method of claim 2,
Wherein the super block including the pixel data has a size of 64 * 64 pixels, the prediction block has a size of 8 * 8 pixels, and the super block is processed in the prediction block unit. Memory based loop filter.

The method according to claim 1,
Wherein the motion picture decoder is a 4K class VP9 decoder.