EP3688987A1 - Interpolation filter for an inter prediction apparatus and method for video coding - Google Patents

Interpolation filter for an inter prediction apparatus and method for video coding

Info

Publication number
EP3688987A1
EP3688987A1 EP17811734.7A EP17811734A EP3688987A1 EP 3688987 A1 EP3688987 A1 EP 3688987A1 EP 17811734 A EP17811734 A EP 17811734A EP 3688987 A1 EP3688987 A1 EP 3688987A1
Authority
EP
European Patent Office
Prior art keywords
current
integer pixel
pixels
full
integer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP17811734.7A
Other languages
German (de)
French (fr)
Inventor
Maxim Borisovitch SYCHEV
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Publication of EP3688987A1 publication Critical patent/EP3688987A1/en
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/182—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a pixel
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/513—Processing of motion vectors
    • H04N19/517—Processing of motion vectors by encoding
    • H04N19/52—Processing of motion vectors by encoding by predictive encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/523—Motion estimation or motion compensation with sub-pixel accuracy
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/537—Motion estimation other than block-based
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/59—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation

Definitions

  • the present invention relates to the field of video coding. More specifically, the invention relates to an interpolation filter for an inter prediction apparatus and method for video coding as well as an encoding apparatus and a decoding apparatus comprising such an inter prediction apparatus.
  • Video coding standards such as ITU-T H.264/AVC or ITU-T H.265/HEVC, provide a good tradeoff between these parameters. For that reason support of video coding standards is a mandatory requirement for almost any video compression application.
  • the state-of-the-art video coding standards are based on partitioning of a source frame or picture into frame or picture blocks. Processing of these blocks depend on their size, spatial position and a coding mode specified by an encoder. Coding modes can be classified into two groups according to the type of prediction: intra- and inter-prediction modes. Intra-prediction modes use pixels of the same frame (also referred to as picture or image) to generate reference samples to calculate the prediction values for the pixels of the block being reconstructed. Intra-prediction is also referred to as spatial prediction. Inter-prediction modes are designed for temporal prediction and uses reference samples of previous or next frames to predict pixels of the block of the current frame.
  • transform coding is performed for a prediction error that is the difference between an original signal and its prediction.
  • the transform coefficients and side information are encoded using an entropy coder (e.g., CABAC for AVC/H.264 and HEVC/H.265).
  • CABAC entropy coder
  • the recently adopted ITU-T H.265/HEVC standard (ISO/IEC 23008- 2:2013, "Information technology - High efficiency coding and media delivery in heterogeneous environments - Part 2: High efficiency video coding", November 2013) declares a set of state-of-the-art video coding tools that provide a reasonable tradeoff between coding efficiency and computational complexity.
  • An overview on the ITU-T H.265/HEVC standard has been given by Gary J. Sullivan, "Overview of the High
  • HEVC Efficiency Video Coding
  • ITU-T H.264/AVC ITU-T H.264/AVC video coding standard
  • the HEVC/H.265 video coding standard provides for a division of the source frame into frame blocks in the form of so- called coding units (CUs).
  • CUs coding units
  • Each of the CUs can be further split into either smaller CUs or prediction units (PUs).
  • a PU can be intra- or inter-predicted according to the type of processing applied for the pixels of PU. In case of inter-prediction, a PU represents an area of pixels that is processed by motion compensation using a motion vector specified for a PU.
  • a PU specifies a prediction mode that is selected from the set of intra-prediction modes for all the transform units (TUs) contained in this PU.
  • a TU can have different sizes (e.g., 4x4, 8x8, 16x16 and 32x32 pixels) and can be processed in different ways.
  • transform coding is performed, i.e. the prediction error is transformed with a discrete cosine transform or a discrete sine transform (in the HEVC/H.265 standard, it is applied to intra-coded blocks) and quantized.
  • reconstructed pixels contain quantization noise (it can become apparent, for examples, as blockiness between units, ringing artifacts along with sharp edges, etc.) that in-loop filters such as Deblocking Filter (DBF), Sample Adaptive Offset (SAO) and Adaptive Loop Filter (ALF) try to suppress.
  • DPF Deblocking Filter
  • SAO Sample Adaptive Offset
  • ALF Adaptive Loop Filter
  • Sub-integer pixel interpolation techniques have been developed in order to improve the prediction accuracy for a level of compression that can be achieved for inter prediction.
  • the predictive data generated during motion compensation which is used to code a video block, may correspond to sub-integer pixels, whose values may be interpolated from values for full pixels of video blocks of a reference video frame or other coded unit to which a motion vector refers.
  • the video encoder may calculate values for sub-integer pixel positions using interpolation techniques, e.g., by applying interpolation filters to a set of support pixels, e.g., values for full integer pixels and/or previously coded values of other sub-integer pixel positions.
  • JVET Joint Video Exploration Team
  • affine transforms In order to estimate and compensate the fractional-pel (or sub- integer) displacements, the full-integer pixels of the reference image has to be interpolated on the fractional-pel, i.e. sub-integer positions. To get the interpolated image on the fractional-pel positions interpolation filters are used.
  • the problem of interpolation for non-translation motion models is the variable fractional-pel displacements.
  • the quality of the interpolated image strongly depends on the properties of the interpolation filter(s).
  • Short-tap filters e.g. bilinear
  • long-tap filters e.g. sinc-based
  • Another consideration is that for motion compensation of non-translational models the complexity has been reduced by decreasing of accuracy of interpolation and prediction.
  • the invention relates to an apparatus for inter prediction of a sample value of a current full-integer pixel of a plurality of pixels of a current block of a current frame of a video signal.
  • the apparatus comprises a processing unit configured to: determine a motion vector of the current full-integer pixel on the basis of the current frame and a reference frame of the video signal and/or a motion compensation model; determine for the current full-integer pixel a corresponding sub-integer pixel in the reference frame on the basis of the motion vector of the current full-integer pixel; generate on the basis of a predefined set of filter support pixels in the current frame a set of corresponding filter support pixels in the reference frame, wherein the predefined set of filter support pixels in the current frame comprises one or more neighboring sub-integer pixels of the current full- integer pixel; determine a respective sample value, in particular luminance value, of the corresponding sub-integer pixel of the current full-integer pixel and the corresponding filter support pixels in the reference frame; and
  • an improved inter prediction apparatus is provided allowing improving the efficiency for video coding.
  • the predefined set of filter support pixels in the current frame comprises one or more vertically and/or horizontally neighboring half-integer pixels of the current full-integer pixel in the current frame.
  • the predefined set of filter support pixels in the current frame comprises one or more vertically and/or horizontally neighboring full-integer pixels of the current full-integer pixel in the current frame.
  • the spatial high-pass filter is a 5-tap filter.
  • the 5-tap filter is a symmetric filter, i.e. a filter where the first and the fifth filter coefficients are identical and the second and the fourth filter coefficients are identical.
  • the first and the fifth filter coefficients are negative, while the other filter coefficients of the 5-tap filter are positive.
  • the processing unit of the apparatus is configured to determine the respective sample values of the corresponding sub-integer pixel of the current full-integer pixel in the reference frame and of the corresponding filter support pixels in the reference frame on the basis of a bilinear interpolation of the respective neighboring full-integer pixels in the reference frame.
  • the processing unit of the apparatus is further configured to determine a respective motion vector of each of the sub- integer pixels of the filter support pixels in the current frame on the basis of the motion vector of the current full-integer pixel and one or more motion vectors of neighboring full- integer pixels of the current full-integer pixel.
  • the processing unit of the apparatus is configured to determine a respective motion vector of each of the sub-integer pixels of the filter support pixels in the current frame by determining a mean vector of the motion vector of the current full-integer pixel and one or more motion vectors of neighboring full-integer pixels of the current full-integer pixel.
  • the processing unit of the apparatus is further configured to determine the one or more motion vectors of neighboring full-integer pixels of the current full-integer pixel on the basis of the current frame and the reference frame of the video signal and/or a motion compensation model.
  • the processing unit of the apparatus is configured to determine the inter predicted sample value of the current pixel in the current frame by applying a spatial high-pass filter to the sample value of the corresponding sub-integer pixel of the current full-integer pixel in the reference frame and to the sample values of the corresponding filter support pixels in the reference frame, in case the motion vector of the current full-integer pixel is determined on the basis of a non- translational motion compensation model, and to determine the inter predicted sample value of the current pixel in the current frame on the basis of a conventional scheme, such as the conventional scheme defined in H.264/AVC and H.265/HEVC, in case the motion vector of the current full-integer pixel is determined on the basis of a translational motion compensation model.
  • a conventional scheme such as the conventional scheme defined in H.264/AVC and H.265/HEVC
  • the non-translational motion compensation model is an affine, warping and/or panoramic motion compensation model.
  • the invention relates to a corresponding method for inter prediction of a sample value of a current full-integer pixel of a plurality of pixels of a current block of a current frame of a video signal.
  • the method comprises the steps of: determining a motion vector of the current full-integer pixel on the basis of the current frame and a reference frame of the video signal and/or on the basis of a motion compensation model; determining for the current full-integer pixel a corresponding sub- integer pixel in the reference frame on the basis of the motion vector of the current full- integer pixel; generating on the basis of a predefined set of filter support pixels in the current frame a set of corresponding filter support pixels in the reference frame, wherein the predefined set of filter support pixels in the current frame comprises one or more neighboring sub-integer pixels of the current full-integer pixel; determining a respective sample value, in particular luminance value, of the corresponding sub-integer pixel of the current full-integer pixel and the corresponding filter support support
  • the inter prediction method according to the second aspect of the invention can be performed by the inter prediction apparatus according to the first aspect of the invention. Further features of the inter prediction method according to the second aspect of the invention result directly from the functionality of the inter prediction apparatus according to the first aspect of the invention and its different implementation forms described above and below.
  • the invention relates to an encoding apparatus for encoding a current frame of a video signal, wherein the encoding apparatus comprises an inter prediction apparatus according to the first aspect.
  • the invention relates to a decoding apparatus for decoding a current reconstructed frame of a compressed video signal, wherein the decoding apparatus comprises an inter prediction apparatus according to the first aspect.
  • the invention relates to a computer program product comprising program code for performing the method according to the second aspect when executed on a computer or a processor.
  • Embodiments of the invention provide in particular for the following advantages.
  • Embodiments of the invention allow performing interpolation with pixel-wise accuracy while keeping the complexity at a low level.
  • Embodiments of the invention support any kind of non-translational movements.
  • Embodiments of the invention allow removing blocking artefacts across sub-blocks edges.
  • Embodiments of the invention reduce memory bandwidth.
  • Embodiments of the invention reduce memory requirements for storing a set of filter coefficients.
  • Embodiments of the invention allow reusing well optimized in HW bilinear transform.
  • Embodiments of the invention align orientation of filtering along transformation.
  • Embodiments of the invention allow reducing ringing artifacts caused by long motion interpolation filter while improving quality of interpolated edges.
  • Embodiments of the invention allow increasing subjective quality of edges in reconstructed pictures.
  • embodiments of the invention do not require any additional signaling and, thus, can seamlessly replace existing interpolation methods for non-translational movements. Splitting the interpolation into two steps allows separating the fractional
  • Fig. 1 schematically illustrates an example of an encoding apparatus according to an embodiment comprising an inter prediction apparatus according to an embodiment
  • Fig. 2 schematically illustrates an example of a decoding apparatus according to an embodiment comprising an inter prediction apparatus according to an embodiment
  • Fig. 3 schematically illustrates an aspect of an interpolation process implemented in an inter prediction apparatus according to an embodiment
  • Fig.4 schematically illustrates different aspects of an interpolation process implemented in an inter prediction apparatus according to an embodiment
  • Fig. 5 schematically illustrates different aspects of an interpolation process implemented in an inter prediction apparatus according to an embodiment
  • Fig. 6 shows a flow diagram illustrating steps of an inter prediction method according to an embodiment.
  • Figure 1 shows an encoding apparatus 100 according to an embodiment comprising an inter prediction apparatus 144 according to an embodiment.
  • the encoding apparatus 100 is configured to encode a block of a frame of a video signal comprising a plurality of frames (also referred to as pictures or images herein), wherein each frame is dividable into a plurality of blocks and each block comprises a plurality of pixels.
  • the blocks could be macro blocks, coding tree units, coding units, prediction units and/or prediction blocks.
  • the encoding apparatus 100 is implemented in the form of a hybrid video coding encoder.
  • the first frame of a video signal is an intra frame, which is encoded using only intra prediction.
  • the embodiment of the encoding apparatus 100 shown in figure 2 further comprises an intra prediction unit 154 for intra prediction.
  • An intra frame can be decoded without information from other frames.
  • the intra prediction unit 154 can perform the intra prediction of a block on the basis of information provided by the intra estimation unit 152.
  • the blocks of subsequent frames following the first intra frame can be coded using inter or intra prediction, as selected by a mode selection unit 160.
  • the inter prediction unit 144 can be configured to perform motion compensation of a block based on motion estimation, as will be described in more detail further below.
  • the motion estimation can be performed by an inter estimation unit 142 of the encoding apparatus.
  • the functionality of the inter estimation unit 142 can be implemented as part of the inter prediction unit 144 as well.
  • a residual calculation unit 104 determines the difference between the original block and its prediction, i.e. the residual block defining the prediction error of the intra/inter picture prediction.
  • This residual block is transformed by the transformation unit 106 (for instance using a DCT) and the transformation coefficients are quantized by the quantization unit 108.
  • the output of the quantization unit 108 as well as the coding or side information provided, for instance, by the inter prediction unit 144 are further encoded by an entropy encoding unit 70.
  • a hybrid video encoder such as the encoding apparatus 00 shown in figure 1 , usually duplicates the decoder processing such that both will generate the same predictions.
  • the inverse quantization unit 10 and the inverse transformation unit perform the inverse operations of the transformation unit 106 and the quantization unit 108 and duplicate the decoded approximation of the residual block.
  • the decoded residual block data is then added to the results of the prediction, i.e. the prediction block, by the reconstruction unit 114.
  • the output of the reconstruction unit 114 can be provided to a line buffer 1 16 to be used for intra prediction and is further processed by an in-loop filter 120 for removing image artifacts.
  • the final picture is stored in a decoded picture buffer 130 and can be used as a reference frame for the inter prediction of subsequent frames.
  • Figure 2 shows a decoding apparatus 200 according to an embodiment comprising an inter prediction apparatus 244 according to an embodiment.
  • the decoding apparatus 200 is configured to decode a block of a frame of an encoded video signal.
  • the decoding apparatus 200 is implemented as a hybrid decoder.
  • An entropy decoding unit 204 performs entropy decoding of the encoded picture data, which generally can comprise prediction errors (i.e. residual blocks), motion data and other side information, which are needed, in particular, for the inter prediction apparatus 244 and an intra prediction unit 254 as well as other components of the decoding apparatus 200.
  • the inter prediction apparatus 244 or the intra prediction unit 254 of the decoding apparatus 200 shown in figure 3 are selected by a mode selection unit 260 and function in the same way as the inter prediction apparatus 144 and the intra prediction unit 154 of the encoding apparatus 100 shown in figure 1 , so that identical predictions can be generated by the encoding apparatus 100 and the decoding apparatus 200.
  • a reconstruction unit 214 of the decoding apparatus 200 is configured to reconstruct the block on the basis of the filtered predicted block and the residual block provided by the inverse quantization unit 210 and the inverse
  • the reconstructed block can be provided to a line buffer 216 used for intra prediction and the filtered block/frame can be provided to a decoded picture buffer 230 by the in-loop filter 220 for future inter predictions.
  • the apparatus 144, 244 is configured to perform an inter prediction of a sample value of a current full-integer pixel of a plurality of pixels of a current block of a current frame of a video signal.
  • the apparatus 44, 244 comprises a processing unit, which can be implemented in software and/or hardware.
  • the processing unit of the apparatus 144, 244 is configured to determine a motion vector of the current full-integer pixel.
  • the processing unit of the apparatus 144, 244 is configured to determine the motion vector of the current full-integer pixel on the basis of the current frame and a reference frame of the video signal by determining the position of the pixel in the reference frame corresponding to the current full-integer pixel in the current frame.
  • the processing unit of the apparatus 144, 244 is configured to determine the motion vector of the current full-integer pixel on the basis of an adopted motion compensation model. For instance, in the case of a translational or affine motion compensation model the motion vector of the current full- integer pixel can be determined using one or more motion vectors of pixels of the same block the current full-integer pixel belongs to.
  • motion compensation model is also referred to as motion transformational model, motion model description and the like.
  • the processing unit of the apparatus 144, 244 is further configured to determine for the current full-integer pixel the corresponding sub-integer pixel in the reference frame on the basis of the motion vector of the current full-integer pixel.
  • the processing unit of the apparatus 144, 244 is further configured to generate on the basis of a predefined set of filter support pixels in the current frame a set of corresponding filter support pixels in the reference frame.
  • the predefined set of filter support pixels in the current frame comprises one or more neighboring sub-integer pixels of the current full- integer pixel.
  • the predefined set of filter support pixels in the current frame comprises one or more vertically and/or horizontally neighboring half-integer pixels of the current full-integer pixel in the current frame.
  • the predefined set of filter support pixels in the current frame comprises the neighboring half- integer pixels above, to the left of, below and to the right of the current full-integer pixel.
  • the predefined set of filter support pixels in the current frame further comprises one or more vertically and/or horizontally neighboring full-integer pixels of the current full-integer pixel in the current frame.
  • the predefined set of filter support pixels in the current frame further comprises the neighboring full-integer pixels above, to the left of, below and to the right of the current full-integer pixel.
  • the predefined set of filter support pixels in the current frame can comprise the neighboring half-integer and full-integer pixels above, to the left of, below and to the right of the current full-integer pixel in the current frame.
  • the processing unit of the apparatus 144, 244 is further configured to determine a respective sample value, in particular a luminance value of the corresponding sub-integer pixel of the current full-integer pixel and the corresponding filter support pixels in the reference frame.
  • the processing unit of the apparatus 144, 244 is configured to determine the respective sample values of the corresponding sub-integer pixel of the current full-integer pixel in the reference frame and of the corresponding filter support pixels in the reference frame on the basis of a bilinear interpolation of the respective neighboring full-integer pixels in the reference frame.
  • Figure 3 illustrates an example of using bilinear interpolation for determining the sample value of the corresponding sub- integer pixel of the current full-integer pixel in the reference frame.
  • a reference block in the reference frame is enlarged and rotated relative to a current block comprising an exemplary current pixel of the current frame.
  • figure 3 illustrated the increased resolution used for the filter support pixels.
  • the sample value L of the corresponding sub-integer pixel of the current full-integer pixel in the reference frame can be determined by the processing unit as follows.
  • the corresponding sub- integer pixel of the current full-integer pixel has the fractional position (fdX, fdY) in a corresponding cell of the sample grid of the reference frame.
  • LO, L1 , L2, L3 are the known sample values of the neighboring full-integer pixels in the reference frame (i.e. the full- integer pixels located at the corners of the corresponding cell of the sample grid of the reference frame the corresponding sub-integer pixel of the current full-integer pixel is located in).
  • the processing unit of the apparatus 144, 244 is further configured to determine an inter predicted sample value of the current pixel in the current frame by applying a spatial high- pass filter to the sample value of the corresponding sub-integer pixel of the current full- integer pixel in the reference frame and to the sample values of the corresponding filter support pixels in the reference frame.
  • the spatial high-pass filter is a 5-tap filter.
  • the 5-tap filter is a symmetric filter, i.e. a filter where the first and the fifth filter coefficients are identical and the second and the fourth filter coefficients are identical.
  • the first and the fifth filter coefficients are negative, while the other filter coefficients of the 5-tap filter are positive.
  • the spatial high-pass filter can be applied separately in the vertical and the horizontal direction.
  • Figure 4 illustrates different stages of the processing unit performed by the processing unit of the apparatus 144, 244 using a 5-tap filter in the vertical and the horizontal direction for the example shown in figure 3.
  • the reference block is enlarged and rotated (corresponding to an affme transformation) relative to the current block, the 5-tap filters, which are vertical and horizontal in the current frame, are rotated in the reference frame.
  • inter prediction apparatus 144, 244 relate to embodiments of the inter prediction apparatus 133 as implemented in the encoding apparatus 100 as well as embodiments of the inter prediction apparatus 244 as implemented in the decoding apparatus 200.
  • the processing unit of the apparatus 144, 244 is further configured to determine a respective motion vector of each of the sub-integer pixels of the filter support pixels in the current frame on the basis of the motion vector of the current full-integer pixel and one or more motion vectors of neighboring full-integer pixels of the current full-integer pixel.
  • the processing unit of the apparatus 144, 244 is configured to determine a mean vector of the motion vector of the current full-integer pixel and one or more motion vectors of neighboring full-integer pixels of the current full-integer pixel.
  • the processing unit of the apparatus 144, 244 can compute the average, i.e. mean of the motion vector of the current full-integer pixel and the motion vector of the neighboring full-integer pixel above the current full-integer pixel. Similar to the determination of the motion vector of the current pixel the processing unit of the apparatus 144, 244 can be configured to determine the one or more motion vectors of neighboring full-integer pixels of the current full-integer pixel on the basis of the current frame and the reference frame of the video signal and/or a motion compensation model.
  • the processing unit of the apparatus 144, 244 is configured to determine the inter predicted sample value of the current pixel in the current frame by applying a spatial high-pass filter to the sample value of the corresponding sub-integer pixel of the current full-integer pixel in the reference frame and to the sample values of the corresponding filter support pixels in the reference frame, in case the motion vector of the current full-integer pixel is determined on the basis of a non-translational motion compensation model, and to determine the inter predicted sample value of the current pixel in the current frame on the basis of a conventional scheme, such as the conventional scheme defined in H.264/AVC and H.265/HEVC, in case the motion vector of the current full-integer pixel is determined on the basis of a translational motion compensation model.
  • the non-translational motion compensation model is an affine, warping and/or panoramic motion compensation model.
  • Figure 5 summarizes several aspects of embodiments of the invention described above.
  • Figure 6 shows a flow diagram illustrating steps of a corresponding inter prediction method 600 according to an embodiment.
  • the method 600 comprises the following steps: determining 601 a motion vector of the current full-integer pixel on the basis of the current frame and a reference frame of the video signal and/or on the basis of a motion compensation model; determining 603 for the current full-integer pixel a corresponding sub-integer pixel in the reference frame on the basis of the motion vector of the current full-integer pixel; generating 605 on the basis of a predefined set of filter support pixels in the current frame a set of corresponding filter support pixels in the reference frame, wherein the predefined set of filter support pixels in the current frame comprises one or more neighboring sub-integer pixels of the current full-integer pixel; determining 607 a respective sample value of the corresponding sub-integer pixel of the current full-integer pixel and the corresponding filter support pixels in the reference frame; and determining
  • the processing unit may be any kind of programmable or non-programmable circuitry that is configured to carry out the respective operations described above.
  • the processing unit may comprise hardware as well as software.
  • the processing unit may comprise one or more processors and a transitory or non-transitory memory that carries a program which causes the processing unit to perform the respective operations when the program is executed by the one or more processors.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The invention relates to an apparatus (144) and method for inter prediction of a sample value of a current full-integer pixel of a plurality of pixels of a current block of a current frame of a video signal. The apparatus (144) comprises a processing unit configured to: determine a motion vector of the current full-integer pixel on the basis of the current frame and a reference frame of the video signal and/or on the basis of a motion compensation model; determine for the current full-integer pixel a corresponding sub-integer pixel in the reference frame on the basis of the motion vector of the current full-integer pixel; generate on the basis of a predefined set of filter support pixels in the current frame a set of corresponding filter support pixels in the reference frame, wherein the predefined set of filter support pixels in the current frame comprises one or more neighboring sub-integer pixels of the current full-integer pixel; determine a respective sample value of the corresponding sub-integer pixel of the current full-integer pixel and the corresponding filter support pixels in the reference frame; and determine the inter predicted sample value of the current pixel in the current frame by applying a spatial high-pass filter to the sample value of the corresponding sub-integer pixel of the current full-integer pixel in the reference frame and to the sample values of the corresponding filter support pixels in the reference frame. Moreover, the invention relates to an encoding apparatus (100) and a decoding apparatus comprising such an inter prediction apparatus (144).

Description

DESCRIPTION
Interpolation filter for an inter prediction apparatus and method for video coding TECHNICAL FIELD
Generally, the present invention relates to the field of video coding. More specifically, the invention relates to an interpolation filter for an inter prediction apparatus and method for video coding as well as an encoding apparatus and a decoding apparatus comprising such an inter prediction apparatus.
BACKGROUND
Digital video communication and storage applications are implemented by a wide range of digital devices, e.g. digital cameras, cellular radio telephones, laptops, broadcasting systems, video teleconferencing systems, etc. One of the most important and challenging tasks of these applications is video compression. The task of video compression is complex and is constrained by two contradicting parameters: compression efficiency and computational complexity. Video coding standards, such as ITU-T H.264/AVC or ITU-T H.265/HEVC, provide a good tradeoff between these parameters. For that reason support of video coding standards is a mandatory requirement for almost any video compression application.
The state-of-the-art video coding standards are based on partitioning of a source frame or picture into frame or picture blocks. Processing of these blocks depend on their size, spatial position and a coding mode specified by an encoder. Coding modes can be classified into two groups according to the type of prediction: intra- and inter-prediction modes. Intra-prediction modes use pixels of the same frame (also referred to as picture or image) to generate reference samples to calculate the prediction values for the pixels of the block being reconstructed. Intra-prediction is also referred to as spatial prediction. Inter-prediction modes are designed for temporal prediction and uses reference samples of previous or next frames to predict pixels of the block of the current frame. After a prediction stage, transform coding is performed for a prediction error that is the difference between an original signal and its prediction. Then, the transform coefficients and side information are encoded using an entropy coder (e.g., CABAC for AVC/H.264 and HEVC/H.265). The recently adopted ITU-T H.265/HEVC standard (ISO/IEC 23008- 2:2013, "Information technology - High efficiency coding and media delivery in heterogeneous environments - Part 2: High efficiency video coding", November 2013) declares a set of state-of-the-art video coding tools that provide a reasonable tradeoff between coding efficiency and computational complexity. An overview on the ITU-T H.265/HEVC standard has been given by Gary J. Sullivan, "Overview of the High
Efficiency Video Coding (HEVC) Standard", in IEEE Transactions on Circuits and Systems for Video Technology, Vol. 22, No. 12, December 2012, the entire content of which is incorporated herein by reference. Similarly to the ITU-T H.264/AVC video coding standard, the HEVC/H.265 video coding standard provides for a division of the source frame into frame blocks in the form of so- called coding units (CUs). Each of the CUs can be further split into either smaller CUs or prediction units (PUs). A PU can be intra- or inter-predicted according to the type of processing applied for the pixels of PU. In case of inter-prediction, a PU represents an area of pixels that is processed by motion compensation using a motion vector specified for a PU. For intra prediction, the adjacent pixels of neighbor blocks are used as reference samples to predict a current block. A PU specifies a prediction mode that is selected from the set of intra-prediction modes for all the transform units (TUs) contained in this PU. A TU can have different sizes (e.g., 4x4, 8x8, 16x16 and 32x32 pixels) and can be processed in different ways. For a TU, transform coding is performed, i.e. the prediction error is transformed with a discrete cosine transform or a discrete sine transform (in the HEVC/H.265 standard, it is applied to intra-coded blocks) and quantized. Hence, reconstructed pixels contain quantization noise (it can become apparent, for examples, as blockiness between units, ringing artifacts along with sharp edges, etc.) that in-loop filters such as Deblocking Filter (DBF), Sample Adaptive Offset (SAO) and Adaptive Loop Filter (ALF) try to suppress.
Sub-integer pixel interpolation techniques have been developed in order to improve the prediction accuracy for a level of compression that can be achieved for inter prediction. In this case, the predictive data generated during motion compensation, which is used to code a video block, may correspond to sub-integer pixels, whose values may be interpolated from values for full pixels of video blocks of a reference video frame or other coded unit to which a motion vector refers. The video encoder may calculate values for sub-integer pixel positions using interpolation techniques, e.g., by applying interpolation filters to a set of support pixels, e.g., values for full integer pixels and/or previously coded values of other sub-integer pixel positions. Today's standards H.264/AVC and H.265/HEVC are based on a ¼ pel displacement resolution. The Joint Video Exploration Team (JVET) group is exploring post-HEVC video compression technologies, including non-translational motion compensation models, such as affine transforms. In order to estimate and compensate the fractional-pel (or sub- integer) displacements, the full-integer pixels of the reference image has to be interpolated on the fractional-pel, i.e. sub-integer positions. To get the interpolated image on the fractional-pel positions interpolation filters are used. The problem of interpolation for non-translation motion models is the variable fractional-pel displacements.
The quality of the interpolated image strongly depends on the properties of the interpolation filter(s). Short-tap filters (e.g. bilinear) may suppress high frequencies and make interpolated picture blurred. On the other hand, long-tap filters (e.g. sinc-based) require more memory bandwidth and may preserve high frequencies but generate some ringing artifacts in the vicinity of sharp edges. Another consideration is that for motion compensation of non-translational models the complexity has been reduced by decreasing of accuracy of interpolation and prediction.
In the proposed JEM affine motion model there are two types of motion supported: zoom and rotation. Most of the fractional-pel positions are non-constant within a prediction unit. Pixels are replaced by sub-blocks to speed up the interpolation. Within one sub-block the displacement vector is constant and translational. The complexity was reduced gradually but accuracy was reduced as well. To improve the quality of prediction the precision of motion compensation was improved by increasing the precision of motion vector displacement for sub-blocks with increasing of amount of interpolation filters. Current accuracy of interpolation filtering for non-translational motion model still requires to be improved.
Thus, there is a need for an improved interpolation filter for an inter prediction apparatus and method for video coding providing an improved video coding efficiency.
SUMMARY
It is an object of the invention to provide an improved interpolation filter for an inter prediction apparatus and method for video coding providing an improved video coding efficiency. The foregoing and other objects are achieved by the subject matter of the independent claims. Further implementation forms are apparent from the dependent claims, the description and the figures.
According to a first aspect the invention relates to an apparatus for inter prediction of a sample value of a current full-integer pixel of a plurality of pixels of a current block of a current frame of a video signal. The apparatus comprises a processing unit configured to: determine a motion vector of the current full-integer pixel on the basis of the current frame and a reference frame of the video signal and/or a motion compensation model; determine for the current full-integer pixel a corresponding sub-integer pixel in the reference frame on the basis of the motion vector of the current full-integer pixel; generate on the basis of a predefined set of filter support pixels in the current frame a set of corresponding filter support pixels in the reference frame, wherein the predefined set of filter support pixels in the current frame comprises one or more neighboring sub-integer pixels of the current full- integer pixel; determine a respective sample value, in particular luminance value, of the corresponding sub-integer pixel of the current full-integer pixel and the corresponding filter support pixels in the reference frame; and determine the inter predicted sample value of the current pixel in the current frame by applying a spatial high-pass filter to the sample value of the corresponding sub-integer pixel of the current full-integer pixel in the reference frame and to the sample values of the corresponding filter support pixels in the reference frame.
Thus, an improved inter prediction apparatus is provided allowing improving the efficiency for video coding.
In a further possible implementation form of the first aspect, the predefined set of filter support pixels in the current frame comprises one or more vertically and/or horizontally neighboring half-integer pixels of the current full-integer pixel in the current frame.
In a further possible implementation form of the first aspect, the predefined set of filter support pixels in the current frame comprises one or more vertically and/or horizontally neighboring full-integer pixels of the current full-integer pixel in the current frame. In a further possible implementation form of the first aspect, the spatial high-pass filter is a 5-tap filter. In an implementation form, the 5-tap filter is a symmetric filter, i.e. a filter where the first and the fifth filter coefficients are identical and the second and the fourth filter coefficients are identical. In an implementation form, the first and the fifth filter coefficients are negative, while the other filter coefficients of the 5-tap filter are positive. In a further possible implementation form of the first aspect, the processing unit of the apparatus is configured to determine the respective sample values of the corresponding sub-integer pixel of the current full-integer pixel in the reference frame and of the corresponding filter support pixels in the reference frame on the basis of a bilinear interpolation of the respective neighboring full-integer pixels in the reference frame.
In a further possible implementation form of the first aspect, the processing unit of the apparatus is further configured to determine a respective motion vector of each of the sub- integer pixels of the filter support pixels in the current frame on the basis of the motion vector of the current full-integer pixel and one or more motion vectors of neighboring full- integer pixels of the current full-integer pixel.
In a further possible implementation form of the first aspect, the processing unit of the apparatus is configured to determine a respective motion vector of each of the sub-integer pixels of the filter support pixels in the current frame by determining a mean vector of the motion vector of the current full-integer pixel and one or more motion vectors of neighboring full-integer pixels of the current full-integer pixel.
In a further possible implementation form of the first aspect, the processing unit of the apparatus is further configured to determine the one or more motion vectors of neighboring full-integer pixels of the current full-integer pixel on the basis of the current frame and the reference frame of the video signal and/or a motion compensation model.
In a further possible implementation form of the first aspect, the processing unit of the apparatus is configured to determine the inter predicted sample value of the current pixel in the current frame by applying a spatial high-pass filter to the sample value of the corresponding sub-integer pixel of the current full-integer pixel in the reference frame and to the sample values of the corresponding filter support pixels in the reference frame, in case the motion vector of the current full-integer pixel is determined on the basis of a non- translational motion compensation model, and to determine the inter predicted sample value of the current pixel in the current frame on the basis of a conventional scheme, such as the conventional scheme defined in H.264/AVC and H.265/HEVC, in case the motion vector of the current full-integer pixel is determined on the basis of a translational motion compensation model.
In a further possible implementation form of the first aspect, the non-translational motion compensation model is an affine, warping and/or panoramic motion compensation model.
According to a second aspect the invention relates to a corresponding method for inter prediction of a sample value of a current full-integer pixel of a plurality of pixels of a current block of a current frame of a video signal. The method comprises the steps of: determining a motion vector of the current full-integer pixel on the basis of the current frame and a reference frame of the video signal and/or on the basis of a motion compensation model; determining for the current full-integer pixel a corresponding sub- integer pixel in the reference frame on the basis of the motion vector of the current full- integer pixel; generating on the basis of a predefined set of filter support pixels in the current frame a set of corresponding filter support pixels in the reference frame, wherein the predefined set of filter support pixels in the current frame comprises one or more neighboring sub-integer pixels of the current full-integer pixel; determining a respective sample value, in particular luminance value, of the corresponding sub-integer pixel of the current full-integer pixel and the corresponding filter support pixels in the reference frame; and determining the inter predicted sample value of the current pixel in the current frame by applying a spatial high-pass filter to the sample value of the corresponding sub-integer pixel of the current full-integer pixel in the reference frame and to the sample values of the corresponding filter support pixels in the reference frame. Thus, an improved inter prediction method is provided allowing improving the efficiency for video coding.
The inter prediction method according to the second aspect of the invention can be performed by the inter prediction apparatus according to the first aspect of the invention. Further features of the inter prediction method according to the second aspect of the invention result directly from the functionality of the inter prediction apparatus according to the first aspect of the invention and its different implementation forms described above and below. According to a third aspect the invention relates to an encoding apparatus for encoding a current frame of a video signal, wherein the encoding apparatus comprises an inter prediction apparatus according to the first aspect. According to a fourth aspect the invention relates to a decoding apparatus for decoding a current reconstructed frame of a compressed video signal, wherein the decoding apparatus comprises an inter prediction apparatus according to the first aspect.
According to a fifth aspect the invention relates to a computer program product comprising program code for performing the method according to the second aspect when executed on a computer or a processor.
Embodiments of the invention provide in particular for the following advantages.
Embodiments of the invention allow performing interpolation with pixel-wise accuracy while keeping the complexity at a low level. Embodiments of the invention support any kind of non-translational movements. Embodiments of the invention allow removing blocking artefacts across sub-blocks edges. Embodiments of the invention reduce memory bandwidth. Embodiments of the invention reduce memory requirements for storing a set of filter coefficients. Embodiments of the invention allow reusing well optimized in HW bilinear transform. Embodiments of the invention align orientation of filtering along transformation. Embodiments of the invention allow reducing ringing artifacts caused by long motion interpolation filter while improving quality of interpolated edges. Embodiments of the invention allow increasing subjective quality of edges in reconstructed pictures. Moreover, embodiments of the invention do not require any additional signaling and, thus, can seamlessly replace existing interpolation methods for non-translational movements. Splitting the interpolation into two steps allows separating the fractional offset compensation from the high-pass filtering.
BRIEF DESCRIPTION OF THE DRAWINGS
Further embodiments of the invention will be described with respect to the following figures, wherein:
Fig. 1 schematically illustrates an example of an encoding apparatus according to an embodiment comprising an inter prediction apparatus according to an embodiment; Fig. 2 schematically illustrates an example of a decoding apparatus according to an embodiment comprising an inter prediction apparatus according to an embodiment;
Fig. 3 schematically illustrates an aspect of an interpolation process implemented in an inter prediction apparatus according to an embodiment;
Fig.4 schematically illustrates different aspects of an interpolation process implemented in an inter prediction apparatus according to an embodiment;
Fig. 5 schematically illustrates different aspects of an interpolation process implemented in an inter prediction apparatus according to an embodiment; and
Fig. 6 shows a flow diagram illustrating steps of an inter prediction method according to an embodiment.
In the various figures, identical reference signs will be used for identical or functionally equivalent features.
DETAILED DESCRIPTION OF EMBODIMENTS
In the following description, reference is made to the accompanying drawings, which form part of the disclosure, and in which are shown, by way of illustration, specific aspects in which the invention may be placed. It is understood that other aspects may be utilized and structural or logical changes may be made without departing from the scope of the invention. The following detailed description, therefore, is not to be taken in a limiting sense, as the scope of the invention is defined by the appended claims.
For instance, it is understood that a disclosure in connection with a described method may also hold true for a corresponding device or system configured to perform the method and vice versa. For example, if a specific method step is described, a corresponding device may include a unit to perform the described method step, even if such unit is not explicitly described or illustrated in the figures. Further, it is understood that the features of the various exemplary aspects described herein may be combined with each other, unless specifically noted otherwise. Figure 1 shows an encoding apparatus 100 according to an embodiment comprising an inter prediction apparatus 144 according to an embodiment. The encoding apparatus 100 is configured to encode a block of a frame of a video signal comprising a plurality of frames (also referred to as pictures or images herein), wherein each frame is dividable into a plurality of blocks and each block comprises a plurality of pixels. In an embodiment, the blocks could be macro blocks, coding tree units, coding units, prediction units and/or prediction blocks.
In the exemplary embodiment shown in figure 1 , the encoding apparatus 100 is implemented in the form of a hybrid video coding encoder. Usually, the first frame of a video signal is an intra frame, which is encoded using only intra prediction. To this end, the embodiment of the encoding apparatus 100 shown in figure 2 further comprises an intra prediction unit 154 for intra prediction. An intra frame can be decoded without information from other frames. The intra prediction unit 154 can perform the intra prediction of a block on the basis of information provided by the intra estimation unit 152.
The blocks of subsequent frames following the first intra frame can be coded using inter or intra prediction, as selected by a mode selection unit 160. Generally, the inter prediction unit 144 can be configured to perform motion compensation of a block based on motion estimation, as will be described in more detail further below. In an embodiment, the motion estimation can be performed by an inter estimation unit 142 of the encoding apparatus. However, in other embodiments, the functionality of the inter estimation unit 142 can be implemented as part of the inter prediction unit 144 as well. Furthermore, in the hybrid encoder embodiment shown in figure 1 a residual calculation unit 104 determines the difference between the original block and its prediction, i.e. the residual block defining the prediction error of the intra/inter picture prediction. This residual block is transformed by the transformation unit 106 (for instance using a DCT) and the transformation coefficients are quantized by the quantization unit 108. The output of the quantization unit 108 as well as the coding or side information provided, for instance, by the inter prediction unit 144 are further encoded by an entropy encoding unit 70.
A hybrid video encoder, such as the encoding apparatus 00 shown in figure 1 , usually duplicates the decoder processing such that both will generate the same predictions. Thus, in the embodiment shown in figure 1 the inverse quantization unit 10 and the inverse transformation unit perform the inverse operations of the transformation unit 106 and the quantization unit 108 and duplicate the decoded approximation of the residual block. The decoded residual block data is then added to the results of the prediction, i.e. the prediction block, by the reconstruction unit 114. Then, the output of the reconstruction unit 114 can be provided to a line buffer 1 16 to be used for intra prediction and is further processed by an in-loop filter 120 for removing image artifacts. The final picture is stored in a decoded picture buffer 130 and can be used as a reference frame for the inter prediction of subsequent frames.
Figure 2 shows a decoding apparatus 200 according to an embodiment comprising an inter prediction apparatus 244 according to an embodiment. The decoding apparatus 200 is configured to decode a block of a frame of an encoded video signal. In the embodiment shown in figure 2 the decoding apparatus 200 is implemented as a hybrid decoder. An entropy decoding unit 204 performs entropy decoding of the encoded picture data, which generally can comprise prediction errors (i.e. residual blocks), motion data and other side information, which are needed, in particular, for the inter prediction apparatus 244 and an intra prediction unit 254 as well as other components of the decoding apparatus 200. In the embodiment shown in figure 2, the inter prediction apparatus 244 or the intra prediction unit 254 of the decoding apparatus 200 shown in figure 3 are selected by a mode selection unit 260 and function in the same way as the inter prediction apparatus 144 and the intra prediction unit 154 of the encoding apparatus 100 shown in figure 1 , so that identical predictions can be generated by the encoding apparatus 100 and the decoding apparatus 200. A reconstruction unit 214 of the decoding apparatus 200 is configured to reconstruct the block on the basis of the filtered predicted block and the residual block provided by the inverse quantization unit 210 and the inverse
transformation unit 212. As in the case of the encoding apparatus 100, the reconstructed block can be provided to a line buffer 216 used for intra prediction and the filtered block/frame can be provided to a decoded picture buffer 230 by the in-loop filter 220 for future inter predictions. As already described above, the apparatus 144, 244 is configured to perform an inter prediction of a sample value of a current full-integer pixel of a plurality of pixels of a current block of a current frame of a video signal. The apparatus 44, 244 comprises a processing unit, which can be implemented in software and/or hardware. The processing unit of the apparatus 144, 244 is configured to determine a motion vector of the current full-integer pixel. In an embodiment, the processing unit of the apparatus 144, 244 is configured to determine the motion vector of the current full-integer pixel on the basis of the current frame and a reference frame of the video signal by determining the position of the pixel in the reference frame corresponding to the current full-integer pixel in the current frame. In an embodiment, the processing unit of the apparatus 144, 244 is configured to determine the motion vector of the current full-integer pixel on the basis of an adopted motion compensation model. For instance, in the case of a translational or affine motion compensation model the motion vector of the current full- integer pixel can be determined using one or more motion vectors of pixels of the same block the current full-integer pixel belongs to. As used herein, "motion compensation model" is also referred to as motion transformational model, motion model description and the like.
The processing unit of the apparatus 144, 244 is further configured to determine for the current full-integer pixel the corresponding sub-integer pixel in the reference frame on the basis of the motion vector of the current full-integer pixel.
The processing unit of the apparatus 144, 244 is further configured to generate on the basis of a predefined set of filter support pixels in the current frame a set of corresponding filter support pixels in the reference frame. The predefined set of filter support pixels in the current frame comprises one or more neighboring sub-integer pixels of the current full- integer pixel.
In an embodiment, the predefined set of filter support pixels in the current frame comprises one or more vertically and/or horizontally neighboring half-integer pixels of the current full-integer pixel in the current frame. For instance, in an embodiment the predefined set of filter support pixels in the current frame comprises the neighboring half- integer pixels above, to the left of, below and to the right of the current full-integer pixel.
In an embodiment, the predefined set of filter support pixels in the current frame further comprises one or more vertically and/or horizontally neighboring full-integer pixels of the current full-integer pixel in the current frame. For instance, in an embodiment the predefined set of filter support pixels in the current frame further comprises the neighboring full-integer pixels above, to the left of, below and to the right of the current full-integer pixel. Thus, in an embodiment, the predefined set of filter support pixels in the current frame can comprise the neighboring half-integer and full-integer pixels above, to the left of, below and to the right of the current full-integer pixel in the current frame. The processing unit of the apparatus 144, 244 is further configured to determine a respective sample value, in particular a luminance value of the corresponding sub-integer pixel of the current full-integer pixel and the corresponding filter support pixels in the reference frame.
In an embodiment, the processing unit of the apparatus 144, 244 is configured to determine the respective sample values of the corresponding sub-integer pixel of the current full-integer pixel in the reference frame and of the corresponding filter support pixels in the reference frame on the basis of a bilinear interpolation of the respective neighboring full-integer pixels in the reference frame. Figure 3 illustrates an example of using bilinear interpolation for determining the sample value of the corresponding sub- integer pixel of the current full-integer pixel in the reference frame. In figure 3, a reference block in the reference frame is enlarged and rotated relative to a current block comprising an exemplary current pixel of the current frame. Moreover, figure 3 illustrated the increased resolution used for the filter support pixels.
As can be taken from the enlarged view in figure 3, in an embodiment the sample value L of the corresponding sub-integer pixel of the current full-integer pixel in the reference frame can be determined by the processing unit as follows. The corresponding sub- integer pixel of the current full-integer pixel has the fractional position (fdX, fdY) in a corresponding cell of the sample grid of the reference frame. LO, L1 , L2, L3 are the known sample values of the neighboring full-integer pixels in the reference frame (i.e. the full- integer pixels located at the corners of the corresponding cell of the sample grid of the reference frame the corresponding sub-integer pixel of the current full-integer pixel is located in). On the basis of the fractional position (fdX, fdY) the respective areas of the rectangles corresponding to sO, s1 , s2, s3 can be calculated as follows: sO = fdX*fdY, s1 = (1-fdX)*fdY, s2 = fdX*(1-fdY), s3 = (1-fdX)*(1-fdY). Bilinear interpolation can be expressed using a 2-tap filter having the following horizontal coefficients (1-fdX, fdX) and the following vertical coefficients (1-fdY, fdY). On the basis of these weighting factors the sample value L of the corresponding sub-integer pixel of the current full-integer pixel in the reference frame can be determined on the basis of the following equation:
L = L0*s3 + L1*s2 + L2*s1 + L3*s0. As already mentioned above, the same bilinear interpolation can be used for determining the sample values for the corresponding filter support pixels in the reference frame. The processing unit of the apparatus 144, 244 is further configured to determine an inter predicted sample value of the current pixel in the current frame by applying a spatial high- pass filter to the sample value of the corresponding sub-integer pixel of the current full- integer pixel in the reference frame and to the sample values of the corresponding filter support pixels in the reference frame.
In an embodiment, the spatial high-pass filter is a 5-tap filter. In an embodiment, the 5-tap filter is a symmetric filter, i.e. a filter where the first and the fifth filter coefficients are identical and the second and the fourth filter coefficients are identical. In an embodiment, the first and the fifth filter coefficients are negative, while the other filter coefficients of the 5-tap filter are positive. In an embodiment, the spatial high-pass filter can be applied separately in the vertical and the horizontal direction.
Figure 4 illustrates different stages of the processing unit performed by the processing unit of the apparatus 144, 244 using a 5-tap filter in the vertical and the horizontal direction for the example shown in figure 3. As in the example shown in figure 3, the reference block is enlarged and rotated (corresponding to an affme transformation) relative to the current block, the 5-tap filters, which are vertical and horizontal in the current frame, are rotated in the reference frame.
In the following further embodiments of the inter prediction apparatus 144, 244, the encoding apparatus 100 and the decoding apparatus 200 will be described. In this context it will be understood that embodiments of the inter prediction apparatus 144, 244 relate to embodiments of the inter prediction apparatus 133 as implemented in the encoding apparatus 100 as well as embodiments of the inter prediction apparatus 244 as implemented in the decoding apparatus 200.
In an embodiment, the processing unit of the apparatus 144, 244 is further configured to determine a respective motion vector of each of the sub-integer pixels of the filter support pixels in the current frame on the basis of the motion vector of the current full-integer pixel and one or more motion vectors of neighboring full-integer pixels of the current full-integer pixel. To this end, in an embodiment, the processing unit of the apparatus 144, 244 is configured to determine a mean vector of the motion vector of the current full-integer pixel and one or more motion vectors of neighboring full-integer pixels of the current full-integer pixel. For instance, for determining the motion vector of the half-integer pixel above the current full-integer pixel in the current frame the processing unit of the apparatus 144, 244 can compute the average, i.e. mean of the motion vector of the current full-integer pixel and the motion vector of the neighboring full-integer pixel above the current full-integer pixel. Similar to the determination of the motion vector of the current pixel the processing unit of the apparatus 144, 244 can be configured to determine the one or more motion vectors of neighboring full-integer pixels of the current full-integer pixel on the basis of the current frame and the reference frame of the video signal and/or a motion compensation model. In an embodiment, the processing unit of the apparatus 144, 244 is configured to determine the inter predicted sample value of the current pixel in the current frame by applying a spatial high-pass filter to the sample value of the corresponding sub-integer pixel of the current full-integer pixel in the reference frame and to the sample values of the corresponding filter support pixels in the reference frame, in case the motion vector of the current full-integer pixel is determined on the basis of a non-translational motion compensation model, and to determine the inter predicted sample value of the current pixel in the current frame on the basis of a conventional scheme, such as the conventional scheme defined in H.264/AVC and H.265/HEVC, in case the motion vector of the current full-integer pixel is determined on the basis of a translational motion compensation model. In an embodiment, the non-translational motion compensation model is an affine, warping and/or panoramic motion compensation model.
Figure 5 summarizes several aspects of embodiments of the invention described above. Figure 6 shows a flow diagram illustrating steps of a corresponding inter prediction method 600 according to an embodiment. The method 600 comprises the following steps: determining 601 a motion vector of the current full-integer pixel on the basis of the current frame and a reference frame of the video signal and/or on the basis of a motion compensation model; determining 603 for the current full-integer pixel a corresponding sub-integer pixel in the reference frame on the basis of the motion vector of the current full-integer pixel; generating 605 on the basis of a predefined set of filter support pixels in the current frame a set of corresponding filter support pixels in the reference frame, wherein the predefined set of filter support pixels in the current frame comprises one or more neighboring sub-integer pixels of the current full-integer pixel; determining 607 a respective sample value of the corresponding sub-integer pixel of the current full-integer pixel and the corresponding filter support pixels in the reference frame; and determining 609 the inter predicted sample value of the current pixel in the current frame by applying a spatial high-pass filter to the sample value of the corresponding sub-integer pixel of the current full-integer pixel in the reference frame and to the sample values of the corresponding filter support pixels in the reference frame.
The processing unit may be any kind of programmable or non-programmable circuitry that is configured to carry out the respective operations described above. The processing unit may comprise hardware as well as software. For example, the processing unit may comprise one or more processors and a transitory or non-transitory memory that carries a program which causes the processing unit to perform the respective operations when the program is executed by the one or more processors.
While a particular feature or aspect of the disclosure may have been disclosed with respect to only one of several implementations or embodiments, such feature or aspect may be combined with one or more other features or aspects of the other implementations or embodiments as may be desired and advantageous for any given or particular application. Furthermore, to the extent that the terms "include", "have", "with", or other variants thereof are used in either the detailed description or the claims, such terms are intended to be inclusive in a manner similar to the term "comprise". Also, the terms "exemplary", "for example" and "e.g." are merely meant as an example, rather than the best or optimal. The terms "coupled" and "connected", along with derivatives may have been used. It should be understood that these terms may have been used to indicate that two elements cooperate or interact with each other regardless whether they are in direct physical or electrical contact, or they are not in direct contact with each other.
Although specific aspects have been illustrated and described herein, it will be appreciated by those of ordinary skill in the art that a variety of alternate and/or equivalent implementations may be substituted for the specific aspects shown and described without departing from the scope of the present disclosure. This application is intended to cover any adaptations or variations of the specific aspects discussed herein.
Although the elements in the following claims are recited in a particular sequence with corresponding labeling, unless the claim recitations otherwise imply a particular sequence for implementing some or all of those elements, those elements are not necessarily intended to be limited to being implemented in that particular sequence. Many alternatives, modifications, and variations will be apparent to those skilled in the art in light of the above teachings. Of course, those skilled in the art readily recognize that there are numerous applications of the invention beyond those described herein. While the invention has been described with reference to one or more particular embodiments, those skilled in the art recognize that many changes may be made thereto without departing from the scope of the invention. It is therefore to be understood that within the scope of the appended claims and their equivalents, the invention may be practiced otherwise than as specifically described herein.

Claims

1. An apparatus (144, 244) for inter prediction of a sample value of a current full- integer pixel of a plurality of pixels of a current block of a current frame of a video signal, wherein the apparatus ( 44, 244) comprises a processing unit configured to: determine a motion vector of the current full-integer pixel on the basis of the current frame and a reference frame of the video signal and/or a motion compensation model; determine for the current full-integer pixel a corresponding sub-integer pixel in the reference frame on the basis of the motion vector of the current full-integer pixel; generate on the basis of a predefined set of filter support pixels in the current frame a set of corresponding filter support pixels in the reference frame, wherein the predefined set of filter support pixels in the current frame comprises one or more neighboring sub-integer pixels of the current full-integer pixel; determine a respective sample value of the corresponding sub-integer pixel of the current full-integer pixel and the corresponding filter support pixels in the reference frame; and determine the inter predicted sample value of the current pixel in the current frame by applying a spatial high-pass filter to the sample value of the corresponding sub-integer pixel of the current full-integer pixel in the reference frame and to the sample values of the corresponding filter support pixels in the reference frame.
2. The apparatus (144, 244) of claim 1 , wherein the predefined set of filter support pixels in the current frame comprises one or more vertically and/or horizontally neighboring half-integer pixels of the current full-integer pixel in the current frame.
3. The apparatus (144, 244) of claim 1 or 2, wherein the predefined set of filter support pixels in the current frame comprises one or more vertically and/or horizontally neighboring full-integer pixels of the current full-integer pixel in the current frame.
4. The apparatus (144, 244) of any one of the preceding claims, wherein the spatial high-pass filter is a 5-tap filter.
5. The apparatus (144, 244) of any one of the preceding claims, wherein the processing unit of the apparatus (144, 244) is configured to determine the respective sample values of the corresponding sub-integer pixel of the current full-integer pixel in the reference frame and of the corresponding filter support pixels in the reference frame on the basis of a bilinear interpolation of the respective neighboring full-integer pixels in the reference frame.
6. The apparatus (144, 244) of any one of the preceding claims, wherein the processing unit of the apparatus (144, 244) is further configured to determine a respective motion vector of each of the sub-integer pixels of the filter support pixels in the current frame on the basis of the motion vector of the current full-integer pixel and one or more motion vectors of neighboring full-integer pixels of the current full-integer pixel.
7. The apparatus (144, 244) of claim 6, wherein the processing unit of the apparatus (144, 244) is configured to determine a respective motion vector of each of the sub-integer pixels of the filter support pixels in the current frame by determining a mean vector of the motion vector of the current full-integer pixel and one or more motion vectors of neighboring full-integer pixels of the current full-integer pixel.
8. The apparatus (144, 244) of claim 6 or 7, wherein the processing unit of the apparatus (144, 244) is further configured to determine the one or more motion vectors of neighboring full-integer pixels of the current full-integer pixel on the basis of the current frame and the reference frame of the video signal and/or a motion compensation model.
9. The apparatus (144, 244) of any one of the preceding claims, wherein the processing unit of the apparatus (144, 244) is configured to determine the inter predicted sample value of the current pixel in the current frame by applying a spatial high-pass filter to the sample value of the corresponding sub-integer pixel of the current full-integer pixel in the reference frame and to the sample values of the corresponding filter support pixels in the reference frame, in case the motion vector of the current full-integer pixel is determined on the basis of a non-translational motion compensation model, and to determine the inter predicted sample value of the current pixel in the current frame on the basis of a conventional scheme, in case the motion vector of the current full-integer pixel is determined on the basis of a translational motion compensation model.
10. The apparatus (144, 244) of claim 9, wherein the non-translational motion compensation model is an affine, warping and/or panoramic motion compensation model.
11. A method (600) for inter prediction of a sample value of a current full-integer pixel of a plurality of pixels of a current block of a current frame of a video signal, wherein the method (600) comprises: determining (601 ) a motion vector of the current full-integer pixel on the basis of the current frame and a reference frame of the video signal and/or on the basis of a motion compensation model; determining (603) for the current full-integer pixel a corresponding sub-integer pixel in the reference frame on the basis of the motion vector of the current full-integer pixel; generating (605) on the basis of a predefined set of filter support pixels in the current frame a set of corresponding filter support pixels in the reference frame, wherein the predefined set of filter support pixels in the current frame comprises one or more neighboring sub-integer pixels of the current full-integer pixel; determining (607) a respective sample value of the corresponding sub-integer pixel of the current full-integer pixel and the corresponding filter support pixels in the reference frame; and determining (609) the inter predicted sample value of the current pixel in the current frame by applying a spatial high-pass filter to the sample value of the corresponding sub-integer pixel of the current full-integer pixel in the reference frame and to the sample values of the corresponding filter support pixels in the reference frame.
12. An encoding apparatus (100) for encoding a current frame of a video signal, wherein the encoding apparatus (100) comprises an inter prediction apparatus (144) according to any one of claims 1 to 10.
13. A decoding apparatus (200) for decoding a current reconstructed frame of a compressed video signal, wherein the decoding apparatus (200) comprises an inter prediction apparatus (244) according to any one of claims 1 to 10.
14. A computer program product comprising program code for performing the method of claim 1 1 , when executed on a computer or a processor.
EP17811734.7A 2017-11-07 2017-11-07 Interpolation filter for an inter prediction apparatus and method for video coding Pending EP3688987A1 (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/RU2017/000830 WO2019093916A1 (en) 2017-11-07 2017-11-07 Interpolation filter for an inter prediction apparatus and method for video coding

Publications (1)

Publication Number Publication Date
EP3688987A1 true EP3688987A1 (en) 2020-08-05

Family

ID=60629774

Family Applications (2)

Application Number Title Priority Date Filing Date
EP17811734.7A Pending EP3688987A1 (en) 2017-11-07 2017-11-07 Interpolation filter for an inter prediction apparatus and method for video coding
EP18715817.5A Pending EP3688988A1 (en) 2017-11-07 2018-03-06 Interpolation filter for an inter prediction apparatus and method for video coding

Family Applications After (1)

Application Number Title Priority Date Filing Date
EP18715817.5A Pending EP3688988A1 (en) 2017-11-07 2018-03-06 Interpolation filter for an inter prediction apparatus and method for video coding

Country Status (8)

Country Link
US (2) US11375199B2 (en)
EP (2) EP3688987A1 (en)
JP (2) JP7005854B2 (en)
KR (2) KR102357142B1 (en)
CN (2) CN111201791B (en)
BR (2) BR112020008896A2 (en)
SG (2) SG11202003260WA (en)
WO (2) WO2019093916A1 (en)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019093916A1 (en) 2017-11-07 2019-05-16 Huawei Technologies Co., Ltd Interpolation filter for an inter prediction apparatus and method for video coding
WO2019190342A1 (en) 2018-03-29 2019-10-03 Huawei Technologies Co., Ltd Image processing apparatus and method
US11356707B2 (en) * 2019-09-23 2022-06-07 Qualcomm Incorporated Signaling filters for video processing
KR20220066974A (en) * 2019-09-30 2022-05-24 후아웨이 테크놀러지 컴퍼니 리미티드 Interpolation filter for inter prediction apparatus and method for video coding
EP4062638A4 (en) * 2019-12-27 2023-01-11 Zhejiang Dahua Technology Co., Ltd Affine prediction method and related devices
CN115567709A (en) * 2020-07-30 2023-01-03 北京达佳互联信息技术有限公司 Method and apparatus for encoding samples at pixel locations in sub-blocks
WO2022032028A1 (en) * 2020-08-06 2022-02-10 Beijing Dajia Internet Information Technology Co., Ltd. Methods and apparatuses for affine motion-compensated prediction refinement
CN113259669B (en) * 2021-03-25 2023-07-07 浙江大华技术股份有限公司 Encoding method, encoding device, electronic device and computer readable storage medium

Family Cites Families (49)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1274254B1 (en) 1995-08-29 2011-07-06 Sharp Kabushiki Kaisha Video coding device and video decoding device with a motion compensated interframe prediction
US20030222998A1 (en) * 2000-12-20 2003-12-04 Satoru Yamauchi Digital still camera system and method
US20030179827A1 (en) * 2002-03-19 2003-09-25 Kenji Sugiyama Video emphasis encoding method and apparatus, video de-emphasis decoding method and apparatus, video emphasis transmitting method and apparatus and video de-emphasis receiving method and apparatus
US7782951B2 (en) 2004-05-13 2010-08-24 Ittiam Systems (P) Ltd. Fast motion-estimation scheme
US7792192B2 (en) 2004-11-19 2010-09-07 Analog Devices, Inc. System and method for sub-pixel interpolation in motion vector estimation
US8208564B2 (en) 2005-06-24 2012-06-26 Ntt Docomo, Inc. Method and apparatus for video encoding and decoding using adaptive interpolation
EP1841230A1 (en) 2006-03-27 2007-10-03 Matsushita Electric Industrial Co., Ltd. Adaptive wiener filter for video coding
US8208553B2 (en) * 2006-05-04 2012-06-26 Altera Corporation Methods and apparatus for quarter-pel refinement in a SIMD array processor
EP2127391A2 (en) 2007-01-09 2009-12-02 Nokia Corporation Adaptive interpolation filters for video coding
US9967590B2 (en) * 2008-04-10 2018-05-08 Qualcomm Incorporated Rate-distortion defined interpolation for video coding based on fixed filter or adaptive filter
US8971412B2 (en) * 2008-04-10 2015-03-03 Qualcomm Incorporated Advanced interpolation techniques for motion compensation in video coding
US9077971B2 (en) * 2008-04-10 2015-07-07 Qualcomm Incorporated Interpolation-like filtering of integer-pixel positions in video coding
US8705622B2 (en) * 2008-04-10 2014-04-22 Qualcomm Incorporated Interpolation filter support for sub-pixel resolution in video coding
US20090257499A1 (en) * 2008-04-10 2009-10-15 Qualcomm Incorporated Advanced interpolation techniques for motion compensation in video coding
US8831086B2 (en) * 2008-04-10 2014-09-09 Qualcomm Incorporated Prediction techniques for interpolation in video coding
US8804831B2 (en) * 2008-04-10 2014-08-12 Qualcomm Incorporated Offsets at sub-pixel resolution
US8462842B2 (en) * 2008-04-10 2013-06-11 Qualcomm, Incorporated Symmetry for interpolation filtering of sub-pixel positions in video coding
US8750378B2 (en) * 2008-09-23 2014-06-10 Qualcomm Incorporated Offset calculation in switched interpolation filters
JP5446198B2 (en) * 2008-10-03 2014-03-19 富士通株式会社 Image prediction apparatus and method, image encoding apparatus, and image decoding apparatus
US8542732B1 (en) 2008-12-23 2013-09-24 Elemental Technologies, Inc. Video encoder using GPU
EP2360925A1 (en) 2010-01-20 2011-08-24 Siemens Aktiengesellschaft Method for reducing noise for coding of noisy images or image sequences
KR101682147B1 (en) 2010-04-05 2016-12-05 삼성전자주식회사 Method and apparatus for interpolation based on transform and inverse transform
US9094658B2 (en) 2010-05-10 2015-07-28 Mediatek Inc. Method and apparatus of adaptive loop filtering
CN103238320B (en) * 2010-09-30 2016-06-01 三星电子株式会社 By the method and apparatus using smooth interpolation wave filter that image is interpolated
US20120134425A1 (en) * 2010-11-29 2012-05-31 Faouzi Kossentini Method and System for Adaptive Interpolation in Digital Video Coding
EP2651137A4 (en) * 2010-12-08 2016-03-02 Lg Electronics Inc Intra prediction method and encoding apparatus and decoding apparatus using same
US10045046B2 (en) * 2010-12-10 2018-08-07 Qualcomm Incorporated Adaptive support for interpolating values of sub-pixels for video coding
US9172972B2 (en) * 2011-01-05 2015-10-27 Qualcomm Incorporated Low complexity interpolation filtering with adaptive tap size
US20120177104A1 (en) 2011-01-12 2012-07-12 Madhukar Budagavi Reduced Complexity Adaptive Loop Filter (ALF) for Video Coding
ES2770320T3 (en) * 2011-06-28 2020-07-01 Samsung Electronics Co Ltd Image interpolation using an asymmetric interpolation filter
NO335667B1 (en) 2011-06-29 2015-01-19 Cisco Systems Int Sarl Method of video compression
US9055304B2 (en) * 2011-07-01 2015-06-09 Qualcomm Incorporated Reduced resolution pixel interpolation
JP6242793B2 (en) * 2011-09-15 2017-12-06 ヴィド スケール インコーポレイテッド System and method for spatial prediction
US9357235B2 (en) 2011-10-13 2016-05-31 Qualcomm Incorporated Sample adaptive offset merged with adaptive loop filter in video coding
US9445088B2 (en) 2012-04-09 2016-09-13 Qualcomm Incorporated LCU-based adaptive loop filtering for video coding
US9906804B2 (en) * 2014-01-16 2018-02-27 Qualcomm Incorporated Reference layer sample position derivation for scalable video coding
US11122262B2 (en) 2014-06-27 2021-09-14 Samsung Electronics Co., Ltd. System and method for motion compensation in video coding
CN104363451B (en) * 2014-10-27 2019-01-25 华为技术有限公司 Image prediction method and relevant apparatus
US10104361B2 (en) * 2014-11-14 2018-10-16 Samsung Electronics Co., Ltd. Coding of 360 degree videos using region adaptive smoothing
CN106331722B (en) 2015-07-03 2019-04-26 华为技术有限公司 Image prediction method and relevant device
KR102146436B1 (en) * 2015-09-25 2020-08-20 후아웨이 테크놀러지 컴퍼니 리미티드 Apparatus and method for video motion compensation
CA2999826C (en) * 2015-09-25 2020-10-13 Huawei Technologies Co., Ltd. Apparatus and method for video motion compensation with selectable interpolation filter
MY185713A (en) * 2015-09-25 2021-05-31 Huawei Tech Co Ltd Apparatus and method for video motion compensation
CN113301334B (en) * 2015-11-17 2023-11-03 华为技术有限公司 Method and apparatus for adaptive filtering of video coding samples
FR3044449B1 (en) 2015-12-01 2017-11-24 E2V Semiconductors METHOD FOR PROCESSING SIGNALS FROM A COLOR IMAGE TAKING ARRAY, AND CORRESPONDING SENSOR
US11134259B2 (en) * 2016-01-15 2021-09-28 Interdigital Madison Patent Holdings, Sas System and method for enhanced motion compensation using adaptive filtering
US11405611B2 (en) * 2016-02-15 2022-08-02 Qualcomm Incorporated Predicting filter coefficients from fixed filters for video coding
US10382781B2 (en) * 2016-09-28 2019-08-13 Qualcomm Incorporated Interpolation filters for intra prediction in video coding
WO2019093916A1 (en) 2017-11-07 2019-05-16 Huawei Technologies Co., Ltd Interpolation filter for an inter prediction apparatus and method for video coding

Also Published As

Publication number Publication date
US11375199B2 (en) 2022-06-28
KR20200055139A (en) 2020-05-20
CN111201791B (en) 2022-05-24
CN111213380B (en) 2022-10-28
BR112020009019A2 (en) 2020-12-01
JP6989699B2 (en) 2022-01-05
US20200244965A1 (en) 2020-07-30
WO2019093922A1 (en) 2019-05-16
US11297324B2 (en) 2022-04-05
CN111201791A (en) 2020-05-26
SG11202003298YA (en) 2020-05-28
CN111213380A (en) 2020-05-29
SG11202003260WA (en) 2020-05-28
JP2021502013A (en) 2021-01-21
KR20200057088A (en) 2020-05-25
JP2021502031A (en) 2021-01-21
WO2019093916A1 (en) 2019-05-16
BR112020008896A2 (en) 2020-10-20
KR102357142B1 (en) 2022-02-08
US20200267407A1 (en) 2020-08-20
KR102359415B1 (en) 2022-02-08
JP7005854B2 (en) 2022-01-24
EP3688988A1 (en) 2020-08-05

Similar Documents

Publication Publication Date Title
US11297324B2 (en) Interpolation filter for an inter prediction apparatus and method for video coding
US20180332292A1 (en) Method and apparatus for intra prediction mode using intra prediction filter in video and image compression
AU2010279841A1 (en) Apparatus and method for deblocking filtering image data and video decoding apparatus and method using the same
EP3685581A1 (en) Devices and methods for image and video coding
US11202082B2 (en) Image processing apparatus and method
US11228780B2 (en) Inter prediction apparatus and method for video coding
EP3698542B1 (en) In-loop filter apparatus, method and computer program product for video coding

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20200427

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20220127

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20240315