EP2241111A1 - Methods and apparatus for video encoding and decoding using parametric filtering - Google Patents
Methods and apparatus for video encoding and decoding using parametric filteringInfo
- Publication number
- EP2241111A1 EP2241111A1 EP09700816A EP09700816A EP2241111A1 EP 2241111 A1 EP2241111 A1 EP 2241111A1 EP 09700816 A EP09700816 A EP 09700816A EP 09700816 A EP09700816 A EP 09700816A EP 2241111 A1 EP2241111 A1 EP 2241111A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- filters
- picture
- parametric
- reference picture
- filtering
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/573—Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/103—Selection of coding mode or of prediction mode
- H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/117—Filters, e.g. for pre-processing or post-processing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/136—Incoming video signal characteristics or properties
- H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
- H04N19/82—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop
Definitions
- the present principles relate generally to video encoding and decoding and, more particularly, to methods and apparatus for video coding using parametric filtering.
- MCP motion compensated prediction
- DCP disparity compensated prediction
- MVC multi-view video coding
- a multi-view video coding (MVC) sequence is a set of two or more video sequences that capture the same scene from a different view point.
- mismatches in the video content that are beyond translational displacement, such as focus changes, motion blur in monoscopic video, and illumination and/or focus mismatches across different views in multi-view video coding.
- the exhibited mismatches may be localized such that different portions of a video frame can undergo different types of change with respect to the corresponding areas in one or more frames used as reference.
- different types of blurriness/sharpness mismatches will be associated to objects with different depths.
- motion blur in monoscopic video objects moving in different directions could result in directional blurring.
- a two-pass encoding scheme can be utilized, in which an initial search and filter estimation are performed first to adaptively design filters based on the differences between the current frame and the reference frame(s). Higher coding efficiency is achieved by the preceding described two-pass encoding scheme as new references are created using the estimated filters.
- Such scheme significantly increases encoding complexity and also increases the overhead since we transmit filter coefficients for every frame encoded with this scheme.
- blur compensation For focus changes and/or camera panning, a technique referred to as blur compensation was proposed, in which a fixed set of blurring (lowpass) filters are used to generate blurred reference frames for video coding.
- This technique has two shortcomings for the scenarios we consider. First, the filter selection is made only at the frame-level, i.e., applying different filters to different parts of a frame was not considered. Second, this method relies on a very limited pre-defined filter set (lowpass only).
- an adaptive reference filtering approach which is a two-pass encoding scheme.
- To encode a frame with inter-view prediction it was proposed to first perform an initial disparity estimation. By exploiting the disparity fields as an estimation of scene depth, video frames are partitioned into regions which correspond to different scene-depth levels. For each depth level, a spatial filter is adaptively designed based on the difference between the current frame and the reference frame to minimize the residue energy. Such a design approach is able to address depth-dependent focus mismatches exhibited across different views. The estimated filters are applied to the reference frame to create filtered references.
- the encoder selects the predictor (filtered or unfiltered) that provides the lowest rate-distortion cost (RD-cost), thus ensuring the highest coding efficiency.
- the overhead frame-wise filter coefficients
- this two-pass method significantly increases encoding complexity. The additional steps (initial search and filter estimation) are necessary if we do not have prior knowledge about the mismatch.
- an apparatus includes an encoder for encoding a picture.
- the encoder performs filtering of at least one reference picture to obtain at least one filtered reference picture, and predictively codes the picture using the at least one filtered reference picture.
- a method includes encoding a picture.
- the encoding step includes performing filtering of at least one reference picture to obtain at least one filtered reference picture, and predictively coding the picture using the at least one filtered reference picture.
- an apparatus there is provided an apparatus.
- the apparatus includes a decoder for decoding a picture.
- the decoder performs filtering of at least one reference picture to obtain at least one filtered reference picture, and predictively decodes the picture using the at least one filtered reference picture.
- a method includes decoding a picture.
- the decoding step includes performing filtering of at least one reference picture to obtain at least one filtered reference picture, and predictively decoding the picture using the at least one filtered reference picture.
- FIG. 1 is a block diagram for an exemplary Multi-view Video Coding (MVC) encoder to which the present principles may be applied, in accordance with an embodiment of the present principles;
- MVC Multi-view Video Coding
- FIG. 2 is a block diagram for an exemplary Multi-view Video Coding (MVC) decoder to which the present principles may be applied, in accordance with an embodiment of the present principles
- FIG. 3 is a flow diagram for an exemplary method for video encoding using a default parametric filter set, in accordance with an embodiment of the present principles
- FIG. 4 is a flow diagram for an exemplary method for video encoding using adaptive selection for a parametric filter set, in accordance with an embodiment of the present principles
- FIG. 5 is a flow diagram for an exemplary method for video decoding using a default parametric filter set, in accordance with an embodiment of the present principles.
- FIG. 6 is a flow diagram for an exemplary method for video decoding using adaptive selection for a parametric filter set, in accordance with an embodiment of the present principles.
- the present principles are directed to methods and apparatus for video coding using parametric filtering.
- processor or “controller” should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (“DSP”) hardware, read-only memory (“ROM”) for storing software, random access memory (“RAM”), and non-volatile storage.
- DSP digital signal processor
- ROM read-only memory
- RAM random access memory
- any switches shown in the figures are conceptual only. Their function may be carried out through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, the particular technique being selectable by the implementer as more specifically understood from the context.
- any element expressed as a means for performing a specified function is intended to encompass any way of performing that function including, for example, a) a combination of circuit elements that performs that function or b) software in any form, including, therefore, firmware, microcode or the like, combined with appropriate circuitry for executing that software to perform the function.
- the present principles as defined by such claims reside in the fact that the functionalities provided by the various recited means are combined and brought together in the manner which the claims call for. It is thus regarded that any means that can provide those functionalities are equivalent to those shown herein.
- T, "and/or”, and “at least one of, for example, in the cases of “A/B”, “A and/or B” and “at least one of A and B”, is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of both options (A and B).
- such phrasing is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of the third listed option (C) only, or the selection of the first and the second listed options (A and B) only, or the selection of the first and third listed options (A and C) only, or the selection of the second and third listed options (B and C) only, or the selection of all three options (A and B and C).
- This may be extended, as readily apparent by one of ordinary skill in this and related arts, for as many items listed.
- cross-view and “inter-view” both refer to pictures that belong to a view other than a current view.
- an exemplary Multi-view Video Coding (MVC) encoder is indicated generally by the reference numeral 100.
- the encoder 100 includes a combiner 105 having an output connected in signal communication with an input of a transformer 1 10.
- An output of the transformer 1 10 is connected in signal communication with an input of quantizer 115.
- An output of the quantizer 1 15 is connected in signal communication with an input of an entropy coder 120 and an input of an inverse quantizer 125.
- An output of the inverse quantizer 125 is connected in signal communication with an input of an inverse transformer 130.
- An output of the inverse transformer 130 is connected in signal communication with a first non-inverting input of a combiner 135.
- An output of the combiner 135 is connected in signal communication with an input of an intra predictor 145 and an input of a deblocking filter 150.
- An output of the deblocking filter 150 is connected in signal communication with an input of a reference picture store 155 (for view i).
- An output of the reference picture store 155 is connected in signal communication with a first input of a motion compensator 175 and a first input of a motion estimator 180.
- An output of the motion estimator 180 is connected in signal communication with a second input of the motion compensator 175
- An output of a reference picture store 160 (for other views) is connected in signal communication with a first input of a disparity/illumination estimator 170 and a first input of a disparity/illumination compensator 165.
- An output of the disparity/illumination estimator 170 is connected in signal communication with a second input of the disparity/illumination compensator 165.
- An output of the entropy decoder 120 is available as an output of
- a non-inverting input of the combiner 105 is available as an input of the encoder 100, and is connected in signal communication with a second input of the disparity/illumination estimator 170, and a second input of the motion estimator 180.
- An output of a switch 185 is connected in signal communication with a second non- inverting input of the combiner 135 and with an inverting input of the combiner 105.
- the switch 185 includes a first input connected in signal communication with an output of the motion compensator 175, a second input connected in signal communication with an output of the disparity/illumination compensator 165, and a third input connected in signal communication with an output of the intra predictor 145.
- a mode decision module 140 has an output connected to the switch 185 for controlling which input is selected by the switch 185.
- an exemplary Multi-view Video Coding (MVC) decoder is indicated generally by the reference numeral 200.
- the decoder 200 includes an entropy decoder 205 having an output connected in signal communication with an input of an inverse quantizer 210.
- An output of the inverse quantizer is connected in signal communication with an input of an inverse transformer 215.
- An output of the inverse transformer 215 is connected in signal communication with a first non- inverting input of a combiner 220.
- An output of the combiner 220 is connected in signal communication with an input of a deblocking filter 225 and an input of an intra predictor 230.
- An output of the deblocking filter 225 is connected in signal communication with an input of a reference picture store 240 (for view i).
- An output of the reference picture store 240 is connected in signal communication with a first input of a motion compensator 235.
- An output of a reference picture store 245 (for other views) is connected in signal communication with a first input of a disparity/illumination compensator 250.
- An input of the entropy coder 205 is available as an input to the decoder 200, for receiving a residue bitstream.
- an input of a mode module 260 is also available as an input to the decoder 200, for receiving control syntax to control which input is selected by the switch 255.
- a second input of the motion compensator 235 is available as an input of the decoder 200, for receiving motion vectors.
- a second input of the disparity/illumination compensator 250 is available as an input to the decoder 200, for receiving disparity vectors and illumination compensation syntax.
- An output of a switch 255 is connected in signal communication with a second non-inverting input of the combiner 220.
- a first input of the switch 255 is connected in signal communication with an output of the disparity/illumination compensator 250.
- a second input of the switch 255 is connected in signal communication with an output of the motion compensator 235.
- a third input of the switch 255 is connected in signal communication with an output of the intra predictor 230.
- An output of the mode module 260 is connected in signal communication with the switch 255 for controlling which input is selected by the switch 255.
- An output of the deblocking filter 225 is available as an output of the decoder.
- the present principles are directed to methods and apparatus for video coding using parametric filtering.
- pre-defined filter sets that are designed based on the characteristics of the mismatch to be compensated.
- Possible examples of such mismatches are focus mismatch in inter-view prediction for multi-view video coding (MVC) and directional motion blur in monoscopic video.
- MVC multi-view video coding
- Multiple parametric filters can be supported in our approach such that localized compensation can be applied.
- parametric filters are designed based on the knowledge of the target mismatch to be compensated for, such as isotropic filters for focus change and directional filters for motion blur.
- embodiments of the present principles are capable of providing application specific pre-defined filter sets.
- Multiple parametric filters can be designed in order to compensate for localized mismatches.
- new reference frames can be generated by applying the parametric filters to improve coding efficiency. Note that although the parametric filters are designed beforehand, the decision of whether to use them, which filter(s) to be used, and the ordering of the filters for generating new reference frames, can be adaptively determined based on estimations of the mismatch.
- embodiments of the present principles can reduce complexity as compared to other methods that also utilize reference filtering (which estimate filters with an initial search for each frame to be encoded), while still efficiently capturing the mismatches exhibited in the video content.
- the filters are parameterized according to the properties of the mismatch and the coding decision can be made adaptive, the coding efficiency is improved. That is, while we keep the overhead very small (pre-defined filter set instead of one filter set for every frame), high coding efficiency can be achieved since the filters are parameterized by exploiting the properties of the mismatch and the selection is adaptive to the estimated difference between video frames.
- the new coding scheme avoids performing two- pass coding as in adaptive reference filtering (ARF), while the parametric filters to be utilized can still be designed/selected according the difference between the current frame and a reference frame(s). For example, we can fix the structure of the filter, and the filter parameters can be adjusted, so that the filter can be changed to a lowpass, bandpass and/or highpass filter accordingly.
- ARF adaptive reference filtering
- Multiple parametric filters can be constructed to compensate for localized and/or different types of mismatch within a frame. After designing the set of filters, the filters are made available to the encoder such that the encoder can use them to generate new filtered reference frames for higher coding efficiency (application specific pre-defined filter set).
- video data captured simultaneously by multiple cameras is encoded.
- views we refer to the respective video from the different cameras as views.
- Inter-view prediction refers to encoding a frame using frames from other views as references.
- Frames from different views may exhibit focus mismatch, due to heterogeneous camera settings and/or different shooting positions/orientations. It has been shown that the differences exhibited in images captured by different cameras can be represented in terms of the focus setting mismatches and the depths of objects.
- the mismatches lead to differences in optical transfer functions which can be represented as blurring (lowpass)/sharpening(enhancement) filters that are circular symmetric in the spatial domain.
- TABLES 1A and 1 B show parametric filters with 3X3 and 5x5 symmetric constraints, respectively.
- TABLES 2A and 2B show parametric filters with 3x3 and 5x5 vertical/horizontal symmetric constraints, respectively.
- we construct a set of filters by estimating and collecting parametric filters that minimize the prediction error for sequences with focus mismatches, and classify/cluster the collected parametric filters into some representative filters. Then, the obtained filter set can serve as the parametric filters that are used to compensate for focus mismatch in multi-view video coding inter-view prediction.
- the pre-defined filter set provides encoder options when coding multi-view video. Depending on the focus setting difference and the depth composition of the scene, the image will exhibit different types of focus mismatch. Thus, it is not necessary for a multi-view video coded view to utilize all the parametric filters. The following provide some examples of when all of the parametric filters need not be used:
- the composition of the mismatch in the scene can still be different.
- one scene may have objects mostly requiring enhancement where only a few of the objects need lowpass filtering, while in another scene a larger portion may need to be smoothed (lowpass filtering).
- the ordering of the parametric filters should be altered accordingly. Putting the most dominant mismatch filter first in the list can further reduce the signaling bitrate.
- the characteristics of the imaging system i.e., the optical transfer functions
- the depth information is also available (for example, in free viewpoint television systems), combined with the optical transfer functions, we will be able to know what kinds of mismatch will be present in the images.
- the depth information also tells us the depth composition of the scene and consequently tells us what type of mismatch is more dominant.
- the ordering of the selected filters can be determined accordingly.
- the estimated disparity can then be used to trace back the actual depth of different objects in the scene. That is to say, given the multi-view camera parameters, we can perform disparity estimation (via search, for example) to identify possible types of focus mismatch. For a given view, after such estimation, we select from the pre-defined filter set the parametric filters which cover the types of mismatch we are considering. The disparity estimation also indicates the depth composition of the scene and consequently what type of mismatch is more dominant. The ordering of the selected filters can be determined accordingly.
- Focus mismatch estimation embodiment III: When the multi-view camera parameters are not fully available, we can use the procedure similar to adaptive reference filtering to estimate the mismatch in the images. For a given view to be encoded, frames are partitioned into different depth- levels based on classification of the disparity vectors. Then, two-dimensional (2D) minimum mean-squared error filters are estimated for all the blocks in each depth level. These estimated filters indicate what the types of focus mismatch are in the current view as compared to the reference view(s). We can then compare the estimated filters with the parametric filters in the pre-defined set. Filters in the predefined set which have the highest correlations to the estimated filters are selected to serve as additional coding options for the current view, in order to generate new reference frames. The ordering of the selected parametric filters can also be decided based on the classification and correlation analysis.
- the disparity estimation (embodiments Il and III) and filter estimation (embodiment III) can be performed in some early frames of a given view and, then the selected parametric filters are applied to the remaining frames.
- the estimations can be carried out at certain particular timestamps (for example, every 10th frame) and the chosen parametric filters are applied to frames between the mismatch- estimation timestamps.
- Parametric filters can be constructed to compensate for motion blur. Since motion can occur in different directions in the video content, the filters can be parameterized with directionality as illustrated below:
- the encoder can select which of them to use as well as the filter ordering by performing estimations of the motion blur mismatches.
- estimations of the motion blur mismatches may be performed by motion estimation to identify the possible motion blur directions.
- the filter set is pre-defined by exploiting the characteristics of the mismatch, the coefficients only have to be transmitted at most once for the entire sequence. Furthermore, if the pre-defined filter set is built as a dictionary which is common to both the encoder and decoder, we do not have to transmit the filter coefficients.
- the order in which they are transmitted can be considered as their default order and in the case when they are a fixed set, the default order can be established for them.
- the filters are applied in their default order. If there are N filters in each list but only M ⁇ N reference pictures in the reference list, then the first M filters are applied.
- the flag is transmitted as a high level syntax and may be present in, for example, but not limited to, the PPS, SPS, NAL unit header and/or slice header.
- This flag is shown in TABLE 6, which shows slice header syntax.
- the ordering of the references corresponds to filtered by f3, unfiltered, and filtered by f1
- f ⁇ lter_flag 0 indicates that is was encoded without applying parametric filtering
- an exemplary method for video encoding using a default parametric filter set is indicated generally by the reference numeral 300.
- the method 300 includes a start block 305 that passes control to a function block 310.
- the function block 310 generates filtered references using a pre-defined parametric filter set (with a default order), and passes control to a function block 315.
- the function block 315 encodes the current frame with unfiltered and filtered references, and passes control to an end block 399.
- FIG. 4 an exemplary method for video encoding using adaptive selection for a parametric filter set is indicated generally by the reference numeral 400.
- the method 400 includes a start block 405 that passes control to a function block 410.
- the function block 410 performs mismatch estimation, which may be based on cameras parameters, and/or camera settings, and/or depth/disparity/motion information, and/or filter estimation (e.g., the latter including performing a comparison with the pre-defined set), to determine, for example, the value of filter_flag (0 or 1 ), the number of filters, and the order of the filters, and passes control to a decision block 415.
- the decision block 415 determines whether or not filter_flag is equal to one. If so, then control is passed to a function block 420. Otherwise, control is passed to a function block 430.
- the function block 420 generates filtered references (with the determined order and number (as per function block 410), and passes control to a function block 425.
- the function block 425 encodes the current frame with unfiltered and filtered references, and passes control to an end block 499.
- the function block 430 encodes the current frame, and passes control to the end block 499.
- an exemplary method for video decoding using a default parametric filter set is indicated generally by the reference numeral 500.
- the method 500 includes a start block 505 that passes control to a function block 510.
- the function block 510 generates filtered references based on predefined filter coefficients, and passes control to a function block 515.
- the function block 515 decodes the current frame with the unfiltered and filtered references, and passes control to an end block 599.
- FIG. 6 an exemplary method for video decoding using adaptive selection for a parametric filter set is indicated generally by the reference numeral 600.
- the method 600 includes a start block 605 that passes control to a decision block 615.
- the decision block 615 determines whether or not filter_flag is equal to 1. If so, then control is passed to a function block 620. Otherwise, control is passed to a function block 635.
- the function block 620 decodes the number of filters that are selected from the pre-defined filter sets, and also decodes the order of the filters, and passes control to a function block 625.
- the function block 625 generates filtered references based on the decoded number of filters and the order of the filters, and passes control to a function block 630.
- the function block 630 decodes the current frame with the filtered and unfiltered references, and passes control to an end block 699.
- the function block 635 decodes the current frame, and passes control to the end block 699.
- one advantage/feature is an apparatus having an encoder for encoding a picture.
- the encoder performs parametric filtering of at least one reference picture to respectively obtain at least one filtered reference picture, and predictively codes the picture using the at least one filtered reference picture.
- Another advantage/feature is the apparatus having the encoder as described above, wherein at least one set of parametric filters used for the parametric filtering is pre-defined before encoding the picture or a video sequence to which the picture belongs.
- Yet another advantage/feature is the apparatus having the encoder where at least one set of parametric filters is pre-defined as described above, wherein the at least one pre-defined set of parametric filters is estimated based on an expected mismatch between the picture and the at least one reference picture.
- Still another advantage/feature is the apparatus having the encoder wherein the at least one pre-defined set of parametric filters is estimated as described above, herein coefficients of the at least one pre-defined set of parametric filters are calculated based on estimating a group of filters which compensate for the expected mismatch and classifying at least a portion of the estimated group of filters.
- another advantage/feature is the apparatus having the encoder where at least one set of parametric filters is pre-defined as described above, wherein the encoder selects at least one set of parametric filters from the at least one pre-defined filter set to use for the parametric filtering, and further selects an ordering of the filters in the at least one selected set.
- another advantage/feature is the apparatus having the encoder that selects at least one set of parametric filters and further selects an ordering of the filters as described above, wherein at least one of the at least one selected set of parametric filters and the ordering of the filters in the at least one selected set is selected based on an estimation of a mismatch between the picture and the at least one reference picture
- another advantage/feature is the apparatus having the encoder that selects based on an estimation of a mismatch as described above, wherein the picture corresponds to at least one view of at least two views of multi-view video content, and the estimation is based on at least one of camera parameters and camera settings for at least two cameras used to capture the multi-view video content
- another advantage/feature is the apparatus having the encoder that selects based on an estimation of a mismatch as described above, wherein the estimation is based on at least one of motion information, depth information, a motion search, and a disparity search
- Another advantage/feature is the apparatus having the encoder that selects based on an estimation of a mismatch as described above, wherein the estimation is performed by partitioning the picture into regions with different types of mismatch
- another advantage/feature is the apparatus having the encoder that selects at least one set of parametric filters and further selects an ordering of the filters as described above, wherein the at least one filtered reference picture obtained by performing the parametric filtering is associated with a different reference index for encoding selection and signaling
- another advantage/feature is the apparatus having the encoder that selects at least one set of parametric filters and further selects an ordering of the filters as described above, wherein the encoder signals the at least one selected set of parametric filters and a reference picture ordering in a resultant bitstream to allow a reconstruction of the picture by a corresponding decoder
- the teachings of the present principles are implemented as a combination of hardware and software.
- the software may be implemented as an application program tangibly embodied on a program storage unit.
- the application program may be uploaded to, and executed by, a machine comprising any suitable architecture.
- the machine is implemented on a computer platform having hardware such as one or more central processing units (“CPU"), a random access memory (“RAM”), and input/output ("I/O") interfaces.
- CPU central processing units
- RAM random access memory
- I/O input/output
- the computer platform may also include an operating system and microinstruction code.
- the various processes and functions described herein may be either part of the microinstruction code or part of the application program, or any combination thereof, which may be executed by a CPU.
- various other peripheral units may be connected to the computer platform such as an additional data storage unit and a printing unit.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US1952508P | 2008-01-07 | 2008-01-07 | |
PCT/US2009/000027 WO2009088976A1 (en) | 2008-01-07 | 2009-01-05 | Methods and apparatus for video encoding and decoding using parametric filtering |
Publications (1)
Publication Number | Publication Date |
---|---|
EP2241111A1 true EP2241111A1 (en) | 2010-10-20 |
Family
ID=40602563
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP09700816A Ceased EP2241111A1 (en) | 2008-01-07 | 2009-01-05 | Methods and apparatus for video encoding and decoding using parametric filtering |
Country Status (7)
Country | Link |
---|---|
US (1) | US8625672B2 (en) |
EP (1) | EP2241111A1 (en) |
JP (2) | JP5400062B2 (en) |
KR (1) | KR101682516B1 (en) |
CN (1) | CN101965732B (en) |
BR (1) | BRPI0907242A2 (en) |
WO (1) | WO2009088976A1 (en) |
Families Citing this family (31)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8750632B2 (en) * | 2008-12-26 | 2014-06-10 | JVC Kenwood Corporation | Apparatus and method for encoding images from multiple viewpoints and associated depth information |
US10178406B2 (en) | 2009-11-06 | 2019-01-08 | Qualcomm Incorporated | Control of video encoding based on one or more video capture parameters |
US8837576B2 (en) | 2009-11-06 | 2014-09-16 | Qualcomm Incorporated | Camera parameter-assisted video encoding |
KR101484280B1 (en) | 2009-12-08 | 2015-01-20 | 삼성전자주식회사 | Method and apparatus for video encoding by motion prediction using arbitrary partition, and method and apparatus for video decoding by motion compensation using arbitrary partition |
CN102726046B (en) | 2010-01-27 | 2015-09-09 | 杜比实验室特许公司 | For the method and system of the reference process in image and coding and decoding video |
EP2375747B1 (en) * | 2010-04-12 | 2019-03-13 | Sun Patent Trust | Filter positioning and selection |
JP5805991B2 (en) * | 2010-05-07 | 2015-11-10 | トムソン ライセンシングThomson Licensing | Method for encoding picture sequence, corresponding reconstruction method, and stream of encoded data representing the sequence |
US9247265B2 (en) | 2010-09-01 | 2016-01-26 | Qualcomm Incorporated | Multi-input adaptive filter based on combination of sum-modified Laplacian filter indexing and quadtree partitioning |
US9819966B2 (en) * | 2010-09-01 | 2017-11-14 | Qualcomm Incorporated | Filter description signaling for multi-filter adaptive filtering |
CN103155559B (en) * | 2010-10-12 | 2016-01-06 | 杜比实验室特许公司 | For the stratum conjunctum optimization of frame compatible video transmission |
US8767835B2 (en) * | 2010-12-28 | 2014-07-01 | Mitsubishi Electric Research Laboratories, Inc. | Method for coding videos using dictionaries |
US8849053B2 (en) | 2011-01-14 | 2014-09-30 | Sony Corporation | Parametric loop filter |
US8917950B2 (en) | 2011-01-18 | 2014-12-23 | Sony Corporation | Simplifying parametric loop filters |
JPWO2012131895A1 (en) * | 2011-03-29 | 2014-07-24 | 株式会社東芝 | Image coding apparatus, method and program, image decoding apparatus, method and program |
US8934026B2 (en) | 2011-05-12 | 2015-01-13 | Cisco Technology, Inc. | System and method for video coding in a dynamic environment |
US8818125B2 (en) | 2011-10-07 | 2014-08-26 | Texas Instruments Incorporated | Scene adaptive filter design for improved stereo matching |
US8682087B2 (en) * | 2011-12-19 | 2014-03-25 | Cisco Technology, Inc. | System and method for depth-guided image filtering in a video conference environment |
JP6394966B2 (en) | 2012-01-20 | 2018-09-26 | サン パテント トラスト | Encoding method, decoding method, encoding device, and decoding device using temporal motion vector prediction |
EP2811743B1 (en) | 2012-02-03 | 2021-03-03 | Sun Patent Trust | Image encoding method, image decoding method, image encoding device, image decoding device, and image encoding/decoding device |
CA2866121C (en) | 2012-03-06 | 2018-04-24 | Panasonic Intellectual Property Corporation Of America | Moving picture coding method, moving picture decoding method, moving picture coding apparatus, moving picture decoding apparatus, and moving picture coding and decoding apparatus |
CN104335587B (en) * | 2012-07-04 | 2018-06-19 | 英特尔公司 | Filter parameter between the view of 3 D video decoding is reused |
CN103533374B (en) * | 2012-07-06 | 2018-02-16 | 乐金电子(中国)研究开发中心有限公司 | A kind of Video coding, the method and device of decoding |
GB2504069B (en) * | 2012-07-12 | 2015-09-16 | Canon Kk | Method and device for predicting an image portion for encoding or decoding of an image |
US9235875B2 (en) * | 2012-11-01 | 2016-01-12 | Google Inc. | Image enhancement using learned non-photorealistic effects |
US9681154B2 (en) | 2012-12-06 | 2017-06-13 | Patent Capital Group | System and method for depth-guided filtering in a video conference environment |
WO2016006894A1 (en) * | 2014-07-06 | 2016-01-14 | 엘지전자 주식회사 | Method for processing video signal, and apparatus therefor |
US20180048890A1 (en) * | 2015-03-02 | 2018-02-15 | Lg Electronics Inc. | Method and device for encoding and decoding video signal by using improved prediction filter |
US10979704B2 (en) * | 2015-05-04 | 2021-04-13 | Advanced Micro Devices, Inc. | Methods and apparatus for optical blur modeling for improved video encoding |
AU2015395514B2 (en) | 2015-05-21 | 2019-10-10 | Huawei Technologies Co., Ltd. | Apparatus and method for video motion compensation |
EP3552390A1 (en) * | 2016-12-23 | 2019-10-16 | Huawei Technologies Co., Ltd. | Mixed domain collaborative in-loop filter for lossy video coding |
JP7132749B2 (en) * | 2018-05-28 | 2022-09-07 | 日本放送協会 | Video encoding device and program |
Family Cites Families (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3360191B2 (en) * | 1994-02-28 | 2002-12-24 | 富士通株式会社 | Image encoding apparatus and image encoding method |
US6735253B1 (en) * | 1997-05-16 | 2004-05-11 | The Trustees Of Columbia University In The City Of New York | Methods and architecture for indexing and editing compressed video over the world wide web |
US6285711B1 (en) * | 1998-05-20 | 2001-09-04 | Sharp Laboratories Of America, Inc. | Block matching-based method for estimating motion fields and global affine motion parameters in digital video sequences |
US6748020B1 (en) * | 2000-10-25 | 2004-06-08 | General Instrument Corporation | Transcoder-multiplexer (transmux) software architecture |
US7499922B1 (en) * | 2001-04-26 | 2009-03-03 | Dakota Software Corp. | Information retrieval system and method |
US7227896B2 (en) * | 2001-10-04 | 2007-06-05 | Sharp Laboratories Of America, Inc. | Method and apparatus for global motion estimation |
BR0303901A (en) * | 2002-07-11 | 2004-08-10 | Matsushita Electric Ind Co Ltd | Filtering power determination method, moving image encoding method and moving image decoding method |
US20130107938A9 (en) | 2003-05-28 | 2013-05-02 | Chad Fogg | Method And Apparatus For Scalable Video Decoder Using An Enhancement Stream |
US8094711B2 (en) | 2003-09-17 | 2012-01-10 | Thomson Licensing | Adaptive reference picture generation |
EP1578137A2 (en) * | 2004-03-17 | 2005-09-21 | Matsushita Electric Industrial Co., Ltd. | Moving picture coding apparatus with multistep interpolation process |
US7671894B2 (en) | 2004-12-17 | 2010-03-02 | Mitsubishi Electric Research Laboratories, Inc. | Method and system for processing multiview videos for view synthesis using skip and direct modes |
JP2007166381A (en) | 2005-12-15 | 2007-06-28 | Univ Of Tokyo | Compression coding method and decoding method of multi-viewpoint image |
CN101371585B (en) * | 2006-01-09 | 2014-07-30 | 汤姆森特许公司 | Method and apparatus for providing reduced resolution update mode for multi-view video coding |
EP1841230A1 (en) | 2006-03-27 | 2007-10-03 | Matsushita Electric Industrial Co., Ltd. | Adaptive wiener filter for video coding |
JP5535625B2 (en) * | 2006-07-18 | 2014-07-02 | トムソン ライセンシング | Method and apparatus for adaptive reference filtering |
BRPI0715507A2 (en) * | 2006-08-02 | 2013-06-18 | Thomson Licensinc S A | Methods and apparatus for adaptive geometric partitioning for video decoding |
US8270472B2 (en) * | 2007-11-09 | 2012-09-18 | Thomson Licensing | Methods and apparatus for adaptive reference filtering (ARF) of bi-predictive pictures in multi-view coded video |
-
2009
- 2009-01-05 WO PCT/US2009/000027 patent/WO2009088976A1/en active Application Filing
- 2009-01-05 US US12/735,331 patent/US8625672B2/en not_active Expired - Fee Related
- 2009-01-05 KR KR1020107014959A patent/KR101682516B1/en active IP Right Grant
- 2009-01-05 BR BRPI0907242-0A patent/BRPI0907242A2/en active Search and Examination
- 2009-01-05 EP EP09700816A patent/EP2241111A1/en not_active Ceased
- 2009-01-05 CN CN200980108035.4A patent/CN101965732B/en not_active Expired - Fee Related
- 2009-01-05 JP JP2010541557A patent/JP5400062B2/en not_active Expired - Fee Related
-
2013
- 2013-04-26 JP JP2013094133A patent/JP5624171B2/en not_active Expired - Fee Related
Non-Patent Citations (4)
Title |
---|
BUDAGAVI M: "Video Compression using Blur Compensation", IMAGE PROCESSING, 2005. ICIP 2005. IEEE INTERNATIONAL CONFERENCE ON, IEEE, PISCATAWAY, NJ, USA,IEEE, vol. 2, 11 September 2005 (2005-09-11), pages 882 - 885, XP010851195, ISBN: 978-0-7803-9134-5, DOI: 10.1109/ICIP.2005.1530197 * |
LAI THOMSON CORPORATE RESEARCH (USA) AND UNIV OF SOUTHERN CALIFORNIA (USA) P ET AL: "Adaptive filtering for cross-view prediction in multi-view video coding", VISUAL COMMUNICATIONS AND IMAGE PROCESSING; 30-1-2007 - 1-2-2007; SAN JOSE,, 30 January 2007 (2007-01-30), XP030081115 * |
POLIN LAI ET AL: "Adaptive filtering for video coding with focus change", INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNALPROCESSING, IEEE, 15 April 2007 (2007-04-15), pages I, XP002454496 * |
See also references of WO2009088976A1 * |
Also Published As
Publication number | Publication date |
---|---|
US8625672B2 (en) | 2014-01-07 |
KR101682516B1 (en) | 2016-12-05 |
CN101965732B (en) | 2015-03-25 |
JP5400062B2 (en) | 2014-01-29 |
JP5624171B2 (en) | 2014-11-12 |
WO2009088976A1 (en) | 2009-07-16 |
US20100278267A1 (en) | 2010-11-04 |
KR20100103556A (en) | 2010-09-27 |
CN101965732A (en) | 2011-02-02 |
JP2011509053A (en) | 2011-03-17 |
BRPI0907242A2 (en) | 2015-07-14 |
JP2013150360A (en) | 2013-08-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8625672B2 (en) | Methods and apparatus for video encoding and decoding using parametric filtering | |
US8270472B2 (en) | Methods and apparatus for adaptive reference filtering (ARF) of bi-predictive pictures in multi-view coded video | |
EP2548372B1 (en) | Methods and apparatus for implicit adaptive motion vector predictor selection for video encoding and decoding | |
US9253504B2 (en) | Methods and apparatus for adaptive reference filtering | |
US9241160B2 (en) | Reference processing using advanced motion models for video coding | |
KR101834688B1 (en) | Method for low-latency illumination compensation process and depth lookup table based coding | |
US10375411B2 (en) | Predictor for depth map intra coding | |
EP3633996A1 (en) | Methods and apparatus for adaptive coding of motion information | |
WO2009005658A2 (en) | Single loop decoding of multi-vieuw coded video | |
WO2011146451A1 (en) | Methods and apparatus for adaptive motion vector candidate ordering for video encoding and decoding | |
WO2007081713A1 (en) | Method and apparatus for providing reduced resolution update mode for multi-view video coding | |
EP2177036A2 (en) | Methods and apparatus for motion skip mode in multi-view coded video using regional disparity vectors | |
CN116800961A (en) | Apparatus for encoding and decoding video signal and apparatus for transmitting data of image | |
EP2526697A1 (en) | Methods and apparatus for reduced complexity template matching prediction for video encoding and decoding | |
WO2009151563A1 (en) | Methods and apparatus for locally adaptive filtering for motion compensation interpolation and reference picture filtering | |
EP3059969A1 (en) | Method and apparatus for coding/decoding video comprising multi-view |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20100705 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK TR |
|
AX | Request for extension of the european patent |
Extension state: AL BA RS |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: YIN, PENG Inventor name: PANDIT, PURVIN, BIBHAS Inventor name: LAI, POLIN |
|
DAX | Request for extension of the european patent (deleted) | ||
17Q | First examination report despatched |
Effective date: 20141111 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R003 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN REFUSED |
|
18R | Application refused |
Effective date: 20160525 |