WO2008065089A2 - Video pre-processing device and method, motion estimation device and method - Google Patents

Video pre-processing device and method, motion estimation device and method Download PDF

Info

Publication number
WO2008065089A2
WO2008065089A2 PCT/EP2007/062834 EP2007062834W WO2008065089A2 WO 2008065089 A2 WO2008065089 A2 WO 2008065089A2 EP 2007062834 W EP2007062834 W EP 2007062834W WO 2008065089 A2 WO2008065089 A2 WO 2008065089A2
Authority
WO
WIPO (PCT)
Prior art keywords
component
video
colour
colour component
pixel
Prior art date
Application number
PCT/EP2007/062834
Other languages
French (fr)
Other versions
WO2008065089A3 (en
Inventor
Jonathan Kervec
Michael Bardy
Didier Doyen
Original Assignee
Thomson Licensing
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Thomson Licensing filed Critical Thomson Licensing
Priority to EP07847362A priority Critical patent/EP2087736A2/en
Priority to KR1020097010723A priority patent/KR101377780B1/en
Publication of WO2008065089A2 publication Critical patent/WO2008065089A2/en
Publication of WO2008065089A3 publication Critical patent/WO2008065089A3/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G06T5/94—Dynamic range modification of images or parts thereof based on local image properties, e.g. for local contrast enhancement
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/77—Circuits for processing the brightness signal and the chrominance signal relative to each other, e.g. adjusting the phase of the brightness signal relative to the colour signal, correcting differential gain or differential phase
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image

Definitions

  • the invention relates to a video pre-processing device and method.
  • the pixel output from the module 4 has a value Iv' in the range between l m and I M such that:

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Processing Of Color Television Signals (AREA)
  • Color Television Systems (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to a method and to a device for the video pre- processing of at least one of the colour components (Ir, Iv, Ib) of the video. The device comprises, for each pixel, - means (2, 5, 8) for the low-pass filtering of the said colour component (Ir, Iv, Ib), - means (3, 6, 9) for the over-accentuation of the contours by applying a mask onto the said colour component (Ir, Iv, Ib).

Description

VIDEO PRE-PROCESSING DEVICE AND METHOD, MOTION ESTIMATION
DEVICE AND METHOD.
The invention relates to a video pre-processing device and method.
The invention may, in particular, be applied to video pre-processing prior to motion estimation and, more particularly, to recursive-pixel-based motion estimation.
Motion estimation by the recursive-pixel method is based on the DFD
(Displaced Frame Difference) criterion coming from the equation for the optical flux. This processing is carried out on luminance images that are low- pass filtered then decimated in twos horizontally and vertically. Subsequently, with each remaining pixel is associated a motion vector; the latter is firstly selected taking into account a spatio-temporal neighbourhood by means of the DFD (concept of prediction), then corrected in order to minimize the DFD in the direction of the spatial gradient of the image (using the optical flux equation).
The invention provides a method of image processing that advantageously allows the contours of an image to be accentuated. Such an accentuation can be useful in order to improve the motion estimation of a video sequence without affecting the core of the estimator. Indeed, the motion estimation is more effective in images whose contours are accentuated because a motion estimator needs the contours of an image to lock onto and to define the various movements. The invention is not limited to motion estimation but may be used for other image processing applications for which such an accentuation of the contours is advantageous.
For this purpose, the invention provides a device for video preprocessing of at least one of the colour components of the video comprising, for each pixel,
- means for the low-pass filtering of the said colour component, - means for over-accentuating the contours by applying a mask onto the said colour component.
According to a preferred embodiment, the device comprises linear rescaling means carrying out, upstream of the low-pass filtering means, a linear rescaling in such a manner as to spread the spectrum of at least one colour component between a minimum value and a predetermined maximum value.
Advantageously, the pre-processing is carried out by choosing from amongst
- preferably the red component of the video,
- the blue component of the video,
- the green component of the video, and any given combination of these components, preferably a combination of the red component and of the blue component.
The invention also relates to a motion estimation device for a video signal comprising, for each pixel : - means for linear rescaling in such a manner as to spread the spectrum of at least one colour component between a minimum value and a predetermined maximum value,
- means for the low-pass filtering of the said colour component,
- means for over-accentuating the contours by applying a mask onto the said colour component,
- means for calculating a luminance value by weighting each colour component, at least one being taken at the output of the over-accentuation means.
The invention also relates to a method for the video pre-processing of at least one of the colour components of the video comprising for each pixel,
- a step for the low-pass filtering of the said colour component,
- a step for the over-accentuation of the contours by applying a mask onto the said colour component. The invention will be better understood and illustrated by means of advantageous, non-limiting, exemplary embodiments and their implementation, with reference to Figure 1 showing a preferred embodiment of the invention.
The various embodiments of the invention may be produced by software, hardware or by combination of these various techniques. For example, in certain cases, the invention can be supplied as a computer program product, or a software application, which can comprise a device or medium readable by a computer having instructions stored for executing the method according to the invention. In other embodiments, the invention may be effected by hardware means containing logic wired so as to carry out the various means of the invention or else by a combination of software and hardware means.
The video image at the input of the device is decomposed for each pixel according to its red, green and blue colour components. Each pixel is therefore represented by its three components.
Each component, red, green, blue, is respectively filtered by a module 1 , 4, 7.
The linear rescaling modules 1 , 4 and 7 are optional for the implementation of the invention. However, they do considerably improve the results obtained.
The purpose of the modules 1 , 4 and 7 is to spread the grey levels so as to distribute the values of each component over a wider interval [L, IM], where lm and IM represent the minimum and maximum values permitted at the output, hence between 0 and 255 for pixels coded over 8 bits. Thus, for a pixel of value Ir at the input for a red component, the pixel output from the module 1 has a value Ir' in the range between lm and IM such that: , b]
Figure imgf000005_0001
It is considered that the minimum lm and maximum IM output values are respectively 0 and 255, the values of the colour components being coded over 8 bits.
The values 20 and 235 are respectively chosen for a and b according to the preferred embodiment.
According to another embodiment, the values 10 and 245 are respectively taken for a and b.
Signals whose values extend over a wider range than at the input are therefore obtained at the output.
For a pixel of value Iv at the input for a green component, the pixel output from the module 4 has a value Iv' in the range between lm and IM such that:
/v'= {7^ 7" * (/v -α) + /m if /ve M [ b - α
IV= In if Iv < α
Figure imgf000005_0002
For a pixel of value Ib at the input for a blue component, the pixel output from the module 4 has a value Ib' in the range between lm and IM such that:
Figure imgf000005_0003
Ib'= I if Ib < α Ib'= I M if Ib > b
In other embodiments of the invention, the b, a, lm and IM can be different for the different color components. The modules 1 , 4 and 7 are respectively connected to modules 2, 5 and 8. These modules 2, 5 and 8 carry out a low-pass filtering on the signals Ir', Ib' and Iv' received from the modules 1 , 4 and 7.
The low-pass filtering allows the image to be smoothed after its linear rescaling but also the noise to be reduced. The following low-pass filter is used:
1 1 1
H= - * 1 1 1 " i l l
For a pixel P with coordinates (i, j) in the image, at the output of the module 2, the following red colour component value is obtained:
Ir"(p,j) = \∑ ∑ mJr' (P1+nj+J y -l -l In other embodiments, other values may be used.
For a pixel P with coordinates (i, j) in the image, at the output of the module 5, the following green colour component value is obtained:
Figure imgf000006_0001
For a pixel P with coordinates (i, j) in the image, at the output of the module 8, the following blue colour component value is obtained:
Ib"(PJ = ^Σ Σ m Jb' (P1+11 J+m) y -l -l
Once each component Ir', Ib' and Iv' has been respectively filtered by the modules 2, 5 and 8, the modules 3, 6 and 9 apply a filter to the signals Ir", Iv" and Ib" received from the modules 2, 5 and 8, respectively.
The following filter is used: -1 -1 -1
H= -1 9 -1
-1 -1 -1
At the output of the module 3, the following red colour component value is obtained:
V(P1J = ∑m,Λ+nj+mIr"(P1+nj+J with ∑mnai+nj+m =1
-1
With
Figure imgf000007_0001
α,-i,J+i=-1
Figure imgf000007_0002
αl+i,j-i=-1
0Cι+i, j+i=-1
At the output of the module 6, the following green colour component value is obtained:
Iv'" (PJ = ∑m^1+nj+Jv"(P1+nj+m) with ∑mnaι+nj+m =1
-1
The same values being used for the α components of the red component.
At the output of the module 9, the following blue colour component value is obtained:
Ib'"(PJ = ∑m,na1+nj+mIb"(P1+nj+m) with ∑mnaι+nj+m =1
-1 The same values being used for the α components of the red component.
If this video pre-processing device or method is designed to facilitate motion estimation, the signals Ir'", Iv'" and Ib'" obtained are combined for each pixel so as to obtain a luminance value which will subsequently be used, for example, by a motion estimator of the recursive pixel type.
The luminance Y is calculated by a module 10 according to the following formula:
Y=0.299*(lr'") + 0.587*(lv'") + 0.114*(lb"')
In other embodiments, as indicated by Figure 1 , the device is not applicable to the three colour components. More generally, the luminance is thus obtained according to the following equation:
Y=0.299*(lr or Ir'") + 0.587*(lv or Iv'") + 0.114*(lb or Ib'")
It is effectively possible for only one of the three components to be filtered by the three successive modules, as previously described, and for the other two not to be filtered. It is also possible for only two of the components to be filtered. According to the best embodiments, the best results are obtained for a filtering of the blue component only, of the red component only or of both the red and blue components. This is explained by the fact that the red and blue components are of less importance in the composition of the luminance. Thus, by applying the device to these two colours, this is sufficient to accentuate the contours by accentuating the differences in grey level.
For reasons of clarity in Figure 1 , each module is replicated for each component. It is clear that, for those skilled in the art, the processing of the various components can be performed by the same module in a sequential fashion. It goes without saying that the invention is not limited to the exemplary embodiments described hereinabove. In particular, those skilled in the art may bring any variant into the embodiments presented and combine them so as to benefit from their various advantages.
The invention may notably be applied to applications relating to video coding other than motion estimation.

Claims

Claims
1. Device for video pre-processing of at least one of the colour components (I1-, lv, Ib) of the video, characterized in that it comprises, for each pixel,
- means (2, 5, 8) for the low-pass filtering of the said colour component (I1-,
- means (3, 6, 9) for over-accentuating the contours by applying a mask onto the said colour component (I1-, lv, Ib)-
2. Device according to Claim 1 , characterized in that it comprises linear rescaling means (1 , 4, 7) carrying out, upstream of the low-pass filtering means (2, 5, 8), a linear rescaling in such a manner as to spread the spectrum of at least one colour component (I1-, lv, Ib) between a minimum value (L) and a predetermined maximum value (IM).
3. Device according to either of Claims 1 and 2, characterized in that the pre-processing is carried out by choosing from amongst
- preferably the red component (I1-) of the video,
- the blue component (lb) of the video,
- the green component (lv) of the video, and any given combination of these colour components (I1-, lv, Ib), preferably a combination of the red component (I1-) and of the blue component (lb).
4. Motion estimation device for a video signal, characterized in that it comprises for each pixel: - means (2, 5, 8) for the low-pass filtering of at least one colour component (I1-, lv, Ib),
- means (3, 6, 9) for over-accentuating the contours by applying a mask onto the said colour component (I1-, lv, Ib), - means (10) for calculating a luminance value (Y) by weighting each colour component (I1-, lv, Ib), one at least being taken at the output of the over-accentuation means (3, 6, 9).
5. Method for the video pre-processing of at least one of the colour components (I1-, lv, Ib) of the video comprising for each pixel,
- A step for the low-pass filtering of the said colour component,
- A step for the over-accentuation of the contours by applying a mask onto the said colour component.
6. Motion estimation method for a video signal, characterized in that it comprises, for each pixel:
- A step for linear rescaling in such a manner as to spread the spectrum of at least one colour component (I1-, lv, Ib) between a minimum value (L) and a predetermined maximum value (IM),
- A step for the low-pass filtering of the said colour component (I1-, lv, Ib),
- A step for the over-accentuation of the contours by applying a mask onto the said colour component (I1-, lv, Ib),
- A step for calculating a luminance value (Y) by weighting each colour component (I1-, lv, Ib), at least one being taken at the output of the over-accentuation step.
7. Computer program product comprising program code instructions for the execution of the steps of the method according to Claim 6, when the said program is run on a computer.
PCT/EP2007/062834 2006-11-27 2007-11-26 Video pre-processing device and method, motion estimation device and method WO2008065089A2 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP07847362A EP2087736A2 (en) 2006-11-27 2007-11-26 Video pre-processing device and method, motion estimation device and method
KR1020097010723A KR101377780B1 (en) 2006-11-27 2007-11-26 Video pre-processing device and method, motion estimation device and method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
FR06/55128 2006-11-27
FR0655128 2006-11-27

Publications (2)

Publication Number Publication Date
WO2008065089A2 true WO2008065089A2 (en) 2008-06-05
WO2008065089A3 WO2008065089A3 (en) 2008-09-04

Family

ID=38222397

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2007/062834 WO2008065089A2 (en) 2006-11-27 2007-11-26 Video pre-processing device and method, motion estimation device and method

Country Status (3)

Country Link
EP (1) EP2087736A2 (en)
KR (1) KR101377780B1 (en)
WO (1) WO2008065089A2 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5933540A (en) * 1995-05-11 1999-08-03 General Electric Company Filter system and method for efficiently suppressing noise and improving edge definition in a digitized image
US20040032906A1 (en) * 2002-08-19 2004-02-19 Lillig Thomas M. Foreground segmentation for digital video
US20050001913A1 (en) * 2003-07-01 2005-01-06 Nikon Corporation Signal processing apparatus, signal processing program and electirc camera
EP1594087A2 (en) * 2004-05-04 2005-11-09 Samsung Electronics Co., Ltd. Apparatus and method for filtering digital image signal
WO2005124664A2 (en) * 2004-06-14 2005-12-29 Precoad, Inc. Image clean-up and pre-coding

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1522963A1 (en) * 2003-10-07 2005-04-13 Deutsche Thomson-Brandt Gmbh Method for processing video pictures for false contours and dithering noise compensation

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5933540A (en) * 1995-05-11 1999-08-03 General Electric Company Filter system and method for efficiently suppressing noise and improving edge definition in a digitized image
US20040032906A1 (en) * 2002-08-19 2004-02-19 Lillig Thomas M. Foreground segmentation for digital video
US20050001913A1 (en) * 2003-07-01 2005-01-06 Nikon Corporation Signal processing apparatus, signal processing program and electirc camera
EP1594087A2 (en) * 2004-05-04 2005-11-09 Samsung Electronics Co., Ltd. Apparatus and method for filtering digital image signal
WO2005124664A2 (en) * 2004-06-14 2005-12-29 Precoad, Inc. Image clean-up and pre-coding

Also Published As

Publication number Publication date
EP2087736A2 (en) 2009-08-12
WO2008065089A3 (en) 2008-09-04
KR20090086560A (en) 2009-08-13
KR101377780B1 (en) 2014-03-26

Similar Documents

Publication Publication Date Title
TWI407800B (en) Improved processing of mosaic images
RU2413384C2 (en) Device of image processing and method of image processing
CN110378861B (en) Image fusion method and device
WO2015198954A1 (en) Method and decoder for predicting and filtering color components in pictures
CN109118437B (en) Method and storage medium capable of processing muddy water image in real time
CN107767356B (en) Image processing method and device
US7474785B2 (en) Video auto enhancing algorithm
US11677938B2 (en) Method to reduce contour artifacts on recursive temporal filters
CN110782414A (en) Dark light image denoising method based on dense connection convolution
US20080219582A1 (en) Apparatus for Filtering an Image Obtained by Block Based Image Decompression
US8385671B1 (en) Digital camera and method
WO2006020532A2 (en) Fast area-selected filtering for pixel-noise and analog artifacts reduction
CN107358578B (en) Yin-yang face treatment method and device
JPH02272894A (en) Color image sigal encoding
CN111815659A (en) Image processing method, image processing device, electronic equipment and computer readable storage medium
CN101778297B (en) Interference elimination method of image sequence
EP2087736A2 (en) Video pre-processing device and method, motion estimation device and method
WO2006003102A1 (en) Pre-processing device and method before encoding of a video image sequence
KR101180409B1 (en) Low illumination intensity image enhancement method and apparatus using histogram normalization and gamma correction
US20100002773A1 (en) Motion-Compensated Temporal Recursive Filter
US10264212B1 (en) Low-complexity deinterlacing with motion detection and overlay compensation
CN110572652A (en) Static image processing method and device
CN115953331B (en) Image edge enhancement method and system
Vinayagamoorthy et al. A multichannel filter for TV signal processing
EP2321958A1 (en) Image sharpening

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 07847362

Country of ref document: EP

Kind code of ref document: A2

WWE Wipo information: entry into national phase

Ref document number: 2007847362

Country of ref document: EP

Ref document number: 1020097010723

Country of ref document: KR

NENP Non-entry into the national phase

Ref country code: DE