EP2633520A1 - Parametric encoder for encoding a multi-channel audio signal - Google Patents

Parametric encoder for encoding a multi-channel audio signal

Info

Publication number
EP2633520A1
EP2633520A1 EP10859153.8A EP10859153A EP2633520A1 EP 2633520 A1 EP2633520 A1 EP 2633520A1 EP 10859153 A EP10859153 A EP 10859153A EP 2633520 A1 EP2633520 A1 EP 2633520A1
Authority
EP
European Patent Office
Prior art keywords
audio signal
encoding parameter
parameter
transformed
frequency
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP10859153.8A
Other languages
German (de)
French (fr)
Other versions
EP2633520A4 (en
EP2633520B1 (en
Inventor
Christof Faller
Lei Miao
Yue Lang
Jianfeng Xu
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Publication of EP2633520A4 publication Critical patent/EP2633520A4/en
Publication of EP2633520A1 publication Critical patent/EP2633520A1/en
Application granted granted Critical
Publication of EP2633520B1 publication Critical patent/EP2633520B1/en
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/0204—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using subband decomposition

Definitions

  • the present invention relates to audio coding.
  • the down-mix audio signals result from a superposition of a plurality of audio channel signals of a multi-channel audio signal, e.g. of a stereo audio signal.
  • These less channels are waveform coded and side information, i.e. the spatial cues, relating to the original signal channel relations is added as encoding parameters to the coded audio channels.
  • the decoder uses this side information to re-generate the original number of audio channels based on the decoded waveform coded audio channels.
  • a basic parametric stereo coder may use inter-channel level differences (ILD) as a cue needed for generating the stereo signal from the mono down-mix audio signal. More sophisticated coders may also use the inter-channel coherence (ICC), which may represent a degree of similarity between the audio channel signals, i.e. audio channels. Furthermore, when coding binaural stereo signals e.g. for 3D audio or headphone based surround rendering, also an inter-channel phase difference (IPD) may play a role to reproduce phase/delay differences between the channels.
  • IPD inter-channel phase difference
  • the synthesis of ICC cues may be relevant for most audio and music contents to re-generate ambience, stereo reverb, source width, and other perceptions related to spatial impression as described in J. Blauert, Spatial Hearing: The
  • Coherence synthesis may be implemented by using de-correlators in frequency domain as described in E. Schuijers, W. Oomen, B. den Brinker, and J. Breebaart, "Advances in parametric coding for high-quality audio," in Preprint 114th Conv. Aud. Eng. Soc, Mar. 2003.
  • the known synthesis approaches for synthesizing multi-channel audio signals may suffer from an increased complexity.
  • the use of ICC parameters e.g. in addition to other parameters, such as inter-channel level differences (ICLDs) and inter- channel phase differences (ICPDs), may increase a bitrate overhead.
  • ICLDs inter-channel level differences
  • ICPDs inter-channel phase differences
  • a goal to be achieved by the present invention is to reduce complexity of a parametric coding scheme. This goal is achieved by the features of the
  • the invention is based on the finding that combining parametric encoding parameters such as ICC parameters may reduce bit rate required for representing the parameters and thus may reduce complexity of the resulting parametric encoding scheme.
  • the combined encoding parameters may be applied e.g. only to a certain frequency region in order to improve an audio quality for e.g. speech whereby the complexity and the memory requirements may further be reduced.
  • the invention relates to a parametric encoder for encoding a multi-channel audio signal having a first audio signal and a second audio signal, the parametric encoder having a transformer for transforming the first audio signal into frequency domain to obtain a first transformed audio signal, and for transforming the second audio signal into frequency domain to obtain a second transformed audio signal, a parameter generator for generating a first encoding parameter from the first transformed audio signal and from the second
  • the first and second encoding parameter may be an inter-channel phase difference.
  • the first and second encoding parameter may be an inter-channel coherence.
  • the first and second encoding parameter may be an inter-channel intensity difference.
  • the first and second encoding parameter may be an inter-channel level difference.
  • the parameter generator is configured to generate the first encoding parameter and the second encoding parameter upon a basis of a multiplication of values of the first transformed audio signal and of the second transformed audio signal.
  • the parameter combiner is configured to add the first encoding parameter and the second encoding parameter to obtain the combined encoding parameter.
  • the parameter combiner is configured to determine an average of the first encoding parameter and the second encoding parameter to obtain the combined encoding parameter.
  • the parameter combiner is configured to determine a weighted average of the first encoding parameter and the second encoding parameter using powers of the a first transformed audio signal and the second transformed signal at the certain frequency as weights to obtain the combined encoding parameter.
  • the parameter combiner is configured to determine a weighted average of the first encoding parameter and the second encoding parameter using a frequency-dependent weight to obtain the combined encoding parameter.
  • the parameter generator is configured to generate a plurality of encoding parameters from the first transformed audio signal and from the second transformed audio signal at a plurality of frequencies, and wherein the parameter combiner is configured to combine the plurality of the encoding parameters to obtain the combined encoding parameter.
  • the parametric encoder further comprises a signal combiner for combining the first transformed audio signal and the second transformed audio signal to obtain a down-mix signal.
  • the parametric encoder further comprises an inverse transformer for inversely transforming a combination of the first transformed audio signal and the second transformed audio signal to obtain a down-mix audio signal.
  • the invention relates to a parametric decoder for decoding a down-mix audio signal upon a basis of a combined encoding parameter, the down-mix audio signal representing a combination of a first audio signal and of a second audio signal, the combined encoding parameter
  • the parametric decoder having a transformer for transforming the down-mix audio signal to obtain a transformed down-mix audio signal having a certain frequency, a provider for providing a frequency-specific encoding parameter associated with the certain frequency upon the basis of the combined encoding parameter, and an audio synthesizer for synthesizing the first audio signal and the second audio signal at the certain frequency from the transformed down-mix audio signal and from the frequency-specific encoding parameter.
  • the parametric decoder may be configured to decode the down-mix audio signal which may be provided by the parametric encoder according to the first aspect.
  • the provider is adapted to provide the combined encoding parameter as the frequency-specific encoding parameter.
  • the parametric decoder further comprises an inverse transformer for inversely transforming the first audio signal and the second audio signal.
  • the invention relates to a method for parametrically encoding a multi-channel audio signal having a first audio signal and a second audio signal, the method having transforming the first audio signal into frequency domain to obtain a first transformed audio signal, and transforming the second audio signal into frequency domain to obtain a second transformed audio signal, generating a first encoding parameter from the first transformed audio signal and from the second transformed audio signal at a first frequency, and generating a second encoding parameter from the first transformed audio signal and from the second transformed audio signal at a second frequency, and combining the first encoding parameter and the second encoding parameter to obtain a combined encoding parameter.
  • the invention relates to a method for parametrically decoding a down-mix audio signal upon a basis of a combined encoding parameter, the down-mix audio signal representing a combination of a first audio signal and of a second audio signal, the combined encoding parameter
  • the method comprising transforming the down-mix audio signal to obtain a
  • Fig. 1 shows a block diagram of a parametric encoder according to an
  • Fig. 2 shows a block diagram of a parametric decoder according to an
  • Fig. 3 shows a diagram of a method for parametrically encoding according to an implementation form
  • Fig. 4 shows a diagram of a method for parametrically decoding according to an implementation form.
  • Fig. 1 shows a diagram of a parametric encoder for encoding a multi-channel audio signal having a first audio signal, x1 , and a second audio signal, x2, according to an implementation form.
  • the parametric encoder comprises a transformer 101 for transforming the first audio signal into frequency domain to obtain a first transformed audio signal, and for transforming the second audio signal into frequency domain to obtain a second transformed audio signal.
  • the transformer 101 may comprise a first transformer 103 for transforming the first audio signal, and a second transformer 105 for transforming the second audio signal.
  • the transformer 101 and/or the transformers 103, 105 may be Fourier transformers, by way of example.
  • the first and second transformed audio signals are provided to a parameter generator 107 for generating a first encoding parameter from the first transformed signal and from the second transformed audio signal at a first frequency, e.g. at the i-th frequency or in the i-th band.
  • the i- th band or "band i" refer to a frequency band i at or in which the parameter generator 107 generates the respective encoding parameter from the first and second transformed signal, and is also referred to as parameter band i.
  • the parameter generator 107 is further configured to generate a second encoding parameter from the first and second transformed audio signal at a second frequency or in a second band.
  • the first and the second encoding parameters are provided to the parameter combiner 109 which combines the first encoding parameter and the second encoding parameter to obtain a combined encoding parameter according to a principle described herein.
  • the parameter combiner 109 may separately obtain encoding parameters for different parameter bands.
  • the e.g. stereo input audio channels x1 and x2 are converted to a plurality of sub-bands or parameter bands.
  • the corresponding ICC parameters may be estimated.
  • One or more ICC parameter combining processes e.g. one of the processes according to the equations (1 )-(4), may be applied to the ICC parameters of all or subsets of parameter bands, to compute the combined ICC parameters.
  • At least one combined ICC parameter may be put into a bit stream 1 11 or transmitted to an audio decoder which is not depicted in Fig. 1 .
  • the encoding parameter may be any encoding parameter or of any encoding parameter type used for parametric encoding, e.g. inter-channel phase difference or inter-channel intensity difference or an inter-channel level difference or the like, and that the encoder may be adapted to produce combined encoding parameters according one, some or all of the aforementioned encoding parameter types and to include combined encoding parameters of different types in the bitstream 1 11 as side information.
  • the parametric encoder of Fig, 1 may form a parametric stereo encoder which estimates in parameter bands perceptual spatial cue parameters, such as ICLD, ICPD, and/or ICC. If the parameter band index is i, then the estimated parameters in that band are denoted ICLD(i), ICPD(i), and ICC(i). The left and right signal power in a parameter band are denoted P1 (i) and P2(i), respectively.
  • one or more combined ICC parameters may be computed, e.g. as an average
  • I is the set of indices of parameter bands of which ICC are used to compute the combined ICC parameter
  • is the number of indices in the set I
  • Another way of computing a combined ICC parameter is to use a weighted average, i.e.
  • Pi(i) denotes a signal power of the first audio channel signal in the i-th band
  • P 2 (i) denotes a signal power of the second audio channel signal in the i-th band
  • g t is a weight given to frequency (parameter band) i
  • Another example is to use an average considering not power but frequency weighting: Additionally, different parameter bands (frequencies) can be weighted differently, when computing the combined ICC:
  • a single full-band ICC performs surprisingly well.
  • the combined ICC is computed using all parameter bands, i.e. I contains all parameter band indices.
  • the speech quality may be improved by only using ICC in a limited frequency range.
  • ICC parameter bands between 500 Hz and 1 .5 kHz
  • the combined ICC is computed using only parameter bands between 500 Hz and 1.5 kHz, i.e. I contains only those indices.
  • the parametric encoder shown in Fig. 1 may estimate one or more combined ICC parameters by:
  • Fig. 2 shows a block diagram of a parametric encoder for decoding a down-mix audio signal according to an implementation form.
  • the down-mix signal may be provided by the parametric encoder as shown e.g. in Fig. 1 .
  • the parametric decoder comprises a transformer 201 for transforming the down-mix audio signal, as, to obtain a transformed down-mix audio signal having a certain frequency, e.g. an i-th frequency of a plurality of frequencies, or correspondingly a certain band, e.g. an i-th band of a plurality of bands.
  • the parametric decoder further comprises a provider 203 for providing a frequency-specific encoding parameter associated with the certain frequency.
  • the frequency-specific encoding parameter may be derived from the combined encoding parameter. However, the frequency-specific encoding parameter may correspond to the combined encoding parameter.
  • the parametric decoder further comprises an audio synthesizer 205, e.g. a stereo synthesizer, for synthesizing a first audio signal and a second audio signal at the certain frequency or in the certain band from the transformed down-mix audio signal provided by the transformer 201 using the frequency-specific encoding parameter as provided by the provider 203.
  • an audio synthesizer 205 e.g. a stereo synthesizer
  • the transformer 201 may be a Fourier transformer, wherein the audio synthesizer may synthesize the first and the second audio signal in frequency domain.
  • the output signal provided by the synthesizer 205 may correspond to the first and second audio channel signal.
  • the parametric encoder may further comprise an inverse transformer 207 for inversely transforming the first and second audio channel signal in time domain in order to obtain a first and second audio channel signal, x1 and x2, in time domain.
  • the parametric decoder shown in Fig. 1 uses for all parameter bands, or a subset J thereof, the combined ICC parameter or a modified version thereof. One can use the same subset of parameter bands at the decoder as at the encoder, i.e.
  • the parametric decoder may receive the down-mix signal s and the stereo parameters, i.e. encoding parameters, amongst which at least one combined ICC parameter may be received.
  • the stereo parameters i.e. encoding parameters
  • the decoder may apply the combined ICCs to parameter bands corresponding to the frequency regions from which the combined ICCs were estimated.
  • Fig. 3 shows a diagram of a method for parametrically encoding a multi-channel audio signal having the first and the second audio signal as mentioned above.
  • the method comprises transforming 301 the first and second audio signal into frequency domain to obtain a first and second transformed audio signal, generating 303 a first encoding parameter from the first and second transformed audio signal at a first frequency, and a second encoding parameter from the first and second transformed audio signal at a second frequency, and combining 305 the first and second encoding parameter to obtain a combined encoding parameter.
  • the method depicted in Fig. 3 may be performed by the parametric encoder as shown in Fig. 1 .
  • Fig. 4 shows a block diagram of a method for parametrically decoding a down-mix audio signal upon a basis of a combined encoding parameter.
  • the down-mix audio signal may represent a combination, e.g. a superposition, of a first and second audio signal.
  • the combined encoding parameter may have features as described above.
  • the method comprises transforming 401 the down-mix audio signal to obtain a transformed down-mix audio signal having a certain frequency, providing 403 a frequency-specific encoding parameter associated with the certain frequency upon the basis of the combined encoding parameter, according to the principle described herein, and synthesizing 405 the first and second audio signal at the certain frequency from the transformed down-mix audio signal and from the frequency-specific encoding parameter.
  • the method depicted in Fig. 4 may be performed by the parametric decoder as shown in Fig. 2.
  • the parametric decoder shown in Fig. 2 may be a parametric stereo decoder adapted for

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Stereophonic System (AREA)

Abstract

A parametric encoder for encoding a multi-channel audio signal having a first audio signal and a second audio signal is provided. The parametric encoder has a transformer (101) for transforming the first audio signal into frequency domain to obtain a first transformed audio signal and for transforming the second audio signal into frequency domain to obtain a second transformed audio signal, a parameter generator (107) for generating a first encoding parameter from the first transformed audio signal and from the second transformed audio signal at a first frequency and for generating a second encoding parameter from the first transformed audio signal and from the second transformed audio signal at a second frequency, and a parameter combiner (109) for combining the first encoding parameter and the second encoding parameter to obtain a combined encoding parameter.

Description

DESCRIPTION
Parametric encoder for encoding a multi-channel audio signal TECHNICAL FIELD
The present invention relates to audio coding. BACKGROUND OF THE INVENTION
Parametric stereo or multi-channel audio coding as described e.g. in C. Faller and F. Baumgarte, "Efficient representation of spatial audio using perceptual
parametrization," in Proc. IEEE Workshop on Appl. of Sig. Proc. to Audio and Acoust., Oct. 2001 , pp. 199-202, uses spatial cues to synthesize down-mix - usually mono or stereo - audio signals to signals with more channels. Usually, the down-mix audio signals result from a superposition of a plurality of audio channel signals of a multi-channel audio signal, e.g. of a stereo audio signal. These less channels are waveform coded and side information, i.e. the spatial cues, relating to the original signal channel relations is added as encoding parameters to the coded audio channels. The decoder uses this side information to re-generate the original number of audio channels based on the decoded waveform coded audio channels.
A basic parametric stereo coder may use inter-channel level differences (ILD) as a cue needed for generating the stereo signal from the mono down-mix audio signal. More sophisticated coders may also use the inter-channel coherence (ICC), which may represent a degree of similarity between the audio channel signals, i.e. audio channels. Furthermore, when coding binaural stereo signals e.g. for 3D audio or headphone based surround rendering, also an inter-channel phase difference (IPD) may play a role to reproduce phase/delay differences between the channels. The synthesis of ICC cues may be relevant for most audio and music contents to re-generate ambience, stereo reverb, source width, and other perceptions related to spatial impression as described in J. Blauert, Spatial Hearing: The
Psychophysics of Human Sound Localization, The MIT Press, Cambridge,
Massachusetts, USA, 1997. Coherence synthesis may be implemented by using de-correlators in frequency domain as described in E. Schuijers, W. Oomen, B. den Brinker, and J. Breebaart, "Advances in parametric coding for high-quality audio," in Preprint 114th Conv. Aud. Eng. Soc, Mar. 2003. However, the known synthesis approaches for synthesizing multi-channel audio signals may suffer from an increased complexity. Furthermore, the use of ICC parameters, e.g. in addition to other parameters, such as inter-channel level differences (ICLDs) and inter- channel phase differences (ICPDs), may increase a bitrate overhead.
SUMMARY OF THE INVENTION
A goal to be achieved by the present invention is to reduce complexity of a parametric coding scheme. This goal is achieved by the features of the
independent claims. Further embodiments are apparent from the description, the drawings and from the dependent claims.
The invention is based on the finding that combining parametric encoding parameters such as ICC parameters may reduce bit rate required for representing the parameters and thus may reduce complexity of the resulting parametric encoding scheme. The combined encoding parameters may be applied e.g. only to a certain frequency region in order to improve an audio quality for e.g. speech whereby the complexity and the memory requirements may further be reduced.
According to a first aspect, the invention relates to a parametric encoder for encoding a multi-channel audio signal having a first audio signal and a second audio signal, the parametric encoder having a transformer for transforming the first audio signal into frequency domain to obtain a first transformed audio signal, and for transforming the second audio signal into frequency domain to obtain a second transformed audio signal, a parameter generator for generating a first encoding parameter from the first transformed audio signal and from the second
transformed audio signal at a first frequency, and for generating a second encoding parameter from the first transformed audio signal and from the second transformed audio signal at a second frequency, and a parameter combiner for combining the first encoding parameter and the second encoding parameter to obtain a combined encoding parameter. According to a first implementation form of the first aspect, the first and second encoding parameter may be an inter-channel phase difference.
According to a third implementation form of the first aspect, the first and second encoding parameter may be an inter-channel coherence.
According to a fourth implementation form of the first aspect, the first and second encoding parameter may be an inter-channel intensity difference.
According to a fifth implementation form of the first aspect, the first and second encoding parameter may be an inter-channel level difference.
According to a sixth implementation form of the first aspect, the parameter generator is configured to generate the first encoding parameter and the second encoding parameter upon a basis of a multiplication of values of the first transformed audio signal and of the second transformed audio signal.
According to a seventh implementation form of the first aspect, the parameter combiner is configured to add the first encoding parameter and the second encoding parameter to obtain the combined encoding parameter. According to an eighth implementation form of the first aspect, the parameter combiner is configured to determine an average of the first encoding parameter and the second encoding parameter to obtain the combined encoding parameter. According to a ninth implementation form of the first aspect, the parameter combiner is configured to determine a weighted average of the first encoding parameter and the second encoding parameter using powers of the a first transformed audio signal and the second transformed signal at the certain frequency as weights to obtain the combined encoding parameter.
According to a tenth implementation form of the first aspect, the parameter combiner is configured to determine a weighted average of the first encoding parameter and the second encoding parameter using a frequency-dependent weight to obtain the combined encoding parameter.
According to an eleventh implementation form of the first aspect, the parameter generator is configured to generate a plurality of encoding parameters from the first transformed audio signal and from the second transformed audio signal at a plurality of frequencies, and wherein the parameter combiner is configured to combine the plurality of the encoding parameters to obtain the combined encoding parameter.
According to a twelfth implementation form of the first aspect, the parametric encoder further comprises a signal combiner for combining the first transformed audio signal and the second transformed audio signal to obtain a down-mix signal.
According to a thirteenth implementation form of the first aspect, the parametric encoder further comprises an inverse transformer for inversely transforming a combination of the first transformed audio signal and the second transformed audio signal to obtain a down-mix audio signal. According to a second aspect, the invention relates to a parametric decoder for decoding a down-mix audio signal upon a basis of a combined encoding parameter, the down-mix audio signal representing a combination of a first audio signal and of a second audio signal, the combined encoding parameter
representing a combination of frequency-specific encoding parameters, the parametric decoder having a transformer for transforming the down-mix audio signal to obtain a transformed down-mix audio signal having a certain frequency, a provider for providing a frequency-specific encoding parameter associated with the certain frequency upon the basis of the combined encoding parameter, and an audio synthesizer for synthesizing the first audio signal and the second audio signal at the certain frequency from the transformed down-mix audio signal and from the frequency-specific encoding parameter.
By way of example, the parametric decoder may be configured to decode the down-mix audio signal which may be provided by the parametric encoder according to the first aspect.
According to a first implementation form of the second aspect, the provider is adapted to provide the combined encoding parameter as the frequency-specific encoding parameter.
According to a second implementation form of the second aspect, the parametric decoder further comprises an inverse transformer for inversely transforming the first audio signal and the second audio signal.
According to a third aspect the invention relates to a method for parametrically encoding a multi-channel audio signal having a first audio signal and a second audio signal, the method having transforming the first audio signal into frequency domain to obtain a first transformed audio signal, and transforming the second audio signal into frequency domain to obtain a second transformed audio signal, generating a first encoding parameter from the first transformed audio signal and from the second transformed audio signal at a first frequency, and generating a second encoding parameter from the first transformed audio signal and from the second transformed audio signal at a second frequency, and combining the first encoding parameter and the second encoding parameter to obtain a combined encoding parameter.
Further method steps of implementation forms according to the third aspect are directly derivable from the functionality of the parametric encoder according to the first aspect.
According to a fourth aspect, the invention relates to a method for parametrically decoding a down-mix audio signal upon a basis of a combined encoding parameter, the down-mix audio signal representing a combination of a first audio signal and of a second audio signal, the combined encoding parameter
representing a combination of frequency-specific encoding parameters, the method comprising transforming the down-mix audio signal to obtain a
transformed down-mix audio signal having a certain frequency, providing a frequency-specific encoding parameter associated with the certain frequency upon the basis of the combined encoding parameter, and synthesizing the first audio signal and the second audio signal at the certain frequency from the transformed down-mix audio signal and from the frequency-specific encoding parameter.
Further method steps of implementation forms according to the fourth aspect are directly derivable from the functionality of the parametric decoder according to the second aspect.
BRIEF DESCRIPTION OF THE DRAWINGS
Further embodiments of the invention will be described with reference to the following drawings, in which: Fig. 1 shows a block diagram of a parametric encoder according to an
implementation form;
Fig. 2 shows a block diagram of a parametric decoder according to an
implementation form;
Fig. 3 shows a diagram of a method for parametrically encoding according to an implementation form; and Fig. 4 shows a diagram of a method for parametrically decoding according to an implementation form.
DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION Fig. 1 shows a diagram of a parametric encoder for encoding a multi-channel audio signal having a first audio signal, x1 , and a second audio signal, x2, according to an implementation form. The parametric encoder comprises a transformer 101 for transforming the first audio signal into frequency domain to obtain a first transformed audio signal, and for transforming the second audio signal into frequency domain to obtain a second transformed audio signal. The transformer 101 may comprise a first transformer 103 for transforming the first audio signal, and a second transformer 105 for transforming the second audio signal. The transformer 101 and/or the transformers 103, 105 may be Fourier transformers, by way of example. The first and second transformed audio signals are provided to a parameter generator 107 for generating a first encoding parameter from the first transformed signal and from the second transformed audio signal at a first frequency, e.g. at the i-th frequency or in the i-th band. The i- th band or "band i" (see also in Fig. 1 ) refer to a frequency band i at or in which the parameter generator 107 generates the respective encoding parameter from the first and second transformed signal, and is also referred to as parameter band i. The parameter generator 107 is further configured to generate a second encoding parameter from the first and second transformed audio signal at a second frequency or in a second band. The first and the second encoding parameters are provided to the parameter combiner 109 which combines the first encoding parameter and the second encoding parameter to obtain a combined encoding parameter according to a principle described herein. However, the parameter combiner 109 may separately obtain encoding parameters for different parameter bands.
With reference to Fig. 1 and to ICC parameters forming an embodiment of encoding parameters, the e.g. stereo input audio channels x1 and x2 are converted to a plurality of sub-bands or parameter bands. In all or in a subset of the parameter bands the corresponding ICC parameters may be estimated. One or more ICC parameter combining processes, e.g. one of the processes according to the equations (1 )-(4), may be applied to the ICC parameters of all or subsets of parameter bands, to compute the combined ICC parameters. At least one combined ICC parameter may be put into a bit stream 1 11 or transmitted to an audio decoder which is not depicted in Fig. 1 .
The following embodiments are exemplarily described with respect to ICC forming an embodiment of an encoding parameter. It is, however, to be understood, that the encoding parameter may be any encoding parameter or of any encoding parameter type used for parametric encoding, e.g. inter-channel phase difference or inter-channel intensity difference or an inter-channel level difference or the like, and that the encoder may be adapted to produce combined encoding parameters according one, some or all of the aforementioned encoding parameter types and to include combined encoding parameters of different types in the bitstream 1 11 as side information.
The parametric encoder of Fig, 1 may form a parametric stereo encoder which estimates in parameter bands perceptual spatial cue parameters, such as ICLD, ICPD, and/or ICC. If the parameter band index is i, then the estimated parameters in that band are denoted ICLD(i), ICPD(i), and ICC(i). The left and right signal power in a parameter band are denoted P1 (i) and P2(i), respectively.
In this regard, one or more combined ICC parameters may be computed, e.g. as an average
where I is the set of indices of parameter bands of which ICC are used to compute the combined ICC parameter and N| is the number of indices in the set I Another way of computing a combined ICC parameter is to use a weighted average, i.e.
∑ (3 (0 + 3(0) wherein Pi(i) denotes a signal power of the first audio channel signal in the i-th band, and wherein P2(i) denotes a signal power of the second audio channel signal in the i-th band.
Additionally, different parameter bands (frequencies) can be weighted differently, when computing the combined ICC:
∑g,-(3(0 + 3(0)ICC(0
ICC = ^= (3)
∑ a (3 (0+3(0)
where gt is a weight given to frequency (parameter band) i Another example is to use an average considering not power but frequency weighting: Additionally, different parameter bands (frequencies) can be weighted differently, when computing the combined ICC:
∑g,-ICC(
ICC = ^L_ (4) iel
A single full-band ICC performs surprisingly well. In this case, the combined ICC is computed using all parameter bands, i.e. I contains all parameter band indices.
According to some implementation forms, the speech quality may be improved by only using ICC in a limited frequency range. When only parameter bands between 500 Hz and 1 .5 kHz are used for generating ICC parameters, less artifacts occur. In this case, the combined ICC is computed using only parameter bands between 500 Hz and 1.5 kHz, i.e. I contains only those indices. According to some implementation forms, the parametric encoder shown in Fig. 1 may estimate one or more combined ICC parameters by:
(a) Combining the ICC parameters from a plurality of parameter bands to a combined ICC parameter,
(b) Putting combined ICC parameters into a bit stream, and
(c) Outputting the bit stream.
Fig. 2 shows a block diagram of a parametric encoder for decoding a down-mix audio signal according to an implementation form. The down-mix signal may be provided by the parametric encoder as shown e.g. in Fig. 1 . The parametric decoder comprises a transformer 201 for transforming the down-mix audio signal, as, to obtain a transformed down-mix audio signal having a certain frequency, e.g. an i-th frequency of a plurality of frequencies, or correspondingly a certain band, e.g. an i-th band of a plurality of bands. The parametric decoder further comprises a provider 203 for providing a frequency-specific encoding parameter associated with the certain frequency. The frequency-specific encoding parameter may be derived from the combined encoding parameter. However, the frequency-specific encoding parameter may correspond to the combined encoding parameter. The parametric decoder further comprises an audio synthesizer 205, e.g. a stereo synthesizer, for synthesizing a first audio signal and a second audio signal at the certain frequency or in the certain band from the transformed down-mix audio signal provided by the transformer 201 using the frequency-specific encoding parameter as provided by the provider 203.
According to some implementation forms, the transformer 201 may be a Fourier transformer, wherein the audio synthesizer may synthesize the first and the second audio signal in frequency domain. Thus, the output signal provided by the synthesizer 205 may correspond to the first and second audio channel signal. However, according to some implementation forms, the parametric encoder may further comprise an inverse transformer 207 for inversely transforming the first and second audio channel signal in time domain in order to obtain a first and second audio channel signal, x1 and x2, in time domain. The parametric decoder shown in Fig. 1 uses for all parameter bands, or a subset J thereof, the combined ICC parameter or a modified version thereof. One can use the same subset of parameter bands at the decoder as at the encoder, i.e. J=I , or a different subset. With respect to Fig. 2, the parametric decoder may receive the down-mix signal s and the stereo parameters, i.e. encoding parameters, amongst which at least one combined ICC parameter may be received. For at least one parameter band an ICC parameter derived from combined ICC parameters is used. To some bands no ICC synthesis may be applied. According to an implementation form, for the "Combined ICC to Band ICC
Conversion" when one combined ICC is used, combined ICC is applied to all parameter bands. Or, if the combined ICC was estimated only for a subset of bands, J , then the decoder may apply the combined ICC to all bands, to the same subset, or to another subset, e.g. a subset of the same subset.
If two combined ICC are used, representing two distinct frequency regions of the audio signal. The decoder may apply the combined ICCs to parameter bands corresponding to the frequency regions from which the combined ICCs were estimated.
Fig. 3 shows a diagram of a method for parametrically encoding a multi-channel audio signal having the first and the second audio signal as mentioned above. The method comprises transforming 301 the first and second audio signal into frequency domain to obtain a first and second transformed audio signal, generating 303 a first encoding parameter from the first and second transformed audio signal at a first frequency, and a second encoding parameter from the first and second transformed audio signal at a second frequency, and combining 305 the first and second encoding parameter to obtain a combined encoding parameter. By way of example, the method depicted in Fig. 3 may be performed by the parametric encoder as shown in Fig. 1 .
Fig. 4 shows a block diagram of a method for parametrically decoding a down-mix audio signal upon a basis of a combined encoding parameter. The down-mix audio signal may represent a combination, e.g. a superposition, of a first and second audio signal. The combined encoding parameter may have features as described above.
The method comprises transforming 401 the down-mix audio signal to obtain a transformed down-mix audio signal having a certain frequency, providing 403 a frequency-specific encoding parameter associated with the certain frequency upon the basis of the combined encoding parameter, according to the principle described herein, and synthesizing 405 the first and second audio signal at the certain frequency from the transformed down-mix audio signal and from the frequency-specific encoding parameter.
According to some implementation forms, the method depicted in Fig. 4 may be performed by the parametric decoder as shown in Fig. 2.
According to some implementation forms, the parametric decoder shown in Fig. 2 may be a parametric stereo decoder adapted for
(a) Receiving one or more combined ICC parameters, and
(b) Using for at least one parameter band an ICC parameter related to the received combined ICC parameters.

Claims

CLAIMS:
1 . Parametric encoder for encoding a multi-channel audio signal having a first audio signal and a second audio signal, the parametric encoder having: a transformer (101 ) for transforming the first audio signal into frequency domain to obtain a first transformed audio signal, and for transforming the second audio signal into frequency domain to obtain a second transformed audio signal; a parameter generator (107) for generating a first encoding parameter from the first transformed audio signal and from the second transformed audio signal at a first frequency, and for generating a second encoding parameter from the first transformed audio signal and from the second transformed audio signal at a second frequency; and a parameter combiner (109) for combining the first encoding parameter and the second encoding parameter to obtain a combined encoding parameter.
2. The parametric encoder of claim 1 , wherein the first encoding parameter and the second encoding parameter is an inter-channel phase difference or an inter-channel coherence or an inter-channel intensity difference or an inter-channel level difference.
3. The parametric encoder of claim 1 or 2, wherein the parameter generator (107) is configured to generate the first encoding parameter and the second encoding parameter upon a basis of a multiplication of values of the first
transformed audio signal and of the second transformed audio signal.
4. The parametric encoder of any of the preceding claims, wherein the parameter combiner (109) is configured to add the first encoding parameter and the second encoding parameter to obtain the combined encoding parameter.
5. The parametric encoder of any of the preceding claims, wherein the parameter combiner (109) is configured to determine an average of the first encoding parameter and the second encoding parameter to obtain the combined encoding parameter.
6. The parametric encoder of any of the preceding claims, wherein the parameter combiner (109) is configured to determine a weighted average of the first encoding parameter and the second encoding parameter using powers of the a first transformed audio signal and the second transformed signal at the certain frequency as weights to obtain the combined encoding parameter.
7. The parametric encoder of any of the preceding claims, wherein the parameter combiner (109) is configured to determine a weighted average of the first encoding parameter and the second encoding parameter using a frequency- dependent weight to obtain the combined encoding parameter.
8. The parametric encoder of any of the preceding claims, wherein the parameter generator (107) is configured to generate a plurality of encoding parameters from the first transformed audio signal and from the second transformed audio signal at a plurality of frequencies, and wherein the parameter combiner is configured to combine the plurality of the encoding parameters to obtain the combined encoding parameter.
9. The parametric encoder of any of the preceding claims, being further configured to combine the first transformed audio signal and the second
transformed audio signal to obtain a down-mix signal.
10. The parametric encoder of any of the preceding claims, further comprising an inverse transformer for inversely transforming a combination of the first transformed audio signal and the second transformed audio signal to obtain a down-mix audio signal.
1 1. A parametric decoder for decoding a down-mix audio signal upon a basis of a combined encoding parameter, the down-mix audio signal representing a combination of a first audio signal and of a second audio signal, the combined encoding parameter representing a combination of frequency-specific encoding parameters, the parametric decoder having: a transformer (201 ) for transforming the down-mix audio signal to obtain a transformed down-mix audio signal having a certain frequency; a provider (203) for providing a frequency-specific encoding parameter associated with the certain frequency upon the basis of the combined encoding parameter; and an audio synthesizer (205) for synthesizing the first audio signal and the second audio signal at the certain frequency from the transformed down-mix audio signal and from the frequency-specific encoding parameter.
12. The parametric decoder of claim 1 1 , wherein the provider (203) is used to provide the combined encoding parameter as the frequency-specific encoding parameter.
13. The parametric decoder of claim 1 1 or 12, further comprising an inverse transformer (207) for inversely transforming the first audio signal and the second audio signal.
14. Method for parametrically encoding a multi-channel audio signal having a first audio signal and a second audio signal, the method having: transforming (301 ) the first audio signal into frequency domain to obtain a first transformed audio signal, and transforming the second audio signal into frequency domain to obtain a second transformed audio signal; generating (303) a first encoding parameter from the first transformed audio signal and from the second transformed audio signal at a first frequency, and generating a second encoding parameter from the first transformed audio signal and from the second transformed audio signal at a second frequency; and combining (305) the first encoding parameter and the second encoding parameter to obtain a combined encoding parameter.
15. Method for parametrically decoding a down-mix audio signal upon a basis of a combined encoding parameter, the down-mix audio signal representing a combination of a first audio signal and of a second audio signal, the combined encoding parameter representing a combination of frequency-specific encoding parameters, the method comprising: transforming (401 ) the down-mix audio signal to obtain a transformed down-mix audio signal having a certain frequency; providing (403) a frequency-specific encoding parameter associated with the certain frequency upon the basis of the combined encoding parameter; and synthesizing (405) the first audio signal and the second audio signal at the certain frequency from the transformed down-mix audio signal and from the frequency- specific encoding parameter.
EP10859153.8A 2010-11-03 2010-11-03 Parametric encoder for encoding a multi-channel audio signal Not-in-force EP2633520B1 (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2010/078358 WO2012058805A1 (en) 2010-11-03 2010-11-03 Parametric encoder for encoding a multi-channel audio signal

Publications (3)

Publication Number Publication Date
EP2633520A4 EP2633520A4 (en) 2013-09-04
EP2633520A1 true EP2633520A1 (en) 2013-09-04
EP2633520B1 EP2633520B1 (en) 2015-09-02

Family

ID=46023928

Family Applications (1)

Application Number Title Priority Date Filing Date
EP10859153.8A Not-in-force EP2633520B1 (en) 2010-11-03 2010-11-03 Parametric encoder for encoding a multi-channel audio signal

Country Status (4)

Country Link
EP (1) EP2633520B1 (en)
CN (1) CN102844808B (en)
ES (1) ES2553398T3 (en)
WO (1) WO2012058805A1 (en)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102739320B (en) * 2012-06-16 2014-11-05 天地融科技股份有限公司 Method, system and device for transmitting audio data and electronic signature tool
RU2628900C2 (en) * 2012-08-10 2017-08-22 Фраунхофер-Гезелльшафт Цур Фердерунг Дер Ангевандтен Форшунг Е.Ф. Coder, decoder, system and method using concept of balance for parametric coding of audio objects
WO2014161996A2 (en) 2013-04-05 2014-10-09 Dolby International Ab Audio processing system
US9838819B2 (en) * 2014-07-02 2017-12-05 Qualcomm Incorporated Reducing correlation between higher order ambisonic (HOA) background channels
CA2997334A1 (en) * 2015-09-25 2017-03-30 Voiceage Corporation Method and system for encoding left and right channels of a stereo sound signal selecting between two and four sub-frames models depending on the bit budget
CN107452387B (en) 2016-05-31 2019-11-12 华为技术有限公司 A kind of extracting method and device of interchannel phase differences parameter
CN117133297A (en) 2017-08-10 2023-11-28 华为技术有限公司 Coding method of time domain stereo parameter and related product
GB2576769A (en) 2018-08-31 2020-03-04 Nokia Technologies Oy Spatial parameter signalling
GB2582749A (en) * 2019-03-28 2020-10-07 Nokia Technologies Oy Determination of the significance of spatial audio parameters and associated encoding
CN110740404B (en) * 2019-09-27 2020-12-25 广州励丰文化科技股份有限公司 Audio correlation processing method and audio processing device
CN112731289B (en) * 2020-12-10 2024-05-07 深港产学研基地（北京大学香港科技大学深圳研修院） Binaural sound source positioning method and device based on weighted template matching

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030219130A1 (en) * 2002-05-24 2003-11-27 Frank Baumgarte Coherence-based audio coding and synthesis
US20050180579A1 (en) * 2004-02-12 2005-08-18 Frank Baumgarte Late reverberation-based synthesis of auditory scenes
WO2010090019A1 (en) * 2009-02-04 2010-08-12 パナソニック株式会社 Connection apparatus, remote communication system, and connection method

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007507726A (en) * 2003-09-29 2007-03-29 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ Audio signal encoding
WO2008132850A1 (en) * 2007-04-25 2008-11-06 Panasonic Corporation Stereo audio encoding device, stereo audio decoding device, and their method
CN100571043C (en) * 2007-11-06 2009-12-16 武汉大学 A kind of space parameter stereo coding/decoding method and device thereof
US8060042B2 (en) * 2008-05-23 2011-11-15 Lg Electronics Inc. Method and an apparatus for processing an audio signal
CN101809656B (en) * 2008-07-29 2013-03-13 松下电器产业株式会社 Sound coding device, sound decoding device, sound coding/decoding device, and conference system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030219130A1 (en) * 2002-05-24 2003-11-27 Frank Baumgarte Coherence-based audio coding and synthesis
US20050180579A1 (en) * 2004-02-12 2005-08-18 Frank Baumgarte Late reverberation-based synthesis of auditory scenes
WO2010090019A1 (en) * 2009-02-04 2010-08-12 パナソニック株式会社 Connection apparatus, remote communication system, and connection method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of WO2012058805A1 *

Also Published As

Publication number Publication date
EP2633520A4 (en) 2013-09-04
CN102844808B (en) 2016-01-13
CN102844808A (en) 2012-12-26
EP2633520B1 (en) 2015-09-02
ES2553398T3 (en) 2015-12-09
WO2012058805A1 (en) 2012-05-10

Similar Documents

Publication Publication Date Title
US20230345176A1 (en) Audio decoder for audio channel reconstruction
EP2633520B1 (en) Parametric encoder for encoding a multi-channel audio signal
EP2088580B1 (en) Audio decoding
CN104885150B (en) The decoder and method of the universal space audio object coding parameter concept of situation are mixed/above mixed for multichannel contracting
JP6141980B2 (en) Apparatus and method for adapting audio information in spatial audio object coding
US8626503B2 (en) Audio encoding and decoding
JP2015527611A (en) Decoder and method for multi-instance spatial acoustic object coding employing parametric concept for multi-channel downmix / upmix configuration
Lang et al. Novel low complexity coherence estimation and synthesis algorithms for parametric stereo coding

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20130530

A4 Supplementary search report drawn up and despatched

Effective date: 20130724

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAX Request for extension of the european patent (deleted)
17Q First examination report despatched

Effective date: 20140414

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

RIC1 Information provided on ipc code assigned before grant

Ipc: G10L 19/02 20130101ALN20150218BHEP

Ipc: G10L 19/008 20130101ALI20150218BHEP

Ipc: G10L 19/00 20130101AFI20150218BHEP

Ipc: G10L 13/00 20060101ALI20150218BHEP

INTG Intention to grant announced

Effective date: 20150311

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 747064

Country of ref document: AT

Kind code of ref document: T

Effective date: 20150915

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 6

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602010027288

Country of ref document: DE

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2553398

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20151209

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 747064

Country of ref document: AT

Kind code of ref document: T

Effective date: 20150902

REG Reference to a national code

Ref country code: NL

Ref legal event code: FP

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151202

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151203

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150902

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150902

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150902

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150902

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150902

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150902

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150902

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150902

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150902

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160102

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150902

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160104

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150902

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602010027288

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151103

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150902

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20151130

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20151130

26N No opposition filed

Effective date: 20160603

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150902

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150902

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 7

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20151103

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150902

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150902

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150902

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20101103

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150902

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150902

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150902

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 8

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150902

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150902

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 9

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150902

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20201015

Year of fee payment: 11

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20201020

Year of fee payment: 11

Ref country code: GB

Payment date: 20201021

Year of fee payment: 11

Ref country code: ES

Payment date: 20201208

Year of fee payment: 11

Ref country code: FR

Payment date: 20201013

Year of fee payment: 11

Ref country code: IT

Payment date: 20201013

Year of fee payment: 11

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602010027288

Country of ref document: DE

REG Reference to a national code

Ref country code: NL

Ref legal event code: MM

Effective date: 20211201

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20211103

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20211201

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20211103

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220601

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20211130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20211103

REG Reference to a national code

Ref country code: ES

Ref legal event code: FD2A

Effective date: 20230214

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20211104