EP3014611B1 - Verbesserte frequenzbanderweiterung in einem audiosignaldecodierer - Google Patents

Verbesserte frequenzbanderweiterung in einem audiosignaldecodierer Download PDF

Info

Publication number
EP3014611B1
EP3014611B1 EP14742262.0A EP14742262A EP3014611B1 EP 3014611 B1 EP3014611 B1 EP 3014611B1 EP 14742262 A EP14742262 A EP 14742262A EP 3014611 B1 EP3014611 B1 EP 3014611B1
Authority
EP
European Patent Office
Prior art keywords
signal
band
extended
frequency
frequency band
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP14742262.0A
Other languages
English (en)
French (fr)
Other versions
EP3014611A1 (de
Inventor
Magdalena KANIEWSKA
Stéphane RAGOT
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Orange SA
Original Assignee
Orange SA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Orange SA filed Critical Orange SA
Publication of EP3014611A1 publication Critical patent/EP3014611A1/de
Application granted granted Critical
Publication of EP3014611B1 publication Critical patent/EP3014611B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/038—Speech enhancement, e.g. noise reduction or echo cancellation using band spreading techniques
    • G10L21/0388—Details of processing therefor
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/012—Comfort noise or silence coding
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/06—Determination or coding of the spectral characteristics, e.g. of the short-term prediction coefficients
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/08—Determination or coding of the excitation function; Determination or coding of the long-term prediction parameters
    • G10L19/083—Determination or coding of the excitation function; Determination or coding of the long-term prediction parameters the excitation function being an excitation gain
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/08—Determination or coding of the excitation function; Determination or coding of the long-term prediction parameters
    • G10L19/12—Determination or coding of the excitation function; Determination or coding of the long-term prediction parameters the excitation function being a code excitation, e.g. in code excited linear prediction [CELP] vocoders
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/26—Pre-filtering or post-filtering
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/038—Speech enhancement, e.g. noise reduction or echo cancellation using band spreading techniques
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/08—Determination or coding of the excitation function; Determination or coding of the long-term prediction parameters

Definitions

  • the present invention relates to the field of coding / decoding and audio-frequency signal processing (such as speech, music or other signals) for their transmission or storage.
  • the invention relates to a method and a device for extending the frequency band in a decoder or a processor performing an audio-frequency signal improvement.
  • state of the art audio signal coding (mono) consists of perceptual encoding by transform or subband, with parametric coding of high frequencies by tape replication.
  • a review of conventional speech and audio coding methods can be found in the books WB Kleijn and KK Paliwal (Eds.), Speech Coding and Synthesis, Elsevier, 1995 ; M. Bosi, RE Goldberg, Introduction to Digital Audio Coding and Standards, Springer 2002 ; J. Benesty, MM Sondhi, Y. Huang (Eds.), Handbook of Speech Processing, Springer 2008 .
  • 3GPP AMR-WB Adaptive Multi-Rate Wideband codec (decoder and decoder), which operates at an input / output frequency of 16 kHz and in which the signal is divided into two sub-bands, the low band (0-6.4 kHz) which is sampled at 12.8 kHz and coded by CELP model and the high band (6.4-7 kHz) which is parametrically reconstructed by " band extension " ( or BWE for "Bandwidth Extension” with or without additional information depending on the mode of the current frame.
  • band extension or BWE for "Bandwidth Extension
  • the limitation of the coded band of the AMR-WB codec at 7 kHz is essentially related to the fact that the transmit frequency response of the broadband terminals has been approximated at the time of standardization (ETSI / 3GPP then ITU-T T) according to the frequency mask defined in the ITU-T P.341 standard and more precisely by using a so-called "P341" filter defined in the ITU-T G.191 standard which cuts frequencies above 7 kHz (this filter respects the mask defined in P.341).
  • a signal sampled at 16 kHz may have a defined audio band of 0 to 8000 Hz; the AMR-WB codec thus introduces a limitation of the high band in comparison with the theoretical bandwidth of 8 kHz.
  • the 3GPP AMR-WB speech codec was standardized in 2001 mainly for circuit-mode (CS) telephony applications over GSM (2G) and UMTS (3G). This same codec was also standardized in 2003 in the ITU-T as Recommendation G.722.2 "Wideband coding speech at around 16kbit / s using Adaptive Multi-Rate Wideband (AMR-WB)".
  • band extension in the AMR-WB codec is rather rudimentary. Indeed, the high band (6.4-7 kHz) is generated by formatting a white noise through a temporal envelope (applied in the form of gains per subframe) and frequency (by the application of a linear prediction synthesis filter or LPC for Linear Predictive Coding). This band extension technique is illustrated in figure 1 .
  • correction information is transmitted by the encoder AMR-WB and decoded (blocks 107, 108) in order to refine the estimated gain per subframe (4 bits every 5ms, ie 0.8 kbit / s) .
  • s HB ( n ) is finally processed by a FIR ("Finite Impulse Response") band-pass filter (block 112), to keep only the band 6 - 7 kHz; at 23.85 kbit / s, a low-pass filter of FIR type (block 113) is added to the processing to further attenuate the frequencies above 7 kHz.
  • the high frequency synthesis (HF) is finally added (block 130) to the low frequency synthesis (BF) obtained with the blocks 120 to 123 and resampled at 16 kHz (block 123).
  • the AMR-WB decoding algorithm has been improved in part with the development of the ITU-T G.718 scalable codec that was standardized in 2008.
  • ITU-T G.718 includes an interoperable mode, for which core coding is compatible with 12.65 kbit / s G.722.2 (AMR-WB) coding; in addition, the G.718 decoder has the distinction of being able to decode a bit stream AMR-WB / G.722.2 at all possible bit rates of the AMR-WB codec (6.6 to 23.85 kbit / s).
  • blind band extension is disclosed by the prior art US 2003/050786 A1 which comprises an LPC analysis of the narrowband signal, the use of these coefficients to obtain an envelope of the broadband signal, the extension of the filtered narrowband excitation signal, a filtering to obtain a signal in high band which is finally combined with the narrowband signal to provide an enlarged band signal.
  • the present invention improves the situation.
  • the excitation signal (resulting from the decoding of the low band or from a low band signal extraction) makes it possible to carry out the band extension with a signal model that is more suitable for certain types of band. signals like music signals.
  • the excitation signal decoded or estimated in the low band has in some cases harmonics, which when they exist, can be transposed in high frequency so that it ensures a certain level of harmonicity in the high band reconstructed.
  • the band extension according to the method thus makes it possible to improve the quality for this type of signal.
  • the band extension according to the method is performed by first extending an excitation signal and then applying a synthesis filtering step; this approach exploits the fact that the decoded excitation in the low band is a signal whose spectrum is relatively flat, which avoids the decoded signal whitening treatments that may exist in the known methods of band extension in the frequency domain in the state of the art.
  • first frequency band the energy at the current frame and that of the subframe in the low band signal
  • second frequency band the energy per frame in the high band
  • This makes it possible to keep in the high band the same ratio of energy between subframe and frame as in the low band, which is particularly beneficial when the energy of the subframes varies greatly, for example in the case of transient sounds. , of attacks.
  • the method further includes an adaptive bandpass filtering step based on the decoding rate of the current frame.
  • This adaptive filtering makes it possible to optimize the extended bandwidth as a function of the bit rate, and therefore the quality of the reconstructed signal after band extension. Indeed, for low bit rates (typically 6.6 and 8.85 kbit / s for AMR-WB), the overall quality of the decoded signal in the low band (by the AMR-WB codec or an interoperable version) not being very good, it is preferable not to extend the decoded band too much and therefore to limit the band extension by adapting the frequency response of the filter bandpass associated to cover for example an approximate band of 6 to 7 kHz; this limitation is all the more advantageous that the excitation signal itself is relatively poorly coded and it is preferable not to use a sub-band too wide for the extension of high frequencies.
  • the quality can be improved with HF synthesis covering a wider band, for example approximately 6 to 7.7 kHz.
  • the high limit of 7.7 kHz (instead of 8 kHz) is an example embodiment, which can be adjusted to values close to 7.7 kHz. This limitation is here justified by the fact that the extension is made in the invention without auxiliary information and that an extension up to 8 kHz (although theoretically possible) could result in artifacts for particular signals.
  • this limitation at 7.7 kHz takes into account the fact that typically the anti-aliasing filters in analog / digital conversion and resampling filters between 16 kHz and other frequencies are not perfect and they typically introduce a rejection at frequencies below 8 kHz.
  • the method comprises a step of transforming the time-frequency of the excitation signal, the step of obtaining an extended signal then taking place in the frequency domain and a reverse time-frequency transforming step. of the extended signal before the scaling and filtering steps.
  • the implementation of the band extension (of the excitation signal) in the frequency domain makes it possible to obtain a fineness of frequency analysis which is not available with a temporal approach, and also makes it possible to have a resolution Frequency sufficient to detect harmonics and transpose high harmonics of the signal (in the low band) to improve quality while respecting the structure of the signal.
  • this function includes a resampling of the excitation signal by adding samples to the spectrum of this signal.
  • the original spectrum is conserved, in order to be able to apply a progressive attenuation response of the high-pass filter in this frequency band and also to not introduce defects. 9 audible during the step of adding the low frequency synthesis to the high frequency synthesis.
  • the method comprises a de-emphasis filtering step of the extended signal at least in the second frequency band.
  • the signal in the second frequency band is brought into a domain coherent with the signal in the first frequency band.
  • the method further comprises a step of generating a noise signal at least in the second frequency band, the extended signal being obtained by combining the extended excitation signal and the noise signal.
  • the combining step is performed by adaptive additive mixing with a level equalization gain between the extended excitation signal and the noise signal.
  • this equalization gain allows the combining step to adapt to the characteristics of the signal to optimize the relative proportion of noise in the mixture.
  • This device has the same advantages as the method described above, which it implements.
  • the invention relates to a decoder comprising a device as described.
  • the invention relates to a storage medium, readable by a processor, integrated or not integrated with the band expansion device, possibly removable, storing a computer program implementing a band extension method as described above.
  • the figure 3 illustrates an example of a decoder, compatible with the norm AMR-WB / G.722.2 in which one finds a postprocessing similar to that introduced in G.718 and described with reference to the figure 2 and an improved tape extension according to the extension method of the invention, implemented by the tape extension device illustrated by block 309.
  • the CELP decoding (BF for low frequencies) always operates at the internal frequency of 12.8 kHz, as in AMR-WB and G.718, and the band extension (HF for high frequencies) subject of the invention operating at the frequency of 16 kHz, the synthesis BF and HF are combined (block 312) at the frequency fs after adequate resampling (block 306 and internal processing block 311).
  • the combination of the low and high bands can be done at 16 kHz, after resampling the low band of 12.8 to 16 kHz, before resampling the extended signal at the frequency fs.
  • the decoding of the low band described above assumes a current frame called "active" with a rate between 6.6 and 23.85 kbit / s.
  • active a current frame
  • some frames can be coded as "inactive” and in this case you can either transmit a silence descriptor (on 35 bits) or not transmit anything.
  • SID frame describes several parameters: ISF parameters averaged over 8 frames, average energy over 8 frames, dithering flag for the non-stationary noise reconstruction.
  • the decoder makes it possible to extend the decoded low band (50-6400 Hz while taking into account the 50 Hz high-pass filtering at the decoder, 0-6400 Hz in the decoder. the general case) to an extended band whose width varies, ranging from approximately 50-6900 Hz to 50-7700 Hz depending on the mode implemented in the current frame.
  • the extension of the excitation is carried out in the frequency domain in a band of 5000 to 8000 Hz, to allow bandpass filtering of width 6000 to 6900 or 7700 Hz.
  • the HF gain correction information (0.8 kbit / s) transmitted at 23.85 kbit / s is here ignored. So at the figure 3 no specific block at 23.85 kbit / s is used.
  • the high band decoding part is produced in block 309 representing the band extension device according to the invention and which is detailed in FIG. figure 5 in a first embodiment and at the figure 7 in a second embodiment.
  • This device comprises at least one module for obtaining an extended signal in at least one second frequency band greater than the first frequency band from an oversampled excitation signal and extended in at least a second band.
  • frequency U HB1 (k)
  • a delay (block 310) is introduced in the first embodiment to synchronize the outputs of the blocks 306 and 307 and the high band synthesized at 16 kHz is resampled from 16 kHz to the frequency fs (block output 311).
  • the delay T 30 samples, which corresponds to the resampling delay of 12.8 to 16 kHz of 15 samples + delay of the low frequency post-processing of 15 samples.
  • the extension method of the invention implemented in block 309 according to the first embodiment introduces preferentially no additional delay with respect to the low band reconstructed at 12.8 kHz; however, in variants of the invention (for example using a time / frequency transformation with overlap), a delay may be introduced.
  • the low and high bands are then combined (added) in block 312 and the resulting synthesis is post-processed by high-order 50 Hz (type IIR) high-pass filtering whose coefficients depend on the frequency fs (block 313) and output post-processing with optional noise gate application similar to G.718 (block 314).
  • high-order 50 Hz type IIR
  • the band extension device according to the invention illustrated by the block 309 according to the embodiment of the decoder of the figure 3 , implements a band extension method described now with reference to the figure 4 .
  • This extension device can also be independent of the decoder and can implement the method described in FIG. figure 4 to perform a band extension of an existing audio signal stored or transmitted to the device, with an analysis of the audio signal to extract an excitation and an LPC filter.
  • This device receives as input an excitation signal in a first so-called low-band frequency band u ( n ) in the case of an implementation in the time domain or U ( k ) in the case of an implementation. in the frequency domain for which a time-frequency transform step is then applied.
  • this received excitation signal is a decoded signal.
  • the low band excitation signal is extracted by analysis of the audio signal.
  • the low band audio signal is resampled before the excitation extraction step, so that the excitation extracted from the audio signal by linear prediction estimated from the low band signal (or LPC parameters associated with the low band) is already resampled.
  • An exemplary embodiment in this case consists in taking a sampled low band signal at 12.8 kHz, which has a low-band LPC filter describing the short-term spectral envelope for the current frame, oversampling it at 16 kHz, and filter it by an LPC prediction filter obtained by extrapolating the LPC filter.
  • Another embodiment is to take a low band signal sampled at 12.8 kHz which is not available LPC model, the oversampler at 16 kHz, perform an LPC analysis on this signal at 16 kHz, and filter this signal by a LPC prediction filter obtained by this analysis.
  • a step E401 for generating an extended oversampled excitation signal ( u ext ( n ) or U HB 1 ( k )) in a second frequency band greater than the first frequency band is performed.
  • This generation step may comprise both a resampling step and an extension step or simply an extension step depending on the excitation signal obtained at the input.
  • This extended oversampled excitation signal is used to obtain an extended signal (U HB2 (k)) in a second frequency band.
  • This extended signal then has a signal model adapted to certain types of signals thanks to the characteristics of the extended excitation signal.
  • This extended signal can be obtained after combining the oversampled and extended excitation signal with another signal, for example a noise signal.
  • a step E402 for generating a noise signal ( u HB ( n ) or U HB ( k )) at least in the second frequency band is performed.
  • the second frequency band is for example a high frequency band ranging from 6000 to 8000 Hz.
  • this noise can be generated pseudo-randomly by a linear congruential generator.
  • this noise generation can be replaced by other methods, for example a signal of constant amplitude (of arbitrary value such as 1) could be defined and random signs applied to each frequency line. generated.
  • the extended excitation signal is then combined with the noise signal in step E403 to obtain the extended signal which may also be called combined signal ( u HB 1 ( n ) or U HB 2 ( k )) in the band extended frequency corresponding to the entire frequency band including the first and the second frequency band.
  • the combination of these two types of signals makes it possible to obtain a combined signal with characteristics more adapted to certain types of signals such as musical signals.
  • the decoded or estimated excitation signal in the low band in some cases has harmonics closer to the musical signals than the noise signal alone.
  • Low-frequency harmonics if they exist, can thus be transposed into high frequency so that their mixing with noise makes it possible to ensure a certain level of harmonicity or relative level of noise or spectral flatness ("spectral flatness"). in English) in the reconstructed high band.
  • the band extension according to the method improves the quality for this type of signals compared to AMR-WB.
  • the combined signal (or extended) is then filtered E404 by a linear prediction filter whose coefficients are derived from the filter coefficients of the low band ( ⁇ (z)) decoded or obtained by analysis and extraction from the low band signal or an oversampled version of it.
  • the band extension according to the method is therefore achieved by first extending an excitation signal and then applying a linear prediction synthesis filtering step (LPC); this approach exploits the fact that the LPC decoded excitation in the low band is a signal whose spectrum is relatively flat, which avoids additional processing of whitening of the decoded signal in the band extension.
  • LPC linear prediction synthesis filtering step
  • the coefficients of this filter can for example be obtained from the decoded parameters of the linear band prediction filter (LPC).
  • LPC filter used at 16 kHz sampled high band is of the form 1 / ⁇ ( z / ⁇ ), where 1 / ⁇ ( z ) is the decoded low band filter, and ⁇ is a weighting factor
  • the frequency response of the filter 1 / ⁇ ( z / ⁇ ) corresponds to a spread of the frequency response of the decoded lowband filter.
  • the filter 1 / A ( z ) can be extended to a higher order (as to 6.6 kbit / s in the block 111) to avoid such spreading.
  • E405 adaptive bandpass filtering and / or E407 scaling can be carried out in order to improve the quality of the extension signal according to the decoding bit rate.
  • additional steps of E405 adaptive bandpass filtering and / or E407 scaling can be carried out in order to improve the quality of the extension signal according to the decoding bit rate.
  • E405 adaptive bandpass filtering and / or E407 scaling can be carried out in order to improve the quality of the extension signal according to the decoding bit rate.
  • additional steps of E405 adaptive bandpass filtering and / or E407 scaling can be carried out in order to improve the quality of the extension signal according to the decoding bit rate.
  • E407 scaling can be carried out in order to improve the quality of the extension signal according to the decoding bit rate.
  • the tape extension device is now described with reference to the figure 5 .
  • This device implements the band extension method described above with reference to the figure 4 .
  • a low band excitation signal decoded or estimated by analysis is received ( u ( n )).
  • the band extension here uses the decoded excitation at 12.8 kHz (exc2 or u ( n )) at the output of block 302.
  • the generation of the oversampled and extended excitation is carried out in a frequency band ranging from 5 to 8 kHz, thus including a second frequency band (6.4-8 kHz) greater than the first one. frequency band (0-6.4 kHz).
  • the generation of the extended excitation signal is effected at least on the second frequency band but also on a part of the first frequency band.
  • this signal is transformed to obtain an excitation signal spectrum U ( k ) by the time-frequency transformation module 500.
  • the DCT-IV transformation is implemented by FFT according to the algorithm called " Evolved DCT ( EDCT )" described in the article.
  • EDCT Evolved DCT
  • DM Zhang, HT Li, Low Complexity Transform - Evolved DCT, IEEE International Conference on Computational Science and Engineering (CSE), Aug. 2011, pp. 144-149 and implemented in ITU-T G.718 Annex B and G.729.1 Annex E.
  • the DCT-IV transformation may be replaced with other short-term time-frequency transformations of the same length and in the field of excitation, such as an FFT (for " Fast Fourier Transform "in English ) or DCT-II ( Discrete Cosine Transform - Type II).
  • FFT Fast Fourier Transform
  • DCT-II Discrete Cosine Transform - Type II
  • MDCT Modified Discrete Cosine Tranform
  • This approach preserves the original spectrum in this band and avoids introducing distortions in the 5000-6000 Hz band during the addition of HF synthesis with BF synthesis - particularly the signal phase (implicitly represented in the DCT-IV domain) in this band is preserved.
  • the band 6000-8000 Hz of U HB 1 ( k ) is here defined by copying the 4000-6000 Hz band of U ( k ) since the value of start_band is preferably fixed at 160.
  • the value of start_band can be made adaptive around the value of 160, without changing the nature of the invention.
  • the details of the adaptation of the value start_band are not described here because they go beyond the scope of the invention without changing the scope.
  • the noise in the 6000-8000 Hz band
  • U HBN k - 1 + 13849 k 240 , ⁇ 319 with the convention that U HBN (239) in the current frame corresponds to the value U HBN (319) of the previous frame.
  • this noise generation can be replaced by other methods.
  • the combination block 503 can be realized in different ways.
  • G HBN is a normalization factor for equalizing the energy level between the two signals
  • the coefficient ⁇ (between 0 and 1) is adjusted according to parameters estimated from the decoded low band and the coefficient ⁇ (between 0 and 1) depends on ⁇ .
  • N ( k 1 , k 2 ) is the set of indices k for which the index coefficient k is classified as being associated with noise.
  • This set can be obtained for example by detecting the local peaks in U '( k ) verifying
  • and in whereas these lines are not associated with noise, ie (by applying the negation of the previous condition): NOT at b at ⁇ k ⁇ b
  • the calculation of ⁇ may be replaced by other methods.
  • the linear regression could for example be estimated in a supervised manner by estimating the factor ⁇ by giving itself the original high band in a learning base. Note that the calculation mode of ⁇ does not limit the nature of the invention.
  • ⁇ 1 - ⁇ 2 to preserve the energy of the extended signal after mixing.
  • block 503 realizes the equivalent of block 101 of the figure 1 to normalize the white noise according to an excitation which is on the other hand here in the frequency domain, already extended to the rate of 16 kHz; in addition, the mix is limited to the band 6000-8000 Hz.
  • an embodiment of block 503 may be considered, where the spectra, U HB 1 ( k ) or G HBN U HBN ( k ), are selected (switched) adaptively, which amounts to allowing only the values 0 or 1 for ⁇ ; this approach amounts to classifying the type of excitation to be generated in the 6000-8000 Hz band
  • the block 504 optionally carries out a dual operation of application of bandpass filter frequency response and deemphasis filtering (or deemphasis) in the frequency domain.
  • G deemph (k) 1
  • ⁇ k can be adjusted (for example for even frequencies).
  • the HF synthesis is not de-emphasized.
  • the high frequency signal is on the contrary de-emphasized so as to bring it back to a domain coherent with the low frequency signal (0-6.4 kHz) coming out of block 305. This is important for the estimation and the subsequent adjustment of the energy of the HF synthesis.
  • the de-emphasis can be performed in an equivalent way in the time domain after inverse DCT.
  • Such an embodiment is implemented at the figure 7 described later.
  • This filtering is performed in the frequency domain, and its frequency response is illustrated in FIG. figure 6 .
  • the 3 dB cut-off frequencies are 6000 Hz for the lower part and for the upper part approximately 6900, 7300, 7600 Hz at 6.6, 8.86 and the bit rates higher than 8.85 kbit / s (respectively).
  • G hp ( k ) , k 0, ⁇ , 55, is given for example in Table 1 below.
  • Table 1 ⁇ / b> K g hp ( k ) K g hp ( k ) K g hp ( k ) k g hp ( k ) 0 0.001622428 14 0.114057967 28 0.403990611 42 0.776551214 1 0.004717458 15 0.128865425 29 0.430149896 43 0.800503267 2 0.008410494 16 0.144662643 30 0.456722014 44 0.823611104 3 0.012747280 17 0.161445005 31 0.483628433 45 0.845788355 4 0.017772424 18 0.179202219 32 0.510787115 46 0.866951597 5 0.023528982 19 0.197918220 33 0.538112915 47 0.887020781 6 0.030058032 20 0.217571104 34 0.56
  • G hp ( k ) may be modified while keeping a gradual attenuation.
  • the variable bandwidth low-pass filtering, G lp ( k ) may be adjusted with different values or frequency support, without changing the principle of this filtering step.
  • bandpass filtering example illustrated in FIG. figure 6 can be adapted by defining a single filtering step combining the high-pass and low-pass filtering.
  • the bandpass filtering may be performed in an equivalent manner in the time domain (as in block 112 of the present invention). figure 1 ) with different filter coefficients according to the flow rate, after a reverse DCT step. Such an embodiment is implemented at the figure 7 described later. However, it will be noted that it is advantageous to carry out this step directly in the frequency domain because the filtering is carried out in the field of LPC excitation and therefore the problems of circular convolution and edge effects are very limited in this field. .
  • the realization of the block 506 differs from that of the block 101 of the figure 1 because the energy at the current frame is taken into account in addition to that of the sub-frame. This makes it possible to have the ratio of the energy of each sub-frame with respect to the energy of the frame. Energy ratios (or relative energies) are compared rather than the absolute energies between low band and high band.
  • this scaling step makes it possible to keep in the high band the energy ratio between the subframe and the frame in the same way as in the low band.
  • Blocks 508 and 509 are useful for adjusting the level of the LPC synthesis filter (block 510), here depending on the tilt of the signal. Other methods of calculating the gain g HB 2 ( m ) are possible without changing the nature of the invention.
  • this filtering can be done in the same way as that described for block 111 of the figure 1 of the AMR-WB decoder, however the order of the filter goes to 20 at the rate of 6.6, which does not significantly change the quality of the synthesized signal.
  • the coding of the low band (0-6.4 kHz) may be replaced by a CELP coder other than that used in AMR-WB, for example the CELP coder in G.718 to 8. kbit / s.
  • a CELP coder other than that used in AMR-WB, for example the CELP coder in G.718 to 8. kbit / s.
  • other encoders in wide band or operating at frequencies higher than 16 kHz in which the coding of the low band operates at an internal frequency at 12.8 kHz could be used.
  • the invention can be obviously adapted to other sampling frequencies than 12.8 kHz, when a low frequency encoder operates at a sampling frequency lower than that of the original or reconstructed signal.
  • the low band decoding does not use a linear prediction, it does not have an excitation signal to be extended, in this case it will be possible to carry out an LPC analysis of the reconstructed signal in the current frame and calculate an LPC excitation. so as to be able to apply the invention.
  • the excitation ( u ( n )) is resampled, for example by linear interpolation or "spline" cubic, from 12.8 to 16 kHz before transformation (for example DCT-IV)
  • This variant has the defect of being more complex, because the transform (DCT-IV) of the excitation is then calculated over a greater length and the resampling is not carried out in the field of the transformed.
  • the scaling step (E407 to figure 4 ) is performed by blocks 508 and 509 identical to the figure 5 .
  • the filtering step (E404 of the figure 4 ) is performed by the filter module (block 510) identical to that described with reference to the figure 5 .
  • the low band excitation u (n) and the LPC filter 1 / ⁇ (z) estimated by frame by LPC analysis of a low-band signal whose band is to be extended.
  • the low band excitation signal is then extracted by analyzing the audio signal.
  • the low band audio signal is resampled before the excitation extraction step, so that the excitation extracted from the audio signal (by linear prediction) is already resolved. sampled.
  • the invention illustrated in figure 5 or the second embodiment not forming part of the invention and described in figure 7 , applies in this case to a low band which is not decoded but analyzed.
  • the figure 8 represents an exemplary hardware embodiment of a band extension device 800 according to the invention. This may be an integral part of an audio-frequency signal decoder or equipment receiving decoded or non-decoded audio signals.
  • This type of device comprises a PROC processor cooperating with a memory block BM having a memory storage and / or work MEM.
  • a PROC processor cooperating with a memory block BM having a memory storage and / or work MEM.
  • Such a device comprises an input module E able to receive a decoded or extracted excitation audio signal in a first so-called low band frequency band ( u ( n ) or U ( k )) and the parameters of a filter of linear prediction synthesis ( ⁇ ( z )). It comprises an output module S adapted to transmit the synthesized high frequency signal (HF_syn) for example to a delay application module such as the block 310 of the figure 3 or a resampling module such as module 311.
  • a delay application module such as the block 310 of the figure 3
  • a resampling module such as module 311.
  • the memory block may advantageously comprise a computer program comprising code instructions for implementing the steps of the band extension method in the sense of the invention, when these instructions are executed by the processor PROC, and in particular the steps for obtaining an extended signal in at least a second frequency band greater than the first frequency band from an oversampled and extended excitation signal in at least a second frequency band, scaling of the signal extended by a subframe defined gain based on a frame and subframe energy ratio and filtering said extended signal scaled by a linear prediction filter whose coefficients are derived from the coefficients of the low band filter.
  • the description of the figure 4 takes the steps of an algorithm of such a computer program.
  • the computer program can also be stored on a memory medium readable by a reader of the device or downloadable in the memory space thereof.
  • the memory MEM generally records all the data necessary for the implementation of the method.
  • the device thus described can also comprise the low band decoding functions and other processing functions described for example in figure 3 in addition to the band extension functions according to the invention.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Quality & Reliability (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Claims (11)

  1. Verfahren zur Frequenzbanderweiterung eines Audiosignals bei einem Decodierungs- oder Verbesserungsprozess, umfassend einen Schritt der Decodierung oder Extraktion, in einem ersten Frequenzband, Tiefband genannt, eines Anregungssignals und der Koeffizienten eines linearen Vorhersagefilters, wobei das Verfahren dadurch gekennzeichnet ist, dass es die folgenden Schritte umfasst:
    - Erhalten eines erweiterten Signals (UHB2(k), E403) in mindestens einem zweiten Frequenzband, das über dem ersten Frequenzband liegt, ausgehend von dem in dem mindestens einen zweiten Frequenzband überabgetasteten und erweiterten Anregungssignal (UHB1(k), E401);
    - Skalieren (E406) des erweiterten Signals durch einen Verstärkungsfaktor, der je Teilrahmen in Abhängigkeit von einem Verhältnis zwischen dem Verhältnis zwischen der Energie je Teilrahmen und der Energie je Rahmen des Anregungssignals des Tiefbandes und dem Verhältnis zwischen der Energie je Teilrahmen und der Energie je Rahmen des erweiterten Signals definiert wird;
    - Filterung (E404) des skalierten erweiterten Signals durch ein lineares Prädiktionsfilter, dessen Koeffizienten von den Koeffizienten des Filters des Tiefbandes abgeleitet sind.
  2. Verfahren nach Anspruch 1, dadurch gekennzeichnet, dass es ferner vor dem Schritt des Skalierens einen Schritt der adaptiven Bandpassfilterung (E405) des erweiterten Signals in Abhängigkeit von der Decodierungsleistung des aktuellen Rahmens umfasst.
  3. Verfahren nach Anspruch 1, dadurch gekennzeichnet, dass es einen Schritt der Zeit-Frequenz-Transformation des Anregungssignals, wobei der Schritt des Erhaltens eines erweiterten Signals in dem Frequenzbereich stattfindet, und einen Schritt der inversen Zeit-Frequenz-Transformation des erweiterten Signals vor den Schritten des Skalierens und der Filterung umfasst.
  4. Verfahren nach Anspruch 3, dadurch gekennzeichnet, dass der Schritt des Generierens eines überabgetasteten und erweiterten Anregungssignals nach der folgenden Gleichung durchgeführt wird: U HB 1 k = { 0 k = 0 , … , 199 U k k = 200 , … , 239 U k + start _ band − 240 k = 240 , … , 319
    Figure imgb0040
     wobei k der Index des Samples, UHB1(k) das Spektrum des erweiterten Anregungssignals, U(k) das Spektrum des nach dem Transformationsschritt erhaltenen Anregungssignals und start_band eine vorgegebene Variable ist.
  5. Verfahren nach einem der Ansprüche 1 bis 4, dadurch gekennzeichnet, dass es einen Schritt der Deakzentuierungsfilterung des erweiterten Signals mindestens in dem zweiten Frequenzband umfasst.
  6. Verfahren nach Anspruch 1, dadurch gekennzeichnet, dass es ferner einen Schritt des Erzeugens (E402) eines Rauschsignals mindestens in dem zweiten Frequenzband umfasst, wobei das erweiterte Signal (UHB2(k)) durch Kombinieren (E403) des erweiterten Anregungssignals und des Rauschsignals erhalten wird.
  7. Verfahren nach Anspruch 6, dadurch gekennzeichnet, dass der Schritt des Kombinierens durch adaptives additives Mischen mit einem Entzerrungsverstärkungsfaktor zwischen dem erweiterten Anregungssignal und dem Rauschsignal durchgeführt wird.
  8. Vorrichtung zur Frequenzbanderweiterung eines Audiosignals, umfassend eine Stufe zur Docodierung oder Extraktion, in einem ersten Frequenzband, Tiefband genannt, eines Anregungssignals und der Koeffizienten eines linearen Prädiktionsfilters, wobei die Vorrichtung dadurch gekennzeichnet ist, dass sie umfasst:
    - ein Modul zum Erhalten eines erweiterten Signals (UHB2(k), 503) in mindestens einem zweiten Frequenzband, das über dem ersten Frequenzband liegt, ausgehend von dem in dem mindestens einen zweiten Frequenzband überabgetasteten und erweiterten Anregungssignal (UHEB1(k)) ;
    - Skalieren (507) des erweiterten Signals durch einen Verstärkungsfaktor, der je Teilrahmen in Abhängigkeit von einem Verhältnis zwischen dem Verhältnis zwischen der Energie je Teilrahmen und der Energie je Rahmen des Anregungssignals des Tiefbandes und dem Verhältnis zwischen der Energie je Teilrahmen und der Energie je Rahmen des erweiterten Signals definiert wird;
    - ein Modul zur Filterung (510) des skalierten erweiterten Signals durch ein lineares Prädiktionsfilter, dessen Koeffizienten von den Koeffizienten des Filters des Tiefbandes abgeleitet sind.
  9. Audiosignaldecoder, dadurch gekennzeichnet, dass er eine Vorrichtung zur Frequenzbanderweiterung nach Anspruch 8 umfasst.
  10. Computerprogramm mit Codeanweisungen zum Umsetzen der Schritte des Frequenzbanderweiterungsverfahrens nach einem der Ansprüche 1 bis 7, wenn diese Anweisungen von einem Prozessor ausgeführt werden.
  11. Speichermedium, das von einer Vorrichtung zur Frequenzbanderweiterung gelesen werden kann, auf dem ein Computerprogramm gespeichert ist, das Codeanweisungen zum Umsetzen der Schritte des Verfahrens zur Frequenzbanderweiterung nach einem der Ansprüche 1 bis 7 umfasst.
EP14742262.0A 2013-06-25 2014-06-24 Verbesserte frequenzbanderweiterung in einem audiosignaldecodierer Active EP3014611B1 (de)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
FR1356100A FR3007563A1 (fr) 2013-06-25 2013-06-25 Extension amelioree de bande de frequence dans un decodeur de signaux audiofrequences
PCT/FR2014/051563 WO2014207362A1 (fr) 2013-06-25 2014-06-24 Extension améliorée de bande de fréquence dans un décodeur de signaux audiofréquences

Publications (2)

Publication Number Publication Date
EP3014611A1 EP3014611A1 (de) 2016-05-04
EP3014611B1 true EP3014611B1 (de) 2019-03-13

Family

ID=49151174

Family Applications (1)

Application Number Title Priority Date Filing Date
EP14742262.0A Active EP3014611B1 (de) 2013-06-25 2014-06-24 Verbesserte frequenzbanderweiterung in einem audiosignaldecodierer

Country Status (6)

Country Link
US (1) US9911432B2 (de)
EP (1) EP3014611B1 (de)
CN (1) CN105324814B (de)
ES (1) ES2724576T3 (de)
FR (1) FR3007563A1 (de)
WO (1) WO2014207362A1 (de)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA3045686C (en) 2010-04-09 2020-07-14 Dolby International Ab Audio upmixer operable in prediction or non-prediction mode
EP3182411A1 (de) 2015-12-14 2017-06-21 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und verfahren zur verarbeitung eines codierten audiosignals
US10249307B2 (en) 2016-06-27 2019-04-02 Qualcomm Incorporated Audio decoding using intermediate sampling rate
EP3382702A1 (de) * 2017-03-31 2018-10-03 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und verfahren zur bestimmung einer im voraus bestimmten eigenschaft bezüglich der künstlichen bandbreitenbeschränkungsverarbeitung eines audiosignals
US10825467B2 (en) * 2017-04-21 2020-11-03 Qualcomm Incorporated Non-harmonic speech detection and bandwidth extension in a multi-source environment
US20190051286A1 (en) * 2017-08-14 2019-02-14 Microsoft Technology Licensing, Llc Normalization of high band signals in network telephony communications
CN107886966A (zh) * 2017-10-30 2018-04-06 捷开通讯(深圳)有限公司 终端及其优化语音命令的方法、存储装置
EP3553777B1 (de) * 2018-04-09 2022-07-20 Dolby Laboratories Licensing Corporation Verdecken von paketverlusten mit niedriger komplexität für transcodierte audiosignale
CN110660409A (zh) * 2018-06-29 2020-01-07 华为技术有限公司 一种扩频的方法及装置
CN110556122B (zh) * 2019-09-18 2024-01-19 腾讯科技（深圳）有限公司 频带扩展方法、装置、电子设备及计算机可读存储介质

Family Cites Families (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE10041512B4 (de) * 2000-08-24 2005-05-04 Infineon Technologies Ag Verfahren und Vorrichtung zur künstlichen Erweiterung der Bandbreite von Sprachsignalen
US6889182B2 (en) * 2001-01-12 2005-05-03 Telefonaktiebolaget L M Ericsson (Publ) Speech bandwidth extension
SE522553C2 (sv) * 2001-04-23 2004-02-17 Ericsson Telefon Ab L M Bandbreddsutsträckning av akustiska signaler
US6988066B2 (en) * 2001-10-04 2006-01-17 At&T Corp. Method of bandwidth extension for narrow-band speech
ATE331280T1 (de) * 2001-11-23 2006-07-15 Koninkl Philips Electronics Nv Bandbreitenvergrösserung für audiosignale
AU2003260958A1 (en) * 2002-09-19 2004-04-08 Matsushita Electric Industrial Co., Ltd. Audio decoding apparatus and method
US20050004793A1 (en) * 2003-07-03 2005-01-06 Pasi Ojala Signal adaptation for higher band coding in a codec utilizing band split coding
KR100707174B1 (ko) * 2004-12-31 2007-04-13 삼성전자주식회사 광대역 음성 부호화 및 복호화 시스템에서 고대역 음성부호화 및 복호화 장치와 그 방법
CA2603246C (en) * 2005-04-01 2012-07-17 Qualcomm Incorporated Systems, methods, and apparatus for anti-sparseness filtering
KR101171098B1 (ko) * 2005-07-22 2012-08-20 삼성전자주식회사 혼합 구조의 스케일러블 음성 부호화 방법 및 장치
US9454974B2 (en) * 2006-07-31 2016-09-27 Qualcomm Incorporated Systems, methods, and apparatus for gain factor limiting
US20090201983A1 (en) * 2008-02-07 2009-08-13 Motorola, Inc. Method and apparatus for estimating high-band energy in a bandwidth extension system
WO2010028297A1 (en) * 2008-09-06 2010-03-11 GH Innovation, Inc. Selective bandwidth extension
WO2010036061A2 (en) * 2008-09-25 2010-04-01 Lg Electronics Inc. An apparatus for processing an audio signal and method thereof
US8463599B2 (en) * 2009-02-04 2013-06-11 Motorola Mobility Llc Bandwidth extension method and apparatus for a modified discrete cosine transform audio coder
FR2947945A1 (fr) * 2009-07-07 2011-01-14 France Telecom Allocation de bits dans un codage/decodage d'amelioration d'un codage/decodage hierarchique de signaux audionumeriques
CA2780971A1 (en) * 2009-11-19 2011-05-26 Telefonaktiebolaget L M Ericsson (Publ) Improved excitation signal bandwidth extension
US8600737B2 (en) * 2010-06-01 2013-12-03 Qualcomm Incorporated Systems, methods, apparatus, and computer program products for wideband speech coding
MX2013009295A (es) * 2011-02-15 2013-10-08 Voiceage Corp Dispositivo y método para cuantificar ganancias de contribuciones adaptativas y fijas de una excitación en un codec celp.
US20140019125A1 (en) * 2011-03-31 2014-01-16 Nokia Corporation Low band bandwidth extended
WO2013066238A2 (en) * 2011-11-02 2013-05-10 Telefonaktiebolaget L M Ericsson (Publ) Generation of a high band extension of a bandwidth extended audio signal

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
CN105324814B (zh) 2019-06-04
EP3014611A1 (de) 2016-05-04
US20160133273A1 (en) 2016-05-12
WO2014207362A1 (fr) 2014-12-31
CN105324814A (zh) 2016-02-10
FR3007563A1 (fr) 2014-12-26
US9911432B2 (en) 2018-03-06
ES2724576T3 (es) 2019-09-12

Similar Documents

Publication Publication Date Title
EP3014611B1 (de) Verbesserte frequenzbanderweiterung in einem audiosignaldecodierer
EP3020043B1 (de) Optimierter skalenfaktor für frequenzbanderweiterung bei einem audiofrequenzsignaldecodierer
EP3103116B1 (de) Verbesserte frequenzbanderweiterung in einem audiosignaldecodierer
RU2386179C2 (ru) Способ и устройство для кодирования речевых сигналов с расщеплением полосы
WO2014009657A1 (fr) Traitement d'amelioration de la qualite des signaux audiofrequences

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20160122

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAX Request for extension of the european patent (deleted)
RIN1 Information on inventor provided before grant (corrected)

Inventor name: RAGOT, STEPHANE

Inventor name: KANIEWSKA, MAGDALENA

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20181106

RIN1 Information on inventor provided before grant (corrected)

Inventor name: KANIEWSKA, MAGDALENA

Inventor name: RAGOT, STEPHANE

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

Ref country code: AT

Ref legal event code: REF

Ref document number: 1108804

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190315

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: FRENCH

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602014042825

Country of ref document: DE

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20190313

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190313

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190313

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190313

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190613

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190613

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190313

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190313

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190313

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190614

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190313

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2724576

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20190912

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1108804

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190313

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190313

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190713

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190313

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190313

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190313

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190313

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190313

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190313

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602014042825

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190713

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190313

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190313

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190313

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

26N No opposition filed

Effective date: 20191216

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190313

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20190630

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190313

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190624

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190630

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190630

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190624

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190630

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190313

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190313

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20140624

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190313

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20230523

Year of fee payment: 10

Ref country code: FR

Payment date: 20230524

Year of fee payment: 10

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20230703

Year of fee payment: 10

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20240521

Year of fee payment: 11

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240521

Year of fee payment: 11