EP3192073A1 - Discrimination and attenuation of pre-echoes in a digital audio signal - Google Patents
Discrimination and attenuation of pre-echoes in a digital audio signalInfo
- Publication number
- EP3192073A1 EP3192073A1 EP15771686.1A EP15771686A EP3192073A1 EP 3192073 A1 EP3192073 A1 EP 3192073A1 EP 15771686 A EP15771686 A EP 15771686A EP 3192073 A1 EP3192073 A1 EP 3192073A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- sub
- echo
- block
- attack
- signal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000005236 sound signal Effects 0.000 title claims abstract description 24
- 238000002592 echocardiography Methods 0.000 title description 6
- 238000000034 method Methods 0.000 claims abstract description 47
- 230000007704 transition Effects 0.000 claims abstract description 22
- 230000002401 inhibitory effect Effects 0.000 claims abstract description 8
- 238000012545 processing Methods 0.000 claims description 36
- 238000001514 detection method Methods 0.000 claims description 20
- 238000004364 calculation method Methods 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 6
- 230000005764 inhibitory process Effects 0.000 claims description 4
- 230000008569 process Effects 0.000 abstract description 3
- 230000006870 function Effects 0.000 description 18
- 238000009499 grossing Methods 0.000 description 16
- 230000015654 memory Effects 0.000 description 15
- 238000012795 verification Methods 0.000 description 14
- 230000002238 attenuated effect Effects 0.000 description 13
- 238000001914 filtration Methods 0.000 description 11
- 238000004458 analytical method Methods 0.000 description 8
- 230000009467 reduction Effects 0.000 description 7
- 238000003786 synthesis reaction Methods 0.000 description 7
- 238000000354 decomposition reaction Methods 0.000 description 6
- 230000007423 decrease Effects 0.000 description 6
- 230000015572 biosynthetic process Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 238000013139 quantization Methods 0.000 description 4
- 238000005070 sampling Methods 0.000 description 4
- 230000002123 temporal effect Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 230000003247 decreasing effect Effects 0.000 description 2
- 238000012417 linear regression Methods 0.000 description 2
- 230000000873 masking effect Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000009527 percussion Methods 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 230000002829 reductive effect Effects 0.000 description 2
- 230000003595 spectral effect Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000001052 transient effect Effects 0.000 description 2
- 238000010162 Tukey test Methods 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000010420 art technique Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000012885 constant function Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000000116 mitigating effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 230000003936 working memory Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
- G10L25/21—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters the extracted parameters being power information
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
- G10L19/0204—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using subband decomposition
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
- G10L19/26—Pre-filtering or post-filtering
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
Definitions
- the invention relates to a method and a device for discriminating and processing pre-echo attenuation when decoding a digital audio signal.
- compression processes for the transmission of digital audio signals over telecommunications networks, whether for example fixed or mobile networks, or for the storage of signals, compression processes (or source coding) using coding systems which are generally of the linear coding type by linear prediction or by transform frequency coding.
- the method and the device which are the subject of the invention, thus have as their field of application the compression of sound signals, in particular frequency-coded digital audio signals.
- FIG. 1 represents by way of illustration, a schematic diagram of the coding and decoding of a digital audio signal by transform including an addition / overlap synthesis analysis according to the prior art.
- Certain musical sequences such as percussion and certain segments of speech like the plosives (/ k /, lit, ...), are characterized by extremely sudden attacks which result in very fast transitions and a very strong variation of the dynamic signal within a few samples.
- An example of transition is given in Figure 1 from sample 410.
- the input signal is cut into blocks of samples of length L whose boundaries are represented in FIG. 1 by dotted vertical lines.
- the input signal is denoted x (ri), where n is the index of the sample.
- L 160 samples.
- the division in blocks, also called frames, operated by the transform coding is totally independent of the sound signal and the transitions can therefore appear at any point in the analysis window.
- the reconstructed signal is tainted by "noise" (or distortion) generated by the quantization (Q) - inverse quantization (Q 1 ) operation.
- This coding noise is temporally distributed relatively uniform throughout the temporal support of the transformed block, that is to say over the entire length of the window length 2L samples (with overlap of L samples).
- the energy of the coding noise is generally proportional to the energy of the block and is a function of the coding / decoding rate.
- an attacking block such as block 320-480 of FIG. 1
- the signal energy is high, so the noise is also high.
- the level of the coding noise is typically lower than that of the signal for the high energy segments that immediately follow the transition, but the level is higher than that of the signal for the lower energy segments, especially on the part preceding the transition (samples 160 - 410 of Figure 1).
- the signal-to-noise ratio is negative and the resulting degradation can appear very troublesome to listen.
- Pre-echo is the coding noise prior to the transition and post-echo the noise after the transition.
- the human ear also performs a post-masking of a longer duration, from 5 to 60 milliseconds, during the passage of high energy sequences to low energy sequences.
- the rate or level of inconvenience acceptable for post-echoes is therefore greater than for pre-echoes.
- MPEG AAC Advanced Audio Coding
- MPEG AAC Advanced Audio Coding
- Transform encoders used for conversational applications such as ITU-T G.722.1, G.722.1C or G.719, often use a 20 ms frame length and a 40 ms window at 16, 32 or 48 kHz (respectively). It should be noted that the ITU-T G.719 encoder incorporates a window switch mechanism with transient detection, however the pre-echo is not completely reduced at low bit rate (typically at 32 kbit / s).
- Window switching has been mentioned previously; it requires transmitting auxiliary information to identify the type of windows used in the current frame.
- Another solution is to apply adaptive filtering. In the area preceding the attack, the reconstructed signal is seen as the sum of the original signal and the quantization noise.
- the aforementioned filtering process does not allow to find the original signal, but provides a strong reduction of pre-echo. However, it requires transmitting the additional parameters to the decoder.
- Attenuation factors are determined by sub-block, in the sub-blocks of weak energy preceding a sub-block in which a transition or attack has been detected.
- the attenuation factor g (k) in the k-th sub-block is calculated for example according to the ratio R (k) between the energy of the sub-block of higher energy and the energy of the k-th under -block in question:
- the factor g (k) is set to an attenuation-inhibiting attenuation value, i.e. 1. Otherwise, the attenuation factor is between 0 and 1.
- the frame that precedes the pre-echo frame has a homogeneous energy that corresponds to the energy of a low energy segment (typically a background noise).
- a background noise typically a background noise
- ⁇ im g (k) serves as a lower limit in the final calculation of the attenuation factor of the sub-block, and is therefore used as follows:
- g (k) max (g (k), lim g (k))
- the attenuation factors (or gains) g (k) determined by sub-blocks can then be smoothed by a sample-by-sample applied smoothing function to avoid abrupt changes in the attenuation factor at the block boundaries.
- L ' represents the length of a sub-block.
- u i 0
- u 5
- FIGS. 2 and 3 illustrate the implementation of the attenuation method as described in the patent application of the state of the art, cited above, and summarized above.
- the signal is sampled at 32 kHz
- a frame of an original signal sampled at 32 kHz is shown.
- An attack (or transition) in the signal is located in the sub-block beginning at the index 320.
- This signal has been coded by a low rate (24 kbit / s) MDCT type transform coder.
- Part c) shows the evolution of the pre-echo attenuation factor (solid line) obtained by the method described in the aforementioned prior art patent application.
- the dotted line represents the factor before smoothing. Note here that the position of the attack is estimated around the sample 380 (in the block delimited by the samples 320 and 400).
- Part d) illustrates the result of decoding after application of pre-echo processing (multiplication of signal b) with signal c)).
- pre-echo has been attenuated.
- Figure 2 also shows that the smoothed factor does not go back to 1 at the time of the attack, which implies a decrease in the amplitude of the attack. The noticeable impact of this decrease is very small but can nevertheless be avoided.
- FIG. 3 illustrates the same example as FIG. 2, in which, before smoothing, the attenuation factor value is forced to 1 for the few samples of the sub-block preceding the sub-block where the attack is located. Part (c) of Figure 3 gives an example of such a correction.
- the value of factor 1 has been assigned to the last 16 samples of the sub-block preceding the attack, starting from the index 364.
- the smoothing function progressively increases the factor to have a value close to 1 at the moment. of the attack.
- the amplitude of the attack is then preserved, as illustrated in part d) of FIG. 3, but some pre-echo samples are not attenuated.
- pre-echo attenuation reduction does not reduce the pre-echo to the level of the attack, because of the smoothing gain.
- This pre-echo reduction technique is, however, perfectible for certain types of signals such as modern music signals, for example. Indeed, in some cases, a false pre-echo detection can take place.
- FIG. 4 illustrates an example of such an original signal, which is thus uncoded without pre-echo. This is a beat of an electronic / synthetic percussion instrument. It can be observed that before the net attack to the index 1600 there is a synthetic noise which starts towards the index 1250. This synthetic noise which is therefore part of the signal would be detected as a pre-echo by detection algorithm pre-echo method described above, assuming perfect signal coding / decoding. The pre-echo attenuation processing would therefore suppress this component of the signal. This would distort the decoded signal (when the coding / decoding is perfect), which is undesirable.
- the present invention improves the state of the art.
- the present invention relates to a method of pre-echo discrimination and attenuation in a digital audio signal generated from a transform coding, in which, for a current frame decomposed into sub-blocks, the Low energy low sub-blocks preceding a sub-block in which a transition or attack is detected determine a pre-echo area in which a pre-echo attenuation processing is performed.
- the method is such that, in the case where an attack is detected from the third sub-block of the current frame, it comprises the following steps:
- the energy director coefficient calculated for the sub-blocks preceding the position of the attack makes it possible to check the tendency of increase of the energy of the signal in the pre-echo zone. This makes reliable pre-echo detection by avoiding false pre-echo detection. Indeed, by looking at Figure 1 we can see that the pre-echo has a typical characteristic: its energy has a growing tendency in approaching the attack origin of the pre-echo. The shape of the addition-overlay weighting windows explain this. Even if the pre-echo has a nearly constant energy before overlap, the signals at the input of the add-over module are multiplied by weighting windows whose weight decreases towards the past. In the case of the example signal of FIG. 4, the energy of the signal before the attack is approximately constant, which makes it possible to differentiate it from a pre-echo. Thus, verification of increasing signal energy in the pre-echo area increases the reliability of the pre-echo detection.
- the method further comprises a step of decomposing the digital audio signal into at least two sub-signals according to a frequency criterion and in that the comparison calculation steps are performed for at least one of the subsignals.
- the energy of two sub-blocks is used in the pre-echo zone to calculate a directional coefficient and compare it to a threshold. With only two points, only the verification for the high-frequency sub-signal in the case of two sub-signal decomposition is sufficient to detect a false pre-echo detection.
- the method further comprises a step of decomposing the digital audio signal into at least two sub-signals as a function of a frequency criterion and in that the calculation and comparison steps are performed for each of the sub-signals, the inhibition of the pre-echo attenuation processing in the pre-echo zone of all the sub-signals performing when a calculated master coefficient is below the predefined threshold for at least one sub-signal.
- the division into sub-signals thus makes it possible to carry out a pre-echo attenuation independently and adapted in the sub-signals.
- the detection reliability of the pre-echo zone is enhanced for each of the sub-signals by checking the value of the respective coefficient coefficients.
- a different threshold is defined by sub-signal.
- the steering coefficient is calculated using a least squares estimation method.
- This calculation method is of low complexity.
- the steering coefficient is normalized.
- a direction coefficient calculated for the previous frame is used for the comparison step.
- the present invention also relates to a device for discriminating and attenuating pre-echo in a digital audio signal generated from a transform coding, comprising a transition detection or attack module, a zone discrimination module. pre-echo and a pre-echo attenuation processing module, a pre-echo attenuation processing being performed for a current sub-block decomposed frame, in the low energy sub-blocks preceding a sub-block wherein a transition or attack is detected determining a pre-echo area.
- the device is such that, in the case where an attack is detected from the third sub-block of the current frame, it further comprises:
- a calculation module calculating an energies directing coefficient for at least two sub-blocks of the current frame preceding the sub-block in which an attack is detected;
- a comparator able to perform a comparison of the steering coefficient at a predefined threshold
- a discrimination module capable of inhibiting the pre-echo attenuation processing in the pre-echo zone in the case where the calculated directing coefficient is lower than the predefined threshold.
- the invention relates to a decoder of a digital audio signal comprising a device as described above.
- the invention also relates to a computer program comprising code instructions for implementing the steps of the method as described above, when these instructions are executed by a processor.
- the invention relates to a storage medium, readable by a processor, integrated or not to the processing device, optionally removable, storing a computer program implementing a method of treatment as described above.
- FIG. 1 previously described illustrates a state-of-the-art transform coding-decoding system
- FIG. 3 illustrates another example of a digital audio signal for which an attenuation method according to the state of the art is carried out
- FIG. 4 previously described illustrates an example of a signal for which the state of the art technique would erroneously detect a pre-echo
- FIG. 5 illustrates an embodiment of a method and a device for discriminating and pre-echo attenuation processing included in a decoder according to the invention
- FIG. 6 illustrates an example of analysis windows and low-delay synthesis windows for coding and transform decoding capable of creating the pre-echo phenomenon
- FIG. 7 illustrates an exemplary digital audio signal for which the pre-echo attenuation method according to one embodiment of the invention is implemented
- FIG. 8 illustrates a hardware example of discrimination and attenuation processing device according to the invention.
- a pre-echo attenuation discrimination and processing device 600 is described.
- the attenuation processing device 600 as described below is included in a decoder comprising an inverse quantization module 610 (Q 1 ) receiving a signal S, a module 620 of inverse transform (MDCT 1 ), a module 630 of reconstruction of the addition / overlap signal (add / rec) as described with reference to FIG. 1 and delivering a reconstructed signal x rec (n) to the attenuation discrimination and processing device according to the invention.
- MDCT inverse quantization module
- a processed signal Sa is provided in which a pre-echo attenuation has been performed.
- the device 600 implements a discrimination and pre-echo attenuation processing method in the decoded signal x rec (n).
- the discrimination and attenuation processing method includes a step of detecting (E601) attacks that may generate a pre-echo, in the decoded signal x rec (n).
- the device 600 comprises a detection module 601 able to implement a step of detecting (E601) the position of an attack in a decoded audio signal.
- An onset is a rapid transition and a sudden change in the dynamics (or amplitude) of the signal.
- This type of signal may be referred to by the more general term "transient”.
- transient we will use only the terms of attack or transition to designate also transients.
- the size of these sub-blocks is therefore identical but the invention remains valid and easily generalizable when the sub-blocks have a variable size. This can be the case for example when the length of the frame L is not divisible by the number of sub-blocks K or if the frame length is variable.
- Special low-delay analysis-synthesis windows similar to those described in ITU-T G.718 are used for the analysis part and for the synthesis part of the MDCT transformation.
- An example of such windows is illustrated with reference to Figure 6.
- the delay generated by the transformation is only 280 samples unlike the delay of 640 samples in the case of use of conventional sinusoidal windows.
- the MDCT memory with special low-delay analysis-synthesis windows contains only 140 independent samples (not folded with the current frame) unlike the 320 samples in the case of using conventional sinusoidal windows.
- the MDCT x MDCT (n) memory is used which gives a time-folding version of the future signal.
- Figure 1 shows that the pre-echo influences the frame that precedes the frame where the attack is located, and it is desirable to detect an attack in the future frame which is partly contained in the MDCT memory.
- the current frame and the MDCT memory can be seen as concatenated signals forming a signal cut into ( ⁇ + ⁇ ') consecutive sub-blocks.
- the energy in the k-th sub-block is defined as:
- the average energy of the sub-blocks in the current frame is thus obtained as:
- the average energy of the sub-blocks in the second part of the current frame is also defined as (assuming that K is an even number):
- R (k) "- ° ⁇ ⁇ + ⁇ ⁇ 1 - exceeds a predefined threshold, in one of the sub-blocks considered.
- the device 600 also comprises a pre-echo zone discrimination module 602 implementing a step of determining (E602) a pre-echo zone (ZPE) preceding the detected driving position.
- Pre-echo zone is here called the zone covering the samples before the estimated position of the attack which are disturbed by the preecho generated by the attack and where attenuation of this pre-echo is desirable.
- the pre-echo zone can be determined on the decoded signal.
- In k) are concatenated in chronological order, first with the time envelope of the decoded signal, then the envelope of the signal of the next frame estimated from the memory of the MDCT transform.
- the pre-echo presence is detected for example if the ratio R (k) exceeds a threshold, typically this threshold is 16.
- the device 600 comprises a calculation module 603 capable of implementing a step of calculating a steering coefficient (or variation trend indicator) of the energies of the sub-blocks preceding the sub-block in which an attack has been detected.
- the value of bi also depends on the magnitude (in absolute value) of the energies; it is indeed homogeneous with energy over time. To be able to better compare the value of bi with a threshold (for example fixed) one can suppress this dependency. For example, the value of bi can be divided by the average value of the energies to obtain the standardized guideline:
- This alternative solution has a higher computational complexity because it requires calculating a square root.
- the steering coefficient is calculated with at most 3 sub-blocks. This makes it possible to limit the maximum complexity of the calculation of the steering coefficient.
- the normalized standard coefficient b ln thus obtained is then compared to the step E604 by a comparator module 604 at a predefined threshold.
- the threshold may be predefined to a fixed value or may be variable depending for example on the classification of the signal according to a speech or music criterion. Typically this threshold is equal to 0 if we only check that the energy does not decrease or equal to 0.2 if we impose a slight increase in energy in the pre-echo zone. If the standardized guideline b in is below this threshold it is concluded that the signal in the pre-echo zone does not correspond to a typical pre-echo and pre-echo attenuation is inhibited in this zone in step E602. Thus, it is avoided that a decoded signal whose original input signal contains a low energy component before an attack is erroneously modified / altered by the pre-echo attenuation module by detecting that component as a pre-echo.
- a pre-echo attenuation is implemented in step E607 by the attenuation module 607 for the discriminated pre-echo area.
- the attenuation factor is for example calculated as in the application FR 08 56248. In the case where the module 604 has detected a false pre-echo detection, the attenuation factor can be forced to 1 thus inhibiting the attenuation or Well the discrimination module 602 does not discriminate this area as a pre-echo area, the attenuation module then not being solicited.
- the device 600 further comprises a signal decomposition module 605, able to perform a step E605 of decomposing the decoded signal into at least two sub-signals according to a predetermined criterion. This method is described in particular in the application FR12 62598 which is recalled here some elements.
- the decoded signal x rec (n) is decomposed in step E605 into two sub-signals as follows:
- the first sub-signal x mc ssl (n) is obtained by low-pass filtering using a FIR filter (finite impulse response filter) with 3 coefficients and a null transfer function phase c (n) z ⁇ l + (l) - 2c (n)) + c (n) z with c (n) a value between 0 and 0.25, where [c (w), l-2c (n), c (n)] are the coefficients of the pass filter low; this filter is implemented with the difference equation:
- a constant value c (n) 0.25 is used. It can be noted that the sub-signal x reC SS i (n) resulting from this filtering, therefore contains rather low-frequency components of the decoded signal.
- the sub-signal x reCtSs2 (n) resulting from this filtering therefore contains rather high-frequency components of the decoded signal.
- the combination of the attenuated sub-signals for obtaining the attenuated signal Sa is made by simply adding the attenuated sub-signals to the step E608 described later.
- a step E606 for calculating pre-echo attenuation factors is implemented in the calculation module 606. This calculation is done separately for the two sub-signals.
- Attenuation factors are obtained by sampling the pre-echo area determined in E602 as a function of the frame in which the attack was detected and the previous frame.
- the attenuation factors are calculated by sub-block. In the method described here, they are additionally calculated separately for each sub signal. For samples preceding the detected attack, the attenuation factors g pre-ss i (n) and
- g.sub.Ss2 ( n ) The calculation of the attenuation factor of a sub-signal (for example g.sub.Ss2 ( n )) can be similar to that described in the patent application FR 08 56248 for the signal decoded as a function of the ratio R (k) (used also for the detection of the attack) between the energy of the sub-block of higher energy and the energy of the k-th sub-block of the decoded signal.
- R (k) used also for the detection of the attack
- the factor is then set to a attenuation value that inhibits the attenuation, that is to say 1. Otherwise, the attenuation factor is between 0 and 1. This initialization can be common for all the sub-signals. .
- the attenuation values are then refined by sub-signal to be able to adjust the optimal sub-signal attenuation level based on the characteristics of the decoded signal.
- the attenuations can be limited according to the average energy of the sub-signal of the previous frame because it is not desirable that after the pre-echo attenuation processing, the signal energy becomes less than the average energy per sub-block of the signal preceding the processing zone (typically that of the previous frame or that of the second half of the previous frame).
- n kL '
- the average energy of the previous frame In ss2 and that of the second half of the previous frame In ss2 'which can be calculated (at the previous frame) are also known by storage as: l Kl
- the limit value of the factor lim ⁇ ss2 (k) can be calculated in order to obtain exactly the same energy as the average energy per sub-block of the segment preceding the sub-block to be processed. This value is of course limited to a maximum of 1 since it is of interest here to the attenuation values. More precisely :
- the pre-echo zone where the attenuation extends from the beginning of the current frame to the beginning of the sub-block in which the attack has been detected, up to the index pos where pos min (. ⁇ arg max (£> ⁇ (&))
- pos min (. ⁇ arg max (£> ⁇ (&))
- pos min (. ⁇ arg max (£> ⁇ (&))
- the attenuations associated with the samples of the sub-block of the attack are all set to 1 even if attack is towards the end of this sub -block.
- the starting position of the attack pos is refined in the sub-block of the attack, for example by cutting the sub-block into sub-sub-blocks and observing the evolution of the energy of these sub-sub-blocks.
- the calculation of the attenuation values based on the sub-signal x rec ss1 (n) may be similar to the calculation of the attenuation values based on the decoded signal x rec (n).
- the attenuation values can be determined based on the decoded signal x rec (n). In the case where the detection of attacks is made on the decoded signal, it is no longer necessary to recalculate energies of the sub-blocks because for this signal the energy values by sub-block are already calculated to detect the attacks.
- the attenuation factors g pre ssi (n) and g pre ss2 (T i) determined by sub-blocks can then be smoothed by an applied smoothing function sample by sample to avoid abrupt variations of the attenuation factor at the boundaries of the blocks. .
- This is particularly important for sub-signals containing low frequency components such as sub-signal x mc ss i (n) but not necessary for sub-signals containing only high frequency components such as sub-signal x rec ss2 [n ).
- Figure 7 illustrates an example of applying an attenuation gain with smoothing functions represented by the L arrows.
- This figure illustrates in a), an example of an original signal, in b), the decoded signal without pre-echo attenuation, in c), the attenuation gains for the two sub-signals obtained according to the decomposition step E605 and in d), the decoded signal with pre-echo attenuation of steps E607 and E608 (i.e. after combining the two attenuated sub-signals).
- the attenuation gain represented in dashed line and corresponding to the gain calculated for the first sub-signal comprising low frequency components comprises smoothing functions as described above.
- the attenuation gain represented in solid line and calculated for the second sub-signal comprising high frequency components does not include smoothing gain.
- the signal represented in d) shows that the pre-echo has been effectively attenuated by the attenuation processing implemented.
- the smoothing function is for example preferably defined by the following equations:
- the pre-echo zone (the number of attenuated samples) may therefore be different for the 2 sub-signals processed separately, even if the detection of the attack is made in common on the basis of the decoded signal. .
- the smoothed attenuation factor does not go back to 1 at the moment of the attack, which implies a decrease in the amplitude of the attack. The noticeable impact of this decrease is very small but must nevertheless be avoided.
- the attenuation factor value can be forced to 1 for the u-1 samples preceding the index pos where the onset of the attack is. This is equivalent to advancing the pos marker of u-1 samples for the sub signal where the smoothing is applied.
- the smoothing function gradually increases the factor to have a value 1 at the time of the attack. The amplitude of the attack is then preserved.
- the verification of the increase of the energy of the pre-echo zone according to the invention is carried out for at least one sub-signal or for each of these sub-signals.
- the comparison threshold used may be different depending on the sub-signals and the number of sub-blocks available before attack.
- the normalized steering coefficient b in is lower than the threshold of this sub-signal, the pre-echo attenuation is inhibited for all the sub-signals.
- pre-echo processing can be done for example by setting the attenuation factors to 1 or not discriminating the area as a pre-echo zone, the module of Pre-echo attenuation processing is then not requested as illustrated by way of example in the embodiment of Figure 5 by the link between the block 604 and 602.
- the attenuation will be inhibited separately for each sub-signal as soon as the normalized steering coefficient b in is lower than the threshold of this sub-signal.
- the inhibition may for example be implemented by setting the attenuation factors to 1 or by not soliciting the pre-echo module for the sub-signal considered.
- the evolution of the two sub-signals is checked in both sub-signals. energy of the sub-blocks preceding the sub-block where the attack was detected, by linear regression.
- This verification can be done according to the steps E603 and E604, at any time after the division of the decoded signal into sub-signals (E605) and before the application of the pre-echo attenuation factors (E607). Verification is possible if at least two sub-blocks precede the sub-block where the attack was detected. If the attack is detected in the first or second sub-block the verification according to the invention is not possible.
- step E603 a simple director coefficient (without normalization) such as:
- bi nss2 is less than 0.2, pre-echo attenuation is inhibited for this pre-echo zone, and for all the sub-signals.
- the module 607 of the device 600 of FIG. 5 implements the pre-echo attenuation step E607 in the pre-echo zone of each of the sub-signals by application to the sub-signals of the attenuation factors thus calculated. .
- a step E608 of the obtaining module 608 makes it possible to obtain the attenuated output signal (the decoded signal after pre-echo attenuation) by combining (in this example by simple addition) the attenuated sub-signals, according to the equation:
- the filtering used is not associated with sub-signal decimation operations and the complexity and delay ("lookahead" or future frame) are reduced to a minimum.
- FIG. 8 An exemplary embodiment of a discrimination and attenuation processing device according to the invention is now described with reference to FIG. 8.
- this device 100 in the sense of the invention typically comprises a ⁇ processor cooperating with a memory block BM including a storage and / or working memory, and a memory buffer MEM mentioned above as a means for storing all data. necessary to implement the method of discrimination and attenuation processing as described with reference to Figure 5.
- This device receives as input successive frames of the digital signal Se and delivers the signal Sa reconstructed with attenuation pre-echo in the pre-echo areas discriminated with the eventual reconstruction of the attenuated signal by combination of attenuated sub-signals.
- the memory block BM may comprise a computer program comprising the code instructions for implementing the steps of the method according to the invention when these instructions are executed by a ⁇ processor of the device and in particular the steps of calculating a control coefficient of energies for at least two sub-blocks preceding the sub-block in which an attack is detected, comparing the steering coefficient to a predefined threshold and inhibiting the pre-echo attenuation processing in the pre-echo zone in the case where the calculated steering coefficient is below the predefined threshold.
- Figure 5 can illustrate the algorithm of such a computer program.
- This discrimination and attenuation processing device can be independent or integrated in a digital signal decoder.
- a decoder can be integrated with equipment for storing or transmitting digital audio signals such as communication gateways, communication terminals or servers of a communication network.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
- Cable Transmission Systems, Equalization Of Radio And Reduction Of Echo (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
FR1458608A FR3025923A1 (en) | 2014-09-12 | 2014-09-12 | DISCRIMINATION AND ATTENUATION OF PRE-ECHO IN AUDIONUMERIC SIGNAL |
PCT/FR2015/052433 WO2016038316A1 (en) | 2014-09-12 | 2015-09-11 | Discrimination and attenuation of pre-echoes in a digital audio signal |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3192073A1 true EP3192073A1 (en) | 2017-07-19 |
EP3192073B1 EP3192073B1 (en) | 2018-08-01 |
Family
ID=51842602
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP15771686.1A Active EP3192073B1 (en) | 2014-09-12 | 2015-09-11 | Discrimination and attenuation of pre-echoes in a digital audio signal |
Country Status (8)
Country | Link |
---|---|
US (1) | US10083705B2 (en) |
EP (1) | EP3192073B1 (en) |
JP (2) | JP6728142B2 (en) |
KR (1) | KR102000227B1 (en) |
CN (2) | CN106716529B (en) |
ES (1) | ES2692831T3 (en) |
FR (1) | FR3025923A1 (en) |
WO (1) | WO2016038316A1 (en) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
FR3025923A1 (en) * | 2014-09-12 | 2016-03-18 | Orange | DISCRIMINATION AND ATTENUATION OF PRE-ECHO IN AUDIONUMERIC SIGNAL |
EP3382700A1 (en) * | 2017-03-31 | 2018-10-03 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Apparatus and method for post-processing an audio signal using a transient location detection |
EP3652867B1 (en) * | 2017-07-14 | 2021-05-26 | Dolby Laboratories Licensing Corporation | Mitigation of inaccurate echo prediction |
JP7172030B2 (en) * | 2017-12-06 | 2022-11-16 | 富士フイルムビジネスイノベーション株式会社 | Display device and program |
Family Cites Families (21)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
NL249503A (en) | 1959-03-19 | |||
JP3104400B2 (en) * | 1992-04-27 | 2000-10-30 | ソニー株式会社 | Audio signal encoding apparatus and method |
FR2739736B1 (en) * | 1995-10-05 | 1997-12-05 | Jean Laroche | PRE-ECHO OR POST-ECHO REDUCTION METHOD AFFECTING AUDIO RECORDINGS |
JP3660599B2 (en) * | 2001-03-09 | 2005-06-15 | 日本電信電話株式会社 | Rising and falling detection method and apparatus for acoustic signal, program and recording medium |
WO2005057801A2 (en) * | 2003-12-05 | 2005-06-23 | Plexus Networks, Inc. | Low-power mixed-mode echo/crosstalk cancellation in wireline communications |
CA2457988A1 (en) * | 2004-02-18 | 2005-08-18 | Voiceage Corporation | Methods and devices for audio compression based on acelp/tcx coding and multi-rate lattice vector quantization |
TWI275074B (en) * | 2004-04-12 | 2007-03-01 | Vivotek Inc | Method for analyzing energy consistency to process data |
FR2897733A1 (en) | 2006-02-20 | 2007-08-24 | France Telecom | Echo discriminating and attenuating method for hierarchical coder-decoder, involves attenuating echoes based on initial processing in discriminated low energy zone, and inhibiting attenuation of echoes in false alarm zone |
CN102160114B (en) * | 2008-09-17 | 2012-08-29 | 法国电信公司 | Method and device of pre-echo attenuation in a digital audio signal |
KR20110001130A (en) * | 2009-06-29 | 2011-01-06 | 삼성전자주식회사 | Apparatus and method for encoding and decoding audio signals using weighted linear prediction transform |
CN102318004B (en) * | 2009-09-18 | 2013-10-23 | 杜比国际公司 | Improved harmonic transposition |
US8582443B1 (en) * | 2009-11-23 | 2013-11-12 | Marvell International Ltd. | Method and apparatus for virtual cable test using echo canceller coefficients |
CN103325379A (en) * | 2012-03-23 | 2013-09-25 | 杜比实验室特许公司 | Method and device used for acoustic echo control |
CN103391381B (en) * | 2012-05-10 | 2015-05-20 | 中兴通讯股份有限公司 | Method and device for canceling echo |
FR2992766A1 (en) | 2012-06-29 | 2014-01-03 | France Telecom | EFFECTIVE MITIGATION OF PRE-ECHO IN AUDIONUMERIC SIGNAL |
CN103730125B (en) * | 2012-10-12 | 2016-12-21 | 华为技术有限公司 | A kind of echo cancelltion method and equipment |
FR3000328A1 (en) * | 2012-12-21 | 2014-06-27 | France Telecom | EFFECTIVE MITIGATION OF PRE-ECHO IN AUDIONUMERIC SIGNAL |
FR3011408A1 (en) * | 2013-09-30 | 2015-04-03 | Orange | RE-SAMPLING AN AUDIO SIGNAL FOR LOW DELAY CODING / DECODING |
FR3015754A1 (en) * | 2013-12-20 | 2015-06-26 | Orange | RE-SAMPLING A CADENCE AUDIO SIGNAL AT A VARIABLE SAMPLING FREQUENCY ACCORDING TO THE FRAME |
FR3023036A1 (en) * | 2014-06-27 | 2016-01-01 | Orange | RE-SAMPLING BY INTERPOLATION OF AUDIO SIGNAL FOR LOW-LATER CODING / DECODING |
FR3025923A1 (en) * | 2014-09-12 | 2016-03-18 | Orange | DISCRIMINATION AND ATTENUATION OF PRE-ECHO IN AUDIONUMERIC SIGNAL |
-
2014
- 2014-09-12 FR FR1458608A patent/FR3025923A1/en active Pending
-
2015
- 2015-09-11 JP JP2017513524A patent/JP6728142B2/en active Active
- 2015-09-11 ES ES15771686.1T patent/ES2692831T3/en active Active
- 2015-09-11 WO PCT/FR2015/052433 patent/WO2016038316A1/en active Application Filing
- 2015-09-11 KR KR1020177009719A patent/KR102000227B1/en active IP Right Grant
- 2015-09-11 CN CN201580048998.5A patent/CN106716529B/en active Active
- 2015-09-11 EP EP15771686.1A patent/EP3192073B1/en active Active
- 2015-09-11 CN CN202010861715.1A patent/CN112086107B/en active Active
- 2015-09-11 US US15/510,831 patent/US10083705B2/en active Active
-
2020
- 2020-06-30 JP JP2020112837A patent/JP7008756B2/en active Active
Also Published As
Publication number | Publication date |
---|---|
CN112086107A (en) | 2020-12-15 |
CN106716529B (en) | 2020-09-22 |
JP6728142B2 (en) | 2020-07-22 |
KR102000227B1 (en) | 2019-07-15 |
EP3192073B1 (en) | 2018-08-01 |
CN112086107B (en) | 2024-04-02 |
FR3025923A1 (en) | 2016-03-18 |
US20170263263A1 (en) | 2017-09-14 |
KR20170055515A (en) | 2017-05-19 |
ES2692831T3 (en) | 2018-12-05 |
CN106716529A (en) | 2017-05-24 |
JP2017532595A (en) | 2017-11-02 |
WO2016038316A1 (en) | 2016-03-17 |
US10083705B2 (en) | 2018-09-25 |
JP2020170187A (en) | 2020-10-15 |
JP7008756B2 (en) | 2022-01-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2002428B1 (en) | Method for trained discrimination and attenuation of echoes of a digital signal in a decoder and corresponding device | |
EP2867893B1 (en) | Effective pre-echo attenuation in a digital audio signal | |
EP1789956B1 (en) | Method of processing a noisy sound signal and device for implementing said method | |
CA2436318C (en) | Noise reduction method and device | |
EP2936488B1 (en) | Effective attenuation of pre-echos in a digital audio signal | |
EP2586133B1 (en) | Controlling a noise-shaping feedback loop in a digital audio signal encoder | |
EP2153438B1 (en) | Post-processing for reducing quantification noise of an encoder during decoding | |
FR2741217A1 (en) | METHOD AND DEVICE FOR REMOVING PARASITE NOISE IN A COMMUNICATION SYSTEM | |
EP3192073B1 (en) | Discrimination and attenuation of pre-echoes in a digital audio signal | |
FR3012928A1 (en) | MODIFIERS BASED ON EXTERNALLY ESTIMATED SNR FOR INTERNAL MMSE CALCULATIONS | |
EP0506535B1 (en) | Method and system for processing of pre-echos of a frequency transform coded digital audio signal | |
EP2347411B1 (en) | Pre-echo attenuation in a digital audio signal | |
WO1999014737A1 (en) | Method for detecting speech activity | |
EP1039736A1 (en) | Method and device for adaptive identification and related adaptive echo canceller | |
EP1021805B1 (en) | Method and apparatus for conditioning a digital speech signal | |
EP2515300A1 (en) | Method and System for noise reduction | |
WO2014199055A1 (en) | Control of the processing of attentuation of quantization noise introduced by a compression coding |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20170404 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R079 Ref document number: 602015014397 Country of ref document: DE Free format text: PREVIOUS MAIN CLASS: G10L0025210000 Ipc: G10L0019260000 |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G10L 25/21 20130101ALI20180306BHEP Ipc: G10L 19/26 20130101AFI20180306BHEP Ipc: G10L 25/51 20130101ALI20180306BHEP |
|
INTG | Intention to grant announced |
Effective date: 20180323 |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: KOVESI, BALAZS Inventor name: RAGOT, STEPHANE |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP Ref country code: AT Ref legal event code: REF Ref document number: 1025252 Country of ref document: AT Kind code of ref document: T Effective date: 20180815 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: FRENCH |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602015014397 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 4 |
|
REG | Reference to a national code |
Ref country code: ES Ref legal event code: FG2A Ref document number: 2692831 Country of ref document: ES Kind code of ref document: T3 Effective date: 20181205 Ref country code: NL Ref legal event code: MP Effective date: 20180801 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 1025252 Country of ref document: AT Kind code of ref document: T Effective date: 20180801 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180801 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180801 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180801 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180801 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181101 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180801 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181201 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181102 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181101 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180801 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180801 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180801 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180801 Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180801 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180801 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180801 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180801 Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180801 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602015014397 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180801 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180801 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180801 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20180930 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180911 |
|
26N | No opposition filed |
Effective date: 20190503 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180911 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180801 Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180930 Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180930 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180930 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180801 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180801 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180801 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180801 Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20150911 Ref country code: MK Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180801 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: IT Payment date: 20230822 Year of fee payment: 9 Ref country code: GB Payment date: 20230823 Year of fee payment: 9 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20230822 Year of fee payment: 9 Ref country code: DE Payment date: 20230822 Year of fee payment: 9 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: ES Payment date: 20231002 Year of fee payment: 9 |