EP3197181B1 - Procédé de réduction du temps de latence d'un banc de filtrage destiné au filtrage d'un signal audio et procédé de fonctionnement sans latence d'un système auditif - Google Patents

Procédé de réduction du temps de latence d'un banc de filtrage destiné au filtrage d'un signal audio et procédé de fonctionnement sans latence d'un système auditif Download PDF

Info

Publication number
EP3197181B1
EP3197181B1 EP16204529.8A EP16204529A EP3197181B1 EP 3197181 B1 EP3197181 B1 EP 3197181B1 EP 16204529 A EP16204529 A EP 16204529A EP 3197181 B1 EP3197181 B1 EP 3197181B1
Authority
EP
European Patent Office
Prior art keywords
signal
audio signal
output
prediction period
block
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP16204529.8A
Other languages
German (de)
English (en)
Other versions
EP3197181A1 (fr
Inventor
Marc Aubreville
Oliver Dressler
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sivantos Pte Ltd
Original Assignee
Sivantos Pte Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sivantos Pte Ltd filed Critical Sivantos Pte Ltd
Publication of EP3197181A1 publication Critical patent/EP3197181A1/fr
Application granted granted Critical
Publication of EP3197181B1 publication Critical patent/EP3197181B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
    • H04R25/50—Customised settings for obtaining desired overall acoustical characteristics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
    • H04R25/35—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception using translation techniques
    • H04R25/353—Frequency, e.g. frequency shift or compression
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
    • H04R25/50—Customised settings for obtaining desired overall acoustical characteristics
    • H04R25/505—Customised settings for obtaining desired overall acoustical characteristics using digital signal processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
    • H04R25/55—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception using an external connection, either wireless or wired
    • H04R25/552—Binaural
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R5/00—Stereophonic arrangements
    • H04R5/04—Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/0017—Lossless audio signal coding; Perfect reconstruction of coded audio signal by transmission of coding error
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2225/00—Details of deaf aids covered by H04R25/00, not provided for in any of its subgroups
    • H04R2225/43—Signal processing in hearing aids to enhance the speech intelligibility

Definitions

  • the invention relates to a method for reducing the latency of a filter bank for filtering an audio signal, wherein a plurality of signal blocks in the time domain is formed from the audio signal, wherein in each case a filter function is specified for at least a plurality of the signal blocks, the signal Block is filtered with the given filter function transformed into the frequency domain, and thereby a transformed signal block is formed, and signal components of the transformed signal block are output for further processing.
  • the invention further relates to a method for low-latency operation of a hearing system, wherein a first audio signal is generated from a sound signal by a first input transducer, wherein the first audio signal is filtered in a signal processing unit by means of a first filter bank, wherein signal components of the filtered first audio signal in the signal processing unit further processed and for generating an output signal, and wherein an output sound signal is generated from the output signal by an output transducer.
  • an audio signal generated by a microphone is usually transformed from the time domain into the frequency domain after digitization, ie after the digitization the audio signal initially exists in the form of time-resolved samples which, if necessary, become individual signal blocks (so-called “signal blocks”).
  • Frames ") are decomposed by a Fourier transform such as FFT into individual spectral signal components of the generated audio signal.
  • FFT Fourier transform
  • This has the advantage that frequency-selective algorithms such as noise reduction, directional microphone or dynamic compression can be applied.
  • the mentioned transformation has the disadvantage that an audio signal converted back into the time domain after a corresponding frequency-selective processing has a delay with respect to the input signal, which is typically of the order of magnitude of several ms. This delay, also known as latency, is greater the higher the resolution in the frequency domain is chosen.
  • an open adaptation of the hearing device is often chosen, in which the sound output from a speaker of the hearing aid output signal via a sound tube with Schirmchen or a listener with Schirmchen in the ear canal is passed to the eardrum.
  • the eardrum itself thus comes a mixture of a frequency selective attenuated direct sound of the environment as well as the output sound signal generated by the hearing aid.
  • different mixing ratios are therefore found as a function of the frequency.
  • the described problems with comb filter effects are not tied to a binaural hearing system, but can also occur in a monaural hearing system with only one hearing aid, in which a direct sound of the environment and an output sound signal of a hearing device overlaid with temporal displacement reach the eardrum of the user ,
  • the temporal offset is primarily due to the internal latency of the hearing system for signal processing and in particular in the filtering.
  • a method for filtering an input signal by means of a desired impulse response in which the impulse response in the time domain is decomposed into individual segments which are transformed into the frequency domain and from which respective coefficient blocks are formed for the filtering of the individual time-delayed frames in the frequency domain ,
  • the frames thus filtered with the coefficient blocks are summed with their corresponding time delay, and from this a signal in the time domain is generated by inverse transformation, of which in a predetermined time Way still individual signal components are discarded to obtain the final, filtered output signal.
  • the US Pat. No. 7,251,271 B1 refers to a method to avoid so-called aliasing effects when filtering a discrete input signal with a discrete impulse response. These can occur during the transformation of the individual frames of the input signal from the time domain into the frequency domain and the inverse transformation of the product of impulse response and frequency spectrum of the input signal into the time domain. To avoid the aliasing effects, individual frames are extended before the respective transformation by adding zeros in order to correspond to the respective filter length.
  • the invention is therefore based on the object of specifying a method for as low-latency spectral filtering of an audio signal with the highest possible spectral resolution.
  • the invention is further based on the object of specifying a method for low-latency operation of a hearing system.
  • the first object is achieved according to the invention by a method for reducing the latency of a filter bank for filtering an audio signal, wherein a plurality of signal blocks in the time domain is formed from the audio signal.
  • a filter function is specified, at least a subinterval of the signal block is given as a prediction period, signal components of the signal block are estimated in the at least one subinterval for the prediction period, and from the for the prediction period of estimated signal components and the signal components of the signal block outside the prediction period, a predicted signal block is generated.
  • the predicted signal block is filtered with the predetermined filter function transformed into the frequency domain, and thereby a transformed signal block is formed, and signal components of the transformed signal block are output for further processing.
  • the second object is achieved by a method for low-latency operation of a hearing system, wherein from a sound signal through a first input transducer, a first audio signal is generated, wherein the first audio signal is transmitted directly to a signal processing unit, and in the signal processing unit directly by means of a first filter bank according to the above-described method for reducing the latency of a filter bank for filtering an audio signal is filtered, wherein signal portions of the filtered first audio signal in the signal processing unit further processed and used to generate an output signal, and wherein from the output signal directly by an output transducer, an output sound signal is generated.
  • a signal block (“frame") in the time domain is formed from the audio signal by the audio signal is converted by time and amplitude discretization in a plurality of each successive time points associated amplitude ("samples"), and a plurality of consecutive samples is combined into a signal block.
  • the further processing of the signal components of the transformed signal block comprises in particular a frequency band-dependent amplification, a frequency band-dependent directional characteristic, a frequency band-dependent noise suppression and a backward transformation of frequency band-dependent treated signal components in the time domain.
  • the estimation of the signal components for the prediction period of a respective signal block preferably takes place via a prediction algorithm, such as by a linear prediction filter.
  • a prediction algorithm such as by a linear prediction filter.
  • an adaptive adaptation of time-correlated coefficients used for the estimation is possible such that an estimation coefficient, which is assigned as a coordinate in the signal block in each case to a sample with a specific time delay, depending on the error between an estimated sample and a real from the Audio signal obtained sample is corrected, whereby the corrector is renewed at periodic intervals.
  • a signal component estimated for a signal block becomes is also used for a signal block which follows later, if the period corresponding to the signal component still falls within the prediction period of the signal block following later.
  • the prediction period preferably comprises the respectively first and / or the last sample of a signal block.
  • the period lying outside the prediction period in each case forms a coherent interval in a signal block.
  • the prediction period comprises the first n samples and / or the last m samples, where n and m natural numbers are smaller than the number of samples in the respective signal block.
  • An input transducer or an output transducer of the hearing system includes any shape of an acousto-electric or an electro-acoustic transducer, for example a microphone or a loudspeaker.
  • Direct transmission of the first audio signal to the signal processing unit is to be understood as meaning that the transmission of the first audio signal takes place immediately after its generation, that is to say in particular without another, via signal preprocessing, such as signal processing.
  • a / D conversion and / or data compression beyond time delay takes place, as z. B. by a long-term physical storage, which is not based on the FIFO principle ("first-in-first-out"), would occur.
  • the transmission takes place in particular locally within a hearing device, for example on the signal path predetermined by the signal lines.
  • the transmission also takes place wirelessly, for example from a first hearing device of a binaural hearing system to a second hearing device of the binaural hearing system.
  • Direct filtering of the first audio signal in the signal processing unit analogously means that the filtering process for the audio signal takes place immediately after its input in the signal processing unit, ie in particular without a further, beyond the direct signal transmission time delay as z. B. by a long-term storage, which is not based on the FIFO principle ("first-in-first-out"), would occur.
  • an immediate generation of the output sound signal from the output signal means that immediately after the generation of the output signal is passed through the further processing, the output signal to the output transducer for output, ie in particular without a further, beyond the direct signal transmission outgoing time delay, z.
  • the filter banks which are used to transform the audio signals generated by the input transducers into the frequency domain (analysis filter banks), and the filter banks for the inverse transformation of the frequency resolved, further processed audio signals into the time domain (Synthetic filter banks), the former usually have a larger proportion.
  • the transmission of an audio signal from one hearing aid to another for the generation of a binaural output signal is associated with a certain delay.
  • the latter is difficult to reduce given the restrictions on encoding for transmission.
  • the effective length of the signal block can be reduced with an appropriate choice of the prediction period, without This affects the frequency resolution of the filter bank.
  • the frequency resolution of the filter bank depends on the temporal information content of the signal blocks to be used for the filter process, ie of their length. Since the signal components are now estimated in a signal block for a period of time, the latency of the filter bank can be reduced by the duration corresponding to the associated prediction period.
  • two temporally successive signal blocks overlap each other in part.
  • the definition of the temporal sequence is preferably carried out via a reference sample for the respective signal block, e.g. the first sample.
  • the consequence of the described overlap is that the relevant, successive signal blocks have several, preferably successive, samples in common.
  • this improves the temporal resolution in the frequency domain, since this allows frequent frequency band information to be updated frequently, and on the other hand, the cost of estimating the signal components can be reduced because already estimated signal components are available for a subsequent block without a new estimation process stand.
  • signal portions of the transformed signal block are output separately according to different frequency bands for further processing.
  • the latency of the filter bank which is reduced by estimating the signal components of the prediction periods, is particularly advantageous given a constant high frequency resolution.
  • the filter function in the prediction period preferably has an average lower transmission amplitude than outside the prediction period. This is to mean that the value of the transmission amplitude of the filter function averaged over the entire prediction period is lower than that over the remaining one Period of the signal block outside the prediction period averaged value of the transmission amplitude of the filter function.
  • the transmission amplitude of the filter function is formed in each case by a logarithmically concave function, wherein the prediction period spans the maximum of the transmission amplitude of the filter function.
  • a logarithmically concave function is defined as a function whose logarithm is concave in the domain of definition - which is given here by the individual samples of the respective signal block. Such a function may for example be given by an approximation of a Gaussian bell curve over a finite, discretized domain of definition.
  • the advantage of the logarithmic concave behavior of the transmission amplitude is that it has a maximum of two inflection points in the domain of definition, and thus is not subject to any oscillations. This results in an advantageous filter behavior, since thus no relevant signal components with a minimum value of an oscillation of the filter function are filtered.
  • a logarithmic concave function can be represented as a function reciprocal to a particular logarithmic convex function.
  • a logarithmically convex function is in turn convex. This means that the reciprocal, logarithmically concave function due to the reciprocity property has a maximum of two inflection points.
  • the maximum of the transmission amplitude lies in a convex region, so that beyond the inflection points the transmission amplitude runs concavely.
  • the transmission amplitude usually already has sufficiently low values, so that the choice of the prediction time period in at least one of the two ranges can ensure that errors which can occur due to deviations in the estimation of the signal components from the real signal components, be largely suppressed due to the sufficiently lower transmission amplitude of the filter function, and thus not significantly enter into the transformed signal block.
  • an empty signal is respectively estimated as signal components for the prediction period of at least one signal block.
  • An empty signal is in this case the signal which has no amplitude for the period in question.
  • the estimation of an empty signal is carried out in particular in the event that the signal components of the audio signal which are used for the estimation method of the signal components of the prediction period do not allow a qualitatively sufficiently high-quality estimate of the signal components as a result of insufficient correlations. This can occur, for example, if there is a high proportion of white noise in the audio signal, which reduces the correlation of successive samples and thus makes prediction more difficult.
  • estimated signal components which are different from the empty signal, with respect to the quality of the estimate, are to be compared with the corresponding real signal components of the audio signal in order to be able to evaluate the quality of the prediction.
  • excessive deviation - defined by a deviation measure such.
  • B. an averaged over several samples difference and an associated upper bound for the deviation measure - is determined instead of the predicted signal components, an empty signal as estimated for the prediction signal component. It is also possible to check the signal components of the audio signal for correlations even before the prediction, and to set an empty signal as signal component for the prediction period if the correlation is too low.
  • a second audio signal is generated from the sound signal by a second input transducer spatially separated from the first input transducer, the second audio signal is transmitted directly to the signal processing unit and filtered by a second filter bank, and wherein signal components of filtered second audio signal in the signal processing unit further processed and used to generate the output signal.
  • the filtering of the second audio signal by means of the second filter bank in accordance with the above-described method for reducing the latency of a filter bank for filtering an audio signal is understood to mean that the transmission of the second audio signal without another, via a signal preprocessing such.
  • a / D conversion and / or data compression and the direct signal transmission beyond time delay takes place, as z. B. by a long-term physical storage, which would not based on the FIFO principle ("first-in-first-out”) would occur.
  • This named embodiment makes possible, in particular, a low-latency operation of a binaural hearing system, taking into account the special features occurring in such a hearing system as a result of the signal transmission from one hearing aid to another occurring for the generation of the binaural auditory sensation. Since, in the case of a binaural hearing system for compression, the real information content of signal portions of the audio signal received by the respective other hearing device for the generation of the binaural hearing sensation is often reduced for better transmission, for example by data compression, this is possible by estimating the signal components in the prediction period reduced errors in its meaning.
  • a further advantage of using the method for low-latency operation of a binaural hearing system is that a certain latency of several ms is already introduced into the hearing system through the described transmission of the audio signals.
  • the reduction of further possible latencies, e.g. in the present case through the filter banks, here helps to minimize the losses of the sound quality by comb filter effects as low as possible.
  • the invention further provides a hearing aid, comprising at least one input transducer for generating an audio signal, an output transducer for generating an output sound signal, and a local signal processing unit having a first filter bank, which for carrying out the above-described method for reducing the latency of a filter bank for filtering an audio signal is set up.
  • the invention also mentions a binaural hearing system with two hearing aids as described above, which is set up to carry out the method for low-latency operation of a hearing system with at least two input transducers.
  • the advantages specified for the method and its developments can be transferred analogously to the binaural hearing system.
  • Fig. 1 1 is a schematic block diagram of a binaural hearing system 1.
  • the binaural hearing system 1 is in this case formed by a first hearing device 2 and a second hearing device 4.
  • the first hearing device 2 has a first input transducer 8 configured as a microphone 6, which generates a first audio signal 10 from a sound signal 9.
  • the second hearing device 4 has a second input transducer 14 configured as a microphone 12, which generates a second audio signal 16 from the sound signal 9.
  • the first audio signal 10 and the second audio signal 16 are respectively prepared in the respective hearing device 2, 4 by a local signal preprocessing 18, 20, which in each case in particular comprises an A / D conversion, for the further signal processing processes.
  • the local signal preprocessing 18, 20 comprises in particular only runtime processes, ie those processes which do not involve any further delay, in particular no longer-term storage and charging processes of the signal components, over the duration of the signal processing taking place.
  • the first audio signal 10 is first transmitted immediately after the local signal pre-processing 18 in a binaural transmission process 22 from the first hearing aid 2 to the second hearing aid 4, where it is filtered in a signal processing unit 24 in a first filter bank 26 in a manner to be described.
  • the binaural transmission process 22 takes place immediately after the local signal preprocessing 18, that is, in particular without further delay, in particular without longer-term storage and recharging the relevant signal components on a FIFO memory addition.
  • the filtered first audio signal 28 will now be subjected to frequency bandwise signal processing algorithms 30, e.g. Noise suppression, directional microphone or dynamic compression applied.
  • the second audio signal 16 is supplied immediately after the local signal pre-processing 20 of the signal processing unit 24, where it is first filtered in a second filter bank 32 in a manner to be described, wherein as a filtered second audio signal 34, the respective signal components are transmitted separately in individual frequency bands.
  • the filtered second audio signal 34 resulting from the second filter bank 32 the respective signal components are output separately in individual frequency bands.
  • frequency bandwise signal processing algorithms 28 such as noise reduction, directional microphone or dynamic compression are now applied. From the filtered first audio signal 26 and the filtered second audio signal 34, an output signal 36 is generated after the frequency band-wise signal processing 28, which locally reflects the binaural hearing at the location of the second hearing device 4.
  • the output signal 36 is converted directly, ie in particular without further long-term storage and recharges of the signal components, from an output transducer 40 designed as a loudspeaker 38 into an output sound signal 42.
  • Fig. 2 is against a time axis t, the first audio signal 10 after Fig. 1 which is split into individual partially overlapping signal blocks 50a-f.
  • the individual signal blocks 50a-f are in this case formed from a large number of successive samples of the first audio signal 10, individual samples occurring as a result of the overlap of the successive signal blocks 50a-f in at least two signal blocks.
  • the individual signal blocks 50a-f are now each transformed in a manner to be described, the frequency domain. Due to the short time interval of each of two consecutive signal blocks 50a-f, the spectral signal components of the first audio signal 10 can thus be updated in short intervals in the frequency domain.
  • the individual signal blocks 50a-f are determined Signal portions, which is shown for the signal block 50c on the basis of a detail representation.
  • the individual real signal components 52a, 52b are shown against a time axis t '.
  • the real signal components 52a, 52b are each given by the amplitude of the corresponding sample.
  • the transmission amplitude 54c of the filter function 56c is shown, which in the present case is approximately given by a Gaussian bell curve.
  • the filter function 56c in this case represents a window function with which the edges of the signal block 50c for the transformation into the frequency domain are to be smoothed out. This is because without such a window function, the Fourier transform of the signal components of the signal block 50c is de facto a Fourier transform of the signal components of the first audio signal 10, which are multiplied by a rectangular function corresponding to the duration of the signal block.
  • this multiplication in the time domain means a convolution of the frequency components of the first audio signal 10 with the Fourier transform of the rectangular function, which is given by a strongly oscillating sin (x) / x or Sinc function.
  • the edges of the signal block 50c for the transformation into the frequency domain are "hidden” by means of a suitable filter function 56c. This is done by the transmission amplitude 54c of the filter function 56c converges to zero at the edges of the signal block 50c as free of oscillation as possible, ie in particular with as few turning points as possible.
  • a function having such properties is given by a logarithmic concave function, such as e.g. the approximated Gaussian bell curve of the present case.
  • the sub-interval 58c lies beyond the point of inflection 62c of the transmission amplitude 54c, ie in particular far away from the maximum 64c of the transmission amplitude 54c, so that in the sub-interval 58c, which defines the prediction time 60c, the transmission amplitude 54c has only low values.
  • the signal components to be used for the transformation are now estimated there by means of a prediction algorithm, for example a linear prediction filter, instead of the real signal components 52b.
  • the signal portions 66b estimated in the prediction period 60c and the signal portions 52a of the signal block 50c outside the prediction period 60c now form a predicted signal block 68c.
  • This predicted signal block 68c is now multiplied by the filter function 56c and transformed into the frequency domain by means of a fast Fourier transformation, so that there the frequency-resolved information of the transformed signal block 50c is available for further processing by means of frequency band-dependent signal processing algorithms stands.
  • the procedure described is used to estimate signal components for a prediction period which is to be chosen favorably on the basis of the respective filter function to be used, in order thus to reduce the latency for the transformation into the frequency domain. since then the last samples of a signal block need not yet exist, so that the transformation can be started several ms earlier due to the estimation.
  • An important role in this case plays the course of the transmission amplitude 54c of the filter function 56c.
  • a possible error which could result from the deviation of the signal portions 66b estimated for the prediction period 60c from the real signal portions 52b is suppressed by the fact that for the prediction period 60c the transmission amplitude 54c has only comparatively small values relative to its maximum 64c, and thus by the corresponding multiplication with the filter function 56c, the estimated signal components 66b make only a small contribution to the transformed signal block anyway.
  • this contribution is important for the spectral resolution.
  • tonal signal components can be estimated relatively well anyway by means of conventional prediction methods. Even with a white noise, which is unfavorable due to its static properties, due to the mentioned suppression of the errors due to possible deviations, the described method gives good results.
  • the binaural hearing system 1 of Fig. 1 is the first audio signal 10 in the first filter bank 24 according to the basis of Fig. 2 described method filtered.
  • the filtering of the second audio signal 16 in the second filter bank 32 can be done in the same way; however, a conventional filter method-that is to say without estimation of signal components for a respective prediction period of the individual signal blocks-can also be used for this purpose.
  • the decision on this is made in particular as a function of the tolerable overall latency of the binaural hearing system 1 and the delay caused by the binaural transmission process.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Neurosurgery (AREA)
  • Otolaryngology (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Stereophonic System (AREA)
  • Circuit For Audible Band Transducer (AREA)

Claims (11)

  1. Procédé, destiné à réduire le temps de latence d'un banc de filtres (26, 32) pour le filtrage d'un signal audio (10, 16)
    lors duquel à partir du signal audio (10, 16) une pluralité de blocs de signaux (50a à f) est créée dans le domaine temporel,
    lors duquel, pour au moins une pluralité des blocs de signaux (50a à f), chaque fois
    - une fonction de filtrage (56c) est prédéfinie,
    - au moins un intervalle partiel (58c) du bloc de signaux (50a à f) est prédéfini comme étant une période de prédiction (60c),
    - des fractions de signaux (66b) du bloc de signaux (50a à f) sont estimées dans l'au moins un intervalle partiel (58c) pour la période de prédiction (60c) et à partir des fractions de signaux (66b) estimées pour la période de prédiction (60c) et des fractions de signaux (52a) du bloc de signaux (50a à f) hors de la période de prédiction (60c), un bloc de signaux (68c) prédit est créé, et
    - le bloc de signaux (68c) prédit filtré avec la fonction de filtrage (56c) prédéfinie est transformé dans les domaines de fréquence et de ce fait, un bloc de signaux transformé est créé, et
    - des fractions de signaux du bloc de signaux transformé sont éditées pour traitement ultérieur.
  2. Procédé selon la revendication 1, lors duquel chaque fois deux blocs de signaux (50a à f) successifs dans le temps se chevauchent.
  3. Procédé selon la revendication 1 ou la revendication 2, lors duquel chaque fois des fractions de signaux du bloc de signaux transformé, séparées en différentes bandes de fréquence sont éditées pour le traitement ultérieur (30).
  4. Procédé selon l'une quelconque des revendications précédentes, chaque fois la fonction de filtrage (56c) dans la période de prédiction (60c) présentant une amplitude de transmission (54c) plus faible en moyenne qu'à l'extérieur de la période de prédiction (60c).
  5. Procédé selon la revendication 4, lors duquel l'amplitude de transmission (54c) de la fonction de filtrage (56c) est créée chaque fois par une fonction logarithmique concave et
    lors duquel la période de prédiction (60c) exclut le maximum (64c) de l'amplitude de transmission (54c) de la fonction de filtrage (56c).
  6. Procédé selon la revendication 5, la période de prédiction (60c) ne contenant que des plages convexes de l'amplitude de transmission (54c) de la fonction de filtrage (56c).
  7. Procédé selon l'une quelconque des revendications précédentes,
    lors duquel, pour la période de prédiction (60c) d'au moins un bloc de signaux (50a à f), chaque fois un signal vide est estimé en tant que fractions de signaux (66b).
  8. Procédé, destiné à faire fonctionner à faible latence un système auditif (1), lors duquel, à partir d'un signal acoustique (9), un premier signal audio (10) est créé par un premier transducteur d'entrée (8)
    lors duquel, le premier signal audio (10) est transmis directement à une unité de traitement des signaux (24) et dans l'unité de traitement des signaux (24), est filtré directement au moyen d'un premier banc de filtres (26) d'après un procédé selon l'une quelconque des revendications précédentes,
    lors duquel des fractions de signaux du premier signal audio (28) filtré sont traitées ultérieurement (30) dans l'unité de traitement des signaux (24) et utilisées pour créer un signal de sortie (36) et
    lors duquel, à partir du signal de sortie (36), un signal acoustique de sortie (42) est créé directement par un transducteur de sortie (40).
  9. Procédé selon la revendication 8, lors duquel, à partir du signal acoustique (9), un deuxième signal audio (16) est créé par un deuxième transducteur d'entrée (14) séparé dans l'espace du premier transducteur d'entrée (8),
    le deuxième signal audio (16) étant directement transmis à l'unité de traitement des signaux (24) et filtré au moyen d'un deuxième banc de filtres (32) et
    lors duquel des fractions de signaux du deuxième signal audio (16) filtré sont traitées ultérieurement dans l'unité de traitement des signaux (24) et utilisées pour créer le signal de sortie (36).
  10. Appareil auditif (2, 4), comprenant au moins un transducteur d'entrée (8, 14) pour créer un signal audio (10, 16), un transducteur de sortie (40) pour créer un signal acoustique de sortie (42), ainsi qu'une unité de traitement des signaux (24) dotée d'un premier banc de filtres (26), lequel appareil auditif est aménagé pour réaliser le procédé selon la revendication 8 ou la revendication 9.
  11. Système auditif (1) binaural, doté de deux appareils auditifs (2, 4) selon la revendication 10, lequel est aménagé pour réaliser le procédé selon la revendication 9.
EP16204529.8A 2016-01-19 2016-12-15 Procédé de réduction du temps de latence d'un banc de filtrage destiné au filtrage d'un signal audio et procédé de fonctionnement sans latence d'un système auditif Active EP3197181B1 (fr)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
DE102016200637.1A DE102016200637B3 (de) 2016-01-19 2016-01-19 Verfahren zur Reduktion der Latenzzeit einer Filterbank zur Filterung eines Audiosignals sowie Verfahren zum latenzarmen Betrieb eines Hörsystems

Publications (2)

Publication Number Publication Date
EP3197181A1 EP3197181A1 (fr) 2017-07-26
EP3197181B1 true EP3197181B1 (fr) 2018-09-26

Family

ID=57714385

Family Applications (1)

Application Number Title Priority Date Filing Date
EP16204529.8A Active EP3197181B1 (fr) 2016-01-19 2016-12-15 Procédé de réduction du temps de latence d'un banc de filtrage destiné au filtrage d'un signal audio et procédé de fonctionnement sans latence d'un système auditif

Country Status (5)

Country Link
US (1) US10142741B2 (fr)
EP (1) EP3197181B1 (fr)
CN (1) CN106982409B (fr)
DE (1) DE102016200637B3 (fr)
DK (1) DK3197181T3 (fr)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102018206689A1 (de) * 2018-04-30 2019-10-31 Sivantos Pte. Ltd. Verfahren zur Rauschunterdrückung in einem Audiosignal
DE102018207780B3 (de) 2018-05-17 2019-08-22 Sivantos Pte. Ltd. Verfahren zum Betrieb eines Hörgerätes
US11330376B1 (en) 2020-10-21 2022-05-10 Sonova Ag Hearing device with multiple delay paths
CN112218221B (zh) * 2020-10-21 2022-06-03 歌尔智能科技有限公司 一种助听适配器及控制方法
DE102021205251A1 (de) 2021-05-21 2022-11-24 Sivantos Pte. Ltd. Verfahren und Vorrichtung zur frequenzselektiven Verarbeitung eines Audiosignals mit geringer Latenz
DE102023200581A1 (de) 2023-01-25 2024-07-25 Sivantos Pte. Ltd. Verfahren zum Betrieb eines Hörinstruments

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA2139511C (fr) 1992-07-07 2004-09-07 David Stanley Mcgrath Filtre numerique de grande precision et de grande efficacite
SE517525C2 (sv) * 1999-09-07 2002-06-18 Ericsson Telefon Ab L M Förfarande och anordning för konstruktion av digitala filter
US7277554B2 (en) * 2001-08-08 2007-10-02 Gn Resound North America Corporation Dynamic range compression using digital frequency warping
US8494193B2 (en) * 2006-03-14 2013-07-23 Starkey Laboratories, Inc. Environment detection and adaptation in hearing assistance devices
AU2006275003A1 (en) * 2006-06-13 2007-02-08 Phonak Ag Method and system for acoustic shock detection and application of said method in hearing devices
TWI559679B (zh) * 2009-02-18 2016-11-21 杜比國際公司 低延遲調變濾波器組及用以設計該低延遲調變濾波器組之方法
CN102256200A (zh) * 2010-05-19 2011-11-23 上海聪维声学技术有限公司 全数字助听器的基于wola滤波器组的信号处理方法
EP2521377A1 (fr) * 2011-05-06 2012-11-07 Jacoti BVBA Dispositif de communication personnel doté d'un support auditif et procédé pour sa fourniture
CN111009249B (zh) * 2013-10-18 2021-06-04 弗劳恩霍夫应用研究促进协会 编码器/解码器、编码/解码方法和非瞬时性存储介质
EP2897382B1 (fr) * 2014-01-16 2020-06-17 Oticon A/s Amélioration des sources binaurales
DE102014204557A1 (de) * 2014-03-12 2015-09-17 Siemens Medical Instruments Pte. Ltd. Übertragung eines windreduzierten Signals mit verminderter Latenzzeit
JP6391197B2 (ja) * 2015-01-14 2018-09-19 ヴェーデクス・アクティーセルスカプ 補聴器システムの動作方法および補聴器システム

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
CN106982409A (zh) 2017-07-25
CN106982409B (zh) 2019-11-26
US10142741B2 (en) 2018-11-27
DE102016200637B3 (de) 2017-04-27
US20170208397A1 (en) 2017-07-20
EP3197181A1 (fr) 2017-07-26
DK3197181T3 (da) 2019-01-21

Similar Documents

Publication Publication Date Title
EP3197181B1 (fr) Procédé de réduction du temps de latence d'un banc de filtrage destiné au filtrage d'un signal audio et procédé de fonctionnement sans latence d'un système auditif
DE69319494T2 (de) Kodierungsvorrichtung für Audiosignalen und Verfahren dazu
EP1853089B1 (fr) Méthode pour la suppression de la rétroaction et pour l'éxpansion spéctrale pour des appareils de correction auditive
EP1005695B1 (fr) Procede et dispositif permettant de detecter un transitoire dans un signal audio discontinu, et dispositif et procede de codage d'un signal audio
DE10017646A1 (de) Geräuschunterdrückung im Zeitbereich
DE2207141C3 (de) Schaltungsanordnung zur Unterdrückung unerwünschter Sprachsignale mittels eines vorhersagenden Filters
EP1386307B1 (fr) Procede et dispositif pour determiner un niveau de qualite d'un signal audio
EP1189419B1 (fr) Procede et appareil pour eliminer l'interference d'un haut-parleur sur de signaux de microphone
EP3454572A1 (fr) Procédé de reconnaissance d'un défaut dans un appareil auditif
EP3926982A2 (fr) Procédé de réduction directionnelle du bruit pour un système auditif comprenant un dispositif auditif
EP3565270B1 (fr) Procédé de réduction du bruit dans un signal audio
EP2981099B1 (fr) Procede et dispositif de suppression de l'effet larsen
EP3065417B1 (fr) Procede de suppression d'un bruit parasite dans un systeme acoustique
DE102015204253B4 (de) Verfahren zur frequenzabhängigen Rauschunterdrückung eines Eingangssignals sowie Hörgerät
EP3355592A1 (fr) Procédé de fonctionnement d'un système pour appareils de correction auditive binaural
DE602004006912T2 (de) Verfahren zur Verarbeitung eines akustischen Signals und ein Hörgerät
EP3340656B1 (fr) Procédé de fonctionnement d'un dispositif de correction auditive
EP1351550B1 (fr) Procédé d'adaptation d'une amplification de signal dans une prothèse auditive et prothèse auditive
EP3403260B1 (fr) Procédé et dispositif de mise en forme d'un signal audio comprimé avec perte
EP3048813B1 (fr) Procédé et dispositif de suppression du bruit basée sur l'inter-corrélation de bandes secondaires
EP2190218B1 (fr) Système d'ensemble de filtres avec atténuation de bande affaiblie spécifique pour un dispositif auditif
DE102005015647A1 (de) Kompandersystem
DE102018207780B3 (de) Verfahren zum Betrieb eines Hörgerätes
DE102016212393A1 (de) Rechnerisch wirkungsvoller Datenratenfehlanpassungsausgleich für Telefonietaktgeber
EP4235664A1 (fr) Procédé de suppression d'une réverbération acoustique dans un signal audio

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

17P Request for examination filed

Effective date: 20171127

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: SIVANTOS PTE. LTD.

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

RIC1 Information provided on ipc code assigned before grant

Ipc: H04R 25/00 20060101AFI20180209BHEP

INTG Intention to grant announced

Effective date: 20180308

GRAJ Information related to disapproval of communication of intention to grant by the applicant or resumption of examination proceedings by the epo deleted

Free format text: ORIGINAL CODE: EPIDOSDIGR1

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20180706

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1047470

Country of ref document: AT

Kind code of ref document: T

Effective date: 20181015

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 502016002092

Country of ref document: DE

REG Reference to a national code

Ref country code: CH

Ref legal event code: NV

Representative=s name: E. BLUM AND CO. AG PATENT- UND MARKENANWAELTE , CH

REG Reference to a national code

Ref country code: DK

Ref legal event code: T3

Effective date: 20190115

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20180926

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181226

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181227

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180926

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180926

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181226

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180926

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180926

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180926

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180926

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180926

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180926

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180926

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180926

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190126

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180926

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180926

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180926

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180926

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190126

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180926

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 502016002092

Country of ref document: DE

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180926

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20181215

26N No opposition filed

Effective date: 20190627

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20181231

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20181215

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180926

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20181231

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180926

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180926

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180926

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20161215

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180926

Ref country code: MK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180926

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 1047470

Country of ref document: AT

Kind code of ref document: T

Effective date: 20211215

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20211215

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20231220

Year of fee payment: 8

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20231219

Year of fee payment: 8

Ref country code: DK

Payment date: 20231219

Year of fee payment: 8

Ref country code: DE

Payment date: 20231214

Year of fee payment: 8

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: CH

Payment date: 20240110

Year of fee payment: 8