EP2245620B1 - Method and means for encoding background noise information - Google Patents

Method and means for encoding background noise information Download PDF

Info

Publication number
EP2245620B1
EP2245620B1 EP09711709.7A EP09711709A EP2245620B1 EP 2245620 B1 EP2245620 B1 EP 2245620B1 EP 09711709 A EP09711709 A EP 09711709A EP 2245620 B1 EP2245620 B1 EP 2245620B1
Authority
EP
European Patent Office
Prior art keywords
background noise
frames
period
speech
energy
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP09711709.7A
Other languages
German (de)
French (fr)
Other versions
EP2245620A1 (en
Inventor
Stefan Schandl
Panji Setiawan
Herve Taddei
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Unify GmbH and Co KG
Original Assignee
Unify GmbH and Co KG
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Unify GmbH and Co KG filed Critical Unify GmbH and Co KG
Publication of EP2245620A1 publication Critical patent/EP2245620A1/en
Application granted granted Critical
Publication of EP2245620B1 publication Critical patent/EP2245620B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/012—Comfort noise or silence coding
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/16—Vocoder architecture
    • G10L19/18—Vocoders using multiple modes

Definitions

  • the invention relates to methods and means for encoding background noise information in speech signal coding methods.
  • Such a limited frequency range is also provided in many speech signal coding methods for today's digital telecommunications.
  • a bandwidth limitation of the analog signal Prior to a coding process, a bandwidth limitation of the analog signal is performed for this purpose.
  • a codec is used which, due to the described bandwidth limitation in the frequency range between 300 Hz and 3400 Hz, is also referred to below as narrow-band speech codec (Narrow Band Speech Codec).
  • the term codec is understood to mean both the coding rule for the digital coding of audio signals and the decoding rule for the decoding of data with the aim of reconstructing the audio signal.
  • a narrowband speech codec is known, for example, from ITU-T Recommendation G.729.
  • a transmission of a narrowband speech signal with a data rate of 8 kbit / s is provided.
  • broadband speech codecs Wide Band Speech Codec
  • Such an extended frequency range is e.g. between a frequency of 50 Hz and 7000 Hz.
  • a wideband speech codec is known from ITU-T Recommendation G.729.EV.
  • coding methods for broadband speech codecs are made scalable.
  • scalability it is meant here that the transmitted coded data includes various demarcated blocks containing the narrowband portion, the wideband portion and / or the full bandwidth of the coded voice signal.
  • such a scalable design allows for backward compatibility on the receiver side and, on the other hand, offers a simple possibility of adapting the data rate and the size of transmitted data frames in the transmission channel in the case of limited data transmission capacities in the transmission channel.
  • a compression of the data to be transmitted For a reduction of the data transmission rate by a codec is usually provided a compression of the data to be transmitted. Compression is achieved, for example, by coding methods in which parameters for an excitation signal and filter parameters are determined for coding the speech data. The filter parameters and parameters specifying the excitation signal are then transmitted to the receiver. There, a synthetic speech signal is synthesized using the codec, which is as similar as possible to the original speech signal in terms of a subjective hearing impression. With the help of this method, also known as "analysis-by-synthesis", the determined and digitized samples are not transmitted, but determined parameters that allow a receiver-side synthesis of the speech signal.
  • a further measure for reducing the data transmission rate is provided by a method for discontinuous transmission (Discontinuous Transmission), which is also familiar in the art under the term DTX.
  • Discontinuous Transmission a method for discontinuous transmission
  • document US 2008/0027716 A1 discloses a DTX signaling.
  • the basic goal of DTX is to reduce the data transfer rate in the event of a speech break.
  • a pause detection (Voice Activity Detection, VAD) is used on the part of the transmitter, which recognizes when a certain signal level falls below a speech break.
  • VAD Voice Activity Detection
  • Comfort noise is noise that is synthesized to fill silence phases on the receiver's side.
  • the comfort noise serves as a subjective impression of a continuing connection, without claiming the data transmission rate intended for the transmission of speech signals. In other words, less effort is required to code the speech data for the transmitter-side coding of the noise. For a receiver-side still perceived as realistic synthesizing the comfort noise data are transmitted at a much lower data rate.
  • the data transmitted here will be Also referred to in the art as SID (Silence Insertion Description).
  • discontinuous transmission methods provide for transmission of SID frames with updated background noise characterization parameters only when significant changes in background noise energy during an inactive speech period (silence) are detected by the encoder. This applies to both narrowband (50Hz to 4kHz) and wideband speech codecs which support discontinuous transmission techniques.
  • an Energy Threshold specified in the decoder is used. This means that no SID frames are sent if the defined energy limit is not exceeded.
  • a suspension of transmission of SID frames is regarded as idle channel.
  • additional data exchange may be required to indicate that the connection is to be maintained.
  • a well-known additional data exchange currently takes place in such a way that administrative bodies in the network management of the transmission network are the sending node, i. request the sending encoder to retransmit the last transmitted SID frame if the elapsed idle period to the last SID frame sent is considered too long for the corresponding connection. For such retransmission, parameters of the retransmitted SID frame are not updated. The encoder does not perform any additional actions.
  • the object of the invention is to provide an improved implementation of the discontinuous transmission in scalable speech codecs.
  • a basic idea of the invention is to design the encoder of a speech codec in such a way that, after a previously determined idle period, it carries out a new determination or calculation of parameters via the background noise, in particular the averaged energy and the autocorrelation function. Said determination of the background noise parameters in other words corresponds to an encoding of the noise signal.
  • Administrative authorities in the network inform the encoder about the idle time set in the transmission network. The encoder thus determines the idle time, for example by requesting administrative bodies in the transmission network. Such a request is necessary only once if the determined idle time is stored by the encoder.
  • An advantage of the method according to the invention is that in order to decide whether to send updated background noise parameters in the form of an updated SID frame, no comparison of the energy of the background noise signal with an energy limit value is required.
  • the method thus saves computational resources compared with the known methods.
  • Another advantage is that the set period of time between two SID frames complies with the requirements of the respective transmission network.
  • An advantageous embodiment of the invention provides a SID structure (SID Bitstream Structure) in which the narrowband portion of the background noise information is separated from the broadband portion of the background noise information.
  • SID Bitstream Structure SID Bitstream Structure
  • Separate handling of narrowband and broadband background noise information in a SID frame enables separate encoding of the narrowband and wideband portions of the background noise and makes the processing transparent.
  • This embodiment also has the advantage that it can be determined on the receiver side whether comfort noise is based on the broadband component of the transmitted data SID framework or on the basis of the narrowband component. This is of particular advantage for the receiver-side acoustic reception in a situation where the transmission rate for speech information frames has been reduced such that only narrowband speech information is transmitted.
  • An advantageous embodiment of the invention provides that energy and autocorrelation function of the background noise are determined to determine the background noise parameters of the narrowband first portion of the background noise.
  • the narrowband portion requires averaging over a relatively long period of speech break, in practice over a period of e.g. 100 ms.
  • the calculation quantities used according to this embodiment include the energy (not the logarithmic energy) and the autocorrelation function.
  • an additional overhang period (hangover period) is introduced according to a further advantageous embodiment of the invention.
  • the newly introduced overhang period in the following: DTX overhang period serves another previously unknown purpose compared to the previously known VAD overhang period (Voice Activity Detection). While both types of overhanging period aim to identify multiple frames as active speech frames and thus avoid misclassification at the end of a speech signal, the DTX overhang period has the additional purpose of gathering information about the background noise.
  • An advantageous embodiment of the invention provides that the broadband second component is attenuated.
  • the attenuation of the broadband component plays a role in the attenuation of the entire energy component in the broadband component. This measure is necessary due to the fact that the generator for generating (synthesizing) the comfort noise in the decoder is unable to produce the same noise characteristics as the original background noise in the encoder.
  • An advantageous embodiment of the invention provides that is applied to the entire background noise signal, ie the combination of broadband and narrowband portion, a downstream emphasis reduction filter ("De-emphasis Post Filter”).
  • the "De-Emphasis Post Filter” leads to a de-emphasis of the energy and the higher frequency components. Since the averaging deforms the spectral envelope in a certain way, this attenuation can advantageously contribute to reducing the disturbing effect of a disturbed broadband noise on a human receiver.
  • the single FIGURE shows a temporal representation of a transition from a classified as a speech to a classified as background noise input signal to a decoder.
  • the inventive method provides for designing the encoder so that it recalculates the averaged energy and the autocorrelation function after a certain given time. Administrative authorities in the network inform the encoder about the required idle time.
  • SID SID Bitstream Structure
  • the calculation quantities used include the energy (not the logarithmized energy) and the autocorrelation function.
  • the autocorrelation function is used for a spectral envelope presentation.
  • An overall amplification factor can be compensated by a combination of all amplification and averaging methods.
  • the values for the autocorrelation function are normalized by summing or averaging (Equally Weighted). This affects all SID frames.
  • a relatively long averaging of the narrowband portion results in smoothening of the narrowband energy and the spectral envelope, so that a sudden change in energy does not significantly affect the synthesizing of comfort noise in the receiver.
  • the same averaging period is used for both energy and averaging of the spectral envelope after a first SID frame is generated after a voice burst is applied. This measure assures a more consistent estimation of the narrowband background noise during a transition from a speech period to a speech pause.
  • FIG. 1 shows a speech signal (Speech Burst) that at a certain time t a certain signal level, threshold, shown in the drawing as a dashed line, falls below.
  • the ordinate is to be understood as the level or energy value of the signal.
  • a pause detection (Voice Activity Detection, VAD) is used on the part of the transmitter, which detects when the threshold falls below a speech break.
  • VAD Voice Activity Detection
  • the VAD method provides for a known overhang period VAD-HO in which active voice frames are still transmitted and only after typically two frame lengths transitions into a mode which provides for generation of SID frames.
  • an additional overhang period DTX-HO is introduced.
  • the new overhang period DTX-HO follows the hitherto known overhang period VAD-HO, which is used as a black box.
  • VAD-HO hitherto known overhang period
  • the signal processed in the encoder is still classified as a speech signal, while at the same time a determination is already made starts from background noise parameters.
  • the data rate of the speech coding is already reduced, since no high-quality encoding is needed at the beginning of a speech break.
  • a portion of the overhead period is used for averaging the first SID frame.
  • the aforementioned embodiments preferably relate to the last frames FRAMES within an overhang period DTX-HO, VAD-HO.
  • the information of the first frames of the overhang period is preferably not used.
  • the newly introduced overhanging period DTX-HO serves a further hitherto unnoticed purpose compared to the well-known overhang period VAD-HO, which was previously motivated by the needs of voice activity detection. While both types of overhang periods DTX-HO, VAD-HO aim to identify multiple frames as active speech frames and thus avoid misclassification at the end of a speech signal, the DTX-overhang period DTX-HO has the additional purpose of providing information about the speech Raise background noise.
  • the new overhanging period DTX-HO provides additional assurance that after expiration of the overhanging period DTX-HO there will definitely be background noise and no speech signals at the input of the decoder.
  • VAD-HO could not be ruled out that it was the background signal exclusively to the adjacent signal.
  • speech fragments speech bursts
  • the new overhang period DTX-HO is used exclusively for learning the background noise.
  • an advantageous setting is e.g. to choose such that a period of two frames - cf. dashed axis FRAMES - is provided for the known overhang period VAD-HO and a period of five frames for the new overhang period DTX-HO.
  • the attenuation of the broadband component plays a role in the attenuation of the entire energy component in the broadband component. This measure is necessary due to the fact that the generator for generating (synthesizing) the comfort noise in the decoder is unable to produce the same noise characteristics as the original background noise in the encoder.
  • the output wideband speech signal ie the combination of broadband and narrowband components
  • a downstream de-emphasis postfilter This filtering mainly attenuates higher frequency components.
  • the De-Emphasis Post Filter also leads to a de-emphasis of the energy and the higher frequency components. Because the averaging deforms the spectral envelope in some way, this attenuation can help to reduce the disturbing effect of a disturbed wideband noise on a human receiver.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Computational Linguistics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Telephonic Communication Services (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Telephone Function (AREA)
  • Transmission Systems Not Characterized By The Medium Used For Transmission (AREA)

Description

Die Erfindung betrifft Verfahren und Mittel zur Enkodierung von Hintergrundrauschinformationen bei Sprachsignalkodierungsverfahren.The invention relates to methods and means for encoding background noise information in speech signal coding methods.

Für Telefongespräche ist seit den Anfängen der Telekommunikation eine Bandbreitenbeschränkung für eine analoge Sprachübertragung vorgesehen. Die Sprachübertragung erfolgt auf einem eingeschränkten Frequenzbereich von 300 Hz bis 3400 Hz.For telephone calls, a bandwidth limitation has been provided for analog voice transmission since the beginning of telecommunications. The voice transmission takes place over a restricted frequency range from 300 Hz to 3400 Hz.

Ein solcher eingeschränkter Frequenzbereich ist auch bei vielen Sprachsignalkodierungsverfahren für die heutige digitale Telekommunikation vorgesehen. Vor einem Kodiervorgang wird hierzu eine Bandbreitenbegrenzung des analogen Signals durchgeführt. Zur Kodierung und zur Dekodierung kommt dabei ein Codec zum Einsatz, welcher aufgrund der beschriebenen Bandbreitenbeschränkung im Frequenzbereich zwischen 300 Hz und 3400 Hz im Folgenden auch als schmalbandiger Sprachcodec (Narrow Band Speech Codec) bezeichnet wird. Unter dem Begriff Codec wird dabei sowohl die Kodiervorschrift zur digitalen Kodierung von Audiosignalen als auch die Dekodiervorschrift zur Dekodierung von Daten mit dem Ziel einer Rekonstruktion des Audiosignals verstanden.Such a limited frequency range is also provided in many speech signal coding methods for today's digital telecommunications. Prior to a coding process, a bandwidth limitation of the analog signal is performed for this purpose. For coding and decoding, a codec is used which, due to the described bandwidth limitation in the frequency range between 300 Hz and 3400 Hz, is also referred to below as narrow-band speech codec (Narrow Band Speech Codec). The term codec is understood to mean both the coding rule for the digital coding of audio signals and the decoding rule for the decoding of data with the aim of reconstructing the audio signal.

Ein schmalbandiger Sprachcodec ist beispielsweise aus der ITU-T-Empfehlung G.729 bekannt. Mittels der dort beschriebenen Kodiervorschrift ist eine Übertragung eines schmalbandigen Sprachsignals mit einer Datenrate von 8 kbit/s vorgesehen.A narrowband speech codec is known, for example, from ITU-T Recommendation G.729. By means of the coding rule described therein, a transmission of a narrowband speech signal with a data rate of 8 kbit / s is provided.

Weiterhin sind sogenannte breitbandige Sprachcodecs (Wide Band Speech Codec) bekannt, welche zur Verbesserung des Höreindrucks eine Kodierung eines Audiosignals in einem erweiterten Frequenzbereich vorsehen. Ein derart erweiterter Frequenzbereich liegt z.B. zwischen einer Frequenz von 50 Hz und 7000 Hz. Ein breitbandiger Sprachcodec ist beispielsweise aus der ITU-T-Empfehlung G.729.EV bekannt.Furthermore, so-called broadband speech codecs (Wide Band Speech Codec) are known, which provide for the improvement of the hearing impression coding of an audio signal in an extended frequency range. Such an extended frequency range is e.g. between a frequency of 50 Hz and 7000 Hz. For example, a wideband speech codec is known from ITU-T Recommendation G.729.EV.

Üblicherweise sind Kodierungsverfahren für breitbandige Sprachcodecs skalierbar gestaltet. Mit einer Skalierbarkeit ist hier gemeint, dass die übertragenen kodierten Daten verschiedene abgegrenzte Blöcke enthalten, welche den schmalbandigen Anteil, den breitbandigen Anteil und/oder die volle Bandbreite des kodierten Sprachsignals enthalten. Eine solche skalierbare Gestaltung gestattet einerseits eine empfängerseitige Abwärtskompatibilität und andererseits bietet sie eine einfache Möglichkeit, im Falle von eingeschränkten Datenübertragungskapazitäten im Übertragungskanal eine sender- und empfängerseitige Anpassung der Datenrate und der Größe von übertragenen Datenrahmen vorzunehmen.Usually, coding methods for broadband speech codecs are made scalable. By scalability, it is meant here that the transmitted coded data includes various demarcated blocks containing the narrowband portion, the wideband portion and / or the full bandwidth of the coded voice signal. On the one hand, such a scalable design allows for backward compatibility on the receiver side and, on the other hand, offers a simple possibility of adapting the data rate and the size of transmitted data frames in the transmission channel in the case of limited data transmission capacities in the transmission channel.

Für eine Reduzierung der Datenübertragungsrate durch einen Codec ist üblicherweise eine Komprimierung der zu übertragenden Daten vorgesehen. Eine Komprimierung wird beispielsweise durch Kodierungsverfahren erreicht, bei denen zur Kodierung der Sprachdaten Parameter für ein Anregungssignal und Filterparameter bestimmt werden. Die Filterparameter sowie das Anregungssignal spezifizierende Parameter werden dann an den Empfänger übertragen. Dort wird mithilfe des Codecs ein synthetisches Sprachsignal synthetisiert, das dem ursprünglichen Sprachsignal hinsichtlich eines subjektiven Höreindrucks möglichst ähnlich ist. Mit Hilfe dieses auch als »Analysis-by-Synthesis« bezeichneten Verfahrens werden nicht die ermittelten und digitalisierten Abtastwerte (Samples) selbst übertragen, sondern ermittelte Parameter, die eine empfängerseitige Synthese des Sprachsignals ermöglichen.For a reduction of the data transmission rate by a codec is usually provided a compression of the data to be transmitted. Compression is achieved, for example, by coding methods in which parameters for an excitation signal and filter parameters are determined for coding the speech data. The filter parameters and parameters specifying the excitation signal are then transmitted to the receiver. There, a synthetic speech signal is synthesized using the codec, which is as similar as possible to the original speech signal in terms of a subjective hearing impression. With the help of this method, also known as "analysis-by-synthesis", the determined and digitized samples are not transmitted, but determined parameters that allow a receiver-side synthesis of the speech signal.

Eine weitere Maßnahme zur Reduzierung der Datenübertragungsrate bietet ein Verfahren zur diskontinuierlichen Übertragung (Discontinuous Transmission), welches in der Fachwelt auch unter dem Begriff DTX geläufig ist. Dokument US 2008/0027716 A1 offenbart eine DTX Signalisierung. Das grundsätzliche Ziel von DTX ist eine Reduzierung der Datenübertragungsrate im Fall einer Sprechpause.A further measure for reducing the data transmission rate is provided by a method for discontinuous transmission (Discontinuous Transmission), which is also familiar in the art under the term DTX. document US 2008/0027716 A1 discloses a DTX signaling. The basic goal of DTX is to reduce the data transfer rate in the event of a speech break.

Hierzu kommt auf Seiten des Senders eine Sprechpausenerkennung (Voice Activity Detection, VAD) zum Einsatz, welche bei Unterschreiten eines bestimmten Signalpegels auf eine Sprechpause erkennt.For this purpose, a pause detection (Voice Activity Detection, VAD) is used on the part of the transmitter, which recognizes when a certain signal level falls below a speech break.

Üblicherweise wird vom Empfänger während einer Sprechpause keine völlige Stille erwartet. Im Gegenteil würde eine völlige Stille empfängerseitig zu Irritationen oder sogar zur Vermutung eines Verbindungsabbaus führen. Aus diesem Grund werden Verfahren zur Erzeugung eines sogenannten Komfortrauschen (Comfort Noise) angewandt.Usually the receiver is not expected to have complete silence during a speech break. On the contrary, a complete silence on the receiver side would lead to irritation or even to the suspicion of a breakdown of the connection. For this reason, methods for generating a so-called comfort noise (Comfort Noise) are applied.

Bei einem Komfortrauschen handelt es sich um Rauschen, welches zur Füllung von Stillephasen auf Seiten des Empfängers synthetisiert wird. Das Komfortrauschen dient einem subjektiven Eindruck einer weiter bestehenden Verbindung, ohne die für die Übertragung von Sprachsignalen vorgesehene Datenübertragungsrate zu beanspruchen. Mit anderen Worten wird zur senderseitigen Kodierung des Rauschens ein geringerer Aufwand als zur Kodierung der Sprachdaten betrieben. Für eine empfängerseitig noch als realistisch empfundene Synthetisierung des Komfortrauschens werden Daten mit einer weitaus niedrigeren Datenrate übertragen. Die hierbei übertragenen Daten werden in der Fachwelt auch als SID (Silence Insertion Description) bezeichnet.Comfort noise is noise that is synthesized to fill silence phases on the receiver's side. The comfort noise serves as a subjective impression of a continuing connection, without claiming the data transmission rate intended for the transmission of speech signals. In other words, less effort is required to code the speech data for the transmitter-side coding of the noise. For a receiver-side still perceived as realistic synthesizing the comfort noise data are transmitted at a much lower data rate. The data transmitted here will be Also referred to in the art as SID (Silence Insertion Description).

Gegenwärtige skalierbare Kodierungsverfahren für breitbandige Sprachcodecs sehen derzeit keine Verfahren zur diskontinuierlichen Übertragung vor.Current scalable coding schemes for wideband speech codecs currently do not provide for discontinuous transmission techniques.

Im Stand der Technik bestehen Probleme mit einer Anwendung des diskontinuierlichen Übertragung (DTX) in Verbindung mit einem Komfortrauschgenerator auf Empfängerseite (CNG Comfort Noise Generator).There are problems in the prior art with the use of discontinuous transmission (DTX) in conjunction with a comfort noise generator on the receiver side (CNG Comfort Noise Generator).

Derzeit bekannte Verfahren zur diskontinuierlichen Übertragung sehen eine Übermittlung SID-Rahmen mit aktualisierten Parametern zur Charakterisierung des Hintergrundrauschens nur dann vor, wenn seitens des Enkoders signifikante Änderungen in der Energie des Hintergrundrauschens während einer inaktiven Sprachperiode (Sprechpause) detektiert werden. Dies betrifft sowohl schmalbandige (50Hz bis 4kHz) als auch breitbandige Sprachcodecs, welche Verfahren zur diskontinuierlichen Übertragung unterstützen. Üblicherweise wird bei der Entscheidung, einen SID-Rahmen mit aktualisierten Parametern zu übermitteln, ein im Dekoder spezifizierter Energiegrenzwert (Energy Threshold) herangezogen. Dies führt dazu, dass bei einer Nichtüberschreitung des definierten Energiegrenzwertes keine SID-Rahmen gesendet werden. Seitens des Übertragungsnetzwerks zwischen Empfänger und Sender wird jedoch ein derartiges Aussetzen des Sendens von SID-Rahmen als Ruhezustand bzw. »Idle Channel« angesehen. Zur Gewährleistung einer Aufrechterhaltung der Verbindung (»Connection Alive«) ist dann eventuell ein zusätzlicher Datenaustausch erforderlich, um anzuzeigen, dass die Verbindung aufrechterhalten werden soll.Presently known discontinuous transmission methods provide for transmission of SID frames with updated background noise characterization parameters only when significant changes in background noise energy during an inactive speech period (silence) are detected by the encoder. This applies to both narrowband (50Hz to 4kHz) and wideband speech codecs which support discontinuous transmission techniques. Typically, when deciding to submit a SID frame with updated parameters, an Energy Threshold specified in the decoder is used. This means that no SID frames are sent if the defined energy limit is not exceeded. However, on the part of the transmission network between receiver and transmitter, such a suspension of transmission of SID frames is regarded as idle channel. To maintain the connection (Connection Alive), additional data exchange may be required to indicate that the connection is to be maintained.

Ein bekannter zusätzlich vorgesehener Datenaustausch erfolgt derzeit in der Weise, dass administrative Stellen im Netzwerkmanagement des Übertragungsnetzwerks den sendenden Knoten, d.h. den sendenden Enkoder auffordern, den zuletzt übermittelten SID-Rahmen erneut zu übermitteln, falls die verstrichene Leerlaufzeit (»Idle Period«) zum letzten gesendeten SID-Rahmen als zu lang für die entsprechende Verbindung erachtet wird. Für eine solche erneute Übermittlung werden Parameter des erneut gesendeten SID-Rahmens nicht aktualisiert. Der Enkoder führt also keine zusätzlichen Aktionen aus.A well-known additional data exchange currently takes place in such a way that administrative bodies in the network management of the transmission network are the sending node, i. request the sending encoder to retransmit the last transmitted SID frame if the elapsed idle period to the last SID frame sent is considered too long for the corresponding connection. For such retransmission, parameters of the retransmitted SID frame are not updated. The encoder does not perform any additional actions.

Aufgabe der Erfindung ist es, eine verbesserte Implementierung der diskontinuierlichen Übertragung in skalierbaren Sprachcodecs anzugeben.The object of the invention is to provide an improved implementation of the discontinuous transmission in scalable speech codecs.

Die Aufgabe wird durch den Gegenstand der unabhängigen Ansprüche gelöst.The object is solved by the subject matter of the independent claims.

Ein Grundgedanke der Erfindung besteht darin, den Enkoder eines Sprachcodecs so auszugestalten, dass dieser nach einer zuvor ermittelten Leerlaufzeit (»Idle Period«) eine erneute Ermittlung bzw. Berechnung von Parameter über das Hintergrundrauschen, insbesondere der gemittelten Energie und der Autokorrelationsfunktion, vornimmt. Die besagte Ermittlung der Hintergrundrauschparameter entspricht mit anderen Worten einer Enkodierung des Rauschsignals. Administrative Stellen im Netzwerk informieren den Enkoder dabei über die im Übertragungsnetzwerk eingestellte Leerlaufzeit. Der Enkoder bestimmt also die Leerlaufzeit z.B. durch Anfrage administrativer Stellen im Übertragungsnetzwerk. Eine solche Anfrage ist nur einmal notwendig, wenn die ermittelte Leerlaufzeit seitens des Enkoders gespeichert wird.A basic idea of the invention is to design the encoder of a speech codec in such a way that, after a previously determined idle period, it carries out a new determination or calculation of parameters via the background noise, in particular the averaged energy and the autocorrelation function. Said determination of the background noise parameters in other words corresponds to an encoding of the noise signal. Administrative authorities in the network inform the encoder about the idle time set in the transmission network. The encoder thus determines the idle time, for example by requesting administrative bodies in the transmission network. Such a request is necessary only once if the determined idle time is stored by the encoder.

Eine Einstellung eines zeitlichen Abstands für zu sendende SID-Rahmen gestattet es administrativen Stellen im Übertragungsnetzwerk, den Enkoder zu einem Senden eines aktualisierten Rahmens zu zwingen. Dies garantiert sowohl eine Aktualisierung zugunsten einer besseren Rekonstruktion des Hintergrundrauschens im CNG als auch ein zuverlässigeres Halten der Verbindung.Setting a time interval for SID frames to be sent allows administrative authorities in the transmission network to force the encoder to send an updated frame. This guarantees both an update in favor of a better reconstruction of background noise in the CNG and a more reliable hold of the connection.

Ein Vorteil des erfindungsgemäßen Verfahrens besteht darin, dass zur Entscheidung, ob aktualisierte Hintergrundrauschparameter in Form eines aktualisierten SID-Rahmen gesendet werden sollen, kein Vergleich der Energie des Hintergrundrauschsignals mit einem Energiegrenzwert erforderlich ist. Das Verfahren spart somit Rechenressourcen gegenüber den bekannten Verfahren.An advantage of the method according to the invention is that in order to decide whether to send updated background noise parameters in the form of an updated SID frame, no comparison of the energy of the background noise signal with an energy limit value is required. The method thus saves computational resources compared with the known methods.

Ein weiterer Vorteil besteht darin, dass die eingestellte Zeitdauer zwischen zwei SID-Rahmen mit den Erfordernissen des jeweiligen Übertragungsnetzwerks übereinstimmt.Another advantage is that the set period of time between two SID frames complies with the requirements of the respective transmission network.

Vorteilhafte Weiterbildungen und Ausgestaltungen der Erfindung sind Gegenstand der Unteransprüche.Advantageous developments and refinements of the invention are the subject of the dependent claims.

Eine vorteilhafte Ausgestaltung der Erfindung sieht eine SID-Struktur (SID Bitstream Structure) bei der der schmalbandige Anteil der Hintergrundrauschinformation vom breitbandigen Anteil der Hintergrundrauschinformation getrennt ist. Eine getrennte Behandlung von schmalbandiger und breitbandiger Hintergrundrauschinformation in einem SID-Rahmen ermöglicht eine getrennte Enkodierung des schmalbandigen und des breitbandigen Anteils des Hintergrundrauschens und macht die Bearbeitung transparent. Diese Ausgestaltung hat weiterhin den Vorteil, dass empfängerseitig bestimmt werden kann, ob ein Komfortrauschen auf Basis des breitbandigen Anteils der übertragenen SID-Rahmen oder auf Basis des schmalbandigen Anteils erfolgen soll. Dies ist von besonderem Vorteil für die empfängerseitige akustische Rezeption in einer Situation, in der die Übertragungsrate für Sprachinformationsrahmen derart verringert wurde, dass nur noch schmalbandige Sprachinformationen übertragen werden. Wird nämlich, wie im derzeitigen Stand der Technik, schmalbandige Sprachinformationen in Verbindung mit breitbandigem Rauschen synthetisiert, ist dies für den Empfänger sehr irritierend. Die besagte Verringerung der Übertragungsrate für Sprachinformationsrahmen kann zum Beispiel durch eine hohe Auslastung (Congestion) des Netzwerks zwischen Sender und Empfänger verursacht sein. Die wesentlich kleineren SID-Rahmen sind von einem solchen Netzwerkengpass nicht betroffen. Für sie besteht also weder ein Zwang zur Reduzierung ihrer Datenübertragungsrate noch ihres Inhalts.An advantageous embodiment of the invention provides a SID structure (SID Bitstream Structure) in which the narrowband portion of the background noise information is separated from the broadband portion of the background noise information. Separate handling of narrowband and broadband background noise information in a SID frame enables separate encoding of the narrowband and wideband portions of the background noise and makes the processing transparent. This embodiment also has the advantage that it can be determined on the receiver side whether comfort noise is based on the broadband component of the transmitted data SID framework or on the basis of the narrowband component. This is of particular advantage for the receiver-side acoustic reception in a situation where the transmission rate for speech information frames has been reduced such that only narrowband speech information is transmitted. Namely, as synthesized in the current state of the art, narrowband speech information in conjunction with broadband noise, this is very irritating for the receiver. The said reduction of the transmission rate for speech information frames can be caused, for example, by a high congestion of the network between transmitter and receiver. The much smaller SID frames are not affected by such a network bottleneck. For them, there is no compulsion to reduce their data transfer rate or their content.

Eine vorteilhafte Ausgestaltung der Erfindung sieht vor, dass zur Bestimmung der Hintergrundrauschparameter des schmalbandigen ersten Anteils des Hintergrundrauschens Energie und Autokorrelationsfunktion des Hintergrundrauschens ermittelt werden. Im schmalbandigen Anteil ist eine Mittlung über einen relativ langen Zeitraum einer Sprechpause notwendig, in der Praxis über einen Zeitraum von z.B. 100 ms. Die verwendeten Berechnungsgrößen gemäß dieser Ausführungsform umfassen dabei die Energie (nicht die logarithmierte Energie) und die Autokorrelationsfunktion.An advantageous embodiment of the invention provides that energy and autocorrelation function of the background noise are determined to determine the background noise parameters of the narrowband first portion of the background noise. The narrowband portion requires averaging over a relatively long period of speech break, in practice over a period of e.g. 100 ms. The calculation quantities used according to this embodiment include the energy (not the logarithmic energy) and the autocorrelation function.

Zu Beginn eines Zeitabschnitts, welcher als inaktiv bzw. als Sprechpause klassifiziert ist, wird gemäß einer weiteren vorteilhaften Ausgestaltung der Erfindung eine zusätzliche Überhangperiode (Hangover Period) eingeführt. Die neu eingeführte Überhangperiode, im Folgenden: DTX-Überhangperiode dient im Vergleich zur bisher bekannten VAD- Überhangperiode (Voice Activity Detection) einem weiteren bislang unbekannten Zweck. Während beide Arten von Überhangperiode das Ziel verfolgen, mehrere Rahmen als aktive Sprachrahmen zu kennzeichnen und damit eine falsche Klassifizierung am Ende eines Sprachsignals zu vermeiden, hat die DTX-Überhangperiode den zusätzlichen Zweck, Informationen über das Hintergrundrauschen zu erheben.At the beginning of a period of time which is classified as inactive or as a speech break, an additional overhang period (hangover period) is introduced according to a further advantageous embodiment of the invention. The newly introduced overhang period, in the following: DTX overhang period serves another previously unknown purpose compared to the previously known VAD overhang period (Voice Activity Detection). While both types of overhanging period aim to identify multiple frames as active speech frames and thus avoid misclassification at the end of a speech signal, the DTX overhang period has the additional purpose of gathering information about the background noise.

Eine vorteilhafte Ausgestaltung der Erfindung sieht vor, dass der breitbandige zweite Anteil gedämpft wird. Die Dämpfung des breitbandigen Anteils spielt eine Rolle bei der Dämpfung des gesamten Energieanteils im breitbandigen Anteil. Diese Maßnahme ist aufgrund der Tatsache notwendig, dass der Generator zur Erzeugung (synthetisieren) des Komfortrauschens im Dekoder nicht in der Lage ist, dieselben Rauscheigenschaften zu produzieren als das Originalhintergrundrauschen im Enkoder.An advantageous embodiment of the invention provides that the broadband second component is attenuated. The attenuation of the broadband component plays a role in the attenuation of the entire energy component in the broadband component. This measure is necessary due to the fact that the generator for generating (synthesizing) the comfort noise in the decoder is unable to produce the same noise characteristics as the original background noise in the encoder.

Eine vorteilhafte Ausgestaltung der Erfindung sieht vor, dass auf das gesamte Hintergrundrauschsignal, also der Kombination aus breitbandigen und schmalbandigen Anteil, eine nachgeschaltete Betonungsminderungsfilterung (»De-emphasis Post Filter«) angewandt wird. Das »De-Emphasis Post Filter« führt zu einer Betonungsminderung (De-Emphasis) der Energie und der höheren Frequenzkomponenten. Da die Mittelung in bestimmter Weise die spektrale Einhüllende deformiert, kann diese Dämpfung in vorteilhafter Weise dazu beitragen, den störenden Effekt eines gestörten breitbandigen Rauschens auf einen menschlichen Empfänger zu reduzieren.An advantageous embodiment of the invention provides that is applied to the entire background noise signal, ie the combination of broadband and narrowband portion, a downstream emphasis reduction filter ("De-emphasis Post Filter"). The "De-Emphasis Post Filter" leads to a de-emphasis of the energy and the higher frequency components. Since the averaging deforms the spectral envelope in a certain way, this attenuation can advantageously contribute to reducing the disturbing effect of a disturbed broadband noise on a human receiver.

Ein Ausführungsbeispiel mit weiteren Vorteilen und Ausgestaltungen der Erfindung wird im Folgenden anhand der Zeichnung näher erläutert.An embodiment with further advantages and embodiments of the invention will be explained in more detail below with reference to the drawing.

Dabei zeigt die einzige FIG eine zeitliche Darstellung eines Übergangs von einem als Sprache klassifizierten zu einem als Hintergrundrauschen klassifizierten Eingangssignal an einem Dekoder.The single FIGURE shows a temporal representation of a transition from a classified as a speech to a classified as background noise input signal to a decoder.

Im Folgenden wird der der Erfindung zugrundeliegende technische Hintergrund, zunächst ohne Bezugnahme auf die Zeichnung, näher beschrieben.In the following, the technical background underlying the invention, first without reference to the drawings, described in more detail.

Im Stand der Technik bestehen Probleme mit einer Anwendung des diskontinuierlichen Übertragung (DTX) in Verbindung mit einem Komfortrauschgenerator auf Empfängerseite (CNG Comfort Noise Generator). Während der DTX/CNG-Operation müssen folgende Überlegungen berücksichtigt werden:

  1. 1. Seitens des CNG ist eine geeignete Erzeugung des Hintergrundrauschens bzw. Komfortrauschens erforderlich, welche von einem Zuhörer auf Empfängerseite als realistisch aufgefasst werden soll. Im Falle von breitbandigen Sprachcodecs, also beispielsweise Sprachcodecs mit einer Bandbreite zwischen Frequenzen von 50 Hz und 7 kHz, wird eine Generierung eines breitbandigen Rauschens als eine Verschlechterung angesehen. Darüber hinaus ist der Charakter bzw. »die Farbe« des Hintergrundrauschens auf Dekoder- und Enkoderseite nicht immer gleich, sodass gegenwärtige Lösungen, welche eine Mittelwertbildung der Energie und der spektralen Einhüllenden vorsehen, eine Verfälschung der ursprünglichen Hintergrundrauschinformation bewirken.
  2. 2. Das DTX-Verfahren übermittelt nur dann aktualisierte SID-Rahmen wenn seitens des Enkoders signifikante Änderungen in der Energie des Hintergrundrauschens während einer inaktiven Sprachperiode (Sprechpause) detektiert werden. Dies betrifft sowohl schmalbandige (50Hz bis 4kHz) als auch breitbandige Sprachcodecs, welche das DTX/CNG-Verfahren unterstützen. Üblicherweise spielt dabei ein Energiegrenzwert (Energy Threshold) eine zentrale Rolle. Dies führt dazu, dass bei einer Nichtüberschreitung eines definierten Energiegrenzwertes keine SID-Rahmen gesendet werden. Seitens des Übertragungsnetzwerks zwischen Empfänger und Sender wird jedoch ein derartiges Aussetzen des Sendens von SID-Rahmen als Ruhezustand bzw. »Idle Channel« angesehen. Zur Gewährleistung einer Aufrechterhaltung der Verbindung (»Connection Alive«) ist dann eventuell ein zusätzlicher Datenaustausch erforderlich, um anzuzeigen, dass die Verbindung aufrechterhalten werden soll.
There are problems in the prior art with the use of discontinuous transmission (DTX) in conjunction with a comfort noise generator on the receiver side (CNG Comfort Noise Generator). During the DTX / CNG operation, the following considerations must be considered:
  1. 1. On the part of the CNG, a suitable generation of background noise or comfort noise is required, which is to be considered by a listener on the receiver side as realistic. In the case of wideband speech codecs, that is, for example, speech codecs having a bandwidth between frequencies of 50 Hz and 7 kHz, generation of broadband noise is considered a degradation. Moreover, the character or "color" of the background noise on the decoder and encoder sides is not always the same, so that current solutions that provide averaging of the energy and the spectral envelope will cause corruption of the original background noise information.
  2. 2. The DTX method transmits updated SID frames only when significant changes in background noise energy during an inactive speech period (silence) are detected by the encoder. This affects both narrowband (50Hz to 4kHz) as also broadband speech codecs which support the DTX / CNG method. Usually an energy limit (Energy Threshold) plays a central role. This means that no SID frames are sent if the defined energy limit value is not exceeded. However, on the part of the transmission network between receiver and transmitter, such a suspension of transmission of SID frames is regarded as idle channel. To maintain the connection (Connection Alive), additional data exchange may be required to indicate that the connection is to be maintained.

Derzeit wird mit dem oben genannten Problemen wie folgt umgegangen:

  • Zu 1.: Die den breitbandigen Anteil betreffende Information wird in dem SID-Rahmen enkodiert. Dabei werden die gemittelte logarithmische Energie und die gemittelte Immitance Spectral Frequency (ISF) zur Beschreibung des breitbandigen Hintergrundrauschens herangezogen, z.B. in den Sprachcodecs G.722.2 und AMR-WB. Es ist dabei keine getrennte Behandlung eines unteren Teils und eines oberen Teils des breitbandigen Hintergrundrauschens vorgesehen. Der schmalbandige Sprachcode G.729 verwendet eine gemittelte logarithmische Energie und eine gemittelte Autokorrelationsfunktion. Die Mittelungsperiode für die Energie und die Mittelungsperiode für die Autokorrelationsfunktion stimmen dabei nicht überein.
  • Zu 2.: Administrative Stellen im Netzwerkmanagement fordern den sendenden Knoten, d.h. den sendenden Enkoder auf, den zuletzt übermittelten SID-Rahmen erneut zu übermitteln, falls die »Idle Period« als zu lang für die zugehörige Verbindung erachtet wird. Der erneut gesendete SID-Rahmen und die darin enthaltene Information wird daher nicht aktualisiert. Der Enkoder führt also keine zusätzlichen Aktionen aus.
Currently, the above issues are handled as follows:
  • Re 1 .: The information concerning the broadband component is encoded in the SID frame. The averaged logarithmic energy and the averaged Immitance Spectral Frequency (ISF) are used to describe the broadband background noise, eg in the speech codecs G.722.2 and AMR-WB. There is no separate treatment of a lower part and an upper part of the broadband background noise provided. The G.729 narrowband language code uses averaged logarithmic energy and an averaged autocorrelation function. The averaging period for the energy and the averaging period for the autocorrelation function do not match.
  • Ad 2 .: Administrative authorities in the network management request the sending node, ie the sending encoder, to transmit the last transmitted SID frame again if the "idle period" is too long for the associated connection is considered. The resent SID frame and the information contained therein will therefore not be updated. The encoder does not perform any additional actions.

Das erfinderische Verfahren sieht vor, den Enkoder so auszugestalten, dass dieser nach einer bestimmten gegebenen Zeit eine erneute Berechnung der gemittelten Energie und der Autokorrelationsfunktion vornimmt. Administrative Stellen im Netzwerk informieren den Enkoder dabei über die benötigte Idle Time.The inventive method provides for designing the encoder so that it recalculates the averaged energy and the autocorrelation function after a certain given time. Administrative authorities in the network inform the encoder about the required idle time.

Im Folgenden werden weitere Ausführungsformen zur Generierung des SID-Rahmens beschrieben.In the following, further embodiments for generating the SID frame will be described.

Eine SID-Struktur (SID Bitstream Structure) bei der der schmalbandige Anteil der Hintergrundrauschinformation vom breitbandigen Anteil der Hintergrundrauschinformation getrennt ist, wird erzeugt. Eine getrennte Behandlung von schmalbandiger und breitbandiger Hintergrundrauschinformation in einem SID-Rahmen ermöglicht eine getrennte Enkodierung des schmalbandigen und des breitbandigen Anteils des Hintergrundrauschens und macht die Bearbeitung transparent.An SID (SID Bitstream Structure) structure in which the narrowband portion of the background noise information is separated from the broadband portion of the background noise information is generated. Separate handling of narrowband and broadband background noise information in a SID frame enables separate encoding of the narrowband and wideband portions of the background noise and makes the processing transparent.

Im schmalbandigen Anteil ist eine Mittlung über einen relativ langen Zeitraum einer Sprechpause notwendig, in der Praxis über einen Zeitraum von z.B. 100 ms. Die verwendeten Berechnungsgrößen umfassen dabei die Energie (nicht die logarithmierte Energie) und die Autokorrelationsfunktion. Die Autokorrelationsfunktion wird benutzt für eine spektrale Hüllkurvenpräsentation. Ein Gesamtverstärkungsfaktor kann dabei durch eine Kombination aller Verstärkungs- und Mittlungsmethoden kompensiert werden. Die Werte für die Autokorrelationsfunktion werden durch Summierung oder Mittelwertbildung jeweils normiert (Equally Weighted). Dies betrifft alle SID-Rahmen. Eine relativ lange Mittelung (Averaging) des schmalbandigen Anteils führt zu einer Glättung der schmalbandigen Energie und der spektralen Einhüllenden, sodass eine plötzliche Energieänderung keinen merklichen Einfluss auf die Synthetisierung des Komfortrauschens im Empfänger zeitigt. Dieselbe Mittelungsperiode wird sowohl für die Energie als auch zur Mittelung der spektralen Einhüllenden verwendet, nachdem ein erster SID-Rahmen nach einem Einsetzen eines Sprachsignals (Speak Burst) erzeugt wird. Diese Maßnahme sichert eine konsistentere Abschätzung des schmalbandigen Hintergrundrauschens während einem Übergang von einer Sprachperiode in eine Sprechpause.In the narrow-band component, a mediation over a relatively long period of a speech break is necessary, in practice over a period of, for example, 100 ms. The calculation quantities used include the energy (not the logarithmized energy) and the autocorrelation function. The autocorrelation function is used for a spectral envelope presentation. An overall amplification factor can be compensated by a combination of all amplification and averaging methods. The values for the autocorrelation function are normalized by summing or averaging (Equally Weighted). This affects all SID frames. A relatively long averaging of the narrowband portion results in smoothening of the narrowband energy and the spectral envelope, so that a sudden change in energy does not significantly affect the synthesizing of comfort noise in the receiver. The same averaging period is used for both energy and averaging of the spectral envelope after a first SID frame is generated after a voice burst is applied. This measure assures a more consistent estimation of the narrowband background noise during a transition from a speech period to a speech pause.

Im Folgenden wird auf die FIG Bezug genommen. Die FIG zeigt ein Sprachsignal (Speech Burst), dass zu einem bestimmten Zeitpunkt t einen bestimmten Signalpegel, Threshold, in der Zeichnung als strichlierte Linie dargestellt, unterschreitet. Die Ordinate ist als Pegel oder als Energiewert des Signals zu verstehen. Hierzu kommt auf Seiten des Senders eine Sprechpausenerkennung (Voice Activity Detection, VAD) zum Einsatz, welche bei Unterschreiten des Thresholds auf eine Sprechpause erkennt. Das VAD-Verfahren sieht eine bekannte Überhangperiode VAD-HO vor, in welcher weiterhin aktive Sprachrahmen gesendet werden und erst nach üblicherweise zwei Rahmenlängen in einen Modus überwechselt, der eine Generierung von SID-Rahmen vorsieht.In the following, reference is made to FIG. The FIG shows a speech signal (Speech Burst) that at a certain time t a certain signal level, threshold, shown in the drawing as a dashed line, falls below. The ordinate is to be understood as the level or energy value of the signal. For this purpose, a pause detection (Voice Activity Detection, VAD) is used on the part of the transmitter, which detects when the threshold falls below a speech break. The VAD method provides for a known overhang period VAD-HO in which active voice frames are still transmitted and only after typically two frame lengths transitions into a mode which provides for generation of SID frames.

Gemäß der hier beschriebenen Ausführungsform der Erfindung wird eine zusätzliche Überhangperiode DTX-HO eingeführt. Die neue Überhangperiode DTX-HO schließt sich an die bislang bekannte Überhangperiode VAD-HO an, welche als »Black Box« verwendet wird. Während dieser Überhangperiode DTX-HO wird das im Enkoder bearbeitete Signal immer noch als Sprachsignal klassifiziert, während parallel dazu bereits eine Bestimmung von Hintergrundrauschparametern beginnt. Die Datenrate der Sprachkodierung ist bereits reduziert, da zu Beginn einer Sprechpause keine hochqualitative Enkodierung benötigt wird. Weiterhin wird für den schmalbandigen Anteil ein Teil der Überhangperiode benutzt für die Mittelwertbildung des ersten SID-Rahmens. Die vorgenannten Ausführungen beziehen sich vorzugsweise auf die letzten Rahmen FRAMES innerhalb einer Überhangperiode DTX-HO, VAD-HO. Die Information der ersten Rahmen der Überhangperiode wird dagegen vorzugsweise nicht benutzt.According to the embodiment of the invention described herein, an additional overhang period DTX-HO is introduced. The new overhang period DTX-HO follows the hitherto known overhang period VAD-HO, which is used as a black box. During this overhanging period DTX-HO, the signal processed in the encoder is still classified as a speech signal, while at the same time a determination is already made starts from background noise parameters. The data rate of the speech coding is already reduced, since no high-quality encoding is needed at the beginning of a speech break. Furthermore, for the narrowband portion, a portion of the overhead period is used for averaging the first SID frame. The aforementioned embodiments preferably relate to the last frames FRAMES within an overhang period DTX-HO, VAD-HO. On the other hand, the information of the first frames of the overhang period is preferably not used.

Die neu eingeführte Überhangperiode DTX-HO dient im Vergleich zur bisher von Bedürfnissen der Voice Activity Detection motivierten bekannten Überhangperiode VAD-HO einem weiteren bislang nicht beachteten Zweck. Während beide Arten von Überhangperioden DTX-HO, VAD-HO das Ziel verfolgen, mehrere Rahmen als aktive Sprachrahmen zu kennzeichnen und damit eine falsche Klassifizierung am Ende eines Sprachsignals zu vermeiden, hat die DTX-Überhangperiode DTX-HO den zusätzlichen Zweck, Informationen über das Hintergrundrauschen zu erheben.The newly introduced overhanging period DTX-HO serves a further hitherto unnoticed purpose compared to the well-known overhang period VAD-HO, which was previously motivated by the needs of voice activity detection. While both types of overhang periods DTX-HO, VAD-HO aim to identify multiple frames as active speech frames and thus avoid misclassification at the end of a speech signal, the DTX-overhang period DTX-HO has the additional purpose of providing information about the speech Raise background noise.

Bezüglich des verfolgten Ziels, eine falsche Klassifizierung am Ende eines Sprachsignals zu vermeiden, stellt die neue Überhangperiode DTX-HO eine zusätzliche Versicherung dar, dass nach Ablauf der Überhangperiode DTX-HO definitiv ein Hintergrundrauschen und keine Sprachsignale am Eingang des Dekoders anliegen. Bei einer bisherigen Verwendung der bekannten Überhangperiode VAD-HO konnte nicht ausgeschlossen werden, dass es sich bei dem anliegenden Signal exklusiv nur um Hintergrundrauschen handelte. In der Praxis konnten während dieser bekannten Überhangperiode VAD-HO noch Sprachanteile (Speech Bursts) auftreten. Im Übrigen dient die neue Überhangperiode DTX-HO ausschließlich zur Anlernung des Hintergrundrauschens.With regard to the pursued goal of avoiding misclassification at the end of a speech signal, the new overhanging period DTX-HO provides additional assurance that after expiration of the overhanging period DTX-HO there will definitely be background noise and no speech signals at the input of the decoder. In a previous use of the known overhang period VAD-HO could not be ruled out that it was the background signal exclusively to the adjacent signal. In practice, speech fragments (speech bursts) could occur during this known overhang period VAD-HO. Incidentally, the new overhang period DTX-HO is used exclusively for learning the background noise.

Bezüglich der Wahl der Zeitdauer dieser Überhangperioden DTX-HO, VAD-HO und damit der Wahl der Anzahl an Rahmen FRAMES ist eine vorteilhafte Einstellung z.B. so zu wählen, dass eine Zeitdauer von zwei Rahmen - vgl. gestrichelte Achse FRAMES - für die bekannte Überhangperiode VAD-HO und eine Zeitdauer von fünf Rahmen für die neue Überhangperiode DTX-HO vorgesehen ist.With regard to the choice of the duration of these overhang periods DTX-HO, VAD-HO and thus the choice of the number of frames FRAMES, an advantageous setting is e.g. to choose such that a period of two frames - cf. dashed axis FRAMES - is provided for the known overhang period VAD-HO and a period of five frames for the new overhang period DTX-HO.

Im breitbandigen Anteil wird eine Energiedämpfung ausgeführt. Die Dämpfung des breitbandigen Anteils spielt eine Rolle bei der Dämpfung des gesamten Energieanteils im breitbandigen Anteil. Diese Maßnahme ist aufgrund der Tatsache notwendig, dass der Generator zur Erzeugung (synthetisieren) des Komfortrauschens im Dekoder nicht in der Lage ist, dieselben Rauscheigenschaften zu produzieren als das Originalhintergrundrauschen im Enkoder.In the broadband portion of an energy attenuation is performed. The attenuation of the broadband component plays a role in the attenuation of the entire energy component in the broadband component. This measure is necessary due to the fact that the generator for generating (synthesizing) the comfort noise in the decoder is unable to produce the same noise characteristics as the original background noise in the encoder.

Auf das ausgegebene Breitbandsprachsignal, also der Kombination aus breitbandigen und schmalbandigen Anteil, wird eine nachgeschaltete Betonungsminderungsfilterung (»De-emphasis Post Filter«) angewandt. Diese Filterung dämpft hauptsächlich höhere Frequenzkomponenten. Das »De-Emphasis Post Filter« führt weiterhin zu einer Betonungsminderung (De-Emphasis) der Energie und der höheren Frequenzkomponenten. Da die Mittelung in bestimmter Weise die spektrale Einhüllende deformiert, kann diese Dämpfung dazu beitragen, den störenden Effekt eines gestörten breitbandigen Rauschens auf einen menschlichen Empfänger zu reduzieren.The output wideband speech signal, ie the combination of broadband and narrowband components, is followed by a downstream de-emphasis postfilter. This filtering mainly attenuates higher frequency components. The De-Emphasis Post Filter also leads to a de-emphasis of the energy and the higher frequency components. Because the averaging deforms the spectral envelope in some way, this attenuation can help to reduce the disturbing effect of a disturbed wideband noise on a human receiver.

Claims (15)

  1. A method for the generation of SID frames for a discontinuous transmission of background noise parameters via a transmission network with which audio signals are transmitted by a sender to a recipient during telecommunication, said method involving a periodic determination of background noise parameters and a generation and transmission of SID frames based on the determined background noise parameters, wherein the period corresponds to a determined idle period of the transmission network that is adjusted in the transmission network.
  2. The method according to claim 1, characterized in that a speech signal is used as the audio signal.
  3. The method according to claim 1 or 2, characterized in that, in the decision as to whether the background noise parameters should be sent in the form of an SID frame, an interval of time between two SID frames to be sent is adjusted, rather than comparing the energy of a background noise signal with an energy threshold.
  4. The method according to any of claims 1 to 3, characterized in that background noise parameters of a narrow band first portion and of a wide band second portion are determined, and in that the SID frame is generated with separate areas for the first portion and the second portion.
  5. The method according to claim 4, characterized in that the recipient determines whether a comfort noise based on the wide band second portion of the transmitted SID frame or based on the narrow band first portion should occur.
  6. The method according to claim 4 or 5, characterized in that the background noise parameters of the narrow band first portion of the background noise are determined by determining an energy and an autocorrelation function of the background noise.
  7. The method according to claim 6, characterized in that the background noise parameters of the narrow band first portion are averaged over a period in 100 millisecond increments.
  8. The method according to claim 6 or 7, characterized in that the energy of the background noise, rather than a logarithmized energy of the background noise, is determined.
  9. The method according to any of the preceding claims, characterized in that during the transition from a signal categorized as speech to a signal categorized as background noise, an additional hangover period (DTX-HO) is provided, during which background noise parameters are determined.
  10. The method according to claim 9, characterized in that the additional hangover period (DTX-HO) follows a known hangover period (VAD-HO), in which active speech frames continue to be sent.
  11. The method according to claim 10, characterized in that the additional hangover period (DTX-HO) and the known hangover period (VAD-HO) each include frames, and the number of frames is selected in such a way that a duration of two frames is provided for the known hangover period (VAD-HO) and a duration of five frames is provided for the additional hangover period (DTX-HO).
  12. The method according to any of claims 4 to 9, characterized in that the wide band second portion is attenuated.
  13. The method according to any of the preceding claims, characterized in that a subsequent de-emphasis post filter is applied to the entire background noise signal.
  14. A codec having means for carrying out the method according to any of claims 1 to 13.
  15. The codec according to claim 14, characterized by an implementation in the known ITU-T Standard G.729.1(05/06).
EP09711709.7A 2008-02-19 2009-02-02 Method and means for encoding background noise information Active EP2245620B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102008009718A DE102008009718A1 (en) 2008-02-19 2008-02-19 Method and means for encoding background noise information
PCT/EP2009/051123 WO2009103610A1 (en) 2008-02-19 2009-02-02 Method and means for encoding background noise information

Publications (2)

Publication Number Publication Date
EP2245620A1 EP2245620A1 (en) 2010-11-03
EP2245620B1 true EP2245620B1 (en) 2017-08-30

Family

ID=40568601

Family Applications (1)

Application Number Title Priority Date Filing Date
EP09711709.7A Active EP2245620B1 (en) 2008-02-19 2009-02-02 Method and means for encoding background noise information

Country Status (8)

Country Link
US (1) US8949121B2 (en)
EP (1) EP2245620B1 (en)
JP (1) JP5415460B2 (en)
KR (1) KR101216496B1 (en)
CN (1) CN101952887B (en)
DE (1) DE102008009718A1 (en)
RU (1) RU2440674C1 (en)
WO (1) WO2009103610A1 (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3285253B1 (en) * 2011-01-14 2020-08-12 III Holdings 12, LLC Method for coding a speech/sound signal
CN103187065B (en) * 2011-12-30 2015-12-16 华为技术有限公司 The disposal route of voice data, device and system
US8868415B1 (en) * 2012-05-22 2014-10-21 Sprint Spectrum L.P. Discontinuous transmission control based on vocoder and voice activity
DK3550562T3 (en) * 2013-02-22 2020-11-23 Ericsson Telefon Ab L M Methods and devices for DTX hangover in audio coding
US9572103B2 (en) * 2014-09-24 2017-02-14 Nuance Communications, Inc. System and method for addressing discontinuous transmission in a network device
EP3830823B1 (en) 2018-07-27 2022-04-27 Dolby Laboratories Licensing Corporation Forced gap insertion for pervasive listening

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5881373A (en) 1996-08-28 1999-03-09 Telefonaktiebolaget Lm Ericsson Muting a microphone in radiocommunication systems
US5893056A (en) 1997-04-17 1999-04-06 Northern Telecom Limited Methods and apparatus for generating noise signals from speech signals
CA2334195A1 (en) * 1998-06-08 1999-12-16 Telefonaktiebolaget Lm Ericsson System for elimination of audible effects of handover
RU2237296C2 (en) 1998-11-23 2004-09-27 Телефонактиеболагет Лм Эрикссон (Пабл) Method for encoding speech with function for altering comfort noise for increasing reproduction precision
DE69940649D1 (en) * 1998-11-24 2009-05-07 Ericsson Telefon Ab L M Efficient in-band signaling for discontinuous transmission and configuration changes in adaptive multi-rate communication systems
DE10084675T1 (en) * 1999-06-07 2002-06-06 Ericsson Inc Method and device for generating artificial noise using parametric noise model measures
US6807525B1 (en) 2000-10-31 2004-10-19 Telogy Networks, Inc. SID frame detection with human auditory perception compensation
CN1617605A (en) 2003-11-12 2005-05-18 皇家飞利浦电子股份有限公司 Method and device for transmitting non-voice data in voice channel
WO2006030865A1 (en) * 2004-09-17 2006-03-23 Matsushita Electric Industrial Co., Ltd. Scalable encoding apparatus, scalable decoding apparatus, scalable encoding method, scalable decoding method, communication terminal apparatus, and base station apparatus
US7693708B2 (en) 2005-06-18 2010-04-06 Nokia Corporation System and method for adaptive transmission of comfort noise parameters during discontinuous speech transmission
US20070136055A1 (en) * 2005-12-13 2007-06-14 Hetherington Phillip A System for data communication over voice band robust to noise
US8725499B2 (en) 2006-07-31 2014-05-13 Qualcomm Incorporated Systems, methods, and apparatus for signal change detection
US8260609B2 (en) * 2006-07-31 2012-09-04 Qualcomm Incorporated Systems, methods, and apparatus for wideband encoding and decoding of inactive frames
US8032359B2 (en) * 2007-02-14 2011-10-04 Mindspeed Technologies, Inc. Embedded silence and background noise compression

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
KR20100123734A (en) 2010-11-24
EP2245620A1 (en) 2010-11-03
US20110004471A1 (en) 2011-01-06
JP2011515705A (en) 2011-05-19
JP5415460B2 (en) 2014-02-12
CN101952887A (en) 2011-01-19
WO2009103610A1 (en) 2009-08-27
RU2440674C1 (en) 2012-01-20
US8949121B2 (en) 2015-02-03
CN101952887B (en) 2013-05-29
DE102008009718A1 (en) 2009-08-20
DE102008009718A8 (en) 2009-12-17
KR101216496B1 (en) 2012-12-31

Similar Documents

Publication Publication Date Title
EP2245621B1 (en) Method and means for encoding background noise information
DE60117471T2 (en) BROADBAND SIGNAL TRANSMISSION SYSTEM
DE60129327T2 (en) ARRANGEMENT AND PROCEDURE RELATING TO THE LANGUAGE PROCESSING
DE60214599T2 (en) SCALABLE AUDIO CODING
EP2047669B1 (en) Method for carrying out an audio conference, audio conference device, and method for switching between encoders
EP2245620B1 (en) Method and means for encoding background noise information
DE69925168T2 (en) DETECTION OF THE ACTIVITY OF COMPLEX SIGNALS FOR IMPROVED VOICE / NOISE CLASSIFICATION FROM AN AUDIO SIGNAL
DE69631318T2 (en) Method and device for generating background noise in a digital transmission system
DE69533734T2 (en) Voice activity detection controlled noise rejection
DE60209888T2 (en) CODING AN AUDIO SIGNAL
EP1953739B1 (en) Method and device for reducing noise in a decoded signal
DE102007018484B4 (en) Apparatus and method for transmitting a sequence of data packets and decoder and apparatus for decoding a sequence of data packets
EP0978172B1 (en) Method for masking defects in a stream of audio data
DE60122751T2 (en) METHOD AND DEVICE FOR OBJECTIVE EVALUATION OF LANGUAGE QUALITY WITHOUT REFERENCE SIGNAL
DE60118631T2 (en) METHOD FOR REPLACING TRACKED AUDIO DATA
DE60224005T2 (en) METHOD AND DEVICE FOR PROCESSING MULTIPLE AUDIOBIT STREAMS
EP1023777B1 (en) Method and device for limiting a stream of audio data with a scaleable bit rate
EP1677286A1 (en) Process for adaptation of comfort noise generation parameters
DE4343366C2 (en) Method and circuit arrangement for increasing the bandwidth of narrowband speech signals
EP2245622B1 (en) Method and means for decoding background noise information
DE10252070B4 (en) Communication terminal with parameterized bandwidth extension and method for bandwidth expansion therefor
DE69921643T2 (en) AV SIGNAL TRANSMISSION WITH VARIABLE BITRATE IN A PACKET NETWORK
WO2002084929A1 (en) Method and device for the transmission of digital signals
EP1390947B1 (en) Method for signal reception
EP2745510B1 (en) Method for handling transmission errors of a video stream

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20100812

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA RS

RIN1 Information on inventor provided before grant (corrected)

Inventor name: SETIAWAN, PANJI

Inventor name: TADDEI, HERVE

Inventor name: SCHANDL, STEFAN

DAX Request for extension of the european patent (deleted)
RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: UNIFY GMBH & CO. KG

17Q First examination report despatched

Effective date: 20140129

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: UNIFY GMBH & CO. KG

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 502009014306

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: G10L0019000000

Ipc: G10L0019012000

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20170331

RIC1 Information provided on ipc code assigned before grant

Ipc: G10L 19/18 20130101ALN20170320BHEP

Ipc: G10L 19/012 20130101AFI20170320BHEP

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 924306

Country of ref document: AT

Kind code of ref document: T

Effective date: 20170915

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 502009014306

Country of ref document: DE

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20170830

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20171130

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 10

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20171130

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20171201

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20171230

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 502009014306

Country of ref document: DE

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20180531

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20180228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180202

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180228

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180228

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180202

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180228

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 924306

Country of ref document: AT

Kind code of ref document: T

Effective date: 20180202

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180202

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20090202

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170830

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 502009014306

Country of ref document: DE

Representative=s name: SCHAAFHAUSEN PATENTANWAELTE PARTNERSCHAFTSGESE, DE

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240216

Year of fee payment: 16

Ref country code: GB

Payment date: 20240222

Year of fee payment: 16

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20240221

Year of fee payment: 16