EP0945854B1 - Vorrichtung zur Sprachdetektion bei Umgebungsgeräuschen - Google Patents
Vorrichtung zur Sprachdetektion bei Umgebungsgeräuschen Download PDFInfo
- Publication number
- EP0945854B1 EP0945854B1 EP99301823A EP99301823A EP0945854B1 EP 0945854 B1 EP0945854 B1 EP 0945854B1 EP 99301823 A EP99301823 A EP 99301823A EP 99301823 A EP99301823 A EP 99301823A EP 0945854 B1 EP0945854 B1 EP 0945854B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- speech
- threshold
- band
- thresholds
- state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Lifetime
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 35
- 230000003044 adaptive effect Effects 0.000 claims abstract description 15
- 238000000034 method Methods 0.000 claims description 10
- 230000003111 delayed effect Effects 0.000 claims description 8
- 230000008859 change Effects 0.000 claims description 5
- 230000002401 inhibitory effect Effects 0.000 claims 1
- 230000007704 transition Effects 0.000 abstract description 7
- 230000007774 longterm Effects 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 14
- 230000007246 mechanism Effects 0.000 description 9
- 238000012545 processing Methods 0.000 description 6
- 238000005070 sampling Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000001228 spectrum Methods 0.000 description 3
- 238000009499 grossing Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000003139 buffering effect Effects 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 230000003292 diminished effect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/78—Detection of presence or absence of voice signals
- G10L25/84—Detection of presence or absence of voice signals for discriminating voice from noise
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/78—Detection of presence or absence of voice signals
- G10L25/87—Detection of discrete points within a voice signal
Definitions
- the present invention relates generally to speech processing and speech recognizing systems. More particularly, the invention relates to a detection system for detecting the beginning and ending of speech within an input signal.
- Speech recognition for speech recognition and for other purposes, is currently one of the most challenging tasks a computer can perform.
- Speech recognition for example, employs a highly complex pattern-matching technology that can be very sensitive to variability.
- recognition systems need to be able to handle a diverse range of different speakers and need to operate under widely varying environmental conditions. The presence of extraneous signals and noise can greatly degrade recognition quality and speech-processing performance.
- EP-A-0 322 797 discloses a method for extracting isolated speech words in which the speech signal is divided into low and high frequency bands, the power levels of which are independently compared to respective thresholds.
- the present invention divides the incoming signal into frequency bands, each band representing a different range of frequencies.
- the short-term energy within each band is then compared with a plurality of thresholds and the results of the comparison are used to drive a state machine that switches from a "speech absent" state to a "speech present” state when the band-limited signal energy of at least one of the bands is above at least one of its associated thresholds.
- the state machine similarly switches from a "speech present” state to a "speech absent” state when the band-limited signal energy of at least one of the bands is below at least one of its associated thresholds.
- the system also includes a partial speech detection mechanism based on an assumed "silence segment" prior to the actual beginning of speech.
- a histogram data structure accumulates long-term data concerning the mean and variance of energy within the frequency bands, and this information is used to adjust adaptive thresholds.
- the frequency bands are allocated based on noise characteristics.
- the histogram representation affords strong discrimination between speech signal, silence and noise, respectively.
- the silence part within the speech signal itself, the silence part (with only background noise) typically dominates, and it is reflected strongly on the histogram. Background noise, being comparatively constant, shows up as noticeable spikes on the histogram.
- the system is well adapted to detecting speech in noisy conditions and it will detect both the beginning and end of speech as well as handling situations where the beginning of speech may have been lost through truncation.
- the present invention separates the input signal into multiple signal paths, each representing a different frequency band.
- Figure 1 illustrates one embodiment of the invention employing two bands, one band corresponding to the entire frequency spectrum of the input signal and the other band corresponding to a high frequency subset of the entire frequency spectrum.
- the illustrated embodiment is particularly suited to examining input signals having a low signal-to-noise ratio (SNR), such as for conditions found within a moving motor vehicle or within a noisy office environment. In these common environments, much of the noise energy is distributed below 2,000 Hz.
- SNR signal-to-noise ratio
- the input signal containing a possible speech signal as well as noise has been represented at 20 .
- the input signal is digitized and processed through a hamming window 22 to subdivide the input signal data into frames.
- the presently preferred embodiment employs a 10 ms frame of a predefined sampling rate (in this case 8,000 Hz.), resulting in 80 digital samples per frame.
- the output of hamming window 22 is a sequence of digital samples representing the input signal (speech plus noise) and arranged into frames of a predetermined size. These frames are then fed to the fast Fourier transform (FFT) converter 24 , which transforms the input signal data from the time domain into the frequency domain. At this point the signal is split into plural paths, a first path at 26 and a second path at 28.
- the first path corresponds to a frequency band containing all frequencies of the input signal, while the second path 28 corresponds to a high-frequency subset of the full spectrum of the input signal. Because the frequency domain content is represented by digital data, the frequency band splitting is accomplished by the summation modules 30 and 32 , respectively.
- the summation module 30 sums the spectral components over the range 10-108; whereas the summation module 32 sums over the range 64-108. In this way, the summation module 30 selects all frequency bands in the input signal, while module 32 selects only the high-frequency bands. In this case, module 32 extracts a subset of the bands selected by module 30 .
- This is the presently preferred arrangement for detecting speech content within a noisy input signal of the type commonly found in moving vehicles or noisy offices. Other noisy conditions may dictate other frequency band-splitting arrangements. For example, plural signal paths could be configured to cover individual, nonoverlapping frequency bands and partially overlapping frequency bands, as desired.
- the summation modules 30 and 32 sum the frequency components one frame at a time.
- the resultant outputs of modules 30 and 32 represent frequency band-limited, short-term energy within the signal.
- this raw data may be passed through a smoothing filter, such as filters 34 and 36.
- filters 34 and 36 In the presently preferred embodiment a 3-tap average is used as the smoothing filter in both locations.
- speech detection is based on comparing the multiple frequency band-limited, short-term energy with a plurality of thresholds. These thresholds are adaptively updated based on the long-term mean and variance of energies associated with the pre-speech silence portion (assumed to be present while the system is active but before the speaker begins speaking).
- the implementation uses a histogram data structure in generating the adaptive thresholds.
- composite blocks 38 and 40 represent the adaptive threshold updating modules for signal paths 26 and 28 , respectively. Further details of these modules will be provided in connection with Figure 2 and several of the associated waveform diagrams.
- the speech state detection modules 42 and its associated partial speech detection module 44 consider the signal energy data from both paths 26 and 28.
- the speech state module 42 implements a state machine whose details are further illustrated in Figure 4.
- the partial speech detection module is shown in greater detail in Figure 3.
- the adaptive threshold updating module 38 uses three different thresholds for each energy band. Thus in the illustrated embodiment there is a total of six thresholds. The purpose of each threshold will be made more clear by considering the waveform diagrams and the associated discussion. For each energy band the three thresholds are identified: Threshold, WThreshold and SThreshold.
- Threshold is a basic threshold used for detecting the beginning of speech.
- WThreshold is a weak threshold for detecting the ending of speech.
- the SThreshold is a strong threshold for assessing the validity of the speech detection decision.
- Threshold Noise_Level + Offset
- Variance is the short term variance, i.e., the variance of M past input frames.
- Figure 6 illustrates the relationship of the three thresholds superimposed upon an exemplary signal. Note that SThreshold is higher than Threshold, while WThreshold is generally lower than Threshold. These thresholds are based on the noise level using a histogram data structure to determine the maximum of all past input energies contained within the pre-speech silence portion of the input signal.
- Figure 5 illustrates an exemplary histogram superimposed upon a waveform illustrating an exemplary noise level. The histogram records as "Counts" the number of times the pre-speech silence portion contains a predetermined noise level energy. The histogram thus plots the number of counts (on the y-axis) as a function of the energy level (on the x-axis). Note that in the example illustrated in Figure 5, the most common (highest count) noise level energy has an energy value of E a . The value E a would correspond to a predetermined noise level energy.
- the noise level energy data recorded in the histogram (Fig. 5) is extracted from the pre-speech silence portion of the input signal.
- the audio channel supplying the input signal is live and sending data to the speech detection system before actual speech commences.
- the system is effectively sampling the energy characteristics of the ambient noise level itself.
- the presently preferred implementation uses a fixed size histogram to reduce computer memory requirements.
- Proper configuration of the histogram data structure represents a tradeoff between the desire for precise estimation (implying small histogram steps) and wide dynamic range (implying large histogram steps).
- the algorithm employed in adjusting histogram step size is described in the following pseudocode, where M is the step size (representing a range of energy values in each step of the histogram).
- the histogram step M is adapted based on mean of the assumed silence part at the beginning that are buffered in the initialization stage.
- the said mean is assumed to show the actual background noise conditions.
- the histogram step is limited to MIN_HISTOGRAM_STEP as a lower bound. This histogram step is fixed after this moment.
- the histogram is updated by inserting a new value for each. frame.
- a forgetting factor in the current implementation 0.90 is introduced for every 10 frames.
- FIG. 2 the basic block diagram of the adaptive threshold updating mechanism is illustrated.
- This block diagram illustrates the operations performed by modules 38 and 40 (Fig. 1).
- the short-term (current data) energy is stored in update buffer 50 and is also used in module 52 to update the histogram data structure as previously described.
- the update buffer is then examined by module 54 which computes the variance over the past frames of data stored in buffer 50.
- module 56 identifies the maximum energy value within the histogram (e.g., value E a in Figure 5) and supplies this to the threshold updating module 58.
- the threshold updating module uses the maximum energy value and the statistical data (variance) from module 54 to revise the primary threshold, Threshold.
- Threshold is equal to the noise level plus a predetermined offset. This offset is based on the noise level as determined by the maximum value in the histogram and upon the variance supplied by module 54 .
- the remaining thresholds, WThreshold and SThreshold are calculated from Threshold according to the equations set forth above.
- the thresholds adaptively adjust, generally tracking the noise level within the pre-speech region.
- Figure 12 illustrates this concept.
- the pre-speech region is shown at 100 and the beginning of speech is shown generally at 200 .
- the Threshold level has been superimposed. Note that the level of this threshold tracks the noise level within the pre-speech region, plus an offset.
- the Threshold (as well as the SThreshold and the WThreshold) applicable to a given speech segment will be those thresholds in effect immediately prior to the beginning of speech.
- the speech state detection and partial speech detection modules 42 and 44 will now be described. Instead of making the speech present/speech absent decision based on one frame of data, the decision is made based on the current frame plus a few frames following the current frame. With regard to beginning of speech detection, the consideration of additional frames following the current frame (look ahead) avoids the false detection in the presence of a short but strong noise pulse, such as an electric pulse. With regard to ending of speech detection, frame look ahead prevents a pause or short silence in an otherwise continuous speech signal from providing a false detection of the end of speech. This delayed decision or look ahead strategy is implemented by buffering the data in the update buffer 50 (Fig. 2) and applying the process described by the following pseudocode:
- the beginning of speech detection algorithm assumes the existence of a pre-speech silence portion of at least a given minimum length. In practice, there are times when this assumption may not be valid, such as in cases where the input signal is clipped due to signal dropout or circuit switching glitches, thereby shortening or eliminating the assumed "silence segment.” When this occurs, the thresholds may be adapted incorrectly, as the thresholds are based on noise level energy, presumably with voice signal absent Furthermore, when the input signal is clipped to the point that there is no silence segment, the speech. detection system could fail to recognize the input signal as containing speech, possibly resulting in a loss of speech in the input stage that makes the subsequent speech processing useless.
- FIG. 3 illustrates the mechanism employed by partial speech detection module 44 (Fig. 1).
- the partial speech detection mechanism works by monitoring the threshold (Threshold) to determine if there is a sudden jump in the adaptive threshold level.
- the jump detection module 60 performs this analysis by first accumulating a value indicative of the change in threshold over a series of frames. This step is performed by module 62 which generates accumulated threshold change ⁇ . This accumulated threshold change ⁇ is compared with a predetermined absolute value Athrd in module 64, and the processing proceeds through either branch 66 or branch 68, depending on whether ⁇ is greater than Athrd or not. If not, module 70 is invoked (if so module 72 is invoked).
- Modules 70 and 72 maintain separate average threshold values.
- Module 70 maintains and updates threshold value T1, corresponding to threshold values before the detected jump and module 72 maintains and updates Threshold 2 corresponding to thresholds after the jump.
- the ratio of these two thresholds (T1/T2) is then compared with a third threshold Rthrd in module 74 . If the ratio is greater than the third threshold then a ValidSpeech flag is set.
- the ValidSpeech flag is used in the speech signal state machine of Figure 4.
- Figures 9A and 9B illustrate the partial speech detection mechanism in operation.
- Figure 9A corresponds to a condition that would take the Yes branch 68 (Fig. 3)
- Figure 9B corresponds to a condition that would take the No branch 66 .
- FIG 9A note that there is a jump in the threshold from 150 to 160 . In the illustrated example this jump is greater than the absolute value Athrd.
- the jump in threshold from position 152 to position 162 represents a jump that is not greater than Athrd.
- the jump position has been illustrated by the dotted line 170 .
- the average threshold value before the jump position is designated T1 and the average threshold after the jump position is designated T2.
- the ratio T1/T2 is then compared with the ratio threshold Rthrd (block 74 in Fig. 3).
- ValidSpeech is discriminated from simply stray noise in the pre-speech region as follows. If the jump in threshold is less than Athrd, or if the ratio T1/T2 is less than Rthrd then the signal responsible for the threshold jump is recognized as noise. On the other hand, if the ratio T1/T2 is greater than Rthrd then the signal responsible for the threshold jump is treated as partial speech and it is not used to update the threshold.
- the speech signal state machine starts, as indicated at 300 in the initialization state 310 . It then proceeds to the silence state 320 , where it remains until the steps performed in the silence state dictate a transition to the speech state 330 . Once in the speech state 330 , the state machine will transition back to the silence state 320 when certain conditions are met as indicated by the steps illustrated within the speech state 330 block.
- each of the frequency band-limited short-term energy values is compared with the basic threshold, Threshold.
- Threshold the threshold applicable to signal path 26 (Fig. 1) is designated Threshold_All and the threshold applicable to signal path 28 is designated Threshold_HPF. Similar nomenclature is used for the other threshold values applied in speech state 330 .
- the Beginning Delayed Decision flag is tested. If that flag was set to TRUE, as previously discussed, a Beginning of Speech message is returned and the state machine transitions to the speech state 330 . Otherwise, the state machine remains in the silent state and the histogram data structure is updated.
- the presently preferred embodiment updates the histogram using a forgetting factor of 0.99 to cause the effect of noncurrent data to evaporate over time. This is done by multiplying existing values in the histogram by 0.99 prior to adding the Count data associated with current frame energy. In this way, the effect of historical data is gradually diminished over time.
- Processing within the speech state 330 proceeds along similar lines, although different sets of threshold values are used.
- the speech state compares the respective energies in signal paths 26 and 28 with the WThresholds. If either signal path is above the WThreshold then a similar comparison is made vis-a-vis the SThresholds. If the energy in either signal path is above the SThreshold then the ValidSpeech flag is set to TRUE. This flag is used in the subsequent comparison steps.
- Figures 10 and 11 show how the various levels affect the state machine operation.
- Figure 10 compares the simultaneous operation of both signal paths, the all-frequency band, Band_All, and the high-frequency band, Band_HPF.
- the signal wave forms are different because they contain different frequency content.
- the final range that is recognized as detected speech corresponds to the beginning of speech generated by the all-frequency band crossing the threshold at b1 and the end of speech corresponds to the crossing of the high-frequency band at e2.
- Different input waveforms would, of course, produce different results in accordance with the algorithm described in Figure 4.
- Figure 11 shows how the strong threshold, SThreshold, is used to confirm the existence of ValidSpeech in the presence of a strong noise level. As illustrated, a strong noise that falls below SThreshold is responsible for region R that would correspond to a ValidSpeech flag being set to FALSE.
- the present invention provides a system that will detect the beginning and ending of speech within an input signal, handling many problems encountered in consumer applications in noisy environments. While the invention has been described in its presently preferred form, it will be understood that the invention is capable of certain modification without departing from the scope of the invention as set forth in the appended claims.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Computational Linguistics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Telephonic Communication Services (AREA)
- Telephone Function (AREA)
- Mobile Radio Communication Systems (AREA)
- Time-Division Multiplex Systems (AREA)
- Image Analysis (AREA)
Claims (14)
- Sprachdetektionssystem zum Untersuchen eines Eingangssignals, um festzustellen, ob ein Sprachsignal anwesend oder abwesend ist, umfassend:einen Frequenzbandteiler (30, 32) zum Teilen des Eingangssignals in eine Vielzahl von Frequenzbändern, wobei jedes Band eine bandbegrenzte Signalenergie darstellt, die einem unterschiedlichen Frequenzbereich entspricht;ein Energievergleichssystem zum Vergleichen der bandbegrenzten Signalenergie der Vielzahl von Frequenzbändern mit einer Vielzahl von Schwellenwerten, so dass jedes Frequenzband mit zumindest einem diesem Band zugeordneten Schwellenwert verglichen wird; undeine Sprachsignal-Zustandsmaschine (42), die mit dem Energievergleichssystem gekoppelt ist und schaltet:(a) von einem Sprachabwesenheitszustand in einen Sprachanwesenheitszustand, wenn die bandbegrenzte Signalenergie von zumindest einem der Bänder über zumindest einem seiner zugehörigen Schwellenwerte liegt, und(b) von einem Sprachanwesenheitszustand in einen Sprachabwesenheitszustand, wenn die bandbegrenzte Signalenergie von zumindest einem der Bänder unter zumindest einem seiner zugehörigen Schwellenwerte liegt; gekennzeichnet durch:ein Mehrfachschwellenwertsystem, das definiert:einen ersten Schwellenwert als ein vorbestimmter Offset über dem Rauschgrund;einen zweiten Schwellenwert als ein vorbestimmter Prozentsatz des ersten Schwellenwertes, wobei der zweite Schwellenwert kleiner als der erste Schwellenwert ist; undeinen dritten Schwellenwert als ein vorbestimmtes Vielfaches des ersten Schwellenwertes, wobei der dritte Schwellenwert größer als der erste Schwellenwert ist; und
wobei der zweite und der dritte Schwellenwert ein Schalten von dem Sprachanwesenheitszustand in den Sprachabwesenheitszustand steuern. - System nach Anspruch 1, das ferner ein adaptives Schwellenwertaktualisierungssystem (38, 40) umfasst, das eine Histogrammdatenstruktur anwendet, um Verlaufsdaten zu sammeln, die die Energien in zumindest einem der Frequenzbänder angeben.
- System nach Anspruch 1 oder 2, das ferner ein separates adaptives Schwellenwertaktualisierungssystem umfasst, das jedem der Frequenzbänder zugeordnet ist.
- System nach Anspruch 1, 2 oder 3, das ferner ein adaptives Schwellenwertaktualisierungssystem umfasst, das die Vielzahl von Schwellenwerten auf der Grundlage des Mittelwertes und der Varianz von Energien in jedem der Frequenzbänder überarbeitet.
- System nach Anspruch 1, 2, 3 oder 4, das ferner ein Teilsprachdetektionssystem (44) umfasst, das auf einen vorbestimmten Sprung in der Änderungsrate in zumindest einem der Vielzahl von Schwellenwerten anspricht, wobei das Teilsprachdetektionssystem verhindert, dass die Zustandsmaschine in einen Sprachanwesenheitszustand schaltet, wenn das Verhältnis vor dem Sprung zu nach dem Sprung des Durchschnittswertes des einen Schwellenwertes einen vorbestimmten Wert übersteigt.
- System nach Anspruch 1, 2, 3, 4 oder 5, wobei die Zustandsmaschine von dem Sprachanwesenheitszustand in den Sprachabwesenheitszustand schaltet, wenn die bandbegrenzte Signalenergie von zumindest einem der Bänder unter dem zweiten Schwellenwert liegt und wenn die bandbegrenzte Signalenergie von zumindest einem der Bänder unter dem dritten Schwellenwert liegt.
- System nach einem der Ansprüche 1 bis 6, das ferner einen Puffer für eine verzögerte Entscheidung umfasst, der Daten speichert, die ein vorbestimmtes Zeitinkrement des Eingangssignals darstellen, und der verhindert, dass die Zustandsmaschine von dem Sprachabwesenheitszustand in den Sprachanwesenheitszustand schaltet, wenn die bandbegrenzte Signalenergie von mindestens einem der Vielzahl von Frequenzbändern zumindest einen Schwellenwert während des gesamten vorbestimmten Zeitinkrementes hindurch nicht übersteigt.
- Verfahren zum Bestimmen, ob ein Sprachsignal in einem Eingangssignal anwesend oder abwesend ist, mit den Schritten:Teilen des Eingangssignals in eine Vielzahl von Frequenzbändern, wobei jedes Band eine bandbegrenzte Signalenergie darstellt, die einem unterschiedlichen Frequenzbereich entspricht;Vergleichen der bandbegrenzten Signalenergie der Vielzahl von Frequenzbändern mit einer Vielzahl von Schwellenwerten, so dass jedes Frequenzband mit mindestens einem diesem Band zugeordneten Schwellenwert verglichen wird; undBestimmen, dass:(a) ein Sprachanwesenheitszustand vorhanden ist, wenn die bandbegrenzte Signalenergie von zumindest einem der Bänder über zumindest einem seiner zugeordneten Schwellenwerte liegt, und(b) ein Sprachabwesenheitszustand vorhanden ist, wenn die bandbegrenzte Signalenergie von zumindest einem der Bänder unter zumindest einem seiner zugeordneten Schwellenwerte liegt;Definieren:eines ersten Schwellenwertes als ein vorbestimmter Offset über dem Rauschgrund;eines zweiten Schwellenwertes als ein vorbestimmter Prozentsatz des ersten Schwellenwertes, wobei der zweite Schwellenwert kleiner als der erste Schwellenwert ist; undeines dritten Schwellenwertes als ein vorbestimmtes Vielfaches des ersten Schwellenwertes, wobei der dritte Schwellenwert größer als der erste Schwellenwert ist; undBestimmen, dass der Sprachanwesenheitszustand vorhanden ist, auf der Grundlage des ersten Schwellenwertes, undBestimmen, dass der Sprachabwesenheitszustand vorhanden ist, auf der Grundlage des zweiten und des dritten Schwellenwertes.
- Verfahren nach Anspruch 8, das ferner umfasst:Definieren von zumindest einem der Vielzahl von Schwellenwerten unter Verwendung eines Histogramms, um Verlaufsdaten zu sammeln, die die Energien in zumindest einem der Frequenzbänder angeben.
- Verfahren nach Anspruch 8 oder 9, das ferner umfasst:adaptives Aktualisieren von zumindest einem der Vielzahl von Schwellenwerten separat für jedes der Frequenzbänder.
- Verfahren nach Anspruch 8, 9 oder 10, das ferner umfasst:Überarbeiten der Vielzahl von Schwellenwerten auf der Grundlage des Mittelwertes und der Varianz von Energien in jedem der Frequenzbänder.
- Verfahren nach Anspruch 8, 9, 10 oder 11, das ferner umfasst:Detektieren eines vorbestimmten Sprunges in der Änderungsrate in zumindest einem der Vielzahl von Schwellenwerten undBestimmen, dass der Sprachanwesenheitszustand nicht vorhanden ist, wenn das Verhältnis vor dem Sprung zu nach dem Sprung des Durchschnittswertes von dem einen Schwellenwert einen vorbestimmten Wert übersteigt.
- Verfahren nach Anspruch 8, 9, 10, 11 oder 12, wobei bestimmt wird, dass der Sprachabwesenheitszustand vorhanden ist, wenn die bandbegrenzte Signalenergie von zumindest einem der Bänder über dem zweiten Schwellenwert liegt, und wenn die bandbegrenzte Signalenergie von zumindest einem der Bänder über dem dritten Schwellenwert liegt.
- Verfahren nach einem der Ansprüche 8 bis 13, das ferner umfasst:Bestimmen, dass der Sprachanwesenheitszustand nicht vorhanden ist, wenn die bandbegrenzte Signalenergie von zumindest einem der Vielzahl von Frequenzbändern zumindest einen Schwellenwert während eines ganzen vorbestimmten Zeitinkrementes hindurch nicht übersteigt.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/047,276 US6480823B1 (en) | 1998-03-24 | 1998-03-24 | Speech detection for noisy conditions |
US47276 | 1998-03-24 |
Publications (3)
Publication Number | Publication Date |
---|---|
EP0945854A2 EP0945854A2 (de) | 1999-09-29 |
EP0945854A3 EP0945854A3 (de) | 1999-12-29 |
EP0945854B1 true EP0945854B1 (de) | 2004-05-19 |
Family
ID=21948048
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP99301823A Expired - Lifetime EP0945854B1 (de) | 1998-03-24 | 1999-03-11 | Vorrichtung zur Sprachdetektion bei Umgebungsgeräuschen |
Country Status (9)
Country | Link |
---|---|
US (1) | US6480823B1 (de) |
EP (1) | EP0945854B1 (de) |
JP (1) | JPH11327582A (de) |
KR (1) | KR100330478B1 (de) |
CN (1) | CN1113306C (de) |
AT (1) | ATE267443T1 (de) |
DE (1) | DE69917361T2 (de) |
ES (1) | ES2221312T3 (de) |
TW (1) | TW436759B (de) |
Families Citing this family (80)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6873953B1 (en) * | 2000-05-22 | 2005-03-29 | Nuance Communications | Prosody based endpoint detection |
US6640208B1 (en) * | 2000-09-12 | 2003-10-28 | Motorola, Inc. | Voiced/unvoiced speech classifier |
US6754623B2 (en) * | 2001-01-31 | 2004-06-22 | International Business Machines Corporation | Methods and apparatus for ambient noise removal in speech recognition |
US7277853B1 (en) * | 2001-03-02 | 2007-10-02 | Mindspeed Technologies, Inc. | System and method for a endpoint detection of speech for improved speech recognition in noisy environments |
US20020147585A1 (en) * | 2001-04-06 | 2002-10-10 | Poulsen Steven P. | Voice activity detection |
US6721411B2 (en) * | 2001-04-30 | 2004-04-13 | Voyant Technologies, Inc. | Audio conference platform with dynamic speech detection threshold |
US6782363B2 (en) * | 2001-05-04 | 2004-08-24 | Lucent Technologies Inc. | Method and apparatus for performing real-time endpoint detection in automatic speech recognition |
US7289626B2 (en) * | 2001-05-07 | 2007-10-30 | Siemens Communications, Inc. | Enhancement of sound quality for computer telephony systems |
US7236929B2 (en) * | 2001-05-09 | 2007-06-26 | Plantronics, Inc. | Echo suppression and speech detection techniques for telephony applications |
US7277585B2 (en) * | 2001-05-25 | 2007-10-02 | Ricoh Company, Ltd. | Image encoding method, image encoding apparatus and storage medium |
JP2003087547A (ja) * | 2001-09-12 | 2003-03-20 | Ricoh Co Ltd | 画像処理装置 |
US6901363B2 (en) * | 2001-10-18 | 2005-05-31 | Siemens Corporate Research, Inc. | Method of denoising signal mixtures |
US7299173B2 (en) | 2002-01-30 | 2007-11-20 | Motorola Inc. | Method and apparatus for speech detection using time-frequency variance |
JP2007501420A (ja) * | 2003-08-01 | 2007-01-25 | コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ | ダイアログシステムの駆動方法 |
JP4587160B2 (ja) * | 2004-03-26 | 2010-11-24 | キヤノン株式会社 | 信号処理装置および方法 |
US7278092B2 (en) * | 2004-04-28 | 2007-10-02 | Amplify, Llc | System, method and apparatus for selecting, displaying, managing, tracking and transferring access to content of web pages and other sources |
JP4483468B2 (ja) * | 2004-08-02 | 2010-06-16 | ソニー株式会社 | ノイズ低減回路、電子機器、ノイズ低減方法 |
US7457747B2 (en) * | 2004-08-23 | 2008-11-25 | Nokia Corporation | Noise detection for audio encoding by mean and variance energy ratio |
US7692683B2 (en) * | 2004-10-15 | 2010-04-06 | Lifesize Communications, Inc. | Video conferencing system transcoder |
US7545435B2 (en) * | 2004-10-15 | 2009-06-09 | Lifesize Communications, Inc. | Automatic backlight compensation and exposure control |
US8149739B2 (en) * | 2004-10-15 | 2012-04-03 | Lifesize Communications, Inc. | Background call validation |
US20060106929A1 (en) * | 2004-10-15 | 2006-05-18 | Kenoyer Michael L | Network conference communications |
KR100677396B1 (ko) * | 2004-11-20 | 2007-02-02 | 엘지전자 주식회사 | 음성인식장치의 음성구간 검출방법 |
US7590529B2 (en) * | 2005-02-04 | 2009-09-15 | Microsoft Corporation | Method and apparatus for reducing noise corruption from an alternative sensor signal during multi-sensory speech enhancement |
US20060241937A1 (en) * | 2005-04-21 | 2006-10-26 | Ma Changxue C | Method and apparatus for automatically discriminating information bearing audio segments and background noise audio segments |
US20060248210A1 (en) * | 2005-05-02 | 2006-11-02 | Lifesize Communications, Inc. | Controlling video display mode in a video conferencing system |
US8170875B2 (en) | 2005-06-15 | 2012-05-01 | Qnx Software Systems Limited | Speech end-pointer |
US7664635B2 (en) * | 2005-09-08 | 2010-02-16 | Gables Engineering, Inc. | Adaptive voice detection method and system |
GB0519051D0 (en) * | 2005-09-19 | 2005-10-26 | Nokia Corp | Search algorithm |
US20070100611A1 (en) * | 2005-10-27 | 2007-05-03 | Intel Corporation | Speech codec apparatus with spike reduction |
KR100800873B1 (ko) * | 2005-10-28 | 2008-02-04 | 삼성전자주식회사 | 음성 신호 검출 시스템 및 방법 |
KR100717401B1 (ko) * | 2006-03-02 | 2007-05-11 | 삼성전자주식회사 | 역방향 누적 히스토그램을 이용한 음성 특징 벡터의 정규화방법 및 그 장치 |
CN101320559B (zh) * | 2007-06-07 | 2011-05-18 | 华为技术有限公司 | 一种声音激活检测装置及方法 |
US8319814B2 (en) | 2007-06-22 | 2012-11-27 | Lifesize Communications, Inc. | Video conferencing system which allows endpoints to perform continuous presence layout selection |
US8139100B2 (en) | 2007-07-13 | 2012-03-20 | Lifesize Communications, Inc. | Virtual multiway scaler compensation |
CN101393744B (zh) * | 2007-09-19 | 2011-09-14 | 华为技术有限公司 | 调整声音激活检测门限值的方法及装置 |
US9661267B2 (en) * | 2007-09-20 | 2017-05-23 | Lifesize, Inc. | Videoconferencing system discovery |
KR101437830B1 (ko) * | 2007-11-13 | 2014-11-03 | 삼성전자주식회사 | 음성 구간 검출 방법 및 장치 |
US8542983B2 (en) * | 2008-06-09 | 2013-09-24 | Koninklijke Philips N.V. | Method and apparatus for generating a summary of an audio/visual data stream |
CN101625857B (zh) * | 2008-07-10 | 2012-05-09 | 新奥特(北京)视频技术有限公司 | 一种自适应的语音端点检测方法 |
US8514265B2 (en) | 2008-10-02 | 2013-08-20 | Lifesize Communications, Inc. | Systems and methods for selecting videoconferencing endpoints for display in a composite video image |
US20100110160A1 (en) * | 2008-10-30 | 2010-05-06 | Brandt Matthew K | Videoconferencing Community with Live Images |
CN102272826B (zh) * | 2008-10-30 | 2015-10-07 | 爱立信电话股份有限公司 | 电话内容信号鉴别 |
WO2010101527A1 (en) * | 2009-03-03 | 2010-09-10 | Agency For Science, Technology And Research | Methods for determining whether a signal includes a wanted signal and apparatuses configured to determine whether a signal includes a wanted signal |
US8643695B2 (en) * | 2009-03-04 | 2014-02-04 | Lifesize Communications, Inc. | Videoconferencing endpoint extension |
US8456510B2 (en) * | 2009-03-04 | 2013-06-04 | Lifesize Communications, Inc. | Virtual distributed multipoint control unit |
WO2010106734A1 (ja) * | 2009-03-18 | 2010-09-23 | 日本電気株式会社 | 音声信号処理装置 |
US8305421B2 (en) * | 2009-06-29 | 2012-11-06 | Lifesize Communications, Inc. | Automatic determination of a configuration for a conference |
ES2371619B1 (es) * | 2009-10-08 | 2012-08-08 | Telefónica, S.A. | Procedimiento de detección de segmentos de voz. |
CN102044243B (zh) * | 2009-10-15 | 2012-08-29 | 华为技术有限公司 | 语音激活检测方法与装置、编码器 |
US8350891B2 (en) * | 2009-11-16 | 2013-01-08 | Lifesize Communications, Inc. | Determining a videoconference layout based on numbers of participants |
CN102201231B (zh) * | 2010-03-23 | 2012-10-24 | 创杰科技股份有限公司 | 语音侦测方法 |
JP2012058358A (ja) * | 2010-09-07 | 2012-03-22 | Sony Corp | 雑音抑圧装置、雑音抑圧方法およびプログラム |
JP5949550B2 (ja) * | 2010-09-17 | 2016-07-06 | 日本電気株式会社 | 音声認識装置、音声認識方法、及びプログラム |
EP2656341B1 (de) * | 2010-12-24 | 2018-02-21 | Huawei Technologies Co., Ltd. | Vorrichtung zur durchführung von sprachaktivitätserkennung |
DK3493205T3 (da) | 2010-12-24 | 2021-04-19 | Huawei Tech Co Ltd | Fremgangsmåde og indretning til adaptiv detektion af stemmeaktivitet i et lydindgangssignal |
US9280982B1 (en) * | 2011-03-29 | 2016-03-08 | Google Technology Holdings LLC | Nonstationary noise estimator (NNSE) |
CN102800322B (zh) * | 2011-05-27 | 2014-03-26 | 中国科学院声学研究所 | 一种噪声功率谱估计与语音活动性检测方法 |
US9280984B2 (en) | 2012-05-14 | 2016-03-08 | Htc Corporation | Noise cancellation method |
CN103455021B (zh) * | 2012-05-31 | 2016-08-24 | 科域半导体有限公司 | 改变检测***和方法 |
CN103730110B (zh) * | 2012-10-10 | 2017-03-01 | 北京百度网讯科技有限公司 | 一种检测语音端点的方法和装置 |
CN103839544B (zh) * | 2012-11-27 | 2016-09-07 | 展讯通信(上海)有限公司 | 语音激活检测方法和装置 |
US9190061B1 (en) * | 2013-03-15 | 2015-11-17 | Google Inc. | Visual speech detection using facial landmarks |
CN103413554B (zh) * | 2013-08-27 | 2016-02-03 | 广州顶毅电子有限公司 | Dsp延时调整的去噪方法及装置 |
JP6045511B2 (ja) * | 2014-01-08 | 2016-12-14 | Psソリューションズ株式会社 | 音響信号検出システム、音響信号検出方法、音響信号検出サーバー、音響信号検出装置、及び音響信号検出プログラム |
US9330684B1 (en) * | 2015-03-27 | 2016-05-03 | Continental Automotive Systems, Inc. | Real-time wind buffet noise detection |
WO2016188593A1 (en) * | 2015-05-26 | 2016-12-01 | Katholieke Universiteit Leuven | Speech recognition system and method using an adaptive incremental learning approach |
US9516373B1 (en) | 2015-12-21 | 2016-12-06 | Max Abecassis | Presets of synchronized second screen functions |
US9596502B1 (en) | 2015-12-21 | 2017-03-14 | Max Abecassis | Integration of multiple synchronization methodologies |
CN106887241A (zh) | 2016-10-12 | 2017-06-23 | 阿里巴巴集团控股有限公司 | 一种语音信号检测方法与装置 |
EP3545691B1 (de) * | 2017-01-04 | 2021-11-17 | Harman Becker Automotive Systems GmbH | Fernfeldschallerfassung |
WO2019061055A1 (zh) * | 2017-09-27 | 2019-04-04 | 深圳传音通讯有限公司 | 电子设备的测试方法及*** |
CN109767774A (zh) | 2017-11-08 | 2019-05-17 | 阿里巴巴集团控股有限公司 | 一种交互方法和设备 |
US10928502B2 (en) | 2018-05-30 | 2021-02-23 | Richwave Technology Corp. | Methods and apparatus for detecting presence of an object in an environment |
US10948581B2 (en) | 2018-05-30 | 2021-03-16 | Richwave Technology Corp. | Methods and apparatus for detecting presence of an object in an environment |
CN108962249B (zh) * | 2018-08-21 | 2023-03-31 | 广州市保伦电子有限公司 | 一种基于mfcc语音特征的语音匹配方法及存储介质 |
CN109065043B (zh) * | 2018-08-21 | 2022-07-05 | 广州市保伦电子有限公司 | 一种命令词识别方法及计算机存储介质 |
CN112687273B (zh) * | 2020-12-26 | 2024-04-16 | 科大讯飞股份有限公司 | 一种语音转写方法及装置 |
CN113345472B (zh) * | 2021-05-08 | 2022-03-25 | 北京百度网讯科技有限公司 | 语音端点检测方法、装置、电子设备及存储介质 |
CN115376513B (zh) * | 2022-10-19 | 2023-05-12 | 广州小鹏汽车科技有限公司 | 语音交互方法、服务器及计算机可读存储介质 |
Family Cites Families (28)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US3909532A (en) * | 1974-03-29 | 1975-09-30 | Bell Telephone Labor Inc | Apparatus and method for determining the beginning and the end of a speech utterance |
US4032711A (en) | 1975-12-31 | 1977-06-28 | Bell Telephone Laboratories, Incorporated | Speaker recognition arrangement |
US4052568A (en) * | 1976-04-23 | 1977-10-04 | Communications Satellite Corporation | Digital voice switch |
JPS56104399A (en) | 1980-01-23 | 1981-08-20 | Hitachi Ltd | Voice interval detection system |
US4357491A (en) * | 1980-09-16 | 1982-11-02 | Northern Telecom Limited | Method of and apparatus for detecting speech in a voice channel signal |
USRE32172E (en) | 1980-12-19 | 1986-06-03 | At&T Bell Laboratories | Endpoint detector |
FR2502370A1 (fr) | 1981-03-18 | 1982-09-24 | Trt Telecom Radio Electr | Dispositif de reduction du bruit dans un signal de parole mele de bruit |
US4410763A (en) | 1981-06-09 | 1983-10-18 | Northern Telecom Limited | Speech detector |
US4531228A (en) | 1981-10-20 | 1985-07-23 | Nissan Motor Company, Limited | Speech recognition system for an automotive vehicle |
JPS5876899A (ja) * | 1981-10-31 | 1983-05-10 | 株式会社東芝 | 音声区間検出装置 |
FR2535854A1 (fr) | 1982-11-10 | 1984-05-11 | Cit Alcatel | Procede et dispositif d'evaluation du niveau de bruit sur une voie telephonique |
JPS59139099A (ja) | 1983-01-31 | 1984-08-09 | 株式会社東芝 | 音声区間検出装置 |
US4627091A (en) | 1983-04-01 | 1986-12-02 | Rca Corporation | Low-energy-content voice detection apparatus |
JPS603700A (ja) | 1983-06-22 | 1985-01-10 | 日本電気株式会社 | 音声検出方式 |
EP0186671A4 (de) * | 1984-06-08 | 1988-11-16 | Plessey Australia | Anpassbares sprachnachweissystem. |
US4630304A (en) * | 1985-07-01 | 1986-12-16 | Motorola, Inc. | Automatic background noise estimator for a noise suppression system |
US4815136A (en) | 1986-11-06 | 1989-03-21 | American Telephone And Telegraph Company | Voiceband signal classification |
JPH01169499A (ja) | 1987-12-24 | 1989-07-04 | Fujitsu Ltd | 単語音声区間切出し方式 |
US5222147A (en) | 1989-04-13 | 1993-06-22 | Kabushiki Kaisha Toshiba | Speech recognition LSI system including recording/reproduction device |
AU633673B2 (en) * | 1990-01-18 | 1993-02-04 | Matsushita Electric Industrial Co., Ltd. | Signal processing device |
US5313531A (en) * | 1990-11-05 | 1994-05-17 | International Business Machines Corporation | Method and apparatus for speech analysis and speech recognition |
US5305422A (en) | 1992-02-28 | 1994-04-19 | Panasonic Technologies, Inc. | Method for determining boundaries of isolated words within a speech signal |
US5323337A (en) | 1992-08-04 | 1994-06-21 | Loral Aerospace Corp. | Signal detector employing mean energy and variance of energy content comparison for noise detection |
US5579431A (en) * | 1992-10-05 | 1996-11-26 | Panasonic Technologies, Inc. | Speech detection in presence of noise by determining variance over time of frequency band limited energy |
US5617508A (en) * | 1992-10-05 | 1997-04-01 | Panasonic Technologies Inc. | Speech detection device for the detection of speech end points based on variance of frequency band limited energy |
US5479560A (en) * | 1992-10-30 | 1995-12-26 | Technology Research Association Of Medical And Welfare Apparatus | Formant detecting device and speech processing apparatus |
US5459814A (en) * | 1993-03-26 | 1995-10-17 | Hughes Aircraft Company | Voice activity detector for speech signals in variable background noise |
US6266633B1 (en) * | 1998-12-22 | 2001-07-24 | Itt Manufacturing Enterprises | Noise suppression and channel equalization preprocessor for speech and speaker recognizers: method and apparatus |
-
1998
- 1998-03-24 US US09/047,276 patent/US6480823B1/en not_active Expired - Fee Related
-
1999
- 1999-03-11 AT AT99301823T patent/ATE267443T1/de not_active IP Right Cessation
- 1999-03-11 EP EP99301823A patent/EP0945854B1/de not_active Expired - Lifetime
- 1999-03-11 DE DE69917361T patent/DE69917361T2/de not_active Expired - Fee Related
- 1999-03-11 ES ES99301823T patent/ES2221312T3/es not_active Expired - Lifetime
- 1999-03-16 KR KR1019990008735A patent/KR100330478B1/ko not_active IP Right Cessation
- 1999-03-23 CN CN99104095A patent/CN1113306C/zh not_active Expired - Fee Related
- 1999-03-23 JP JP11077884A patent/JPH11327582A/ja active Pending
- 1999-03-23 TW TW088104608A patent/TW436759B/zh not_active IP Right Cessation
Also Published As
Publication number | Publication date |
---|---|
US6480823B1 (en) | 2002-11-12 |
ATE267443T1 (de) | 2004-06-15 |
CN1242553A (zh) | 2000-01-26 |
CN1113306C (zh) | 2003-07-02 |
DE69917361T2 (de) | 2005-06-02 |
ES2221312T3 (es) | 2004-12-16 |
TW436759B (en) | 2001-05-28 |
JPH11327582A (ja) | 1999-11-26 |
KR100330478B1 (ko) | 2002-04-01 |
EP0945854A3 (de) | 1999-12-29 |
EP0945854A2 (de) | 1999-09-29 |
DE69917361D1 (de) | 2004-06-24 |
KR19990077910A (ko) | 1999-10-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP0945854B1 (de) | Vorrichtung zur Sprachdetektion bei Umgebungsgeräuschen | |
US10971169B2 (en) | Sound signal processing device | |
US9916841B2 (en) | Method and apparatus for suppressing wind noise | |
US6154721A (en) | Method and device for detecting voice activity | |
US6236970B1 (en) | Adaptive speech rate conversion without extension of input data duration, using speech interval detection | |
KR950011964B1 (ko) | 신호처리장치 | |
US4630304A (en) | Automatic background noise estimator for a noise suppression system | |
CA2169424C (en) | Method and apparatus for noise reduction by filtering based on a maximum signal-to-noise ratio and an estimated noise level | |
US5970441A (en) | Detection of periodicity information from an audio signal | |
WO2009009522A1 (en) | Voice activity detector and a method of operation | |
CA2485644A1 (en) | Voice activity detection | |
EP1751740B1 (de) | System und verfahren zur plapper-geräuschdetektion | |
US7487067B2 (en) | Signal-driven recovery of a digital pulse stream | |
US8917886B2 (en) | Method of distortion-free signal compression | |
JPH06164278A (ja) | ハウリング抑制装置 | |
US8392197B2 (en) | Speaker speed conversion system, method for same, and speed conversion device | |
KR20020082643A (ko) | 고속 푸우리에 변환(fft) 및 역고속 푸우리에변환(ifft)을 이용한 송,수신기의 동기검출장치 | |
KR200237439Y1 (ko) | 고속 푸우리에 변환(fft) 및 역고속 푸우리에변환(ifft)을 이용한 송,수신기의 동기검출장치 | |
US9148520B2 (en) | Low complexity tone/voice discrimination method using a rising edge of a frequency power envelope | |
JPS63226698A (ja) | 不特定話者電話音声認識装置 | |
JPS60216399A (ja) | 音声認識装置における音声区間検出回路 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
AK | Designated contracting states |
Kind code of ref document: A2 Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE |
|
AX | Request for extension of the european patent |
Free format text: AL;LT;LV;MK;RO;SI |
|
PUAL | Search report despatched |
Free format text: ORIGINAL CODE: 0009013 |
|
AK | Designated contracting states |
Kind code of ref document: A3 Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE |
|
AX | Request for extension of the european patent |
Free format text: AL;LT;LV;MK;RO;SI |
|
17P | Request for examination filed |
Effective date: 20000502 |
|
AKX | Designation fees paid |
Free format text: AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE |
|
AXX | Extension fees paid |
Free format text: AL PAYMENT 20000502;LT PAYMENT 20000502;LV PAYMENT 20000502;MK PAYMENT 20000502;RO PAYMENT 20000502;SI PAYMENT 20000502 |
|
17Q | First examination report despatched |
Effective date: 20030221 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: 7G 10L 11/02 A |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE |
|
AX | Request for extension of the european patent |
Extension state: AL LT LV MK RO SI |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20040519 Ref country code: LI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20040519 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20040519 Ref country code: CH Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20040519 Ref country code: BE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20040519 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20040519 |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REF | Corresponds to: |
Ref document number: 69917361 Country of ref document: DE Date of ref document: 20040624 Kind code of ref document: P |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20040819 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20040819 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20040819 |
|
LTIE | Lt: invalidation of european patent or patent extension |
Effective date: 20040519 |
|
NLV1 | Nl: lapsed or annulled due to failure to fulfill the requirements of art. 29p and 29m of the patents act | ||
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
REG | Reference to a national code |
Ref country code: ES Ref legal event code: FG2A Ref document number: 2221312 Country of ref document: ES Kind code of ref document: T3 |
|
ET | Fr: translation filed | ||
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20050311 Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20050311 Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20050311 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20050331 |
|
26N | No opposition filed |
Effective date: 20050222 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20070307 Year of fee payment: 9 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20070308 Year of fee payment: 9 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: ES Payment date: 20070329 Year of fee payment: 9 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20041019 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: IT Payment date: 20070528 Year of fee payment: 9 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20070308 Year of fee payment: 9 |
|
GBPC | Gb: european patent ceased through non-payment of renewal fee |
Effective date: 20080311 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: ST Effective date: 20081125 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20081001 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20080331 |
|
REG | Reference to a national code |
Ref country code: ES Ref legal event code: FD2A Effective date: 20080312 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GB Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20080311 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: ES Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20080312 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20080311 |