EP2760022A1 - Audio bandwidth dependent noise suppression - Google Patents
Audio bandwidth dependent noise suppression Download PDFInfo
- Publication number
- EP2760022A1 EP2760022A1 EP13153105.5A EP13153105A EP2760022A1 EP 2760022 A1 EP2760022 A1 EP 2760022A1 EP 13153105 A EP13153105 A EP 13153105A EP 2760022 A1 EP2760022 A1 EP 2760022A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- audio
- noise suppression
- audio bandwidth
- bandwidth
- gains
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000001629 suppression Effects 0.000 title claims abstract description 85
- 230000001419 dependent effect Effects 0.000 title claims abstract description 20
- 238000000034 method Methods 0.000 claims abstract description 34
- 230000005236 sound signal Effects 0.000 claims abstract description 28
- 230000006835 compression Effects 0.000 claims abstract description 15
- 238000007906 compression Methods 0.000 claims abstract description 15
- 230000015654 memory Effects 0.000 claims description 20
- 230000003044 adaptive effect Effects 0.000 claims description 2
- 230000009467 reduction Effects 0.000 abstract description 15
- 238000005070 sampling Methods 0.000 abstract description 5
- 238000004891 communication Methods 0.000 description 19
- 239000003607 modifier Substances 0.000 description 9
- 230000003595 spectral effect Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000011946 reduction process Methods 0.000 description 2
- 239000000654 additive Substances 0.000 description 1
- 230000000996 additive effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
- G10L19/16—Vocoder architecture
- G10L19/18—Vocoders using multiple modes
- G10L19/24—Variable rate codecs, e.g. for generating different qualities using a scalable representation such as hierarchical encoding or layered encoding
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L21/0264—Noise filtering characterised by the type of parameter measurement, e.g. correlation techniques, zero crossing techniques or predictive techniques
Definitions
- the present disclosure relates to the field of audio noise suppression.
- a system and method for audio bandwidth dependent noise suppression are also possible.
- Low-bandwidth (a.k.a. limited-bandwidth) communication systems typically use low-bitrate codecs that may be generally intolerant of noise. Reduction of noise is very important when the communication system is intolerant of noise. Higher (a.k.a. wider) bandwidth communication systems may tolerate more noise and may be more likely to be used for multimedia application that may include music. The application of significant noise reduction in higher bandwidth communications may create more undesirable artifacts than allowing the noise to pass through the system or applying less significant noise reduction.
- Fig. 1 is a schematic representation of a system for audio bandwidth dependent noise suppression.
- Fig. 2 is a further schematic representation of a system for audio bandwidth dependent noise suppression.
- Fig. 3 is flow diagram representing a method for audio bandwidth dependent noise suppression.
- a system and method for audio bandwidth dependent noise suppression may detect the audio bandwidth of an audio signal responsive to one or more audio indicators.
- the audio indicators may include the audio sampling rate and characteristics of an associated compression format.
- Noise suppression gains may be calculated responsive to the audio bandwidth. Noise suppression gains may mitigate undesirable noise in a reproduced output signal.
- the noise suppression gains may be modified responsive to the detected audio bandwidth. Less noise reduction may be desirable when more audio bandwidth is available.
- the modified noise suppression gains may be applied to the audio signal.
- x ( t ) and n ( t ) denote a clean audio signal, and a noise signal, respectively.
- G i , k are the noise suppression gains.
- Various methods are known in the literature to calculate these gains.
- One example further described below is a recursive Wiener filter.
- the parameter ⁇ in (3) is a constant noise floor, which defines a maximum amount of noise attenuation across all frequency bins. For example, when ⁇ is set to 0.3, the system will attenuate the noise by a maximum of 10 dB (decibel) at each frequency bin k .
- the noise reduction process may produce limited noise suppression gains that will range from 0 dB to 10 dB at each frequency bin k .
- Narrowband communication may have, for example, an audio sample rate of 8 kHz with a 4 kHz audio bandwidth.
- Wideband communication may have, for example, an audio sample rate of 16 kHz with an 8 kHz audio bandwidth.
- Full band communication may have, for example, an audio sample rate of 32 kHz or greater with a 16 kHz or greater audio bandwidth.
- the given associated audio sample rate and audio bandwidth for narrowband, wideband and full band are exemplary in nature and the values may be greater or less than the example values.
- FIG. 1 is a schematic representation of a system for audio bandwidth dependent noise suppression.
- a microphone 102 may receive a sound field that is an audible environment associated with the microphones 102. Many audible environments associated with the microphones 102 may include undesirable content that may be mitigated by processing a microphone signal output by the microphone 102 responsive to the received sound field.
- a noise suppression gain calculator 104 calculates noise suppression gains G i,k using any of various methods that are know in the literature to calculate noise suppression gains.
- a noise suppression gain applier 106 may apply the noise suppression gains to the microphone signal to mitigate undesirable content.
- An exemplary noise suppression method is a recursive Wiener filter.
- the background noise estimate may include signal information from previously processed frames.
- the spectral magnitude of the background noise may be calculated using the background noise estimation techniques disclosed in U.S. Patent No. 7,844,453 , which is incorporated in its entirety herein by reference, except that in the event of any inconsistent disclosure or definition from the present specification, the disclosure or definition herein shall be deemed to prevail.
- alternative background noise estimation techniques may be used, such as a noise power estimation technique based on minimum statistics.
- One or more audio indicators 110 may indicate the audio bandwidth.
- One audio indicator 110 of the audio bandwidth may include the audio sample rate of the microphone signal.
- the audio sampling rate utilized by the noise suppression gain calculator 104 may be an alternative audio indicator 110 as the audio signal 116 may be processed using a sample rate converter.
- Another audio indicator 110 may include the type of compression format applied to the output signal 108.
- Compression formats utilized for voice communication may include the 3 rd Generation Partnership Project (3GPP) Adaptive Multi-Rate (AMR) and 3 rd Generation Partnership Project 2 (3GPP2) Enhanced Variable Rate Codec B (EVRC-B).
- Compression formats utilized for general audio communication may include Motion Pictures Expert Group (MPEG) Advanced Audio Coding (AAC).
- MPEG Motion Pictures Expert Group
- AAC Advanced Audio Coding
- Another audio indicator 110 may include the data rate of the compression format applied to the output signal 108.
- Another audio indicator 110 may include an energy detector that detects the energy of the audio signal 116 at various frequencies.
- the energy detector may allow for an estimation of the audio bandwidth of a remote side audio signal.
- the remote side device may be capable of only narrowband audio signals where it may be desirable to increase the amount of noise reduction.
- a device that may be capable of limited audio bandwidth, for example narrowband may select a compression format suitable to the limited audio bandwidth. In this case, narrowband audio may cause a voice codec to be selected including AMR or EVRC-B.
- An audio bandwidth detector 112 may detect the audio bandwidth of the audio signal responsive to one or more audio indicators 110.
- Low-bandwidth audio communications may utilize low-bitrate compression formats, or codecs, including AMR and EVRC that may be intolerant of noise. Noise reduction may be important when codecs are intolerant of noise. Low audio-bandwidth communication may not be used for music or multimedia applications, so again, reduction of noise may be important. Higher bandwidth communication may tolerate more noise and may be more likely to be used for multimedia applications that involve music where less noise reduction may be desirable.
- the data rate and type of codec used may change the desired about of noise reduction. For example, an audio codec operating at a low data rate may be perceptibly improved by utilizing more noise suppression. In this case, more noise removal may allow the audio codec to allocate more data rate to the desired signal content.
- a noise suppression gain modifier 114 may modify the noise suppression gains responsive to the audio bandwidth detected by the audio bandwidth detector 112.
- the noise suppression gain modifier 114 may, for example, utilize a mechanism described by equation (3) where the audio bandwidth detector 112 may modify the parameter ⁇ .
- the noise suppression gain modifier 114 may produce limited noise suppression gains that may, for example, have a maximum suppression varying from 10 dB to 12 dB when the audio bandwidth detector 112 detects narrowband audio.
- the noise suppression gain modifier 114 may produce limited noise suppression gains that may, for example, have a maximum suppression varying from 6 dB to 8 dB when the audio bandwidth detector 112 detects wideband audio.
- the noise suppression gain modifier 114 may produce limited noise suppression gains that may, for example, have a maximum suppression varying from 0 dB to 6 dB when the audio bandwidth detector 112 detects full band audio.
- the audio bandwidth detector 112 may detect full band audio when a low data rate audio codec is utilized and the noise suppression gain modifier 114 may produce limited noise suppression gains that may have a maximum suppression varying from 6 dB to 10 dB.
- a subband filter may process the microphone 102 to extract frequency information.
- the subband filter may be accomplished by various methods, such as a Fast Fourier Transform (FFT), critical filter bank, octave filter band, or one-third octave filter bank.
- the subband analysis may include a time-based filter bank.
- the time-based filter bank may be composed of a bank of overlapping bandpass filters, where the center frequencies have non-linear spacing such as octave, 3 rd octave, bark, mel, or other spacing techniques.
- the noise suppression gains may be calculated for each frequency bin or band of the subband filter. The resulting noise suppression gains may be filtered, or smoothed, over time and/or frequency.
- Many communications channels may have a variable amount of available communication bandwidth over time. As the amount of communication bandwidth increases, the audio bandwidth of a signal carried by a communications channel may increase. The increased audio bandwidth may be utilized to support one or more of a higher audio sampling rate, utilizing a compression format with increased signal quality and a higher data rate for the associated compression format. Conversely, the amount of audio bandwidth may be reduced over time. The reduction in audio bandwidth may result in one or more of a lower audio sampling rate, utilizing a compression format with reduced signal quality and a lower data rate for the associated compression format.
- the audio bandwidth detector 112 may trigger the noise suppression gain modifier 114 to cause a change in amount of noise suppression responsive to the dynamic bandwidth conditions of the communication channel and thereby the audio bandwidth.
- Figure 3 is a representation of a method for audio bandwidth dependent noise suppression.
- the method 300 may be, for example, implemented using the systems 100 and 200 described herein with reference to Figure 1 and Figure 2 .
- the method 300 includes the act of detecting the audio bandwidth of an audio signal responsive to one or more audio indicators 302.
- Noise suppression gaind may be calculated responsive to the audio signal 304.
- the noise suppression gains may be modified responsive to the detected audio bandwidth 306.
- the modified noise suppression gains may be applied to the audio signal 308.
- FIG. 2 is a further schematic representation of a system for audio bandwidth dependent noise suppression.
- the system 200 comprises a processor 202, memory 204 (the contents of which are accessible by the processor 202), one or more microphones 102 and an I/O interface 206.
- the memory 204 may store instructions which when executed using the process 202 may cause the system 200 to render the functionality associated with audio bandwidth dependent noise suppression as described herein.
- the memory 204 may store instructions which when executed using the process 202 may cause the system 200 to render the functionality associated with the noise suppression gain calculator module 104, the noise suppression gain applier module 106, the audio indicators 110, the audio bandwidth detector module 112 and the noise suppression gain modifier 114 described herein.
- data structures, temporary variables and other information may store data in data storage 208.
- the processor 202 may comprise a single processor or multiple processors that may be disposed on a single chip, on multiple devices or distributed over more that one system.
- the processor 202 may be hardware that executes computer executable instructions or computer code embodied in the memory 204 or in other memory to perform one or more features of the system.
- the processor 202 may include a general purpose processor, a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a digital signal processor (DSP), a field programmable gate array (FPGA), a digital circuit, an analog circuit, a microcontroller, any other type of processor, or any combination thereof.
- the memory 204 may comprise a device for storing and retrieving data, processor executable instructions, or any combination thereof.
- the memory 204 may include non-volatile and/or volatile memory, such as a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a flash memory.
- RAM random access memory
- ROM read-only memory
- EPROM erasable programmable read-only memory
- flash memory a flash memory.
- the memory 204 may comprise a single device or multiple devices that may be disposed on one or more dedicated memory devices or on a processor or other similar device.
- the memory 204 may include an optical, magnetic (hard-drive) or any other form of data storage device.
- the memory 204 may store computer code, such as the noise suppression gain calculator module 104, the noise suppression gain applier module 106, the audio indicators 110, the audio bandwidth detector module 112 and the noise suppression gain modifier 114 described herein.
- the computer code may include instructions executable with the processor 202.
- the computer code may be written in any computer language, such as C, C++, assembly language, channel program code, and/or any combination of computer languages.
- the memory 204 may store information in data structures including, for example, noise suppression gains and state variables.
- the I/O interface 206 may be used to connect devices such as, for example, the one or more microphones 102, and to other components of the system 200.
- the system 200 may include more, fewer, or different components than illustrated in Figure 2 . Furthermore, each one of the components of system 200 may include more, fewer, or different elements than is illustrated in Figure 2 .
- Flags, data, databases, tables, entities, and other data structures may be separately stored and managed, may be incorporated into a single memory or database, may be distributed, or may be logically and physically organized in many different ways.
- the components may operate independently or be part of a same program or hardware.
- the components may be resident on separate hardware, such as separate removable circuit boards, or share common hardware, such as a same memory and processor for implementing instructions from the memory. Programs may be parts of a single program, separate programs, or distributed across several memories and processors.
- the functions, acts or tasks illustrated in the figures or described may be executed in response to one or more sets of logic or instructions stored in or on computer readable media.
- the functions, acts or tasks are independent of the particular type of instructions set, storage media, processor or processing strategy and may be performed by software, hardware, integrated circuits, firmware, micro code and the like, operating alone or in combination.
- processing strategies may include multiprocessing, multitasking, parallel processing, distributed processing, and/or any other type of processing.
- the instructions are stored on a removable media device for reading by local or remote systems.
- the logic or instructions are stored in a remote location for transfer through a computer network or over telephone lines.
- the logic or instructions may be stored within a given computer such as, for example, a CPU.
Landscapes
- Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Soundproofing, Sound Blocking, And Sound Damping (AREA)
- Circuit For Audible Band Transducer (AREA)
Abstract
Description
- The present disclosure relates to the field of audio noise suppression. In particular, to a system and method for audio bandwidth dependent noise suppression.
- Low-bandwidth (a.k.a. limited-bandwidth) communication systems typically use low-bitrate codecs that may be generally intolerant of noise. Reduction of noise is very important when the communication system is intolerant of noise. Higher (a.k.a. wider) bandwidth communication systems may tolerate more noise and may be more likely to be used for multimedia application that may include music. The application of significant noise reduction in higher bandwidth communications may create more undesirable artifacts than allowing the noise to pass through the system or applying less significant noise reduction.
- The system may be better understood with reference to the following drawings and description. The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the disclosure. Moreover, in the figures, like referenced numerals designate corresponding parts throughout the different views.
- Other systems, methods, features and advantages will be, or will become, apparent to one with skill in the art upon examination of the following figures and detailed description. It is intended that all such additional systems, methods, features and advantages be included with this description and be protected by the following claims.
-
Fig. 1 is a schematic representation of a system for audio bandwidth dependent noise suppression. -
Fig. 2 is a further schematic representation of a system for audio bandwidth dependent noise suppression. -
Fig. 3 is flow diagram representing a method for audio bandwidth dependent noise suppression. - A system and method for audio bandwidth dependent noise suppression may detect the audio bandwidth of an audio signal responsive to one or more audio indicators. The audio indicators may include the audio sampling rate and characteristics of an associated compression format. Noise suppression gains may be calculated responsive to the audio bandwidth. Noise suppression gains may mitigate undesirable noise in a reproduced output signal. The noise suppression gains may be modified responsive to the detected audio bandwidth. Less noise reduction may be desirable when more audio bandwidth is available. The modified noise suppression gains may be applied to the audio signal.
-
- where x(t) and n(t) denote a clean audio signal, and a noise signal, respectively.
- Let |Yi,k |, |Xi,k |, and |Ni,k | designate, respectively, the short-time spectral magnitudes of the noisy audio signal, the clean audio signal, and noise signal at the ith frame and the kth frequency bin. A noise reduction process may involve the application of a suppression gain Gi,k to each short-time spectral value. For the purpose of noise reduction the clean audio signal and the noise signal may both be estimates because their exact relationship may be unknown. As such, the spectral magnitude of an estimated clean audio signal is given by:
- Where Gi,k are the noise suppression gains. Various methods are known in the literature to calculate these gains. One example further described below is a recursive Wiener filter.
- A typical problem with noise reduction methods is that they create audible artifacts such as musical tones in the resulting signal, the estimated clean audio signal |X̂i,k |. These audible artifacts are due to errors in signal estimates that cause further errors in the noise suppression gains. For example, the noise signal |Ni,k | can only be estimated. To mitigate or mask the audible artifacts, the noise suppression gains may be floored (e.g. limited or constrained):
- The parameter σ in (3) is a constant noise floor, which defines a maximum amount of noise attenuation across all frequency bins. For example, when σ is set to 0.3, the system will attenuate the noise by a maximum of 10 dB (decibel) at each frequency bin k. The noise reduction process may produce limited noise suppression gains that will range from 0 dB to 10 dB at each frequency bin k.
- The noise reduction method based on the above noise suppression gain limiting applies the same maximum amount of noise attenuation to all frequencies. The constant noise floor in the noise suppression gain limiting may result in good performance for noise reduction in narrowband communication. However, it is not ideal for reducing noise in wideband and full band communications that may utilize less noise suppression gain limiting. Narrowband communication may have, for example, an audio sample rate of 8 kHz with a 4 kHz audio bandwidth. Wideband communication may have, for example, an audio sample rate of 16 kHz with an 8 kHz audio bandwidth. Full band communication may have, for example, an audio sample rate of 32 kHz or greater with a 16 kHz or greater audio bandwidth. The given associated audio sample rate and audio bandwidth for narrowband, wideband and full band are exemplary in nature and the values may be greater or less than the example values.
-
Figure 1 is a schematic representation of a system for audio bandwidth dependent noise suppression. Amicrophone 102 may receive a sound field that is an audible environment associated with themicrophones 102. Many audible environments associated with themicrophones 102 may include undesirable content that may be mitigated by processing a microphone signal output by themicrophone 102 responsive to the received sound field. A noisesuppression gain calculator 104 calculates noise suppression gains Gi,k using any of various methods that are know in the literature to calculate noise suppression gains. A noisesuppression gain applier 106 may apply the noise suppression gains to the microphone signal to mitigate undesirable content. An exemplary noise suppression method is a recursive Wiener filter. The Wiener suppression gain, or noise suppression gain, is defined as: -
-
- Where |N̂i,k | is the background noise estimate. The background noise estimate may include signal information from previously processed frames. In one implementation, the spectral magnitude of the background noise may be calculated using the background noise estimation techniques disclosed in
U.S. Patent No. 7,844,453 , which is incorporated in its entirety herein by reference, except that in the event of any inconsistent disclosure or definition from the present specification, the disclosure or definition herein shall be deemed to prevail. In other implementations, alternative background noise estimation techniques may be used, such as a noise power estimation technique based on minimum statistics. - One or more
audio indicators 110 may indicate the audio bandwidth. Oneaudio indicator 110 of the audio bandwidth may include the audio sample rate of the microphone signal. The audio sampling rate utilized by the noisesuppression gain calculator 104 may be analternative audio indicator 110 as theaudio signal 116 may be processed using a sample rate converter. Anotheraudio indicator 110 may include the type of compression format applied to theoutput signal 108. Compression formats utilized for voice communication may include the 3rd Generation Partnership Project (3GPP) Adaptive Multi-Rate (AMR) and 3rd Generation Partnership Project 2 (3GPP2) Enhanced Variable Rate Codec B (EVRC-B). Compression formats utilized for general audio communication may include Motion Pictures Expert Group (MPEG) Advanced Audio Coding (AAC). Anotheraudio indicator 110 may include the data rate of the compression format applied to theoutput signal 108. Anotheraudio indicator 110 may include an energy detector that detects the energy of theaudio signal 116 at various frequencies. The energy detector may allow for an estimation of the audio bandwidth of a remote side audio signal. For example, the remote side device may be capable of only narrowband audio signals where it may be desirable to increase the amount of noise reduction. A device that may be capable of limited audio bandwidth, for example narrowband, may select a compression format suitable to the limited audio bandwidth. In this case, narrowband audio may cause a voice codec to be selected including AMR or EVRC-B. - An
audio bandwidth detector 112 may detect the audio bandwidth of the audio signal responsive to one or moreaudio indicators 110. Low-bandwidth audio communications may utilize low-bitrate compression formats, or codecs, including AMR and EVRC that may be intolerant of noise. Noise reduction may be important when codecs are intolerant of noise. Low audio-bandwidth communication may not be used for music or multimedia applications, so again, reduction of noise may be important. Higher bandwidth communication may tolerate more noise and may be more likely to be used for multimedia applications that involve music where less noise reduction may be desirable. The data rate and type of codec used may change the desired about of noise reduction. For example, an audio codec operating at a low data rate may be perceptibly improved by utilizing more noise suppression. In this case, more noise removal may allow the audio codec to allocate more data rate to the desired signal content. - A noise
suppression gain modifier 114 may modify the noise suppression gains responsive to the audio bandwidth detected by theaudio bandwidth detector 112. The noisesuppression gain modifier 114 may, for example, utilize a mechanism described by equation (3) where theaudio bandwidth detector 112 may modify the parameter σ. The noisesuppression gain modifier 114 may produce limited noise suppression gains that may, for example, have a maximum suppression varying from 10 dB to 12 dB when theaudio bandwidth detector 112 detects narrowband audio. The noisesuppression gain modifier 114 may produce limited noise suppression gains that may, for example, have a maximum suppression varying from 6 dB to 8 dB when theaudio bandwidth detector 112 detects wideband audio. The noisesuppression gain modifier 114 may produce limited noise suppression gains that may, for example, have a maximum suppression varying from 0 dB to 6 dB when theaudio bandwidth detector 112 detects full band audio. In an alternative example, theaudio bandwidth detector 112 may detect full band audio when a low data rate audio codec is utilized and the noisesuppression gain modifier 114 may produce limited noise suppression gains that may have a maximum suppression varying from 6 dB to 10 dB. - A subband filter may process the
microphone 102 to extract frequency information. The subband filter may be accomplished by various methods, such as a Fast Fourier Transform (FFT), critical filter bank, octave filter band, or one-third octave filter bank. Alternatively, the subband analysis may include a time-based filter bank. The time-based filter bank may be composed of a bank of overlapping bandpass filters, where the center frequencies have non-linear spacing such as octave, 3rd octave, bark, mel, or other spacing techniques. The noise suppression gains may be calculated for each frequency bin or band of the subband filter. The resulting noise suppression gains may be filtered, or smoothed, over time and/or frequency. - Many communications channels may have a variable amount of available communication bandwidth over time. As the amount of communication bandwidth increases, the audio bandwidth of a signal carried by a communications channel may increase. The increased audio bandwidth may be utilized to support one or more of a higher audio sampling rate, utilizing a compression format with increased signal quality and a higher data rate for the associated compression format. Conversely, the amount of audio bandwidth may be reduced over time. The reduction in audio bandwidth may result in one or more of a lower audio sampling rate, utilizing a compression format with reduced signal quality and a lower data rate for the associated compression format. The
audio bandwidth detector 112 may trigger the noisesuppression gain modifier 114 to cause a change in amount of noise suppression responsive to the dynamic bandwidth conditions of the communication channel and thereby the audio bandwidth. -
Figure 3 is a representation of a method for audio bandwidth dependent noise suppression. Themethod 300 may be, for example, implemented using thesystems Figure 1 andFigure 2 . Themethod 300 includes the act of detecting the audio bandwidth of an audio signal responsive to one or moreaudio indicators 302. Noise suppression gaind may be calculated responsive to theaudio signal 304. The noise suppression gains may be modified responsive to the detectedaudio bandwidth 306. The modified noise suppression gains may be applied to theaudio signal 308. -
Figure 2 is a further schematic representation of a system for audio bandwidth dependent noise suppression. Thesystem 200 comprises aprocessor 202, memory 204 (the contents of which are accessible by the processor 202), one ormore microphones 102 and an I/O interface 206. Thememory 204 may store instructions which when executed using theprocess 202 may cause thesystem 200 to render the functionality associated with audio bandwidth dependent noise suppression as described herein. For example, thememory 204 may store instructions which when executed using theprocess 202 may cause thesystem 200 to render the functionality associated with the noise suppressiongain calculator module 104, the noise suppressiongain applier module 106, theaudio indicators 110, the audiobandwidth detector module 112 and the noisesuppression gain modifier 114 described herein. In addition, data structures, temporary variables and other information may store data indata storage 208. - The
processor 202 may comprise a single processor or multiple processors that may be disposed on a single chip, on multiple devices or distributed over more that one system. Theprocessor 202 may be hardware that executes computer executable instructions or computer code embodied in thememory 204 or in other memory to perform one or more features of the system. Theprocessor 202 may include a general purpose processor, a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a digital signal processor (DSP), a field programmable gate array (FPGA), a digital circuit, an analog circuit, a microcontroller, any other type of processor, or any combination thereof. - The
memory 204 may comprise a device for storing and retrieving data, processor executable instructions, or any combination thereof. Thememory 204 may include non-volatile and/or volatile memory, such as a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a flash memory. Thememory 204 may comprise a single device or multiple devices that may be disposed on one or more dedicated memory devices or on a processor or other similar device. Alternatively or in addition, thememory 204 may include an optical, magnetic (hard-drive) or any other form of data storage device. - The
memory 204 may store computer code, such as the noise suppressiongain calculator module 104, the noise suppressiongain applier module 106, theaudio indicators 110, the audiobandwidth detector module 112 and the noisesuppression gain modifier 114 described herein. The computer code may include instructions executable with theprocessor 202. The computer code may be written in any computer language, such as C, C++, assembly language, channel program code, and/or any combination of computer languages. Thememory 204 may store information in data structures including, for example, noise suppression gains and state variables. - The I/
O interface 206 may be used to connect devices such as, for example, the one ormore microphones 102, and to other components of thesystem 200. - All of the disclosure, regardless of the particular implementation described, is exemplary in nature, rather than limiting. The
system 200 may include more, fewer, or different components than illustrated inFigure 2 . Furthermore, each one of the components ofsystem 200 may include more, fewer, or different elements than is illustrated inFigure 2 . Flags, data, databases, tables, entities, and other data structures may be separately stored and managed, may be incorporated into a single memory or database, may be distributed, or may be logically and physically organized in many different ways. The components may operate independently or be part of a same program or hardware. The components may be resident on separate hardware, such as separate removable circuit boards, or share common hardware, such as a same memory and processor for implementing instructions from the memory. Programs may be parts of a single program, separate programs, or distributed across several memories and processors. - The functions, acts or tasks illustrated in the figures or described may be executed in response to one or more sets of logic or instructions stored in or on computer readable media. The functions, acts or tasks are independent of the particular type of instructions set, storage media, processor or processing strategy and may be performed by software, hardware, integrated circuits, firmware, micro code and the like, operating alone or in combination. Likewise, processing strategies may include multiprocessing, multitasking, parallel processing, distributed processing, and/or any other type of processing. In one embodiment, the instructions are stored on a removable media device for reading by local or remote systems. In other embodiments, the logic or instructions are stored in a remote location for transfer through a computer network or over telephone lines. In yet other embodiments, the logic or instructions may be stored within a given computer such as, for example, a CPU.
- While various embodiments of the system and method for audio bandwidth dependent noise suppression have been described, it will be apparent to those of ordinary skill in the art that many more embodiments and implementations are possible within the scope of the present invention. Accordingly, the invention is not to be restricted except in light of the attached claims and their equivalents.
Claims (10)
- A computer implemented method for audio bandwidth dependent noise suppression comprising:detecting (302) the audio bandwidth of an audio signal (116) responsive to one or more audio indicators;calculating (304) noise suppression gains responsive to the audio signal;modifying (306) the noise suppression gains responsive to the detected audio bandwidth; andapplying (308) the modified noise suppression gains to the audio signal (116).
- The method for audio bandwidth dependent noise suppression of claim 1, where the audio indicators comprise one or more of an audio sample rate associated with the audio signal (116), an energy detector, a compression format associated with the audio signal (116) and a data rate associated with the compression format.
- The method for audio bandwidth dependent noise suppression of claims 1 to 2, where the compression format comprise one or more of Adaptive Multi-Rate, Enhanced Variable Rate Codec B and Motion Pictures Expert Group Advanced Audio Coding.
- The method for audio bandwidth dependent noise suppression of claims 1 to 3, where modifying the noise suppression gains responsive to the detected audio bandwidth comprises limiting the magnitude of the noise suppression gains.
- The method for audio bandwidth dependent noise suppression of claim 4, where limiting the magnitude of the noise suppression gains comprises increasing a magnitude limit of the noise suppression gains with increased detected audio bandwidth.
- The method for audio bandwidth dependent noise suppression of claim 5, where the magnitude limit of the noise suppression gains comprises:10 to 12 dB for detected audio bandwidth up to and including 4 kHz;6 to 8 dB for detected audio bandwidth from 4 kHz up to and including 8 kHz; and0 to 6 dB for detected audio bandwidth over 8 kHz.
- The method for audio bandwidth dependent noise suppression of claims 1 to 6, further comprising:detecting changes in the audio bandwidth of an audio signal (116) responsive to one or more audio indicators;where modifying the noise suppression gains responsive to the detected audio bandwidth includes the changed detected audio bandwidth.
- The computer implemented method of claims 1 to 7, further comprising generating a set of sub-bands for the audio signal (116) using a subband filter or a Fast Fourier Transform.
- The computer implemented method of claims 1 to 8, further comprising generating a set of sub-bands for the audio signal (116) according to a critical, octave, mel, or bark band spacing technique.
- A system for audio bandwidth dependent noise suppression, the system comprising:a processor (202); anda memory (204) coupled to the processor (202) containing instructions, executable by the processor (202), for performing the instructions executing the steps of any of method claims 1 to 9.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP13153105.5A EP2760022B1 (en) | 2013-01-29 | 2013-01-29 | Audio bandwidth dependent noise suppression |
CA2840851A CA2840851C (en) | 2013-01-29 | 2014-01-27 | Audio bandwidth dependent noise suppression |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP13153105.5A EP2760022B1 (en) | 2013-01-29 | 2013-01-29 | Audio bandwidth dependent noise suppression |
Publications (2)
Publication Number | Publication Date |
---|---|
EP2760022A1 true EP2760022A1 (en) | 2014-07-30 |
EP2760022B1 EP2760022B1 (en) | 2017-11-01 |
Family
ID=47713885
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP13153105.5A Active EP2760022B1 (en) | 2013-01-29 | 2013-01-29 | Audio bandwidth dependent noise suppression |
Country Status (1)
Country | Link |
---|---|
EP (1) | EP2760022B1 (en) |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5012519A (en) * | 1987-12-25 | 1991-04-30 | The Dsp Group, Inc. | Noise reduction system |
US20060020450A1 (en) * | 2003-04-04 | 2006-01-26 | Kabushiki Kaisha Toshiba. | Method and apparatus for coding or decoding wideband speech |
US7844453B2 (en) | 2006-05-12 | 2010-11-30 | Qnx Software Systems Co. | Robust noise estimation |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2012158157A1 (en) * | 2011-05-16 | 2012-11-22 | Google Inc. | Method for super-wideband noise supression |
-
2013
- 2013-01-29 EP EP13153105.5A patent/EP2760022B1/en active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5012519A (en) * | 1987-12-25 | 1991-04-30 | The Dsp Group, Inc. | Noise reduction system |
US20060020450A1 (en) * | 2003-04-04 | 2006-01-26 | Kabushiki Kaisha Toshiba. | Method and apparatus for coding or decoding wideband speech |
US7844453B2 (en) | 2006-05-12 | 2010-11-30 | Qnx Software Systems Co. | Robust noise estimation |
Non-Patent Citations (1)
Title |
---|
"G.718: Frame error robust narrow-band and wideband embedded variable bit-rate coding of speech and audio from 8-32 kbit/s", ITU-T; STUDY PERIOD 2009-2012, INTERNATIONAL TELECOMMUNICATION UNION, GENEVA ; CH, vol. Study Group 16, 13 September 2010 (2010-09-13), pages 1 - 257, XP017452920 * |
Also Published As
Publication number | Publication date |
---|---|
EP2760022B1 (en) | 2017-11-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2905779B1 (en) | System and method for dynamic residual noise shaping | |
JP4440937B2 (en) | Method and apparatus for improving speech in the presence of background noise | |
KR101266894B1 (en) | Apparatus and method for processing an audio signal for speech emhancement using a feature extraxtion | |
US8244526B2 (en) | Systems, methods, and apparatus for highband burst suppression | |
US20150081287A1 (en) | Adaptive noise reduction for high noise environments | |
KR20160125984A (en) | Systems and methods for speaker dictionary based speech modeling | |
RU2682923C2 (en) | Improved extension of frequency band in an audio signal decoder | |
US9524729B2 (en) | System and method for noise estimation with music detection | |
CA2867250C (en) | System and method for dynamically mixing audio signals | |
US9349383B2 (en) | Audio bandwidth dependent noise suppression | |
US9210507B2 (en) | Microphone hiss mitigation | |
EP2760022B1 (en) | Audio bandwidth dependent noise suppression | |
CA2840851C (en) | Audio bandwidth dependent noise suppression | |
CN113593604A (en) | Method, device and storage medium for detecting audio quality | |
EP2760221A1 (en) | Microphone hiss mitigation | |
Rahali et al. | Enhancement of noise-suppressed speech by spectral processing implemented in a digital signal processor | |
Abd Almisreb et al. | Noise reduction approach for Arabic phonemes articulated by Malay speakers | |
EP2760021B1 (en) | Sound field spatial stabilizer | |
BRPI0911932B1 (en) | EQUIPMENT AND METHOD FOR PROCESSING AN AUDIO SIGNAL FOR VOICE INTENSIFICATION USING A FEATURE EXTRACTION |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20130129 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: HETHERINGTON, PHILLIP ALAN |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: 2236008 ONTARIO INC. |
|
17Q | First examination report despatched |
Effective date: 20150901 |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G10L 19/24 20130101ALN20170509BHEP Ipc: G10L 21/0264 20130101ALN20170509BHEP Ipc: G10L 21/0208 20130101AFI20170509BHEP |
|
INTG | Intention to grant announced |
Effective date: 20170526 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP Ref country code: AT Ref legal event code: REF Ref document number: 942767 Country of ref document: AT Kind code of ref document: T Effective date: 20171115 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602013028625 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 6 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20171101 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 942767 Country of ref document: AT Kind code of ref document: T Effective date: 20171101 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180201 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180201 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180202 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180301 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602013028625 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
26N | No opposition filed |
Effective date: 20180802 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180129 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20180131 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180131 Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180131 Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180131 Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180129 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180129 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20130129 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R082 Ref document number: 602013028625 Country of ref document: DE Representative=s name: MERH-IP MATIAS ERNY REICHL HOFFMANN PATENTANWA, DE Ref country code: DE Ref legal event code: R081 Ref document number: 602013028625 Country of ref document: DE Owner name: BLACKBERRY LIMITED, WATERLOO, CA Free format text: FORMER OWNER: 2236008 ONTARIO INC., WATERLOO, ONTARIO, CA |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20171101 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: 732E Free format text: REGISTERED BETWEEN 20200730 AND 20200805 |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230518 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20240129 Year of fee payment: 12 Ref country code: GB Payment date: 20240129 Year of fee payment: 12 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20240125 Year of fee payment: 12 |