EP2760022A1 - Audiobandbreitenabhängige Geräuschunterdrückung - Google Patents
Audiobandbreitenabhängige Geräuschunterdrückung Download PDFInfo
- Publication number
- EP2760022A1 EP2760022A1 EP13153105.5A EP13153105A EP2760022A1 EP 2760022 A1 EP2760022 A1 EP 2760022A1 EP 13153105 A EP13153105 A EP 13153105A EP 2760022 A1 EP2760022 A1 EP 2760022A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- audio
- noise suppression
- audio bandwidth
- bandwidth
- gains
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000001629 suppression Effects 0.000 title claims abstract description 85
- 230000001419 dependent effect Effects 0.000 title claims abstract description 20
- 238000000034 method Methods 0.000 claims abstract description 34
- 230000005236 sound signal Effects 0.000 claims abstract description 28
- 230000006835 compression Effects 0.000 claims abstract description 15
- 238000007906 compression Methods 0.000 claims abstract description 15
- 230000015654 memory Effects 0.000 claims description 20
- 230000003044 adaptive effect Effects 0.000 claims description 2
- 230000009467 reduction Effects 0.000 abstract description 15
- 238000005070 sampling Methods 0.000 abstract description 5
- 238000004891 communication Methods 0.000 description 19
- 239000003607 modifier Substances 0.000 description 9
- 230000003595 spectral effect Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000011946 reduction process Methods 0.000 description 2
- 239000000654 additive Substances 0.000 description 1
- 230000000996 additive effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
- G10L19/16—Vocoder architecture
- G10L19/18—Vocoders using multiple modes
- G10L19/24—Variable rate codecs, e.g. for generating different qualities using a scalable representation such as hierarchical encoding or layered encoding
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L21/0264—Noise filtering characterised by the type of parameter measurement, e.g. correlation techniques, zero crossing techniques or predictive techniques
Definitions
- the present disclosure relates to the field of audio noise suppression.
- a system and method for audio bandwidth dependent noise suppression are also possible.
- Low-bandwidth (a.k.a. limited-bandwidth) communication systems typically use low-bitrate codecs that may be generally intolerant of noise. Reduction of noise is very important when the communication system is intolerant of noise. Higher (a.k.a. wider) bandwidth communication systems may tolerate more noise and may be more likely to be used for multimedia application that may include music. The application of significant noise reduction in higher bandwidth communications may create more undesirable artifacts than allowing the noise to pass through the system or applying less significant noise reduction.
- Fig. 1 is a schematic representation of a system for audio bandwidth dependent noise suppression.
- Fig. 2 is a further schematic representation of a system for audio bandwidth dependent noise suppression.
- Fig. 3 is flow diagram representing a method for audio bandwidth dependent noise suppression.
- a system and method for audio bandwidth dependent noise suppression may detect the audio bandwidth of an audio signal responsive to one or more audio indicators.
- the audio indicators may include the audio sampling rate and characteristics of an associated compression format.
- Noise suppression gains may be calculated responsive to the audio bandwidth. Noise suppression gains may mitigate undesirable noise in a reproduced output signal.
- the noise suppression gains may be modified responsive to the detected audio bandwidth. Less noise reduction may be desirable when more audio bandwidth is available.
- the modified noise suppression gains may be applied to the audio signal.
- x ( t ) and n ( t ) denote a clean audio signal, and a noise signal, respectively.
- G i , k are the noise suppression gains.
- Various methods are known in the literature to calculate these gains.
- One example further described below is a recursive Wiener filter.
- the parameter ⁇ in (3) is a constant noise floor, which defines a maximum amount of noise attenuation across all frequency bins. For example, when ⁇ is set to 0.3, the system will attenuate the noise by a maximum of 10 dB (decibel) at each frequency bin k .
- the noise reduction process may produce limited noise suppression gains that will range from 0 dB to 10 dB at each frequency bin k .
- Narrowband communication may have, for example, an audio sample rate of 8 kHz with a 4 kHz audio bandwidth.
- Wideband communication may have, for example, an audio sample rate of 16 kHz with an 8 kHz audio bandwidth.
- Full band communication may have, for example, an audio sample rate of 32 kHz or greater with a 16 kHz or greater audio bandwidth.
- the given associated audio sample rate and audio bandwidth for narrowband, wideband and full band are exemplary in nature and the values may be greater or less than the example values.
- FIG. 1 is a schematic representation of a system for audio bandwidth dependent noise suppression.
- a microphone 102 may receive a sound field that is an audible environment associated with the microphones 102. Many audible environments associated with the microphones 102 may include undesirable content that may be mitigated by processing a microphone signal output by the microphone 102 responsive to the received sound field.
- a noise suppression gain calculator 104 calculates noise suppression gains G i,k using any of various methods that are know in the literature to calculate noise suppression gains.
- a noise suppression gain applier 106 may apply the noise suppression gains to the microphone signal to mitigate undesirable content.
- An exemplary noise suppression method is a recursive Wiener filter.
- the background noise estimate may include signal information from previously processed frames.
- the spectral magnitude of the background noise may be calculated using the background noise estimation techniques disclosed in U.S. Patent No. 7,844,453 , which is incorporated in its entirety herein by reference, except that in the event of any inconsistent disclosure or definition from the present specification, the disclosure or definition herein shall be deemed to prevail.
- alternative background noise estimation techniques may be used, such as a noise power estimation technique based on minimum statistics.
- One or more audio indicators 110 may indicate the audio bandwidth.
- One audio indicator 110 of the audio bandwidth may include the audio sample rate of the microphone signal.
- the audio sampling rate utilized by the noise suppression gain calculator 104 may be an alternative audio indicator 110 as the audio signal 116 may be processed using a sample rate converter.
- Another audio indicator 110 may include the type of compression format applied to the output signal 108.
- Compression formats utilized for voice communication may include the 3 rd Generation Partnership Project (3GPP) Adaptive Multi-Rate (AMR) and 3 rd Generation Partnership Project 2 (3GPP2) Enhanced Variable Rate Codec B (EVRC-B).
- Compression formats utilized for general audio communication may include Motion Pictures Expert Group (MPEG) Advanced Audio Coding (AAC).
- MPEG Motion Pictures Expert Group
- AAC Advanced Audio Coding
- Another audio indicator 110 may include the data rate of the compression format applied to the output signal 108.
- Another audio indicator 110 may include an energy detector that detects the energy of the audio signal 116 at various frequencies.
- the energy detector may allow for an estimation of the audio bandwidth of a remote side audio signal.
- the remote side device may be capable of only narrowband audio signals where it may be desirable to increase the amount of noise reduction.
- a device that may be capable of limited audio bandwidth, for example narrowband may select a compression format suitable to the limited audio bandwidth. In this case, narrowband audio may cause a voice codec to be selected including AMR or EVRC-B.
- An audio bandwidth detector 112 may detect the audio bandwidth of the audio signal responsive to one or more audio indicators 110.
- Low-bandwidth audio communications may utilize low-bitrate compression formats, or codecs, including AMR and EVRC that may be intolerant of noise. Noise reduction may be important when codecs are intolerant of noise. Low audio-bandwidth communication may not be used for music or multimedia applications, so again, reduction of noise may be important. Higher bandwidth communication may tolerate more noise and may be more likely to be used for multimedia applications that involve music where less noise reduction may be desirable.
- the data rate and type of codec used may change the desired about of noise reduction. For example, an audio codec operating at a low data rate may be perceptibly improved by utilizing more noise suppression. In this case, more noise removal may allow the audio codec to allocate more data rate to the desired signal content.
- a noise suppression gain modifier 114 may modify the noise suppression gains responsive to the audio bandwidth detected by the audio bandwidth detector 112.
- the noise suppression gain modifier 114 may, for example, utilize a mechanism described by equation (3) where the audio bandwidth detector 112 may modify the parameter ⁇ .
- the noise suppression gain modifier 114 may produce limited noise suppression gains that may, for example, have a maximum suppression varying from 10 dB to 12 dB when the audio bandwidth detector 112 detects narrowband audio.
- the noise suppression gain modifier 114 may produce limited noise suppression gains that may, for example, have a maximum suppression varying from 6 dB to 8 dB when the audio bandwidth detector 112 detects wideband audio.
- the noise suppression gain modifier 114 may produce limited noise suppression gains that may, for example, have a maximum suppression varying from 0 dB to 6 dB when the audio bandwidth detector 112 detects full band audio.
- the audio bandwidth detector 112 may detect full band audio when a low data rate audio codec is utilized and the noise suppression gain modifier 114 may produce limited noise suppression gains that may have a maximum suppression varying from 6 dB to 10 dB.
- a subband filter may process the microphone 102 to extract frequency information.
- the subband filter may be accomplished by various methods, such as a Fast Fourier Transform (FFT), critical filter bank, octave filter band, or one-third octave filter bank.
- the subband analysis may include a time-based filter bank.
- the time-based filter bank may be composed of a bank of overlapping bandpass filters, where the center frequencies have non-linear spacing such as octave, 3 rd octave, bark, mel, or other spacing techniques.
- the noise suppression gains may be calculated for each frequency bin or band of the subband filter. The resulting noise suppression gains may be filtered, or smoothed, over time and/or frequency.
- Many communications channels may have a variable amount of available communication bandwidth over time. As the amount of communication bandwidth increases, the audio bandwidth of a signal carried by a communications channel may increase. The increased audio bandwidth may be utilized to support one or more of a higher audio sampling rate, utilizing a compression format with increased signal quality and a higher data rate for the associated compression format. Conversely, the amount of audio bandwidth may be reduced over time. The reduction in audio bandwidth may result in one or more of a lower audio sampling rate, utilizing a compression format with reduced signal quality and a lower data rate for the associated compression format.
- the audio bandwidth detector 112 may trigger the noise suppression gain modifier 114 to cause a change in amount of noise suppression responsive to the dynamic bandwidth conditions of the communication channel and thereby the audio bandwidth.
- Figure 3 is a representation of a method for audio bandwidth dependent noise suppression.
- the method 300 may be, for example, implemented using the systems 100 and 200 described herein with reference to Figure 1 and Figure 2 .
- the method 300 includes the act of detecting the audio bandwidth of an audio signal responsive to one or more audio indicators 302.
- Noise suppression gaind may be calculated responsive to the audio signal 304.
- the noise suppression gains may be modified responsive to the detected audio bandwidth 306.
- the modified noise suppression gains may be applied to the audio signal 308.
- FIG. 2 is a further schematic representation of a system for audio bandwidth dependent noise suppression.
- the system 200 comprises a processor 202, memory 204 (the contents of which are accessible by the processor 202), one or more microphones 102 and an I/O interface 206.
- the memory 204 may store instructions which when executed using the process 202 may cause the system 200 to render the functionality associated with audio bandwidth dependent noise suppression as described herein.
- the memory 204 may store instructions which when executed using the process 202 may cause the system 200 to render the functionality associated with the noise suppression gain calculator module 104, the noise suppression gain applier module 106, the audio indicators 110, the audio bandwidth detector module 112 and the noise suppression gain modifier 114 described herein.
- data structures, temporary variables and other information may store data in data storage 208.
- the processor 202 may comprise a single processor or multiple processors that may be disposed on a single chip, on multiple devices or distributed over more that one system.
- the processor 202 may be hardware that executes computer executable instructions or computer code embodied in the memory 204 or in other memory to perform one or more features of the system.
- the processor 202 may include a general purpose processor, a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a digital signal processor (DSP), a field programmable gate array (FPGA), a digital circuit, an analog circuit, a microcontroller, any other type of processor, or any combination thereof.
- the memory 204 may comprise a device for storing and retrieving data, processor executable instructions, or any combination thereof.
- the memory 204 may include non-volatile and/or volatile memory, such as a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a flash memory.
- RAM random access memory
- ROM read-only memory
- EPROM erasable programmable read-only memory
- flash memory a flash memory.
- the memory 204 may comprise a single device or multiple devices that may be disposed on one or more dedicated memory devices or on a processor or other similar device.
- the memory 204 may include an optical, magnetic (hard-drive) or any other form of data storage device.
- the memory 204 may store computer code, such as the noise suppression gain calculator module 104, the noise suppression gain applier module 106, the audio indicators 110, the audio bandwidth detector module 112 and the noise suppression gain modifier 114 described herein.
- the computer code may include instructions executable with the processor 202.
- the computer code may be written in any computer language, such as C, C++, assembly language, channel program code, and/or any combination of computer languages.
- the memory 204 may store information in data structures including, for example, noise suppression gains and state variables.
- the I/O interface 206 may be used to connect devices such as, for example, the one or more microphones 102, and to other components of the system 200.
- the system 200 may include more, fewer, or different components than illustrated in Figure 2 . Furthermore, each one of the components of system 200 may include more, fewer, or different elements than is illustrated in Figure 2 .
- Flags, data, databases, tables, entities, and other data structures may be separately stored and managed, may be incorporated into a single memory or database, may be distributed, or may be logically and physically organized in many different ways.
- the components may operate independently or be part of a same program or hardware.
- the components may be resident on separate hardware, such as separate removable circuit boards, or share common hardware, such as a same memory and processor for implementing instructions from the memory. Programs may be parts of a single program, separate programs, or distributed across several memories and processors.
- the functions, acts or tasks illustrated in the figures or described may be executed in response to one or more sets of logic or instructions stored in or on computer readable media.
- the functions, acts or tasks are independent of the particular type of instructions set, storage media, processor or processing strategy and may be performed by software, hardware, integrated circuits, firmware, micro code and the like, operating alone or in combination.
- processing strategies may include multiprocessing, multitasking, parallel processing, distributed processing, and/or any other type of processing.
- the instructions are stored on a removable media device for reading by local or remote systems.
- the logic or instructions are stored in a remote location for transfer through a computer network or over telephone lines.
- the logic or instructions may be stored within a given computer such as, for example, a CPU.
Landscapes
- Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Soundproofing, Sound Blocking, And Sound Damping (AREA)
- Circuit For Audible Band Transducer (AREA)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP13153105.5A EP2760022B1 (de) | 2013-01-29 | 2013-01-29 | Audiobandbreitenabhängige Geräuschunterdrückung |
CA2840851A CA2840851C (en) | 2013-01-29 | 2014-01-27 | Audio bandwidth dependent noise suppression |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP13153105.5A EP2760022B1 (de) | 2013-01-29 | 2013-01-29 | Audiobandbreitenabhängige Geräuschunterdrückung |
Publications (2)
Publication Number | Publication Date |
---|---|
EP2760022A1 true EP2760022A1 (de) | 2014-07-30 |
EP2760022B1 EP2760022B1 (de) | 2017-11-01 |
Family
ID=47713885
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP13153105.5A Active EP2760022B1 (de) | 2013-01-29 | 2013-01-29 | Audiobandbreitenabhängige Geräuschunterdrückung |
Country Status (1)
Country | Link |
---|---|
EP (1) | EP2760022B1 (de) |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5012519A (en) * | 1987-12-25 | 1991-04-30 | The Dsp Group, Inc. | Noise reduction system |
US20060020450A1 (en) * | 2003-04-04 | 2006-01-26 | Kabushiki Kaisha Toshiba. | Method and apparatus for coding or decoding wideband speech |
US7844453B2 (en) | 2006-05-12 | 2010-11-30 | Qnx Software Systems Co. | Robust noise estimation |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2710590B1 (de) * | 2011-05-16 | 2015-10-07 | Google, Inc. | Superbreitband-rauschunterdrückung |
-
2013
- 2013-01-29 EP EP13153105.5A patent/EP2760022B1/de active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5012519A (en) * | 1987-12-25 | 1991-04-30 | The Dsp Group, Inc. | Noise reduction system |
US20060020450A1 (en) * | 2003-04-04 | 2006-01-26 | Kabushiki Kaisha Toshiba. | Method and apparatus for coding or decoding wideband speech |
US7844453B2 (en) | 2006-05-12 | 2010-11-30 | Qnx Software Systems Co. | Robust noise estimation |
Non-Patent Citations (1)
Title |
---|
"G.718: Frame error robust narrow-band and wideband embedded variable bit-rate coding of speech and audio from 8-32 kbit/s", ITU-T; STUDY PERIOD 2009-2012, INTERNATIONAL TELECOMMUNICATION UNION, GENEVA ; CH, vol. Study Group 16, 13 September 2010 (2010-09-13), pages 1 - 257, XP017452920 * |
Also Published As
Publication number | Publication date |
---|---|
EP2760022B1 (de) | 2017-11-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2905779B1 (de) | System und verfahren für dynamische restgeräuschformung | |
JP4440937B2 (ja) | 暗騒音存在時の音声を改善するための方法および装置 | |
KR101266894B1 (ko) | 특성 추출을 사용하여 음성 향상을 위한 오디오 신호를 프로세싱하기 위한 장치 및 방법 | |
US8244526B2 (en) | Systems, methods, and apparatus for highband burst suppression | |
US20150081287A1 (en) | Adaptive noise reduction for high noise environments | |
KR20160125984A (ko) | 화자 사전 기반 스피치 모델링을 위한 시스템들 및 방법들 | |
RU2682923C2 (ru) | Улучшенное расширение диапазона частот в декодере звукового сигнала | |
US9524729B2 (en) | System and method for noise estimation with music detection | |
CA2867250C (en) | System and method for dynamically mixing audio signals | |
US9349383B2 (en) | Audio bandwidth dependent noise suppression | |
CN113593604A (zh) | 检测音频质量方法、装置及存储介质 | |
US9210507B2 (en) | Microphone hiss mitigation | |
EP2760022B1 (de) | Audiobandbreitenabhängige Geräuschunterdrückung | |
CA2840851C (en) | Audio bandwidth dependent noise suppression | |
EP2760221A1 (de) | Mikrofonzischabschwächung | |
Rahali et al. | Enhancement of noise-suppressed speech by spectral processing implemented in a digital signal processor | |
Abd Almisreb et al. | Noise reduction approach for Arabic phonemes articulated by Malay speakers | |
EP2760021B1 (de) | Räumlicher Schallfeldstabilisator | |
BRPI0911932B1 (pt) | Equipamento e método para processamento de um sinal de áudio para intensificação de voz utilizando uma extração de característica |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20130129 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: HETHERINGTON, PHILLIP ALAN |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: 2236008 ONTARIO INC. |
|
17Q | First examination report despatched |
Effective date: 20150901 |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G10L 19/24 20130101ALN20170509BHEP Ipc: G10L 21/0264 20130101ALN20170509BHEP Ipc: G10L 21/0208 20130101AFI20170509BHEP |
|
INTG | Intention to grant announced |
Effective date: 20170526 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP Ref country code: AT Ref legal event code: REF Ref document number: 942767 Country of ref document: AT Kind code of ref document: T Effective date: 20171115 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602013028625 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 6 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20171101 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 942767 Country of ref document: AT Kind code of ref document: T Effective date: 20171101 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180201 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180201 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180202 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180301 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602013028625 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
26N | No opposition filed |
Effective date: 20180802 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180129 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20180131 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180131 Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180131 Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180131 Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180129 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180129 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20130129 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R082 Ref document number: 602013028625 Country of ref document: DE Representative=s name: MERH-IP MATIAS ERNY REICHL HOFFMANN PATENTANWA, DE Ref country code: DE Ref legal event code: R081 Ref document number: 602013028625 Country of ref document: DE Owner name: BLACKBERRY LIMITED, WATERLOO, CA Free format text: FORMER OWNER: 2236008 ONTARIO INC., WATERLOO, ONTARIO, CA |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20171101 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171101 |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: 732E Free format text: REGISTERED BETWEEN 20200730 AND 20200805 |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230518 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20240129 Year of fee payment: 12 Ref country code: GB Payment date: 20240129 Year of fee payment: 12 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20240125 Year of fee payment: 12 |