EP2355094B1 - Subband zur Verarbeitung der Komplexitätsverringerung - Google Patents
Subband zur Verarbeitung der Komplexitätsverringerung Download PDFInfo
- Publication number
- EP2355094B1 EP2355094B1 EP11151856.9A EP11151856A EP2355094B1 EP 2355094 B1 EP2355094 B1 EP 2355094B1 EP 11151856 A EP11151856 A EP 11151856A EP 2355094 B1 EP2355094 B1 EP 2355094B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- sub
- band
- phase
- data
- bin
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 title claims description 20
- 230000009467 reduction Effects 0.000 title description 4
- 238000001228 spectrum Methods 0.000 claims description 50
- 230000006835 compression Effects 0.000 claims description 34
- 238000007906 compression Methods 0.000 claims description 34
- 238000000034 method Methods 0.000 claims description 16
- 230000006870 function Effects 0.000 claims description 13
- 230000003595 spectral effect Effects 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 8
- 239000013598 vector Substances 0.000 claims description 5
- 238000005192 partition Methods 0.000 claims description 4
- 230000035945 sensitivity Effects 0.000 claims description 2
- 238000000638 solvent extraction Methods 0.000 claims description 2
- 230000026676 system process Effects 0.000 claims description 2
- 238000004590 computer program Methods 0.000 claims 4
- 238000013139 quantization Methods 0.000 description 9
- 230000000717 retained effect Effects 0.000 description 4
- 230000003044 adaptive effect Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 230000015572 biosynthetic process Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000005236 sound signal Effects 0.000 description 3
- 238000003786 synthesis reaction Methods 0.000 description 3
- 230000002238 attenuated effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 239000000470 constituent Substances 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 230000002085 persistent effect Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 206010021403 Illusion Diseases 0.000 description 1
- 240000004759 Inga spectabilis Species 0.000 description 1
- 238000007792 addition Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000006837 decompression Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 230000000873 masking effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000003607 modifier Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000007493 shaping process Methods 0.000 description 1
- 238000010183 spectrum analysis Methods 0.000 description 1
- 230000032258 transport Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
- G10L25/18—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
- G10L19/0204—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using subband decomposition
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
- G10L19/0212—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using orthogonal transformation
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L2021/02082—Noise filtering the noise being echo, reverberation of the speech
Definitions
- This disclosure relates to sub-band processing, and more particularly to systems that reduce computational complexity and memory requirements.
- Wideband networks receive and transmit data through radio frequency signals through inbound and outbound transmissions.
- the networks may transmit data, voice, and video simultaneously through multiple channels that may be distinguished in frequency.
- Some wideband networks are capable of high speed operations and may have a considerably higher throughput than some narrowband networks.
- the increased bandwidth of these networks may increase the processing loads and memory requirements of other applications.
- Frequency domain based adaptive filtering may be computationally intensive because it translates a time domain signal into multiple frequency components that are separately processed. Translating a time domain signal into multiple frequency components increases the computational complexity and memory usage of some systems when a signal's bandwidth increases. As the number of frequency components increase with bandwidth, the computational load and the required memory increase.
- EP 0527374 relates to good speech quality (also for music signals) which is intended to be achieved for audio signals (50 to 7000 Hz) with a data rate of 32 kbits/s, for which purpose considerably more intensive data reduction must be achieved than was known in the past.
- the absolute audible threshold and a relative audible threshold which is known as a masking effect, are taken into account for defining the quantisation levels, and a variable quantisation is subsequently carried out.
- GB 2362549 relates to an apparatus and a method for quantizing the phase of a speech signal using a perceptual weighting function.
- the apparatus includes a phase information extractor for obtaining the phase of each harmonic frequency in a speech signal represented by the discrete sum of periodic signals having different harmonic frequency components, a quantization noise shaping unit for controlling the amount of quantization noise of each phase using a perceptual weighting function, which makes quantization noise less than a predetermined just noticeable difference (JND) of the phase, a quantization bit assigner for assigning quantization bits to each phase according to the controlled amount of quantization noise, and a scalar quantizer for quantizing each phase by the assigned quantization bits.;
- JND just noticeable difference
- a quantization bit assigner for assigning quantization bits to each phase according to the controlled amount of quantization noise
- a scalar quantizer for quantizing each phase by the assigned quantization bits.
- EP 1852848 relates to lossy based lossless coding a PCM audio signal which passes through a lossy encoder to a lossy decoder.
- the lossy encoder provides a lossy bit stream.
- the difference signal (S Diff) between the PCM signal and the lossy decoder output is lossless encoded, providing an extension bit stream.
- the invention facilitates enhancing a lossy perceptual audio encoding/decoding by an extension that enables mathematically exact reproduction of the original waveform using enhanced de-correlation, and provides additional data for reconstructing at decoder site an intermediate-quality audio signal.
- the lossless extension can be used to extend the widely used mp3 encoding/decoding to lossless encoding/decoding and superior quality mp3 encoding/de-coding.
- a sub-band processing system that reduces computational complexity and memory requirements includes a processor and a local or a distributed memory.
- Logic stored in the memory partitions a frequency spectrum of bins into sub-bands.
- the logic enables a lossy compression by designating a magnitude and a designated or derived phase of each bin in the enables a lossy compression by designating a magnitude and a designated or derived phase of each bin in the frequency spectrum as representative.
- the logic renders a lossless compression by decompressing the lossy compressed data and providing lost data based on original spectral relationships contained within the frequency spectrum.
- a sub-band processing system processes data such that, after it is compressed and decompressed it is restored to its original format.
- the system may compress video, sound, text, code, and/or numeric data such that little or no data is lost after a bin or file is decompressed. While the data may contain more information than may be heard or seen (e.g., perceived by a user), some systems preserve the original data (or a representative data set) while compressing and decompressing operating data through a lossy compression.
- the sub-band processing system After further (optional) processing (by an ancillary device or system) the sub-band processing system reconstructs and restores the data.
- the restored data may maintain the relative magnitude and phase of the original data.
- the restored data may match the original relationships (e.g., relative magnitudes and phases) frequency-for-frequency.
- the sub-band processing system analysis may occur on frequency domain characteristics.
- the signal may be broken into intervals though a multiplier function (retained in a local or a distributed computer readable medium) or multiplier device that multiplies the signal by a "window" function or a "frame” of fixed duration.
- smooth window functions such as Hann, Hamming, etc. retained in the local or the distributed computer readable medium
- a window filter may be used for the short-time spectral analysis.
- a time-to-frequency transform device, a Discrete Fourier Transform (DFT) device, or a Fast Fourier Transform (FFT) device may transform (or decompose) the short-time based signals into a complex spectrum.
- DFT Discrete Fourier Transform
- FFT Fast Fourier Transform
- the spectrum may be separated into bins of magnitude and phase data or substantially equivalent complex (e.g. real and imaginary) data.
- a sub-band (or band) may be represented by a single bin of magnitude and phase spectra, or a collection of consecutive or successive bins represented by a common or single magnitude and phase spectra.
- Table 1 shows representative characteristics of an exemplary FFT device. Table 1 Parameters Sample rate (kHz) 8 11.025 16 22.05 32 44.1 FFT length ( N ) 256 256 512 512 1024 1024 Number of useful output bins ( R ) 129 129 257 257 513 513 Hz / bin 31.25 43.07 31.25 43.07 31.25 43.07
- an FFT device may transform the time domain signal into about 256 bins. Due to the complex symmetry, the FFT device may yield about 129 useful bins (e.g., 256/2 + 1). Each bin may represent a frequency resolution of about 31.25 Hz (e.g., 8 kHz/256).
- the frequency resolution of other sample rates e.g., 16 kHz and 32 kHz
- the FFT length may be about double the FFT length of the 8 kHz sample rate.
- the FFT length may be about double the FFT length of the 16 kHz sample rate.
- the magnitude and phase spectra may be obtained from one or more signal processors that execute a Discrete Fourier Transform (DFT) stored in a local or a distributed memory.
- the output of the DFT may be represented by X(k) .
- the sub-band processing system may reduce M to a lowest possible integer that does not affect the performance or quality of a later process.
- the system may generate a number of sub-bands that minimize perceptual error.
- the applications may exploit the sensitivity of the human auditory system or other systems that do not detect or process certain frequencies or are affected by certain signal distortions.
- a lossy compression may compress the data such that some data is lost when the data is compressed into the sub-bands.
- Some sub-band processing systems compress 2 q bins (q is an integer) into individual sub-bands.
- Other systems apply a perceptual scale (through a processor or controller, for example) where the bins are grouped into sub-bands that match the frequency selectivity of the human auditory system.
- the sub-bands may comprise non-overlapping or overlapping frequency regions that account for a selected or critical band (e.g., a frequency bandwidth that may model an auditory filter) or apply a perceptual scale like a single or multiple stage rectangular-like bandwidth filter or filter bank, logarithmic spacing filter or filter bank, Bark filter or filter bank, Mel or Mel-like filters or filter bank.
- Figures 1 and 2 respectively, describe exemplary non-overlapping and band-like overlapping compressions.
- the uncompressed bins are shown above the corresponding compressed sub-bands.
- the compressions divide a variable sequence of uncompressed bins into a substantially equal sequence of compressed sub-bands.
- a substantially equal gain or a variable gain may be applied to render compressed sub-bands that are substantially flat across the frequency spectrum. Perceptual distortions may be minimized by applying lower compression ratios at lower frequencies while applying higher compression ratios at higher frequencies.
- Table 2 describes an exemplary non-overlapping compression scheme in which each sub-band represents 2 q bins. Approximate freq range (kHz) Input bin numbers Compression ratio Output sub-bands #s 0 - 1 0 .. 31 1:1 0 .. 31 1 - 2 32 .. 63 2:1 32 .. 47 2 - 4 64 .. 127 4:1 48 .. 63 4 - Nyquist 128 .. M 8:1 64 .. xx
- Each sub-band may be approximately equal to a first predetermined frequency band such as 1 ⁇ 2 ERB (Equivalent Rectangular Bandwidth) for frequencies below about 4 kHz, and a second predetermined frequency band such as 1 ERB for frequencies above about 4 kHz. More aggressive compression schemes may be applied when the level of distortion or artifacts do not affect (or have little affect on) the performance of other systems.
- 1 ⁇ 2 ERB Equivalent Rectangular Bandwidth
- Some systems such as a system that may divide fifteen bins of the spectrum into five sub-bands (e.g., as shown in Figure 3 ) may group sub-bands such that each sub-band is about 0.4 ERB (at a low compression) to about 0.875 (at a high compression) ERB. When there is less processor execution speed the sub-bands may be increased. If there is a need to reduce a processors speed by a millions of instructions per second (MIPS), for example, some systems increase the sub-bands to larger ERB values (e.g., each sub-band may be about 1.25 ERB)
- the sub-band processing system may select or designate a representative phase for each sub-band. Some sub-band processing system "preserve" or select the phase of a bin within the sub-band that has the lowest frequency (as shown in Figure 3 ) within that sub-band. Other systems may select bins near or at the center of the sub-band, and others may select a phase based on other structural, functional, or qualitative measures.
- An alterative sub-band processing system may derive phase through an average or weighted average (e.g., an averaging filter, a programmable dynamic weighting filter, a perceptual weighting filter, etc.).
- An average may comprise a logical operation stored in a local or remote central or distributed memory such as an arithmetic mean of the phases within each sub-band.
- the weights of a weighted average may be based on the phase correlations common to one or all of the bins that comprise one or more sub-bands.
- the selected magnitudes an average magnitude (e.g., an average of bins that makeup a band), peaks in the magnitude spectrum, or a function or algorithm that selects or synthesizes a magnitude of each sub-band may be designated as representative.
- an average magnitude e.g., an average of bins that makeup a band
- peaks in the magnitude spectrum or a function or algorithm that selects or synthesizes a magnitude of each sub-band
- a function or algorithm that selects or synthesizes a magnitude of each sub-band may be designated as representative.
- the bin containing that magnitude is indexed, stored in memory, and the magnitude is rotated or shifted (e.g., through a phase shifter) to attain the selected or designated phase.
- a resulting sub-band value may be transformed to a maximum magnitude selected from its constituent bins and the phase of the "preserved" or selected bin (through a rotation through or shift by a phase differential, e.g., beta sub-band1 , beta sub-band2
- common bins may be selected from the divided spectrum to preserve the phase of the sub-bands relative to each processed frame.
- j m and D m may be constant (e.g., temporally invariant) while h m may change (e.g., time variant) from one aural or sound frame (or video, sound, text, code, and/or numeric data) to the next.
- Such systems may preserve the phase of the same bin within a sub-band on a frame-by-frame basis such as for example, always the first bin of a sub-band in each frame or a common bin of a sub-band in each frame.
- Figure 4 is an uncompressed spectrum of complex vectors representing bins 4, 5, 6 and 7 that comprise an exemplary sub-band 3.
- bin 5 has the largest magnitude and is therefore designated as representative (e.g., through a peak magnitude detector).
- the phase of bin 4 is the designated phase.
- the vector representing bin 5 is rotated counter-clockwise or otherwise adjusted to substantially match the phase of bin 4 while maintaining its original maximum magnitude (as shown in Figure 5 ).
- the rotated or adjusted version of bin 5 represents sub-band 3, which effectively attenuates the remaining spectrum within the sub-band (e.g., effectively setting the remaining spectrum to substantially to zero) as shown in Figure 6 .
- the magnitudes and phases of the sparse spectrum may be further processed before the spectrum is reconstructed.
- the spectrum may be further processed in the frequency domain (or other domains).
- Adaptive filtering techniques or devices used by an acoustic echo canceller, noise cancellation, or a beam-former, for example may process a consistent phase that does not change abruptly from frame to frame.
- Abrupt phase changes that may be a characteristic of other systems may be identified as an impulse response that causes an acoustic echo canceller to diverge.
- divergence occurs, a sub-optimal, reduced, or no echo cancellation may occur due to the mismatch between the filter coefficients and the echo path characteristics.
- an adaptive filter may require time to achieve a convergence.
- the original spectral data (or a representative data set or a data set of relative measures) is processed so that little or no data is lost when the decompression is complete.
- the sub-band processing system may achieve a lossless or nearly lossless compression. Some systems may preserve almost the entire original spectrum to avoid generating perceivable artifacts when the spectrum is reconstructed.
- An overlap-add synthesis may partially reconstruct the spectrum from the processed sparse spectrum.
- An overlap-add synthesis may avoid discontinuities in the reconstructed spectrum.
- the system rotates the processed sub-band to its original relative phase (or a substantially original relative phase), which is relative to the preserved bin (e.g., through a counter rotation through the phase differential, e.g., beta sub-band1 , beta sub-band2 , etc.). For example, if a bin containing the largest magnitude was rotated beta degrees in one direction, then the system rotates the processed sub-band by beta degrees in the opposite direction to restore the peak magnitude bin.
- the remaining bins that made up the sub-band are reconstructed by maintaining relative magnitudes and phases of the original spectrum (or representative data or relative measure data set).
- the magnitude and phase of the remaining reconstructed bins maintain the same relative magnitude and phase relationship with the restored peak magnitude bin, as the original spectral bins had with the original peak magnitude bin.
- frequency-criteria may affect phase reconstruction.
- sub-bands that exceed a predetermined value e.g., over about 4 kHz, may not maintain relative phase relationships.
- Equations 6 - 10 describe how the magnitude and phase for each sub-band may be expanded to its constituent bins.
- Equation (7) establishes that the magnitude of the restored peak magnitude bin is equal (or may be substantially equal) to the magnitude of the processed sub-band.
- Equation (8) establishes that the phase of the restored peak magnitude bin maintains substantially the same relative phase relationship measured during the partitioning process.
- Equations (9) and (10), respectively, establish how the remaining bins may be reconstructed.
- SBY m SBY m
- arg SBY m Y h m SBY m
- arg Y h m arg SBY m ⁇ arg X j m + arg X h m
- Y p Y h m ⁇ X p X h m
- processed sub-band 3 may be somewhat attenuated and rotated as shown in Figure 7 .
- bin 5 since bin 5 was designated as representative, it may be restored by rotating sub-band 3 clockwise by beta degrees to maintain the original relative phase to bin 4.
- the restored bin 5 maintains a new attenuated (or adjusted) magnitude.
- the remaining bins are then scaled and rotated to maintain their original relative phase and magnitude relationships to the restored bin as shown in Figure 8 .
- the original spectrum (or the representative data set) may be retained in a computer readable medium or memory so that the original relative magnitude and phase relationships may be maintained or restored in the decompressed spectrum. This retention potentially reduces audible artifacts that may be introduced by a compression scheme.
- the system, methods, and descriptions described may be programmed in one or more controllers, devices, processors (e.g., signal processors).
- the processors may comprise one or more central processing units that supervise the sequence of micro-operations that execute the instruction code and data coming from memory (e.g., computer readable medium) that generate, support, and/or complete an operation, compression, or signal modifications.
- the dedicated applications may support and define the functions of the special purpose processor or general purpose processor that is customized by instruction code (and in some applications may be resident to vehicles).
- a front-end processor may perform the complementary tasks of gathering data for a processor or program to work with, and for making the data and results available to other processors, controllers, or devices.
- the systems, methods, and descriptions may program one or more signal processors or may be encoded in a signal bearing storage medium, a computer-readable medium, or may comprise logic 902 stored in a memory that may be accessible through an interface and is executable by one or more processors 904 as shown in Figure 9 (in Figure 9 , N comprises an integer).
- Some signal-bearing storage medium or computer-readable medium comprise a memory that is unitary or separate (e.g., local or remote) from a device, programmed within a device, such as one or more integrated circuits, or retained in memory and/or processed by a controller or a computer.
- the software or logic may reside in a memory resident to or interfaced to one or more processors, devices, or controllers that may support a tangible or visual communication interface (e.g., to a display), wireless communication interface, or a wireless system.
- the memory may retain an ordered listing of executable instructions in a processor, device, or controller accessible medium for implementing logical functions.
- a logical function may be implemented through digital circuitry, through source code, or through analog circuitry.
- the software may be embodied in any computer-readable medium or signal-bearing medium, for use by, or in connection with, an instruction executable system, apparatus, and device, resident to system that may maintain persistent or non-persistent connections.
- Such a system may include a computer system, a processor-based system, or another system that includes an input and output interface that may communicate with a publicly accessible or privately accessible distributed network through a wireless or tangible communication bus through a public and/or proprietary protocol.
- a “computer-readable storage medium,” “machine-readable medium,” “propagated-signal” medium, and/or “signal-bearing medium” may comprise a medium that stores, communicates, propagates, or transports software or data for use by or in connection with an instruction executable system, apparatus, or device.
- the machine-readable medium may selectively be, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium.
- a non-exhaustive list of examples of a machine-readable medium would include: an electrical connection having one or more wires, a portable magnetic or optical disk, a volatile memory, such as a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM or Flash memory), or an optical fiber.
- a machine-readable medium may also include a tangible medium, as the software may be electronically stored as an image or in another format (e.g., through an optical scan), then compiled, and/or interpreted or otherwise processed. The processed medium may then be stored in a computer and/or machine memory.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Computational Linguistics (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Claims (15)
- System, umfassend:eine erste Logik (902), die in einem computerlesbaren Medium gespeichert ist und durch einen Prozessor (904) ausführbar ist, der ein Frequenzspektrum von Tondaten, das eine gewisse Anzahl von Bins von reellen und imaginären Daten umfasst, in eine kleinere Anzahl von Teilbändern als die Anzahl von Bins partitioniert und speichert;gekennzeichnet durch:eine zweite Logik (902), die in dem computerlesbaren Medium gespeichert ist und durch den Prozessor ausführbar ist, der eine verlustbehaftete Kompression ausführt, die das Frequenzspektrum von Tondaten basierend auf einer bestimmten Größe eines Bins in jedem Teilband, die für dieses Teilband repräsentativ ist, und einer bestimmten Phase eines Bins in jedem Teilband, die für dieses Teilband repräsentativ ist, derart komprimiert, dass die Größendaten und die Phasendaten jedes Bins des Frequenzspektrums durch die verlustbehaftete Kompression nicht bewahrt werden, wobei die zweite Logik eine Vielzahl von Tondatenrahmen verarbeitet und ein gemeinsames Bin in jedem Teilband als eine repräsentative Phase für jeden Tondatenrahmen, den das System verarbeitet, bestimmt, wobei ein Index des gemeinsamen Bins, dessen Phase für jedes Teilband beibehalten wird, von einem Rahmen zu einem nächsten Rahmen konstant ist; undeine dritte Logik (902), die in dem computerlesbaren Medium gespeichert ist und von dem Prozessor ausführbar ist, der eine verlustfreie Kompression durch Dekomprimieren von verlustbehaftet komprimierten Daten, die von der zweiten Logik wiedergegeben werden, und Bereitstellen von Größendaten und Phasendaten, die von der verlustbehafteten Kompression nicht bewahrt werden, basierend auf ursprünglichen spektralen Beziehungen, die in dem Frequenzspektrum enthalten sind, das in dem computerlesbaren Medium gespeichert ist, wiedergibt, wobei die ursprünglichen spektralen Beziehungen ursprüngliche Größen- und Phasenbeziehungen in Bezug auf das Bin, das für jedes Teilband beibehalten wird, angeben.
- System nach Anspruch 1, wobei eine Phase des gemeinsamen Bins in einem ersten Teilband der Teilbänder als bestimmte Phase für das erste Teilband dient.
- System nach Anspruch 1, wobei die bestimmte Phase mindestens eines Bins in einem ersten Teilband der Teilbänder anders als eine ursprüngliche Phase des mindestens einen Bins ist, welches die bestimmte Größe in dem ersten Teilband umfasst.
- System nach Anspruch 2, wobei das gemeinsame Bin ein erstes Bin jedes Teilbandes ist.
- System nach Anspruch 1, wobei sich ein Index eines Bins, welches die ausgewählte Größe enthält, die als repräsentativ bestimmt wird und eine maximale Größe für jedes Teilband aufweist, von dem einen Rahmen zum nächsten Rahmen ändert.
- System nach Anspruch 1, ferner umfassend eine Logik, die eine Verstärkung auf ein oder mehrere der Teilbänder anwendet, um komprimierte Teilbänder wiederzugeben, die über das Frequenzspektrum geglättet sind.
- System nach Anspruch 1, ferner umfassend eine Multiplikatorvorrichtung, die das Frequenzspektrum mit einer Fensterfunktion multipliziert, bevor das Frequenzspektrum partitioniert wird.
- System nach Anspruch 1, wobei die erste Logik das Frequenzspektrum von Bins von reellen und imaginären Daten in Teilbänder partitioniert, die einer Frequenzempfindlichkeit des menschlichen Gehörs entsprechen.
- System nach Anspruch 1, wobei das computerlesbare Medium einen verteilten Speicher umfasst, und wobei die erste Logik eine Vielzahl von Kompressionsverhältnissen basierend auf einem Frequenzbereich der Bins anwendet.
- System nach Anspruch 1, wobei die zweite Logik umfasst:Computerprogrammcode, der jedes Bin indiziert, das eine bestimmte Spitzengröße enthält, die eine größte Größe in jedem Teilband aufweist; undComputerprogrammcode, der die Phase jedes Bins, welches die bestimmte Spitzengröße enthält, an die bestimmte Phase in jedem Teilband anpasst, wobei die Anpassung eine Vektordrehung über eine Phasendifferenz umfasst, die in dem computerlesbaren Medium gespeichert ist.
- System nach Anspruch 1, ferner umfassend eine akustische Echo-Unterdrückungsvorrichtung, die das Frequenzspektrum verarbeitet nach der verlustbehafteten Kompression und bevor die dritte Logik die Größendaten und Phasendaten bereitstellt.
- System nach Anspruch 1, ferner umfassend eine Rauschunterdrückungsvorrichtung, die das Frequenzspektrum verarbeitet nach der verlustbehafteten Kompression und bevor die dritte Logik die Größendaten und Phasendaten bereitstellt.
- System nach Anspruch 1, ferner umfassend einen Strahlformer, der das Frequenzspektrum verarbeitet nach der verlustbehafteten Kompression und bevor die dritte Logik die Größendaten und die Phasendaten bereitstellt.
- System nach Anspruch 1, wobei die dritte Logik umfasst:Computerprogrammcode, der einen Vektor, der jede der bestimmten Größen in jedem Teilband darstellt, in eine ursprüngliche Phasenposition dreht; undComputerprogrammcode, der die Bins, welche die Teilbänder umfassen, die von der ersten Logik wiedergegeben werden, durch Wiederaufbauen und Bewahren der relativen Größen und relativen Phasen zwischen den Bins des Frequenzspektrums, das von der ersten Logik partitioniert wird, wiederherstellt.
- Verlustfreies Kompressionsverfahren, umfassend:Partitionieren und Speichern eines Frequenzspektrums von Tondaten, die eine gewisse Anzahl von Bins von reellen und imaginären Daten umfassen, in eine kleinere Anzahl von Teilbändern als die Anzahl von Bins;gekennzeichnet durch folgende Schritte:Komprimieren des Frequenzspektrums von Tondaten basierend auf einer bestimmten Spitzengröße eines Bins in jedem Teilband, die für dieses Teilband repräsentativ ist, und einer bestimmten Phase eines Bins in jedem Teilband, die für dieses Teilband repräsentativ ist, durch eine verlustbehaftete Kompression;Verarbeiten einer Vielzahl von Rahmen der Tondaten und Bestimmen eines gemeinsamen Bins in jedem Teilband als eine repräsentative Phase für jeden Rahmen von verarbeiteten Tondaten, wobei ein Index des gemeinsamen Bins, dessen Phase für jedes Teilband beibehalten wird, von einem Rahmen zu einem nächsten Rahmen konstant ist;Dekomprimieren der verlustbehafteten komprimierten Daten, die durch die verlustbehaftete Kompression wiedergegeben werden; undWiederaufbauen und Wiederherstellen von Größendaten und Phasendaten, die durch die verlustbehaftete Kompression nicht bewahrt werden, basierend auf ursprünglichen spektralen Beziehungen, die in dem Frequenzspektrum enthalten sind, so dass die wiederhergestellten Größendaten und die wiederhergestellten Phasendaten eine relative Größe und eine relative Phase zwischen den Bins des Frequenzspektrums bewahren, wobei die ursprünglichen spektralen Beziehungen ursprüngliche Größen- und Phasenbeziehungen in Bezug auf das Bin angeben, das für jedes Teilband beibehalten wird.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP14157074.7A EP2755205B1 (de) | 2010-01-29 | 2011-01-24 | Subband-Verarbeitung zur Komplexitätsverringerung |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/696,533 US8457976B2 (en) | 2009-01-30 | 2010-01-29 | Sub-band processing complexity reduction |
Related Child Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP14157074.7A Division EP2755205B1 (de) | 2010-01-29 | 2011-01-24 | Subband-Verarbeitung zur Komplexitätsverringerung |
EP14157074.7A Division-Into EP2755205B1 (de) | 2010-01-29 | 2011-01-24 | Subband-Verarbeitung zur Komplexitätsverringerung |
Publications (2)
Publication Number | Publication Date |
---|---|
EP2355094A1 EP2355094A1 (de) | 2011-08-10 |
EP2355094B1 true EP2355094B1 (de) | 2017-04-12 |
Family
ID=44023086
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP14157074.7A Active EP2755205B1 (de) | 2010-01-29 | 2011-01-24 | Subband-Verarbeitung zur Komplexitätsverringerung |
EP11151856.9A Active EP2355094B1 (de) | 2010-01-29 | 2011-01-24 | Subband zur Verarbeitung der Komplexitätsverringerung |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP14157074.7A Active EP2755205B1 (de) | 2010-01-29 | 2011-01-24 | Subband-Verarbeitung zur Komplexitätsverringerung |
Country Status (2)
Country | Link |
---|---|
EP (2) | EP2755205B1 (de) |
CA (1) | CA2729707C (de) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10984808B2 (en) | 2019-07-09 | 2021-04-20 | Blackberry Limited | Method for multi-stage compression in sub-band processing |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE4212339A1 (de) * | 1991-08-12 | 1993-02-18 | Standard Elektrik Lorenz Ag | Codierverfahren fuer audiosignale mit 32 kbit/s |
KR100363259B1 (ko) * | 2000-05-16 | 2002-11-30 | 삼성전자 주식회사 | 인지 특성 가중 함수를 이용한 음성신호의 위상 양자화장치 및 방법 |
KR100851970B1 (ko) * | 2005-07-15 | 2008-08-12 | 삼성전자주식회사 | 오디오 신호의 중요주파수 성분 추출방법 및 장치와 이를이용한 저비트율 오디오 신호 부호화/복호화 방법 및 장치 |
EP1852848A1 (de) * | 2006-05-05 | 2007-11-07 | Deutsche Thomson-Brandt GmbH | Verfahren und Vorrichtung für verlustfreie Kodierung eines Quellensignals unter Verwendung eines verlustbehafteten kodierten Datenstroms und eines verlustfreien Erweiterungsdatenstroms |
-
2011
- 2011-01-24 EP EP14157074.7A patent/EP2755205B1/de active Active
- 2011-01-24 EP EP11151856.9A patent/EP2355094B1/de active Active
- 2011-01-27 CA CA2729707A patent/CA2729707C/en active Active
Non-Patent Citations (1)
Title |
---|
None * |
Also Published As
Publication number | Publication date |
---|---|
CA2729707C (en) | 2014-04-01 |
CA2729707A1 (en) | 2011-07-29 |
EP2755205A1 (de) | 2014-07-16 |
EP2355094A1 (de) | 2011-08-10 |
EP2755205B1 (de) | 2019-12-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10269362B2 (en) | Methods, apparatus and systems for determining reconstructed audio signal | |
US9225318B2 (en) | Sub-band processing complexity reduction | |
US6263312B1 (en) | Audio compression and decompression employing subband decomposition of residual signal and distortion reduction | |
KR100913987B1 (ko) | 다중-채널 출력 신호를 발생시키기 위한 다중-채널합성장치 및 방법 | |
US8032387B2 (en) | Audio coding system using temporal shape of a decoded signal to adapt synthesized spectral components | |
JP3093179B2 (ja) | 高品質オーディオ用短時間遅延変換エンコーダ及びデコーダ | |
JP5539203B2 (ja) | 改良された音声及びオーディオ信号の変換符号化 | |
JP3297051B2 (ja) | 適応ビット配分符号化装置及び方法 | |
RU2439718C1 (ru) | Способ и устройство для обработки звукового сигнала | |
US20080140405A1 (en) | Audio coding system using characteristics of a decoded signal to adapt synthesized spectral components | |
JPH10511243A (ja) | 知覚符号化システムのサブバンドに波形予測を適用する装置及び方法 | |
US10170126B2 (en) | Effective attenuation of pre-echoes in a digital audio signal | |
EP2227682A1 (de) | Ein kodierer | |
US8665914B2 (en) | Signal analysis/control system and method, signal control apparatus and method, and program | |
JP3297050B2 (ja) | デコーダスペクトル歪み対応電算式適応ビット配分符号化方法及び装置 | |
EP2355094B1 (de) | Subband zur Verarbeitung der Komplexitätsverringerung | |
EP3764353A1 (de) | Verfahren zur mehrstufigen kompression in der unterbandverarbeitung | |
JP5491193B2 (ja) | 音声コード化の方法および装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20110124 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
17Q | First examination report despatched |
Effective date: 20120220 |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: QNX SOFTWARE SYSTEMS LIMITED |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: 2236008 ONTARIO INC. |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G10L 19/02 20130101AFI20161013BHEP Ipc: G10L 25/18 20130101ALI20161013BHEP Ipc: G10L 21/0208 20130101ALN20161013BHEP |
|
INTG | Intention to grant announced |
Effective date: 20161028 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G10L 25/18 20130101ALI20161019BHEP Ipc: G10L 21/0208 20130101ALN20161019BHEP Ipc: G10L 19/02 20130101AFI20161019BHEP |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 884616 Country of ref document: AT Kind code of ref document: T Effective date: 20170515 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602011036813 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20170412 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 884616 Country of ref document: AT Kind code of ref document: T Effective date: 20170412 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170412 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170713 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170412 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170412 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170712 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170412 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170412 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170412 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170812 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170712 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170412 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170412 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170412 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170412 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602011036813 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 8 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170412 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170412 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170412 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170412 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170412 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170412 Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170412 |
|
26N | No opposition filed |
Effective date: 20180115 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170412 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180124 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20180131 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180131 Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180131 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180131 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180124 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170412 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180124 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170412 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20110124 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170412 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R081 Ref document number: 602011036813 Country of ref document: DE Owner name: MALIKIE INNOVATIONS LTD., IE Free format text: FORMER OWNER: 2236008 ONTARIO INC., WATERLOO, ONTARIO, CA Ref country code: DE Ref legal event code: R082 Ref document number: 602011036813 Country of ref document: DE Representative=s name: MERH-IP MATIAS ERNY REICHL HOFFMANN PATENTANWA, DE Ref country code: DE Ref legal event code: R081 Ref document number: 602011036813 Country of ref document: DE Owner name: BLACKBERRY LIMITED, WATERLOO, CA Free format text: FORMER OWNER: 2236008 ONTARIO INC., WATERLOO, ONTARIO, CA |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20170412 Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170412 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170412 |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: 732E Free format text: REGISTERED BETWEEN 20200730 AND 20200805 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20240129 Year of fee payment: 14 Ref country code: GB Payment date: 20240123 Year of fee payment: 14 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R082 Ref document number: 602011036813 Country of ref document: DE Ref country code: DE Ref legal event code: R081 Ref document number: 602011036813 Country of ref document: DE Owner name: MALIKIE INNOVATIONS LTD., IE Free format text: FORMER OWNER: BLACKBERRY LIMITED, WATERLOO, ONTARIO, CA |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20240125 Year of fee payment: 14 |