EP1599868A1 - Method and device for the spectral reconstruction of an audio signal - Google Patents
Method and device for the spectral reconstruction of an audio signalInfo
- Publication number
- EP1599868A1 EP1599868A1 EP04716626A EP04716626A EP1599868A1 EP 1599868 A1 EP1599868 A1 EP 1599868A1 EP 04716626 A EP04716626 A EP 04716626A EP 04716626 A EP04716626 A EP 04716626A EP 1599868 A1 EP1599868 A1 EP 1599868A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- frequency
- coded
- core
- audio signal
- decoder
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000005236 sound signal Effects 0.000 title claims abstract description 91
- 238000000034 method Methods 0.000 title claims abstract description 31
- 230000003595 spectral effect Effects 0.000 title claims abstract description 30
- 238000001228 spectrum Methods 0.000 claims abstract description 100
- 230000000295 complement effect Effects 0.000 claims abstract description 10
- 238000004590 computer program Methods 0.000 claims abstract description 5
- 230000005540 biological transmission Effects 0.000 description 20
- 238000012546 transfer Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 230000008929 regeneration Effects 0.000 description 5
- 238000011069 regeneration method Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 4
- 230000010076 replication Effects 0.000 description 4
- 238000012937 correction Methods 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- 101000969688 Homo sapiens Macrophage-expressed gene 1 protein Proteins 0.000 description 2
- 102100021285 Macrophage-expressed gene 1 protein Human genes 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000003111 delayed effect Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000004061 bleaching Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 238000007493 shaping process Methods 0.000 description 1
- 230000017105 transposition Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/005—Correction of errors induced by the transmission channel, if related to the coding algorithm
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
- G10L19/0204—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using subband decomposition
- G10L19/0208—Subband vocoders
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
- G10L19/16—Vocoder architecture
- G10L19/18—Vocoders using multiple modes
- G10L19/24—Variable rate codecs, e.g. for generating different qualities using a scalable representation such as hierarchical encoding or layered encoding
Definitions
- the present invention relates to a method and a device for coding and decoding an audio signal using spectrum reconstruction techniques.
- the invention lies in improving the decoding of an audio signal coded by an encoder with spectral band limitation, called the core coder.
- an encoder with spectral band limitation called the core coder.
- This coding can be a bit rate reduction coding.
- Known rate reduction coders are, for example, transform type coders such as MPEG1, MPEG2 or MPEG4-GA coders, CELP type coders and even parametric type coders, such as a parametric MPEG4 type coder.
- the audio signal In bit rate audio coding, the audio signal often has to undergo bandwidth limitation when the bit rate becomes low. This bandwidth limitation is necessary to avoid the introduction of audible quantization noise into the coded signal. It is therefore desirable to complete as far as possible the full spectral content of the original signal.
- Bandwidth is known from the state of the art, such as, for example, the spectral widening method known as the HFR (High-Frequency Regeneration) method.
- the decoded low-frequency, band-limited signal is subjected to a non-linear device to obtain a signal enriched in harmonics.
- This signal after bleaching and shaping based on information describing the spectral envelope of the full band signal before coding, allows the generation of a high frequency signal corresponding to the high frequency content of the signal before coding.
- the core encoder and the band extension encoder distribute the bandwidth according to the suitable cut-off frequency.
- This type of system is particularly interesting for coding audio signals.
- Certain communication networks such as the Internet, wireless communication networks and the like do not guarantee perfect routing of data between the sender and the recipient. Certain data may thus never arrive at the recipient or arrive too late. By arriving too late, the recipient considers them lost. In these networks, the bandwidth available to route data also varies continuously and significantly.
- the invention attempts to solve the drawbacks of the prior art by proposing a method for coding an audio signal, in which a part of the frequency spectrum of the audio signal is coded with a spectral band limiting coder known as the core coder and wherein the complementary part of the frequency spectrum of the audio signal is coded with an extension coder, characterized in that at least part of the spectrum coded with the core coder is also coded with the extension coder.
- the invention proposes a device for coding an audio signal, in which a part of the frequency spectrum of the audio signal is coded with an encoder with limitation of spectral band called core coder and in which the complementary part of the frequency spectrum of the audio signal is coded with an extension coder, characterized in that it comprises means for coding at least part of the spectrum coded with the core coder with the extension coder.
- the cut-off frequency of the core encoder can be adapted to the operating conditions of the core encoder. More particularly; the coded digital signal is transferred over a network and the or each determined frequency is transferred with the coded digital signal.
- the decoder can process this information quickly by reading it in the coded digital signal.
- the core coder is a hierarchical coder and for each coding layer, at least one cutoff frequency of each coding layer is determined.
- the cut-off frequency of the core coder can be adapted to the operating conditions of the core coder. More precisely, from each coding layer of the coded digital signal is transferred over a network and the or each frequency determined for the layer is transferred with said layer.
- the decoder has all of the information quickly. No particular processing of the decoded signal is then necessary. More precisely, the part of the spectrum coded with the core coder and the extension coder is determined.
- the part of the audio signal coded by the two coders can change over time and take account, for example, of network conditions.
- the part of the frequency spectrum of the audio signal coded with the core encoder is the lower part of the frequency spectrum of the audio signal.
- the invention also relates to a method of spectral reconstruction of an audio signal encoded in the form of data, in which a part of the frequency spectrum of the audio signal is decoded with an encoder with limitation of spectral band called core encoder and in which the complementary part of the frequency spectrum of the audio signal is decoded with an extension decoder, characterized in that the method comprises:
- the invention provides a device for spectral reconstruction of an audio signal encoded in the form of data, in which a part of the frequency spectrum of the audio signal is decoded with an encoder with limitation of spectral band called core encoder and in which the complementary part of the frequency spectrum of the audio signal is decoded with an extension encoder, characterized in that the device comprises:
- the decoded signal will be of better quality, no spectral component of the signal is missing, the frequency spectrum decoded with the extension encoder being modified in accordance with the cut-off frequency of the signal decoded by the core encoder.
- the part of the frequency spectrum of the audio signal decoded with a core decoder is the low part of the frequency spectrum of the audio signal.
- the information representative of at least one cutoff frequency of the signal decoded by the core decoder is obtained by carrying out an estimation of the high cutoff frequency of the signal decoded by the core decoder.
- the core decoder is a hierarchical decoder and information representative of the bandwidth of the signal decoded by the core decoder is obtained for each layer of the decoded signal.
- the information representative of at least one cut-off frequency of the signal decoded by the core decoder is obtained from information included in the data stream comprising the coded digital signal.
- the core decoder is a hierarchical decoder and information representative of the bandwidth of the signal decoded by the core decoder is obtained for each layer of the decoded signal.
- the decoder can adapt the processing to each coding layer, the decoder has this information at each layer and can thus modify the frequency spectrum decoded with the extension decoder as a function of this information.
- the invention proposes a data signal representative of an encoded audio signal, in which a part of the frequency spectrum of the audio signal is encoded with an encoder with limitation of spectral band called core encoder and in which the complementary part of the spectrum frequency of the audio signal is encoded with an extension encoder, characterized in that the signal comprises part of the spectrum encoded with the core encoder and with the extension encoder.
- the signal also includes information representative of at least one cut-off frequency of the core encoder or of the extension encoder.
- the invention also relates to the computer program stored on an information medium, said program comprising instructions making it possible to implement the processing method described above, when it is loaded and executed by a computer system.
- Figs. la to ld represent the different frequency spectra of an audio signal coded with a core coder and an extension coder
- Figs. the to lg represent the different frequency spectra of an audio signal transmitted over a network and decoded with a core decoder and an extension decoder
- Figs. 2a to 2e represent the different frequency spectra of an audio signal coded with a hierarchical core coder and an extension coder
- Figs. la to ld represent the different frequency spectra of an audio signal coded with a core coder and an extension coder
- Figs. the to lg represent the different frequency spectra of an audio signal transmitted over a network and decoded with a core decoder and an extension decoder
- Figs. 2a to 2e represent the different frequency spectra of an audio signal coded with a hierarchical core coder and an extension coder
- 2f to 2i represent the different frequency spectra of an audio signal transmitted over a network and decoded with a hierarchical core decoder and an extension decoder;
- Figs. 3a to 3c represent the different frequency spectra of an audio signal coded with a core coder and an extension coder according to the invention;
- Figs. 3d to 3f represent the different frequency spectra of an audio signal transmitted over a network and decoded with a core decoder and an extension decoder according to the invention;
- Fig. 4a represents a block diagram describing the coding device according to the invention
- Fig. 4b represents a block diagram describing the main elements of a hierarchical core encoder
- - Fig. 5 represents a block diagram describing the decoding device according to the invention
- Fig. 6 shows according to the invention, the algorithm performed at the level of the coder
- Fig. 7 shows according to the invention, the algorithm performed at the decoder.
- Fig. 1a represents a frequency spectrum of an audio signal to be coded.
- the low frequencies of the spectrum (FIG. 1b) are coded by a core coder, while the high frequencies are coded by a d coder. 'extension. This part of the high frequencies is represented in FIG. the.
- Fig. the represents the frequency spectrum of an audio signal decoded with a core decoder, the coded audio signal having been transmitted over a network and certain data 10 have been lost.
- This type of loss is particularly troublesome for the information coded by the core coder.
- the absence of the data 10 constitutes a hole in the spectrum of the decoded frequencies and this hole creates significant noise disturbances such as hissing at the restitution of the sound signal.
- the information encoded by the extension encoder is much more limited in number.
- the frequency spectrum of an audio signal transmitted over a network and decoded with an extension decoder is considered to be correct. This is shown in Fig. If.
- the reconstruction of the audio signal respectively by the core decoder and the extension decoder shows in FIG. lg a frequency spectrum comprising frequency components 10 which have disappeared. These disappeared frequency components 10 significantly tarnish the quality of reproduction of the audio signal.
- Fig. 2a represents the frequency spectrum of the total audio signal to be coded by a hierarchical core coder and an extension coder.
- a hierarchical core encoder will successively encode different sub-portions of the frequency spectrum of the audio signal to be encoded.
- a first part of the spectrum for example the part containing the lowest frequency components, such as the spectrum shown in FIG. 2b, will be coded.
- the first layer This is called the first layer.
- Another part containing additional frequency components will be coded. It is the second layer, it is represented in FIG. 2c.
- the information representative of the lowest frequencies is generally transmitted in the first layers.
- the other layers are, for example, then transmitted in an order depending on the frequencies of the spectrum they represent.
- some of the layers among the transmitted layers have a higher priority than others.
- the layers comprising the lowest frequencies are regarded as priority
- the layers comprising the highest frequencies are regarded as least priority.
- the layers comprising the lowest frequencies are associated with high-performance error correcting codes, ensuring good decoding, therefore no transmission losses.
- Fig. 2d represents the part of the spectrum allocated to the band extension coder, it is identical to that described in FIG. the.
- FIG. 2e The cumulation of the three spectra of Figs. 2b, 2c and 2d then gives a total spectrum shown in FIG. 2e which is identical if not similar to the spectrum of FIG. 2a.
- Figs. 2f and 2g represent the frequency spectra of an audio signal decoded with a hierarchical core decoder comprising two layers of hierarchy, the coded audio signal having been transmitted over a network and some of whose layers have been lost. During the transmission of the first layer, the spectrum equivalent to this layer was not marred by transmission errors, as shown in FIG. 2 F.
- the spectrum equivalent to this layer comprises frequency components, 25 in FIG. 2g, absent.
- the part of the spectrum allocated to the band extension coder is identical to that described in FIG. the. It is shown in Fig 2h.
- the reconstruction of the audio signal respectively by the hierarchical core decoder and the extension decoder shows in FIG. 2i a frequency spectrum comprising frequency components 25 having disappeared.
- Fig. 3a represents the frequency spectrum of the total audio signal to be coded by a core coder and an extension coder according to the invention.
- the core encoder codes the low frequency components of the frequency spectrum of the audio signal, this is shown in FIG. 3b.
- the extension decoder encodes not only the high frequency components of the frequency spectrum of the audio signal to be coded but also a part of the low frequency components which the core coder codes . These components are shown in Fig. 3c.
- Fig. 3d represents the frequency spectrum of an audio signal decoded with a core decoder, the coded audio signal having been transmitted over a network and of which certain layers 31 have been lost.
- An estimation of the bandwidth of the audio signal decoded by the core decoder is carried out, if it is different from that expected, the core decoder informs the decoder of extension of the missing bandwidth.
- the extension decoder adapts the decoding so that the decoding also applies to the missing bandwidth.
- the frequency spectrum equivalent to the coded information received by the extension decoder is shown. This spectrum is composed of the components 32, 33 and 34. If no transmission error linked to the variation in network bandwidth or transmission errors has occurred, the information corresponding to component 34 is sufficient for decoding.
- the reconstruction of the audio signal respectively by the hierarchical core decoder and the extension decoder shows in FIG. 3f a frequency spectrum no longer comprising missing frequency components.
- the decoded audio signal remains of quality.
- Fig. 4a represents a block diagram describing the coding device according to the invention.
- the coding device consists of an analog to digital converter 400 which converts the analog signal to be coded into a digital signal.
- an analog to digital converter 400 which converts the analog signal to be coded into a digital signal.
- the analog to digital converter is not necessary.
- the digital signal is delivered to the core encoder which encodes this signal.
- the core encoder is for example a bit rate reduction encoder as conforming to one of the MPEGl, MPEG2 or MPEG4-GA standards, or a CELP type encoder, a hierarchical encoder, or even a parametric MPEG4 encoder.
- the output of the core encoder represents the signal data covering the frequency spectrum such as that shown in FIG. 3b.
- the band extension encoder is for example an HFR (High Frequency) type encoder
- the output of the band extension encoder represents the data of the signal envelope covering the frequency spectrum such as that shown in FIG. 3c.
- a cutoff frequency adjustment module 402 is connected to the band extension coder 403 and to the core coder 401.
- This module 402 defines the frequency spectrum that the extension coder takes into account for coding. This module 402 determines this spectrum as a function of the high cut-off frequency of the core encoder 401, and of a variable frequency band which allows the decoder according to the invention to be able to compensate for possible transmission losses.
- variable frequency band is adjusted so as to guarantee the good signal redialing for layers without a robust error correction code.
- the frequency spectrum of the core encoder 401 can be adjusted from the frequency spectrum of the extension encoder 403.
- the module 402 defines the frequency spectrum as the core encoder
- the coding device also comprises a multiplexer 404 which multiplexes the audio signals coded by the core coder 401 and by the extension coder 403.
- the module 402 transfers to the multiplexer 404 the information representative of the passband of the core encoder 401 or its cut-off frequencies, or even of the low cut-off frequency of the extension encoder 403 so that these these are included in the transmitted data.
- the inclusion is carried out in the case of a hierarchical coder for each coding layer.
- Fig. 4b represents a block diagram describing the main elements of a hierarchical core encoder.
- This hierarchical encoder can replace the encoder 401 previously described with reference to FIG. 4a.
- a hierarchical core encoder usually subdivides the frequency spectrum to be encoded into different layers.
- a layer represents a frequency band of the spectrum to be coded. The number of layers is variable and allows progressive transmission of the coded signal.
- the coder is made up of a first coder 410 which codes the lowest part of the frequency spectrum of the original signal.
- the coded information is transferred to a multiplexer 416 which transfers this data to the multiplexer 404.
- module 402 transfers to the multiplexer 404 the information representative of the bandwidth of the core encoder 410 so that it is included in the data stream associated with this layer.
- the coded information is also transferred to a decoder 411. This decoder decodes this information to then transmit it to a subtraction circuit 413 which will subtract from the original signal the decoded signal.
- the original signal was previously delayed by a delay equal to the coding time of the encoder 410 and to the decoding time of the decoder 411.
- the signal obtained at the output of the subtractor circuit is then the original signal in which the previously coded low frequency components have been removed to the nearest coding residue.
- This signal is again encoded by an encoder 415 which can be of the same type as the encoder 410.
- the frequency components of the signal are encoded greater than those encoded by the encoder 410.
- the coded information is transferred to a multiplexer 416 which transfers this data to the multiplexer 404.
- the module 402 transfers to the multiplexer 404 the information representative of the passband of the core coder 415 so that it is included in the data stream associated with this layer. It can also transfer the total number of coding layers, the high or low cut-off frequency of the 415 core encoder.
- each encoder can be variable.
- the invention is applicable for audio signals of monophonic, stereophonic or multichannel type.
- the bandwidth information transmitted by the encoder can be transmitted in a conjugate manner or in a preferential mode, the bandwidth of each of the channels can be deduced from the other channels by differential coding.
- Fig. 5 shows a block diagram describing the decoding device according to the invention.
- the decoding device consists of a demultiplexer 510 which separates the signals received via the network 405 into data intended for the core decoder 511 and in data intended for the extension decoder 512. It also extracts received signals, the information representative of the bandwidth of the core coder 401 of the coding device, of the coders 410 and 415 if the signal has been coded with a hierarchical coder, or even of the low cut-off frequency of the extension coder 403 of the coding device if these have been included in the fields transmitted.
- the core decoder 511 decodes the data to provide a decoded signal such as the signal shown in FIG. 3d.
- the core decoder 511 is for example a decoder as conforming to one of the MPEGl, MPEG2 or MPEG4-GA standards, or a CELP type decoder, a hierarchical decoder, or even a parametric MPEG4 decoder.
- the core decoder 511 comprises a module 511b for obtaining information representative of at least one cutoff frequency which estimates, according to a first embodiment, the frequency spectrum of the signal received by it.
- the module 511b achieves this for example by carrying out a time-frequency transformation on the decoded signal and by determining the frequency from which the energy of the signal becomes negligible. Preferably, this can be done with the assistance of a perception model.
- the decoder 511 more precisely its module 511b then transfers information representative of the cut-off frequency or of the bandwidth to the extension decoder 512.
- the extension decoder 512 selects, from the representative information transmitted by the decoder 511, from the coded data which it has received from the multiplexer 510, the data corresponding to a representation of the spectral envelope greater than the determined frequency by the encoder 511. In this way, the losses linked to the transmission of the coded signal are compensated.
- the core decoder 511 more precisely the module 511b for obtaining information representative of at least one cutoff frequency obtains from the demultiplexer 510, according to a second embodiment, the information representative of the passband of the core encoder 401 or of the coders 410 and 415 of the coding device, or even the number of layers of the coded signal, or even of the low cut-off frequency of the extension coder 403 of the coding device if these have been included in the transmitted data.
- the module 511b checks, if it is a hierarchical decoder, if each layer is well received and if not transfers information representative of the bandwidth of one or more lost layers to the extension decoder 512.
- the extension decoder 512 selects, from the representative information transmitted by the module 511b, from the coded data received from the multiplexer 510, the data corresponding to the signal envelope corresponding to a representation of the frequency spectral envelope higher than the lowest frequency corresponding to the lost frequency bands.
- the extension decoder corrects the losses due to the network, whether on losses affecting the last layers received or losses affecting an intermediate layer.
- the band extension decoder 512 is for example an HFR (High Frequency Regeneration) type decoder, for example an SBR (Spectral Band Replication) type decoder as described in the document "Audio Engineering Society, convention paper 5553", presented to the 112th AES convention by Mr. Martin Dietz. It should be noted that, as a variant, the extension decoder 512 decodes all of the information received. A selection from the decoded data is made so as to keep only those corresponding to a representation of the spectral envelope greater than the frequency determined by the coder 511.
- HFR High Frequency Regeneration
- SBR Spectral Band Replication
- the envelope decoded by the extension decoder 512 or selected is transferred to a gain control module 515.
- the signal decoded by the core decoder 511 is sent to a transposition module 513 which generates a signal in the high frequencies of the spectrum from the decoded low frequency signal.
- This signal is introduced into the gain control module 515 in order to allow the adjustment of the high frequency signal envelope.
- the adjusted envelope signal is then added to the signal decoded by the core decoder 511 with an adder 516.
- the adder 516 can in a preferred mode favor certain frequency components by multiplying for example certain components by coefficients. It should be noted that the signal decoded by the core decoder 511 was previously delayed by a delay equal to the difference in processing time between the added signals. This delay is effected by the delay circuit 514.
- the frequency spectrum of the signal obtained is thus similar to that of FIG. 3f.
- the summation signal can then be converted into analog form using a digital to analog converter 517.
- Fig. 6 shows the algorithm performed according to the invention at the level of the coder.
- the invention as described with reference to Figs. above is also achievable in software form in which a processor executes the executable code associated with steps E1 to E7 of the algorithm of FIG. 6.
- the processor When the coding device is switched on, and more particularly when a computer is used as a coding device, the processor reads from the computer's read-only memory or from a storage medium. information such as a CD-ROM compact disc, the program instructions corresponding to steps E1 to E7 of FIG. 6 and loads them into RAM memory to execute them.
- step E1 on receiving audio data to be coded, the processor determines the bandwidth of the core coder or at least one cutoff frequency.
- the bandwidth of the core encoder may or may not be variable over time depending for example on the load of the core encoder.
- the processor codes the data according to a so-called core coding algorithm conforming to one of the MPEG1, MPEG2 or MPEG4-GA standards, or of CELP type, of hierarchical type, or even of parametric MPEG4 type.
- Step E2 consists in checking whether, and in the case of a hierarchical coding, all the layers have been coded or not. If not, and if the core coding is a hierarchical coding, the processor repeats step E1 for each layer of the coded audio signal.
- step E3 the processor determines a frequency margin.
- This margin can be predetermined and stored in a register or be in the form of a variable. This variable depends for example on the type of error correction that will be applied to the coded data during their transmission over the network.
- step E4 the processor determines in step E4, from the margin and the high cutoff frequency of the core encoder, the low cutoff frequency of the extension encoder.
- the processor transfers this information to the extension coding subroutine in step E5.
- the processor stores this information.
- the processor in step E7, performs extension coding by encoding the data whose spectrum is greater than the information transferred in step E5.
- the band extension coding is for example an HFR (High Frequency Regeneration) type coding, for example SBR (Spectral Band Replication) as described in the document "Audio Engineering Society, convention paper 5553", presented to the 112th convention. AES by Mr Martin Dietz.
- step E7 which consists in multiplexing the audio signals coded in step E1 and the audio signals coded in step E7 to form a stream of coded data transmitted over a network.
- the processor inserts into the stream of coded and transmitted data, the information stored in step E6 or inserts one or more of the following information: bandwidth of the core encoder, bandwidth of the encoder extension, low and high frequency of each coding layer, number of coding layers if a hierarchical coder is used.
- the insertion is carried out in the case of a hierarchical coder for each coding layer.
- step E1 awaiting new audio data to be coded.
- Fig. 7 shows the algorithm performed according to the invention at the level of the decoder.
- the invention as described with reference to Figs. above is also achievable in software form in which a processor executes the code associated with steps El 0 to El 5 of the algorithm of FIG. 7.
- the processor When the receiving device is switched on, and more particularly when a computer is used as a receiving device, the processor reads from the computer's read-only memory or from an information medium. such as a compact disc CD-ROM, the program instructions corresponding to the steps
- the processor on receiving audio data to be decoded, separates the signals received via the network 405 into data intended for the core decoder and data intended for the extension decoder. It also extracts from the signals received, the information representative of the bandwidth or of at least one cutoff frequency of the core coder having coded the audio signal, or of coders having coded the audio signal if the signal has been coded with an coder hierarchical, or even the low cut-off frequency of the extension coder that coded the audio signal if these have been included in the transmitted data.
- step El i the processor then proceeds to decode this data.
- the processor proceeds to the decoding of the data according to a decoding algorithm called core decoding algorithm such as conforming to one of the MPEGl, MPEG2 or MPEG4-GA standards, or of CELP type, a hierarchical decoding, or even a decoding of parametric MPEG4 type .
- core decoding algorithm such as conforming to one of the MPEGl, MPEG2 or MPEG4-GA standards, or of CELP type, a hierarchical decoding, or even a decoding of parametric MPEG4 type .
- step E12 is a step of obtaining information representative of at least one cutoff frequency which estimates, according to a first embodiment, the frequency spectrum of the signal received by him. This is achieved for example by carrying out a time-frequency transformation on the signal decoded in step El i and by determining the frequency from which the energy of the signal becomes negligible. Preferably, this can be done with the assistance of a perception model.
- the processor obtains the information extracted in step E1 and checks, in the case where this is a hierarchical decoder, if each layer is well received and if not transfers information representative of the bandwidth one or more layers lost to the extension decoder.
- step El 3 consists in adapting the low cut-off frequency of the extension decoder so that it compensates for the losses due to the network.
- the adaptation is carried out on the basis of the information representative of the cut-off frequency or of the bandwidth obtained in step E12 or if the decoding of step El i is a hierarchical decoding of the information representative of the band frequency or cutoff frequency of one or more lost layers.
- step El 4 decodes, according to a so-called extension decoding algorithm, the data corresponding to the frequencies above this low cut-off frequency previously determined.
- the processor selects, from the adapted frequency, from the data separated in step E1 and intended for extension decoding, the data corresponding to the signal envelope corresponding to a representation of the spectral envelope of the frequencies greater than the lowest frequency corresponding to the lost frequency bands.
- the extension decoding corrects the losses due to the network, whether on losses affecting the last layers received or losses affecting an intermediate layer.
- Extension decoding is a band extension decoding algorithm, for example HFR (High Frequency Regeneration) type decoding, for example SBR (Spectral Band Replication) type decoding as described in the document.
- HFR High Frequency Regeneration
- SBR Spectrum Band Replication
- step El 5 the data decoded by the core decoder and the extension decoder are added to form the audio signal decoded in step El 5.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Quality & Reliability (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
- Stereophonic System (AREA)
Abstract
Description
Claims
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
FR0302730A FR2852172A1 (en) | 2003-03-04 | 2003-03-04 | Audio signal coding method, involves coding one part of audio signal frequency spectrum with core coder and another part with extension coder, where part of spectrum is coded with both core coder and extension coder |
FR0302730 | 2003-03-04 | ||
PCT/FR2004/000488 WO2004081918A1 (en) | 2003-03-04 | 2004-03-03 | Method and device for the spectral reconstruction of an audio signal |
Publications (2)
Publication Number | Publication Date |
---|---|
EP1599868A1 true EP1599868A1 (en) | 2005-11-30 |
EP1599868B1 EP1599868B1 (en) | 2010-05-19 |
Family
ID=32865273
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP04716626A Expired - Lifetime EP1599868B1 (en) | 2003-03-04 | 2004-03-03 | Method and device for the spectral reconstruction of an audio signal |
Country Status (9)
Country | Link |
---|---|
US (1) | US7720676B2 (en) |
EP (1) | EP1599868B1 (en) |
JP (1) | JP4660470B2 (en) |
KR (1) | KR101091593B1 (en) |
AT (1) | ATE468584T1 (en) |
DE (1) | DE602004027219D1 (en) |
ES (1) | ES2345489T3 (en) |
FR (1) | FR2852172A1 (en) |
WO (1) | WO2004081918A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8069035B2 (en) | 2005-10-14 | 2011-11-29 | Panasonic Corporation | Scalable encoding apparatus, scalable decoding apparatus, and methods of them |
Families Citing this family (33)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8605911B2 (en) | 2001-07-10 | 2013-12-10 | Dolby International Ab | Efficient and scalable parametric stereo coding for low bitrate audio coding applications |
SE0202159D0 (en) | 2001-07-10 | 2002-07-09 | Coding Technologies Sweden Ab | Efficientand scalable parametric stereo coding for low bitrate applications |
DE60202881T2 (en) | 2001-11-29 | 2006-01-19 | Coding Technologies Ab | RECONSTRUCTION OF HIGH-FREQUENCY COMPONENTS |
SE0202770D0 (en) | 2002-09-18 | 2002-09-18 | Coding Technologies Sweden Ab | Method of reduction of aliasing is introduced by spectral envelope adjustment in real-valued filterbanks |
US8010349B2 (en) * | 2004-10-13 | 2011-08-30 | Panasonic Corporation | Scalable encoder, scalable decoder, and scalable encoding method |
WO2006049204A1 (en) * | 2004-11-05 | 2006-05-11 | Matsushita Electric Industrial Co., Ltd. | Encoder, decoder, encoding method, and decoding method |
KR100818268B1 (en) | 2005-04-14 | 2008-04-02 | 삼성전자주식회사 | Apparatus and method for audio encoding/decoding with scalability |
RU2418385C2 (en) * | 2005-07-14 | 2011-05-10 | Конинклейке Филипс Электроникс Н.В. | Coding and decoding of sound |
US8626503B2 (en) * | 2005-07-14 | 2014-01-07 | Erik Gosuinus Petrus Schuijers | Audio encoding and decoding |
WO2007119368A1 (en) * | 2006-03-17 | 2007-10-25 | Matsushita Electric Industrial Co., Ltd. | Scalable encoding device and scalable encoding method |
US7461106B2 (en) * | 2006-09-12 | 2008-12-02 | Motorola, Inc. | Apparatus and method for low complexity combinatorial coding of signals |
JP4918841B2 (en) * | 2006-10-23 | 2012-04-18 | 富士通株式会社 | Encoding system |
US8295507B2 (en) * | 2006-11-09 | 2012-10-23 | Sony Corporation | Frequency band extending apparatus, frequency band extending method, player apparatus, playing method, program and recording medium |
WO2008084688A1 (en) * | 2006-12-27 | 2008-07-17 | Panasonic Corporation | Encoding device, decoding device, and method thereof |
JP4708446B2 (en) | 2007-03-02 | 2011-06-22 | パナソニック株式会社 | Encoding device, decoding device and methods thereof |
GB0705328D0 (en) * | 2007-03-20 | 2007-04-25 | Skype Ltd | Method of transmitting data in a communication system |
BRPI0815972B1 (en) * | 2007-08-27 | 2020-02-04 | Ericsson Telefon Ab L M | method for spectrum recovery in spectral decoding of an audio signal, method for use in spectral encoding of an audio signal, decoder, and encoder |
DK2186089T3 (en) * | 2007-08-27 | 2019-01-07 | Ericsson Telefon Ab L M | Method and apparatus for perceptual spectral decoding of an audio signal including filling in spectral holes |
US8576096B2 (en) * | 2007-10-11 | 2013-11-05 | Motorola Mobility Llc | Apparatus and method for low complexity combinatorial coding of signals |
US20090234642A1 (en) * | 2008-03-13 | 2009-09-17 | Motorola, Inc. | Method and Apparatus for Low Complexity Combinatorial Coding of Signals |
FR2929466A1 (en) * | 2008-03-28 | 2009-10-02 | France Telecom | DISSIMULATION OF TRANSMISSION ERROR IN A DIGITAL SIGNAL IN A HIERARCHICAL DECODING STRUCTURE |
US8639519B2 (en) * | 2008-04-09 | 2014-01-28 | Motorola Mobility Llc | Method and apparatus for selective signal coding based on core encoder performance |
EP2313885B1 (en) * | 2008-06-24 | 2013-02-27 | Telefonaktiebolaget L M Ericsson (PUBL) | Multi-mode scheme for improved coding of audio |
US8423355B2 (en) * | 2010-03-05 | 2013-04-16 | Motorola Mobility Llc | Encoder for audio signal including generic audio and speech frames |
US8428936B2 (en) * | 2010-03-05 | 2013-04-23 | Motorola Mobility Llc | Decoder for audio signal including generic audio and speech frames |
CN102208188B (en) | 2011-07-13 | 2013-04-17 | 华为技术有限公司 | Audio signal encoding-decoding method and device |
WO2013142650A1 (en) | 2012-03-23 | 2013-09-26 | Dolby International Ab | Enabling sampling rate diversity in a voice communication system |
EP2830064A1 (en) | 2013-07-22 | 2015-01-28 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Apparatus and method for decoding and encoding an audio signal using adaptive spectral tile selection |
US20150170655A1 (en) * | 2013-12-15 | 2015-06-18 | Qualcomm Incorporated | Systems and methods of blind bandwidth extension |
EP3107096A1 (en) | 2015-06-16 | 2016-12-21 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Downscaled decoding |
MY191093A (en) * | 2016-02-17 | 2022-05-30 | Fraunhofer Ges Forschung | Post-processor, pre-processor, audio encoder, audio decoder and related methods for enhancing transient processing |
WO2017153006A1 (en) * | 2016-03-07 | 2017-09-14 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Hybrid concealment method: combination of frequency and time domain packet loss concealment in audio codecs |
US20240212704A1 (en) * | 2021-09-22 | 2024-06-27 | Boe Technology Group Co., Ltd. | Audio adjusting method, device and apparatus, and storage medium |
Family Cites Families (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5495552A (en) * | 1992-04-20 | 1996-02-27 | Mitsubishi Denki Kabushiki Kaisha | Methods of efficiently recording an audio signal in semiconductor memory |
FR2729030B1 (en) * | 1994-12-30 | 1997-03-28 | France Telecom | METHOD FOR DYNAMICALLY RECONFIGURING A SIGNAL HAVING A TIME INTERLACEMENT, CORRESPONDING RECEIVER AND SIGNAL |
JP3344944B2 (en) * | 1997-05-15 | 2002-11-18 | 松下電器産業株式会社 | Audio signal encoding device, audio signal decoding device, audio signal encoding method, and audio signal decoding method |
US6023233A (en) * | 1998-03-20 | 2000-02-08 | Craven; Peter G. | Data rate control for variable rate compression systems |
FR2791167B1 (en) * | 1999-03-17 | 2003-01-10 | Matra Nortel Communications | AUDIO ENCODING, DECODING AND TRANSCODING METHODS |
US6226616B1 (en) * | 1999-06-21 | 2001-05-01 | Digital Theater Systems, Inc. | Sound quality of established low bit-rate audio coding systems without loss of decoder compatibility |
US6704703B2 (en) * | 2000-02-04 | 2004-03-09 | Scansoft, Inc. | Recursively excited linear prediction speech coder |
FI119576B (en) * | 2000-03-07 | 2008-12-31 | Nokia Corp | Speech processing device and procedure for speech processing, as well as a digital radio telephone |
US7742927B2 (en) * | 2000-04-18 | 2010-06-22 | France Telecom | Spectral enhancing method and device |
FI109393B (en) * | 2000-07-14 | 2002-07-15 | Nokia Corp | Method for encoding media stream, a scalable and a terminal |
EP1199812A1 (en) * | 2000-10-20 | 2002-04-24 | Telefonaktiebolaget Lm Ericsson | Perceptually improved encoding of acoustic signals |
SE0004187D0 (en) * | 2000-11-15 | 2000-11-15 | Coding Technologies Sweden Ab | Enhancing the performance of coding systems that use high frequency reconstruction methods |
DE60202881T2 (en) * | 2001-11-29 | 2006-01-19 | Coding Technologies Ab | RECONSTRUCTION OF HIGH-FREQUENCY COMPONENTS |
KR100636145B1 (en) * | 2004-06-04 | 2006-10-18 | 삼성전자주식회사 | Exednded high resolution audio signal encoder and decoder thereof |
EP1987513B1 (en) * | 2006-02-06 | 2009-09-09 | France Telecom | Method and device for the hierarchical coding of a source audio signal and corresponding decoding method and device, programs and signal |
FR2898443A1 (en) * | 2006-03-13 | 2007-09-14 | France Telecom | AUDIO SOURCE SIGNAL ENCODING METHOD, ENCODING DEVICE, DECODING METHOD, DECODING DEVICE, SIGNAL, CORRESPONDING COMPUTER PROGRAM PRODUCTS |
-
2003
- 2003-03-04 FR FR0302730A patent/FR2852172A1/en active Pending
-
2004
- 2004-03-03 ES ES04716626T patent/ES2345489T3/en not_active Expired - Lifetime
- 2004-03-03 US US10/547,759 patent/US7720676B2/en active Active
- 2004-03-03 AT AT04716626T patent/ATE468584T1/en not_active IP Right Cessation
- 2004-03-03 KR KR1020057016445A patent/KR101091593B1/en active IP Right Grant
- 2004-03-03 EP EP04716626A patent/EP1599868B1/en not_active Expired - Lifetime
- 2004-03-03 DE DE602004027219T patent/DE602004027219D1/en not_active Expired - Lifetime
- 2004-03-03 WO PCT/FR2004/000488 patent/WO2004081918A1/en active Application Filing
- 2004-03-03 JP JP2006505683A patent/JP4660470B2/en not_active Expired - Lifetime
Non-Patent Citations (1)
Title |
---|
See references of WO2004081918A1 * |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8069035B2 (en) | 2005-10-14 | 2011-11-29 | Panasonic Corporation | Scalable encoding apparatus, scalable decoding apparatus, and methods of them |
Also Published As
Publication number | Publication date |
---|---|
KR20060007371A (en) | 2006-01-24 |
JP4660470B2 (en) | 2011-03-30 |
US7720676B2 (en) | 2010-05-18 |
FR2852172A1 (en) | 2004-09-10 |
ES2345489T3 (en) | 2010-09-24 |
KR101091593B1 (en) | 2011-12-13 |
EP1599868B1 (en) | 2010-05-19 |
WO2004081918A1 (en) | 2004-09-23 |
DE602004027219D1 (en) | 2010-07-01 |
JP2006520487A (en) | 2006-09-07 |
ATE468584T1 (en) | 2010-06-15 |
US20060265087A1 (en) | 2006-11-23 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP1599868B1 (en) | Method and device for the spectral reconstruction of an audio signal | |
US7668723B2 (en) | Scalable lossless audio codec and authoring tool | |
US8374858B2 (en) | Scalable lossless audio codec and authoring tool | |
EP1581930B1 (en) | Method for encoding and decoding audio at a variable rate | |
FR2761801A1 (en) | AUDIO CODING METHOD AND DEVICE | |
WO2007010158A2 (en) | Method for switching rate- and bandwidth-scalable audio decoding rate | |
EP1794748A1 (en) | Data processing method by passage between different sub-band domains | |
EP0111612A1 (en) | Speech signal coding method and apparatus | |
FR2929466A1 (en) | DISSIMULATION OF TRANSMISSION ERROR IN A DIGITAL SIGNAL IN A HIERARCHICAL DECODING STRUCTURE | |
EP2228791B1 (en) | Scalable lossless audio codec and authoring tool | |
EP1037196B1 (en) | Method for coding, decoding and transcoding an audio signal | |
EP1356455B1 (en) | Method and device for processing numerous audio binary streams | |
WO2011086253A2 (en) | Improved method for encoding/decoding a stereo digital stream and associated encoding/decoding device | |
WO2008080609A1 (en) | Audio encoding method and device | |
FR2759510A1 (en) | AUDIO CODING METHOD AND DEVICE | |
FR2796785A1 (en) | COMMUNICATION SYSTEM, RECEIVER, DEVICE AND METHOD FOR CORRECTING CHANNEL ERRORS | |
CA2300646A1 (en) | Coding, decoding and transcoding procedures | |
FR2914480A1 (en) | Digital audio stream transmitting device for e.g. internet network, has selection unit for selecting two digital audio streams based on selection signal, and transmission unit for transmitting selected digital audio stream | |
FR2911738A1 (en) | TRANSCODING METHOD AND DEVICE IMPLEMENTING SAID METHOD. |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20050823 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PL PT RO SE SI SK TR |
|
AX | Request for extension of the european patent |
Extension state: AL LT LV MK |
|
DAX | Request for extension of the european patent (deleted) | ||
17Q | First examination report despatched |
Effective date: 20081023 |
|
RTI1 | Title (correction) |
Free format text: METHOD AND DEVICE FOR THE SPECTRAL RECONSTRUCTION OF AN AUDIO SIGNAL |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PL PT RO SE SI SK TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: FRENCH |
|
REF | Corresponds to: |
Ref document number: 602004027219 Country of ref document: DE Date of ref document: 20100701 Kind code of ref document: P |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: VDEP Effective date: 20100519 |
|
REG | Reference to a national code |
Ref country code: ES Ref legal event code: FG2A Ref document number: 2345489 Country of ref document: ES Kind code of ref document: T3 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100519 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100519 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100519 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100519 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100519 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100820 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100519 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FD4D |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100920 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100519 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100519 Ref country code: IE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100519 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100519 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100519 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100519 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100519 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
26N | No opposition filed |
Effective date: 20110222 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602004027219 Country of ref document: DE Effective date: 20110221 |
|
BERE | Be: lapsed |
Owner name: FRANCE TELECOM Effective date: 20110331 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20110331 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20110331 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20110331 Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20110331 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20110303 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100819 Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100519 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100519 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 13 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 14 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 15 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20230222 Year of fee payment: 20 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: IT Payment date: 20230221 Year of fee payment: 20 Ref country code: GB Payment date: 20230221 Year of fee payment: 20 Ref country code: DE Payment date: 20230221 Year of fee payment: 20 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: ES Payment date: 20230403 Year of fee payment: 20 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R071 Ref document number: 602004027219 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: PE20 Expiry date: 20240302 |
|
REG | Reference to a national code |
Ref country code: ES Ref legal event code: FD2A Effective date: 20240402 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: ES Free format text: LAPSE BECAUSE OF EXPIRATION OF PROTECTION Effective date: 20240304 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: ES Free format text: LAPSE BECAUSE OF EXPIRATION OF PROTECTION Effective date: 20240304 Ref country code: GB Free format text: LAPSE BECAUSE OF EXPIRATION OF PROTECTION Effective date: 20240302 |