EP2452337A1 - Allocation of bits in an enhancement coding/decoding for improving a hierarchical coding/decoding of digital audio signals - Google Patents
Allocation of bits in an enhancement coding/decoding for improving a hierarchical coding/decoding of digital audio signalsInfo
- Publication number
- EP2452337A1 EP2452337A1 EP10745328A EP10745328A EP2452337A1 EP 2452337 A1 EP2452337 A1 EP 2452337A1 EP 10745328 A EP10745328 A EP 10745328A EP 10745328 A EP10745328 A EP 10745328A EP 2452337 A1 EP2452337 A1 EP 2452337A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- coding
- decoding
- bits
- allocated
- band
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000005236 sound signal Effects 0.000 title claims abstract description 9
- 238000000034 method Methods 0.000 claims abstract description 27
- 238000004590 computer program Methods 0.000 claims description 5
- 239000010410 layer Substances 0.000 description 29
- OVOUKWFJRHALDD-UHFFFAOYSA-N 2-[2-(2-acetyloxyethoxy)ethoxy]ethyl acetate Chemical compound CC(=O)OCCOCCOCCOC(C)=O OVOUKWFJRHALDD-UHFFFAOYSA-N 0.000 description 23
- 230000006872 improvement Effects 0.000 description 17
- 238000001228 spectrum Methods 0.000 description 11
- 230000003595 spectral effect Effects 0.000 description 10
- 238000013139 quantization Methods 0.000 description 8
- 238000001914 filtration Methods 0.000 description 5
- 230000015654 memory Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 230000015572 biosynthetic process Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 238000003786 synthesis reaction Methods 0.000 description 3
- 241001362574 Decodes Species 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000002592 echocardiography Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 239000012792 core layer Substances 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000017105 transposition Effects 0.000 description 1
- 230000003936 working memory Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
- G10L19/16—Vocoder architecture
- G10L19/18—Vocoders using multiple modes
- G10L19/24—Variable rate codecs, e.g. for generating different qualities using a scalable representation such as hierarchical encoding or layered encoding
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/002—Dynamic bit allocation
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
- G10L19/0212—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using orthogonal transformation
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
- G10L19/032—Quantisation or dequantisation of spectral components
- G10L19/038—Vector quantisation, e.g. TwinVQ audio
Definitions
- the present invention relates to a binary allocation method for sound data processing.
- This processing is adapted in particular to the transmission and / or storage of digital signals such as audio-frequency signals (speech, music, or other).
- the invention applies more particularly to hierarchical coding (or "scalable” coding) that generates a so-called “hierarchical” bit stream because it comprises a core rate and one or more improvement layer (s) (the standardized coding according to G .722 at 48, 56 and 64 kbit / s being typically bitrate scalable, while the ITU-T G.729.1 and MPEG-4 CELP codecs are scalable in both rate and bandwidth).
- s improvement layer
- Hierarchical coding having the capacity to provide varied bit rates, is described below by distributing the information relating to an audio signal to be coded in hierarchical subsets, so that this information can be used in order of importance. in terms of audio rendering quality.
- the criterion taken into account for determining the order is a criterion for optimizing (or rather reducing) the quality of the coded audio signal.
- Hierarchical coding is particularly suited to transmission over heterogeneous networks or having variable available rates over time, or to transmission to terminals with varying capacities.
- the basic concept of hierarchical audio coding can be described as follows.
- the bit stream includes a base layer and one or more enhancement layers.
- the base layer is generated by a fixed rate codec, called “core coded”, which guarantees the minimum quality of the coding.
- This layer must be received by the decoder to maintain an acceptable level of quality. Improvement layers are used to improve quality. However, they may not all be received by the decoder.
- the main advantage of hierarchical coding is that it allows an adaptation of the bit rate simply by "truncation of the bit stream".
- the number of layers i.e., the number of possible truncations of the bitstream
- the number of layers defines the granularity of the coding.
- scalable bandwidth and scalability encoding techniques are described below, with a CELP core-type coder, in a telephone band, and one or more broadband enhancement layer (s).
- An example of such systems is given in the ITU-T G.729.1 8-32 kbit / s fine grain standard.
- the G.729.1 coding / decoding algorithm is summarized below.
- the G.729.1 encoder is an extension of the ITU-T G.729 coder. It is a modified G.729 heart-shaped hierarchical encoder producing a bandwidth ranging from narrow band (50-4000 Hz) to wide band (50-7000 Hz) at a rate of 8 to 32 kbit / s for conversational services. This codec is compatible with existing VoIP devices that use the G.729 codec.
- the G.729.1 coder is shown diagrammatically in FIG. 1.
- the broadband input signal s WB sampled at 16 kHz, is first broken down into two subbands by QMF (for "Quadrature Mirror Filter”) filtering.
- the low band (0-4000 Hz) is obtained by LP low-pass filtering (block 100) and decimation (block 101), and the high band (4000-8000 Hz) by HP high-pass filtering (block 102) and decimation (block 103).
- the LP and HP filters are of length 64.
- the low band is preprocessed by a high-pass filter eliminating the components below 50 Hz (block 104), to obtain the signal s LB , before coding
- the narrow-band CELP coding is a cascaded CELP coding comprising as a first stage a modified G.729 coding without a filter. preprocessing and as a second stage an additional fixed CELP dictionary.
- the high band is first pretreated (block 106) to compensate for the folding due to the high-pass filter (block 102) combined with the decimation (block 103).
- the high band is then filtered by a low pass filter (block 107) eliminating the components between 3000 and 4000 Hz from the high band (that is, the components between 7000 and
- a parametric band extension (block 108) is then performed.
- the error signal d LB of the low band is calculated (block 109) from the output of the CELP coder (block 105) and a transform predictive coding (TDAC type for "Time Domain Aliasing Cancellation" in the standard G .729.1) is carried out at block 110.
- TDAC type for "Time Domain Aliasing Cancellation" in the standard G .729.1
- Additional parameters can be transmitted by the block 111 to a homologous decoder, this block 111 performing a so-called “FEC” treatment for "Frame Erasure Concealment", in order to reconstitute possible erased frames.
- the different bit streams generated by the coding blocks 105, 108, 110 and 111 are finally multiplexed and structured into a hierarchical bit stream in the multiplexing block 112.
- the coding is performed by 20 ms sample blocks (or frames). 320 samples per frame.
- the G.729.1 codec therefore has a three-step coding architecture comprising: cascading CELP coding,
- TDAC transform predictive coding applied after an MDCT (Modified Discrete Cosine Transform) transformation.
- MDCT Modified Discrete Cosine Transform
- the G.729.1 decoder is illustrated in Figure 2.
- the bits describing each 20 ms frame are demultiplexed in the block 200.
- the bit stream of the 8 and 12 kbit / s layers is used by the CELP decoder
- block 201 to generate the narrow-band synthesis (0-4000 Hz).
- the portion of the bit stream associated with the 14 kbit / s layer is decoded by the tape extension module (block 202).
- the portion of the bit stream associated with data rates greater than 14 kbit / s is decoded by the TDAC module (block 203).
- LJn treatment of pre-echoes and post-echoes is performed by the blocks 204 and 207 as well as an enrichment (block 205) and a post-processing of the low band (block 206).
- the broadband output signal s wh sampled at 16 kHz, is obtained via the QMF synthesis filterbank (blocks 209, 210, 211, 212 and 213) incorporating reverse folding (block 208).
- the TDAC type transform coding in the G.729.1 encoder is illustrated in FIG.
- the filter W LB (z) (block 300) is a perceptual weighting filter, with gain compensation, applied to the low band error signal d LB. MDCT transforms are then calculated (blocks 301 and 302) to obtain:
- MDCT transforms (blocks 301 and 302) apply to 20 ms of sampled signal at 8 kHz (160 coefficients).
- the spectrum Y (Jc) from the block 303 of fusion thus comprises 2 x 160, or 320 coefficients. It is defined as follows:
- a subband j comprises the coefficients F (A :) with sb bound bound (j) ⁇ k ⁇ sb _ bound (j + 1).
- the coefficients 280-319 corresponding to the frequency band 7000 Hz - 8000 Hz are not coded; they are set to zero at the decoder because the bandwidth of the codec is 50-7000 Hz.
- Table 1 TDAC Encoding Boundary Limits and Size
- the spectral envelope ⁇ og__rms (j) ⁇ ⁇ is calculated in block 304 according to the formula:
- This quantized value rms_index (j) is transmitted to the bit allocation block 306.
- two types of coding can be chosen according to a given criterion, and, more precisely, the rms values _index (j):
- LJn bit (0 or 1) is transmitted to the decoder to indicate the encoding mode that has been chosen.
- the number of bits allocated to each subband for its quantization is determined at block 306 from the quantized spectral envelope from block 305.
- the allocation of the bits performed minimizes the squared error while respecting the constraint of an integer number of bits allocated per subband and a maximum number of bits not to be exceeded.
- the spectral content of the subbands is then encoded by spherical vector quantization (block 307).
- the different bit streams generated by the blocks 305 and 307 are then multiplexed and structured into a hierarchical bit stream at the multiplexing block 308.
- the TDAC-type transform decoding step in the G.729.1 decoder is illustrated in FIG. 4.
- block 401 allows to find the allocation of bits (block 402).
- Envelope decoding (block 401) reconstructs the quantized values of the spectral envelope
- each of the subbands is found by inverse spherical vector quantization (block 403).
- the non-transmitted sub-bands, due to a lack of "budget" of bits, are extrapolated (block 404) from the MDCT transform of the signal at the output of the band extension block (block 202 of FIG. 2).
- the MDCT spectrum is separated into two (block 407): with first 160 coefficients corresponding to the spectrum D ⁇ 0 of the decoded difference signal in low band, filtered perceptually,
- IMDCT inverse MDCT transform
- W LB inverse perceptual weighting
- the purpose of the binary allocation is to distribute between each of the sub-bands a certain bit budget (variable) noted nbits _ VQ, with:
- nbits _VQ 351 - nbits _ rms, where nbits _ rms is the number of bits used by the coding of the spectral envelope.
- Table 2 Possible values of number of bits allocated in TDAC subbands.
- nbit (j) arg min nb _ coef (j) x (ip (j) - ⁇ ) - r
- ⁇ is a parameter optimized by dichotomy to satisfy the global constraint ⁇ nbit (j) ⁇ nbits_VQ
- SWB Super Wide Band
- This document describes a super-wideband encoding / decoding system having a G.729.1 or G.718 core encoding stage and a band extension stage.
- the core coding performs the coding of the frequency band from 0 to 7 kHz while the extension band performs coding in the frequency band from 7 to 14 kHz.
- a first extension coding layer is based on a parametric model based on two coding modes: a generic mode and a sinusoidal mode.
- the generic mode uses a transposition method in the MDCT domain for the artificial generation of high frequency MDCT coefficients (7-14 kHz) from low frequencies (0-7 kHz).
- the low frequency band for encoding a high frequency band is selected on a criterion of maximizing the normalized correlation.
- Sinusoidal mode is normally used for particularly harmonic or tonal signals. In this mode, the most energetic components are selected. We then transmit their positions, their amplitudes and their signs.
- This first layer is transmitted with a bit rate of 4 kbit / s.
- a second 7-14 kHz band enhancement layer is proposed which is based on the additional sinusoidal coding approaching the MDCT spectrum of the input signal.
- the bit allocation for this second extension layer is fixed once and for all.
- extension coding presented in this document improves the signal only in the extension frequency band from 7 to 14 kHz.
- the 0 to 7 kHz frequency band of the core coding is not changed.
- the decoder then directly uses the synthesized signal from the first TDBWE band extension coding layer for the 4-7kHz band, to bridge the non-band allocated.
- a band extension can accentuate the coding defects of the core layer.
- the present invention improves the situation.
- a first number of bits (nbit_enhanced (j)) is allocated to a coding / decoding correction coding / decoding in the first frequency band and according to a first coding / decoding mode and a second number of bits (nb_sin) are allocated to an encoding / decoding enhancement of the extension coding / decoding in the second frequency band and in a second coding / decoding mode.
- the allocation method makes it possible while performing an improvement of the frequency band extension coding of a core coding, of allocating additional bits to also correct the core coding in the first frequency band.
- the overall quality of the coded signal is thus improved.
- the method comprises the following steps:
- the allocation according to one embodiment of the invention makes it possible to allocate additional bits for these sub-bands. frequency in order to improve the core coding in these sub-bands and this while also guaranteeing an improvement for the extension coding.
- a minimum number of bits is set per frequency sub-band for the allocation of the first number of bits.
- each frequency sub-band has a guaranteed associated bit rate and therefore a guaranteed coding.
- the predetermined threshold is set to 0.
- the predetermined threshold is greater than 0 and if the first allocated number of bits is greater than the predetermined number of bits, the value of the threshold is reduced.
- the allocation is more adapted to the signal, a maximum correction of the core coding then being performed to optimize the maximum rate allocated. This optimization is done as and when adjusting the threshold.
- the method comprises a step of receiving tone information of a residual signal resulting from a difference between a signal from a first band extension layer and the original signal and in case of residual tone signal, the second number of bits allocated for coding / decoding enhancement of the band extension is larger than the first number.
- this tone information is calculated directly on the original signal, for example by detecting peak energy in the spectrum.
- the enhancement layer of the band extension is adapted to the type of signal that it has to code.
- the coding according to the extension coding mode being particularly adapted to the tonal type signal, the priority is thus given to this coding mode.
- the core coding / decoding is of G.729.1 standardized coding / decoding type, the first coding / decoding mode being a transform coding / decoding and the second coding / decoding mode being a parametric encoding / decoding.
- the present invention also relates to a bit allocation module in a coder / decoder for improving a hierarchical encoder / decoder of digital audio signals comprising a core coding / decoding module in a first frequency band and a coding / coding module. band extension decoding in a second frequency band.
- This allocation module comprises:
- bit_enhanced (j) means for allocating a first number of bits (nbit_enhanced (j)) to a correction coding / decoding module of the heart encoder / decoder in the first frequency band and according to a first coding / decoding mode, for a predetermined number of bits to be allocated for the enhancement coder / decoder, and
- nb_sin a second number of bits
- the invention relates to a hierarchical coder comprising an allocation module according to the invention.
- the invention also relates to a hierarchical decoder comprising an allocation module according to the invention.
- the invention relates to a computer program comprising code instructions for implementing the steps of an allocation method according to the invention, when they are executed by a processor.
- FIG. 1 illustrates the structure of a G.729.1 type encoder described above
- FIG. 2 illustrates the structure of a G.729.1 type decoder described above
- FIG. 3 illustrates the structure of a TDAC encoder included in the G.729.1 type encoder and described previously:
- FIG. 4 illustrates the structure of a TDAC decoder included in a G.729.1 decoder and as described above;
- FIG. 5 illustrates the structure of a frequency-band extended G.729.1 encoder in which the invention can be implemented;
- FIG. 6 illustrates the structure of a G.729.1 extended frequency band decoder in which the invention can be implemented
- FIG. 7 illustrates an enhancement coder comprising a bit allocation module according to the invention implementing an allocation method according to one embodiment of the invention
- FIG. 8 illustrates an exemplary hardware embodiment of an allocation module according to the invention
- Such an encoder as represented consists of an extension of the frequencies coded by the module 515, the frequency band used passing from [50Hz-7KHz] to
- the encoder as shown in FIG. 5 comprises the same modules as the G.729.1 core coding shown in FIG. 1 and an additional band extension module 515 which provides an extension signal to the multiplexing module 512.
- This extension coding module 515 operates in the frequency band from 7 to 14 kHz, said second frequency band with respect to the first frequency band ranging from 0 to 7 kHz of the core coding.
- This frequency band extension is calculated on the original full-band signal S SWB while the input signal of the core encoder is obtained by decimation (block 516) and low-pass filtering (block 517). At the output of these blocks, the broadband input signal S WB is obtained.
- the module 515 includes a first extension coding layer based on a parametric model based on two coding modes, a generic mode and a sinusoidal mode, depending on whether the original signal S WB is tonal or non-tonal as described in FIG. paper by Tammi, L. Laaksonen, A. Ramo, H. Toukomaa, entitled “Scalable Superwideband Extension for Wideband Coding", ICASSP, 2009.
- It also comprises a coding layer for improving this first coding layer by a sinusoidal mode coding and whose bit allocation is performed according to a bit allocation method as described with reference to FIG. 7.
- the extension module 515 receives information from the encoder
- TDAC 510 in particular, the number of bits allocated in the frequency sub-bands of the core coding.
- the allocation module as described later with reference to FIG. 7, is integrated in the extension module 515.
- this module is integrated with the TDAC module
- this module is independent of the two modules 510 and 515 and communicates the bit allocation results to the respective two modules.
- a bit allocation module allocates a first number of bits to a core coding correction coding in the first frequency band and according to a first coding mode, in this case a coding by transformed. This allocation is performed according to a predetermined number of bits to be allocated for the improvement coding.
- the module allocates a second number of bits to a coding for improving the extension coding in the second frequency band and according to a second coding mode, here the sinusoidal parametric mode.
- the bit rate allocation between these two models may be difficult. Indeed, there will usually be a waveform coding model for the core, for example a transform coder that attempts to best encode the original signal.
- a waveform coding model for the core, for example a transform coder that attempts to best encode the original signal.
- band extension more generally parametric models are used which are designed to represent high frequencies perceptually without focusing on accurately coding the waveform.
- the flow allocation between the two models can in this case be difficult.
- the criteria for improving the core coder and the band extension are different and can hardly be compared to each other.
- the TDAC coding module 510 receives an additional bit allocation for performing core coding correction in a number of subbands. It provides the multiplexing module in addition to the core coded signal, additional cores encoding correction coding bits.
- G.729.1 decoder in super-expanded mode is described with reference to FIG. 6. It comprises the same modules as the G.729.1 decoder described with reference to FIG. 6.
- band extension module 614 which receives from the demultiplexing module 600 the band extension signal as well as the improvement signal of the extension coding according to the allocation defined by the module. allocation device described with reference to Figure 7.
- the decoder also comprises the synthesis filter bank (blocks 616, 615) for obtaining the super-wideband output signal S SWh .
- the TDAC decoding module 603 receives from the multiplexing module, in addition to the coded core signal, additional cores correction bits according to the bit allocation defined by the allocation module described with reference to FIG. 7.
- the decoder thus described thus benefits from the improvement coding implemented by the enhancement coder as now described with reference to FIG. 7.
- bit allocation can not be recalculated to the decoder, this information is then transmitted in the corresponding enhancement layer.
- the decoder may perform the same binary allocation calculation as the encoder by distributing the rate between the core encoder correction and the band extension.
- the allocation module is based on the binary allocation of the core encoder and possibly on information coming from the first band extension layer, namely the tone indication.
- An allocation module as described with reference to FIG. 7 implements the allocation method according to the invention.
- This module can in the same way as for the encoder, integrate into the TDAC 603 decoder module, in the extension module 614 or be independent.
- FIG. 7 represents a bit allocation module 701 according to the invention and shows the main steps of a bit allocation method according to the invention.
- the block 306 shown in FIG. 7 corresponds to the bit allocation block for the core coding and as described in the TDAC coder of FIG. 3, for the G.729.1 core coding.
- This heart allocation block delivers bit allocation information nbit (j) of the core coding, by frequency subband of the core frequency band.
- This information is received by the joint bit allocation module 701. Based on a rate available for enhancement coding, the module 701 allocates a first number of nbit_enhanced (j) bits to perform a correction of the transformed type core encoding in a first frequency band and a second number of nb_sin bits. for parametric sinusoidal type coding, improvement of the extension coding in a second frequency band.
- the module 701 receives a number of bits allocated for core coding for each of the subbands of the first frequency band.
- This number of bits per subband is compared to a predetermined threshold.
- the module 701 allocates a minimum number of bits of a predefined value, for example 9 bits.
- the remaining available bits in relation to the bit rate allowed for improvement coding for example an allowable bit rate of 4 kbit / s, are allocated for the enhancement coding of the extension coding, i.e. the second extension coding layer as described with reference to FIG. 5.
- the threshold can be set to 0.
- the threshold can be set to 0.
- the predetermined threshold is greater than 0.
- a first test is performed with a minimum number of bits to be allocated for the sub-bands that have an allocation less than this threshold. In the case where many subbands have a bit allocation below the threshold, the available rate may be exceeded. In this case, the threshold is decreased to perform a second trial. This reduction can be done for example by dichotomy, until finding a threshold that allows to allocate the minimum number of bits per subbands.
- the remaining number of bits is then allocated for sinusoidal band extension coding. It corresponds to the number of sine waves that can be encoded for the enhancement coding enhancement coding.
- the allocation module 701 therefore provides a first bit allocation per sub-band, nbit-enhanced (j) to a core coding correction coding block 703 which performs a spherical vector quantization of a residual signal resulting from the quantization.
- the correction coding block 703 thus provides the multiplexer block 704 with a correction signal of the core coding according to the number of bits allocated for this coding.
- the allocation module 701 delivers a second allocation of bits nb_sin to a block 702 coding for improving the band extension coding.
- This coding block receives the signal from the first band extension layer
- the module 701 also receives tone information from the residual signal. This calculation of tone is given for example in the document ICASSP 2009 referenced above.
- the coded improvement signal from the block 702 is transmitted to the multiplexing block 704 according to the bit allocation determined by the allocation method.
- the improvement coding illustrated in this FIG. 7 is, for example, integrated in a G.729.1 super-wideband encoder as described with reference to FIG. 5.
- the allocation module is for example located in the Tape Expansion Module 515. It receives from the TDAC 510, the allocation information of the core coding. It transmits the first number of bits allocated to the TDAC encoder which performs the spherical vector quantization of the block 703. It transmits to the second coding layer of the extension module 515, the second number of bits allocated for the sinusoidal mode coding of the block 702.
- this bit allocation module is integrated with the TDAC module 510 of FIG. 5. It delivers the first number of bits allocated to the quantization block of the TDAC coder and the second number of bits allocated to the module. extension 515 for the enhancement coding of block 702.
- the allocation module is independent of the modules 510 and 515 and sends respectively to the two modules, the first allocated number of bits and the second allocated number of bits.
- This figure 7 represents the improvement coding stage.
- An allocation module 701 then gives the number of bits nbit_enhanced (j) for the improvement decoding
- SVQ decod heart decoding realized for example in the decoding module TDAC 603 of FIG. 6 and the number of bits nb_sin for the enhancement layer decoding enhancement (sinus decod), realized for example by the extension decoding module 614 of FIG. 6.
- FIG. 8 illustrates an allocation module comprising a PROC processor cooperating with a memory block BM comprising a storage and / or working memory MEM.
- This module comprises an input module adapted to receive a number of bits per sub-band nbit (j) of the first frequency band of a core encoder.
- the memory block BM may advantageously comprise a computer program comprising code instructions for implementing the steps of the allocation method in the sense of the invention, when these instructions are executed by the processor PROC, and in particular the steps for a predetermined number of bits to be allocated for improvement coding / decoding:
- the description of FIG. 7 repeats the steps of an algorithm of such a computer program.
- the computer program can also be stored on a memory medium readable by a reader of the module or an encoder integrating the allocation module or downloadable in the memory space thereof.
- the allocation module comprises an output module capable of transmitting the first number of bits nbit_enhanced (j) allocated for the correction coding of the core coding and a second number of nb_sin bits for the coding of improvement of the extension coding.
- This allocation module can be integrated in a G.729.1 hierarchical coder / decoder in super wide band or more generally in any hierarchical coder / decoder with extension in frequency band.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
- Spectroscopy & Molecular Physics (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
FR0954688A FR2947945A1 (en) | 2009-07-07 | 2009-07-07 | BIT ALLOCATION IN ENCODING / DECODING ENHANCEMENT OF HIERARCHICAL CODING / DECODING OF AUDIONUMERIC SIGNALS |
PCT/FR2010/051308 WO2011004098A1 (en) | 2009-07-07 | 2010-06-25 | Allocation of bits in an enhancement coding/decoding for improving a hierarchical coding/decoding of digital audio signals |
Publications (2)
Publication Number | Publication Date |
---|---|
EP2452337A1 true EP2452337A1 (en) | 2012-05-16 |
EP2452337B1 EP2452337B1 (en) | 2013-05-29 |
Family
ID=41531495
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP10745328.4A Not-in-force EP2452337B1 (en) | 2009-07-07 | 2010-06-25 | Allocation of bits in an enhancement coding/decoding for improving a hierarchical coding/decoding of digital audio signals |
Country Status (8)
Country | Link |
---|---|
US (1) | US8965775B2 (en) |
EP (1) | EP2452337B1 (en) |
KR (1) | KR101703810B1 (en) |
CN (1) | CN102511062B (en) |
CA (1) | CA2766777C (en) |
FR (1) | FR2947945A1 (en) |
WO (1) | WO2011004098A1 (en) |
ZA (1) | ZA201200906B (en) |
Families Citing this family (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8924222B2 (en) * | 2010-07-30 | 2014-12-30 | Qualcomm Incorporated | Systems, methods, apparatus, and computer-readable media for coding of harmonic signals |
US9208792B2 (en) | 2010-08-17 | 2015-12-08 | Qualcomm Incorporated | Systems, methods, apparatus, and computer-readable media for noise injection |
US9767822B2 (en) * | 2011-02-07 | 2017-09-19 | Qualcomm Incorporated | Devices for encoding and decoding a watermarked signal |
CN102737636B (en) * | 2011-04-13 | 2014-06-04 | 华为技术有限公司 | Audio coding method and device thereof |
NO2669468T3 (en) * | 2011-05-11 | 2018-06-02 | ||
CN102509547B (en) * | 2011-12-29 | 2013-06-19 | 辽宁工业大学 | Method and system for voiceprint recognition based on vector quantization based |
CN103854653B (en) * | 2012-12-06 | 2016-12-28 | 华为技术有限公司 | The method and apparatus of signal decoding |
BR112015025092B1 (en) | 2013-04-05 | 2022-01-11 | Dolby International Ab | AUDIO PROCESSING SYSTEM AND METHOD FOR PROCESSING AN AUDIO BITS FLOW |
CN104217727B (en) * | 2013-05-31 | 2017-07-21 | 华为技术有限公司 | Signal decoding method and equipment |
FR3007563A1 (en) * | 2013-06-25 | 2014-12-26 | France Telecom | ENHANCED FREQUENCY BAND EXTENSION IN AUDIO FREQUENCY SIGNAL DECODER |
JP6319753B2 (en) | 2013-12-02 | 2018-05-09 | 華為技術有限公司Huawei Technologies Co.,Ltd. | Encoding method and apparatus |
CN111312277B (en) | 2014-03-03 | 2023-08-15 | 三星电子株式会社 | Method and apparatus for high frequency decoding of bandwidth extension |
CN111105806B (en) * | 2014-03-24 | 2024-04-26 | 三星电子株式会社 | High-frequency band encoding method and apparatus, and high-frequency band decoding method and apparatus |
BR112016019838B1 (en) * | 2014-03-31 | 2023-02-23 | Fraunhofer-Gesellschaft Zur Forderung Der Angewandten Forschung E.V. | AUDIO ENCODER, AUDIO DECODER, ENCODING METHOD, DECODING METHOD, AND NON-TRANSITORY COMPUTER READABLE RECORD MEDIA |
US9847087B2 (en) | 2014-05-16 | 2017-12-19 | Qualcomm Incorporated | Higher order ambisonics signal compression |
US11276412B2 (en) | 2017-09-20 | 2022-03-15 | Voiceage Corporation | Method and device for efficiently distributing a bit-budget in a CELP codec |
WO2020253941A1 (en) * | 2019-06-17 | 2020-12-24 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Audio encoder with a signal-dependent number and precision control, audio decoder, and related methods and computer programs |
Family Cites Families (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
FR2849727B1 (en) * | 2003-01-08 | 2005-03-18 | France Telecom | METHOD FOR AUDIO CODING AND DECODING AT VARIABLE FLOW |
KR100923300B1 (en) * | 2003-03-22 | 2009-10-23 | 삼성전자주식회사 | Method and apparatus for encoding/decoding audio data using bandwidth extension technology |
US7343291B2 (en) * | 2003-07-18 | 2008-03-11 | Microsoft Corporation | Multi-pass variable bitrate media encoding |
FR2888699A1 (en) * | 2005-07-13 | 2007-01-19 | France Telecom | HIERACHIC ENCODING / DECODING DEVICE |
US8032359B2 (en) * | 2007-02-14 | 2011-10-04 | Mindspeed Technologies, Inc. | Embedded silence and background noise compression |
JP4871894B2 (en) * | 2007-03-02 | 2012-02-08 | パナソニック株式会社 | Encoding device, decoding device, encoding method, and decoding method |
JP4708446B2 (en) * | 2007-03-02 | 2011-06-22 | パナソニック株式会社 | Encoding device, decoding device and methods thereof |
KR100921867B1 (en) * | 2007-10-17 | 2009-10-13 | 광주과학기술원 | Apparatus And Method For Coding/Decoding Of Wideband Audio Signals |
-
2009
- 2009-07-07 FR FR0954688A patent/FR2947945A1/en not_active Withdrawn
-
2010
- 2010-06-25 CN CN2010800396761A patent/CN102511062B/en not_active Expired - Fee Related
- 2010-06-25 EP EP10745328.4A patent/EP2452337B1/en not_active Not-in-force
- 2010-06-25 CA CA2766777A patent/CA2766777C/en not_active Expired - Fee Related
- 2010-06-25 US US13/382,794 patent/US8965775B2/en not_active Expired - Fee Related
- 2010-06-25 WO PCT/FR2010/051308 patent/WO2011004098A1/en active Application Filing
- 2010-06-25 KR KR1020127003329A patent/KR101703810B1/en active IP Right Grant
-
2012
- 2012-02-07 ZA ZA2012/00906A patent/ZA201200906B/en unknown
Non-Patent Citations (1)
Title |
---|
See references of WO2011004098A1 * |
Also Published As
Publication number | Publication date |
---|---|
CN102511062A (en) | 2012-06-20 |
FR2947945A1 (en) | 2011-01-14 |
CN102511062B (en) | 2013-07-31 |
KR20120061826A (en) | 2012-06-13 |
EP2452337B1 (en) | 2013-05-29 |
US20120185256A1 (en) | 2012-07-19 |
ZA201200906B (en) | 2012-10-31 |
CA2766777A1 (en) | 2011-01-13 |
WO2011004098A1 (en) | 2011-01-13 |
US8965775B2 (en) | 2015-02-24 |
CA2766777C (en) | 2015-12-15 |
KR101703810B1 (en) | 2017-02-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2452337B1 (en) | Allocation of bits in an enhancement coding/decoding for improving a hierarchical coding/decoding of digital audio signals | |
EP2115741B1 (en) | Advanced encoding / decoding of audio digital signals | |
EP2452336B1 (en) | Improved coding /decoding of digital audio signals | |
EP1989706B1 (en) | Device for perceptual weighting in audio encoding/decoding | |
EP1905010B1 (en) | Hierarchical audio encoding/decoding | |
EP2366177B1 (en) | Encoding of an audio-digital signal with noise transformation in a scalable encoder | |
JP5161069B2 (en) | System, method and apparatus for wideband speech coding | |
EP2239731B1 (en) | Encoding device, decoding device, and method thereof | |
EP1692689B1 (en) | Optimized multiple coding method | |
CA2512179C (en) | Method for encoding and decoding audio at a variable rate | |
WO2007096551A2 (en) | Method for binary coding of quantization indices of a signal envelope, method for decoding a signal envelope and corresponding coding and decoding modules | |
WO2007107670A2 (en) | Method for post-processing a signal in an audio decoder | |
WO2010031951A1 (en) | Pre-echo attenuation in a digital audio signal | |
EP2203915A1 (en) | Transmission error dissimulation in a digital signal with complexity distribution | |
WO2011073600A1 (en) | Parametric stereo encoding/decoding having downmix optimisation | |
FR2737360A1 (en) | Audio digital signal coding method of successive sample blocks - using spectral analysis to select vector dictionary for each sample block and allocating vector and scalar quantisation bits | |
FR2987931A1 (en) | MODIFICATION OF THE SPECTRAL CHARACTERISTICS OF A LINEAR PREDICTION FILTER OF A AUDIONUMERIC SIGNAL REPRESENTED BY ITS COEFFICIENTS LSF OR ISF. |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20120203 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK SM TR |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: VIRETTE, DAVID Inventor name: BERTHET, PIERRE |
|
DAX | Request for extension of the european patent (deleted) | ||
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G10L 19/16 20130101ALI20130412BHEP Ipc: G10L 19/02 20130101AFI20130412BHEP |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 614841 Country of ref document: AT Kind code of ref document: T Effective date: 20130615 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: FRENCH |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602010007406 Country of ref document: DE Effective date: 20130801 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PUE Owner name: ORANGE, FR Free format text: FORMER OWNER: FRANCE TELECOM, FR |
|
RAP2 | Party data changed (patent owner data changed or rights of a patent transferred) |
Owner name: ORANGE |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 614841 Country of ref document: AT Kind code of ref document: T Effective date: 20130529 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130829 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130909 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130529 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130930 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130529 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130529 Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130529 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130529 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130830 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130929 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: VDEP Effective date: 20130529 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130529 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130829 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130529 |
|
BERE | Be: lapsed |
Owner name: FRANCE TELECOM Effective date: 20130630 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130529 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130529 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130529 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130529 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130529 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130529 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130529 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130529 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20130630 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20130625 |
|
26N | No opposition filed |
Effective date: 20140303 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602010007406 Country of ref document: DE Effective date: 20140303 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130529 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20140630 Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20140630 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130529 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130529 Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130529 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20100625 Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130529 Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130529 Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20130625 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 7 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20160524 Year of fee payment: 7 Ref country code: GB Payment date: 20160527 Year of fee payment: 7 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20160526 Year of fee payment: 7 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R119 Ref document number: 602010007406 Country of ref document: DE |
|
GBPC | Gb: european patent ceased through non-payment of renewal fee |
Effective date: 20170625 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: ST Effective date: 20180228 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180103 Ref country code: GB Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20170625 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20170630 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130529 |