EP2720477B1 - Synthèse virtuelle de graves à l'aide de transposition harmonique - Google Patents
Synthèse virtuelle de graves à l'aide de transposition harmonique Download PDFInfo
- Publication number
- EP2720477B1 EP2720477B1 EP13188415.7A EP13188415A EP2720477B1 EP 2720477 B1 EP2720477 B1 EP 2720477B1 EP 13188415 A EP13188415 A EP 13188415A EP 2720477 B1 EP2720477 B1 EP 2720477B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- frequency
- signal
- harmonics
- audio signal
- stage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000017105 transposition Effects 0.000 title claims description 67
- 230000015572 biosynthetic process Effects 0.000 title claims description 29
- 238000003786 synthesis reaction Methods 0.000 title claims description 29
- 230000005236 sound signal Effects 0.000 claims description 83
- 238000000034 method Methods 0.000 claims description 54
- 230000004044 response Effects 0.000 claims description 50
- 238000007781 pre-processing Methods 0.000 claims description 8
- 238000005070 sampling Methods 0.000 description 17
- 230000006870 function Effects 0.000 description 13
- 238000012545 processing Methods 0.000 description 13
- 239000000872 buffer Substances 0.000 description 9
- 230000006835 compression Effects 0.000 description 9
- 238000007906 compression Methods 0.000 description 9
- 238000001228 spectrum Methods 0.000 description 8
- 230000002238 attenuated effect Effects 0.000 description 7
- 230000008447 perception Effects 0.000 description 6
- RVRCFVVLDHTFFA-UHFFFAOYSA-N heptasodium;tungsten;nonatriacontahydrate Chemical group O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.[Na+].[Na+].[Na+].[Na+].[Na+].[Na+].[Na+].[W].[W].[W].[W].[W].[W].[W].[W].[W].[W].[W] RVRCFVVLDHTFFA-UHFFFAOYSA-N 0.000 description 5
- 230000001052 transient effect Effects 0.000 description 5
- 238000012937 correction Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000002592 echocardiography Methods 0.000 description 4
- 230000010363 phase shift Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000001914 filtration Methods 0.000 description 3
- 238000009877 rendering Methods 0.000 description 3
- 238000001308 synthesis method Methods 0.000 description 3
- 230000007704 transition Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 239000011295 pitch Substances 0.000 description 2
- 238000012805 post-processing Methods 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- YBJHBAHKTGYVGT-ZKWXMUAHSA-N (+)-Biotin Chemical compound N1C(=O)N[C@@H]2[C@H](CCCCC(=O)O)SC[C@@H]21 YBJHBAHKTGYVGT-ZKWXMUAHSA-N 0.000 description 1
- 238000012935 Averaging Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000035807 sensation Effects 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- FEPMHVLSLDOMQC-UHFFFAOYSA-N virginiamycin-S1 Natural products CC1OC(=O)C(C=2C=CC=CC=2)NC(=O)C2CC(=O)CCN2C(=O)C(CC=2C=CC=CC=2)N(C)C(=O)C2CCCN2C(=O)C(CC)NC(=O)C1NC(=O)C1=NC=CC=C1O FEPMHVLSLDOMQC-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
- H04R3/04—Circuits for transducers, loudspeakers or microphones for correcting frequency response
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/038—Speech enhancement, e.g. noise reduction or echo cancellation using band spreading techniques
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2430/00—Signal processing covered by H04R, not provided for in its groups
- H04R2430/03—Synergistic effects of band splitting and sub-band processing
Definitions
- the invention relates to methods and systems for virtual bass synthesis.
- Typical embodiments employ harmonic transposition to generate an enhancement signal which is combined with an audio signal to generate an enhanced audio signal, such that the enhanced audio signal provides an increased perceived level of bass content during playback by one or more loudspeakers that cannot physically reproduce bass frequencies of the audio signal or the enhanced audio signal.
- Bass synthesis is the collective name for a class of techniques that add in components to the low frequency range of an audio signal in order to enhance the bass that is perceived during playback of the enhanced signal.
- Some such techniques (sometimes referred to as sub bass synthesis methods) create low frequency components below the signal's existing frequency components in order to extend and improve the lowest frequency range.
- Other techniques in the class known as "virtual pitch" algorithms, generate audible harmonics from an inaudible bass range (e.g., a bass range that is inaudible when the signal is rendered by small loudspeakers), so that the generated harmonics improve the perceived bass response.
- Some virtual pitch methods are designed to increase the perceived level of bass content of an audio signal during playback of the signal by one or more loudspeakers (e.g., small loudspeakers) that cannot physically reproduce bass frequencies of the audio signal.
- Such methods typically include steps of analyzing the bass frequencies present in input audio and enhancing the input audio by generating (and including in the enhanced audio) audible harmonics that aid the perception of lower frequencies that are missing during playback of the enhanced audio (e.g., playback by small loudspeakers that cannot physically reproduce the missing lower frequencies).
- Such methods perform harmonic transposition of frequency components of the input audio that are expected to be inaudible during playback of the input audio (i.e., having frequencies too low to be audible during playback on the expected speaker(s)), to generate audible higher frequency components (i.e., having frequencies that are sufficiently high to be audible during playback on the expected speaker(s)).
- FIG. 1 shows the frequency-amplitude spectrum of an audio signal, having an inaudible range 100 of frequency components, and an audible range of frequency components above the inaudible range.
- Harmonic transposition of frequency components in the inaudible range 100 can generate transposed frequency components in portion 101 of the audible range, which can enhance the perceived level of bass content of the audio signal during playback.
- Such harmonic transposition may include application of multiple transposition factors to each relevant frequency component of the input audio, to generate multiple harmonics of the component.
- United States Patent Application Publication No. 2007/0253576 A1 discloses a method for virtual bass synthesis.
- a low frequency signal is attained by applying a low pass filter to an original signal.
- process of down sampling the low frequency signal, moving the low frequency signal to a series of harmonics whose frequencies are integral times as large as the frequency of low frequency signals, and then up sampling them are provided.
- Typical embodiments of the inventive method are designed to increase the perceived level of bass content of an audio signal during playback of the signal by one or more loudspeakers (e.g., small loudspeakers) that cannot physically reproduce bass frequencies of the audio signal.
- Typical embodiments include steps of: applying harmonic transposition to bass frequencies present in the input audio signal (but expected to be inaudible during playback of the input audio signal using an expected speaker or speaker set) to generate harmonics that are expected to be audible during playback of the enhanced audio signal using the expected speaker(s), and generating enhanced audio (an enhanced version of the input audio) by including the harmonics in the enhanced audio.
- the method typically includes steps of performing a time-to-frequency domain transform (e.g., an FFT) on the input audio to generate frequency components indicative of bass content of the input audio, and enhancing the input audio by generating (and including in an enhanced version of the input audio) audible harmonics of these frequency components that aid the perception of lower frequencies that are expected to be missing during playback of the enhanced audio (e.g., by small loudspeakers that cannot physically reproduce the missing lower frequencies).
- a time-to-frequency domain transform e.g., an FFT
- the invention is a virtual bass generation method, including steps of: (a) performing harmonic transposition on low frequency components of an input audio signal (typically, bass frequency components expected to be inaudible during playback of the input audio signal using an expected speaker or speaker set) to generate transposed data indicative of harmonics (which are expected to be audible during playback, using the expected speaker(s), of an enhanced version of the input audio which includes the harmonics); (b) generating an enhancement signal in response to the transposed data (e.g., such that the enhancement signal is indicative of the harmonics or amplitude modified (e.g., scaled) versions of the harmonics); and (c) generating an enhanced audio signal by combining (e.g., mixing) the enhancement signal with the input audio signal.
- an input audio signal typically, bass frequency components expected to be inaudible during playback of the input audio signal using an expected speaker or speaker set
- transposed data indicative of harmonics (which are expected to be audible during playback, using the expected speaker(s), of an enhanced version of the input
- the enhanced audio signal provides an increased perceived level of bass content during playback of the enhanced audio signal by one or more loudspeakers that cannot physically reproduce the low frequency components.
- combining the enhancement signal with the input audio signal aids the perception of low frequencies that are missing during playback of the enhanced audio signal (e.g., playback by small loudspeakers that cannot physically reproduce the missing low frequencies).
- the harmonic transposition performed in step (a) employs combined transposition to generate harmonics, by means of a second order (“base") transposer and at least one higher order transposer (typically, a third order transposer and a fourth order transposer, and optionally also at least one transposer of order higher than four), of each of the low frequency components, such that all of the harmonics (and typically also the transposed data) are generated in response to frequency-domain values determined by a single, common time-to-frequency domain transform stage (e.g., by performing phase multiplication on frequency coefficients resulting from a single time-to-frequency domain transform), and a single, common frequency-to-time domain transform is subsequently performed.
- the harmonic transposition is performed using integer transposition factors, which eliminates the need for unstable (or inexact) phase estimation, phase unwrapping and/or phase locking techniques (e.g., as implemented in conventional phase vocoders).
- step (a) is performed on low frequency components of the input audio signal which have been generated by performing a frequency domain oversampled transform on the input audio signal, by generating windowed, zero-padded samples, and performing a time-to-frequency domain transform on the windowed, zero-padded samples.
- the frequency domain oversampling typically improves the quality of the virtual bass generation in response to impulse-like (transient) signals.
- the method includes a preprocessing step on the input audio signal to generate critically sampled audio indicative of the low frequency components, and step (a) is performed on the critically sampled audio.
- the input audio signal is a sub-banded, complex-valued QMF domain (CQMF) signal
- the critically sampled audio is indicative of content of a set of low frequency sub-bands of the CQMF signal.
- CQMF complex-valued QMF domain
- the input audio signal is indicative of low frequency audio content (in a range from 0 to B Hz, where B is a number less than 500)
- the critically sampled audio is an at least substantially critically sampled (critically sampled or close to critically sampled) signal indicative of the low frequency audio content, and has sampling frequency Fs/Q, where Fs is the sampling frequency of the input audio signal, and Q is a downsampling factor.
- Q is the largest factor which makes Fs/Q at least substantially equal to (but not less than) two times the bandwidth B of the input signal (i.e., Q ⁇ Fs/2B).
- step (a) is performed in a subsampled (downsampled) domain, which is the first (lowest frequency) band (channel 0) of a CQMF bank for the transposer analysis stage (input), and the first two (lowest frequency) bands (channels 0 and 1) of a CQMF bank for the transposer synthesis stage (output).
- the separation of CQMF channels 0 and 1 is accomplished by a splitting of processed frequency coefficients (i.e., frequency coefficients formerly processed by non-linear processing stages 9-11 and energy adjusting stages 13-15 of Fig.
- the first set of frequency components and the second set of frequency components are magnitude compensated to account for the CQMF channel 0 and CQMF channel 1 frequency responses.
- the magnitude compensations are applied to the frequency components indicative of the overlapping regions between CQMF channel 0 and CQMF channel 1 (e.g., for the frequency components of CQMF channel 0 indicative of the middle of the pass band and upwards in frequency, and for the frequency components of CQMF channel 1 indicative of the middle of the pass band and downwards in frequency).
- the transposed data are energy adjusted (e.g., attenuated).
- the transposed data may be attenuated in a manner determined by the well-known Equal Loudness Contours (ELCs) or an approximation thereof.
- ELCs Equal Loudness Contours
- the transposed data indicative of each generated harmonic overtone spectrum may have an additional attenuation (e.g., a slope gain in dB per octave) applied thereto.
- the attenuation may depend on a tonality metric (e.g., for the frequency range of the low frequency components of the input audio signal), e.g., so that a strong tonality results in a larger attenuation (in dB per octave) within the spectrum of each generated harmonic overtone.
- a tonality metric e.g., for the frequency range of the low frequency components of the input audio signal
- data indicative of the harmonics are energy adjusted (e.g., attenuated) in accordance with a control function which determines a gain to be applied to each hybrid sub-band of the transposed data (where a hybrid sub-band may constitute a frequency band division of the audio data, indicative of a frequency resolution somewhere in-between the resolution provided by the time-to-frequency domain transform of the "base" transposer and the bandwidth of the sub-banded input signal respectively).
- g b H G ⁇ nrg orig b - nrg vb b / G ⁇ nrg orig b + nrg vb b + B , where H, G and B are constants, and nrg orig ( b
- Another aspect of the invention is a system (e.g., a device having physically-limited or otherwise limited bass reproduction capabilities, such as, for example, a notebook, tablet, mobile phone, or other device with small speakers) configured to perform any embodiment of the inventive method on an input audio signal.
- a system e.g., a device having physically-limited or otherwise limited bass reproduction capabilities, such as, for example, a notebook, tablet, mobile phone, or other device with small speakers
- the invention is an audio playback system which has limited (e.g., physically-limited) bass reproduction capabilities (e.g., a notebook, tablet, mobile phone, or other device with small speakers), and is configured to perform virtual bass generation on audio (in accordance with an embodiment of the inventive method) to generate enhanced audio, and to playback the enhanced audio.
- the virtual bass generation is performed such that playback of the enhanced audio by the system provides the perception of enhanced bass response (relative to the bass response perceived during playback of the non-enhanced input audio by the device), including by synthesizing audible harmonics of frequencies (of the input audio) which are below the system's low-frequency roll-off (e.g., below approximately 100-300 Hz).
- the bass perceived during playback of the enhanced audio using headphones or full-range loudspeakers is also increased.
- the invention is a method for performing harmonic transposition of inaudible signal components of input audio (components having frequencies too low to be audible during playback by an expected speaker or set of speakers), to generate enhanced audio including audible harmonics of the inaudible components (i.e., harmonics having frequencies that are audible during playback on the expected speaker or set of speakers), including by application of plural transposition factors (to produce the audible harmonics) followed by energy adjustment.
- Other aspects of the invention are systems and devices configured to perform such harmonic transposition.
- the upper (audible) harmonics thereof that are included in an enhanced audio signal typically must constitute an at least substantially complete (but truncated) harmonic series.
- typical embodiments of the invention transpose all frequency components in a predetermined source range and these components might themselves be harmonics of unknown order.
- a missing fundamental itself may not be perceived when the enhanced audio is rendered.
- the sensation of bass will be typically recognized because a source (e.g., a musical instrument) generating a bass signal will be perceived as being present in the enhanced audio although at a higher pitch (e.g., at the first harmonic of the fundamental).
- the inventive system comprises a preprocessing stage (e.g., a summation stage) coupled to receive input audio indicative of low frequency audio content (in a range from 0 to B Hz, so that B is the bandwidth of the low frequency audio content) and configured to generate critically sampled audio indicative of the low frequency audio content; a bass enhancement stage (including a harmonic transposer) coupled and configured to generate a bass enhancement signal in response to the critically sampled audio; and a bass enhanced audio generation stage coupled and configured to generate to a bass enhanced audio signal by combining (e.g., mixing) the bass enhancement signal and the input audio.
- a preprocessing stage e.g., a summation stage
- a bass enhancement stage including a harmonic transposer
- a bass enhanced audio generation stage coupled and configured to generate to a bass enhanced audio signal by combining (e.g., mixing) the bass enhancement signal and the input audio.
- the preprocessing stage is preferably configured to provide an at least substantially critically sampled (critically sampled or close to critically sampled) signal to the bass enhancement stage.
- the at least substantially critically sampled signal is indicative of the low frequency audio content (in the range from 0 to B Hz), and has sampling frequency Fs/Q, where Fs is the sampling frequency of the input audio, and Q is a downsampling factor.
- Q is the largest factor which makes Fs/Q at least substantially equal to (but not less than) two times the bandwidth B of the input signal (i.e., Q ⁇ Fs/2B).
- Transposed frequency components produced in the bass enhancement stage
- the downsampling factor Q preferably forces the output signal of the summation stage to be critically sampled or close to critically sampled.
- the inventive system is or includes a general or special purpose processor programmed with software (or firmware) and/or otherwise configured to perform an embodiment of the inventive method.
- the inventive system is a general purpose processor, coupled to receive input audio data, and programmed (with appropriate software) to generate output audio data by performing an embodiment of the inventive method.
- the inventive system is a digital signal processor, coupled to receive input audio data, and configured (e.g., programmed) to generate output audio data in response to the input audio data by performing an embodiment of the inventive method.
- aspects of the invention include a system configured (e.g., programmed) to perform any embodiment of the inventive method, and a computer readable medium (e.g., a disc) which stores code for implementing any embodiment of the inventive method.
- a computer readable medium e.g., a disc
- performing an operation "on" a signal or data e.g., filtering, scaling, transforming, or applying gain to, the signal or data
- a signal or data e.g., filtering, scaling, transforming, or applying gain to, the signal or data
- performing the operation directly on the signal or data or on a processed version of the signal or data (e.g., on a version of the signal that has undergone preliminary filtering or pre-processing prior to performance of the operation thereon).
- system is used in a broad sense to denote a device, system, or subsystem.
- a subsystem that implements a decoder may be referred to as a decoder system, and a system including such a subsystem (e.g., a system that generates X output signals in response to multiple inputs, in which the subsystem generates M of the inputs and the other X - M inputs are received from an external source) may also be referred to as a decoder system.
- processor is used in a broad sense to denote a system or device programmable or otherwise configurable (e.g., with software or firmware) to perform operations on data (e.g., audio, or video or other image data).
- data e.g., audio, or video or other image data.
- processors include a field-programmable gate array (or other configurable integrated circuit or chip set), a digital signal processor programmed and/or otherwise configured to perform pipelined processing on audio or other sound data, a programmable general purpose processor or computer, and a programmable microprocessor chip or chip set.
- Coupled is used to mean either a direct or indirect connection.
- that connection may be through a direct connection, or through an indirect connection via other devices and connections.
- the inventive virtual bass synthesis method implements the following basic features:
- the input data to be processed in accordance with the invention are sub-banded CQMF (complex-valued quadrature mirror filter) domain audio data.
- the CQMF data for the low frequency sub-band channels can undergo further frequency band splittings (e.g., in order to increase the frequency resolution for the low frequency range) by means of Nyquist filter banks of different sizes.
- Nyquist filter banks do not employ downsampling of the sub-band samples.
- the Nyquist filter banks have a particularly straightforward synthesis step, i.e. pure addition of the sub-band samples.
- hybrid sub-band samples In such systems, the combination of low frequency sub-band samples from the Nyquist analysis stages and the remaining CQMF channels (i.e., the CQMF channels that were not subjected to Nyquist filtering) are herein referred to as "hybrid" sub-band samples.
- a number of the lowest hybrid sub-bands can be combined (e.g., added together).
- the lowest frequency hybrid sub-bands of the data e.g., sub-bands 0-7, as shown in Fig. 2 , where the sub-bands together span the range from 0-375 Hz
- the sub-bands together span the range from 0-375 Hz
- the latter signal is a low-pass filtered, complex-valued, time-domain audio signal (preferably, a critically sampled signal) whose pass band is 0 Hz to 375 Hz.
- the CQMF channel 0 signal undergoes optional compression (e.g., in stage 45 of the Fig. 2 system), windowing and zero-padding (e.g., in stage 3 of the Fig. 2 system), and then time-to-frequency domain transformation (e.g., in transform stage 5 of the Fig. 2 system).
- optional compression e.g., in stage 45 of the Fig. 2 system
- windowing and zero-padding e.g., in stage 3 of the Fig. 2 system
- time-to-frequency domain transformation e.g., in transform stage 5 of the Fig. 2 system.
- the transform stage typically implements an FFT (Fast Fourier Transform)
- the transform stage implements a time-to-frequency domain transform of another type (e.g., in variations on the Fig.
- transform stage 5 implements a Fourier Transform, a Discrete Fourier Transform, or a Wavelet Transform, or another time-to-frequency domain transform or analysis filter bank which is not an FFT, and each of inverse transform stages 29 and 31 implements a corresponding inverse transform (a frequency-to-time domain transform) or synthesis filter bank.
- a typical conventional harmonic transposer operates on a time domain signal having full sampling rate (44.1 kHz or 48 kHz), and employs an FFT (e.g., of size equal to roughly 1024 to 4096 lines) to generate (in the frequency domain) output audio indicative of frequency transposed samples of the input signal.
- FFT e.g., of size equal to roughly 1024 to 4096 lines
- Such a typical transposer also employs an inverse FFT to generate time domain output audio in response to the frequency domain output.
- the samples of the single, critically sampled (or nearly critically sampled) channel can be efficiently transformed into the frequency domain by an FFT transform of much smaller size (e.g., an FFT with block size of 32-256 samples) than the FFT transform (e.g., of block size equal to 1024 to 4096) that would be needed if the raw, unfiltered time-domain input data were transformed directly into the frequency domain.
- an FFT transform of much smaller size e.g., an FFT with block size of 32-256 samples
- the FFT transform e.g., of block size equal to 1024 to 4096
- Performing frequency transposition directly on the sub-bands of the hybrid data (the input to stage 1 of Fig. 2 ), and combining the resulting transposed data, is a suboptimal option. This is because, each of the low frequency hybrid sub-bands (shown as the input to stage 1 of Fig. 2 ) is oversampled data, and if stage 1 of Fig. 2 were omitted, each of the low frequency hybrid sub-bands would be transformed into the frequency domain, so that the processing power required for each of the hybrid sub-bands would be as high as the processing power required for the single CQMF band (channel 0) in the Fig. 2 system.
- the inventive system When performing frequency transposition on a single CQMF band (e.g., channel 0), the inventive system preferably changes the phase response that would be needed if the transposition were performed directly on the CQMF sub-bands (frequency transposition in the CQMF domain is indeed possible.
- the frequency resolution provided by the sub-band samples of the CQMF bank is inadequate for virtual bass processing in accordance with the invention).
- this phase response compensation is applied by element 2 of the Fig. 2 system.
- the phase relations between the neighboring channels in a CQMF bank will not be correct when performing an FFT split (in element 19 of the Fig. 2 system). Therefore, a phase compensation factor needs to be applied (in element 37 of the Fig. 2 system).
- the general CQMF analysis modulation may have the expression
- T e.g., in stage 9 of the Fig.
- CQMF channel 1 of the output (the signal output from stage 35 of Fig. 2 ) needs a multiplication by e -i ⁇ /2 to preserve the phase relationship and emulate that it has passed a CQMF analysis stage. This multiplication is performed in element 37 of Fig. 2 .
- the 8-channel Nyquist filter bank has pass-bands with center frequencies 47 Hz, 141 Hz, 234 Hz, 328 Hz, 422 Hz, 516 Hz , -141 Hz, and -47 Hz.
- the Nyquist filter bank uses complex-valued arithmetic and operates on complex-valued CQMF samples (channel 0) as input.
- the first 4 pass-bands (0-3) constitute the pass-band of CQMF channel 0, while the last 4 pass-bands filters the CQMF transition regions: channel 4 and 5 filters the overlap/transition region of CQMF channel 0 towards CQMF channel 1, and channel 6 and 7 filters the transition region to negative frequencies of CQMF channel 0.
- the output from the Nyquist filter bank is simply band-passed versions of the input CQMF signal.
- stage 1 adds the eight streams of Nyquist samples back together (Nyquist synthesis), the result is an exact reconstruction of the CQMF channel 0, which is critically sampled in terms of sampling frequency (actually the CQMF bank may be oversampled by a factor of 2 due to the complex-valued sub-band samples, while the real part only of its output may be critically sampled (maximally decimated)).
- the Nyquist synthesis step (implemented in a typical implementation of stage 1 of the Fig. 2 system) is particularly straightforward since it is just a simple summation of the samples from the 8 lowest hybrid channels of the sub-banded input data for each CQMF time slot.
- the summation generates a conventional CQMF channel 0 signal, which is input to element 2 of the Fig. 2 system (or to compressor 45, in implementations in which the optional compressor 45 is included in the Fig. 2 system).
- the output signals from the inventive transposer are two CQMF signals (the outputs of elements 33 and 35 of Fig. 2 ), containing the bass enhancement signal (sometime referred to as a virtual bass signal) to be mixed (in stage 43) with an appropriately delayed version of the original input signal.
- Both output signals are filtered through 8- and 4-channel Nyquist analysis stages (stages 39 and 41 of Fig. 2 ) respectively to convert them back to the original hybrid sub-banded domain.
- Stage 39 implements 8-channel analysis to output, in parallel, 8 sub-band channels in response to the CQMF signal (CQMF channel 0) asserted to its input.
- Stage 41 implements 4-channel analysis to output, in parallel, four sub-band channels in response to the CQMF signal (CQMF channel 1) asserted to its input.
- the CQMF channel 0 signal (produced in stage 1 of Fig. 2 ) optionally undergoes dynamic range compression (e.g., in compressor 45 of Fig. 2 ).
- dynamic range compression e.g., in compressor 45 of Fig. 2
- compression is used in a broad sense to denote either broadening of the dynamic range (sometimes referred to dynamic range expansion) or narrowing of the dynamic range, so that compressor 45 may be what is sometimes referred to as a compander (compressor/expander).
- a low pass filtered, down-mixed (mono) version of the CQMF channel 0 signal can be used as the control signal for the compressor.
- stage 1 of the Fig. 2 system can sum the lowest four sub-bands of the hybrid, sub-banded input data, and assert the control signal to compressor 45.
- compressor 45 (or element 1B of the Fig. 4 system, to be described below) performs an averaged energy calculation, and computes the compression gain required to perform the appropriate dynamic range compression.
- stage 3 performs the following operations on the complex-valued CQMF channel 0 samples asserted thereto (to implement frequency domain oversampling by a factor of 2):
- stage 5 performs a 128-point complex FFT on each windowed, zero-padded block.
- stage 19 splits (in a manner to be described in more detail below) each block of the processed coefficients into two half sized blocks (each comprising 64 coefficients): a first block indicative of content in the frequency range 0-375 Hz; and a second block indicative of content in the frequency range 375-750 Hz.
- stage 29 performs a 64-point IFFT on each first block
- stage 31 performs a 64-point IFFT on each second block.
- Windowing and overlap/adding stage 33 discards the first and last 16 samples from each transformed block output from stage 29, windows the remaining 32 samples with a 32-point synthesis window, and overlap-adds the resulting samples, to generate a conventional CQMF channel 0 signal indicative of the transposed content in the range 0 to 375 Hz.
- Element 37 performs the above-described phase shift on this signal to generate a conventional CQMF channel 1 signal indicative of the transposed content in the range 375 to 750 Hz.
- the block size of the input to stage 3 is quite small (32-256 samples per block).
- the block size of the forward transform implemented by stage 5 is typically larger, and the specific forward transform block size depends on the frequency domain oversampling (typically a factor of 2, but sometimes a factor of 4).
- the inventive system uses asymmetric analysis and synthesis windows for the forward (e.g., FFT) and inverse (e.g., IFFT) transforms in contrast to the symmetric windows used in typical implementations.
- the size (number of points) of the analysis window (e.g., the window applied in stage 3) and the forward transform (e.g., the transform applied by stage 5) may be different from that of the synthesis window (e.g., the window applied in stage 33 or 35) and the inverse transform (e.g., the inverse transform applied in stage 29 or 31).
- the shape and size of each window and size of each transform maybe chosen so as to achieve adequate frequency resolution while lowering the inherent algorithmic delay of the transposer.
- computational complexity is reduced by processing only the signal of interest (e.g., the CQMF channel 0 data, generated in stage 1 of the Fig. 2 system in response to hybrid, sub-banded input data, are critically sampled).
- the signal of interest e.g., the CQMF channel 0 data, generated in stage 1 of the Fig. 2 system in response to hybrid, sub-banded input data, are critically sampled.
- the inventive system comprises a preprocessing stage (e.g., summation stage 1 of the Fig. 2 system), coupled to receive input audio indicative of low frequency audio content (in a range from 0 to B Hz, so that B is the bandwidth of the low frequency audio content) and configured to generate critically sampled audio indicative of the low frequency audio content (e.g., the CQMF channel 0 signal output from stage 1 of Fig. 2 ); a bass enhancement stage (including a harmonic transposer) coupled and configured to generate a bass enhancement signal (e.g., the output of stages 39 and 41 of the Fig. 2 system) in response to the critically sampled audio; and a bass enhanced audio generation stage (e.g., stage 43 of the Fig.
- a preprocessing stage e.g., summation stage 1 of the Fig. 2 system
- receive input audio indicative of low frequency audio content in a range from 0 to B Hz, so that B is the bandwidth of the low frequency audio content
- the bass enhanced audio signal is a full frequency range signal generated by mixing the bass enhancement signal output from stages 39 and 41 of Fig. 2 ), and the input audio (sub-bands 0-7 of the hybrid sub-band signal) asserted to the summation stage, and also the other sub-bands (e.g., sub-bands 8-76) of the hybrid signal.
- the preprocessing stage e.g., summation stage 1 of Fig.
- the at least substantially critically sampled signal is indicative of the low frequency audio content (in the range from 0 to B Hz), and has sampling frequency Fs/Q, where Fs is the sampling frequency of the input audio, and Q is a downsampling factor.
- Q is the largest factor which makes Fs/Q at least substantially equal to (but not less than) two times the bandwidth B of the input signal (i.e., Q ⁇ Fs/2B).
- Transposed frequency components produced in the bass enhancement stage
- the downsampling factor Q preferably forces the output signal of the summation stage to be critically sampled or close to critically sampled.
- the 2nd order "base" transposer (stage 9 of Fig. 2 ) of the inventive system extends the bandwidth of the input signal by a factor of two, thus generating harmonic components of 2 nd order, and transposers of other orders (e.g., stage 11 of Fig. 2 ) generate harmonics of greater factors.
- the frequency-transposed output of the inventive virtual bass system typically does not need to include frequency components above about 500 Hz (otherwise, the audio signal frequency range to be transposed would extend above what is considered the bass range).
- the first CQMF channel (channel 0), whose bandwidth is from 0 to 375 Hz (at 48 kHz), has bandwidth which is typically more than adequate for the virtual bass synthesis system input.
- the first two CQMF channels (channel 0 and 1) have combined bandwidth (0 to 750 Hz at 48 kHz) that is typically sufficient for the virtual bass synthesis system output.
- each complex coefficient output from transform stage 5 corresponds to a frequency identified by index k .
- Element 7 of Fig. 2 multiplies each complex coefficient by e i ⁇ k .
- Stage 5 and element 7, considered together, are a subsystem (which may be referred to as a transform stage) which implements a single time-to-frequency domain transform.
- Element 7 is used to center the analysis window at time 0 in the FFT, an important step in a transposer (or phase vocoder).
- the Fig. 2 system also includes transposers of other orders (e.g., fifth and optionally also higher orders), not shown in Fig. 2 .
- Each of such optional transposers operates in parallel with stages 9 and 11, and multiplies the phase of each complex coefficient asserted thereto by a transposition factor T, where T is an integer greater than 4, either directly or by interpolation of coefficients, so as to produce a harmonic (or corresponding order) of such coefficient.
- phase multiplier stages 9 and 11 implement nonlinear processing which determines contributions to different frequency bands (e.g., different frequency bands of the enhanced low frequency audio output from stages 39 and 41) in response to one frequency band of the input low frequency audio to be enhanced (i.e., in response to a complex coefficient generated by transform stage 5 having a single frequency index k, or in response to complex coefficients generated by transform stage 5 having frequency indices, k, in a range).
- different frequency bands e.g., different frequency bands of the enhanced low frequency audio output from stages 39 and 41
- one frequency band of the input low frequency audio to be enhanced i.e., in response to a complex coefficient generated by transform stage 5 having a single frequency index k, or in response to complex coefficients generated by transform stage 5 having frequency indices, k, in a range.
- the interpolation scheme for transposition orders higher than 2 enables the use of a single, common time-to-frequency transform or analysis filter bank (including transform stage 5) and a single common frequency-to-time transform or synthesis filter bank (including inverse transform stages 29 and 31) for all orders of transposition, thereby significantly reducing the computational complexity when using multiple harmonic transposers.
- the overall gains for the coefficients to which different transposition factors have been applied are set independently (in stages 13-15).
- Gain stage 13 sets the gain of the coefficients output from stage 9
- gain stage 15 sets the gain of the coefficients output from stage 11
- an additional gain stage (not shown in Fig. 2 ) for each other phase multiplier stage sets the gain of the coefficients output from the corresponding phase multiplier stage.
- One such additional gain stage is gain stage 14 of Fig. 4 , which sets the gain of the coefficients output from stage 10 of Fig. 4 .
- the coefficients output from the gain stages 13-15 are summed in element 17, generating a single stream of frequency-transposed (and level adjusted) coefficients which is indicative of the enhanced audio (virtual bass) determined in accordance with the invention. This single stream of frequency-transposed coefficients is asserted to the input of element 19.
- the gains can be set to approximate the well-known Equal Loudness Contours (ELCs), since the ELCs can be adequately modeled by a straight line on a logarithmic scale for frequencies below 400 Hz.
- ELCs Equal Loudness Contours
- the odd order harmonics (the 3 rd order harmonic, 5 th order harmonic, etc.) can sometimes be perceived as being more harsh than the even order harmonics (the 2 nd order harmonic, 4 th order harmonic, etc.), although their presence is typically important (or vital) for the virtual bass effect.
- the odd order harmonics may be attenuated (in stages 13-15) by more than the amount determined by the ELCs.
- each gain stage may apply (to one of the streams of transposed coefficients) a slope gain, i.e.
- a roll-off attenuation factor (e.g., measured in Decibels per octave). This attenuation is applied on a per bin basis (i.e., an attenuation value is applied independently for each frequency index, k).
- a control signal indicative of a tonality metric (indicated in Fig. 2 , although this signal is not applied in some implementations) for CQMF channel 0 is asserted to the gain stages, and the gain stages apply gain on a per bin basis in response to the control signal.
- the slope gain may be applied (e.g., increased by 6 dB or some other amount per octave) so that the roll-off is steeper. This can improve the listening experience for audio (e.g., music) with bass (e.g., bass guitar) sounds consisting of strong harmonic series, which otherwise would result in an over-exaggerated virtual bass effect.
- a control signal indicative of a tonality measure is asserted to the gain stages (e.g., stages 13-15), and the gain stages apply gain on a per bin basis in response to the control signal.
- the tonality measure has been obtained by the conventional method used for CQMF subband samples in conventional HE-AAC audio encoding, where LPC coefficients are used to calculate the relation between the predictable part of the signal and the prediction error (the un-predictable part).
- a control (correction) function is typically used.
- this level compensation operation is performed in the hybrid sub-band
- index i is the block index, i.e. the index of the blocks that are made up of subsequent hybrid sub-band samples over which the averaging is performed.
- a block consists of 4 hybrid sub-band samples.
- nrg vb ( c,i,b ) is a function of energy, E vb ( c, n, b ) , of the transposed signal contained in the hybrid sub-band sample in channel c, sub-band time slot n, and hybrid sub-band b , and is calculated in the way in which nrg org ( c,i,b ) is determined in equation (6), with E vb ( c, n, b ) replacing E org ( c, n, b).
- the correction function of Eq. 5 is illustrated in Fig.
- V(c, i, b) is plotted on the axis labeled "Level compensation factor”
- energy E vb ( c, n, b ) is plotted on the axis labeled "VB energy”
- energy E org ( c, n, b ) is plotted on the axis labeled "Original energy.”
- the frequency-transposed data asserted from the output of element 17 of Fig. 2 is preferably transformed into a CQMF channel 0 signal and a CQMF channel 1 signal.
- Stage 19 is configured to split each block of frequency-transposed coefficients (typically comprising 128 coefficients) that is output from element 17 into two half sized blocks: a first half sized block (typically comprising 64 coefficients) indicative of content in the frequency range 0-375 Hz; and a second half sized block (typically comprising 64 coefficients) indicative of content in the frequency range 375-750 Hz.
- Stages 21 and 23 perform CQMF prototype filter frequency response compensation in the frequency domain.
- the CQMF response compensation performed in stage 21 changes the gains of the 0-375 Hz components output from stage 19 to match the normal profile produced in conventional processing of CQMF data
- the CQMF response compensation performed in stage 23 changes the gains of the 375-750 Hz components output from stage 19 to match the normal profile produced in conventional processing of CQMF data.
- the CQMF compensations are applied to the frequency components indicative of the overlapping regions between CQMF channel 0 and CQMF channel 1 (e.g., for the frequency components of CQMF channel 0 indicative of the middle of the pass band and upwards in frequency, and for the frequency components of CQMF channel 1 indicative of the middle of the pass band and downwards in frequency).
- the levels of compensation are set to distribute the energy of the overlapping parts of the spectrum in a manner that a conventional CQMF analysis filter bank would do between CQMF channel 0 and CQMF channel 1 in the absence of the FFT splitting stage 19 of Fig. 2 .
- Element 25 multiplies each complex coefficient output from stage 21 (and having frequency index k ) by e -i ⁇ k , to cancel the shift applied by element 7.
- Element 27 multiplies each complex coefficient output from stage 23 (and having frequency index k) by e -i ⁇ k , to cancel the shift applied by element 7.
- Stage 29 performs a frequency-to-time domain transform (e.g., an IFFT, where stage 5 had performed an FFT) on each block of the coefficients output from element 25.
- Stage 31 performs a frequency-to-time domain transform (e.g., an IFFT, where stage 5 had performed an FFT) on each block of the coefficients output from element 27.
- Windowing and overlap/adding stage 33 discards the first and last m samples (where m is typically equal to 16) from each transformed block output from inverse transform stage 29, windows the remaining samples, and overlap-adds the resulting samples, to generate a conventional CQMF channel 0 signal indicative of the transposed content in the range 0 to 375 Hz.
- windowing and overlap/adding stage 35 discards the first and last m samples (where m is typically equal to 16) from each transformed block output from inverse transform stage 31, windows the remaining samples, and overlap-adds the resulting samples, to generate a signal indicative of the transposed content in the range 375 to 750 Hz.
- Element 37 performs the above-described phase shift on this signal to generate a conventional CQMF channel 1 signal indicative of the transposed content in the range 375 to 750 Hz.
- the output signals of elements 33 and 37 are filtered in Nyquist 8- and 4-channel analysis stages (stages 39 and 41 of Fig. 2 ) respectively to convert them back to the original hybrid sub-banded domain.
- Stage 39 implements 8-channel analysis to output, in parallel, 8 sub-band channels in response to the CQMF channel 0 signal asserted to its input.
- Stage 41 implements 4-channel analysis to output, in parallel, four sub-band channels in response to the CQMF channel 1 signal asserted to its input.
- the outputs of stages 39 and 41 together comprise a bass enhancement signal (i.e., when mixed together, they determine the bass enhancement signal) which has been generated in the bass enhancement stage of the Fig. 2 system.
- the bass enhancement stage includes a harmonic transposer configured to apply transpositions having several transposition factors to low frequency content of input audio (i.e., to sub-bands 0-7 of the hybrid sub-banded input audio, whose content is in the range from 0 Hz to 375 Hz).
- the bass enhancement signal (including content in the range from 0 Hz to 750 Hz) is combined (e.g., mixed) with the input audio in bass enhanced audio generation stage 43 to generate a bass enhanced audio signal (the output of stage 43).
- the high frequency content (sub-bands 8-76) of the hybrid sub-banded input audio is also mixed with the bass enhancement signal in stage 43.
- the output of stage 43 is full range audio (the bass enhanced audio signal) which has been bass enhanced in accordance with an embodiment of the inventive virtual bass synthesis method.
- FIG. 4 is a block diagram of an implementation of the Fig. 2 system. Elements of the Fig. 4 implementation that are identical to corresponding elements of the Fig. 2 system are identically numbered in Figs. 2 and 4 , and the description of them above will not be repeated with reference to Fig. 4 .
- Fig. 4 includes input data buffer 110, which buffers the hybrid, sub-banded input audio data, whose sub-bands 0-7 are input to stage 1.
- Fig. 4 also includes Nyquist synthesis stage 1A which is coupled to buffer 110 and configured to implement simple summation of the samples from the e.g. 4 lowest sub-bands (sub-bands 0-3) of the sub-banded input audio data in buffer 110, for each hybrid sub-band time slot.
- a stereo or a multi-channel signal would also be mixed down to a mono signal by the stage 1A.
- the output of stage 1A is indicative of a low-passed, mixed down for all input speaker channels, version of the CQMF sub-band signal of channel 0 (i.e., the output from stage 1).
- the output of stage 1A is employed by compression gain determination stage 1B to generate a control signal for compressor 45.
- stage 1B In response to the output of stage 1A, stage 1B performs an averaged energy calculation, and computes the compression gain required to perform appropriate dynamic range compression on the corresponding segments of the output of stage 2. Stage 1B asserts (to compressor 45) the control signal to cause compressor 45 to perform such dynamic range compression.
- compressor 45 is buffered in buffer 111 (coupled between elements 45 and 3 as shown in Fig. 4 ), and then asserted to stage 3 for windowing and zero-padding.
- stage 112 (coupled between elements 5 and stages 9-11 as shown in Fig. 4 , if included), the complex coefficients output from transform stage 5 are employed to calculate cross-products which can be used in some implementations of phase multiplication stages 9-11, as described in the paper by Lars Villemoes, Per Ekstrand, and Per Hedelin, entitled "Methods for Enhanced Harmonic Transposition," 2011 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, October 16-19, 2011.
- element 113 coupled between elements 5 and stages 13-15 as shown in Fig. 4 , if included, the complex coefficients output from transform stage 5 are employed to determine spectrum magnitudes, which are in turn used to generate control signals which are asserted to stages 13-15 to control the gains (applied by stages 13-15) for the coefficients to which transposition factors have been applied by phase multiplier stages 9-11.
- the Fig. 4 system also includes output buffer 116 (coupled between element 33 and stage 39 as shown in Fig. 4 ) for the CQMF channel 0 data output from element 33), and output buffer 117 (coupled between element 37 and stage 41 as shown in Fig. 4 ) for the CQMF channel 1 data output from element 37.
- the Fig. 4 system optionally includes limiter 114 (coupled between element 33 and buffer 116 as shown in Fig. 4 , if included), and limiter 115 (coupled between element 37 and buffer 117 as shown in Fig. 4 , if included).
- limiter 114 coupled between element 33 and buffer 116 as shown in Fig. 4 , if included
- limiter 115 coupled between element 37 and buffer 117 as shown in Fig. 4 , if included.
- Such limiters would function to limit the magnitudes of the transposed samples output from elements 33 and 37, e.g., to maintain averaged values of the magnitudes within predetermined limiting values.
- the invention is a virtual bass generation method, including steps of:
- the harmonic transposition performed in step (a) employs combined transposition to generate harmonics, including a second order (“base”) transposer and at least one higher order transposer (typically, a third order transposer and a fourth order transposer, and optionally also at least one transposer of order higher than four), of each of the low frequency components, such that all of the harmonics (and typically also the transposed data) are generated in response to frequency-domain values determined by a single, common time-to-frequency domain transform stage (e.g., by performing phase multiplication, either direct or by interpolation, on frequency coefficients resulting from a single time-to-frequency domain transform, for example, implemented by transform stage 5 and element 7 of the Fig.
- base second order
- at least one higher order transposer typically, a third order transposer and a fourth order transposer, and optionally also at least one transposer of order higher than four
- the harmonic transposition is performed using integer transposition factors (e.g., the factors two, three, and four applied respectively by stages 9, 10, and 11 of Fig. 4 ), which eliminates the need for unstable (or inexact) phase estimation, phase unwrapping and/or phase locking techniques (e.g., as implemented in conventional phase vocoders).
- integer transposition factors e.g., the factors two, three, and four applied respectively by stages 9, 10, and 11 of Fig. 4 .
- step (a) is performed on low frequency components of the input audio signal which have been generated by performing a frequency domain oversampled transform on the input audio signal (e.g., frequency domain oversampling as implemented by stage 3 of Fig. 2 ), by means of generating windowed, zero-padded samples, and performing a time-to-frequency domain transform on the windowed, zero-padded samples.
- the frequency domain oversampling typically improves the quality of the virtual bass generation in response to impulse-like (transient) signals.
- the method includes a step to generate critically sampled audio indicative of the low frequency components (e.g., as implemented by stage 1 of Fig. 2 ), and step (a) is performed on the critically sampled audio.
- the input audio signal is a complex-valued QMF domain (CQMF) signal
- the critically sampled audio is indicative of a set of low frequency sub-bands (e.g., sub-bands 0-7) of the hybrid signal.
- CQMF complex-valued QMF domain
- the input audio signal is indicative of low frequency audio content (in a range from 0 to B Hz, where B is a number less than 500)
- the critically sampled audio is an at least substantially critically sampled (critically sampled or close to critically sampled) signal indicative of the low frequency audio content, and has sampling frequency Fs/Q, where Fs is the sampling frequency of the input audio signal, and Q is a downsampling factor.
- Q is the largest factor which makes Fs/Q at least substantially equal to (but not less than) two times the bandwidth B of the input signal (i.e., Q ⁇ Fs/2B).
- step (a) is performed in a subsampled (downsampled) domain, which is the first (lowest frequency) band (channel 0) of a CQMF bank for the transposer analysis stage (input), and the first two (lowest frequency) bands (channels 0 and 1) of a CQMF bank for the transposer synthesis stage (output).
- the separation of CQMF channels 0 and 1 is accomplished by a splitting of the transposed data (e.g., as in element 19 of Fig.
- each frequency-to-time domain transform e.g., the transform implemented by stage 29 of Fig. 2 and the transform implemented by stage 31 of Fig.
- the first set of frequency components and the second set of frequency components are magnitude compensated to account for the CQMF channel 0 and CQMF channel 1 frequency responses.
- the transposed data are energy adjusted (e.g., attenuated), for example, as in elements 13-15 of Fig. 2 .
- the transposed data may be attenuated in a manner determined by the well-known Equal Loudness Contours (ELCs) or an approximation thereof.
- ELCs Equal Loudness Contours
- the transposed data indicative of each generated harmonic overtone spectrum may have an additional attenuation (e.g., a slope gain in dB per octave) applied thereto.
- the attenuation may depend on a tonality metric (e.g., for the frequency range of the low frequency components of the input audio signal), e.g., so that a strong tonality results in a larger attenuation (in dB per octave) within each generated harmonic overtone.
- a tonality metric e.g., for the frequency range of the low frequency components of the input audio signal
- data indicative of the harmonics are energy adjusted (e.g., attenuated) in accordance with a control function which determines a gain to be applied to each hybrid sub-band of the transposed data.
- the invention is a system or device (e.g., device having physically-limited or otherwise limited bass reproduction capabilities, such as, for example, a notebook, tablet, mobile phone, or other device with small speakers) configured to perform any embodiment of the inventive method on an input audio signal.
- Device 200 of Fig. 5 is an example of such a device.
- Device 200 includes a virtual bass synthesis subsystem 201, which is coupled to receive an input audio signal and configured to generate enhanced audio in response thereto in accordance with any embodiment of the inventive method, rendering subsystem 202, and left and right speakers (L and R), connected as shown.
- Subsystem 201 may (but need not) have the structure and functionality of the above-described Fig. 2 or Fig. 4 embodiment of the invention.
- Rendering subsystem 202 is configured to generate speaker feeds for speakers L and R in response to the enhanced audio signal generated in subsystem 201.
- the inventive system is or includes a general or special purpose processor (e.g., an implementation of subsystem 201 of Fig. 5 , or an implementation of Fig. 2 or Fig. 4 ) programmed with software (or firmware) and/or otherwise configured to perform an embodiment of the inventive method.
- the inventive system is a general purpose processor, coupled to receive input audio data, and programmed (with appropriate software) to generate output audio data in response to the input audio data by performing an embodiment of the inventive method.
- the inventive system is a digital signal processor (e.g., an implementation of subsystem 201 of Fig. 5 , or an implementation of Fig. 2 or Fig. 4 ), coupled to receive input audio data, and configured (e.g., programmed) to generate output audio data in response to the input audio data by performing an embodiment of the inventive method.
Landscapes
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Human Computer Interaction (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Health & Medical Sciences (AREA)
- Quality & Reliability (AREA)
- Computational Linguistics (AREA)
- Multimedia (AREA)
- Stereophonic System (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
- Circuit For Audible Band Transducer (AREA)
- Auxiliary Devices For Music (AREA)
Claims (15)
- Procédé de génération de basses virtuelles, comprenant une étape consistant à pré-traiter des échantillons d'un signal audio d'entrée CQMF (filtre miroir en quadrature à valeur complexe) en sous-bande afin de générer un audio échantillonné critiquement représentant un contenu d'un ensemble de sous-bandes basse fréquence du signal audio d'entrée, et comprenant les étapes consistant à :(a) effectuer une transposition d'harmoniques sur l'audio échantillonné critiquement afin de générer des données transposées représentant des harmoniques, lesquelles harmoniques doivent être audibles pendant la lecture d'une version améliorée de l'audio d'entrée comprenant ledites harmoniques ;(b) générer un signal d'amélioration en réponse aux données transposées ; et(c) générer un signal audio amélioré en combinant le signal d'amélioration et le signal audio d'entrée ;- dans lequel la transposition d'harmoniques effectuée lors de l'étape (a) utilise une transposition combinée de sorte que les harmoniques comprennent une harmonique de second ordre et au moins une harmonique d'ordre supérieur pour chacune des composantes basse fréquence, et de sorte que toutes les harmoniques soient générées en réponse à des valeurs de domaine de fréquence déterminées par un étage unique de transformation de domaine temps-fréquence commun, et qu'une transformée inverse suivante déterminée par un étage unique de transformation de domaine fréquence-temps commun soit effectuée.
- Procédé selon la revendication 1, dans lequel le signal audio d'entrée indique un contenu audio basse fréquence dans une plage de 0 à B Hz, où B est un nombre inférieur à 500, et l'audio échantillonné critiquement est un signal échantillonné au moins essentiellement de manière critique représentant le contenu audio basse fréquence.
- Procédé de génération de basses virtuelles, comprenant une étape consistant à pré-traiter des échantillons d'un signal audio d'entrée afin de générer un audio échantillonné critiquement représentant des composantes basse fréquence du signal audio d'entrée, et comprenant les étapes consistant à :(a) effectuer une transposition d'harmoniques sur l'audio échantillonné critiquement afin de générer des données transposées représentant des harmoniques, lesquelles harmoniques doivent être audibles pendant la lecture d'une version améliorée de l'audio d'entrée comprenant ledites harmoniques ;(b) générer un signal d'amélioration en réponse aux données transposées ; et(c) générer un signal audio amélioré en combinant le signal d'amélioration et le signal audio d'entrée ;- dans lequel la transposition d'harmoniques effectuée lors de l'étape (a) utilise une transposition combinée de sorte que les harmoniques comprennent une harmonique de second ordre et au moins une harmonique d'ordre supérieur pour chacune des composantes basse fréquence, et de sorte que toutes les harmoniques soient générées en réponse à des valeurs de domaine de fréquence déterminées par un étage unique de transformation de domaine temps-fréquence commun, et qu'une transformée inverse suivante déterminée par un étage unique de transformation de domaine fréquence-temps commun soit effectuée, et dans lequel l'audio échantillonné critiquement est un signal de canal 0 CQMF et le signal d'amélioration généré lors de l'étape (b) comprend un signal d'amélioration de canal 0 CQMF et un signal d'amélioration de canal 1 CQMF.
- Procédé de génération de basses virtuelles, comprenant les étapes consistant à :(a) effectuer une transposition d'harmoniques sur des composantes basse fréquence d'un signal audio d'entrée afin de générer des données transposées représentant des harmoniques, lesquelles harmoniques doivent être audibles pendant la lecture d'une version améliorée de l'audio d'entrée comprenant ledites harmoniques ;(b) générer un signal d'amélioration en réponse aux données transposées ; et(c) générer un signal audio amélioré en combinant le signal d'amélioration et le signal audio d'entrée ;- dans lequel la transposition d'harmoniques effectuée lors de l'étape (a) utilise une transposition combinée de sorte que les harmoniques comprennent une harmonique de second ordre et au moins une harmonique d'ordre supérieur pour chacune des composantes basse fréquence, et de sorte que toutes les harmoniques soient générées en réponse à des valeurs de domaine de fréquence déterminées par un étage unique de transformation de domaine temps-fréquence commun, et qu'une transformée inverse suivante déterminée par un étage unique de transformation de domaine fréquence-temps commun soit effectuée ;
et comprenant également l'étape consistant à générer les composantes basse fréquence en effectuant une transformée sur-échantillonnée de domaine de fréquence sur le signal audio d'entrée, en générant des échantillons à complétion de zéros fenêtrés, et en effectuant une transformée de domaine temps-fréquence sur les échantillons à complétion de zéros fenêtrés afin de générer ledites composantes basse fréquence, et dans lequel l'étape (b) comprend une étape consistant à diviser des composantes de fréquence traitées en un premier ensemble de composantes de fréquence dans une première bande de fréquence et en un second ensemble de composantes de fréquence dans une seconde bande de fréquence, et effectuer une première transformée de domaine fréquence-temps sur le premier ensemble de composantes de fréquence et une seconde transformée de domaine fréquence-temps sur le second ensemble de composantes de fréquence, chacune de la première transformée de domaine fréquence-temps et de la seconde transformée de domaine fréquence-temps possédant une taille de bloc inférieure à celle de la transformée de domaine temps-fréquence. - Procédé selon la revendication 4, dans lequel la première bande de fréquence est la bande de fréquence de canal 0 CQMF tandis que la seconde bande de fréquence est la bande de fréquence de canal 1 CQMF ; et éventuellement
dans lequel le premier ensemble de composantes de fréquence et le second ensemble de composantes de fréquence sont compensés en magnitude pour tenir compte des réponses de fréquence de canal 0 CQMF et de canal 1 CQMF, respectivement. - Procédé de génération de basses virtuelles, comprenant les étapes consistant à :(a) effectuer une transposition d'harmoniques sur des composantes basse fréquence d'un signal audio d'entrée afin de générer des données transposées représentant des harmoniques, lesquelles harmoniques doivent être audibles pendant la lecture d'une version améliorée de l'audio d'entrée comprenant ledites harmoniques ;(b) générer un signal d'amélioration en réponse aux données transposées ; et(c) générer un signal audio amélioré en combinant le signal d'amélioration et le signal audio d'entrée ;- dans lequel la transposition d'harmoniques effectuée lors de l'étape (a) utilise une transposition combinée de sorte que les harmoniques comprennent une harmonique de second ordre et au moins une harmonique d'ordre supérieur pour chacune des composantes basse fréquence, et de sorte que toutes les harmoniques soient générées en réponse à des valeurs de domaine de fréquence déterminées par un étage unique de transformation de domaine temps-fréquence commun, et qu'une transformée inverse suivante déterminée par un étage unique de transformation de domaine fréquence-temps commun soit effectuée ; et- dans lequel la transformée de domaine temps-fréquence et la transformée inverse utilisent une analyse asymétrique et des fenêtres de synthèse.
- Procédé selon la revendication 1, comprenant également l'étape consistant à générer les composantes basse fréquence en effectuant une transformée sur-échantillonnée de domaine de fréquence sur le signal audio d'entrée, en générant des échantillons à complétion de zéros fenêtrés, et en effectuant une transformée de domaine temps-fréquence sur les échantillons à complétion de zéros fenêtrés afin de générer ledites composantes basse fréquence.
- Procédé selon la revendication 1, dans lequel le signal audio amélioré offre un niveau perçu amélioré du contenu de basses pendant la lecture dudit signal audio amélioré par au moins un haut-parleur qui ne peut physiquement pas reproduire les composantes basse fréquence.
- Procédé selon la revendication 1, comprenant également une étape de lecture du signal audio amélioré par des haut-parleurs qui ne peuvent physiquement pas reproduire les composantes basse fréquence.
- Procédé selon la revendication 1, dans lequel les composantes basse fréquence du signal audio d'entrée sont des composantes de fréquence de basses sensées être inaudibles pendant la lecture du signal audio d'entrée à l'aide d'un haut-parleur ou d'un ensemble de haut-parleurs escompté.
- Procédé selon la revendication 1, dans lequel les données transposées représentent des versions modifiées en amplitude dedites harmoniques, comme des versions modifiées en amplitude des harmoniques dont les valeurs sont déterminées au moins approximativement par des contours de volume égaux (ELC).
- Procédé selon la revendication 1, dans lequel l'étape (a) comprend une étape consistant à atténuer les harmoniques d'une manière déterminée par une mesure de tonalité afin de déterminer les données transposées.
- Procédé selon la revendication 1, dans lequel l'une au moins de l'étape (a) et de l'étape (b) comprend une étape consistant à atténuer des données représentant les harmoniques en accord avec une fonction de commande, laquelle fonction de commande détermine un gain à appliquer à chaque sous-bande de fréquence des données transposées.
- Procédé selon la revendication 13, dans lequel la fonction de commande détermine un gain, g(b), à appliquer à des coefficients d'harmoniques dans une sous-bande de fréquence b, et se présente sous la forme suivants :- où H, G et B sont des constantes, nrgorig(b) représente l'énergie du signal audio d'entrée dans la sous-bande b, et nrgvb(b) représente l'énergie des données transposées ou du signal d'amélioration dans la sous-bande b.
- Système de génération de basses virtuelles, conçu pour mettre en oeuvre le procédé de génération de basses virtuelles selon l'une quelconque des revendications précédentes.
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/652,023 US8971551B2 (en) | 2009-09-18 | 2012-10-15 | Virtual bass synthesis using harmonic transposition |
Publications (2)
Publication Number | Publication Date |
---|---|
EP2720477A1 EP2720477A1 (fr) | 2014-04-16 |
EP2720477B1 true EP2720477B1 (fr) | 2016-03-02 |
Family
ID=49293633
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP13771123.0A Active EP2907324B1 (fr) | 2012-10-15 | 2013-09-27 | Système et procédé de réduction de latence dans des systèmes de basse virtuelle basés sur réémetteur |
EP13188415.7A Active EP2720477B1 (fr) | 2012-10-15 | 2013-10-14 | Synthèse virtuelle de graves à l'aide de transposition harmonique |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP13771123.0A Active EP2907324B1 (fr) | 2012-10-15 | 2013-09-27 | Système et procédé de réduction de latence dans des systèmes de basse virtuelle basés sur réémetteur |
Country Status (4)
Country | Link |
---|---|
EP (2) | EP2907324B1 (fr) |
JP (1) | JP5894347B2 (fr) |
CN (1) | CN104704855B (fr) |
WO (1) | WO2014060204A1 (fr) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2023280356A1 (fr) * | 2021-07-09 | 2023-01-12 | Soundfocus Aps | Procédé et système de réseau de transducteurs pour la reproduction directionnelle d'un signal audio d'entrée |
WO2023280357A1 (fr) * | 2021-07-09 | 2023-01-12 | Soundfocus Aps | Procédé et système de haut-parleur pour traiter un signal audio d'entrée |
RU2819779C1 (ru) * | 2020-03-20 | 2024-05-24 | Долби Интернешнл Аб | Усиление низких частот для громкоговорителей |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105280189B (zh) * | 2015-09-16 | 2019-01-08 | 深圳广晟信源技术有限公司 | 带宽扩展编码和解码中高频生成的方法和装置 |
KR102578008B1 (ko) * | 2019-08-08 | 2023-09-12 | 붐클라우드 360 인코포레이티드 | 심리음향적 주파수 범위 확장을 위한 비선형 적응성 필터뱅크 |
KR102511377B1 (ko) * | 2020-03-20 | 2023-03-17 | 돌비 인터네셔널 에이비 | 라우드스피커들을 위한 저음 향상 |
JP2023130644A (ja) * | 2022-03-08 | 2023-09-21 | アルプスアルパイン株式会社 | 音響信号処理装置、音響システム及び低音感の増強方法 |
Family Cites Families (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
SE0101175D0 (sv) | 2001-04-02 | 2001-04-02 | Coding Technologies Sweden Ab | Aliasing reduction using complex-exponential-modulated filterbanks |
TWI339991B (en) * | 2006-04-27 | 2011-04-01 | Univ Nat Chiao Tung | Method for virtual bass synthesis |
US8036903B2 (en) * | 2006-10-18 | 2011-10-11 | Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. | Analysis filterbank, synthesis filterbank, encoder, de-coder, mixer and conferencing system |
JP4983694B2 (ja) * | 2008-03-31 | 2012-07-25 | 株式会社Jvcケンウッド | 音声再生装置 |
BR122019023704B1 (pt) * | 2009-01-16 | 2020-05-05 | Dolby Int Ab | sistema para gerar um componente de frequência alta de um sinal de áudio e método para realizar reconstrução de frequência alta de um componente de frequência alta |
CN101505443B (zh) * | 2009-03-13 | 2013-12-11 | 无锡中星微电子有限公司 | 一种虚拟重低音增强方法及*** |
GB0906594D0 (en) * | 2009-04-17 | 2009-05-27 | Sontia Logic Ltd | Processing an audio singnal |
KR101613684B1 (ko) * | 2009-12-09 | 2016-04-19 | 삼성전자주식회사 | 음향 신호 보강 처리 장치 및 방법 |
US8638953B2 (en) * | 2010-07-09 | 2014-01-28 | Conexant Systems, Inc. | Systems and methods for generating phantom bass |
EP4016527B1 (fr) * | 2010-07-19 | 2023-02-22 | Dolby International AB | Traitement de signaux audio au cours d'une reconstruction haute fréquence |
JP5375861B2 (ja) * | 2011-03-18 | 2013-12-25 | ヤマハ株式会社 | オーディオ再生の効果付加方法およびその装置 |
CN102354500A (zh) * | 2011-08-03 | 2012-02-15 | 华南理工大学 | 一种基于谐波控制的虚拟低音增强处理方法 |
TWI575962B (zh) * | 2012-02-24 | 2017-03-21 | 杜比國際公司 | 部份複數處理之重疊濾波器組中的低延遲實數至複數轉換 |
-
2013
- 2013-09-27 CN CN201380053450.0A patent/CN104704855B/zh active Active
- 2013-09-27 EP EP13771123.0A patent/EP2907324B1/fr active Active
- 2013-09-27 JP JP2015536058A patent/JP5894347B2/ja active Active
- 2013-09-27 WO PCT/EP2013/070262 patent/WO2014060204A1/fr active Application Filing
- 2013-10-14 EP EP13188415.7A patent/EP2720477B1/fr active Active
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
RU2819779C1 (ru) * | 2020-03-20 | 2024-05-24 | Долби Интернешнл Аб | Усиление низких частот для громкоговорителей |
WO2023280356A1 (fr) * | 2021-07-09 | 2023-01-12 | Soundfocus Aps | Procédé et système de réseau de transducteurs pour la reproduction directionnelle d'un signal audio d'entrée |
WO2023280357A1 (fr) * | 2021-07-09 | 2023-01-12 | Soundfocus Aps | Procédé et système de haut-parleur pour traiter un signal audio d'entrée |
Also Published As
Publication number | Publication date |
---|---|
CN104704855B (zh) | 2016-08-24 |
WO2014060204A1 (fr) | 2014-04-24 |
EP2907324B1 (fr) | 2016-11-09 |
CN104704855A (zh) | 2015-06-10 |
JP5894347B2 (ja) | 2016-03-30 |
EP2720477A1 (fr) | 2014-04-16 |
EP2907324A1 (fr) | 2015-08-19 |
JP2015531575A (ja) | 2015-11-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8971551B2 (en) | Virtual bass synthesis using harmonic transposition | |
US20230074883A1 (en) | Apparatus and method for processing an input audio signal using cascaded filterbanks | |
US10043526B2 (en) | Harmonic transposition in an audio coding method and system | |
KR101201167B1 (ko) | 압축된 서브밴드 필터 임펄스 응답들을 형성하는 필터 압축기 및 방법 | |
EP2720477B1 (fr) | Synthèse virtuelle de graves à l'aide de transposition harmonique | |
US9640187B2 (en) | Method and an apparatus for processing an audio signal using noise suppression or echo suppression | |
JP4934427B2 (ja) | 音声信号復号化装置及び音声信号符号化装置 | |
US8886346B2 (en) | Oversampling in a combined transposer filter bank | |
US20060053018A1 (en) | Advanced processing based on a complex-exponential-modulated filterbank and adaptive time signalling methods | |
RU2666316C2 (ru) | Аппарат и способ улучшения аудиосигнала, система улучшения звука | |
EP2334103A2 (fr) | Appareil et procédé pour l'amélioration du son | |
JP2007011341A (ja) | 高調波信号の周波数拡張 | |
US9514767B2 (en) | Device, method and computer program for freely selectable frequency shifts in the subband domain | |
SG183966A1 (en) | Improved magnitude response and temporal alignment in phase vocoder based bandwidth extension for audio signals | |
JP7260101B2 (ja) | 情報処理装置、これを用いたミキシング装置、及びレイテンシ減少方法 | |
US8788277B2 (en) | Apparatus and methods for processing a signal using a fixed-point operation | |
JP2023518794A (ja) | スピーカのための低音強調 | |
RU2782168C1 (ru) | Система и способ для генерирования ряда сигналов высокочастотных поддиапазонов | |
RU2805938C1 (ru) | Система и способ для генерирования ряда сигналов высокочастотных поддиапазонов | |
CN104078048B (zh) | 一种声音解码装置及其方法 | |
EP3783912B1 (fr) | Dispositif de mélange, procédé de mélange et programme de mélange |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
17P | Request for examination filed |
Effective date: 20141016 |
|
RBV | Designated contracting states (corrected) |
Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G10L 21/02 20130101ALI20150715BHEP Ipc: G10L 21/038 20130101ALI20150715BHEP Ipc: H04R 3/04 20060101AFI20150715BHEP |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
INTG | Intention to grant announced |
Effective date: 20150918 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 778700 Country of ref document: AT Kind code of ref document: T Effective date: 20160315 Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602013005230 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20160302 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 778700 Country of ref document: AT Kind code of ref document: T Effective date: 20160302 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160603 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160602 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160302 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160302 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160302 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160302 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160302 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160302 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160302 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160302 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160302 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160302 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 4 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160702 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160302 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160302 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160302 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160302 Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160302 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160704 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602013005230 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160302 Ref country code: BE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160302 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160302 |
|
26N | No opposition filed |
Effective date: 20161205 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160302 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160602 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20161031 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20161031 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20161014 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 5 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20161014 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20131014 Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160302 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20161031 Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160302 Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160302 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 6 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160302 Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160302 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 10 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R081 Ref document number: 602013005230 Country of ref document: DE Owner name: DOLBY INTERNATIONAL AB, IE Free format text: FORMER OWNER: DOLBY INTERNATIONAL AB, AMSTERDAM, NL Ref country code: DE Ref legal event code: R081 Ref document number: 602013005230 Country of ref document: DE Owner name: DOLBY INTERNATIONAL AB, NL Free format text: FORMER OWNER: DOLBY INTERNATIONAL AB, AMSTERDAM, NL |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R081 Ref document number: 602013005230 Country of ref document: DE Owner name: DOLBY INTERNATIONAL AB, IE Free format text: FORMER OWNER: DOLBY INTERNATIONAL AB, DP AMSTERDAM, NL |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230512 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20230920 Year of fee payment: 11 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20230920 Year of fee payment: 11 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20230920 Year of fee payment: 11 |