EP2612437B1 - Spectrally uncolored optimal crosstalk cancellation for audio through loudspeakers - Google Patents
Spectrally uncolored optimal crosstalk cancellation for audio through loudspeakers Download PDFInfo
- Publication number
- EP2612437B1 EP2612437B1 EP11825672.6A EP11825672A EP2612437B1 EP 2612437 B1 EP2612437 B1 EP 2612437B1 EP 11825672 A EP11825672 A EP 11825672A EP 2612437 B1 EP2612437 B1 EP 2612437B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- audio
- loudspeakers
- crosstalk
- xtc
- frequency
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001228 spectrum Methods 0.000 claims description 57
- 230000004044 response Effects 0.000 claims description 53
- 239000011159 matrix material Substances 0.000 claims description 33
- 238000000034 method Methods 0.000 claims description 33
- 230000006870 function Effects 0.000 claims description 31
- 238000012546 transfer Methods 0.000 claims description 30
- 230000005236 sound signal Effects 0.000 claims description 17
- 230000001419 dependent effect Effects 0.000 claims description 13
- 230000000694 effects Effects 0.000 claims description 13
- 238000001914 filtration Methods 0.000 claims description 11
- 230000003595 spectral effect Effects 0.000 description 32
- 210000005069 ears Anatomy 0.000 description 22
- 230000003447 ipsilateral effect Effects 0.000 description 14
- 238000013461 design Methods 0.000 description 9
- 210000003128 head Anatomy 0.000 description 9
- 238000005457 optimization Methods 0.000 description 7
- 239000013598 vector Substances 0.000 description 6
- 235000009508 confectionery Nutrition 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 210000000613 ear canal Anatomy 0.000 description 4
- 238000004519 manufacturing process Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000009472 formulation Methods 0.000 description 3
- 239000000203 mixture Substances 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 239000004065 semiconductor Substances 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 230000004075 alteration Effects 0.000 description 2
- 230000002238 attenuated effect Effects 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 230000003247 decreasing effect Effects 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000004091 panning Methods 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000001066 destructive effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 210000000624 ear auricle Anatomy 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000005404 monopole Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000001755 vocal effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
- H04R3/04—Circuits for transducers, loudspeakers or microphones for correcting frequency response
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S1/00—Two-channel systems
- H04S1/002—Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2430/00—Signal processing covered by H04R, not provided for in its groups
- H04R2430/03—Synergistic effects of band splitting and sub-band processing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
- H04R3/12—Circuits for transducers, loudspeakers or microphones for distributing signals to two or more loudspeakers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R5/00—Stereophonic arrangements
- H04R5/04—Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
- H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
Definitions
- Binaural audio with loudspeakers also known as transauralization, aims to reproduce, at the entrance of each of the listener's ear canals, the sound pressure signals recorded on only the ipsilateral channel of a stereo signal. That is, only the sound signal of the left stereo channel is reproduced at the left ear and only the sound signal of the right stereo channel is reproduced at the right ear.
- the source signal was encoded with a head-related transfer function (HRTF) of the listener, or includes the proper interaural time difference (ITD) and interaural level difference (ILD) cues
- HRTF head-related transfer function
- ITD interaural time difference
- ILD interaural level difference
- Crosstalk occurs when the left ear (right ear) hears sounds from the right (left) audio channel, originating from the right speaker (left speaker). In other words, crosstalk occurs when the sound on one of the stereo channels is heard by the contralateral ear of the listener.
- Crosstalk corrupts HRTF information and ITD or ILD cues so that a listener may not properly or completely comprehend the soundfield's binaural cues that are embedded in the recording. Therefore, approaching the goal of BAL requires an effective cancellation of this unintended crosstalk, i.e. crosstalk cancellation or XTC for short.
- Previous XTC filter design methods based on system transfer matrix inversion strive to maintain a flat amplitude vs. frequency response at the ears of the listener by imposing a non-flat amplitude vs frequency response at the loudspeakers (as explained below), which causes a loss in the dynamic range of the processed sound, and, for reasons that will be explained below, leads to a spectral coloration of the sound as heard by the listener, even if the listener is sitting in the intended sweet spot.
- US 6,668,061 relates to a Crosstalk canceler wherein different frequency bands are cancelled at different locations so as to allow grater listener movement about the "sweet spot" while maintaining effective crosstalk cancellation.
- a method and system for calculating the frequency-dependent regularization parameter (FDRP) used in inverting the analytically derived or experimentally measured system transfer matrix for crosstalk cancellation (XTC) filter design is described.
- the method relies on calculating the FDRP that results in a flat amplitude vs frequency response at the loudspeakers (as opposed to a flat amplitude vs frequency response at the ears of the listener, as inherently done in prior art methods) thus forcing XTC to be effected into the phase domain only and relieving the XTC filter from the drawbacks of audible spectral coloration and dynamic range loss.
- XTC filters that yield optimal XTC levels over any desired portion of the audio band, impose no spectral coloration on the processed sound beyond the spectral coloration inherent in the playback hardware and/or loudspeakers, and cause no dynamic range loss.
- XTC filters designed with this method and used in the system are not only optimal but, due to their being free from Drawbacks D1, D2 and D3, allow for a most natural and spectrally transparent 3D audio reproduction of binaural or stereo audio through loudspeakers.
- the method and system do not attempt to correct the spectral characteristics of the playback hardware, and therefore are best suited for use with audio playback hardware and loudspeakers that are designed to meet a desired spectral fidelity level without the help of additional signal processing for spectral correction.
- an idealized situation consisting of two point sources (idealized loudspeakers) 12, 14 in free space (no sound reflections) and two listening points 16, 18 corresponding to the location of the ears of an idealized listener 20 (no HRTF).
- actual data corresponding to the impulse responses of real loudspeakers in a real room measured at the ear canal entrances of a dummy head will be used.
- l 1 and l 2 are the path lengths between any of the two sources 12, 14 and the ipsilateral and contralateral ear, respectively, as shown in Figure 1 .
- uppercase letters represent frequency variables
- lowercase represent time-domain variables
- uppercase bold letters represent matrices
- lowercase bold letters represent vectors, and define ⁇ ⁇ l ⁇ l 2 - l 1 and g ⁇ l 1 / l 2 as the path length difference and path length ratio, respectively.
- ⁇ r the effective distance between the entrances of the ear canals
- l the distance between either source and the interaural mid-point of the listener.
- ⁇ 2 ⁇ is the loudspeaker span.
- ⁇ c ⁇ ⁇ l c s defined as the time it takes a sound wave to traverse the path length difference ⁇ l .
- ⁇ e - i ⁇ l 1 / c s l 1 which, in the time domain, is a transmission delay (divided by the constant l 1 ) that does not affect the shape of the received signal.
- the diagonal elements of R i.e., R LL ( i ⁇ ) and R RR ( i ⁇ )
- R LL ( i ⁇ ) and R RR ( i ⁇ ) represent the ipsilateral transmission of the recorded sound signal to the ears
- off-diagonal elements i.e., R RL ( i ⁇ ) and R LR ( i ⁇ )
- the undesired contralateral transmission i.e., the crosstalk.
- S i is double (i.e., 6 dB above)
- S ci describes a signal of amplitude 1 panned to center (i.e., split equally between L and R inputs), while the former describes two signals of amplitude 1 fed in phase to the two inputs of the system.
- ⁇ ( ⁇ ) is the envelope spectrum that describes the maximum amplitude that could be expected at the loudspeakers, and is given by S ⁇ ⁇ ⁇ max S i ⁇ , S o ⁇ . It is relevant to note that ⁇ ( ⁇ ) is equivalent to the 2-norm of H , ⁇ H ⁇ , and that S i and S o are the two singular values of H .
- a perfect crosstalk cancellation (P-XTC) filter may be defined as one that, theoretically, yields infinite crosstalk cancellation at the ears of the listener, for all frequencies.
- the superscript [ P ] denotes perfect XTC.
- the spectra has a frequency varying behavior at the sources ( S si / / P ⁇ , S si X P ⁇ , S ci P ⁇ , and ⁇ [ P ] ( ⁇ )) that constitute severe spectral coloration, which, as we shall see below, only in an ideal world (i.e. under the idealized assumptions of the model) is not heard at the ears.
- Fig. 2 shows the frequency responses of a Perfect XTC filter at the loudspeakers: amplitude envelope (curve 22), side image (curve 24), and central image (curve 26).
- ⁇ C max ( 2 ⁇ g 2 + 1 g 2 + 2 ⁇ g cos ⁇ ⁇ ⁇ c + 1 - 1 , 2 ⁇ g 2 + 1 g 2 - 2 ⁇ g cos ⁇ ⁇ ⁇ c + 1 - 1 )
- Regularization methods allow controlling the norm of the approximate solution of an ill-conditioned linear system at the price of some loss in the accuracy of the solution.
- the control of the norm through regularization can be done subject to an optimization prescription, such as the minimization of a cost function.
- Regularization may be discussed analytically in the context of XTC filter optimization, which may be defined as the maximization of XTC performance for a desired tolerable level of spectral coloration or, equivalently, the minimization of spectral coloration for a desired minimum XTC performance.
- H ⁇ C H ⁇ C + ⁇ ⁇ I - 1 ⁇ C H
- H the superscript H denotes the Hermitian operator
- ⁇ the regularization parameter which essentially causes a departure from H [ P ]
- ⁇ is taken to be a constant, 0 ⁇ ⁇ ⁇ 1.
- the pseudoinverse matrix, H [ ⁇ ] is the regularized filter, and the superscript [ ⁇ ] is used to denote constant-parameter regularization.
- H ⁇ H LL ⁇ i ⁇ ⁇ H LR ⁇ i ⁇ ⁇ H RL ⁇ i ⁇ ⁇ H RR ⁇ i ⁇ ⁇ .
- E si ⁇ ⁇ ⁇ g 4 + ⁇ ⁇ g 2 - 2 ⁇ g 2 cos 2 ⁇ ⁇ ⁇ ⁇ c + ⁇ + 1 - 2 g 2 ⁇ cos 2 ⁇ ⁇ ⁇ ⁇ c + g 2 + ⁇ 2 + 2 ⁇ + 1 ;
- E si x ⁇ ⁇ 2 ⁇ g ⁇ ⁇ cos ⁇ ⁇ ⁇ c - 2 g 2 ⁇ cos 2 ⁇ ⁇ ⁇ ⁇ c + g 2 + ⁇ 2 + 2 ⁇ + 1 ;
- E ci ⁇ ⁇ 1 2 - ⁇ 2 ⁇ g 2 + 2 ⁇ gcos ⁇ ⁇ ⁇ c + ⁇ + 1 ;
- S Si ⁇ ⁇ ⁇ g 4 - 2 ⁇ ⁇ + 1 ⁇ g 2 cos 2 ⁇ ⁇ ⁇ ⁇ c + ⁇ + 1 2 - 2 g
- the envelope spectrum, ⁇ [ ⁇ ] ( ⁇ ), is plotted in Fig. 3 for three values of ⁇ .
- Two features can be noted in that plot: 1) increasing the regularization parameter attenuates the peaks in the spectrum without affecting the minima, and 2) with increasing ⁇ the spectral maxima split into doublet peaks (two closely-spaced peaks).
- the first and second derivatives of ⁇ [ ⁇ ] ( ⁇ ) with respect to ⁇ c are used to find the conditions for which the first derivative is nil and the second is negative.
- ⁇ * defined as ⁇ ⁇ ⁇ * ⁇ g - 1 2 .
- 20 ⁇ log 10 S ⁇ p ⁇ i ⁇ S ⁇ ⁇ ⁇ n 20 ⁇ log 10 ⁇ 2 ⁇ ⁇ 1 - g ⁇ d ⁇ B .
- the method and system of the present invention rely on the use of a specific scheme for calculating the frequency-dependent regularization parameter (FDRP) that would result in the flattening of the amplitude vs frequency spectrum measured at the loudspeakers and not at the ears of the listeners as is implicit in previous XTC filter designs that are based on the inversion of the system transfer matrix.
- FDRP frequency-dependent regularization parameter
- a frequency-dependent regularization parameter is calculated that would cause the envelope spectrum ⁇ ( ⁇ ) to be flat at a desired level ⁇ (in dB) over the frequency bands where the perfect filter's envelope spectrum exceeds ⁇ . Outside these bands (i.e., where the ⁇ [ P ] ( ⁇ ) is below ⁇ ), we apply no regularization.
- ⁇ 10 ⁇ / 20 with ⁇ given in dB.
- ⁇ cannot exceed the magnitude of the peaks in the ⁇ [ P ] ( ⁇ ) spectrum, ⁇ is bounded by: ⁇ ⁇ 1 1 - g where the bound is the maxima of the ⁇ [ P ] spectra, ⁇ [ P ] ⁇ , given by Eq. (18).
- the frequency-dependent regularization parameter needed to effect the spectral flattening required by Eq. (33) is obtained by setting ⁇ [ ⁇ ] ( ⁇ ), given by Eq. (27), equal to ⁇ and solving for ⁇ ( ⁇ ), which is now a function of frequency.
- ⁇ I ⁇ - g 2 + 2 ⁇ gcos ⁇ ⁇ ⁇ c + g 2 - 2 ⁇ gcos ⁇ ⁇ ⁇ c + 1 ⁇ - 1
- II ⁇ - g 2 - 2 ⁇ gcos ⁇ ⁇ ⁇ c + g 2 + 2 ⁇ gcos ⁇ ⁇ ⁇ c + 1 ⁇ - 1.
- the first solution, ⁇ I ( ⁇ ), applies for frequency bands where the out-of-phase response of the perfect filter (i.e., the second singular value, which is the second argument of the max function in Eq. (16)) dominates over the in-phase response (i.e., the first argument of that function):
- S o P 1 g 2 - 2 ⁇ gcos ⁇ ⁇ ⁇ c + 1 ⁇
- S i P 1 g 2 - 2 ⁇ gcos ⁇ ⁇ ⁇ c + 1 .
- Y is specifically chosen to be at or below the value equal to the lowest value of the ⁇ [ ⁇ ] ( ⁇ ) spectrum, i.e. S ⁇ P ⁇ ⁇ ⁇ as this would insure that the entire spectrum ⁇ [ ⁇ ] ( ⁇ ) is flat (i.e. the inequality in (34) does not hold and Branch P disappears) and XTC would be forced to be effected through phase effects only, resulting in no amplitude coloration due to XTC filtering and no dynamic range loss, all while insuring the minimization of whatever cost function is prescribed by the adopted optimization scheme (in this particular example, Eq. (23)).
- step 30 the system's transfer matrix in the frequency domain (i.e. matrix C as in Eq. (12) and the input 28) is inverted, either analytically (if it results from a tractable idealized model) or numerically (if it results from experimental measurements), using zero or a very small constant regularization parameter (large enough to avoid machine inversion problems) to obtain the corresponding perfect XTC filter, H [ P ] .
- FDRP frequency-dependent regularization parameter
- Step 40 the FDRP thus obtained, ⁇ ( ⁇ ), is used to calculate the pseudo-inverse of the system's transfer matrix (e.g. according to Eqn. (22)), which yields the sought regularized optimal XTC filter H [ ⁇ ] that has a flat frequency response at the loudspeakers.
- a time domain version (impulse response) of the filter is obtained in step 44 by simply taking the inverse Fourier transform of H [ ⁇ ] (output 42).
- the loudspeakers had a span of 60 degrees at the listening position, which was about 2.5 meters from each loudspeaker.
- Fig. 7 shows the four (windowed) measured impulse responses (IR) representing the transfer function in the time domain.
- the x-axis of each plot in Fig, 7 is time in ms, and the y- axis is the normalized amplitude of the measured signal.
- the top left plot shows the IR of the left loudspeaker measured at the left ear of the dummy head, and the bottom left plot shows the IR of the left loudspeaker measured at the right ear of the dummy head.
- the top right plot is the IR of the right speaker -left ear transfer function and the bottom plot is the IR of the right speaker - right ear transfer function.
- Fig 8 shows relevant spectra where the x-axis is frequency in Hz and the y -axis is amplitude in dB.
- the curve 48 in that plot is the frequency response C LL that corresponds to the left speaker-left ear transfer function in the frequency domain obtained by panning the test sound completely to the left channel.
- the ripples in curve 48 above 5 kHz are due to the HRTF of the head and the left ear pinna.
- Curve 50 is the response at the left loudspeaker ⁇ [ P ] ( ⁇ ) and shows a dynamic range loss of 31.45 dB (difference between the maximum and minimum in that curve).
- Curve 52 is the frequency response at the left (ipsilateral) ear, E si II , which, as expected from a perfect XTC filter, is essentially flat over the entire audio band.
- the curve 54 is the corresponding frequency response measured at the right (contralateral) ear, E si X , and shows significant attenuation with respect to curve 52 due to XTC.
- the difference in amplitude between the curves 52 and 54 linearly averaged over frequencies is the average XTC level, which for this case is 21.3 dB.
- Fig. 9 shows the responses due to a filter designed in accordance with the present invention.
- curve 60 representing , ⁇ [ ⁇ ] ( ⁇ )
- the response at the left loudspeaker is completely flat over the entire audio spectrum. Consequently, the frequency response at the left ear , curve 62, matches very well the corresponding measured system transfer function, C LL , shown in curve 64,. Since ⁇ [ ⁇ ] ( ⁇ ) is flat, there is no dynamic range loss associated with this filter.
- the average XTC level for this filter (obtained by taking the linear average of the difference between curve 62 and 66) is 19.54 dB, which is only 1.76 dB lower than the XTC level obtained with the perfect filter, testifying to the optimal nature of the regularized filter.
- the filter designed with the method of the present invention imposes no audible coloration to the sound of the playback system, has no dynamic range loss, and yields an XTC level that is essentially the same as that of a perfect XTC filter.
- the method described herein may be implemented in software, or firmware incorporated in a computer-readable storage medium for execution by a general purpose computer or a processor, such as a DSP chipset.
- suitable computer-readable storage mediums include a read only memory (ROM), a random access memory (RAM), a register, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs).
- Embodiments of the present invention may be represented as instructions and data stored in a computer-readable storage medium.
- aspects of the present invention may be implemented using Verilog, which is a hardware description language (HDL).
- Verilog data instructions may generate other intermediary data, (e.g., netlists, GDS data, or the like), that may be used to perform a manufacturing process implemented in a semiconductor fabrication facility.
- the manufacturing process may be adapted to manufacture semiconductor devices (e.g., processors) that embody various aspects of the present invention.
- Suitable processors include, by way of example, a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, a graphics processing unit (GPU), a DSP core, a controller, a microcontroller, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), any other type of integrated circuit (IC), and/or a state machine, or combinations thereof.
- DSP digital signal processor
- GPU graphics processing unit
- DSP core DSP core
- controller a microcontroller
- ASICs application specific integrated circuits
- FPGAs field programmable gate arrays
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Stereophonic System (AREA)
Description
- Binaural audio with loudspeakers (BAL), also known as transauralization, aims to reproduce, at the entrance of each of the listener's ear canals, the sound pressure signals recorded on only the ipsilateral channel of a stereo signal. That is, only the sound signal of the left stereo channel is reproduced at the left ear and only the sound signal of the right stereo channel is reproduced at the right ear. For example, if the source signal was encoded with a head-related transfer function (HRTF) of the listener, or includes the proper interaural time difference (ITD) and interaural level difference (ILD) cues, then delivering the signal on each of the channels of the stereo signal to the ipsilateral ear, and only to that ear, would ideally guarantee that the ear-brain system receives the cues it needs to hear an accurate 3-dimensional (3-D) reproduction of a recorded soundfield.
- However, an unintended consequence of binaural audio playback through loudspeakers is crosstalk. Crosstalk occurs when the left ear (right ear) hears sounds from the right (left) audio channel, originating from the right speaker (left speaker). In other words, crosstalk occurs when the sound on one of the stereo channels is heard by the contralateral ear of the listener.
- Crosstalk corrupts HRTF information and ITD or ILD cues so that a listener may not properly or completely comprehend the soundfield's binaural cues that are embedded in the recording. Therefore, approaching the goal of BAL requires an effective cancellation of this unintended crosstalk, i.e. crosstalk cancellation or XTC for short.
- While there are various techniques for effecting some level of crosstalk cancellation (XTC) for a two loudspeaker system, they all have one or more of the following drawbacks:
- D1: Severe spectral coloration to the sound heard by the listener, even if that listener is sitting in the intended sweet spot.
- D2: Useful XTC levels are reached only at limited frequency ranges of the audio band.
- D3: Severe dynamic range loss when the sound is processed through the XTC filter or processor (while avoiding distortion and/or clipping).
- The above drawbacks can be seen by analyzing XTC using the most fundamental formulation of the XTC problem - that is by looking at the inverse of the system transfer matrix (as will be shown and discussed below) that describes sound propagation from the loudspeakers to the ears of the listener.
- While the technique of constant parameter (non-frequency dependent) regularization, commonly used in XTC filter design to make the inversion of the system transfer matrix better behaved, may alleviate some of Drawback D3, it inherently introduces spectral artifice of its own (specifically, at the expense of reducing the amplitude of the spectral peaks in the inverted transfer matrix, constant-parameter regularization results in undesirable narrow-band artifacts at higher frequencies and a rolloff at lower frequencies at the loudspeakers) and does little to alleviate the other two drawbacks (D1 and D2).
- Prior art frequency-dependent regularization, even when coupled with an effective optimization scheme, is not enough to deal away with Drawbacks D1, D2 and D3.
- Previous XTC filter design methods based on system transfer matrix inversion (with or without regularization) strive to maintain a flat amplitude vs. frequency response at the ears of the listener by imposing a non-flat amplitude vs frequency response at the loudspeakers (as explained below), which causes a loss in the dynamic range of the processed sound, and, for reasons that will be explained below, leads to a spectral coloration of the sound as heard by the listener, even if the listener is sitting in the intended sweet spot.
- Therefore, while previous methods are useful for designing XTC filters that can inherently correct for non-idealities in the amplitude vs frequency response of the playback hardware and loudspeakers, they do not address all of Drawbacks D1, D2 and D3.
-
US 6,668,061 relates to a Crosstalk canceler wherein different frequency bands are cancelled at different locations so as to allow grater listener movement about the "sweet spot" while maintaining effective crosstalk cancellation. - A method and system for calculating the frequency-dependent regularization parameter (FDRP) used in inverting the analytically derived or experimentally measured system transfer matrix for crosstalk cancellation (XTC) filter design is described. The method relies on calculating the FDRP that results in a flat amplitude vs frequency response at the loudspeakers (as opposed to a flat amplitude vs frequency response at the ears of the listener, as inherently done in prior art methods) thus forcing XTC to be effected into the phase domain only and relieving the XTC filter from the drawbacks of audible spectral coloration and dynamic range loss. When the method is used with any effective optimization scheme it results in XTC filters that yield optimal XTC levels over any desired portion of the audio band, impose no spectral coloration on the processed sound beyond the spectral coloration inherent in the playback hardware and/or loudspeakers, and cause no dynamic range loss. XTC filters designed with this method and used in the system are not only optimal but, due to their being free from Drawbacks D1, D2 and D3, allow for a most natural and spectrally transparent 3D audio reproduction of binaural or stereo audio through loudspeakers. The method and system do not attempt to correct the spectral characteristics of the playback hardware, and therefore are best suited for use with audio playback hardware and loudspeakers that are designed to meet a desired spectral fidelity level without the help of additional signal processing for spectral correction.
- A more detailed understanding of the present invention may be had from the following detailed description which should read in light of the accompanying drawings wherein:
-
FIG. 1 is a diagram of a listener and a two-source model; -
FIG. 2 is a plot of the frequency responses of the perfect XTC filter at the loudspeakers, -
FIG. 3 is a plot showing the effects of regularization on the envelope spectrum at the loudspeakers, -
FIG. 4 shows the effects of regularization on the crosstalk cancellation spectrum, -
FIG. 5 is a plot showing the envelope spectrum at the loudspeakers, -
FIG. 6 is a flow chart of the method of the present invention. -
FIG. 7 shows four (windowed) measured impulse responses (IR) representing the transfer function in the time domain. -
FIG. 8 is a graph showing measured spectra associated with a perfect XTC filter -
FIG. 9 is a graph showing measured spectra for an XTC filter of the present invention. - In order to explain the advantages of the method and system of the present invention an analytical formulation of the fundamental XTC problem in an idealized situation will be described and the "perfect XTC filter" will be defined, which will serve as a benchmark illustrating the severe problem of audible spectral coloration inherent to all XTC filters.
- In the following description, for the sake of clarity and to allow analytical insight, an idealized situation will be used consisting of two point sources (idealized loudspeakers) 12, 14 in free space (no sound reflections) and two
listening points - In the frequency domain, the air pressure at a free-field point located a distance r from a point source (monopole) radiating a sound wave of frequency ω, under the idealizing assumptions that sound propagation occurs in a free field (with no diffraction or reflection from the head and pinnae of the listener or any other physical objects), and that the loudspeakers radiate like point sources, is given by:
where ρo is the air density, k = 2π / λ = ω / cs is the wavenumber, λ is the wavelength, cs is the speed of sound (340.3 m/s), and q is the source strength (in units of volume per unit time). Defining the mass flow rate of air from the center of the source, V, as:
which is the time derivative ofFigure 1 the air pressure due to the twosources
Similarly, at theright ear 18 of thelistener 20 the following is the sensed pressure:
Here, l 1 and l 2 are the path lengths between any of the twosources Figure 1 . -
- Because the contralateral distance in the geometry of
Fig. 1 is greater than the ipsilateral distance, then 0 < g < 1. Further, from the geometry inFigure 1 , the two distances may be expressed as:
where Δr is the effective distance between the entrances of the ear canals, and l is the distance between either source and the interaural mid-point of the listener. As defined inFig. 1 , Θ = 2θ is the loudspeaker span. Note that for l >> Δrsin(θ), as in many loudspeaker-based listening set-ups, which leads to g ≈ 1. Another important parameter is the time delay,
defined as the time it takes a sound wave to traverse the path length difference Δl. - Using equations (1) and (2), the received signal at the listener's
left ear 16 and the received signal at the listener'sright ear 18 may be written in vector form as:
where
which, in the time domain, is a transmission delay (divided by the constant l 1) that does not affect the shape of the received signal. The source vector at the loudspeaker comprising a left channel, VL , and a right channel, VR , is written in vector form as v = [VL (iω),VR (iω)] T. v may be obtained from the two channels of "recorded" signals, denoted d = [DL (iω),DR (iω)] T, using the transformation
where
is the sought 2 × 2 filter or transformation matrix for XTC. Therefore, from Eq. (7), the following result may be obtained
where p = [PL (iω),PR (iω)] T is the vector of pressures at the ears, and C is the system's transfer matrix
which is symmetric due to the symmetry of the geometry shown inFig. 1 . -
- The diagonal elements of R (i.e., RLL (iω) and RRR (iω)) represent the ipsilateral transmission of the recorded sound signal to the ears, and the off-diagonal elements (i.e., RRL (iω) and RLR (iω)) represent the undesired contralateral transmission, i.e., the crosstalk.
- A set of metrics by which to judge the spectral coloration and performance of XTC filters will now be described. The amplitude spectrum (to a factor α) of a signal fed to only one (either left or right) of the two inputs of the system, as heard at the ipsilateral ear is
where the subscripts "si" and || stand for "side image" and "ipsilateral ear (with respect to the input signal)", respectively, since E si||, as defined, is the frequency response (at the ipsilateral ear) for the side image that would result from the input being panned to one side. Similarly, at the contralateral ear to the input signal (subscript X), the following is the side-image frequency response:
The system's frequency response at either ear when the same signal is split equally between left and right inputs is another spectral coloration metric:
Here the subscript "ci" stands for "center image" since Eci , as defined, is the frequency response (at either ear) for the center image that would result from the input being panned to the center. - Also of importance are the frequency responses that would be measured at the sources (i.e., the loudspeakers), which are denoted by S and may be obtained from the elements of the filter matrix H:
They are given using the same subscript convention used with the amplitude spectrum above (with "∥" and "X" referring to the loudspeakers that are ipsilateral and contralateral to the input signal, respectively). An intuitive interpretation of the significance of the above metrics is that a signal panned from a single input to both inputs to the system will result in frequency responses going from Esi to Eci at the ears, and Ssi to Sci at the loudspeakers. - Two other spectral coloration metrics are the frequency responses of the system to in-phase and out-of-phase inputs to the system. These two responses are given by:
The subscripts i and o denote the in-phase and out-of-phase responses, respectively. Note that, as defined, Si is double (i.e., 6 dB above) S ci, as the latter describes a signal ofamplitude 1 panned to center (i.e., split equally between L and R inputs), while the former describes two signals ofamplitude 1 fed in phase to the two inputs of the system. - Since a real signal can comprise various components having different phase relationships, it is useful to combine Si (ω) and So (ω) into a single metric, Ŝ(ω), which is the envelope spectrum that describes the maximum amplitude that could be expected at the loudspeakers, and is given by
It is relevant to note that Ŝ(ω) is equivalent to the 2-norm of H, ∥H∥, and that Si and So are the two singular values of H. - Finally, an important metric that will allow for the evaluation and comparison of the XTC performance of various filters is χ(ω), the crosstalk cancellation spectrum:
It is the ratio of the amplitude spectrum at the ipsilateral ear to the amplitude spectrum at the contralateral ear and, therefore, the greater the value of the crosstalk cancellation spectrum, χ(ω), the more effective is the crosstalk cancellation filter. The above definitions give a total of eight metrics, (EsiII , EsiX ,Eci ,SsiII ,SsiX ,Sci ,Ŝ,χ), all real functions of frequency, by which to evaluate and compare the spectral coloration and XTC performance of XTC filters. - A perfect crosstalk cancellation (P-XTC) filter may be defined as one that, theoretically, yields infinite crosstalk cancellation at the ears of the listener, for all frequencies. Crosstalk cancellation requires that the received signal at each of the two ears be that which would have resulted from the ipsilateral signal alone. Therefore, in order to achieve perfect cancellation of the crosstalk, Eq. (13) requires that R = CH = I, where I is the unity matrix (identity matrix), and thus, as per the definition of R in Eq. (14), the P-XTC filter is the inverse of the system transfer matrix expressed in Eq. (12), and may be expressed exactly:
where the superscript [P] denotes perfect XTC. For this filter, the eight metrics defined above become: - The perfect XTC filter (χ [P] = ∞) gives flat frequency responses at the ears (as evidenced by the constant
- The extent of spectral coloration at the loudspeakers is plotted in
Fig. 2 which shows the frequency responses of a Perfect XTC filter at the loudspeakers: amplitude envelope (curve 22), side image (curve 24), and central image (curve 26). The dotted horizontal line marks the envelope ceiling, which for this case (g=.985) is 36.5 dB. The non-dimensional frequency ω / τc is given on the bottom axis, and the corresponding frequency in Hz, shown on the top axis, is to illustrate a particular (typical) case of τc =3 samples at the redbook CD sampling rate of 44.1 kHz.(which would be the case, for instance, of a set-up with Δr = 15 cm, l = 1.6 m, and Θ = 18°.) - The peaks in the
Fig. 2 at frequencies for which the amplitude of the signal at the loudspeakers must be boosted in order to effect XTC at the ears while compensating for the destructive interference at that location. Similarly, minima in the spectra occur when the amplitude must be attenuated due to constructive interference. -
- For a typical listening set-up, g ≈ 1, say, a reference g = 985 case shown in
Fig. 2 , the envelope peaks (i.e., Ŝ [P]↑) correspond to a boost of
(and the peaks in the other spectra, - The severe spectral coloration associated with these high-amplitude peaks presents three practical problems: 1) it would be heard by a listener outside the sweet spot, 2) it would cause a relative increase (compared to unprocessed sound playback) in the physical strain on the playback transducers, and 3) it would correspond to a loss in the dynamic range.
- These penalties might be a justifiable price if infinitely good XTC performance (χ = ∞) and perfectly flat frequency response (E [P](ω) = constant) that the perfect XTC filter promises were guaranteed at the ears of a listener in the sweet spot. However, in practice, these theoretically promised benefits are unachievable due to the solution's sensitivity to unavoidable errors. This problem can best be appreciated by evaluating the condition number of the transfer matrix C.
- It is well known that in matrix inversion problems the sensitivity of the solution to errors in the system is given by the condition number of the matrix. The condition number κ(C) of the matrix C is given by
(It is also, equivalently, the ratio of largest to smallest singular values of the matrix.) Therefore, we have
Using the first and second derivatives of this function, as was done for the previous spectra, the following are the maxima and minima:
First, it is noted that the peaks and minima in the condition number occur at the same frequencies as those of the amplitude envelope spectrum at the loudspeakers, Ŝ [P]. Second, it is noted that the minima have a condition number of unity (the lowest possible value), which implies that the XTC filter resulting from the inversion of C is most robust (i.e., least sensitive to errors in the transfer matrix) at the non-dimensional frequencies - Regularization methods allow controlling the norm of the approximate solution of an ill-conditioned linear system at the price of some loss in the accuracy of the solution. The control of the norm through regularization can be done subject to an optimization prescription, such as the minimization of a cost function. Regularization may be discussed analytically in the context of XTC filter optimization, which may be defined as the maximization of XTC performance for a desired tolerable level of spectral coloration or, equivalently, the minimization of spectral coloration for a desired minimum XTC performance.
- A pseudoinverse representing a nearby solution to the matrix inversion problem is sought:
where the superscript H denotes the Hermitian operator, and β is the regularization parameter which essentially causes a departure from H [P], the exact inverse of C. β is taken to be a constant, 0< β <<1. The pseudoinverse matrix, H [β], is the regularized filter, and the superscript [β] is used to denote constant-parameter regularization. The regularization stated in Eq. (22) corresponds to a minimization of a cost function, J(iω),
where the vector e represents a performance metric that is a measure of the departure from the signal reproduced by the perfect filter. Physically, then, the first term in the sum constituting the cost function represents a measure of the performance error, and the second term represents an "effort penalty," which is a measure of the power exerted by the loudspeakers. For β > 0 , Eq. (22) leads to an optimum, which corresponds to the least-square minimization of the cost function J(iω). - Therefore, an increase of the regularization parameter β leads to a minimization of the effort penalty at the expense of a larger performance error and thus to an abatement of the peaks in the norm of H, i.e., the coloration peaks in the S(ω) spectra, at the price of a decrease in XTC performance at and near the frequencies where the system is ill-conditioned.
- Using the explicit form for C given by Eq. (12), the frequency response of the constant parameter regularization XTC filter becomes:
where
The eight metric spectra we defined herein become:
It is worth noting that as β → 0, H [β] → H [P] and the spectra of the perfect XTC filter are recovered from the expressions above as expected. - The envelope spectrum, Ŝ [β](ω), is plotted in
Fig. 3 for three values of β. Two features can be noted in that plot: 1) increasing the regularization parameter attenuates the peaks in the spectrum without affecting the minima, and 2) with increasing β the spectral maxima split into doublet peaks (two closely-spaced peaks). - To get a measure of peak attenuation and the conditions for the formation of doublet peaks, the first and second derivatives of Ŝ [β](ω) with respect to ωτ c are used to find the conditions for which the first derivative is nil and the second is negative. These conditions are summarized as follows: If β is below a threshold β* defined as
the peaks are singlets and occur at the same non-dimensional frequencies as for the envelope spectrum peaks of the P-XTC filter (Ŝ [P]↑), and have the following amplitude:
at ωτc = nπ, with n = 0,1,2,3,4, ... - If the condition
is satisfied, the maxima are doublet peaks located at the following non-dimensional frequencies:
and have an amplitude
which does not depend on g. (The superscripts ↑ and †↑ denote singlet and doublet peaks, respectively.) The attenuation of peaks in the Ŝ [β] spectrum due to regularization can be obtained by dividing the amplitude of the peaks in the P-XTC (i.e., β = 0) spectrum by that of peaks in the regularized spectrum. For the case of singlet peaks, the attenuation is
and for doublet peaks, it is given by - For the typical case of g = .985 illustrated in
Fig. 2 , we have β* = 2.225 × 10-4, and for β = .005 and 0.05 we get doublet peaks that are attenuated (with respect to the peaks in the P-XTC spectrum) by 19.5 and 29.5 dB, respectively, as marked on that plot. Therefore, increasing the regularization parameter above this (typically low) threshold causes the maxima in the envelope spectrum to split into doublet peaks shifted by a frequencyFig. 3 . Therefore, constant-β regularization transforms the bass boost of the perfect XTC filter into a bass roll-off. - Since regularization is essentially a deliberate introduction of error into system inversion, it is expected that both the XTC spectrum and the frequency responses at the ears will suffer (i.e., depart from their ideal P-XTC filter levels of ∞ and 0 dB, respectively) with increasing β. The effects of constant-parameter regularization on responses at the ears are illustrated in
Fig. 4 which shows the effects of regularization on the crosstalk cancellation spectrum, χ [β](ω) (top two curves), and the ipsilateral frequency response at the ear for a side image, EsiII (ω). The black horizontal bars on the top axis mark the frequency ranges for which an XTC level of 20∼dB or higher is reached with β=.05, and the grey bars represent the same for the case of β =.005. (Other parameters are the same as forFig. 2 ). - The black curves in that plot represent the crosstalk cancellation spectra and show that XTC control is lost within frequency bands centered around the frequencies where the system is ill-conditioned (ωτc ) = nπ with n = 0,1,2,3,4,... ) and whose frequency extent widens with increasing regularization. For example, increasing β to .05 limits XTC of 20 dB or higher to the frequency ranges marked by black horizontal bars on the top axis of that figure, with the first range extending only from 1.1 to 6.3 kHz and the second and third ranges located above 8.4 kHz. In many practical applications, such high (20 dB) XTC levels may not be needed or achievable (e.g., because of room reflections and/or mismatch between the HRTF of the listener and that used (e.g. dummy head) to design the filter, and the higher values of β needed to tame the spectral coloration peaks below a required level at the loudspeakers may be tolerated.
- The
Fig. 4 , depart only by a few dB from the corresponding P-XTC (i.e., β = 0) filter response (which is a flat curve at 0 dB). More precisely and generally, the maxima and minima of the
For the typical (g = .985) example shown in the figure, for - In sum, while constant-parameter regularization, a commonly used technique in the design of XTC filters, is effective at reducing the amplitude of peaks (including the "low-frequency boost") in the envelope spectrum at the loudspeakers, it typically results in undesirable narrow-band artifacts at higher frequencies and a rolloff of the lower frequencies at the loudspeakers. This non-optimal behavior can be avoided if the regularization parameter is allowed to be a function of the frequency, as described herein.
- The method and system of the present invention rely on the use of a specific scheme for calculating the frequency-dependent regularization parameter (FDRP) that would result in the flattening of the amplitude vs frequency spectrum measured at the loudspeakers and not at the ears of the listeners as is implicit in previous XTC filter designs that are based on the inversion of the system transfer matrix.
- Flattening of the amplitude vs frequency spectrum measured at the loudspeakers, as opposed to at the ear of the listener, forces XTC to result from phase effects only, and not from amplitude effects, since the amplitude is flat with frequency at the loudspeakers. This means that any inherent spectral (i.e. amplitude vs frequency) coloration in the loudspeaker and/or playback hardware will not be corrected for (as is inherently done in previous inversion-based XTC filter design methods where the XTC filter aims to reproduce at the ears the same amplitude vs frequency response of the recorded the signal).
- Flattening of the amplitude vs frequency spectrum measured at the loudspeakers, results in the listener hearing the same amplitude vs frequency response that would be heard without processing the sound through the XTC filter. This implies that the listener would not hear any spectral coloration beyond that due to the playback hardware and loudspeakers without the filter. Equally important is the fact that such a flat filter response at the loudspeakers also means no dynamic range loss in the processed audio.
- In order to explain method and system of the present invention, an idealized analytical description of how to calculate a frequency-dependent regularization parameter will be described that results in the specific goal of flattening the XTC filter response at the loudspeakers.
- For the sake of clarity, the same optimization scheme described with respect to the minimization of the cost function expressed in Eq. (23)) will be used, keeping in mind that the method and system of the present invention are completely independent of the adopted optimization scheme
- In order to avoid the frequency-domain artifacts discussed above and illustrated in
Fig. 3 , a frequency-dependent regularization parameter is calculated that would cause the envelope spectrum Ŝ(ω) to be flat at a desired level Γ (in dB) over the frequency bands where the perfect filter's envelope spectrum exceeds Γ. Outside these bands (i.e., where the Ŝ [P](ω) is below Γ), we apply no regularization. This can be stated symbolically as:
where the P-XTC envelope spectrum, Ŝ [P](ω), is given by Eq. (16), and
with Γ given in dB. Γ cannot exceed the magnitude of the peaks in the Ŝ [P](ω) spectrum, γ is bounded by:
where the bound is the maxima of the Ŝ [P] spectra, Ŝ [P]↑, given by Eq. (18). - The frequency-dependent regularization parameter needed to effect the spectral flattening required by Eq. (33) is obtained by setting Ŝ [β](ω), given by Eq. (27), equal to γ and solving for β(ω), which is now a function of frequency. Since the regularized spectral envelope, Ŝ [β](ω), (which is also ||H [β]||, the 2-norm of the regularized XTC filter) is the maximum of two functions, two solutions for β(ω) are obtained:
The first solution, βI (ω), applies for frequency bands where the out-of-phase response of the perfect filter (i.e., the second singular value, which is the second argument of the max function in Eq. (16)) dominates over the in-phase response (i.e., the first argument of that function): - Similarly, regularization with βII (ω) applies for frequency bands where
- Branch I: applies where Ŝ [P](ω) ≥ γ and
and requires setting Ŝ(ω) = γ, β = βI (ω); - Branch II: applies where Ŝ [P](ω) ≥ γ and
and requires setting Ŝ(ω) = γ, β = βII (ω);
Branch P: applies where Ŝ [P](ω) < γ,
and requires setting Ŝ(ω) = Ŝ [P](ω), β = 0. - Following this three-branch division, the envelope spectrum at the loudspeakers, Ŝ(ω), for the case of frequency-dependent regularization is plotted as the thick black curve in
Fig. 5 for Γ=7 dB. This value was chosen because it corresponds to the magnitude of the (doublet) peaks in the β = .05 spectrum (i.e., - It is seen from that figure that the low-frequency boost and the high-frequency peaks of the perfect XTC spectrum, which would be transformed into a low-frequency roll-off and narrow-band artifacts, respectively, by constant-β regularization, are now flat at the desired maximum coloration level, Γ. The rest of the spectrum, i.e., the frequency bands with amplitude below Γ, is allowed to benefit from the infinite XTC level of the perfect XTC filter and the robustness associated with relatively low condition numbers.
- In the method of the present invention Y is specifically chosen to be at or below the value equal to the lowest value of the Ŝ [β](ω) spectrum, i.e.
as this would insure that the entire spectrum Ŝ [β](ω) is flat (i.e. the inequality in (34) does not hold and Branch P disappears) and XTC would be forced to be effected through phase effects only, resulting in no amplitude coloration due to XTC filtering and no dynamic range loss, all while insuring the minimization of whatever cost function is prescribed by the adopted optimization scheme (in this particular example, Eq. (23)). - The above leads us to a general description of the method of the present invention in terms of specific steps that are taken in the XTC filter design procedure (the steps are also shown schematically in
Fig. 6 along with the associated input and output for each step): - In
step 30, the system's transfer matrix in the frequency domain (i.e. matrix C as in Eq. (12) and the input 28) is inverted, either analytically (if it results from a tractable idealized model) or numerically (if it results from experimental measurements), using zero or a very small constant regularization parameter (large enough to avoid machine inversion problems) to obtain the corresponding perfect XTC filter, H [P]. - In step 34 Γ is set equal to Γ*,be the lowest value (in dB) reached by the amplitude vs frequency response at the loudspeakers, Ŝ [P]↓ in Step 34. This is found from either Eq. (19) (or a similar equation resulting from another tractable analytical model) or from plotting the H [P] spectra (if the inversion was done numerically using actual measurements as in the example given further below) then calculate γ from γ* = 10Γ*/20 (36).
- In
Step 38, the frequency-dependent regularization parameter (FDRP) β(ω) that would result in a flat frequency response at the loudspeakers is calculated, so that Ŝ [β](ω) = constant ≤ γ* (as, for instance, is done by using Equations (37) and (38)) thus forcing XTC to be caused by phase effects only. - In
Step 40, the FDRP thus obtained, β(ω), is used to calculate the pseudo-inverse of the system's transfer matrix (e.g. according to Eqn. (22)), which yields the sought regularized optimal XTC filter H [β] that has a flat frequency response at the loudspeakers. (Finally, if needed for applying the resulting filter through a time-base convolution, as is often done in practical XTC implementation), a time domain version (impulse response) of the filter is obtained instep 44 by simply taking the inverse Fourier transform of H [β] (output 42).) - It should be noted that in
Step 38, if the FDRP is calculated so that Ŝ [β](ω) = constant ≤ γ*, the spectral flattening occurs for a side image (i.e. a sound panned to either the left or right channel and thus would be perceived by a listener to be located at or near his or left or right ear when the XTC level is sufficiently high). However, the same method can be used to flatten the response at the loudspeakers for an image that is not a pure side image by simply requiring that S [β](ω) = constant ≤ γ*, where S [β](ω) is the XTC filter's frequency response for an image of source panned anywhere between the left and right channels. For instance, to flatten for a central image, we set S [β] ci (ω), (given, for instance, by the equation preceding Eqn. 27) to a constant ≤ γ*, and proceed with the steps of the method as outlined above. In this context it is relevant to mention that for some applications, for instance pop music recording where the lead vocal audio is panned dead center, it might be desirable to flatten the response for a center image, i.e. Sci (ω), (or an image of any other desired panning) in order to avoid coloration of that image. It should also be noted in that context that since Ŝ [β](ω) ≥ S [β](ω) only flattening the side image (i.e. setting Ŝ [β](ω) = constant ≤ γ*) would result in no dynamic range loss due to the XTC filter. In other words, flattening for anything but the side image would incur a dynamic range loss that must be balanced by the benefit of a reduced spectral coloration for the desired panned image. For instance, for binaural recordings of real acoustic soundfields, which typically contain no dead-center panned images, flattening of the side image is advisable as this leads to no dynamic range loss. - An example based on the transfer function of two loudspeakers in a room measured by microphones placed at the ear canal entrances of a dummy head (Neumann KU-100) will now be described. The loudspeakers had a span of 60 degrees at the listening position, which was about 2.5 meters from each loudspeaker.
-
Fig. 7 shows the four (windowed) measured impulse responses (IR) representing the transfer function in the time domain. The x-axis of each plot inFig, 7 is time in ms, and the y-axis is the normalized amplitude of the measured signal. The top left plot shows the IR of the left loudspeaker measured at the left ear of the dummy head, and the bottom left plot shows the IR of the left loudspeaker measured at the right ear of the dummy head. The top right plot is the IR of the right speaker -left ear transfer function and the bottom plot is the IR of the right speaker - right ear transfer function. -
Fig 8 , shows relevant spectra where the x-axis is frequency in Hz and the y-axis is amplitude in dB. Thecurve 48 in that plot is the frequency response CLL that corresponds to the left speaker-left ear transfer function in the frequency domain obtained by panning the test sound completely to the left channel. The ripples incurve 48 above 5 kHz are due to the HRTF of the head and the left ear pinna. The other curves 50, 52 54 in that plot are the measured frequency responses associated with the perfect XTC filter, that is an XTC filter obtained by inverting the transfer function with essentially no regularization (β=10-5), In particular,Curve 50 is the response at the left loudspeaker Ŝ [P](ω) and shows a dynamic range loss of 31.45 dB (difference between the maximum and minimum in that curve).Curve 52 is the frequency response at the left (ipsilateral) ear, EsiII , which, as expected from a perfect XTC filter, is essentially flat over the entire audio band. Thecurve 54 is the corresponding frequency response measured at the right (contralateral) ear, EsiX , and shows significant attenuation with respect tocurve 52 due to XTC. The difference in amplitude between thecurves - We contrast these curves with those curves in
Fig. 9 which shows the responses due to a filter designed in accordance with the present invention. By design,curve 60, representing , Ŝ [β](ω), the response at the left loudspeaker, is completely flat over the entire audio spectrum. Consequently, the frequency response at the left ear ,curve 62, matches very well the corresponding measured system transfer function, CLL , shown incurve 64,. Since Ŝ [β](ω) is flat, there is no dynamic range loss associated with this filter. The average XTC level for this filter (obtained by taking the linear average of the difference betweencurve 62 and 66) is 19.54 dB, which is only 1.76 dB lower than the XTC level obtained with the perfect filter, testifying to the optimal nature of the regularized filter. [In sum, the filter designed with the method of the present invention, imposes no audible coloration to the sound of the playback system, has no dynamic range loss, and yields an XTC level that is essentially the same as that of a perfect XTC filter. - The method described herein may be implemented in software, or firmware incorporated in a computer-readable storage medium for execution by a general purpose computer or a processor, such as a DSP chipset. Examples of suitable computer-readable storage mediums include a read only memory (ROM), a random access memory (RAM), a register, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs).
- Embodiments of the present invention may be represented as instructions and data stored in a computer-readable storage medium. For example, aspects of the present invention may be implemented using Verilog, which is a hardware description language (HDL). When processed, Verilog data instructions may generate other intermediary data, (e.g., netlists, GDS data, or the like), that may be used to perform a manufacturing process implemented in a semiconductor fabrication facility. The manufacturing process may be adapted to manufacture semiconductor devices (e.g., processors) that embody various aspects of the present invention.
- Suitable processors include, by way of example, a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, a graphics processing unit (GPU), a DSP core, a controller, a microcontroller, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), any other type of integrated circuit (IC), and/or a state machine, or combinations thereof.
- While the foregoing invention has been described with reference to its preferred embodiments, various alterations and modifications will occur to those skilled in the art. All such alterations and modifications are intended to fall within the scope of the appended claims.
Claims (10)
- A method for filtering audio signals to cancel loudspeaker crosstalk in an audio system comprising the steps of
inverting a transfer matrix or function of the audio system;
using information from the inverted transfer matrix or function to calculate a frequency-dependent regularization parameter that is used to calculate a regularized inverse of said transfer matrix or function to obtain crosstalk cancellation filters that, when applied to audio signals, produces a flat frequency response at any of the loudspeakers of the audio system over an audio band or a portion thereof. - The method for filtering audio signals to cancel loudspeaker crosstalk of claim 1, wherein said crosstalk cancellation filters effect crosstalk cancellation only though phase effects over said audio band or portion thereof.
- The method for filtering audio signals to cancel loudspeaker crosstalk of claim 1, wherein said frequency-dependent regularization parameter when applied to audio signals produces a flat frequency response at one or more of the loudspeakers for a desired image panned anywhere between left and right channels.
- The method for filtering audio signals to cancel loudspeaker crosstalk of claim 1 wherein said audio system is a binaural audio system through loudspeakers.
- The method for filtering audio signals to cancel loudspeaker crosstalk of claim 1 wherein said audio system is a stereo audio system.
- A system for filtering audio signals to cancel loudspeaker crosstalk in an audio system comprising:an audio input stage;a processor forinverting a transfer matrix or function of the audio system;calculating a frequency-dependent regularization parameter that is it used to calculate a regularized inverse of said transfer matrix or function to obtain cross-talk cancellation filters that, when applied to audio signals, produces a flat frequency response at any of the loudspeakers of the audio system over an audio band or a portion thereof.
- The system for filtering audio signals to cancel loudspeaker crosstalk in an audio system of claim 6 wherein said crosstalk cancellation is effected by said processor only though phase effects over said audio band or portion thereof.
- The system for filtering audio signals to cancel loudspeaker crosstalk in an audio system of claim 6 wherein said processor has the capability of applying said frequency-dependent regularization parameter to filter audio signals to produce a flat frequency response at one or more of the loudspeakers for a desired image panned anywhere between left and right channels.
- The method of filtering audio signals to cancel loudspeaker crosstalk of claim 1 wherein the step of inverting the transfer matrix or function of the audio system comprises of calculating the inverse of the transfer matrix or functions over an entire audio spectrum.
- The system for filtering audio signals to cancel loudspeaker crosstalk in an audio system of claim 6 wherein said processor calculates the inverse of the transfer matrix or functions over an entire audio spectrum.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US37989110P | 2010-09-03 | 2010-09-03 | |
PCT/US2011/050181 WO2012036912A1 (en) | 2010-09-03 | 2011-09-01 | Spectrally uncolored optimal croostalk cancellation for audio through loudspeakers |
Publications (3)
Publication Number | Publication Date |
---|---|
EP2612437A1 EP2612437A1 (en) | 2013-07-10 |
EP2612437A4 EP2612437A4 (en) | 2014-04-02 |
EP2612437B1 true EP2612437B1 (en) | 2015-11-18 |
Family
ID=54290154
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP11825672.6A Active EP2612437B1 (en) | 2010-09-03 | 2011-09-01 | Spectrally uncolored optimal crosstalk cancellation for audio through loudspeakers |
Country Status (1)
Country | Link |
---|---|
EP (1) | EP2612437B1 (en) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10595150B2 (en) | 2016-03-07 | 2020-03-17 | Cirrus Logic, Inc. | Method and apparatus for acoustic crosstalk cancellation |
GB2556663A (en) * | 2016-10-05 | 2018-06-06 | Cirrus Logic Int Semiconductor Ltd | Method and apparatus for acoustic crosstalk cancellation |
CN112235691B (en) * | 2020-10-14 | 2022-09-16 | 南京南大电子智慧型服务机器人研究院有限公司 | Hybrid small-space sound reproduction quality improving method |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6668061B1 (en) * | 1998-11-18 | 2003-12-23 | Jonathan S. Abel | Crosstalk canceler |
GB0712998D0 (en) * | 2007-07-05 | 2007-08-15 | Adaptive Audio Ltd | Sound reproducing systems |
-
2011
- 2011-09-01 EP EP11825672.6A patent/EP2612437B1/en active Active
Also Published As
Publication number | Publication date |
---|---|
EP2612437A4 (en) | 2014-04-02 |
EP2612437A1 (en) | 2013-07-10 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9167344B2 (en) | Spectrally uncolored optimal crosstalk cancellation for audio through loudspeakers | |
US9918179B2 (en) | Methods and devices for reproducing surround audio signals | |
US9577595B2 (en) | Sound processing apparatus, sound processing method, and program | |
US9215544B2 (en) | Optimization of binaural sound spatialization based on multichannel encoding | |
CN110557710B (en) | Low complexity multi-channel intelligent loudspeaker with voice control | |
US8873762B2 (en) | System and method for efficient sound production using directional enhancement | |
WO2004103025A1 (en) | Loudspeaker system for virtual sound synthesis | |
Sakamoto et al. | Sound-space recording and binaural presentation system based on a 252-channel microphone array | |
US11381909B2 (en) | Method and apparatus for forming differential beam, method and apparatus for processing signal, and chip | |
TW201732785A (en) | Subband spatial and crosstalk cancellation for audio reproduction | |
Masiero | Individualized binaural technology: measurement, equalization and perceptual evaluation | |
Choueiri | Optimal crosstalk cancellation for binaural audio with two loudspeakers | |
TWI690220B (en) | Spectral defect compensation for crosstalk processing of spatial audio signals | |
US9510124B2 (en) | Parametric binaural headphone rendering | |
EP2612437B1 (en) | Spectrally uncolored optimal crosstalk cancellation for audio through loudspeakers | |
Choueiri | Binaural audio through loudspeakers | |
US20170127210A1 (en) | Acoustic signal processing device, acoustic signal processing method, and program | |
KR102358310B1 (en) | Crosstalk cancellation for opposite-facing transaural loudspeaker systems | |
Cecchi et al. | A multichannel and multiple position adaptive room response equalizer in warped domain: Real-time implementation and performance evaluation | |
Delikaris-Manias et al. | Parametric binaural rendering utilizing compact microphone arrays | |
WO2023010691A1 (en) | Earphone virtual space sound playback method and apparatus, storage medium, and earphones | |
US20240163630A1 (en) | Systems and methods for a personalized audio system | |
WO2019041213A1 (en) | Acoustic radiation control method and system | |
Brännmark et al. | Controlling the impulse responses and the spatial variability in digital loudspeaker-room correction. | |
Yuyama et al. | Hybrid structure of inverse filtering and DOA-parameterized wavefront synthesis |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20130329 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
DAX | Request for extension of the european patent (deleted) | ||
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R079 Ref document number: 602011021507 Country of ref document: DE Free format text: PREVIOUS MAIN CLASS: H03B0029000000 Ipc: H04S0001000000 |
|
A4 | Supplementary search report drawn up and despatched |
Effective date: 20140304 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: H04S 1/00 20060101AFI20140226BHEP |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
INTG | Intention to grant announced |
Effective date: 20150508 |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: CHOUEIRI, EDGAR, Y. |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 762069 Country of ref document: AT Kind code of ref document: T Effective date: 20151215 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602011021507 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: NV Representative=s name: JACOBACCI AND PARTNERS S.P.A., CH |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: FP |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 762069 Country of ref document: AT Kind code of ref document: T Effective date: 20151118 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160218 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151118 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151118 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151118 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160318 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160318 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151118 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151118 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151118 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151118 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151118 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160219 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151118 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151118 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602011021507 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151118 Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151118 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151118 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151118 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151118 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 6 |
|
26N | No opposition filed |
Effective date: 20160819 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151118 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151118 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151118 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20160901 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20160901 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 7 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151118 Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20110901 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20160930 Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151118 Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151118 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151118 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 8 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151118 |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230524 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: NL Payment date: 20230816 Year of fee payment: 13 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: IT Payment date: 20230810 Year of fee payment: 13 Ref country code: GB Payment date: 20230803 Year of fee payment: 13 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20230808 Year of fee payment: 13 Ref country code: DE Payment date: 20230808 Year of fee payment: 13 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: CH Payment date: 20231001 Year of fee payment: 13 |