EP1992198A2 - Optimization of binaural sound spatialization based on multichannel encoding - Google Patents
Optimization of binaural sound spatialization based on multichannel encodingInfo
- Publication number
- EP1992198A2 EP1992198A2 EP07731684A EP07731684A EP1992198A2 EP 1992198 A2 EP1992198 A2 EP 1992198A2 EP 07731684 A EP07731684 A EP 07731684A EP 07731684 A EP07731684 A EP 07731684A EP 1992198 A2 EP1992198 A2 EP 1992198A2
- Authority
- EP
- European Patent Office
- Prior art keywords
- encoding
- functions
- filters
- decoding
- hrir
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000005457 optimization Methods 0.000 title claims abstract description 30
- 230000006870 function Effects 0.000 claims abstract description 175
- 238000012546 transfer Methods 0.000 claims abstract description 23
- 238000000034 method Methods 0.000 claims description 59
- 238000009877 rendering Methods 0.000 claims description 17
- 230000000694 effects Effects 0.000 claims description 8
- 238000004091 panning Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 2
- 239000000284 extract Substances 0.000 claims description 2
- 230000001934 delay Effects 0.000 description 29
- 238000004364 calculation method Methods 0.000 description 12
- 238000001914 filtration Methods 0.000 description 7
- 210000003128 head Anatomy 0.000 description 7
- 230000008901 benefit Effects 0.000 description 6
- 238000000354 decomposition reaction Methods 0.000 description 6
- 238000013459 approach Methods 0.000 description 4
- 239000011159 matrix material Substances 0.000 description 4
- 210000005069 ears Anatomy 0.000 description 3
- 230000015654 memory Effects 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 241001362574 Decodes Species 0.000 description 2
- 230000007423 decrease Effects 0.000 description 2
- 210000000613 ear canal Anatomy 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000002156 mixing Methods 0.000 description 2
- 238000000513 principal component analysis Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 206010011878 Deafness Diseases 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000007429 general method Methods 0.000 description 1
- 238000007654 immersion Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 230000005236 sound signal Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 230000003936 working memory Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S1/00—Two-channel systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S1/00—Two-channel systems
- H04S1/002—Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
- H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S5/00—Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation
Definitions
- the present invention aims at processing sound signals for their spatialization.
- Spatial sound reproduction allows a listener to perceive sound sources coming from a direction or from any position in space.
- HRTF Head Related Transfer Functions
- HRIR Head Related Impulse Response
- the term "binaural” aims at restitution on a stereophonic headphones with nevertheless effects of spatialization.
- the present invention is not limited to this technique and also applies in particular to techniques derived from the binaural such as rendering techniques called “transaural”, that is to say on remote speakers.
- Such techniques can then use what is called a “crosstalk cancellation” (or “cross-talk cancellation”) which consists in canceling the acoustic cross paths so that a sound, thus processed and then emitted by the loudspeakers. speakers, can be perceived only by one of the two ears of a listener.
- this decomposition makes it possible to do encoding and decoding called "binaural multichannel".
- the decoding functions which are actually filters
- a set of spatial encoding functions which are in fact encoding gains
- when they are optimal in rendering ensure a feeling of immersion perfect for the listener inside a sound stage, whereas it actually has, for the binaural restitution, only two loudspeakers (headphones of a headphone or distant loudspeakers).
- the encoding is generally inexpensive in memory and / or in calculations since the spatial functions are gains that depend solely on the effects of the effects. sources to encode and not the number of sources themselves. Decoding also has a cost independent of the number of sources to be spatialised.
- the decoding functions can be individualized for each of the listeners.
- the present invention aims in particular an improved obtaining of decoding filters and / or encoding gains in binaural multichannel technique.
- the context is as follows: sources are spatialised by multichannel encoding and the restitution of the spatially encoded content is done by applying appropriate decoding filters.
- the reference WO-00/19415 discloses a binaural multichannel processing which provides for the calculation of decoding filters. By denoting: g t ( ⁇ p , ⁇ p ) fixed encoding spatial functions where g is the gain corresponding to the channel i ⁇ ⁇ , .., Nei at the position /?
- e 1, .., P defined by its angles of incidence ⁇ (azimuth) and ⁇ (elevation), h [ ⁇ p , ⁇ p , /) and R ⁇ ⁇ p , ⁇ p , f ⁇ bases of HRTF functions obtained by measuring the acoustic transfer functions of each ear L and R of an individual for a number P of positions of the space (pel, ..., P) and for a given frequency / it is provided in this document WO- 00/19415 essentially two steps to obtain filters from these spatial functions.
- each HRTF The delays of each HRTF are extracted. Indeed, the shape of a head is usually such that, for a given position, a sound arrives at one ear a certain time before reaching the other ear (a sound to the left arriving of course to the ear left before reaching the right ear).
- the delay difference t between the two ears is an interaural location index called ITD (for "Interaural Time Difference").
- ITD Interaural Time Difference
- decoding filters Z, ( (/) and R 1 (/) of channel i are obtained which satisfy the equations:
- a second approach, proposed in US-5,500,900, for jointly calculating the decoding filters and the spatial encoding functions is to decompose the HRIR sets by performing a principal component analysis (PCA) and then selecting a reduced number of components (which corresponds to the number of channels).
- PCA principal component analysis
- HRIR is very good with a small number of components.
- the techniques of the prior art require the extraction of delays HRIR.
- the techniques of sound recording or multi-channel encoding at a point in space are widely used since it is then possible to make transformations to the encoded signals (for example rotations).
- the delay information is not extractable from the signal alone.
- the decoding filters must then be able to reproduce the delays for an optimal sound reproduction.
- the number of channels can be low and the techniques of the prior art do not allow good decoding with few channels without extracting delays.
- the multichannel signal acquired may consist of only four channels, typically.
- ambiophonic microphones means microphones composed of coinciding directional sensors. The interaural delays must then be reproduced at decoding.
- the extraction of delays has at least two other major drawbacks: - the delays must be taken into account (addition of a step) at the time of the encoding, which increases the resources necessary in computation,
- the signals must be encoded for each ear and the number of filtering necessary for decoding is double.
- the present invention improves the situation.
- a sound spatialization method with multichannel encoding and binaural reproduction on two loudspeakers comprising a spatial encoding defined by encoding functions associated with a plurality of encoding channels and an application decoding. filters for binaural playback on the two speakers.
- the method according to the invention comprises the steps of: a) obtaining an original set of acoustic transfer functions specific to an individual morphology (HRIR, HRTF), b) choosing spatial encoding functions and / or filters decoding, and c) by successive iterations, optimizing the filters associated with the chosen encoding functions or the encoding functions associated with the selected filters, or together the filters and the encoding functions chosen, minimizing an error calculated according to a comparison between:
- the invention proposes the optimization calculation of the filters associated with a set of chosen encoding gains or encoding gains associated with a set of selected decoding filters, or a joint optimization of the decoding filters. and encoding gains.
- These filters and / or these gains have for example been fixed or initially calculated by the techniques of the pseudo-inverse or the virtual loudspeakers, described in particular in the document WO-00/19415. Then, these filters and / or the associated gains are improved, within the meaning of the invention, by an iterative optimization which aims to reduce a predetermined error function.
- the invention thus proposes the determination of decoding filters and encoding gains which allow both a good reconstruction of the delay but also a good reconstruction of the HRTF amplitude (modulus of HRTF), and this, for a weak number of channels, as will be seen with reference to the detailed description below.
- FIG. 1 illustrates the general steps of a method in the sense of the invention
- FIG. 2 illustrates the amplitude (gray levels) of the time functions HRIR (on several successive samples Ech) which have been chosen for the implementation implementation of step EO of FIG. 1, as a function of the azimuth (in degrees denoted deg °),
- FIG. 3 illustrates the appearance of some first spherical harmonics in ambiophonic context, as spatial encoding functions in a first embodiment
- FIGS. 4A, 4B, 4C compare the performances of the processing according to the first embodiment, for a non-optimized solution (FIG. 4A), for a solution partially optimized by a few processing iterations (FIG. 4B) and for a completely optimized solution. by the treatment in the sense of the invention (FIG. 4C),
- FIG. 5 illustrates the encoding functions in the virtual loudspeaker technique used in a second embodiment
- FIG. 6 compares a real average HRTF function (represented in solid line) with the reconstructed average HRTF functions using the solution of the pseudo-inverse in the sense of the prior art (represented by dotted lines), the starting solution given by the virtual loudspeaker method (shown in long broken lines) and the convergent optimized solution, in the sense of the second embodiment of the invention (shown in phantom),
- FIG. 7 compares the variations of the original ITD interaural delay (solid lines) with that obtained by the solution optimized in the sense of the second embodiment of the invention (mixed lines), with that reconstructed from the technique of the virtual speakers (long broken lines) and the one reconstructed from the filters obtained by the solution of the pseudo-inverse in the sense of the prior art (dashed lines), - Figure 8 schematically represents a spatialization system that can be obtained by the implementation of the first embodiment, taking into account interaural delays in encoding,
- FIG. 9 schematically represents a spatialization system that can be obtained by implementing the second embodiment, without taking into account the interaural delays in the encoding but including these delays in the decoding filters.
- the method within the meaning of the invention can be broken down into three steps:
- b) set spatial encoding functions and / or base filters, the encoding functions being denoted by g ( ⁇ , ⁇ , n) (or also g ( ⁇ , ⁇ , «, /)), where: ⁇ , ⁇ are the angles of incidence in azimuth and elevation,
- n is the index of the encoding channel considered
- obtaining the HRTF of the second ear can be deduced from the measurement of the first ear by symmetry.
- the set of functions HRIR can for example be measured on a subject by positioning microphones at the entrance of his ear canal.
- this HRIR game can also be calculated by numerical simulation methods (modeling of the morphology of the subject or calculation by network of artificial neurons) or having been the subject of a chosen treatment (reduction of the number of samples, phase correction, or other).
- step a) it is possible in this step a) to extract the delays from the HRIRs, to store them and then to add them at the time of the spatial encoding, the steps b) and c) remaining unchanged.
- This first step a) has the reference EO in FIG.
- step b if one seeks to obtain optimized filters on the one hand, it is necessary to set the spatial encoding functions g ( ⁇ , ⁇ , n) (or g ( ⁇ , ⁇ , n, f)) and, to obtain optimized spatial functions, the decoding filters denoted F (t, n) must be fixed.
- the spatial encoding functions are fixed, they are then reproducible and universal and the individualization of the filters is simply decoding. Moreover, the spatial encoding functions, when they comprise a large number of zeros among n encoding channels as in the second embodiment described below, make it possible to limit the number of operations during encoding. Pan-intensity laws between two-dimensional virtual loudspeakers and their three-dimensional extensions can be represented by encoding functions with only two non-zero gains, at most, for two dimensions and three non-zero gains for three dimensions, for a single given source. The number of non-zero winnings is, of course, independent of the number of channels and, most importantly, the zero winnings make it possible to lighten the encoding calculations.
- Spherical harmonic space functions in ambiophonic context have mathematical qualities that make it possible to transform the encoded signals (for example rotations of the sound field).
- such functions provide compatibility between binaural decoding and surround sound recordings based on a decomposition of the sound field into spherical harmonics.
- the encoding functions may be real or simulated directivity functions of microphones to allow listening of binaural multichannel recordings.
- the encoding functions can be arbitrary (non-universal) and determined by any method, the rendering then having to be optimized during subsequent steps of the method within the meaning of the invention.
- Spatial functions may also be a function of time or frequency.
- the optimization will be done taking into account this dependence (for example by optimizing independently each time sample or frequency).
- these can be fixed so that the decoding can be universal.
- Decoding filters can also be chosen to reduce the resource cost of filtering. For example, the use of filters called “infinite impulse response” or "HR” is advantageous.
- the decoding filters can also be chosen according to a psychoacoustic criterion, for example constructed from standardized Bark bands.
- the decoding filters can be determined by any method.
- the rendering, in particular for an individual listener, can then be optimized during the next steps of the method relating to the encoding functions.
- This second step b) relating to the calculation of an initial solution SO bears the reference E1 in FIG. 1.
- it consists in choosing the decoding filters (referenced “F”) and / or the spatial encoding functions ( referenced “g") and determine an initial solution SO for the encoding functions or the decoding filters, by a method also chosen.
- the filters of the starting solution SO at step E1 may be directly the HRIR functions given to the corresponding positions of the virtual speakers.
- the starting solution SO being further determined by functions defining the pan-effect laws. as encoding functions and by the HRIR functions, themselves, given to the positions of the virtual loudspeakers, as decoding filters.
- the decoding filters in step E1 are calculated from the pseudo-inverse to determine the starting solution SO. More generally, the starting solution SO in step E1 can be calculated from the least squares solution:
- the elements F, HRIR and g are matrices.
- the starting solution SO can be arbitrary (random or fixed), the essential being that it leads to obtaining a converged solution SC in step E6 of FIG.
- FIG. 1 also illustrates the operations E2, E3, T4, E5, E6 of general step c), optimization within the meaning of the invention.
- this optimization is conducted by iterations.
- step E3 the calculation of an error function is an important point of the optimization method within the meaning of the invention.
- a proposed error function is to simply minimize the difference in modules between the HRTF * Fourier transform of the reconstructed HRIR function set and the HRTF Fourier transform of the original HRIR function set (given in step EO).
- the error function can also minimize the energy difference between the modules, ie:
- any error function calculated entirely or in part from the HRIR functions can be provided (module, phase, delay or estimated ITD, interaural differences, or other).
- the optimization iterations can be successively applied to each frequency sample, with the advantage of then reducing the number of simultaneous variables, to have an error function specific to each frequency / and to meet a stop criterion according to the convergence specific to each frequency.
- Step T4 is a test for stopping or not the iteration of the optimization according to a chosen stopping criterion. It may be a criterion characterizing the fact that:
- variable c has reached a minimum value ⁇ , and / or that
- the filters F (n, t) or the gains g ( ⁇ , ⁇ , n) or the calculated filter / gain pairs make it possible to obtain an optimal spatial rendering, as will be seen in particular with reference to Figure 4C or Figure 6 below.
- the treatment then stops by obtaining a converged solution (step E6).
- this embodiment illustrated in FIG. 1 applies just as well when it was chosen to fix the decoding filters in step E1, and then to optimize the spatial encoding functions during the steps E2, E3. , E5, E6. It also applies when has been chosen to iteratively optimize both the encoding functions and the decoding filters.
- a set of HRIR functions measured for the left ear in the deaf chamber and for 64 different azimuth angle values ranging from 0 to about 0 are used.
- the HRIRs of the right ear are the symmetries of the HRIRs of the left ear.
- the HRIR functions can be obtained from standard databases ("Kemar head") or by modeling the morphology of the individual, or the like.
- the starting solution SO for step E1 is given by calculating the pseudo-inverse (with linear resolution).
- This starting solution constitutes the decoding solution which was proposed as such in the document WO-00/19415 of the prior art described above.
- the optimization technique used in the sense of the invention is preferably that of the gradient described above.
- the error function c used corresponds to the least squares on the module of the Fourier transform of the HRIR functions, namely:
- FIGS. 4A, 4B, 4C show the time course (over a few tens of time samples) of the five decoding filters and the module reconstruction errors (in dB, illustrated by gray levels) and the phase ( in radians, illustrated by gray levels) of the Fourier transform of the HRIR functions for each position (ordinates indicated in azimuth) and for each frequency (abscissa located in frequencies), respectively:
- Panoramic laws are commonly used by sound technicians to produce audio content, including multichannel content in so-called “surround” formats that are used in sound reproduction 5.1, 6.1, or other.
- panning encoding by panning laws is achieved by mixing a sound environment in a "surround” format (tracks 5.1 of a digital recording for example). Optimized filters from the same panning laws then allow for optimal binaural decoding for the desired rendering with this "surround" effect.
- the present invention is advantageously applied to the case where the positions of the virtual speakers correspond to positions of a multichannel rendering system for the general public, with a "surround” effect.
- the optimized decoding filters then allow decoding of multimedia consumer content (typically multi-channel content with "surround” effect) for playback on two speakers, for example on binaural headphones.
- This binaural rendering of a content that is for example initially in 5.1 format is optimized thanks to the implementation of the invention.
- the HRIR functions are obtained at 64 positions around the listener, as described with reference to the first embodiment above.
- R is the gain of the right speaker
- ⁇ v is the angle for which it is desired calculate the gains (typically the angle between the plane of symmetry of the two speakers and the desired direction).
- the optimization method used in the second embodiment is still that of the gradient.
- the starting solution SO at step E1 is given by the ten decoding filters which correspond to the ten HRIR functions given to the positions of the loudspeakers. virtual speakers.
- the fixed spatial functions are the encoding functions representing the panning laws.
- the error function c is based on the module of the Fourier transform of the HRIR functions, namely:
- FIG. 6 compares a real HRTF function (shown in solid lines), averaged over a set of 64 measured positions (for azimuth angles ranging from 0 to about 350 °), to the average HRTF functions reconstructed using: - the pseudo-inverse starting solution, without optimization (represented in dotted lines),
- Figure 7 illustrates the variations of the interaural delay ITD as a function of the azimuth position of the HRIR functions.
- the optimized solution makes it possible to reconstruct a delay ITD (mixed lines) relatively close to the original ITD (solid lines), but just as close as that reconstructed from the initial solution, here obtained by the technique of the loudspeakers virtual (long broken lines).
- the reconstructed ITD delay from the filters obtained by linear (pseudo-inverse) resolution, represented by dashed lines in FIG. 7, is rather irregular and remote from the original ITD.
- the case treated in the example described here is that of two spatially distinct sources to encode multichannel and restore binaural.
- the two exemplary embodiments of FIGS. 8 and 9 use the symmetry properties of the HRIR functions.
- FIG. 9 corresponds to the case where the encoding gains are obtained by applying the virtual loudspeaker method according to the second embodiment described above.
- Figure 8 shows an implementation of multichannel encoding and decoding when delays are not included in the decoding filters but must be taken into account as soon as encoding. It may correspond to that of the prior art described hereinbefore WO-00/19415, provided that the decoding filters (and / or the encoding functions) have not been optimized within the meaning of the invention. .
- FIG. 8 consists, in generic terms, of extracting, from the transfer functions obtained in step a), interaural delay information, while that the optimization, within the meaning of the invention, encoding functions and / or decoding filters is conducted here from the transfer functions from which these delay information has been extracted. Then, these interaural delays can be stored and then applied later, in particular to the encoding.
- FIGS. 8 and 9 the same notations S 1 and S 2 have, of course, been adopted for the two sources to be encoded, each being placed at a given position in space.
- ⁇ ⁇ ⁇ D and ⁇ ⁇ 2 D denote the delays (ITD) corresponding to the positions of the sources S 1 and S 2 .
- ITD delays
- both sounds are supposed to arrive at the right ear before reaching the left ear.
- F ⁇ The decoding filter for channel j and F JL symmetrical filters filters F, L. It is indicated here that in the case of virtual loudspeakers, the symmetrical filter of a given virtual loudspeaker (a given channel) is the filter of the virtual symmetrical loudspeaker (considering the left / right plane of symmetry of the head).
- L and R are the left and right binaural channels.
- the ITD delay is introduced at the time of encoding, the multichannel signals for the left channel are different from those for the right channel.
- the consequences of the introduction of coding delays are therefore the doubling of the number of encoding operations and the doubling of the number of channels, compared to the second implementation illustrated in FIG. 9 and taking advantage of the advantages offered by the second embodiment of the invention.
- each signal coming from a source S 1 in the encoding block ENCOD is split in order to apply to one of them a delay (positive or negative) ⁇ I ⁇ TD , T ⁇ 2 0 , and each doubled signal is multiplied by each gain g ⁇ ' L , the multiplication results being then grouped by channel index y (n channels) and whether interaural delay has been applied or not (2 times n channels in total).
- the 2n signals obtained are conveyed through a network, stored, or otherwise, for restitution and, for this purpose, are applied to a DECOD decoding block comprising n filters F JL for a left channel L and n filters. symmetrical F JL for a straight line R.
- the symmetry of the filters results from the fact that we consider a symmetry of HRTF functions.
- the signals to which the filters are applied are grouped together in each channel and the signal resulting from this grouping is intended to supply one of the two speakers with playback on two distant loudspeakers (in which case it is necessary to add an operation of cross paths cancellation) or directly one of the two channels of a headset with auricles in binaural restitution.
- FIG. 9 shows an implementation of multichannel encoding and decoding when the delays are, on the contrary, included in the decoding filters in the sense of the second embodiment using the virtual loudspeaker method. and exploiting the observation resulting from Figures 6 and 7 above.
- each sum or each difference of filters is to be considered as a filter in itself. What is indicated here as being a sum or a difference of filters is to be considered in relation to the expressions of the filters F JL and F j L described above with reference to FIG. 8.
- the decoding processing of FIG. 9 continues with a grouping of the sums SS and a grouping of the differences SD supplying by their sum the channel L (module SL delivering the signal SS + SD) and by their difference the channel R (module DR delivering the SS-SD signal).
- the useful working memory (buffer) for the implementation of FIG. 8 requires more than twice that useful for the implementation of FIG. 9, since 2n channels transit between the encoding and the decoding and it is necessary to implement a delay line by source in the implementation of FIG. figure 8.
- the present invention thus aims at a sound spatialization system with multichannel encoding and for a two-channel reproduction comprising an ENCOD spatial encoding block defined by encoding functions associated with a plurality of encoding channels and a decoding block. DECOD by applying filters for binaural rendition.
- the spatial encoding functions and / or the decoding filters are determined by the implementation of the method described above.
- Such a system may correspond to that illustrated in FIG. 8, in one embodiment for which the delays are integrated at the time of encoding, which corresponds to the state of the art within the meaning of document WO-00/19415.
- Another advantageous embodiment consists in implementing the method according to the second embodiment to then build a spatialization system with a direct encoding block, without applying a delay, so as to reduce a number of encoding channels and a corresponding number of decoding filters, which directly include the ITD interaural delays, according to an advantage offered by the implementation of the invention, as illustrated in FIG. 9.
- This embodiment of FIG. 9 makes it possible to achieve a quality of spatial rendering that is at least as good, if not better, than the techniques of the prior art, and this, with a number of filters half as great and a lower calculation cost. . Indeed, as shown with reference to FIGS. 6 and 7, in the case where the decomposition is aimed at a set of HRIR functions, this embodiment allows a quality of reconstruction of the HRTF module and the interaural delay better than the techniques of the prior art with a reduced number of channels.
- the present invention also provides a computer program comprising instructions for implementing the method described above and whose algorithm can be illustrated by a general flowchart of the type shown in Figure 1.
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Stereophonic System (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
FR0602098 | 2006-03-09 | ||
PCT/FR2007/050867 WO2007101958A2 (en) | 2006-03-09 | 2007-03-01 | Optimization of binaural sound spatialization based on multichannel encoding |
Publications (2)
Publication Number | Publication Date |
---|---|
EP1992198A2 true EP1992198A2 (en) | 2008-11-19 |
EP1992198B1 EP1992198B1 (en) | 2016-07-20 |
Family
ID=37452726
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP07731684.2A Active EP1992198B1 (en) | 2006-03-09 | 2007-03-01 | Optimization of binaural sound spatialization based on multichannel encoding |
Country Status (3)
Country | Link |
---|---|
US (1) | US9215544B2 (en) |
EP (1) | EP1992198B1 (en) |
WO (1) | WO2007101958A2 (en) |
Families Citing this family (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
ES2343723B2 (en) * | 2009-02-05 | 2011-05-18 | Universidad De Vigo | SYSTEM FOR THE EXPLORATION OF VIRTUAL AND REAL ENVIRONMENTS THROUGH VECTOR ACOUSTIC SPACES. |
KR20120004909A (en) * | 2010-07-07 | 2012-01-13 | 삼성전자주식회사 | Method and apparatus for 3d sound reproducing |
EP2645748A1 (en) | 2012-03-28 | 2013-10-02 | Thomson Licensing | Method and apparatus for decoding stereo loudspeaker signals from a higher-order Ambisonics audio signal |
GB201211512D0 (en) * | 2012-06-28 | 2012-08-08 | Provost Fellows Foundation Scholars And The Other Members Of Board Of The | Method and apparatus for generating an audio output comprising spartial information |
US20140081627A1 (en) * | 2012-09-14 | 2014-03-20 | Quickfilter Technologies, Llc | Method for optimization of multiple psychoacoustic effects |
US9736609B2 (en) * | 2013-02-07 | 2017-08-15 | Qualcomm Incorporated | Determining renderers for spherical harmonic coefficients |
CN108810793B (en) | 2013-04-19 | 2020-12-15 | 韩国电子通信研究院 | Multi-channel audio signal processing device and method |
WO2014171791A1 (en) | 2013-04-19 | 2014-10-23 | 한국전자통신연구원 | Apparatus and method for processing multi-channel audio signal |
US9319819B2 (en) | 2013-07-25 | 2016-04-19 | Etri | Binaural rendering method and apparatus for decoding multi channel audio |
EP3090576B1 (en) | 2014-01-03 | 2017-10-18 | Dolby Laboratories Licensing Corporation | Methods and systems for designing and applying numerically optimized binaural room impulse responses |
KR20230105002A (en) | 2015-08-25 | 2023-07-11 | 돌비 레버러토리즈 라이쎈싱 코오포레이션 | Audio encoding and decoding using presentation transform parameters |
US10142755B2 (en) * | 2016-02-18 | 2018-11-27 | Google Llc | Signal processing methods and systems for rendering audio on virtual loudspeaker arrays |
US10325610B2 (en) | 2016-03-30 | 2019-06-18 | Microsoft Technology Licensing, Llc | Adaptive audio rendering |
US10492018B1 (en) * | 2016-10-11 | 2019-11-26 | Google Llc | Symmetric binaural rendering for high-order ambisonics |
US9992602B1 (en) | 2017-01-12 | 2018-06-05 | Google Llc | Decoupled binaural rendering |
US10158963B2 (en) | 2017-01-30 | 2018-12-18 | Google Llc | Ambisonic audio with non-head tracked stereo based on head position and time |
US10009704B1 (en) | 2017-01-30 | 2018-06-26 | Google Llc | Symmetric spherical harmonic HRTF rendering |
US10764684B1 (en) | 2017-09-29 | 2020-09-01 | Katherine A. Franco | Binaural audio using an arbitrarily shaped microphone array |
DK180449B1 (en) * | 2019-10-05 | 2021-04-29 | Idun Aps | A method and system for real-time implementation of head-related transfer functions |
WO2021138517A1 (en) | 2019-12-30 | 2021-07-08 | Comhear Inc. | Method for providing a spatialized soundfield |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE68921890T2 (en) * | 1988-07-08 | 1995-07-20 | Adaptive Audio Ltd | SOUND PLAYING SYSTEMS. |
WO1994010816A1 (en) | 1992-10-29 | 1994-05-11 | Wisconsin Alumni Research Foundation | Methods and apparatus for producing directional sound |
GB9417185D0 (en) * | 1994-08-25 | 1994-10-12 | Adaptive Audio Ltd | Sounds recording and reproduction systems |
US5596644A (en) | 1994-10-27 | 1997-01-21 | Aureal Semiconductor Inc. | Method and apparatus for efficient presentation of high-quality three-dimensional audio |
US6181800B1 (en) * | 1997-03-10 | 2001-01-30 | Advanced Micro Devices, Inc. | System and method for interactive approximation of a head transfer function |
WO2000019415A2 (en) | 1998-09-25 | 2000-04-06 | Creative Technology Ltd. | Method and apparatus for three-dimensional audio display |
US7231054B1 (en) * | 1999-09-24 | 2007-06-12 | Creative Technology Ltd | Method and apparatus for three-dimensional audio display |
FR2880755A1 (en) * | 2005-01-10 | 2006-07-14 | France Telecom | METHOD AND DEVICE FOR INDIVIDUALIZING HRTFS BY MODELING |
EP1946612B1 (en) * | 2005-10-27 | 2012-11-14 | France Télécom | Hrtfs individualisation by a finite element modelling coupled with a corrective model |
-
2007
- 2007-03-01 US US12/224,840 patent/US9215544B2/en active Active
- 2007-03-01 WO PCT/FR2007/050867 patent/WO2007101958A2/en active Application Filing
- 2007-03-01 EP EP07731684.2A patent/EP1992198B1/en active Active
Non-Patent Citations (1)
Title |
---|
See references of WO2007101958A3 * |
Also Published As
Publication number | Publication date |
---|---|
WO2007101958A2 (en) | 2007-09-13 |
US9215544B2 (en) | 2015-12-15 |
US20090067636A1 (en) | 2009-03-12 |
WO2007101958A3 (en) | 2007-11-01 |
EP1992198B1 (en) | 2016-07-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP1992198B1 (en) | Optimization of binaural sound spatialization based on multichannel encoding | |
EP2000002B1 (en) | Method and device for efficient binaural sound spatialization in the transformed domain | |
EP1600042B1 (en) | Method for the treatment of compressed sound data for spatialization | |
EP1999998B1 (en) | Method for binaural synthesis taking into account a spatial effect | |
EP1563485B1 (en) | Method for processing audio data and sound acquisition device therefor | |
EP2042001B1 (en) | Binaural spatialization of compression-encoded sound data | |
EP2901718B1 (en) | Method and system for playing back an audio signal | |
WO2006075077A2 (en) | Method and device for individualizing hrtfs by modeling | |
WO2011045506A1 (en) | Processing of sound data encoded in a sub-band domain | |
EP2374124A1 (en) | Advanced encoding of multi-channel digital audio signals | |
FR2995754A1 (en) | OPTIMIZED CALIBRATION OF A MULTI-SPEAKER SOUND RESTITUTION SYSTEM | |
EP2005420A1 (en) | Device and method for encoding by principal component analysis a multichannel audio signal | |
EP3475943B1 (en) | Method for conversion and stereophonic encoding of a three-dimensional audio signal | |
EP2920979B1 (en) | Acquisition of spatialised sound data | |
FR2776461A1 (en) | METHOD FOR IMPROVING THREE-DIMENSIONAL SOUND REPRODUCTION | |
EP3025514B1 (en) | Sound spatialization with room effect | |
EP3559947A1 (en) | Processing in sub-bands of an actual ambisonic content for improved decoding | |
FR3065137A1 (en) | SOUND SPATIALIZATION METHOD | |
EP1994526B1 (en) | Joint sound synthesis and spatialization | |
EP3384688B1 (en) | Successive decompositions of audio filters | |
WO2005096268A2 (en) | Method for processing audio data, in particular in an ambiophonic context |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20080828 |
|
AK | Designated contracting states |
Kind code of ref document: A2 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC MT NL PL PT RO SE SI SK TR |
|
DAX | Request for extension of the european patent (deleted) | ||
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: ORANGE |
|
17Q | First examination report despatched |
Effective date: 20150608 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R079 Ref document number: 602007047069 Country of ref document: DE Free format text: PREVIOUS MAIN CLASS: H04S0001000000 Ipc: H04S0005000000 |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: H04S 5/00 20060101AFI20160121BHEP Ipc: H04S 1/00 20060101ALI20160121BHEP |
|
INTG | Intention to grant announced |
Effective date: 20160212 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC MT NL PL PT RO SE SI SK TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: FRENCH |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 814926 Country of ref document: AT Kind code of ref document: T Effective date: 20160815 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602007047069 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20160720 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 814926 Country of ref document: AT Kind code of ref document: T Effective date: 20160720 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160720 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160720 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160720 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160720 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161120 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 11 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161121 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160720 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160720 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160720 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161021 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160720 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160720 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602007047069 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160720 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160720 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160720 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160720 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160720 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161020 |
|
26N | No opposition filed |
Effective date: 20170421 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160720 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160720 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20170301 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 12 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20170331 Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20170331 Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20170301 |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20170331 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20170331 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160720 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20070301 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20160720 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160720 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20240220 Year of fee payment: 18 Ref country code: GB Payment date: 20240220 Year of fee payment: 18 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20240220 Year of fee payment: 18 |