EP3475943A1 - Method for conversion, stereophonic encoding, decoding and transcoding of a three-dimensional audio signal - Google Patents
Method for conversion, stereophonic encoding, decoding and transcoding of a three-dimensional audio signalInfo
- Publication number
- EP3475943A1 EP3475943A1 EP17787331.2A EP17787331A EP3475943A1 EP 3475943 A1 EP3475943 A1 EP 3475943A1 EP 17787331 A EP17787331 A EP 17787331A EP 3475943 A1 EP3475943 A1 EP 3475943A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- signal
- vector
- spherical
- complex
- phase
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 99
- 238000006243 chemical reaction Methods 0.000 title abstract description 22
- 230000005236 sound signal Effects 0.000 title description 9
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims abstract description 12
- 238000012986 modification Methods 0.000 claims abstract description 12
- 230000004048 modification Effects 0.000 claims abstract description 12
- 239000013598 vector Substances 0.000 claims description 117
- 230000008569 process Effects 0.000 claims description 19
- 238000012545 processing Methods 0.000 claims description 10
- 238000009877 rendering Methods 0.000 claims description 9
- 230000008859 change Effects 0.000 claims description 6
- 230000000750 progressive effect Effects 0.000 claims description 5
- 230000002123 temporal effect Effects 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 4
- 238000012544 monitoring process Methods 0.000 claims description 2
- 238000012546 transfer Methods 0.000 claims description 2
- JFSPBVWPKOEZCB-UHFFFAOYSA-N fenfuram Chemical compound O1C=CC(C(=O)NC=2C=CC=CC=2)=C1C JFSPBVWPKOEZCB-UHFFFAOYSA-N 0.000 claims 1
- 229920006395 saturated elastomer Polymers 0.000 claims 1
- 238000004091 panning Methods 0.000 abstract description 7
- 238000000926 separation method Methods 0.000 description 10
- 238000004458 analytical method Methods 0.000 description 9
- 239000011159 matrix material Substances 0.000 description 9
- 230000005540 biological transmission Effects 0.000 description 6
- 239000000203 mixture Substances 0.000 description 5
- 238000009472 formulation Methods 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 238000010606 normalization Methods 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000021615 conjugation Effects 0.000 description 2
- 238000000354 decomposition reaction Methods 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 238000009792 diffusion process Methods 0.000 description 2
- 238000009499 grossing Methods 0.000 description 2
- 230000004807 localization Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000012732 spatial analysis Methods 0.000 description 2
- 230000003595 spectral effect Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000013213 extrapolation Methods 0.000 description 1
- 230000004886 head movement Effects 0.000 description 1
- 238000007654 immersion Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000005304 joining Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R5/00—Stereophonic arrangements
- H04R5/033—Headphones for stereophonic communication
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R5/00—Stereophonic arrangements
- H04R5/04—Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S3/00—Systems employing more than two channels, e.g. quadraphonic
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S3/00—Systems employing more than two channels, e.g. quadraphonic
- H04S3/02—Systems employing more than two channels, e.g. quadraphonic of the matrix type, i.e. in which input signals are combined algebraically, e.g. after having been phase shifted with respect to each other
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S5/00—Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation
- H04S5/02—Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation of the pseudo four-channel type, e.g. in which rear channel signals are derived from two-channel stereo signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
- H04S2400/03—Aspects of down-mixing multi-channel audio to configurations with lower numbers of playback channels, e.g. 7.1 -> 5.1
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
- H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
- H04S2420/11—Application of ambisonics in stereophonic audio systems
Definitions
- the present invention relates to a method and method for processing the audio signal, and more particularly to a stereophonic conversion and encoding method of a three-dimensional audio signal, its decoding and its transcoding for rendering it.
- the production, transmission and reproduction of a three-dimensional audio signal is an important part of any audiovisual immersion experience, for example in the context of virtual reality content presentations, but also when viewing cinematic content or in the framework of fun applications. All three-dimensional audio content thus passes through a production or capture phase, a transmission or storage phase, and a reproduction phase.
- the phase of production or of obtaining the content can be carried out by many very widely used and used techniques: stereophonic, multichannel or peripheral capture, or synthesis of content from separate elements.
- the content is then represented either by a number of separate channels, or in the form of a perimeter sound field (for example in Ambisonics format of order 1 or higher), or even in the form of sound objects and information separated spatial
- the reproduction phase is also known and widely used in the professional or consumer fields: stereophonic or binaural headphones, stereophonic loudspeakers (optionally with transaural processing), multi-channel or three-dimensional layout.
- the transmission phase may consist of a simple transmission channel by channel, or a transmission of the separate elements and spatial information to reconstruct the content, or else an encoding allowing, most often with losses, to describe the spatial content of the original signal.
- Gerzon A critical analysis of type 4-2-4 stereophonic mastering systems (ie 4 original channels, stamped and transported on 2 channels, then decoded and reproduced on 4 channels) is provided by Gerzon in "Whither Four Channels "(Audio Annual, 1971). In “Geometry Model for Two-Channel Four-Speaker Matrix Stereo System” (JAES, 1975), Gerzon investigates and proposes several possibilities of 4-2-4 matrixing, and again describes the possibilities of describing a three-dimensional field on the energy sphere (whose principle is identical to the "Scheiber sphere"), and therefore three-dimensional encoding on two channels. This latter ability is recalled by Sommerwerck and Scheiber in "The Threat of Dolby Surround” (Multichannel Sound, Vol. 1, Nos.4 / 5, 1986).
- this approach suffers, whether during the encoding or decoding phase, a major problem of discontinuity of the phase representation: there is a spatial discontinuity of the phase with a temporally static correspondence of the phase introduced by a generic "panoramic law", introducing artifacts when a sound source is placed in certain directions of the sphere or moves on the sphere by carrying out certain trajectories.
- the present invention makes it possible to solve this discontinuity problem, and does not require separation of the incoming signal into an ambient part and a direct part.
- One of the objectives of the present invention is to disclose a method that allows, in the context of encoding to a stereophonic stream or in the context of a decoding of a stereophonic still stream, a continuity of the signal y. phase, irrespective of the source position and whatever the trajectory it describes, without requiring any non-directional component in the input signal, or matrix encoding of the signal, or compromise between stability and location accuracy for extreme positions in the interchannel domain.
- Another object of the present invention is to provide decoding and transcoding from a stereophonic signal, optionally encoded with one of the implementations of the invention, or encoded with the existing systems of matrix encoding, and from make a return on any means of broadcasting and in any audio format, without compromising stability and location accuracy.
- Another object of the present invention is to provide a complete chain of transport or storage of a three-dimensional acoustic field, in a compact format and accepted by the standard means of transport or storage, while retaining the relevant three-dimensional spatial information. from the original field.
- Figure 1 shows the Scheiber sphere (also called Stokes-Poincaré sphere or energy sphere) as defined, for example, in “Analyzing Phase-Amplitude Matrices", Journal of the Audio Engineering Society, Vol. 19, No. 10, p. 835 (November 1971).
- Scheiber sphere also called Stokes-Poincaré sphere or energy sphere
- Figure 2 illustrates in the form of a panoramic-phase map an example of an arbitrary phase matching choice.
- FIG. 3 gives an example of a partial map of phase correspondence providing a continuity between the edges of the panoramic-phase definition domain.
- FIG. 4 illustrates the principle of folding of the correspondence card of FIG. 2 on the Scheiber sphere of FIG. 1.
- Figure 5 illustrates the folding of Figure 4, once it is fully performed.
- Figure 6 shows the Scheiber sphere on which is present a vector field corresponding to the complex frequency coefficient c L local.
- the sum of the indices with the authorized singularities, in L, or the cancellation of the vector field, in R is different from 2, expected value if it were possible not to to have other singularity on the sphere.
- the left and right boxes are presented the possible local structures of the vector field in the vicinity of the singularities of the points L and R, with their respective indices.
- phase correspondence described by this map is continuous at all points except ⁇ .
- Figure 8 shows the map of Figure 7 after its folding on the Scheiber sphere.
- FIG. 9 illustrates the phase correspondence map for a singularity positioned in ⁇ of panorama coordinates and phase difference (-1 / 4, -3 ⁇ / 4).
- Figure 10 shows the map of Figure 9 after its folding on the Scheiber sphere.
- Figure 11 shows the diagram of the encoding process, converting a signal from the spherical domain to the interchannel domain.
- Figure 12 shows the diagram of the decoding process, converting a signal from the interchannel domain to the spherical domain.
- Figure 13 illustrates the process of deformation of the spherical space according to the azimuth values.
- the operator atan2 (y, x) which is the operator which gives the oriented angle between a vector (1,0) T and a vector (x, y) T ; this operator is available as a std :: atan2 function of the C ++ STL library.
- two channels in time form for example forming a stereophonic signal, can be transformed to the frequency domain into two tables of complex coefficients.
- the complex frequency coefficients of the two channels may be paired, so as to have one pair for each frequency or frequency band among a plurality of frequencies, and for each time window of the signal.
- Each pair of complex frequency coefficients can be analyzed using two metrics, combining information from two stereophonic channels, which are introduced below: the panorama and the phase difference, which form what we will name in the remainder of this document the "interchannel domain".
- the panorama of two complex frequency coefficients c 1 and c 2 is defined as the ratio between the difference of their powers and the sum of their powers:
- the panorama thus takes values in the interval [-1,1]. If the two coefficients are simultaneously of zero magnitude, there is no signal in the frequency band they represent, and the use of the panorama is not relevant.
- a variant of the panorama formulation is as follows:
- phase difference between two complex frequency coefficients c 1 and c 2 is further defined as follows:
- a unit vector of azimuth and elevation a and e will have Cartesian coordinates:
- a signal expressed in the form of a "First Order Ambisonics" (FOA) field is composed of four corresponding channels W, X, Y, Z, pressure and pressure gradient at a point in the space following each direction:
- FOA First Order Ambisonics
- the X channel is the signal of the pressure gradient at the point along the X axis
- the Y channel is the signal of the pressure gradient at the point along the Y axis
- the Z channel is the signal of the pressure gradient at the point along the Z axis
- a normalization standard for spherical harmonics can be defined as follows: a monochromatic progressive plane wave (OPPM) of complex frequency component c and direction of origin the unit vector Cartesian coordinates (v x , v y , v z ) or azimuth and elevation coordinates (a, e) will generate for each channel a coefficient of the same phase but of altered magnitude:
- OPPM monochromatic progressive plane wave
- a preferred implementation of the invention comprises a first method of converting such an FOA field into complex coefficients and spherical coordinates.
- This first method allows a loss-based, perceptually based conversion of the FOA field to a format composed of complex frequency coefficients and their spatial correspondence in azimuth and elevation coordinates (or a Cartesian vector of unit norm).
- Said method is based on a frequency representation of the FOA signals obtained after temporal windowing and time-to-frequency transform, for example via the use of the short-term Fourier Transform (STFT). .
- STFT short-term Fourier Transform
- the following method is applied to each group of four complex coefficients corresponding to a frequency "bin", that is to say the complex coefficients of the frequency representation of each of the channels W, X, Y, Z which correspond to the same frequency band, and this for any frequency or frequency band among a plurality of frequencies.
- a frequency bin that is to say the complex coefficients of the frequency representation of each of the channels W, X, Y, Z which correspond to the same frequency band, and this for any frequency or frequency band among a plurality of frequencies.
- An exception is made for the frequency bin (s) corresponding to the DC component (because of the "padding" applied to the time-to-frequency transformed forward signal, the following few frequency bins may also be concerned).
- a part A corresponding to a monochromatic progressive plane wave (OPPM), directional,
- I is an average intensity three-dimensional vector, directed towards the origin of ⁇ , of magnitude proportional to the square of the magnitude of ⁇ ,
- I e is three-dimensional vector composed of complex coefficients corresponding to respective pressure gradients along the axis X, Y, and Z, that is to say
- ⁇ the operator is the conjugation operator of the complex components of the vector.
- e w is the reintroduction elevation of w, in [- ⁇ / 2,0], chosen by the user, and by default set to- ⁇ / 2.
- r x, r y, r and z are vectors depending on the frequency or frequency band, described hereafter.
- the separated parts A, B, and C are grouped into a direction vector of provenance and a complex coefficient c tota i:
- ⁇ ⁇ , and ⁇ ⁇ ⁇ p z are phases which will be defined later in this document.
- the first conversion method presented above does not consider any divergence character that can be introduced during the FOA pan.
- a second preferred implementation makes it possible to consider the divergence character.
- the unit vector of direction is calculated as
- the complex frequency coefficient is meanwhile:
- the separated portions A and C are ultimately combined into a source direction vector and a complex coefficient c i tota:
- vectors and phases have the responsibility to establish a diffuse character to the signal whose direction they give and whose phase they modify. They depend on the frequency band being processed, that is, there is a set of vectors and phase for each frequency "bin". In order to establish this diffuse character, they come from a random process, which allows them to be smoothed spectrally, as well as temporally if it is desired that they be dynamic.
- o the unit vectors are generated from an azimuth resulting from a pseudo-random generator of uniform reals in] - ⁇ , ⁇ ] and from an elevation resulting from the arcsine of a real of a pseudo-generator uniform random in [-1,1]; o the phases are obtained using a pseudo-random generator of uniform reals in] - ⁇ , ⁇ ].
- ⁇ is the frequency equivalent of a characteristic time, allowing the user to choose the spectral smoothing of the diffuse character; a possible value for a sampling frequency of 48 kHz, a window size of 2048 and a padding of 100% is 0.65.
- the lowest frequency vectors for example those corresponding to frequencies below 150 Hz are modified to be directed to a preferred direction, for example and preferably (1.0, 0) T. To do this, the generation of random vectors is modi
- the spectral smoothing for obtaining the vectors is unchanged.
- phases ⁇ ⁇ , ⁇ ⁇ and ⁇ ⁇ can be determined by impulse response measurements: it is possible to obtain them by the analysis of complex frequency coefficients resulting from multiple sound captures of the spherical field of the first order, using signals emitted by loudspeakers, in phase around the measuring point for both sides and out of phase
- the processing is distinct. Note that because of the padding, the continuous regime corresponds to one or more frequency (s) or frequency band (s):
- the first two frequencies or frequency bands are applied the treatment as defined below (as well as the frequency or "negative" frequency band that is conjugate-symmetrical with the second frequency or frequency band);
- the first four frequencies or frequency bands are applied the processing as defined below (as well as frequencies or "negative" frequency bands that are conjugate-symmetric with the second, third and fourth frequencies or frequency bands);
- This (or these) frequency (s) or frequency band (s) are real value and not complex, which does not allow to know the phase of the signal for the corresponding frequencies; direction analysis is not possible.
- a human being can not perceive a direction of origin for the low frequencies concerned (those below 80 to 100 Hz, in this case). It is thus possible to analyze only the pressure wave, therefore the coefficient c w , and to choose an arbitrary source direction, frontal: (1,0, 0) T.
- the representation in the spherical domain of the first (or more) bin (s) frequency (s) is:
- the Scheiber sphere corresponding in the field of optics, to the Stokes-Poincaré sphere, is used in what follows.
- the Scheiber sphere symbolically represents the magnitude and phase relationships of two monochromatic waves, that is, also two complex frequency coefficients representing these waves. It consists of semicircles joining the opposing points L and R, each semicircle being derived from a rotation about the axis LR of the frontal arc in bold of an angle ⁇ and representing a difference value of phase ⁇ E] - ⁇ , ⁇ ].
- the frontal semicircle represents a zero phase difference.
- Each point of the semicircle represents a distinct value of panorama, with a value close to 1 for points close to L, and a value close to -1 for points close to R.
- Figure 1 illustrates the principle of the Scheiber sphere.
- the sphere of Scheiber (100) symbolically represents the magnitude and phase relationships with points on a sphere of two monochromatic waves, that is to say also of two complex frequency coefficients representing these waves, in the form of semicircles of equal phase difference and indexed on the panorama.
- Peter Scheiber has established in "Analyzing Phase-Amplitude Matrices" (QAES, 1971) that it is possible to match this sphere, symbolically constructed, with the sphere of physical positions of sound sources, allowing a spherical encoding of sound sources.
- the axis LR (101, 102) becomes the Y axis (103), the X axis (105) pointing in the direction of the half-circle (104) of zero phase difference.
- the coordinate system of the Scheiber sphere is spherical with polar axis Y, and we can express the coordinates in X, Y, Z according to the panorama and the phase difference:
- the objective is to achieve a fully determined correspondence between a pair of complex frequency coefficients (interchannel domain) on the one hand and a complex frequency coefficient and spherical coordinates of other (spherical domain).
- phase of a coefficient in the spherical domain is a function of the position in the interchannel domain (panorama, phasediff), as well as the absolute phase of said coefficients (which will be represented by an intermediate phase value, as will be seen later).
- the pairs of coefficients are represented locally, so the map represents a field of pairs of complex coefficients.
- the choice of a phase match corresponds to the local rotation of the complex plane containing the pair of complex frequency coefficients. It can be seen that the map is a two-dimensional representation of the Scheiber sphere, to which phase information is added.
- FIG. 2 illustrates an example of a map (200) of phase correspondence between the spherical domain and the interchannel domain, representing, for different measurements of abscissa panorama (201) and of the ordinate phase difference (202), a choice of arbitrary phase matching which is simply the subtraction of the phase difference half for the L channel and the addition of half of the phase difference for the R channel.
- the abscissa axis (201) is inverted so that the Left lateral positions correspond to a preponderant power signal in the L channel and respectively for the right side and the R channel.
- the ordinate axis (201) is also inverted for the positive elevation hemisphere, ie the upper half of the figure. .
- the field of complex coefficient pairs is represented in sections of complex planes around the origin; in each reference, the complex frequency coefficient c L is represented by a vector whose vertex is a circle, the complex frequency coefficient c R is represented by a vector whose vertex is a cross.
- This phase match card is not usable because it contravenes the principles outlined later.
- the criterion chosen for the design of a correspondence is that of the spatial continuity of the phase of the signal, that is to say that a minute change of position of a sound source must result in a minute change of the phase .
- the phase continuity criterion imposes constraints for a phase matching at the edges of the domain:
- the set of values to the left of the domain corresponds to the neighborhood of the point L (respectively of the point R) of the sphere of locations.
- the phase of the complex frequency coefficient with the greatest magnitude must be constant.
- the phase of the complex frequency coefficient having the smallest magnitude is then imposed by the phase difference; it performs a rotation of 2 ⁇ when a curve is traversed around the points L or R of the sphere but it is not problematic because the magnitude vanishes at the point of phase discontinuity, arising on a continuity of the complex frequency coefficient.
- Figure 3 gives an example of a phase match that can be constructed from these constraints to provide phase continuity at the edges of the board (300). The constancy of the phase value is ensured on each of the lateral edges, and there is equality of the values by the correspondence of the top and the bottom of the domain. This solution is not unique, other correspondence cards are possible.
- Fig. 4 illustrates how the two-dimensional map (200) of Fig. 2 is folded over the Scheiber sphere (100) of Fig. 1.
- the directions of the local landmarks are maintained by folding; the local landmarks thus have their continuous direction on the sphere, except at the points L and R, but this is not a problem because the continuity of phase is already ensured at these points. It is thus obtained, for a map, two fields of complex coefficients. These complex coefficients correspond to vectors tangent to the sphere, except at the points L and R.
- the card (200) once folded in full as illustrated in FIG. 5, presents on the rear arc (in continuous continuous pattern) (500) a phase discontinuity, which discontinuity is solved by the method illustrated in FIG.
- the sum of the indices of the zeros isolated from the vector field is equal to the Euler-Poincaré characteristic of the surface.
- a vector field on a sphere has an Euler-Poincaré characteristic of 2.
- the vector field from c L vanishes at R with a 0 or 2 index and vanishes. by the modification around L with an index 1 as can be seen in figure 6.
- the sum of the indices is therefore odd, and this imposes at least another zero in the vector field, of adequate index so that the sum indices equal to the characteristic of Euler-Poincaré.
- the method disclosed in the present invention solves this problem of phase continuity. It is based on the observation that in real cases the whole sphere is not in- completely and simultaneously traversed by signals. A phase matching discontinuity localized at a point in the sphere traversed by signals (fixed signals or spatial signal trajectories) will cause a phase discontinuity. A phase matching discontinuity localized at a point in the sphere not traversed by signals (fixed signals or spatial signal trajectories) does not cause phase discontinuity. Without prior knowledge of the signals, a discontinuity at a fixed point can not guarantee that no signal will pass through this point. A discontinuity in a moving point may instead "avoid" to be traversed by a signal, if its location is a function of the signal.
- This point of moving discontinuity may be part of a dynamic phase match that is continuous on any other point of the sphere.
- the principle of dynamic phase matching based on the avoidance of the spatial location of the signal by the discontinuity is thus established. We will establish such a phase match based on this principle, other phase matches being possible.
- phase matching function ⁇ panorama, phasediff
- ⁇ p s is the phase of the complex frequency coefficient of the spherical domain
- ⁇ is the intermediate phase of the interchannel domain
- phase matching function is dynamic, i.e. it varies from one time window to the next.
- ⁇ (panorama Singu i ar i ty, i phasediff Singu ar i ty) of interchannel area defined by a panorama panorama value Singu i ar i ty [-1 / 2,1 / 2] and phase difference phasediff Singu i ar i ty] -7 ⁇ , - ⁇ / 2].
- This corresponds to an area at the back of the listener, slightly in height. It is possible to arbitrarily choose other areas.
- the singularity is initially located in the center of this zone, at a position ⁇ 0 which is called “anchor” thereafter. It is possible to arbitrarily choose other initial locations of the anchor within the area.
- the index of the phase-matching function is the choice of panorama and phase difference corresponding to the singularity.
- the point of the singularity ⁇ is situated, spatially speaking, near a signal, it is moved in the zone in order to "leak" the signal localization, treatment window after treatment window.
- all the frequency bands are analyzed in order to determine their respective location of panorama and phase difference in the interchannel domain, and for each a modification vector is calculated, intended to move the point of singularity.
- the change from a frequency band can be calculated as follows:
- N is the number of frequency bands and the distance between the point ⁇ and the coordinate point (panorama, phasediff), if d ⁇ 0, 0 otherwise, and
- FIG. 9 represents the phase correspondence map if ⁇ has for panorama and phase difference coordinates (-1 / 4, - 3 ⁇ / 4).
- the phase correspondence described by this map is continuous everywhere except in ⁇ .
- Figure 10 shows the phase match map of Figure 9, when folded over the Scheiber sphere.
- a spherical domain signal is characterized for any frequency or frequency band by azimuth and elevation, magnitude, and phase.
- Implementations of the present invention include transcoding means from the spherical domain to a given audio format selected by the user. Some techniques are presented by way of example but their adaptation to other audio formats will be trivial for a person who knows the state of the art of the sound reproduction or the encoding of the sound signal. Transcoding in spherical harmonics of the first order (or First-Order Ambisonic, FOA) can be performed in the frequency domain. For each complex coefficient c corresponding to a frequency band, knowing the azimuth a and the corresponding elevation e, four complex coefficients w, x, y, z corresponding to the same frequency band can be generated by the following formulas:
- the coefficients w, x, y, z obtained for each frequency band are assembled to respectively generate frequency representations W, X, Y, and Z of four channels, and the application of the frequency-to-time transform (inverse of the one used for the time-to-frequency transform), the possible windowing, then the overlap of the successive time windows obtained makes it possible to obtain four channels which are a temporal representation in spatial harmonics of the first order of the three-dimensional audio signal.
- a similar approach can be used for transcoding to a format (HOA) of order greater than or equal to 2, completing equation (54) with the encoding formulas for the order in question.
- Transcoding to 5.0 surround format with five left, center, right, left rear and right rear channels can be performed as follows.
- the coefficients c L , c C , c R , c Ls , c Rs respectively corresponding to the loudspeakers usually named L, C, R, Ls, Rs are calculated as follows, starting from azimuth and elevation coordinates a and e of the direction vector of provenance and the complex frequency coefficient c s .
- the gains g L , g C , g R , g Ls , g Rs are defined as the gains to be applied to the coefficient c s to obtain the complex frequency coefficients of the output coefficient tables, as well as two gains g B and g T corresponding to virtual speakers allowing a redistribution of the signals down ("Bottom"), that is to say at negative elevation, and at the top (“Top”), that is to say at elevation positive, to the other speakers.
- the six complex coefficients thus obtained for each frequency band are assembled to respectively generate frequency representations of six L, C, R, Ls, Rs and T channels, and the application of the frequency-to-time transform (inverse of that used for the time-to-frequency transform), the possible windowing, then the overlapping of the successive time windows obtained makes it possible to obtain six channels in the time domain.
- Transcoding a signal expressed in the spherical domain to a binaural format may also be performed. It can for example be based on the following elements:
- a database including, for a plurality of frequencies, for a plurality of directions in space, and for each ear, the expression in complex coefficients (magnitude and phase) of the Head-Related Transfer Function (HRTF) filters in the frequency domain;
- HRTF Head-Related Transfer Function
- This interpolation can be carried out bilinearly or spline, or by means of spherical harmonic functions.
- a plurality of functions is thus obtained on the unit sphere, for any frequency, describing the frequency behavior of said HRTF database for any point of the spherical space. Since, for any frequency among a plurality of frequencies, it is established that said spherical signal is described by a direction of origin (azimuth, elevation) and a complex coefficient (magnitude, phase), said interpolation-projection makes it possible to perform binauralization operation of the spherical signal, as follows:
- said complex coefficient HRTF is then multiplied by the complex coefficient corresponding to the spherical signal, resulting in a left ear frequency signal and a right ear frequency signal;
- a frequency-to-time transform is then performed, giving a two-channel binaural signal.
- spherical harmonic formats are often used as intermediate formats before decoding on speaker constellations or binaural decoding.
- Multi-channel formats obtained via VBAP rendering are also likely to be binauralised.
- Other types of transcoding can be obtained by the use of usual spatialization techniques such as panning pair-wise with or without horizontal layers, SPCAP, VBIP or WFS.
- This method allows a perceptual gain of precision of location of sound sources in space; this is a phenomenon known in the field of psychoacoustics: small head movements allow the human auditory device to perform a better localization of sound sources.
- the spherical signal consists of temporally successive tables each corresponding to a representation on a time window of the signal, these windows overlapping each other.
- Each array consists of pairs (complex frequency coefficient, coordinates on the sphere in azimuth and elevation), each pair corresponding to a frequency band.
- the original spherical signal is obtained from spatial analysis techniques such as the one presented which transforms an FOA signal into a spherical signal.
- Encoding makes it possible to obtain temporally successive pairs of tables of complex frequency coefficients, each array corresponding to a channel, for example left (L) and right (R).
- Figure 11 shows the diagram of the encoding process, converting from the spherical domain to the interchannel domain.
- the sequence of the encoding technique for each time window successively processed, is thus illustrated:
- a first step (1100) consists in determining for each element of the input array the panorama and the phase difference corresponding to each spherical coordinate, as indicated in equations 43.
- Optionally widening the azimuth from the interval [ -30 °, 30 °] to the interval [-90 °, 90 °] can be performed according to the method indicated above, before the determination of the panorama and the phase difference, this widening corresponding to the operation (1302) of Figure 13.
- a second step (1101) consists in determining the new position of the singularity in the interchannel domain, by analyzing the coordinates of panorama and phase difference determined in the first step.
- a third step (1102) consists of determining the phase correspondence ⁇ ⁇ (panorama, phasediff) for each complex coefficient of the input array,
- a fourth step (1103) consists in constructing an array of pairs of complex coefficients c L and c R , according to the complex frequency coefficients of the spherical domain ⁇ 3 ⁇ 4, the calculated values of panorama and phase difference, and the function of phase difference:
- Equation 5 An alternative technique for determining the magnitude of complex frequency coefficients is presented in Equation 5.
- the decoding of a stereo signal encoded with the technique presented above can be performed in the following manner. Since the input signal is in the form of a pair of generally temporal channels, a transformation such as the short-term Fourier transform is used to obtain temporally successive pairs of tables of complex frequency coefficients, each coefficient of each table corresponding to a frequency band. In each pair of tables corresponding to a time window, the coefficients corresponding to the same frequency band are paired.
- the decoding makes it possible to obtain for each time window a spherical representation of the signal, in the form of an array of pairs (complex frequency coefficient, coordinates on the sphere in azimuth and elevation).
- Figure 12 the sequence of the decoding technique for each time window successively processed
- a first step (1200) consists in determining the panorama and the phase difference for each pair, as indicated in equations 2 or 4, and 6.
- a second step (1201) consists in determining the new position of the singularity ⁇ in the inter-channel domain, by analyzing the coordinates of panorama and phase difference determined in the first step.
- a third step (1202) consists in determining the phase correspondence ⁇ ⁇ (panorama, phasediff) for each complex coefficient of the input array, from the results of the first and second stages.
- a fourth step (1203) consists in determining, from the results of the first (1200) and third (1202) stages, the complex frequency coefficient ⁇ 3 ⁇ 4 in the spherical domain:
- ⁇ is the intermediate phase, obtained for example with: phasediff.
- a fifth step (1204) consists in determining, from the results of the first step (1200), the azimuth and elevation coordinates as indicated in equations 41.
- the azimuth constriction from the interval [ -90 °, 90 °] to the interval [-30 °, 30 °] can be performed, according to the method indicated above, this step corresponding to the operation (1301) of Figure 13.
- stereo contents are encoded in surround form with a mastering technique, and the coordinates of the mastering points are generally positioned in the inter-channel domain at consistent positions, the decoding of such surround contents works, with some absolute positioning defects. sources.
- stereo contents not intended to be played on a device other than a pair of speakers take advantage of being processed by the decoding process, resulting in a "upmix" 2D or 3D content, the term “ upmix "corresponding to processing a signal to be able to broadcast on devices with a number of speakers greater than the number of original channels, each speaker receiving a signal of its own, or its equivalent virtualized helmet.
- the stereophonic signal resulting from the encoding of a three-dimensional audio field can be reproduced properly without decoding on a standard stereo listening device, for example headphones, sound bar or stereophonic system.
- the signal can also be processed by multichannel decoders of commercially available matrix surround content without audible artifacts appearing.
- the decoder of the invention is versatile: it makes it possible both to decode specially encoded contents for it, to decode in a relatively satisfactory manner pre-existing contents in the matrixed surround format (for example cinematographic sound contents), as well as to upmixing stereo content.
- a software or hardware for example in the form of a chip
- any system dedicated to sound broadcasting television, stereophonic high-fidelity channel, home or home theater amplifier, system embedded audio in a vehicle, equipped with a multichannel broadcast system, or even any system broadcasting for headphone listening, via binaural rendering, possibly with monitoring of the headtracking, such as a computer, a mobile phone, a digital audio player.
- a crosstalk canceling listening device also allows binaural listening without headphones from at least two speakers, and allows listening surround or 3D sound content decoded by the invention and rendered in binaural.
- the decoding algorithm presented in the present invention makes it possible to rotate the sound space on the direction vectors from which the spherical field obtained is obtained, the direction of provenance being that which would be perceived by a listener located in the center. of the said sphere; this ability makes it possible to implement the tracking of the head of the listener (or "head-tracking") in the processing chain as close as possible to its rendering, an important element to reduce the latency between the movements of the head and their compensation in the audible signal.
- An audio headset in itself can embark the decoding system presented in an implementation of the present invention, possibly adding head-tracking and binaural rendering functions.
- stereo audio connectors for example stereo audio connectors, stereophonic digital codecs such as MPEG-2 layer 3 or AAC, radio broadcasting techniques Stereo FM or DAB, or broadcast, cable or IP stereo stereophonic broadcast standards.
- stereophonic digital codecs such as MPEG-2 layer 3 or AAC
- radio broadcasting techniques Stereo FM or DAB or broadcast, cable or IP stereo stereophonic broadcast standards.
- the encoding in the format presented in this invention is performed at the end of "mastering" (finalization) multichannel or 3D, from a FOA field via a conversion to a spherical field such as one of those presented in this document or another technique.
- the encoding can also be performed on each source added to the sound mix, independently of each other, using spatialization or panning tools embodying the described method, which makes it possible to perform 3D mixing on stations digital audio workstations supporting only 2 channels.
- This encoded format can also be stored or archived on any medium comprising only two channels, or for the purpose of size compression.
- the decoding algorithm makes it possible to obtain a spherical field, which can be altered, by removing the spherical coordinates and keeping only the complex frequency coefficients, in order to obtain a mono "downmix".
- This method can be implemented in software or hardware for embedding in an electronic chip, embedded for example in monophonic FM listening devices.
- the contents of video games and virtual reality or augmented reality systems can be stored in stereo encoded form, and then decoded to be re-spatialised by transcoding, for example as a FOA field.
- the availability of the direction vectors of provenance also makes it possible to manipulate the sound field by means of geometrical operations, allowing for example zooms, distortions according to the sound environment such as by the projection of the sphere of the directions on the inside a piece of a video game, then parallax deformation of the vectors of direction of origin.
- a video game or other virtual reality or augmented reality system having as an internal sound format a surround or 3D audio format may also encode its content before broadcast; accordingly, if the listener's final listening device implemented the decoding method disclosed in the present invention, it thus provides a three-dimensional spatialization, and if the device is a headphones that implement head-tracking, binaural customization and head-tracking enable dynamic immersive listening.
- implementations of the present invention may be implemented in the form of one or more computer programs, said computer programs operating on at least one computer or on at least one onboard signal processing circuit, locally, remote or distributed (for example as part of a cloud-like infrastructure).
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Spectroscopy & Molecular Physics (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Pure & Applied Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Algebra (AREA)
- Stereophonic System (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
MC2624A MC200186B1 (en) | 2016-09-30 | 2016-09-30 | Method for conversion, stereo encoding, decoding and transcoding of a three-dimensional audio signal |
PCT/EP2017/025274 WO2018059742A1 (en) | 2016-09-30 | 2017-09-28 | Method for conversion, stereophonic encoding, decoding and transcoding of a three-dimensional audio signal |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3475943A1 true EP3475943A1 (en) | 2019-05-01 |
EP3475943B1 EP3475943B1 (en) | 2021-12-01 |
Family
ID=60153256
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP17787331.2A Active EP3475943B1 (en) | 2016-09-30 | 2017-09-28 | Method for conversion and stereophonic encoding of a three-dimensional audio signal |
Country Status (5)
Country | Link |
---|---|
US (1) | US11232802B2 (en) |
EP (1) | EP3475943B1 (en) |
CN (1) | CN109791768B (en) |
MC (1) | MC200186B1 (en) |
WO (1) | WO2018059742A1 (en) |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
TWI703557B (en) * | 2017-10-18 | 2020-09-01 | 宏達國際電子股份有限公司 | Sound reproducing method, apparatus and non-transitory computer readable storage medium thereof |
CN110493701B (en) * | 2019-07-16 | 2020-10-27 | 西北工业大学 | HRTF (head related transfer function) personalization method based on sparse principal component analysis |
CN110751956B (en) * | 2019-09-17 | 2022-04-26 | 北京时代拓灵科技有限公司 | Immersive audio rendering method and system |
CN113449255B (en) * | 2021-06-15 | 2022-11-11 | 电子科技大学 | Improved method and device for estimating phase angle of environmental component under sparse constraint and storage medium |
CN115497485A (en) * | 2021-06-18 | 2022-12-20 | 华为技术有限公司 | Three-dimensional audio signal coding method, device, coder and system |
US11910177B2 (en) * | 2022-01-13 | 2024-02-20 | Bose Corporation | Object-based audio conversion |
CN114994608B (en) * | 2022-04-21 | 2024-05-14 | 西北工业大学深圳研究院 | Multi-device self-organizing microphone array sound source positioning method based on deep learning |
Family Cites Families (44)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US3632886A (en) | 1969-12-29 | 1972-01-04 | Peter Scheiber | Quadrasonic sound system |
US4334740A (en) | 1978-09-12 | 1982-06-15 | Polaroid Corporation | Receiving system having pre-selected directional response |
US4696036A (en) | 1985-09-12 | 1987-09-22 | Shure Brothers, Inc. | Directional enhancement circuit |
US4862502A (en) | 1988-01-06 | 1989-08-29 | Lexicon, Inc. | Sound reproduction |
US5136650A (en) | 1991-01-09 | 1992-08-04 | Lexicon, Inc. | Sound reproduction |
US5664021A (en) | 1993-10-05 | 1997-09-02 | Picturetel Corporation | Microphone system for teleconferencing system |
IT1283803B1 (en) | 1996-08-13 | 1998-04-30 | Luca Gubert Finsterle | TWO-CHANNEL SOUND RECORDING SYSTEM AND SOUND REPRODUCTION SYSTEM THROUGH AT LEAST FOUR SPEAKERS WITH |
US6041127A (en) | 1997-04-03 | 2000-03-21 | Lucent Technologies Inc. | Steerable and variable first-order differential microphone array |
US6507659B1 (en) | 1999-01-25 | 2003-01-14 | Cascade Audio, Inc. | Microphone apparatus for producing signals for surround reproduction |
WO2001076319A2 (en) | 2000-03-31 | 2001-10-11 | Clarity, L.L.C. | Method and apparatus for voice signal extraction |
EP2299735B1 (en) | 2000-07-19 | 2014-04-23 | Koninklijke Philips N.V. | Multi-channel stereo-converter for deriving a stereo surround and/or audio center signal |
EP1184676B1 (en) | 2000-09-02 | 2004-05-06 | Nokia Corporation | System and method for processing a signal being emitted from a target signal source into a noisy environment |
EP1356706A2 (en) | 2000-09-29 | 2003-10-29 | Knowles Electronics, LLC | Second order microphone array |
GB2373154B (en) * | 2001-01-29 | 2005-04-20 | Hewlett Packard Co | Audio user interface with mutable synthesised sound sources |
WO2007106399A2 (en) | 2006-03-10 | 2007-09-20 | Mh Acoustics, Llc | Noise-reducing directional microphone array |
JP5280837B2 (en) | 2005-03-22 | 2013-09-04 | ブルームライン アコースティックス ベースローテン フェンノートシャップ | Transducer device for improving the naturalness of speech |
US20060222187A1 (en) | 2005-04-01 | 2006-10-05 | Scott Jarrett | Microphone and sound image processing system |
FI20055260A0 (en) | 2005-05-27 | 2005-05-27 | Midas Studios Avoin Yhtioe | Apparatus, system and method for receiving or reproducing acoustic signals |
FI20055261A0 (en) | 2005-05-27 | 2005-05-27 | Midas Studios Avoin Yhtioe | An acoustic transducer assembly, system and method for receiving or reproducing acoustic signals |
EP1737265A1 (en) | 2005-06-23 | 2006-12-27 | AKG Acoustics GmbH | Determination of the position of sound sources |
US20070237340A1 (en) | 2006-04-10 | 2007-10-11 | Edwin Pfanzagl-Cardone | Microphone for Surround-Recording |
US8712061B2 (en) | 2006-05-17 | 2014-04-29 | Creative Technology Ltd | Phase-amplitude 3-D stereo encoder and decoder |
US8345899B2 (en) | 2006-05-17 | 2013-01-01 | Creative Technology Ltd | Phase-amplitude matrixed surround decoder |
GB0619825D0 (en) | 2006-10-06 | 2006-11-15 | Craven Peter G | Microphone array |
JP2010506211A (en) * | 2006-10-06 | 2010-02-25 | ラリー ブラザーズ サイエンティフィック、エルエルシー | Three-dimensional internal backprojection system and method of using the system |
FR2908586B1 (en) | 2006-11-10 | 2011-05-13 | Huyssen Antoine Victor Hurtado | DEVICE FOR CONVERTING A STEREO AUDIO SIGNAL TO A MULTICANAL AUDIO SIGNAL |
DK2121944T3 (en) | 2007-01-19 | 2011-10-31 | Probiodrug Ag | In vivo screening models for the treatment of Alzheimer's disease and other QPCT-related disorders |
US8229134B2 (en) | 2007-05-24 | 2012-07-24 | University Of Maryland | Audio camera using microphone arrays for real time capture of audio images and method for jointly processing the audio images with video images |
GB2467668B (en) | 2007-10-03 | 2011-12-07 | Creative Tech Ltd | Spatial audio analysis and synthesis for binaural reproduction and format conversion |
WO2009046460A2 (en) | 2007-10-04 | 2009-04-09 | Creative Technology Ltd | Phase-amplitude 3-d stereo encoder and decoder |
WO2010076460A1 (en) | 2008-12-15 | 2010-07-08 | France Telecom | Advanced encoding of multi-channel digital audio signals |
US8705750B2 (en) * | 2009-06-25 | 2014-04-22 | Berges Allmenndigitale Rådgivningstjeneste | Device and method for converting spatial audio signal |
EP2346028A1 (en) | 2009-12-17 | 2011-07-20 | Fraunhofer-Gesellschaft zur Förderung der Angewandten Forschung e.V. | An apparatus and a method for converting a first parametric spatial audio signal into a second parametric spatial audio signal |
EP2469741A1 (en) * | 2010-12-21 | 2012-06-27 | Thomson Licensing | Method and apparatus for encoding and decoding successive frames of an ambisonics representation of a 2- or 3-dimensional sound field |
EP2541547A1 (en) * | 2011-06-30 | 2013-01-02 | Thomson Licensing | Method and apparatus for changing the relative positions of sound objects contained within a higher-order ambisonics representation |
US9232310B2 (en) | 2012-10-15 | 2016-01-05 | Nokia Technologies Oy | Methods, apparatuses and computer program products for facilitating directional audio capture with multiple microphones |
FR2998438A1 (en) | 2012-11-16 | 2014-05-23 | France Telecom | ACQUISITION OF SPATIALIZED SOUND DATA |
US9525938B2 (en) | 2013-02-06 | 2016-12-20 | Apple Inc. | User voice location estimation for adjusting portable device beamforming settings |
US9685163B2 (en) * | 2013-03-01 | 2017-06-20 | Qualcomm Incorporated | Transforming spherical harmonic coefficients |
US9516412B2 (en) | 2014-03-28 | 2016-12-06 | Panasonic Intellectual Property Management Co., Ltd. | Directivity control apparatus, directivity control method, storage medium and directivity control system |
US9838819B2 (en) * | 2014-07-02 | 2017-12-05 | Qualcomm Incorporated | Reducing correlation between higher order ambisonic (HOA) background channels |
EP3007167A1 (en) * | 2014-10-10 | 2016-04-13 | Thomson Licensing | Method and apparatus for low bit rate compression of a Higher Order Ambisonics HOA signal representation of a sound field |
JP6539846B2 (en) | 2015-07-27 | 2019-07-10 | 株式会社オーディオテクニカ | Microphone and microphone device |
EP3804356A1 (en) | 2018-06-01 | 2021-04-14 | Shure Acquisition Holdings, Inc. | Pattern-forming microphone array |
-
2016
- 2016-09-30 MC MC2624A patent/MC200186B1/en unknown
-
2017
- 2017-09-28 US US16/333,433 patent/US11232802B2/en active Active
- 2017-09-28 EP EP17787331.2A patent/EP3475943B1/en active Active
- 2017-09-28 CN CN201780051834.7A patent/CN109791768B/en active Active
- 2017-09-28 WO PCT/EP2017/025274 patent/WO2018059742A1/en unknown
Also Published As
Publication number | Publication date |
---|---|
MC200186B1 (en) | 2017-10-18 |
CN109791768A (en) | 2019-05-21 |
US20200168235A1 (en) | 2020-05-28 |
WO2018059742A1 (en) | 2018-04-05 |
CN109791768B (en) | 2023-11-07 |
US11232802B2 (en) | 2022-01-25 |
EP3475943B1 (en) | 2021-12-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3475943B1 (en) | Method for conversion and stereophonic encoding of a three-dimensional audio signal | |
ES2907377T3 (en) | Apparatus, method and computer program for encoding, decoding, scene processing and other methods related to DirAC-based spatial audio coding | |
US9794721B2 (en) | System and method for capturing, encoding, distributing, and decoding immersive audio | |
EP2374123B1 (en) | Improved encoding of multichannel digital audio signals | |
EP1999998B1 (en) | Method for binaural synthesis taking into account a spatial effect | |
EP2042001B1 (en) | Binaural spatialization of compression-encoded sound data | |
EP1992198A2 (en) | Optimization of binaural sound spatialization based on multichannel encoding | |
US20150244869A1 (en) | Spatial Multiplexing in a Soundfield Teleconferencing System | |
Rafaely et al. | Spatial audio signal processing for binaural reproduction of recorded acoustic scenes–review and challenges | |
US20140372107A1 (en) | Audio processing | |
EP2920979B1 (en) | Acquisition of spatialised sound data | |
CN111183479A (en) | Concept for generating an enhanced or modified sound field description using a multi-layer description | |
WO2018162803A1 (en) | Method and arrangement for parametric analysis and processing of ambisonically encoded spatial sound scenes | |
JP2024023412A (en) | Sound field related rendering | |
EP3400599B1 (en) | Improved ambisonic encoder for a sound source having a plurality of reflections | |
WO2018050292A1 (en) | Device and method for capturing and processing a three-dimensional acoustic field | |
FR3065137A1 (en) | SOUND SPATIALIZATION METHOD | |
Hold et al. | Parametric binaural reproduction of higher-order spatial impulse responses | |
He et al. | Literature review on spatial audio | |
EP3384688B1 (en) | Successive decompositions of audio filters | |
WO2020221431A1 (en) | Device and method for rendering a binaural audio signal | |
EP3108670B1 (en) | Method and device for rendering of a multi-channel audio signal in a listening zone | |
Chanda et al. | Immersive rendering of coded audio streams using reduced rank models of subband-domain head-related transfer functions | |
WO2006075079A1 (en) | Method for encoding audio tracks of a multimedia content to be broadcast on mobile terminals |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20190128 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17Q | First examination report despatched |
Effective date: 20200127 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
INTG | Intention to grant announced |
Effective date: 20210624 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 1452563 Country of ref document: AT Kind code of ref document: T Effective date: 20211215 Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: FRENCH |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602017050275 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG9D |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20211201 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 1452563 Country of ref document: AT Kind code of ref document: T Effective date: 20211201 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211201 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211201 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211201 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220301 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211201 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211201 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211201 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220301 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211201 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211201 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220302 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211201 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211201 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211201 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211201 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211201 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220401 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211201 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211201 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602017050275 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220401 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211201 Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211201 |
|
26N | No opposition filed |
Effective date: 20220902 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211201 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211201 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
GBPC | Gb: european patent ceased through non-payment of renewal fee |
Effective date: 20220928 |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20220930 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211201 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20220928 |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230526 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20220930 Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20220928 Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20220930 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20220930 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20220930 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GB Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20220928 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20230802 Year of fee payment: 7 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20170928 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211201 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211201 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211201 |