WO2009050487A1 - Acoustic source separation - Google Patents
Acoustic source separation Download PDFInfo
- Publication number
- WO2009050487A1 WO2009050487A1 PCT/GB2008/003538 GB2008003538W WO2009050487A1 WO 2009050487 A1 WO2009050487 A1 WO 2009050487A1 GB 2008003538 W GB2008003538 W GB 2008003538W WO 2009050487 A1 WO2009050487 A1 WO 2009050487A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- source
- pressure
- signals
- directions
- sources
- Prior art date
Links
- 238000000926 separation method Methods 0.000 title description 48
- 239000000203 mixture Substances 0.000 claims abstract description 76
- 238000000034 method Methods 0.000 claims abstract description 60
- 238000009826 distribution Methods 0.000 claims description 19
- 238000012545 processing Methods 0.000 claims description 14
- 239000011159 matrix material Substances 0.000 claims description 13
- 230000009467 reduction Effects 0.000 claims description 4
- 230000006870 function Effects 0.000 description 50
- 239000013598 vector Substances 0.000 description 36
- 238000004422 calculation algorithm Methods 0.000 description 12
- 230000000875 corresponding effect Effects 0.000 description 10
- 230000003044 adaptive effect Effects 0.000 description 8
- 238000004458 analytical method Methods 0.000 description 8
- 230000004044 response Effects 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000001914 filtration Methods 0.000 description 5
- 239000002245 particle Substances 0.000 description 5
- 230000005236 sound signal Effects 0.000 description 5
- 238000012360 testing method Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 238000012880 independent component analysis Methods 0.000 description 4
- 238000007476 Maximum Likelihood Methods 0.000 description 3
- 238000003491 array Methods 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 208000023514 Barrett esophagus Diseases 0.000 description 2
- 230000002457 bidirectional effect Effects 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 2
- 230000002596 correlated effect Effects 0.000 description 2
- 230000004807 localization Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000003786 synthesis reaction Methods 0.000 description 2
- 238000001283 Kuiper's test Methods 0.000 description 1
- 239000012080 ambient air Substances 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 239000002775 capsule Substances 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000005056 compaction Methods 0.000 description 1
- 230000021615 conjugation Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000009472 formulation Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0272—Voice signal separating
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
- H04R3/005—Circuits for transducers, loudspeakers or microphones for combining the signals of two or more microphones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/10—Earpieces; Attachments therefor ; Earphones; Monophonic headphones
- H04R1/1083—Reduction of ambient noise
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2225/00—Details of deaf aids covered by H04R25/00, not provided for in any of its subgroups
- H04R2225/43—Signal processing in hearing aids to enhance the speech intelligibility
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
- H04S2400/15—Aspects of sound capture and related signal processing for recording or reproduction
Definitions
- the present invention relates to the processing of acoustic signals, and in particular to the separation of a mixture of sounds from different sound sources.
- the separation of convolutive mixtures aims to estimate the individual sound signals in the presence of other such signals in reverberant environments. As sound mixtures are almost always convolutive in enclosures, their separation is a useful pre-processing stage for speech recognition and speaker identification problems. Other direct application areas also exist such as in hearing aids, teleconferencing, multichannel audio and acoustical surveillance.
- Several techniques have been proposed before for the separation of convolutive mixtures, which can be grouped into three different categories: stochastic, adaptive and deterministic.
- ICA ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇
- adaptive beamforming utilizes spatial selectivity to improve the capture of the target source while suppressing the interferences from other sources.
- These adaptive algorithms are similar to stochastic methods in the sense that they both depend on the properties of the signals to reach a solution. It has been shown that the frequency domain adaptive beamforming is equivalent to the frequency domain blind source separation (BSS) . These algorithms need to adaptively converge to a solution which may be suboptimal. They also need to tackle with all the targets and interferences jointly. Furthermore, the null beamforming applied for the interference signal is not very effective under reverberant conditions due to the reflections, creating an upper bound for the performance of the BSS.
- Deterministic methods do not make any assumptions about the source signals and depend solely on the deterministic aspects of the problem such as the source directions and the multipath characteristics of the reverberant environment. Although there have been efforts to exploit direction-of-arrival (DOA) information and the channel characteristics for solving the permutation problem, these were used in an indirect way, merely to assist the actual separation algorithm, which was usually stochastic or adaptive.
- DOE direction-of-arrival
- the present invention provides a technique that can be used to provide a closed form solution for the separation of convolutive mixtures captured by a compact, coincident microphone array.
- the technique may depend on the channel characterization in the frequency domain based on the analysis of the intensity vector statistics. This can avoid the permutation problem which normally occurs due to the lack of channel modeling in the frequency domain methods.
- the present invention provides a method of separating a mixture of acoustic signals from a plurality of sources, the method comprising any one or more of the following: providing pressure signals indicative of time-varying acoustic pressure in the mixture; defining a series of time windows; and for each time window: a) generating from the pressure signals a series of sample values of measured directional pressure gradient; b) identifying different frequency components of the pressure signals; c) for each frequency component defining an associated direction; d) from the frequency components and their associated directions generating a separated signal for one of the sources.
- the separation may be performed in two dimensions, or three dimensions.
- the method may include generating the pressure signals, or may be performed on pressure signals which have already been obtained
- the method may include defining from the pressure signals a series of values of a pressure function.
- the directionality function may be applied to the pressure function to generate the separated signal for the source.
- the pressure function may be, or be derived from, one or more of the pressure signals, which may be generated from one or more omnidirectional pressure sensors, or the pressure function may be, or be derived from, one or more pressure gradients.
- the separated signal may be an electrical signal.
- the separated signal may define an associated acoustic signal.
- the separated signal may be used to generate a corresponding acoustic signal.
- the associated direction may be determined from the pressure gradient sample values.
- the directions of the frequency components may be combined to form a probability distribution from which the directionality function is obtained.
- the directionality function may be obtained by modelling the probability distribution so as to include a set of source components each comprising a probability distribution from a single source.
- the probability distribution may be modelled so as also to include a uniform density component.
- the source components may be estimated numerically from the measured intensity vector direction distribution.
- Each of the source components may have a beamwidth and a direction, each of which may be selected from a set of discrete possible values.
- the directionality function may define a weighting factor which varies as a function of direction, and which is applied to each frequency component of the omnidirectional pressure signal depending on the direction associated with that frequency.
- the present invention further provides a system for separating a mixture of acoustic signals from a plurality of sources, the system comprising: sensing means arranged to provide pressure signals indicative of time varying acoustic pressure in the mixture; and processing means arranged to define a series of time windows; and for each time window to: a) generate from the pressure signals a series of sample values of measured directional pressure gradient; b) identify different frequency components of the pressure signals c) for each frequency component define an associated direction; d) from the frequency components and their associated directions generate a separated signal for the selected one or more sources.
- the system may be arranged to carry out any of the method steps of the method of the invention.
- Figure 1 is a schematic diagram of a system according to an embodiment of the invenion
- Figure 2 is a diagram of a microphone array forming part of the system of Figure 1 ;
- Figure 3 is a graph showing examples of some von Mises functions of different beamwidths used in the processing performed by the system of Figure 1 ;
- Figure 4 is a graph showing probability density functions, estimated individual mixture components, and fitted mixture for two active sources in the system of Figure 1;
- Figure 5 is a graph, similar to Figure 5, for three active sources in the system of Figure 1;
- Figure 6 is a functional diagram of the processing stages performed by the system of Figure 1 ;
- Figures 7 is a graph of signal to interference ratio as a function of angular source separation for a two source system in two different rooms;
- Figure 8 is a graph of signal to distortion ratio as a function of angular source separation for a two source system in two different rooms;
- Figure 9 is a graph of signal to interference ratio as a function of angular source separation for a three source system in two different rooms
- Figure 10 is a graph of signal to distortion ratio as a function of angular source separation for a three source system in two different rooms.
- Figure 11 is schematic diagram of a microphone array of a system according to a further embodiment of the invention.
- Figure 12 is a schematic diagram of the microphone array of a system according to a further embodiment of the invenion
- Figure 13 is a graph showing examples of some von Mises functions of different beamwidths used in the processing performed by the system of Figure 12
- Figures 14a-g show a mixture signal p w ⁇ t) ( Figure 14a) , reverberant originals of three signals making up the mixture signal ( Figures 14b-d)) and separated signals ( Figures 14e-g) obtained from the mixture using the system of Figure 12
- Figure 15 is a graph showing the r.m.s. energies of the signals in the mixture of Figure 14;
- Figure 16 is a graph showing the signal to interference ratio (SIR) for the separated signals for 2-, 3- and 4-source mixtures at different source positions, as obtained with the system of Figure 12; and
- Figure 17 is a graph showing the relationship between actual source direction and the direction of r.m.s. energy peaks calculated for 2- 3- and 4-source mixtures using the sytem of Figure 12. DESCRIPTION OF THE PREFERRED EMBODIMENTS
- an audio source separation system comprises a microphone array 10, a processing system, in this case a personal computer 12, arranged to receive audio signals from the microphone array and process them, and a speaker system 14 arranged to generate sounds based on the processed audio signals.
- the microphone array 10 is located at the centre of a circle of 36 nominal source positions 16. Sound sources 18 can be placed at any of these positions and the system is arranged to separate the sounds from each of the source positions 16. Clearly in a practical system the sound source positions could be spaced apart in a variety of ways.
- the microphone array 10 comprises four omnidirectional microphones, or pressure sensors, 21, 22, 23, 24 arranged in a square array in a horizontal plane.
- the diagonals of the square define x and y axes with two of the microphones 21 , 22 lying on the x axis and two 23, 24 lying on the y axis.
- the pressure signal recorded by th microphone of the array, with N sources can be written as
- h nm ( ⁇ ,t) is the time-frequency representation of the transfer function from the n"' source to the m"' microphone
- s n ( ⁇ ,t) is the time-frequency representation of the n"' original source.
- each h mn ( ⁇ ,t) coefficient can be represented as a plane wave arriving from direction ⁇ n ( ⁇ ,t) with respect to the center of the array. Assuming the pressure at the center of the array due to this plane wave is p o ⁇ ,t) . Then,
- the p w is similar to the pressure signal from an omnidirectional microphone
- p ⁇ and p ⁇ are similar to the signals from two bidirectional microphones that approximate pressure gradients along the X and Y directions, respectively.
- These signals are also known as B-format signals which can also be obtained by four capsules positioned at the sides of a tetrahedron (P. G. Craven and M. A. Gerzon, "Coincident microphone simulation covering three dimensional space and yielding various directional outputs, US 4,042,779) or by, coincidently placed, one omnidirectional and two bidirectional microphones facing the X and Y directions.
- v(r,w,t) The acoustic particle velocity, v(r,w,t) is defined in two dimensions as where p 0 is the ambient air density, c is the speed of sound, U x and u y are unit vectors in the directions of corresponding axes.
- the product of the pressure and the particle velocity gives instantaneous intensity.
- the active intensity can be found as,
- the direction of the intensity vector ⁇ ( ⁇ ,t) i.e. the direction of a single frequency component of the sound mixture at one time, can be obtained by
- the reverberant estimate of the ' source is obtained by beamforming the omnidirectional pressure signal p w in the source direction with a directivity function so that,
- the p w can be considered as comprising a number of components each at a respective frequency, each component varying with time.
- the directivity function takes each frequency component with its associated direction ⁇ ,t) and multiplies it by a weighting factor which is a function of that direction, giving an amplitude value for each frequency.
- the weighted frequency components can then be combined to form a total signal for the source.
- this weighting By this weighting, the time-frequency components of the omnidirectional microphone signal are amplified more if the direction of the corresponding intensity vector (i.e. the intensity vector with the same frequency and time) is closer to the direction of the target source. It should be noted that, this weighting also has the effect of partial deconvolution as the reflections are also suppressed depending on their arrival directions.
- the directivity function used for the source is a function of ⁇ only in the analyzed time-frequency bin. It is determined by the local statistics of the calculated intensity vector directions ⁇ ( ⁇ ,t) , of which there is one for each frequency, for the analyzed short-time window.
- the pressure and particle velocity components have Gaussian distributions. It may be suggested that the directions of the resulting intensity vectors for all frequencies within the analyzed short-time window are also Gaussian distributed.
- the probability density function of the intensity vector directions (i.e. the number of intesity vectors as a function of direction) for each time window can be modeled as a mixture g( ⁇ ) of N von Mises probability density functions each with a respective mean direction of corresponding to the source directions, and a circular uniform density due to the isotropic late reverberation: where, are the component weights, and
- Figures 4 and 5 show examples of the probability density functions of the intensity vector directions, individual mixture components and the fitted mixtures for two and three speech sources, respectively.
- the sources are at 50° and 280° for Figure 4 and 50° , 200° and 300° for Figure 5.
- the intensity vector directions were calculated for an exemplary analysis window of length 4096 samples at 44.1 kHz in a room with reverberation time of 0.83 s.
- the pressure and pressure gradient signals p w (t) p x (t) p y (t) are obtained from the microphone array 10. These signals are sampled at a sample rate of, in this case, 44.1 kHz, and the samples divided into time windows each of 4096 samples. Then, for each time window the modified discrete cosine transform (MDCT) of these signals are calculated. Next, the intensity vector directions are calculated and using the known source directions, von Mises mixture parameters are estimated. Next, beamforming is applied to the pressure signal for each of the target sources using the directivity functions obtained from the von Mises functions. Finally, inverse modified cosine transform (IMDCT) of the separated signals for the different sources are calculated, which reveals the time-domain estimates of the sound sources.
- MDCT modified discrete cosine transform
- the pressure and pressure gradient signals are calculated from the signals from the microphone array 10 as described above. However they can be obtained directly in B-format by using one of the commercially available tetrahedron microphones.
- the spacing between the microphones should be small to avoid aliasing at high frequencies. Phase errors at low frequencies should also be taken into account if a reliable frequency range for operation is essential (FJ. Fahy, Sound Intensity, 2 nd ed. London: E&FN SPON, 1995).
- Time-frequency representations of the pressure and pressure gradient signals are calculated using the modified discrete cosine transform (MDCT) where subsequent time window blocks are overlapped by 50% (J.P.Princen and A. Bradley, "Analysis/synthesis filter bank design based on time domain aliasing cancellation, "IEEE Trans.
- MDCT modified discrete cosine transform
- the MDCT is chosen due to its overlapping and energy compaction properties to decrease the edge effects across blocks that occur as the directivity function used for each time-frequency bin changes. Perfect reconstruction is achieved with a window function w k that satisfies , where 2M is the window length. In this work, the following window function is used:
- the intensity vector directions are calculated for each frequency within each time window, and rounded to the nearest degree.
- the mixture probability density is obtained from the histogram of the found directions for all frequencies. Then, the statistics of these directions are analyzed in order to estimate the mixture component parameters as in (17) .
- the 6 dB beamwidth is spanned linearly from 10° to 180° with 10° intervals and the related concentration parameters are calculated by using (19) . Beamwidths smaller than 10° were not included since very sharp clustering around a source direction was not observed from the densities of the intensity vector directions. As the point source assumption does not hold for real sound sources, such clustering is not expected even in anechoic environments due to the observed finite aperture of a sound source at the recording position. Beamwidths more than 180° were also not considered as the resulting von Mises functions are not very much different from the uniform density functions.
- the individual acoustic signals for the different sources can be used in a number of ways. For example, they can be played back through the speaker system 14 either individually or in groups. It will also be appreciated that the separation is carried out independently for each time window, and can be carried out at high speed. This means that, for each sound source, the separated signals from the series of time windows can be combined together into a continuous acoustic signal, providing continuous real time source separation.
- the algorithm was tested for mixtures of two and three sources for various source positions, in two rooms with different reverberation times.
- the recording setup, procedure for obtaining the mixtures, and the performance measures are discussed first below, followed by the results presenting various factors that affect the separation performance.
- the convolutive mixtures used in the testing of the algorithm were obtained by first measuring the B-format room impulse responses, convolving anechoic sound sources with these impulse responses and summing the resulting reverberant recordings. This method exploits the linearity and time-invariance assumptions of the linear acoustics.
- the impulse responses were measured in two different rooms.
- the first room was an ITU-R BSl 116 standard listening room with a reverberation time of 0.32 s.
- the source and recording positions were 1.2 m high above the floor.
- the loudspeaker had a width of 20 cm, corresponding to the observed source apertures of 7.15° and 5.72° at the recording positions for the first and second rooms, respectively.
- Anechoic sources sampled at 44.1 kHz were used from a commercially available CD entitled "Music for Archimedes” .
- the 5- second long portions of male English speech (M) , female English speech (F), male Danish speech (D), cello music (C) and guitar music (G) sounds were first equalized for energy, then convolved with the B-format impulse responses of the desired directions.
- the B-format sounds were then summed to obtain FM, CG, FC and MG for two source mixtures and FMD, CFG, MFC, DGM for three source mixtures.
- SIR signal-to-interference ratio
- N is the total number of sources, is the estimated source when only source S 1 is active, is the estimated source when only source S j is active and E ⁇ is the expectation operator. It has been suggested for convolutive mixtures that values of SIR above 15 dB indicate a good separation.
- SDR signal-to-distortion ratio
- any of the B-format signals or cardioid microphone signals that can be obtained from them can be used as the reference of that source. All of these signals can be said to have perfect sound quality, as the reverberation is not distortion. Therefore, it is fair to choose the reference signal that results in the best SDR values.
- a hypercardioid microphone has the highest directional selectivity that can be obtained by using B-format signals providing the best signal- to-reverberation gain. Since, the proposed technique performs partial deconvolution in addition to reverberation, a hypercardioid microphone most sensitive in the direction of the sound source is synthesized from the B-format recordings when only one source is active, such that,
- the source signal obtained in this way is used as the reference signal in the SDR calculation,
- Figures 7 and 8 show the signal-to-interference (SIR) and signal-to- distortion (SDR) ratios in dB plotted against the angular interval between the two sound sources.
- the first sound source was positioned at 0° and the position of the second source was varied from 0° to 180° with 10° intervals to yield the corresponding angular interval.
- the tests were repeated both for the listening room and for the reverberant room.
- the error bars were calculated using the lowest and highest deviations from the mean values considering all four mixtures (FM, CG, FC and MG) .
- the SIR values increase, in general, when the angular interval between the sound sources increases, although at around 180° , the SIR values decrease slightly because for this angle both sources lie on the same axis causing vulnerability to phase errors.
- the SDR values also increase when the angular interval between the two sources increases. Similar to the SIR values, the SDR values are better for the listening room which has the lower reverberation time. The similar trend observed for the SDR and SIR values indicates that the distortion is mostly due to the interferences rather than the processing artifacts.
- Figures 9 and 10 show the signal-to-interference (SIR) and signal- to-distortion (SDR) ratios in dB plotted against the angular interval between the three sound sources.
- the first sound source was positioned at 0°
- the position of the second source was varied from 0° to 120° with 10° increasing intervals
- the position of the third source was varied from 360° to 240° with 10° decreasing intervals to yield the corresponding equal angular intervals from the first source.
- the tests were repeated both for the listening room and the reverberant room.
- the error bars were calculated using the lowest and highest deviations from the mean values considering all four mixtures (FMD, CFG, MFC and DMG) .
- the SIR values display a similar trend to the two-source mixtures, increasing with increasing angular intervals and taking higher values in the room with less reverberation time.
- the values are lower in general from those obtained for the two-source mixtures, as expected.
- the SDR values indicate better sound quality for larger angular intervals between the sources and for the room with less reverberation time. However, the quality is usually less than that obtained for the two- source mixtures.
- an acoustic source separation method for convolutive mixtures has been presented.
- the intensity vector directions can be found by using the pressure and pressure gradient signals obtained from a closely spaced microphone array.
- the method assumes a priori knowledge of the sound source directions.
- the densities of the observed intensity vector directions are modeled as mixtures of von Mises density functions with mean values around the source directions and a uniform density function corresponding to the isotropic late reverberation.
- the statistics of the mixture components are then exploited for separating the mixture by beamforming in the directions of the sources in the time-frequency domain.
- the method has been extensively tested for two and three source mixtures of speech and instrument sounds, for various angular intervals between the sources, and for two rooms with different reverberation times.
- the embodiments described provide good separation as quantified by the signal-to-interference (SIR) and signal-to-distortion (SDR) ratios.
- SIR signal-to-interference
- SDR signal-to-distortion
- the method performs better when the angular interval between the sources is large.
- the method performs slightly better for the two-source mixtures in comparison with three-source mixtures.
- higher reverberation time reduces the separation performance and increases distortion.
- Important advantages of the embodiment described are the compactness of the array, low number of individual channels to be processed, and the simple closed-form solution it provides as opposed to adaptive or iterative source separation algorithms.
- the method of this embodiment can be used in teleconferencing applications, hearing aids, acoustical surveillance, and speech recognition among others.
- the method can be used to extract sound from one source so that the remaining sounds, possibly from a large number of other sources, can be analysed together. This can be used, for example, to remove unwanted interference such as a loud siren, which otherwise interferes with analysis of the recorded sound.
- the method can also be used as a pre-processing stage in hearing aid devices or in automatic speech recognition and speaker identification applications, as a clean signal free from interferences improves the performance of recognition and identification algorithms.
- the directions of the intensity vectors can be calculated using only two pressure gradient microphones 110 L , HO R with directivity patterns of D L ( ⁇ ) and D R ( ⁇ ) .
- the microphone signals become,
- a compact microphone array used for intensity vector direction calculation is made up of four microphones 120a, 120b, 120c, 120d placed at positions which correspond to the four non-adjacent corners of a cube of side length d. This geometry forms a tetrahedral microphone array.
- B-format signals, p w , p x and p Y can be obtained as:
- the acoustic particle velocity, instantaneous intensity, and direction of the intensity vector, can be obtained from and p w using equations (12) , (13) and (14) above.
- the microphones 120a, 120b, 120c, 12Od in the array are closely spaced, plane wave assumption can safely be made for incident waves and their directions can be calculated. If simultaneously active sound signals do not overlap directionally in short time-frequency windows, the directions of the intensity vectors correspond to those of the sound sources randomly shifted by major reflections.
- the pressure signal can be written as the sum of pressure waves arriving from all directions, independent of the number of sound sources. Then, a crude approximation of the plane wave s( ⁇ , ⁇ ,t) arriving from direction ⁇ can be obtained by spatial filtering p as, where is the directional filter defined by the von Mises function, which is the circular equivalent of the Gaussian function defined by equation (16) as described above. Spatial filtering involves, for each possible source direction or 'look direction' multiplying each frequency component by a factor which varies (as defined by the filter) with the difference between the look direction and the direction from which the frequency component is detected as coming.
- Figure 13 shows the plot of the three von Mises directional filters with 10 dB, 30 dB and 45 dB beamwidths and 100° , 240° and 330° pointing directions, respectively normalised to have maximum values of 1.
- the time-frequency samples of the pressure signal p w are emphasized if the intensity vectors for these samples are on or around the look direction ⁇ ; otherwise, they are suppressed.
- N directional filters are used with look directions ⁇ varied by 2 ⁇ /N intervals. Then, the spatial filtering yields a row vector of size N for each time-frequency component:
- the elements of this vector can be considered as the proportion of the frequency component that is detected as coming from each of the N possibe source directions.
- This method implies block-based processing, such as with the overlap-add technique.
- the recorded signals are windowed, i.e. divided into time periods or windows of equal length, and converted into frequency domain after which each sample is processed as in (37) . These are then converted back into time-domain, windowed with a matching window function, overlapped and added to remove block effects.
- the selection of the time window size is important. If the window size is too short, then low frequencies can not be calculated efficiently. If, however, the window size is too long, both the correlated interference sounds and reflections contaminate the calculated intensity vector directions due to simultaneous arrivals.
- the dimension of the data matrix can be reduced by only considering a signal subspace of rank m , which is selected according to the relative magnitudes of the singular values as,
- Figure 14a shows the mixture signal p w (t)
- Figures 14b, 14c and 14d show the reverberant originals of each mixture signal
- Figures 14e, 14f and 14g show the separated signals for three speech sounds at directions 30° , 100° and 300° recorded in a room with reverberation time of 0.32 s.
- the signal subspace has been decomposed using the highest three singular values.
- the three rows of the data matrix with highest r.m.s. energy has been plotted.
- the number of the highest singular values that are used in dimensionality reduction is selected to be equal to or higher than a practical estimate of the number of sources in the environment. Alternatively, this number is estimated by simple thresholding of the singular values.
- the 2-source mixture contained MF sounds where the first source direction was fixed at 0° and the second source direction was varied from 30° to 330° with 30° intervals. Therefore, the angular interval between the sources was varied and 11 different mixtures were obtained.
- the 3- source mixture contained MFC sounds, where the direction of M was varied from 0° to 90° , direction of F was varied from 120° to 210° and direction of C was varied from 240° to 330° with 30° intervals. Therefore, 4 different mixtures were obtained while the angular separation between the sources were fixed at 120° .
- the 4-source mixture contained MFCT sounds, where the direction of M was varied from 0° to 60° , direction of F was varied from 90° to 150° , direction of C was varied from 180° to 240° and direction of T was varied from 270° to 330° with 30° intervals. Therefore, 3 different mixtures were obtained while the angular separation between the sources were fixed at 90° . Processing was done with a block size of 4096 and a beamwidth of 10° for creating a data matrix of size 360x88200 with a sampling frequency of 44.1 kHz. Dimension reduction was carried out using only the highest six singular values.
- Figure 16 shows the signal-to-interference ratios (SIR) for each separated source at the corresponding directions for the 2-, 3- and A- source mixtures.
- SIR signal-to-interference ratios
- Figure 17 shows how the directions of the r.m.s. energy peaks in the reduced dimension data matrix, calculated for the 2-, 3- and 4-source mixtures, vary with actual directions of the sources. As explained above, the discrepancies result from the early reflection in the environment, rather than the number of mixtures or their content.
- the signal- to-distortion ratios have also been calculated as described above.
- the active intensity in 3D can be written as:
- the directivity function is obtained by using this function, which then enables spatial filtering considering both the horizontal and vertical intensity vector directions.
Landscapes
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Quality & Reliability (AREA)
- Computational Linguistics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- Otolaryngology (AREA)
- Circuit For Audible Band Transducer (AREA)
- Obtaining Desirable Characteristics In Audible-Bandwidth Transducers (AREA)
- Measurement Of Mechanical Vibrations Or Ultrasonic Waves (AREA)
- Investigating Or Analyzing Materials By The Use Of Ultrasonic Waves (AREA)
Abstract
A method of separating a mixture of acoustic signals from a plurality of sources comprises: providing pressure signals indicative of time- varying acoustic pressure in the mixture; defining a series of time windows; and for each time window: a) providing from the pressure signals a series of sample values of measured directional pressure gradient; b) identifying different frequency components of the pressure signals c) for each frequency component defining an associated direction; and d) from the frequency components and their associated directions generating a separated signal for one of the sources.
Description
ACOUSTIC SOURCE SEPARATION
FIELD OF THE INVENTION
The present invention relates to the processing of acoustic signals, and in particular to the separation of a mixture of sounds from different sound sources.
BACKGROUND TO THE INVENTION
The separation of convolutive mixtures aims to estimate the individual sound signals in the presence of other such signals in reverberant environments. As sound mixtures are almost always convolutive in enclosures, their separation is a useful pre-processing stage for speech recognition and speaker identification problems. Other direct application areas also exist such as in hearing aids, teleconferencing, multichannel audio and acoustical surveillance. Several techniques have been proposed before for the separation of convolutive mixtures, which can be grouped into three different categories: stochastic, adaptive and deterministic.
Stochastic methods, such as the independent component analysis
(ICA) , are based on a separation criterion that assumes the statistical independence of the source signals. ICA was originally proposed for instantaneous mixtures. It is applied in the frequency domain for convolutive mixtures, as the convolution corresponds to multiplication in the frequency domain. Although faster implementations exist such as the FastICA, stochastic methods are usually computationally expensive due to the several iterations required for the computation of the demixing filters. Furthermore, frequency domain ICA-based techniques suffer from the scaling and permutation issues resulting from the independent application of the separation algorithms in each frequency bin.
The second group of methods are based on adaptive algorithms that optimize a multichannel filter structure according to the signal properties. Depending on the type of the microphone array used, adaptive beamforming (ABF) utilizes spatial selectivity to improve the capture of the target source while suppressing the interferences from other sources. These adaptive algorithms are similar to stochastic methods in the sense that they both depend on the properties of the signals to reach a solution. It has been shown that the frequency domain adaptive beamforming is equivalent to the frequency domain blind source separation (BSS) . These algorithms need to adaptively converge to a solution which may be suboptimal. They also need to tackle with all the targets and interferences jointly. Furthermore, the null beamforming applied for the interference signal is not very effective under reverberant conditions due to the reflections, creating an upper bound for the performance of the BSS.
Deterministic methods, on the other hand, do not make any assumptions about the source signals and depend solely on the deterministic aspects of the problem such as the source directions and the multipath characteristics of the reverberant environment. Although there have been efforts to exploit direction-of-arrival (DOA) information and the channel characteristics for solving the permutation problem, these were used in an indirect way, merely to assist the actual separation algorithm, which was usually stochastic or adaptive.
A deterministic approach that leads to a closed-form solution is very desirable from the computational point of view. However, no such method with satisfactory performance has been proposed so far. There are two reasons for this. Firstly, the knowledge of the source directions is not sufficient for good separation, because without adaptive algorithms, the source directions can be exploited only by simple delay-and-sum beamformers. However, due to the limited number of microphones in an
array, the spatial selectivity of such beamformers is not sufficient to perform well under reverberant conditions. Secondly, the multipath characteristics of the environment can not be found with sufficient accuracy while using non-coincident arrays, as the channel characteristics are different at each sensor position which in turn makes it difficult to determine the room responses from the mixtures.
Almost all of the source separation methods employ non-coincident microphone arrays to the extent that the existence of such an array geometry is an inherent assumption by default in the formulation of the problem. The use of a coincident microphone array was previously proposed to exploit the directivities of two closely positioned directional microphones (J. M. Sanchis and J. J. Rieta, "Computational Cost Reduction using coincident boundary microphones for convolutive blind signal separation" Electronics Lett., vol. 41 , no. 6 pp. 374-376 March 2005) . However, the construction of the solution disregarded the fact that the reflections are weighted with different directivity factors according to their arrival directions for two directional microphones pointing at different angles. Therefore, the method was, in fact, not suitable for convolutive mixtures. In literature, coincident microphone arrays have been investigated mostly for intensity vector calculations and sound source localization (H. E. de Bree, W. F. Druyvesteyn, E. Berenschot, and M. Elwenspoek, "Three dimensional sound intensity measurements using Microflown particle velocity sensors" , in Proc. 12th IEEE Int. Conf. on Micro Electro Mech. Syst. , Orlando, FL, USA, January 1999, pp. 124- 129; J. Merimaa and V. Pulkki, "Spatial impulse response rendering I: Analysis and synthesis, " J. Audio Eng. Soc , vol. 53, no.12, pp. 1115- 1127, December 2005; B. Gunel, H. Hacihabiboglu, and A.M. Kondoz, "Wavelet-packet based passive analysis of sound fields using a coincident microphone array, " Appl. Acoust. , vol. 68, no.7, pp. 778-796, July 2007) .
SUMMARY TO THE INVENTION
The present invention provides a technique that can be used to provide a closed form solution for the separation of convolutive mixtures captured by a compact, coincident microphone array. The technique may depend on the channel characterization in the frequency domain based on the analysis of the intensity vector statistics. This can avoid the permutation problem which normally occurs due to the lack of channel modeling in the frequency domain methods.
Accordingly the present invention provides a method of separating a mixture of acoustic signals from a plurality of sources, the method comprising any one or more of the following: providing pressure signals indicative of time-varying acoustic pressure in the mixture; defining a series of time windows; and for each time window: a) generating from the pressure signals a series of sample values of measured directional pressure gradient; b) identifying different frequency components of the pressure signals; c) for each frequency component defining an associated direction; d) from the frequency components and their associated directions generating a separated signal for one of the sources.
The separation may be performed in two dimensions, or three dimensions.
The method may include generating the pressure signals, or may be performed on pressure signals which have already been obtained
The method may include defining from the pressure signals a series of values of a pressure function. The directionality function may be applied to the pressure function to generate the separated signal for the source. For example, the pressure function may be, or be derived from, one or more of the pressure signals, which may be generated from one or more omnidirectional pressure sensors, or the pressure function may be, or be derived from, one or more pressure gradients.
The separated signal may be an electrical signal. The separated signal may define an associated acoustic signal. The separated signal may be used to generate a corresponding acoustic signal.
The associated direction may be determined from the pressure gradient sample values.
The directions of the frequency components may be combined to form a probability distribution from which the directionality function is obtained.
The directionality function may be obtained by modelling the probability distribution so as to include a set of source components each comprising a probability distribution from a single source.
The probability distribution may be modelled so as also to include a uniform density component.
The source components may be estimated numerically from the measured intensity vector direction distribution.
Each of the source components may have a beamwidth and a direction, each of which may be selected from a set of discrete possible values.
The directionality function may define a weighting factor which varies as a function of direction, and which is applied to each frequency component of the omnidirectional pressure signal depending on the direction associated with that frequency.
The present invention further provides a system for separating a mixture of acoustic signals from a plurality of sources, the system comprising: sensing means arranged to provide pressure signals indicative of time varying acoustic pressure in the mixture; and processing means arranged to define a series of time windows; and for each time window to: a) generate from the pressure signals a series of sample values of measured directional pressure gradient; b) identify different frequency components of the pressure signals c) for each frequency component define an associated direction; d) from the frequency components and their associated directions generate a separated signal for the selected one or more sources.
The system may be arranged to carry out any of the method steps of the method of the invention.
Preferred embodiments of the present invention will now be described by way of example only with reference to the accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
Figure 1 is a schematic diagram of a system according to an embodiment of the invenion;
Figure 2 is a diagram of a microphone array forming part of the system of Figure 1 ;
Figure 3 is a graph showing examples of some von Mises functions of different beamwidths used in the processing performed by the system of Figure 1 ;
Figure 4 is a graph showing probability density functions, estimated individual mixture components, and fitted mixture for two active sources in the system of Figure 1;
Figure 5 is a graph, similar to Figure 5, for three active sources in the system of Figure 1;
Figure 6 is a functional diagram of the processing stages performed by the system of Figure 1 ;
Figures 7 is a graph of signal to interference ratio as a function of angular source separation for a two source system in two different rooms;
Figure 8 is a graph of signal to distortion ratio as a function of angular source separation for a two source system in two different rooms;
Figure 9 is a graph of signal to interference ratio as a function of angular source separation for a three source system in two different rooms;
Figure 10 is a graph of signal to distortion ratio as a function of angular source separation for a three source system in two different rooms.
Figure 11 is schematic diagram of a microphone array of a system according to a further embodiment of the invention;
Figure 12 is a schematic diagram of the microphone array of a system according to a further embodiment of the invenion;
Figure 13 is a graph showing examples of some von Mises functions of different beamwidths used in the processing performed by the system of Figure 12
Figures 14a-g show a mixture signal pw {t) (Figure 14a) , reverberant originals of three signals making up the mixture signal (Figures 14b-d)) and separated signals (Figures 14e-g) obtained from the mixture using the system of Figure 12
Figure 15 is a graph showing the r.m.s. energies of the signals in the mixture of Figure 14;
Figure 16 is a graph showing the signal to interference ratio (SIR) for the separated signals for 2-, 3- and 4-source mixtures at different source positions, as obtained with the system of Figure 12; and
Figure 17 is a graph showing the relationship between actual source direction and the direction of r.m.s. energy peaks calculated for 2- 3- and 4-source mixtures using the sytem of Figure 12.
DESCRIPTION OF THE PREFERRED EMBODIMENTS
Referring to Figure 1 , an audio source separation system according to a first embodiment of the invention comprises a microphone array 10, a processing system, in this case a personal computer 12, arranged to receive audio signals from the microphone array and process them, and a speaker system 14 arranged to generate sounds based on the processed audio signals. The microphone array 10 is located at the centre of a circle of 36 nominal source positions 16. Sound sources 18 can be placed at any of these positions and the system is arranged to separate the sounds from each of the source positions 16. Clearly in a practical system the sound source positions could be spaced apart in a variety of ways.
Referring to Figure 2, the microphone array 10 comprises four omnidirectional microphones, or pressure sensors, 21, 22, 23, 24 arranged in a square array in a horizontal plane. The diagonals of the square define x and y axes with two of the microphones 21 , 22 lying on the x axis and two 23, 24 lying on the y axis. The four sensors 21 , 22,
23, 24 are arranged to generate pressure signals p1, p2, p3, p4 respectively. This allows the pressure pw at the centre of the array and the pressure gradients px and py in the x and y directions to be determined using:
In general, in the time-frequency domain, the pressure signal recorded by th
microphone of the array, with N sources, can be written as
where hnm(ω,t) is the time-frequency representation of the transfer function from the n"' source to the m"' microphone, and sn(ω,t) is the time-frequency representation of the n"' original source. The aim of the sound source separation is estimating the individual mixture components from the observation of the microphone signals only.
Assuming that four omnidirectional microphones are positioned very closely on a plane in the geometry as shown in Figure 2, each hmn(ω,t) coefficient can be represented as a plane wave arriving from direction φn(ω,t) with respect to the center of the array. Assuming the pressure at the center of the array due to this plane wave is po{ω,t) . Then,
where k is the wave number related to the wavelength λ as , j is the imaginary unit and 2d is the distance between the two
microphones on the same axis. Now, define
and Then,
If kd « \ , i.e. , when the microphones are positioned close to each other in comparison to the wavelength, it can be shown by using the relations and
that,
The pw is similar to the pressure signal from an omnidirectional microphone, and pχ and pγ are similar to the signals from two bidirectional microphones that approximate pressure gradients along the X and Y directions, respectively. These signals are also known as B-format signals which can also be obtained by four capsules positioned at the sides of a tetrahedron (P. G. Craven and M. A. Gerzon, "Coincident microphone simulation covering three dimensional space and yielding various directional outputs, US 4,042,779) or by, coincidently placed, one omnidirectional and two bidirectional microphones facing the X and Y directions.
The use of these signals for source separation based on intensity vector analysis will now be described.
The acoustic particle velocity, v(r,w,t) is defined in two dimensions as
where p0 is the ambient air density, c is the speed of sound, Ux and uy are unit vectors in the directions of corresponding axes.
The product of the pressure and the particle velocity gives instantaneous intensity. The active intensity can be found as,
Then, the direction of the intensity vector γ(ω,t) , i.e. the direction of a single frequency component of the sound mixture at one time, can be obtained by
The reverberant estimate of the
' source,
is obtained by beamforming the omnidirectional pressure signal pw in the source direction with a directivity function
so that,
The pw can be considered as comprising a number of components each at a respective frequency, each component varying with time. The directivity function, for a particular source and a particular time window, takes each frequency component with its associated direction γ{ω,t) and multiplies it by a weighting factor which is a function of that direction,
giving an amplitude value for each frequency. The weighted frequency components can then be combined to form a total signal for the source.
By this weighting, the time-frequency components of the omnidirectional microphone signal are amplified more if the direction of the corresponding intensity vector (i.e. the intensity vector with the same frequency and time) is closer to the direction of the target source. It should be noted that, this weighting also has the effect of partial deconvolution as the reflections are also suppressed depending on their arrival directions.
Calculation of the directivity function from the intensity vector statistics will now be described.
The directivity function
used for the
source is a function of θ only in the analyzed time-frequency bin. It is determined by the local statistics of the calculated intensity vector directions γ(ω,t) , of which there is one for each frequency, for the analyzed short-time window.
For a reverberant room, the pressure and particle velocity components have Gaussian distributions. It may be suggested that the directions of the resulting intensity vectors for all frequencies within the analyzed short-time window are also Gaussian distributed.
In circular statistics, the equivalent of a Gaussian distribution is a von Mises distribution whose probability density function is given as:
for a circular random variable θ where,
the mean direction,
is the concentration parameter and
) is the modified Bessel function of order zero.
For N sound sources, the probability density function of the intensity vector directions (i.e. the number of intesity vectors as a function of direction) for each time window can be modeled as a mixture g( θ) of N von Mises probability density functions each with a respective mean direction of corresponding to the source directions, and a
circular uniform density due to the isotropic late reverberation:
where,
are the component weights, and
As analytical methods do not exist for finding the maximum likelihood estimates of the mixture parameters, it can be assumed that the an and Kn take discrete values within some boundary and the values of these parameters that maximize the likelihood can be determined numerically. The directivity function for beamforming in the direction of the n"' source for a given time-frequency bin is then defined as
For simplicity, the component weights can be assumed to be equal to each other, i.e. an = \/(N+\) . It can be shown by using the definition of the von Mises function in (16) that the concentration parameter K is logarithmically related to the 6 dB beamwidth θm of this directivity function as
Then, in numerical maximum likelihood estimation, it is appropriate to determine the concentration parameters from linearly increasing beamwidth values. Figure 3 shows four von Mises functions for 6 dB beamwidths of 10° ( K = 182.15), 45° (K = 9.10), 90° ( K- = 2.37) and 180° ( K = 0.69) .
Figures 4 and 5 show examples of the probability density functions of the intensity vector directions, individual mixture components and the fitted mixtures for two and three speech sources, respectively. The sources are at 50° and 280° for Figure 4 and 50° , 200° and 300° for Figure 5. The intensity vector directions were calculated for an exemplary analysis window of length 4096 samples at 44.1 kHz in a room with reverberation time of 0.83 s.
It should be noted that the fitting is applied to determine the directivity functions. Therefore, testing the goodness-of-fit by methods such as the Kuiper test is not discussed here.
The processing stages of the method of this embodiment, as carried out by the PC 12 can be divided into 5 steps as shown in Figure 6.
Initially, the pressure and pressure gradient signals pw(t) px(t) py(t) are obtained from the microphone array 10. These signals are sampled at a sample rate of, in this case, 44.1 kHz, and the samples divided into time windows each of 4096 samples. Then, for each time window the modified discrete cosine transform (MDCT) of these signals are calculated. Next, the intensity vector directions are calculated and using the known source directions, von Mises mixture parameters are estimated. Next, beamforming is applied to the pressure signal for each of the target sources using the directivity functions obtained from the von Mises
functions. Finally, inverse modified cosine transform (IMDCT) of the separated signals for the different sources are calculated, which reveals the time-domain estimates of the sound sources.
The pressure and pressure gradient signals are calculated from the signals from the microphone array 10 as described above. However they can be obtained directly in B-format by using one of the commercially available tetrahedron microphones. The spacing between the microphones should be small to avoid aliasing at high frequencies. Phase errors at low frequencies should also be taken into account if a reliable frequency range for operation is essential (FJ. Fahy, Sound Intensity, 2nd ed. London: E&FN SPON, 1995).
Time-frequency representations of the pressure and pressure gradient signals are calculated using the modified discrete cosine transform (MDCT) where subsequent time window blocks are overlapped by 50% (J.P.Princen and A. Bradley, "Analysis/synthesis filter bank design based on time domain aliasing cancellation, "IEEE Trans.
Acoustic, Speech, Signal Process. , vol. 34, no. 5, pp. 1153-1161, October 1986) . The MDCT is chosen due to its overlapping and energy compaction properties to decrease the edge effects across blocks that occur as the directivity function used for each time-frequency bin changes. Perfect reconstruction is achieved with a window function wk that satisfies
, where 2M is the window length. In this work, the following window function is used:
The intensity vector directions are calculated for each frequency within each time window, and rounded to the nearest degree. The mixture probability density is obtained from the histogram of the found directions for all frequencies. Then, the statistics of these directions are analyzed in order to estimate the mixture component parameters as in (17) . For numerical maximum likelihood estimation, the 6 dB beamwidth is spanned linearly from 10° to 180° with 10° intervals and the related concentration parameters are calculated by using (19) . Beamwidths smaller than 10° were not included since very sharp clustering around a source direction was not observed from the densities of the intensity vector directions. As the point source assumption does not hold for real sound sources, such clustering is not expected even in anechoic environments due to the observed finite aperture of a sound source at the recording position. Beamwidths more than 180° were also not considered as the resulting von Mises functions are not very much different from the uniform density functions.
Once the individual acoustic signals for the different sources have been obtained it will be appreciated that they can be used in a number of ways. For example, they can be played back through the speaker system 14 either individually or in groups. It will also be appreciated that the separation is carried out independently for each time window, and can be carried out at high speed. This means that, for each sound source, the separated signals from the series of time windows can be combined together into a continuous acoustic signal, providing continuous real time source separation.
The algorithm was tested for mixtures of two and three sources for various source positions, in two rooms with different reverberation times. The recording setup, procedure for obtaining the mixtures, and the
performance measures are discussed first below, followed by the results presenting various factors that affect the separation performance.
The convolutive mixtures used in the testing of the algorithm were obtained by first measuring the B-format room impulse responses, convolving anechoic sound sources with these impulse responses and summing the resulting reverberant recordings. This method exploits the linearity and time-invariance assumptions of the linear acoustics.
The impulse responses were measured in two different rooms. The first room was an ITU-R BSl 116 standard listening room with a reverberation time of 0.32 s. The second one was a meeting room with a reverberation time of 0.83 s. Both rooms were geometrically similar ( L = 8 m; W = 5.5 m; H = 3 m) and were empty during the tests.
For both rooms, 36 B-format impulse response recordings were obtained at 44.1 kHz with a SoundField microphone system (SPS422B) and a loudspeaker (Genelec 1030A), using a 16th-order maximum length sequence (MLS) signal. Each of the 36 measurement positions were located on a circle of 1.6 m radius for the first room, and 2.0 m radius for the second room, as shown in Figure 1. The recording points were at the center of the circles, and the frontal directions of the recording setup were fixed in each room. Source locations were selected between 0° to 350° with 10° intervals with respect to the recording setup. At each measurement position, the acoustical axis of the loudspeaker was facing towards the array location, while the orientation of the microphone system was kept fixed. The source and recording positions were 1.2 m high above the floor. The loudspeaker had a width of 20 cm, corresponding to the observed source apertures of 7.15° and 5.72° at the recording positions for the first and second rooms, respectively.
Anechoic sources sampled at 44.1 kHz were used from a commercially available CD entitled "Music for Archimedes" . The 5- second long portions of male English speech (M) , female English speech (F), male Danish speech (D), cello music (C) and guitar music (G) sounds were first equalized for energy, then convolved with the B-format impulse responses of the desired directions. The B-format sounds were then summed to obtain FM, CG, FC and MG for two source mixtures and FMD, CFG, MFC, DGM for three source mixtures.
There exist various criteria for the performance measure of source separation techniques. In this work, one-at-a-time signal-to-interference ratio (SIR) is used for quantifying the separation, as separately synthesized sources are summed together to obtain the mixture. This metric is defined as:
where N is the total number of sources, is the estimated
source
when only source S1 is active, is the estimated source
when only source Sj is active and E{●} is the expectation operator. It has been suggested for convolutive mixtures that values of SIR above 15 dB indicate a good separation.
In addition to SIR, signal-to-distortion ratio (SDR) has also been used in order to quantify the quality of the separated sources. However, the SDR is sensitive to the reverberation content of the original source used as the reference. If the anechoic source is used for comparison, this measure penalizes the effect of the reverberation even if the separation is quite good. On the other hand, if the reverberant source as observed at
the recording position is used, then any deconvolution achieved in addition to the separation is also penalized as distortion.
When only one sound source is active, any of the B-format signals or cardioid microphone signals that can be obtained from them can be used as the reference of that source. All of these signals can be said to have perfect sound quality, as the reverberation is not distortion. Therefore, it is fair to choose the reference signal that results in the best SDR values.
A hypercardioid microphone has the highest directional selectivity that can be obtained by using B-format signals providing the best signal- to-reverberation gain. Since, the proposed technique performs partial deconvolution in addition to reverberation, a hypercardioid microphone most sensitive in the direction of the
sound source is synthesized from the B-format recordings when only one source is active, such that,
The source signal obtained in this way is used as the reference signal in the SDR calculation,
Figures 7 and 8 show the signal-to-interference (SIR) and signal-to- distortion (SDR) ratios in dB plotted against the angular interval between the two sound sources. The first sound source was positioned at 0° and the position of the second source was varied from 0° to 180° with 10° intervals to yield the corresponding angular interval. The tests were
repeated both for the listening room and for the reverberant room. The error bars were calculated using the lowest and highest deviations from the mean values considering all four mixtures (FM, CG, FC and MG) .
As expected, better separation is achieved in the listening room than in the reverberant room. The SIR values increase, in general, when the angular interval between the sound sources increases, although at around 180° , the SIR values decrease slightly because for this angle both sources lie on the same axis causing vulnerability to phase errors.
The SDR values also increase when the angular interval between the two sources increases. Similar to the SIR values, the SDR values are better for the listening room which has the lower reverberation time. The similar trend observed for the SDR and SIR values indicates that the distortion is mostly due to the interferences rather than the processing artifacts.
Figures 9 and 10 show the signal-to-interference (SIR) and signal- to-distortion (SDR) ratios in dB plotted against the angular interval between the three sound sources. The first sound source was positioned at 0° , the position of the second source was varied from 0° to 120° with 10° increasing intervals, and the position of the third source was varied from 360° to 240° with 10° decreasing intervals to yield the corresponding equal angular intervals from the first source. The tests were repeated both for the listening room and the reverberant room. The error bars were calculated using the lowest and highest deviations from the mean values considering all four mixtures (FMD, CFG, MFC and DMG) .
The SIR values display a similar trend to the two-source mixtures, increasing with increasing angular intervals and taking higher values in
the room with less reverberation time. The values, however, are lower in general from those obtained for the two-source mixtures, as expected.
The SDR values indicate better sound quality for larger angular intervals between the sources and for the room with less reverberation time. However, the quality is usually less than that obtained for the two- source mixtures.
In the embodiments described above an acoustic source separation method for convolutive mixtures has been presented. Using this method, the intensity vector directions can be found by using the pressure and pressure gradient signals obtained from a closely spaced microphone array. The method assumes a priori knowledge of the sound source directions. The densities of the observed intensity vector directions are modeled as mixtures of von Mises density functions with mean values around the source directions and a uniform density function corresponding to the isotropic late reverberation. The statistics of the mixture components are then exploited for separating the mixture by beamforming in the directions of the sources in the time-frequency domain.
As described above, the method has been extensively tested for two and three source mixtures of speech and instrument sounds, for various angular intervals between the sources, and for two rooms with different reverberation times. The embodiments described provide good separation as quantified by the signal-to-interference (SIR) and signal-to-distortion (SDR) ratios. The method performs better when the angular interval between the sources is large. Similarly, the method performs slightly better for the two-source mixtures in comparison with three-source mixtures. As expected, higher reverberation time reduces the separation performance and increases distortion.
Important advantages of the embodiment described are the compactness of the array, low number of individual channels to be processed, and the simple closed-form solution it provides as opposed to adaptive or iterative source separation algorithms. As such, the method of this embodiment can be used in teleconferencing applications, hearing aids, acoustical surveillance, and speech recognition among others.
For example, in a teleconferencing system it might be desirable for speech from a single participant to be separated from other noise and interfering speech sounds and played back, or it might be desirable for the separated sound source signals to be played back from different relative positions than the relative positions of the original sources. In acoustical surveillance the method can be used to extract sound from one source so that the remaining sounds, possibly from a large number of other sources, can be analysed together. This can be used, for example, to remove unwanted interference such as a loud siren, which otherwise interferes with analysis of the recorded sound. The method can also be used as a pre-processing stage in hearing aid devices or in automatic speech recognition and speaker identification applications, as a clean signal free from interferences improves the performance of recognition and identification algorithms.
Further improvements could be achieved by applying this method together with other source separation methods that exploit the differences in the frequency content of the sound sources.
Referring to Figure 11, in a further embodiment of the invention, if all sound sources and their reflections are restricted to the horizontal half plane from - ^/2 ^0 π/2 t then the directions of the intensity vectors can be calculated using only two pressure gradient microphones 110L,
HOR with directivity patterns of DL (θ) and DR(θ) . For a plane wave, P(ω,t) arriving from direction γ , the microphone signals become,
For example, assume that two cardioid microphones are coincidently placed with look directions of -Ψ and Ψ as shown in Figure 11. The recorded signals for a plane wave P(ω,t) arriving from direction ^ can be written as:
By defining the ratio of these signals as K,
This enables the direction of the intensity vectors to be determined, and a directivity function to be derived which can then be used for beamforming to determine the separated acoustic signals for the sources.
Referring to Figure 12, in a further embodiment of the invention a compact microphone array used for intensity vector direction calculation is made up of four microphones 120a, 120b, 120c, 120d placed at positions which correspond to the four non-adjacent corners of a cube of side length d. This geometry forms a tetrahedral microphone array.
Let us consider a plane wave arriving from the direction γ(ω,t) on the horizontal plane with respect to the center of the cube. If the pressure at the centre due to this plane wave is po{ω,i) , then the pressure signals pa, pb, pc, pd recorded by the four microphones 120a, 120b, 120c, 12Od can be written as,
where k is the wave number related to the wavelength λ as k = 2π/λ , j is the imaginary unit and d is the length of the one side of the cube. Using these four pressure signals, B-format signals, pw , px and pY can be obtained as:
If, kd« 1 i.e. , when the microphones are positioned close to each other in comparison to the wavelength, it can be shown by using the relations
and
that,
The acoustic particle velocity,
instantaneous intensity, and direction of the intensity vector, can be obtained from
and pw using equations (12) , (13) and (14) above.
Since the microphones 120a, 120b, 120c, 12Od in the array are closely spaced, plane wave assumption can safely be made for incident waves and their directions can be calculated. If simultaneously active sound signals do not overlap directionally in short time-frequency windows, the directions of the intensity vectors correspond to those of the sound sources randomly shifted by major reflections.
The exhaustive separation of the sources by decomposing the sound field into plane waves using intensity vector directions will now be described. This essentially comprises taking N possible directions, and identifying from which of those possible directions the sound is coming, which indicates the likely positions of the sources.
In a short time-frequency window, the pressure signal
can be written as the sum of pressure waves arriving from all directions, independent of the number of sound sources. Then, a crude approximation of the plane wave s(μ,ω,t) arriving from direction μ can be obtained by spatial filtering p as,
where is the directional filter defined by the von
Mises function, which is the circular equivalent of the Gaussian function defined by equation (16) as described above.
Spatial filtering involves, for each possible source direction or 'look direction' multiplying each frequency component by a factor which varies (as defined by the filter) with the difference between the look direction and the direction from which the frequency component is detected as coming.
Figure 13 shows the plot of the three von Mises directional filters with 10 dB, 30 dB and 45 dB beamwidths and 100° , 240° and 330° pointing directions, respectively normalised to have maximum values of 1. By this directional filtering, the time-frequency samples of the pressure signal pw are emphasized if the intensity vectors for these samples are on or around the look direction μ ; otherwise, they are suppressed.
For exhaustive separation, i.e. separation of the mixture between a total set of N possible source directions, N directional filters are used with look directions μ varied by 2π/N intervals. Then, the spatial filtering yields a row vector
of size N for each time-frequency component:
The elements of this vector can be considered as the proportion of the frequency component that is detected as coming from each of the N possibe source directions.
This method implies block-based processing, such as with the overlap-add technique. The recorded signals are windowed, i.e. divided into time periods or windows of equal length, and converted into frequency domain after which each sample is processed as in (37) . These are then converted back into time-domain, windowed with a matching window function, overlapped and added to remove block effects.
The selection of the time window size is important. If the window size is too short, then low frequencies can not be calculated efficiently. If, however, the window size is too long, both the correlated interference sounds and reflections contaminate the calculated intensity vector directions due to simultaneous arrivals.
It should also be noted that although the processing is done in the frequency domain, the deterministic application of the spatial filter eliminates any permutation problem, which is normally observed in other frequency-domain BSS techniques due to independent application of the separation algorithms in each frequency bin.
Let us assume that the exhaustive separation by block-based processing yields a time-domain signal matrix
of size NxL , where L is the common length (in terms of the number of samples) of the signals and typically N<< L . Using (36) and (37) , it can be shown that the column wise sum of
equals to pw{t) , because,
due to the fact that . Therefore, the exhaustive
separation does not introduce additional noise or artifact, which is not present in pw(t) originally.
The singular value decomposition (SVD) of the signal matrix
can be expressed as,
where
is an orthonormal matrix of left singular vectors
is an orthonormal matrix of right singular vectors
is a pseudo-diagonal matrix with values along the diagonals
and p = min(N, L) .
The dimension of the data matrix
can be reduced by only considering a signal subspace of rank m , which is selected according to the relative magnitudes of the singular values as,
By selecting only the highest m singular values, independent rows of the S matrix are obtained that correspond to the individual signals of the mixture. Figure 14a shows the mixture signal pw(t) , Figures 14b, 14c and 14d show the reverberant originals of each mixture signal and Figures 14e, 14f and 14g show the separated signals for three speech sounds at directions 30° , 100° and 300° recorded in a room with reverberation time of 0.32 s. The data matrix is of size N = 360 and L = 88200 samples at 44.1 kHz sampling frequency, calculated using a block window size of 4096 samples. The signal subspace has been decomposed using the highest three singular values. The three rows of the data matrix with highest r.m.s. energy has been plotted. The number of the highest singular values that are used in dimensionality reduction is selected to be equal to or
higher than a practical estimate of the number of sources in the environment. Alternatively, this number is estimated by simple thresholding of the singular values.
When, the energies of the signals at each row of the reduced S matrix are calculated and plotted, peaks are observed at some directions. Figure 15 shows these r.m.s. energies for the previously given separation example. These directions can be used as an indication of the directions of the separated sources. However, the accuracy of the source directions found by these local maxima can change due to the fact that highly correlated early reflections of a sound may cause a shift in the calculated intensity vector directions. While the selection of the observed direction, rather than the actual one is preferable to obtain better SIR for the purposes of BSS, for source localisation problems, a correction should be applied if dominant early reflections are present in the enviroment.
The algorithm has been tested with 2-, 3- and 4-source mixtures of 2-second long sound signals consisting of male speech (M) , female speech (F), cello (C) and trumpet (T) music of equal energy recorded in a room of size ( L = 8 m; W = 5.5 m; H = 3 m) with a reverberation time of 0.32s. The 2-source mixture contained MF sounds where the first source direction was fixed at 0° and the second source direction was varied from 30° to 330° with 30° intervals. Therefore, the angular interval between the sources was varied and 11 different mixtures were obtained. The 3- source mixture contained MFC sounds, where the direction of M was varied from 0° to 90° , direction of F was varied from 120° to 210° and direction of C was varied from 240° to 330° with 30° intervals. Therefore, 4 different mixtures were obtained while the angular separation between the sources were fixed at 120° . The 4-source mixture contained MFCT sounds, where the direction of M was varied from 0° to
60° , direction of F was varied from 90° to 150° , direction of C was varied from 180° to 240° and direction of T was varied from 270° to 330° with 30° intervals. Therefore, 3 different mixtures were obtained while the angular separation between the sources were fixed at 90° . Processing was done with a block size of 4096 and a beamwidth of 10° for creating a data matrix of size 360x88200 with a sampling frequency of 44.1 kHz. Dimension reduction was carried out using only the highest six singular values.
Figure 16 shows the signal-to-interference ratios (SIR) for each separated source at the corresponding directions for the 2-, 3- and A- source mixtures. Angular interval between the sources increase with 30° intervals for the 2-source mixtures. For the 3-source and 4-source mixtures, the angular interval is fixed at 120° and 90° , respectively. The separation performance is not affected by the number of sources in the mixture as long as the angular separation between them is large enough.
Figure 17 shows how the directions of the r.m.s. energy peaks in the reduced dimension data matrix, calculated for the 2-, 3- and 4-source mixtures, vary with actual directions of the sources. As explained above, the discrepancies result from the early reflection in the environment, rather than the number of mixtures or their content.
In order to quantify the quality of the separated signals, the signal- to-distortion ratios (SDR) have also been calculated as described above.
For each separated source, the reverberant pw{i) signal recorded when only that source is active at the corresponding direction was used as the original source with no distortion for comparison. The mean SDRs for the
2-, 3-, and 4-source mixtures were found as 6.46 dB, 5.98 dB, 5.59 dB, respectively. It should also be noted that this comparison based SDR
calculation penalises dereverberation or other suppression of reflections, because the resulting changes on the signal are also considered as artifacts. Therefore, the actual SDRs are generally higher.
Due to the 3D symmetry of the tetrahedral microphone array of
Figure 12, the pressure gradient along the z axis, Pz (ω,t) can also be calculated and used for estimating both the horizontal and the vertical directions of the intensity vectors.
The active intensity in 3D can be written as:
Then, the horizontal and vertical directions of the intensity vector, μ(ω,t) and ν(ω,t) , respectively, can be obtained by
The extension of the von Mises distribution to 3D case yields a Fisher distribution which is defined as
where and are the horizontal and vertical
spherical polar coordinates and κ is the concentration parameter. This distribution is also known as von Mises-Fisher distribution. For
(on the horizontal plane) , this distribution reduces to the simple von Mises distribution.
For separation of sources in 3D, the directivity function is obtained by using this function, which then enables spatial filtering considering both the horizontal and vertical intensity vector directions.
Claims
1. A method of separating a mixture of acoustic signals from a plurality of sources, the method comprising: providing pressure signals indicative of time-varying acoustic pressure in the mixture; defining a series of time windows; and for each time window: a) providing from the pressure signals a series of sample values of measured directional pressure gradient; b) identifying different frequency components of the pressure signals c) for each frequency component defining an associated direction; and d) from the frequency components and their associated directions generating a separated signal for one of the sources.
2. A method according to claim 1 including generating from the pressure signals a series of sample values of a pressure function.
3. A method according to claim 2 wherein the directionality function is applied to the pressure function to generate the separated signal for the source.
4. A method according to claim 2 or claim 3 wherein the pressure function is one of: an omnidirectional pressure, an average pressure, and a pressure gradient.
5. A method according to claim 4 wherein the associated direction is determined from the pressure gradient sample values.
6. A method according to any foregoing claim wherein the directions of the sources is known.
7. A method according to any foregoing claim further comprising defining a directionality function for at least one source direction and using the directionality function to estimate the frequency components of the acoustic signal from the at least one source direction.
8. A method according to claim 7 wherein the directions of the frequency components are combined to form a probability distribution from which the directionality function is obtained.
9. A method according to claim 8 wherein the directionality function is obtained by modelling the probability distribution so as to include a set of source components each comprising a probability distribution from a single source.
10. A method according to claim 9 wherein the probability distribution is modelled so as also to include a uniform density component.
11. A method according to claim 9 or claim 10 wherein the source components are estimated numerically from the measured intensity distribution.
12. A method according to any of claims 9 to 11 wherein each of the source components has a beamwidth and a direction.
13. A method according to claim 12 wherein the beamwidth of each source component is selected from a set of discrete possible values.
14. A method according to claim 12 or claim 13 wherein the direction of each component is selected from a set of discrete possible values.
15. A method according to any of claims 7 to 14 wherein the directionality function defines a weighting factor which varies as a function of direction, and which is applied to each frequency component of the pressure function depending on the direction associated with that frequency.
16. A method according to any of claims 1 to 5 wherein the directions of the sources is unknown, and the method includes defining a set of possible source directions and, for at least one frequency component, generating a directional signal component associated with each of the possible source directions.
17. A method according to claim 16 further comprising generating the separated source signal from the directional signal components.
18. A method according to claim 17 wherein the separated source signal is generated using dimensional reduction of a matrix having the directional signal components as elements.
19. A system for separating a mixture of acoustic signals from a plurality of sources, the system comprising: sensing means arranged to provide pressure signals indicative of time varying acoustic pressure in the mixture; and processing means arranged to define a series of time windows; and for each time window to: a) generate from the pressure signals a series of sample values of measured directional pressure gradient; b) identify different frequency components of the pressure signals c) for each frequency component define an associated direction; and d) from the frequency components and their associated directions generate a separated signal for one of the sources.
20. A system according to claim 19 wherein the processing means is arranged to perform the method of any of claims 2 to 18.
21. A method of separating a mixture of acoustic signals substantially as hereinbefore described with reference to any one or more of the accompanying drawings.
22. A system for separating a mixture of acoustic signals substantially as hereinbefore described with reference to any one or more of the accompanying drawings.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP08806629.5A EP2203731B1 (en) | 2007-10-19 | 2008-10-17 | Acoustic source separation |
US12/734,195 US9093078B2 (en) | 2007-10-19 | 2008-10-17 | Acoustic source separation |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
GBGB0720473.8A GB0720473D0 (en) | 2007-10-19 | 2007-10-19 | Accoustic source separation |
GB0720473.8 | 2007-10-19 |
Publications (2)
Publication Number | Publication Date |
---|---|
WO2009050487A1 true WO2009050487A1 (en) | 2009-04-23 |
WO2009050487A8 WO2009050487A8 (en) | 2009-07-09 |
Family
ID=38814119
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/GB2008/003538 WO2009050487A1 (en) | 2007-10-19 | 2008-10-17 | Acoustic source separation |
Country Status (4)
Country | Link |
---|---|
US (1) | US9093078B2 (en) |
EP (1) | EP2203731B1 (en) |
GB (1) | GB0720473D0 (en) |
WO (1) | WO2009050487A1 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2013144609A1 (en) | 2012-03-26 | 2013-10-03 | University Of Surrey | Acoustic source separation |
WO2014177855A1 (en) | 2013-04-29 | 2014-11-06 | University Of Surrey | Microphone array for acoustic source separation |
EP3011762B1 (en) * | 2013-06-18 | 2020-04-22 | Dolby Laboratories Licensing Corporation | Adaptive audio content generation |
Families Citing this family (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
FR2948484B1 (en) * | 2009-07-23 | 2011-07-29 | Parrot | METHOD FOR FILTERING NON-STATIONARY SIDE NOISES FOR A MULTI-MICROPHONE AUDIO DEVICE, IN PARTICULAR A "HANDS-FREE" TELEPHONE DEVICE FOR A MOTOR VEHICLE |
US8818800B2 (en) * | 2011-07-29 | 2014-08-26 | 2236008 Ontario Inc. | Off-axis audio suppressions in an automobile cabin |
US9274239B2 (en) * | 2012-01-13 | 2016-03-01 | Westerngeco L.L.C. | Wavefield deghosting |
US9131295B2 (en) | 2012-08-07 | 2015-09-08 | Microsoft Technology Licensing, Llc | Multi-microphone audio source separation based on combined statistical angle distributions |
US9269146B2 (en) | 2012-08-23 | 2016-02-23 | Microsoft Technology Licensing, Llc | Target object angle determination using multiple cameras |
US9078057B2 (en) * | 2012-11-01 | 2015-07-07 | Csr Technology Inc. | Adaptive microphone beamforming |
US9460732B2 (en) | 2013-02-13 | 2016-10-04 | Analog Devices, Inc. | Signal source separation |
US9640179B1 (en) * | 2013-06-27 | 2017-05-02 | Amazon Technologies, Inc. | Tailoring beamforming techniques to environments |
US9420368B2 (en) | 2013-09-24 | 2016-08-16 | Analog Devices, Inc. | Time-frequency directional processing of audio signals |
US20170178664A1 (en) * | 2014-04-11 | 2017-06-22 | Analog Devices, Inc. | Apparatus, systems and methods for providing cloud based blind source separation services |
US10313808B1 (en) | 2015-10-22 | 2019-06-04 | Apple Inc. | Method and apparatus to sense the environment using coupled microphones and loudspeakers and nominal playback |
EP3293733A1 (en) * | 2016-09-09 | 2018-03-14 | Thomson Licensing | Method for encoding signals, method for separating signals in a mixture, corresponding computer program products, devices and bitstream |
JP6472823B2 (en) * | 2017-03-21 | 2019-02-20 | 株式会社東芝 | Signal processing apparatus, signal processing method, and attribute assignment apparatus |
JP6591477B2 (en) | 2017-03-21 | 2019-10-16 | 株式会社東芝 | Signal processing system, signal processing method, and signal processing program |
US10299039B2 (en) | 2017-06-02 | 2019-05-21 | Apple Inc. | Audio adaptation to room |
FR3067511A1 (en) * | 2017-06-09 | 2018-12-14 | Orange | SOUND DATA PROCESSING FOR SEPARATION OF SOUND SOURCES IN A MULTI-CHANNEL SIGNAL |
US10535361B2 (en) * | 2017-10-19 | 2020-01-14 | Kardome Technology Ltd. | Speech enhancement using clustering of cues |
EP3704873B1 (en) | 2017-10-31 | 2022-02-23 | Widex A/S | Method of operating a hearing aid system and a hearing aid system |
WO2019086433A1 (en) * | 2017-10-31 | 2019-05-09 | Widex A/S | Method of operating a hearing aid system and a hearing aid system |
EP3837861B1 (en) | 2018-08-15 | 2023-10-04 | Widex A/S | Method of operating a hearing aid system and a hearing aid system |
WO2020035158A1 (en) * | 2018-08-15 | 2020-02-20 | Widex A/S | Method of operating a hearing aid system and a hearing aid system |
US11270712B2 (en) | 2019-08-28 | 2022-03-08 | Insoundz Ltd. | System and method for separation of audio sources that interfere with each other using a microphone array |
Family Cites Families (30)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US2284749A (en) * | 1940-04-02 | 1942-06-02 | Rca Corp | System for sound recording |
US3159807A (en) * | 1958-03-24 | 1964-12-01 | Atlantic Res Corp | Signal analysis method and system |
US3704931A (en) * | 1971-08-30 | 1972-12-05 | Bendix Corp | Method and apparatus for providing an enhanced image of an object |
GB1512514A (en) * | 1974-07-12 | 1978-06-01 | Nat Res Dev | Microphone assemblies |
US4333170A (en) * | 1977-11-21 | 1982-06-01 | Northrop Corporation | Acoustical detection and tracking system |
DE3406343A1 (en) * | 1984-02-22 | 1985-08-29 | Messerschmitt-Bölkow-Blohm GmbH, 2800 Bremen | METHOD FOR LOCATING SIGNAL SOURCES WITH INTERFERENCE CANCELLATION |
JP3522954B2 (en) * | 1996-03-15 | 2004-04-26 | 株式会社東芝 | Microphone array input type speech recognition apparatus and method |
US6317703B1 (en) * | 1996-11-12 | 2001-11-13 | International Business Machines Corporation | Separation of a mixture of acoustic sources into its components |
US6260013B1 (en) * | 1997-03-14 | 2001-07-10 | Lernout & Hauspie Speech Products N.V. | Speech recognition system employing discriminatively trained models |
CA2294262A1 (en) * | 1997-06-18 | 1998-12-23 | Clarity, L.L.C. | Methods and apparatus for blind signal separation |
US6603861B1 (en) * | 1997-08-20 | 2003-08-05 | Phonak Ag | Method for electronically beam forming acoustical signals and acoustical sensor apparatus |
US6225948B1 (en) * | 1998-03-25 | 2001-05-01 | Siemens Aktiengesellschaft | Method for direction estimation |
US6167417A (en) * | 1998-04-08 | 2000-12-26 | Sarnoff Corporation | Convolutive blind source separation using a multiple decorrelation method |
US6898612B1 (en) * | 1998-11-12 | 2005-05-24 | Sarnoff Corporation | Method and system for on-line blind source separation |
JP2001166025A (en) * | 1999-12-14 | 2001-06-22 | Matsushita Electric Ind Co Ltd | Sound source direction estimating method, sound collection method and device |
US6879952B2 (en) * | 2000-04-26 | 2005-04-12 | Microsoft Corporation | Sound source separation using convolutional mixing and a priori sound source knowledge |
AU2001261344A1 (en) * | 2000-05-10 | 2001-11-20 | The Board Of Trustees Of The University Of Illinois | Interference suppression techniques |
US7076433B2 (en) * | 2001-01-24 | 2006-07-11 | Honda Giken Kogyo Kabushiki Kaisha | Apparatus and program for separating a desired sound from a mixed input sound |
EP1415506A2 (en) | 2001-08-10 | 2004-05-06 | Rasmussen Digital APS | Sound processing system that exhibits arbitrary gradient response |
US7088831B2 (en) * | 2001-12-06 | 2006-08-08 | Siemens Corporate Research, Inc. | Real-time audio source separation by delay and attenuation compensation in the time domain |
US20030199857A1 (en) * | 2002-04-17 | 2003-10-23 | Dornier Medtech Systems Gmbh | Apparatus and method for manipulating acoustic pulses |
US7146014B2 (en) * | 2002-06-11 | 2006-12-05 | Intel Corporation | MEMS directional sensor system |
GB0229473D0 (en) * | 2002-12-18 | 2003-01-22 | Qinetiq Ltd | Signal separation system and method |
DE602004029867D1 (en) * | 2003-03-04 | 2010-12-16 | Nippon Telegraph & Telephone | POSITION INFORMATION IMPRESSION DEVICE, METHOD THEREFOR AND PROGRAM |
KR100486736B1 (en) * | 2003-03-31 | 2005-05-03 | 삼성전자주식회사 | Method and apparatus for blind source separation using two sensors |
US7505902B2 (en) * | 2004-07-28 | 2009-03-17 | University Of Maryland | Discrimination of components of audio signals based on multiscale spectro-temporal modulations |
JP4449871B2 (en) * | 2005-01-26 | 2010-04-14 | ソニー株式会社 | Audio signal separation apparatus and method |
JP2007129373A (en) | 2005-11-01 | 2007-05-24 | Univ Waseda | Method and system for adjusting sensitivity of microphone |
JP5098176B2 (en) * | 2006-01-10 | 2012-12-12 | カシオ計算機株式会社 | Sound source direction determination method and apparatus |
US7885688B2 (en) * | 2006-10-30 | 2011-02-08 | L-3 Communications Integrated Systems, L.P. | Methods and systems for signal selection |
-
2007
- 2007-10-19 GB GBGB0720473.8A patent/GB0720473D0/en not_active Ceased
-
2008
- 2008-10-17 WO PCT/GB2008/003538 patent/WO2009050487A1/en active Application Filing
- 2008-10-17 US US12/734,195 patent/US9093078B2/en active Active
- 2008-10-17 EP EP08806629.5A patent/EP2203731B1/en active Active
Non-Patent Citations (4)
Title |
---|
BANU GUNEL ET AL: "Acoustic Source Separation of Convolutive Mixtures Based on Intensity Vector Statistics", IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, IEEE SERVICE CENTER, NEW YORK, NY, US, vol. 16, no. 4, 1 May 2008 (2008-05-01), pages 748 - 756, XP011204999, ISSN: 1558-7916 * |
MERIMAA J ET AL: "SPATIAL IMPULSE RESPONSE RENDERING I: ANALYSIS AND SYNTHESIS", JOURNAL OF THE AUDIO ENGINEERING SOCIETY, AUDIO ENGINEERING SOCIETY, NEW YORK, NY, US, vol. 53, no. 12, 1 December 2005 (2005-12-01), pages 1115 - 1127, XP001243409, ISSN: 1549-4950 * |
MITIANOUDIS N ET AL: "Batch and Online Underdetermined Source Separation Using Laplacian Mixture Models", IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, IEEE SERVICE CENTER, NEW YORK, NY, US, vol. 15, no. 6, 1 August 2007 (2007-08-01), pages 1818 - 1832, XP011187715, ISSN: 1558-7916 * |
NIKOLAOS MITIANOUDIS ET AL: "Underdetermined Source Separation Using Mixtures of Warped Laplacians", INDEPENDENT COMPONENT ANALYSIS AND SIGNAL SEPARATION; [LECTURE NOTES IN COMPUTER SCIENCE], SPRINGER BERLIN HEIDELBERG, BERLIN, HEIDELBERG, vol. 4666, 9 September 2007 (2007-09-09), pages 236 - 243, XP019080818, ISBN: 978-3-540-74493-1 * |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2013144609A1 (en) | 2012-03-26 | 2013-10-03 | University Of Surrey | Acoustic source separation |
KR20150107589A (en) * | 2012-03-26 | 2015-09-23 | 유니버시티 오브 서레이 | Acoustic source separation |
US9473841B2 (en) | 2012-03-26 | 2016-10-18 | University Of Surrey | Acoustic source separation |
KR102049620B1 (en) | 2012-03-26 | 2019-11-27 | 유니버시티 오브 서레이 | Directional Sound Receiving System |
WO2014177855A1 (en) | 2013-04-29 | 2014-11-06 | University Of Surrey | Microphone array for acoustic source separation |
US9774953B2 (en) | 2013-04-29 | 2017-09-26 | University Of Surrey | Microphone array for acoustic source separation |
EP3011762B1 (en) * | 2013-06-18 | 2020-04-22 | Dolby Laboratories Licensing Corporation | Adaptive audio content generation |
Also Published As
Publication number | Publication date |
---|---|
WO2009050487A8 (en) | 2009-07-09 |
EP2203731B1 (en) | 2018-01-10 |
US20110015924A1 (en) | 2011-01-20 |
EP2203731A1 (en) | 2010-07-07 |
US9093078B2 (en) | 2015-07-28 |
GB0720473D0 (en) | 2007-11-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2203731B1 (en) | Acoustic source separation | |
Gunel et al. | Acoustic source separation of convolutive mixtures based on intensity vector statistics | |
JP5657127B2 (en) | Apparatus and method for obtaining direction information, system, and computer program | |
Teutsch et al. | Acoustic source detection and localization based on wavefield decomposition using circular microphone arrays | |
Mohan et al. | Localization of multiple acoustic sources with small arrays using a coherence test | |
KR101442446B1 (en) | Sound acquisition via the extraction of geometrical information from direction of arrival estimates | |
KR101555416B1 (en) | Apparatus and method for spatially selective sound acquisition by acoustic triangulation | |
KR101591220B1 (en) | Apparatus and method for microphone positioning based on a spatial power density | |
KR102357287B1 (en) | Apparatus, Method or Computer Program for Generating a Sound Field Description | |
EP1571875A2 (en) | A system and method for beamforming using a microphone array | |
Dey et al. | Direction of arrival estimation and localization of multi-speech sources | |
Salvati et al. | A low-complexity robust beamforming using diagonal unloading for acoustic source localization | |
Huleihel et al. | Spherical array processing for acoustic analysis using room impulse responses and time-domain smoothing | |
Herzog et al. | Direction preserving wiener matrix filtering for ambisonic input-output systems | |
Benesty et al. | Array beamforming with linear difference equations | |
Corey et al. | Motion-tolerant beamforming with deformable microphone arrays | |
Mabande et al. | On 2D localization of reflectors using robust beamforming techniques | |
Firoozabadi et al. | Combination of nested microphone array and subband processing for multiple simultaneous speaker localization | |
Dey et al. | Microphone array principles | |
Berkun et al. | A tunable beamformer for robust superdirective beamforming | |
Jin et al. | Ray space analysis with sparse recovery | |
Sun et al. | Design of experimental adaptive beamforming system utilizing microphone array | |
Kavruk | Two stage blind dereverberation based on stochastic models of speech and reverberation | |
Guillaume et al. | Sound field analysis based on analytical beamforming | |
Riaz | Adaptive blind source separation based on intensity vector statistics |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 08806629 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2008806629 Country of ref document: EP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 12734195 Country of ref document: US |