US20170011753A1 - Methods And Apparatus For Adaptive Gain Control In A Communication System - Google Patents
Methods And Apparatus For Adaptive Gain Control In A Communication System Download PDFInfo
- Publication number
- US20170011753A1 US20170011753A1 US15/115,804 US201415115804A US2017011753A1 US 20170011753 A1 US20170011753 A1 US 20170011753A1 US 201415115804 A US201415115804 A US 201415115804A US 2017011753 A1 US2017011753 A1 US 2017011753A1
- Authority
- US
- United States
- Prior art keywords
- gain
- snr
- noise
- speech
- further including
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 230000006854 communication Effects 0.000 title claims abstract description 33
- 238000004891 communication Methods 0.000 title claims abstract description 33
- 238000000034 method Methods 0.000 title claims abstract description 20
- 230000003044 adaptive effect Effects 0.000 title description 2
- 238000001514 detection method Methods 0.000 claims description 51
- 230000000694 effects Effects 0.000 claims description 44
- 238000012545 processing Methods 0.000 claims description 21
- 230000008859 change Effects 0.000 claims description 15
- 230000003595 spectral effect Effects 0.000 claims description 13
- 230000003247 decreasing effect Effects 0.000 claims description 6
- 230000004044 response Effects 0.000 claims description 6
- 230000001131 transforming effect Effects 0.000 claims description 2
- 102100026436 Regulator of MON1-CCZ1 complex Human genes 0.000 description 7
- 101710180672 Regulator of MON1-CCZ1 complex Proteins 0.000 description 7
- 230000003321 amplification Effects 0.000 description 7
- 238000004590 computer program Methods 0.000 description 7
- 238000003199 nucleic acid amplification method Methods 0.000 description 7
- 230000001629 suppression Effects 0.000 description 7
- 230000002457 bidirectional effect Effects 0.000 description 6
- 230000008878 coupling Effects 0.000 description 6
- 238000010168 coupling process Methods 0.000 description 6
- 238000005859 coupling reaction Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 230000006978 adaptation Effects 0.000 description 4
- 230000001419 dependent effect Effects 0.000 description 4
- 230000004807 localization Effects 0.000 description 4
- 230000002829 reductive effect Effects 0.000 description 4
- 238000009499 grossing Methods 0.000 description 3
- 230000002123 temporal effect Effects 0.000 description 3
- 230000002411 adverse Effects 0.000 description 2
- 230000007175 bidirectional communication Effects 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000000873 masking effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0316—Speech enhancement, e.g. noise reduction or echo cancellation by changing the amplitude
- G10L21/0324—Details of processing therefor
- G10L21/034—Automatic adjustment
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L21/0216—Noise filtering characterised by the method used for estimating noise
- G10L21/0232—Processing in the frequency domain
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/78—Detection of presence or absence of voice signals
- G10L25/81—Detection of presence or absence of voice signals for discriminating voice from music
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03G—CONTROL OF AMPLIFICATION
- H03G3/00—Gain control in amplifiers or frequency changers
- H03G3/20—Automatic control
- H03G3/30—Automatic control in amplifiers having semiconductor devices
- H03G3/3089—Control of digital or coded signals
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03G—CONTROL OF AMPLIFICATION
- H03G3/00—Gain control in amplifiers or frequency changers
- H03G3/20—Automatic control
- H03G3/30—Automatic control in amplifiers having semiconductor devices
- H03G3/32—Automatic control in amplifiers having semiconductor devices the control being dependent upon ambient noise level or sound level
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
- H04R3/005—Circuits for transducers, loudspeakers or microphones for combining the signals of two or more microphones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
- H04R3/12—Circuits for transducers, loudspeakers or microphones for distributing signals to two or more loudspeakers
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0316—Speech enhancement, e.g. noise reduction or echo cancellation by changing the amplitude
- G10L21/0324—Details of processing therefor
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/78—Detection of presence or absence of voice signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2430/00—Signal processing covered by H04R, not provided for in its groups
- H04R2430/01—Aspects of volume control, not necessarily automatic, in sound systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2499/00—Aspects covered by H04R or H04S not otherwise provided for in their subgroups
- H04R2499/10—General applications
- H04R2499/13—Acoustic transducers and sound field adaptation in vehicles
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R27/00—Public address systems
Definitions
- ICC in-car communication
- Conventional systems can include microphones and loudspeakers to receive and transmit sound within the car.
- an ICC system can enable a driver to focus on the road while talking to passengers in the rear of the car without turning towards a passenger. Due to the driver's forward-facing head orientation, the driver may have to speak loudly in order to compensate for acoustic loss. For the reverse problem of having a conversation from rear to front, the problem is less dominant but still present.
- a communication system includes a series of microphones for picking up the speech of a speaker by performing speech enhancement processing.
- the system can play back the enhanced speech signal via loudspeakers dedicated to the listener.
- the system provides a relatively constant audio impression for a variety of noise and driving conditions.
- a communication system in accordance with exemplary embodiments of the invention optimizes gain control to achieve an improved audio impression.
- a target function for gain control can take psychoacoustic effects into consideration, provide improved gain adaptation, and enable a relatively compact implementation. With this arrangement, a system is provided with considerably reduced tuning effort and more natural audio impression as compared with conventional systems.
- a communication system fuses AGC (automatic gain control), which compensates for speaker-dependent speech volumes, and NDGC (noise dependent gain control), which provides voice amplification in adverse environments.
- AGC automatic gain control
- NDGC noise dependent gain control
- loss control which is used to switch between front and rear passengers, can also be fused into a unitary gain control module that uses a target function for gain control.
- a communication system provides a relatively constant audio impression for listeners given by a predefined Signal-To-Noise (SNR) target range at the listener's position.
- SNR Signal-To-Noise
- AGC and NDGC are combined with loss control, a flexible gain provides fast switching in cross-talk scenarios and smooth transitions when the background noise changes.
- a more natural audio impression can be achieved by using psychoacoustic measures to determine the actual SNR and the target SNR.
- a method comprises: for a communication system having microphones and loudspeakers, transforming a signal received by a first one of the microphones to the frequency domain; determining a noise level estimate for the transformed signal; determining a speech level estimate for the transformed signal; determining a SNR from the noise and speech level estimates; and determining a gain for the transformed signal to achieve a selected SNR range at a given position, wherein determining the gain comprises: adapting an actual gain to follow a target gain, wherein the target gain is adjusted to achieve the selected SNR range; comparing the target gain and the actual gain to determine a gain change increment; increasing the actual gain if the SNR at the given position is lower than a minimum SNR in the SNR range; and decreasing the actual gain if the SNR at the given position is higher than a maximum SNR in the SNR range.
- the method can further include one or more of the following features: adapting the SNR range in response to a change in the noise level, using a psychoacoustic measure for determining the noise and/or speech level estimate, determining a noise level estimate for the transformed signal using spectral weights based upon the psychoacoustic measure, performing voice activity detection on the transformed signal and generating weighting factors from the voice activity detection to detect an active speaker, determining mixer weights to select a speaker-specific gain for the active speaker, attenuating the gain during speech pauses detected by the voice activity detection, determining the gain for different frequencies, in a bi-directional system, setting a maximum gain to block a loudspeaker output of an inactive communication system, using a first one of the microphones to determine the SNR at the given position, and/or the given position corresponds to a position proximate an expected location of a user ear.
- an article comprises: a non-transitory computer-readable medium having stored instructions that enable a machine to: for a communication system having microphones and loudspeakers, transform a signal received by a first one of the microphones to the frequency domain; determine a noise level estimate for the transformed signal; determine a speech level estimate for the transformed signal; determine a SNR from the noise and speech level estimates; and determine a gain for the transformed signal to achieve a selected SNR range at a given position, wherein determining the gain comprises: adapt an actual gain to follow a target gain, wherein the target gain is adjusted to achieve the selected SNR range; compare the target gain and the actual gain to determine a gain change increment; increase the actual gain if the SNR at the given position is lower than a minimum SNR in the SNR range; and decrease the actual gain if the SNR at the given position is higher than a maximum SNR in the SNR range.
- the article can further include one or more of the following features: instructions to adapt the SNR range in response to a change in the noise level, instructions to use a psychoacoustic measure for determining the noise and/or speech level estimate, instructions to perform voice activity detection on the transformed signal and generate weighting factors from the voice activity detection to detect an active speaker, instructions to determine mixer weights to select a speaker-specific gain for the active speaker, and/or, in a bi-directional system, instructions to set a maximum gain to block a loudspeaker output of an inactive communication system.
- a communication system comprises: microphones to receive sound in an enclosed environment; loudspeakers to generate sound into the enclosed environment; a sound processing module to transform a signal received by a first one of the microphones to the frequency domain; a noise estimate module to determine a noise level estimate for the transformed signal; a speech estimate module to determine a speech level estimate for the transformed signal; a gain control module comprising: a SNR module to determine a SNR from the noise and speech level estimates; and a gain module to determine a gain for the transformed signal to achieve a selected SNR range at a given position, the gain module including a processor configured to: adapt an actual gain to follow a target gain, wherein the target gain is adjusted to achieve the selected SNR range; compare the target gain and the actual gain to determine a gain change increment; increase the actual gain if the SNR at the given position is lower than a minimum SNR in the SNR range; and/or decrease the actual gain if the SNR at the given position is higher than a maximum SNR in
- the system can further be configured to include one or more of the following features: the gain module is further configured to adapt the SNR range in response to a change in the noise level, use a psychoacoustic measure for determining the noise and/or speech level estimate, determine a noise level estimate for the transformed signal using spectral weights based upon the psychoacoustic measure, perform voice activity detection on the transformed signal and generate weighting factors from the voice activity detection to detect an active speaker, determine mixer weights to select a speaker-specific gain for the active speaker, attenuate the gain during speech pauses detected by the voice activity detection, determine the gain for different frequencies, in a bi-directional system, set a maximum gain to block a loudspeaker output of an inactive communication system, use a first one of the microphones to determine the SNR at the given position, and/or the given position corresponds to a position proximate an expected location of a user ear.
- FIG. 1 is a schematic representation of a communication system having gain control in accordance with exemplary embodiments of the invention
- FIG. 2A is a schematic representation of a unidirectional in car communication system having gain control
- FIG. 2B is a schematic representation of a bidirectional in car communication system having gain control
- FIG. 3 is a schematic representation of a communication system having microphones and loudspeakers and gain control
- FIG. 4 is a schematic representation of a gain control module that can form a part of the system of FIG. 3 ;
- FIG. 5 is a flow diagram of an exemplary sequence of steps for implementing gain control.
- FIG. 6 is a schematic representation of a speech signal enhancement system having exemplary robust speaker activity detection that can be used in the gain control system of FIG. 3 ;
- FIG. 7 is a schematic representation of an exemplary microphone selection system having robust speaker activity detection.
- FIG. 8 is a schematic representation of an exemplary computer that can perform at least a portion of the processing described herein.
- FIG. 1 shows an exemplary communication system 100 including a speech signal enhancement system 102 having a gain control module 104 in accordance with exemplary embodiments of the invention.
- a microphone array 106 includes one or more microphones 106 a -N to receive sound information, such as speech from a human speaker. It is understood that any practical number of microphones 106 can be used to form a microphone array.
- Respective pre-processing modules 108 a -N can process information from the microphones 106 a -N.
- Exemplary pre-processing modules 108 can include echo cancellation.
- Additional signal processing modules can include beamforming 110 , noise suppression 112 , wind noise suppression 114 , transient removal 116 , speaker/voice activity detection 118 , etc.
- FIG. 2A shows a unidirectional in car communication (ICC) system 150 .
- a vehicle 152 includes a series of loudspeakers 154 and microphones 156 within the passenger compartment.
- the passenger compartment includes a microphone 156 for each passenger.
- each passenger has a microphone array.
- the system 150 should achieve a relatively constant audio impression for a variety of noise and driving conditions.
- One or more microphones 156 are directed to the driver and the front passenger. Speech enhancement is applied to the microphone 156 signals, as described more fully below.
- FIG. 2B shows a vehicle 152 ′ having first and second ICC systems 150 a,b to provide bidirectional operation.
- a loss control module 158 can be provided to apply appropriate attenuation for the inactive ICC system.
- Bidirectional communication is provided in which speech signals originating from rear speakers are picked up by additional microphones 157 in the rear of the car.
- the speech signal is enhanced and amplified before it is played back over front loudspeakers 155 .
- Front loudspeakers 155 and rear microphones 157 are added over the unidirectional embodiment of FIG. 2A
- Both front-to-rear and rear-to-front communication are typically calculated more or less independently from each other by the respective ICC systems 150 a , 150 b .
- the first and second ICC systems 150 a,b are used in parallel.
- only one direction is active in order to avoid instabilities caused by feedback from loudspeakers of one direction into the microphones of the opposite direction.
- bidirectional communication is more convenient for rear and front passengers.
- Using more microphones allows retrieval of more precise information about the acoustic environment within the car.
- Having only one ICC system 150 a,b active at any given time avoids the speech of rear passengers being picked up by front microphones and played back again to the rear passengers.
- the ICC system 150 compensates for acoustic loss between passengers in a car or other enclosed space.
- the communication system 150 can use a number of microphones 156 for picking up the speech of a speaker, performing speech enhancement, and playing back the enhanced speech signal via loudspeakers 154 dedicated to the listener.
- Voice amplification can compensate for the acoustic loss from the speaker's mouth to the listener's ear.
- signal gain is controlled depending on the speaker's speech volume and the surrounding background noise level. Appropriate equalizing ensures stability of the system and sound quality.
- the enhanced speech signal is played back over the rear loudspeakers 154 of the car. Equalization and gain control preserve the natural localization of the speaker. The listener should still have the impression that the speech signal is coming from the speaker, e.g. driver or front passenger, and not from the loudspeaker. However, the speaker does not want to be aware of the speaker's voice played back over the loudspeakers.
- FIG. 3 shows an exemplary communication system 300 having gain control 302 in accordance with exemplary embodiments of the invention.
- Signals from microphones 304 a,b are first high-pass filtered to suppress low-frequency background noise (not shown).
- the filtered microphone 304 signal is then transformed by a filter bank into frequency domain, such as by FFT processing modules 306 a,b .
- the microphone noise levels are tracked in respective noise estimation modules 308 a,b.
- Noise suppression modules 310 a,b can calculate frequency domain spectral weighting coefficients to moderate sibilant sounds (de-essing) and to suppress background noise, for example, in the microphone signals.
- De-essing refers to any technique for reducing excessive prominence of sibilant consonants, such as “s”, “z” and “sh” in recordings of the human voice. Excessive sibilance can be caused by compression, microphone choice and technique, and the like. Sibilance typically lies in frequencies between 2-10 kHz.
- Speech estimation modules 312 a,b receive the noise suppressed microphone signals and output respective speech estimate signals that are provided to the gain control module 302 and respective voice activity detection modules 314 a,b .
- Generation of speech estimate signals can be performed using any suitable technique known to one of ordinary skill in the art.
- a mixer 316 detects which speaker, e.g. driver or front-passenger, is talking.
- Feedback suppression detects residual feedback artifacts (from loudspeaker to microphone) and applies temporal attenuation if required.
- the gain control module 302 provides Automatic Gain Control (AGC) and Noise Dependent Gain Control (NDGC) to control the overall amplification of the speaker's speech signal.
- AGC Automatic Gain Control
- NDGC Noise Dependent Gain Control
- the enhanced signal is then split into a number of output channels each of which is coupled to a respective equalizer 320 .
- Noise estimation 308 and speech level estimation 312 are calculated in frequency domain to allow spectral weighting.
- noise estimation 308 includes the use of spectral weights according to the ITU-R 468 standard, which is based on psychoacoustic experiments under the objective to obtain aurally compensated noise estimates.
- speech level estimation 312 uses so-called A-/B-/C-weighting in order to approximate the well-known equal loudness contours in a manner known in the art.
- the aggressiveness of noise suppression may be controlled by the noise estimate 308 including spectral weighting to achieve a good compromise between speech quality and masking background noise. It is understood that any suitable speech and noise level estimation techniques can be used.
- speech and noise level are jointly estimated in a voice activity detection module.
- the enhanced signal is transferred back into time domain 318 , equalized 320 , and then played back over the loudspeaker(s) 322 .
- Exemplary embodiments of the invention provide AGC, NDGC, and loss control.
- automatic gain control attenuates the speech signal of loud speakers.
- speakers typically speak louder in adverse environments, e.g., high noise situations, known as the Lombard effect.
- the AGC module tracks speech energy and determines whether the speech volume is still within an acceptable range. If the speech volume exceeds a certain limit, the ICC gain is reduced relatively rapidly. Otherwise, attenuation is slowly reduced.
- Noise Dependent Gain Control provides a constant audio impression for listeners. Since the background noise level in automotive environments is highly time variant, the ICC system adapts voice amplification continuously. Beyond a given noise threshold, the ICC gain is continuously increased according to configurable NDGC characteristics. NDGC is expected to react relatively slowly to moderate changes in background noise.
- Loss control is used in a bidirectional ICC system having two or more independent ICC systems. Loss control avoids closed-loop amplification of both ICC systems by applying appropriate attenuation for the inactive ICC system. Loss control ensures fast switching in case of speaker changes between speakers in the front and the rear of a car. Loss control ensures that only one ICC system is active at a given time.
- Exemplary embodiments of the invention fuse AGC, NDGC and loss control in a gain control module that uses a target function to provide a relatively constant audio impression for listeners given by a selected SNR target range at the listener's position.
- a natural audio impression is achieved by using psychoacoustic measures to determine the actual SNR and the target SNR. The fact that a listener hears both the loudspeaker output signal and the direct sound from the speaker can be taken into consideration since the superposition of both signals affects some other psychoacoustic effects such as acoustic localization.
- FIG. 4 shows an exemplary gain control module 400 , which can be provided as the gain control module 302 of FIG. 3 .
- speakers typically differ in speech volume, which complicates designing an ICC system that supports all speakers in the same manner. With respect to system stability, speakers who speak loudly can be more problematic. On the other hand, one can expect that loud speakers are more intelligible for other passengers since they achieve better Signal-to-Noise Ratios (SNR). In this case less amplification of the system is required.
- SNR Signal-to-Noise Ratios
- Noise estimate modules 308 a,b ( FIG. 3 ) provide respective noise estimate signals p N Mic0 , p N Mic1 for the signals from the first and second microphones 304 a,b .
- Speech estimate modules 312 a,b provide speech estimate signals p s Mic0 , p s Mic1 for the signals from the first and second microphones 304 a,b .
- the weighting factors ⁇ Mixer and the input signal sig In are provided by the signal mixer 316 .
- the voice activity detection (VAD) modules 314 a,b provide respective signals for the first and second microphones b VAD Mic0 , b VAD Mic1 to indicate which speaker, e.g., driver or front passenger, is active.
- the gain control module 400 provides an output signal sig out that can be equalized across the various channels, as described above.
- the speech level estimate signals p s Mic0 , p s Mic1 are buffered 402 a,b for each speaker separately.
- noise and speech estimates can be adapted independently during speech pauses/speech utterances.
- the gain 404 a,b is calculated to achieve a predetermined SNR range 406 a,b at the listener's ear.
- the VAD tags b VAD Mic0 , b VAD Mic1 can be used to attenuate the ICC gain during long speech pauses to avoid high amplification of residual background noise or non-stationary noise.
- the mixer weights ⁇ Mixer determine which speaker-specific gain should be used which can be useful in cross-talk situations.
- a target function instead of calculating AGC based on speech level and NDGC based on noise level independently, a target function achieves an approximately constant SNR range with respect to the listener's ear position. To avoid mismatches between perceived and measured SNRs, the use of psychoacoustic measures may be helpful, as noted above.
- the coupling factors can be used for speaker-to-microphone, speaker-to-listener and loudspeaker-to-listener and the corresponding delays, in order to estimate the SNR at the listener's ear position based on the SNR measured at the microphone.
- a model of the audio localization can be created according to the Haas effect. Both help achieve a constant audio impression for the listener and to improve the audio localization for speaker and listener.
- this is achieved by an appropriate upper limit of the ICC gain.
- the microphones dedicated to the listener can be used in order to measure the SNR directly.
- a bidirectional system such as the system shown in FIG. 2B , can use the microphone configuration to estimate the ratio of direct sound and loudspeaker output signal.
- a loss control module can evaluate the power ratio of the microphone signal and the direct sound recorded by the listener's microphone in case of single-talk scenarios. In one embodiment, the system measures this ratio a priori and allows some adaptation in a predetermined range. For example, this can be used to detect whether the driver turns his head while speaking. This would increase the direct sound and lower the signal power measured at the microphone.
- a target gain and an actual gain are used.
- Actual gain refers to the gain which is directly applied to the speech signal.
- the actual gain is continuously adapted to follow the target gain.
- the target gain is calculated as follows:
- G target ( ⁇ ) min ⁇ G ( ⁇ ), G max ( ⁇ ) ⁇ (1)
- G denotes the gain necessary to achieve the target SNR.
- G[dB] SNR_target[dB] ⁇ SNR_measured[dB] so that the SNR estimates include knowledge about speech level and background noise, as well as the psychoacoustic weighting.
- G is limited to obtain the target gain which should be achieved.
- Equation (2) below defines the adaptation scheme if conditions (3) and (4) are fulfilled.
- the gain can be calculated for each frequency separately or it can be averaged over all frequencies or a given frequency band.
- the maximum gain Gmax is the limit where the feedback from loudspeaker to microphone severely affects system stability and sound quality.
- the limit Gmax can be controlled adaptively. For example, if instabilities are detected, one could reduce the gain only for a few frequencies instead of reducing the gain for all frequencies. Spectral weighting factors of other modules such as noise suppression can also be taken into account.
- the actual gain is adapted step-by-step until the target gain G target is reached in order to achieve smooth gain transitions.
- Target gain and actual gain are compared to obtain the gain increment/decrement defined below:
- the gain increment is applied to the actual gain.
- the ICC gain will react slightly on small log-ratios but will adapt rapidly on large values.
- the first effect stabilizes the audio impression in relatively time-invariant acoustic environments.
- the second effect minimizes the delay of gain control in case of rapid changes.
- the gain increases only if the SNR at the listener's ear position is lower than the predefined SNR range:
- the gain should be decreased only if the SNR is too high:
- This arrangement combines NDGC and AGC in one gain control module.
- a SNR range [SNRmin; SNRmax] is useful to preserve natural SNR fluctuations during speech utterances.
- the SNR range is adapted when the noise level increases to reflect the Lombard effect.
- FIG. 5 shows an exemplary sequence of steps for implementing gain control.
- step 500 target and actual gain are compared. From the comparison, a gain change increment is determined in step 502 .
- step 504 it is determined whether the SNR is less than a minimum SNR of the SNR range. If so, in step 506 the gain is incremented by the computed gain change increment. If not, in step 508 it is determined whether the SNR is greater than a maximum SNR of the SNR range. If so, in step 510 the gain is decreased by the computed gain change increment. After any gain increase/decrease, optional loss control is performed in step 512 .
- a bidirectional system such as that shown in FIG. 2B , requires gain control for the multiple ICC systems 150 a,b .
- an appropriate upper limit Gmax is set in accordance with Equation (1) to block the loudspeaker output of the inactive ICC system. A large difference of actual and target gain results in rapid adaptation and small delay.
- any suitable speaker activity detection can be used for the VAD module 314 a,b of FIG. 3 . It is further understood that SAD and VAD are herein used interchangeably.
- An exemplary SAD system is shown and described in T. Matheja, M. Buck, and T. Fingscheidt, entitled “Speaker Activity Detection for Distributed Microphone Systems in Cars,” Proc. of the 6th Biennial Workshop on Digital Signal Processing for In-Vehicle Systems, September, 2013, which is incorporated herein by reference.
- FIG. 6 An exemplary SAD system is shown in FIG. 6 , which shows an exemplary speech signal enhancement system 600 having a speaker activity detection (SAD) module 602 and an event detection module 604 coupled to a robust speaker detection module 606 that provides information to a speech enhancement module 608 .
- the event detection module 604 includes at least one of a local noise detection module 650 , a wind noise detection module 652 , a diffuse sound detection module 654 , and a double-talk detection module 656 .
- the basic speaker activity detection (SAD) module 602 output is combined with outputs from one or more of the event detection modules 650 , 652 , 654 , 656 to avoid a possible positive SAD result during interfering sound events.
- a robust SAD result can be used for further speech enhancement 608 .
- robust SAD refers to a preliminary SAD evaluated against at least one event type so that the event does not result in a false SAD indication, wherein the event types include one or more of local noise, wind noise, diffuse sound, and/or double-talk.
- the local noise detection module 650 detects local distortions by evaluation of the spectral flatness of the difference between signal powers across the microphones, such as based on the signal power ratio.
- the spectral flatness measure in channel m for ⁇ tilde over (K) ⁇ subbands can be provided as:
- SAD Energy-based speaker activity detection
- SPR signal power ratio
- Temporal smoothing of the spectral flatness with ⁇ SF can be provided during speaker activity (S ⁇ D m (l)>0) and decreasing with ⁇ dec SF when there is not speaker activity as set forth below:
- ⁇ _ m , K ⁇ SF ⁇ ( ⁇ ) ⁇ ⁇ SF ⁇ ⁇ _ m , K ⁇ SF ⁇ ( ⁇ - 1 ) + ( 1 - ⁇ SF ) ⁇ ⁇ m , K ⁇ SF ⁇ ( ⁇ ) , if ⁇ ⁇ m ⁇ ( ⁇ ) > 0 , ⁇ dec SF ⁇ ⁇ _ m , K ⁇ SF ⁇ ( ⁇ - 1 ) , else . ( 6 )
- the smoothed spectral flatness can be thresholded to determine whether local noise is detected.
- LND Local Noise Detection
- K whole frequency range
- ⁇ LND threshold ⁇ LND
- LND m ⁇ ( ⁇ ) ⁇ 1 , if ⁇ ⁇ ⁇ _ m , K ⁇ SF ⁇ ( ⁇ ) > ⁇ LND , 0 , else . ( 7 )
- the wind noise detection module 650 thresholds the smoothed spectral flatness using a selected maximum frequency for wind.
- Wind noise detection (WND) in channel m with ⁇ tilde over (K) ⁇ being the number of subbands up to, e.g., 2000 Hz and the threshold ⁇ WND can be expressed as:
- WND m ⁇ ( ⁇ ) ⁇ 1 , if ⁇ ⁇ ( ⁇ ⁇ _ m , K ⁇ SF ⁇ ( ⁇ ) > ⁇ WND ) ⁇ ( LND m ⁇ ( ⁇ ) ⁇ 1 ) , 0 , else . ( 8 )
- the diffuse sound detection module 354 indicates regions where diffuse sound sources may be active that might harm the speaker activity detection. Diffuse sounds are detected if the power across the microphones is similar.
- the diffuse sound detection module is based on the speaker activity detection measure ⁇ m SAD (l). To detect diffuse events a certain positive threshold has to be exceeded by this measure in all of the available channels, whereas ⁇ m SAD (l) has to be always lower than a second higher threshold.
- the double-talk module 356 estimates the maximum speaker activity detection measure based on the speaker activity detection measure ⁇ m SAD (l) set forth above, with an increasing constant ⁇ inc ⁇ applied during fullband speaker activity if the current maximum is smaller than the currently observed SAD measure.
- the decreasing constant ⁇ dec ⁇ is applied otherwise, as set forth below.
- Double talk detection is indicated if more than one channel shows a smoothed maximum measure of speaker activity larger than a threshold ⁇ DTD , as follows:
- ⁇ _ DTD ⁇ ( ⁇ ) ⁇ ⁇ DTD ⁇ ⁇ _ DTD ⁇ ( ⁇ - 1 ) + ( 1 - ⁇ DTD ) , if ⁇ ⁇ DTD ⁇ ( ⁇ ) > 0 , ⁇ DTD ⁇ ⁇ _ DTD ⁇ ( ⁇ - 1 ) , else . ( 13 )
- FIG. 7 shows an exemplary microphone selection system 700 to select a microphone channel using information from a SNR module 702 , an event detection module 704 , which can be similar to the event detection module 604 of FIG. 6 , and a robust SAD module 706 , which can be similar to the robust SAD module 606 of FIG. 6 , all of which are coupled to a channel selection module 708 .
- a first microphone select/signal mixer 710 which receives input from M driver microphones, for example, is coupled to the channel selection module 708 .
- a second microphone select/signal mixer 712 which receives input from M passenger microphones, for example, is coupled to the channel selection module 708 .
- the channel selection module 708 selects the microphone channel prior to any signal enhancement processing.
- an intelligent signal mixer combines the input channels to an enhanced output signal. By selecting the microphone channel prior to signal enhancement, significant processing resources are saved in comparison with signal processing of all the microphone channels.
- the SNR calculation module 702 can estimate SNRs for related microphones.
- the channel selection module 708 receives information from the event detection module 704 , the robust SAD module 706 and the SNR module 702 . If the event of local disturbances is detected locally on a single microphone, that microphone should be excluded from the selection. If there is no local distortion, the signal with the best SNR should be selected. In general, for this decision, the speaker should have been active.
- the two selected signals, one driver microphone and one passenger microphone can be passed to a further signal processing module (not shown), that can include noise suppression, for example. Since not all channels need to be processed by the signal enhancement module, the amount of processing resources required is significantly reduced.
- speech communication between driver and passenger is supported by picking up the speaker's voice over microphones on the seat belt or other structure, and playing the speaker's voice back over loudspeakers close to the other passenger. If a microphone is hidden or distorted, another microphone on the belt can be selected. For each of the driver and passenger, only the ‘best’ microphone will be further processed.
- signal powers/spectra ⁇ SS can be compared pairwise, e.g., symmetric microphone arrangements for two speakers in a car with three microphones on each seat belts, for example.
- the top microphone m for the driver Dr can be compared to the top microphone of the passenger Pa, and similarly for the middle microphones and the lower microphones, as set forth below:
- a signal power ratio (SPR) for the microphones is used:
- the expected power can be used to detect wind noise, such as if the actual power exceeds the expected power considerably.
- specific coupling factors can be observed and evaluated, such as the coupling factors K above.
- the power ratios of different microphones are coupled in case of a speaker, where this coupling is not given in case of local distortions, e.g. wind or scratch noise.
- FIG. 8 shows an exemplary computer 800 that can perform at least part of the processing described herein.
- the computer 800 includes a processor 802 , a volatile memory 804 , a non-volatile memory 806 (e.g., hard disk), an output device 807 and a graphical user interface (GUI) 808 (e.g., a mouse, a keyboard, a display, for example).
- the non-volatile memory 806 stores computer instructions 812 , an operating system 816 and data 818 .
- the computer instructions 812 are executed by the processor 802 out of volatile memory 804 .
- an article 820 comprises non-transitory computer-readable instructions.
- Processing may be implemented in hardware, software, or a combination of the two. Processing may be implemented in computer programs executed on programmable computers/machines that each includes a processor, a storage medium or other article of manufacture that is readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and one or more output devices. Program code may be applied to data entered using an input device to perform processing and to generate output information.
- the system can perform processing, at least in part, via a computer program product, (e.g., in a machine-readable storage device), for execution by, or to control the operation of, data processing apparatus (e.g., a programmable processor, a computer, or multiple computers).
- a computer program product e.g., in a machine-readable storage device
- data processing apparatus e.g., a programmable processor, a computer, or multiple computers.
- Each such program may be implemented in a high level procedural or object-oriented programming language to communicate with a computer system.
- the programs may be implemented in assembly or machine language.
- the language may be a compiled or an interpreted language and it may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.
- a computer program may be deployed to be executed on one computer or on multiple computers at one site or distributed across multiple sites and interconnected by a communication network.
- a computer program may be stored on a storage medium or device (e.g., CD-ROM, hard disk, or magnetic diskette) that is readable by a general or special purpose programmable computer for configuring and operating the computer when the storage medium or device is read by the computer.
- Processing may also be implemented as a machine-readable storage medium, configured with a computer program, where upon execution, instructions in the computer program cause the computer to operate.
- Processing may be performed by one or more programmable processors executing one or more computer programs to perform the functions of the system. All or part of the system may be implemented as, special purpose logic circuitry (e.g., an FPGA (field programmable gate array) and/or an ASIC (application-specific integrated circuit)).
- special purpose logic circuitry e.g., an FPGA (field programmable gate array) and/or an ASIC (application-specific integrated circuit)
Landscapes
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- General Health & Medical Sciences (AREA)
- Otolaryngology (AREA)
- Circuit For Audible Band Transducer (AREA)
Abstract
Description
- As is known in the art, certain communication systems, such as in-car communication (ICC) systems, enable passengers in a car to communicate with each other in a more comfortable manner. Conventional systems can include microphones and loudspeakers to receive and transmit sound within the car. For example an ICC system can enable a driver to focus on the road while talking to passengers in the rear of the car without turning towards a passenger. Due to the driver's forward-facing head orientation, the driver may have to speak loudly in order to compensate for acoustic loss. For the reverse problem of having a conversation from rear to front, the problem is less dominant but still present.
- In exemplary embodiments of the invention, a communication system includes a series of microphones for picking up the speech of a speaker by performing speech enhancement processing. The system can play back the enhanced speech signal via loudspeakers dedicated to the listener. The system provides a relatively constant audio impression for a variety of noise and driving conditions.
- In general, a communication system in accordance with exemplary embodiments of the invention optimizes gain control to achieve an improved audio impression. A target function for gain control can take psychoacoustic effects into consideration, provide improved gain adaptation, and enable a relatively compact implementation. With this arrangement, a system is provided with considerably reduced tuning effort and more natural audio impression as compared with conventional systems.
- In exemplary embodiments, a communication system fuses AGC (automatic gain control), which compensates for speaker-dependent speech volumes, and NDGC (noise dependent gain control), which provides voice amplification in adverse environments. In some embodiments, loss control, which is used to switch between front and rear passengers, can also be fused into a unitary gain control module that uses a target function for gain control.
- In one embodiment, a communication system provides a relatively constant audio impression for listeners given by a predefined Signal-To-Noise (SNR) target range at the listener's position. When AGC and NDGC are combined with loss control, a flexible gain provides fast switching in cross-talk scenarios and smooth transitions when the background noise changes. A more natural audio impression can be achieved by using psychoacoustic measures to determine the actual SNR and the target SNR.
- In one aspect of the invention, a method comprises: for a communication system having microphones and loudspeakers, transforming a signal received by a first one of the microphones to the frequency domain; determining a noise level estimate for the transformed signal; determining a speech level estimate for the transformed signal; determining a SNR from the noise and speech level estimates; and determining a gain for the transformed signal to achieve a selected SNR range at a given position, wherein determining the gain comprises: adapting an actual gain to follow a target gain, wherein the target gain is adjusted to achieve the selected SNR range; comparing the target gain and the actual gain to determine a gain change increment; increasing the actual gain if the SNR at the given position is lower than a minimum SNR in the SNR range; and decreasing the actual gain if the SNR at the given position is higher than a maximum SNR in the SNR range.
- The method can further include one or more of the following features: adapting the SNR range in response to a change in the noise level, using a psychoacoustic measure for determining the noise and/or speech level estimate, determining a noise level estimate for the transformed signal using spectral weights based upon the psychoacoustic measure, performing voice activity detection on the transformed signal and generating weighting factors from the voice activity detection to detect an active speaker, determining mixer weights to select a speaker-specific gain for the active speaker, attenuating the gain during speech pauses detected by the voice activity detection, determining the gain for different frequencies, in a bi-directional system, setting a maximum gain to block a loudspeaker output of an inactive communication system, using a first one of the microphones to determine the SNR at the given position, and/or the given position corresponds to a position proximate an expected location of a user ear.
- In another aspect of the invention, an article comprises: a non-transitory computer-readable medium having stored instructions that enable a machine to: for a communication system having microphones and loudspeakers, transform a signal received by a first one of the microphones to the frequency domain; determine a noise level estimate for the transformed signal; determine a speech level estimate for the transformed signal; determine a SNR from the noise and speech level estimates; and determine a gain for the transformed signal to achieve a selected SNR range at a given position, wherein determining the gain comprises: adapt an actual gain to follow a target gain, wherein the target gain is adjusted to achieve the selected SNR range; compare the target gain and the actual gain to determine a gain change increment; increase the actual gain if the SNR at the given position is lower than a minimum SNR in the SNR range; and decrease the actual gain if the SNR at the given position is higher than a maximum SNR in the SNR range.
- The article can further include one or more of the following features: instructions to adapt the SNR range in response to a change in the noise level, instructions to use a psychoacoustic measure for determining the noise and/or speech level estimate, instructions to perform voice activity detection on the transformed signal and generate weighting factors from the voice activity detection to detect an active speaker, instructions to determine mixer weights to select a speaker-specific gain for the active speaker, and/or, in a bi-directional system, instructions to set a maximum gain to block a loudspeaker output of an inactive communication system.
- In a further aspect of the invention, a communication system comprises: microphones to receive sound in an enclosed environment; loudspeakers to generate sound into the enclosed environment; a sound processing module to transform a signal received by a first one of the microphones to the frequency domain; a noise estimate module to determine a noise level estimate for the transformed signal; a speech estimate module to determine a speech level estimate for the transformed signal; a gain control module comprising: a SNR module to determine a SNR from the noise and speech level estimates; and a gain module to determine a gain for the transformed signal to achieve a selected SNR range at a given position, the gain module including a processor configured to: adapt an actual gain to follow a target gain, wherein the target gain is adjusted to achieve the selected SNR range; compare the target gain and the actual gain to determine a gain change increment; increase the actual gain if the SNR at the given position is lower than a minimum SNR in the SNR range; and/or decrease the actual gain if the SNR at the given position is higher than a maximum SNR in the SNR range.
- The system can further be configured to include one or more of the following features: the gain module is further configured to adapt the SNR range in response to a change in the noise level, use a psychoacoustic measure for determining the noise and/or speech level estimate, determine a noise level estimate for the transformed signal using spectral weights based upon the psychoacoustic measure, perform voice activity detection on the transformed signal and generate weighting factors from the voice activity detection to detect an active speaker, determine mixer weights to select a speaker-specific gain for the active speaker, attenuate the gain during speech pauses detected by the voice activity detection, determine the gain for different frequencies, in a bi-directional system, set a maximum gain to block a loudspeaker output of an inactive communication system, use a first one of the microphones to determine the SNR at the given position, and/or the given position corresponds to a position proximate an expected location of a user ear.
- The foregoing features of this invention, as well as the invention itself, may be more fully understood from the following description of the drawings in which:
-
FIG. 1 is a schematic representation of a communication system having gain control in accordance with exemplary embodiments of the invention; -
FIG. 2A is a schematic representation of a unidirectional in car communication system having gain control; -
FIG. 2B is a schematic representation of a bidirectional in car communication system having gain control; -
FIG. 3 is a schematic representation of a communication system having microphones and loudspeakers and gain control; -
FIG. 4 is a schematic representation of a gain control module that can form a part of the system ofFIG. 3 ; -
FIG. 5 is a flow diagram of an exemplary sequence of steps for implementing gain control; and -
FIG. 6 is a schematic representation of a speech signal enhancement system having exemplary robust speaker activity detection that can be used in the gain control system ofFIG. 3 ; -
FIG. 7 is a schematic representation of an exemplary microphone selection system having robust speaker activity detection; and -
FIG. 8 is a schematic representation of an exemplary computer that can perform at least a portion of the processing described herein. -
FIG. 1 shows anexemplary communication system 100 including a speechsignal enhancement system 102 having again control module 104 in accordance with exemplary embodiments of the invention. A microphone array 106 includes one or more microphones 106 a-N to receive sound information, such as speech from a human speaker. It is understood that any practical number of microphones 106 can be used to form a microphone array. Respective pre-processing modules 108 a-N can process information from the microphones 106 a-N. Exemplary pre-processing modules 108 can include echo cancellation. Additional signal processing modules can include beamforming 110,noise suppression 112,wind noise suppression 114,transient removal 116, speaker/voice activity detection 118, etc. - While exemplary embodiments of the invention are shown and described in conjunction with an in car communication (ICC) system, it is understood that embodiments of the invention are applicable to communication systems in general in which SNR-based gain control is desirable within an enclosed space.
-
FIG. 2A shows a unidirectional in car communication (ICC)system 150. Avehicle 152 includes a series ofloudspeakers 154 andmicrophones 156 within the passenger compartment. In one embodiment, the passenger compartment includes amicrophone 156 for each passenger. In another embodiment (not shown), each passenger has a microphone array. - In general, the
system 150 should achieve a relatively constant audio impression for a variety of noise and driving conditions. One ormore microphones 156 are directed to the driver and the front passenger. Speech enhancement is applied to themicrophone 156 signals, as described more fully below. -
FIG. 2B shows avehicle 152′ having first andsecond ICC systems 150 a,b to provide bidirectional operation. As described more fully below, aloss control module 158 can be provided to apply appropriate attenuation for the inactive ICC system. - Bidirectional communication is provided in which speech signals originating from rear speakers are picked up by
additional microphones 157 in the rear of the car. The speech signal is enhanced and amplified before it is played back overfront loudspeakers 155.Front loudspeakers 155 andrear microphones 157 are added over the unidirectional embodiment ofFIG. 2A - Both front-to-rear and rear-to-front communication are typically calculated more or less independently from each other by the
respective ICC systems second ICC systems 150 a,b are used in parallel. However, in exemplary embodiments, only one direction is active in order to avoid instabilities caused by feedback from loudspeakers of one direction into the microphones of the opposite direction. - In general, bidirectional communication is more convenient for rear and front passengers. Using more microphones allows retrieval of more precise information about the acoustic environment within the car. Having only one
ICC system 150 a,b active at any given time avoids the speech of rear passengers being picked up by front microphones and played back again to the rear passengers. - The
ICC system 150 compensates for acoustic loss between passengers in a car or other enclosed space. Thecommunication system 150 can use a number ofmicrophones 156 for picking up the speech of a speaker, performing speech enhancement, and playing back the enhanced speech signal vialoudspeakers 154 dedicated to the listener. Voice amplification can compensate for the acoustic loss from the speaker's mouth to the listener's ear. - As described more fully below, signal gain is controlled depending on the speaker's speech volume and the surrounding background noise level. Appropriate equalizing ensures stability of the system and sound quality. The enhanced speech signal is played back over the
rear loudspeakers 154 of the car. Equalization and gain control preserve the natural localization of the speaker. The listener should still have the impression that the speech signal is coming from the speaker, e.g. driver or front passenger, and not from the loudspeaker. However, the speaker does not want to be aware of the speaker's voice played back over the loudspeakers. -
FIG. 3 shows anexemplary communication system 300 havinggain control 302 in accordance with exemplary embodiments of the invention. Signals frommicrophones 304 a,b are first high-pass filtered to suppress low-frequency background noise (not shown). The filtered microphone 304 signal is then transformed by a filter bank into frequency domain, such as byFFT processing modules 306 a,b. In parallel, the microphone noise levels are tracked in respective noise estimation modules 308 a,b. - Noise suppression modules 310 a,b can calculate frequency domain spectral weighting coefficients to moderate sibilant sounds (de-essing) and to suppress background noise, for example, in the microphone signals. As is known in the art, De-essing refers to any technique for reducing excessive prominence of sibilant consonants, such as “s”, “z” and “sh” in recordings of the human voice. Excessive sibilance can be caused by compression, microphone choice and technique, and the like. Sibilance typically lies in frequencies between 2-10 kHz.
-
Speech estimation modules 312 a,b receive the noise suppressed microphone signals and output respective speech estimate signals that are provided to thegain control module 302 and respective voiceactivity detection modules 314 a,b. Generation of speech estimate signals can be performed using any suitable technique known to one of ordinary skill in the art. - A
mixer 316 detects which speaker, e.g. driver or front-passenger, is talking. Feedback suppression detects residual feedback artifacts (from loudspeaker to microphone) and applies temporal attenuation if required. - As described more fully below, the
gain control module 302 provides Automatic Gain Control (AGC) and Noise Dependent Gain Control (NDGC) to control the overall amplification of the speaker's speech signal. After transformation back to the time domain, such as by anIFFT module 318, the enhanced signal is then split into a number of output channels each of which is coupled to arespective equalizer 320. Each signal played back over a loudspeaker orloudspeaker pair 322 dedicated to the corresponding output channel. - Since Voice Activity Detection (VAD) 314 is used prior to gain
control 302, the system can handle speech utterances and speech pauses appropriately, e.g., by reducing the gain during long speech pauses. Noise estimation 308 and speech level estimation 312 are calculated in frequency domain to allow spectral weighting. In one embodiment, noise estimation 308 includes the use of spectral weights according to the ITU-R 468 standard, which is based on psychoacoustic experiments under the objective to obtain aurally compensated noise estimates. - In one embodiment, speech level estimation 312 uses so-called A-/B-/C-weighting in order to approximate the well-known equal loudness contours in a manner known in the art. The aggressiveness of noise suppression may be controlled by the noise estimate 308 including spectral weighting to achieve a good compromise between speech quality and masking background noise. It is understood that any suitable speech and noise level estimation techniques can be used. In one embodiment, speech and noise level are jointly estimated in a voice activity detection module.
- After
gain control 302, the enhanced signal is transferred back intotime domain 318, equalized 320, and then played back over the loudspeaker(s) 322. - Exemplary embodiments of the invention provide AGC, NDGC, and loss control. In exemplary embodiments of the invention, automatic gain control attenuates the speech signal of loud speakers. As is known in the art, speakers typically speak louder in adverse environments, e.g., high noise situations, known as the Lombard effect. In general, the AGC module tracks speech energy and determines whether the speech volume is still within an acceptable range. If the speech volume exceeds a certain limit, the ICC gain is reduced relatively rapidly. Otherwise, attenuation is slowly reduced.
- Noise Dependent Gain Control (NDGC) provides a constant audio impression for listeners. Since the background noise level in automotive environments is highly time variant, the ICC system adapts voice amplification continuously. Beyond a given noise threshold, the ICC gain is continuously increased according to configurable NDGC characteristics. NDGC is expected to react relatively slowly to moderate changes in background noise.
- Loss control is used in a bidirectional ICC system having two or more independent ICC systems. Loss control avoids closed-loop amplification of both ICC systems by applying appropriate attenuation for the inactive ICC system. Loss control ensures fast switching in case of speaker changes between speakers in the front and the rear of a car. Loss control ensures that only one ICC system is active at a given time.
- Exemplary embodiments of the invention fuse AGC, NDGC and loss control in a gain control module that uses a target function to provide a relatively constant audio impression for listeners given by a selected SNR target range at the listener's position. In one embodiment, a natural audio impression is achieved by using psychoacoustic measures to determine the actual SNR and the target SNR. The fact that a listener hears both the loudspeaker output signal and the direct sound from the speaker can be taken into consideration since the superposition of both signals affects some other psychoacoustic effects such as acoustic localization.
-
FIG. 4 shows an exemplarygain control module 400, which can be provided as thegain control module 302 ofFIG. 3 . As noted above, speakers typically differ in speech volume, which complicates designing an ICC system that supports all speakers in the same manner. With respect to system stability, speakers who speak loudly can be more problematic. On the other hand, one can expect that loud speakers are more intelligible for other passengers since they achieve better Signal-to-Noise Ratios (SNR). In this case less amplification of the system is required. - Noise estimate modules 308 a,b (
FIG. 3 ) provide respective noise estimate signals pN Mic0, pN Mic1 for the signals from the first andsecond microphones 304 a,b.Speech estimate modules 312 a,b provide speech estimate signals ps Mic0, ps Mic1 for the signals from the first andsecond microphones 304 a,b. The weighting factors ωMixer and the input signal sigIn are provided by thesignal mixer 316. The voice activity detection (VAD)modules 314 a,b provide respective signals for the first and second microphones bVAD Mic0, bVAD Mic1 to indicate which speaker, e.g., driver or front passenger, is active. Thegain control module 400 provides an output signal sigout that can be equalized across the various channels, as described above. - In an exemplary embodiment, the speech level estimate signals ps Mic0, ps Mic1 are buffered 402 a,b for each speaker separately. In general, noise and speech estimates can be adapted independently during speech pauses/speech utterances. Based on noise and speech level estimates pN Mic0, pN Mic1, ps Mic0, ps Mic1, the
gain 404 a,b is calculated to achieve a predetermined SNR range 406 a,b at the listener's ear. In addition to a target SNR range, the VAD tags bVAD Mic0, bVAD Mic1 can be used to attenuate the ICC gain during long speech pauses to avoid high amplification of residual background noise or non-stationary noise. The mixer weights ωMixer determine which speaker-specific gain should be used which can be useful in cross-talk situations. - In exemplary embodiments of the invention, instead of calculating AGC based on speech level and NDGC based on noise level independently, a target function achieves an approximately constant SNR range with respect to the listener's ear position. To avoid mismatches between perceived and measured SNRs, the use of psychoacoustic measures may be helpful, as noted above.
- As long as the ICC gain is relatively small compared to the inverse of the coupling factor representing the feedback from loudspeaker to microphone, a proportional relation is expected between the increase in gain and the resulting SNR. When ICC gain approaches the maximum gain, it is expected that the resulting SNR reacts disproportionately high on increasing the ICC gain. For example, the coupling factors can be used for speaker-to-microphone, speaker-to-listener and loudspeaker-to-listener and the corresponding delays, in order to estimate the SNR at the listener's ear position based on the SNR measured at the microphone. Furthermore, a model of the audio localization can be created according to the Haas effect. Both help achieve a constant audio impression for the listener and to improve the audio localization for speaker and listener.
- In one embodiment, this is achieved by an appropriate upper limit of the ICC gain. In a bidirectional ICC system the microphones dedicated to the listener can be used in order to measure the SNR directly.
- A bidirectional system, such as the system shown in
FIG. 2B , can use the microphone configuration to estimate the ratio of direct sound and loudspeaker output signal. A loss control module can evaluate the power ratio of the microphone signal and the direct sound recorded by the listener's microphone in case of single-talk scenarios. In one embodiment, the system measures this ratio a priori and allows some adaptation in a predetermined range. For example, this can be used to detect whether the driver turns his head while speaking. This would increase the direct sound and lower the signal power measured at the microphone. - In an exemplary embodiment, a target gain and an actual gain are used. Actual gain refers to the gain which is directly applied to the speech signal. The actual gain is continuously adapted to follow the target gain. In one embodiment, the target gain is calculated as follows:
-
G target(Ω)=min{G(Ω),G max(Ω)} (1) - where G denotes the gain necessary to achieve the target SNR. For example: G[dB]=SNR_target[dB]−SNR_measured[dB] so that the SNR estimates include knowledge about speech level and background noise, as well as the psychoacoustic weighting. In equation (1) above, G is limited to obtain the target gain which should be achieved. Equation (2) below defines the adaptation scheme if conditions (3) and (4) are fulfilled.
- The gain can be calculated for each frequency separately or it can be averaged over all frequencies or a given frequency band. The maximum gain Gmax is the limit where the feedback from loudspeaker to microphone severely affects system stability and sound quality. In addition, the limit Gmax can be controlled adaptively. For example, if instabilities are detected, one could reduce the gain only for a few frequencies instead of reducing the gain for all frequencies. Spectral weighting factors of other modules such as noise suppression can also be taken into account.
- The actual gain is adapted step-by-step until the target gain Gtarget is reached in order to achieve smooth gain transitions. Target gain and actual gain are compared to obtain the gain increment/decrement defined below:
-
G Δ[dB]=G target[dB]−G actual[dB])·Δinc/dec (2) - The gain increment is applied to the actual gain. By using an adaptive gain, the ICC gain will react slightly on small log-ratios but will adapt rapidly on large values. The first effect stabilizes the audio impression in relatively time-invariant acoustic environments. The second effect minimizes the delay of gain control in case of rapid changes.
In one embodiment, the gain increases only if the SNR at the listener's ear position is lower than the predefined SNR range: - On the other hand, the gain should be decreased only if the SNR is too high:
- This arrangement combines NDGC and AGC in one gain control module. In addition, a SNR range [SNRmin; SNRmax] is useful to preserve natural SNR fluctuations during speech utterances. In one embodiment, the SNR range is adapted when the noise level increases to reflect the Lombard effect.
-
FIG. 5 shows an exemplary sequence of steps for implementing gain control. Instep 500, target and actual gain are compared. From the comparison, a gain change increment is determined instep 502. Instep 504, it is determined whether the SNR is less than a minimum SNR of the SNR range. If so, instep 506 the gain is incremented by the computed gain change increment. If not, instep 508 it is determined whether the SNR is greater than a maximum SNR of the SNR range. If so, instep 510 the gain is decreased by the computed gain change increment. After any gain increase/decrease, optional loss control is performed instep 512. - A bidirectional system, such as that shown in
FIG. 2B , requires gain control for themultiple ICC systems 150 a,b. In one embodiment, an appropriate upper limit Gmax is set in accordance with Equation (1) to block the loudspeaker output of the inactive ICC system. A large difference of actual and target gain results in rapid adaptation and small delay. - It is understood that any suitable speaker activity detection (SAD) can be used for the
VAD module 314 a,b ofFIG. 3 . It is further understood that SAD and VAD are herein used interchangeably. An exemplary SAD system is shown and described in T. Matheja, M. Buck, and T. Fingscheidt, entitled “Speaker Activity Detection for Distributed Microphone Systems in Cars,” Proc. of the 6th Biennial Workshop on Digital Signal Processing for In-Vehicle Systems, September, 2013, which is incorporated herein by reference. - An exemplary SAD system is shown in
FIG. 6 , which shows an exemplary speechsignal enhancement system 600 having a speaker activity detection (SAD)module 602 and anevent detection module 604 coupled to a robust speaker detection module 606 that provides information to aspeech enhancement module 608. In one embodiment, theevent detection module 604 includes at least one of a localnoise detection module 650, a windnoise detection module 652, a diffusesound detection module 654, and a double-talk detection module 656. - The basic speaker activity detection (SAD)
module 602 output is combined with outputs from one or more of theevent detection modules further speech enhancement 608. - It is understood that the term robust SAD refers to a preliminary SAD evaluated against at least one event type so that the event does not result in a false SAD indication, wherein the event types include one or more of local noise, wind noise, diffuse sound, and/or double-talk.
- In one embodiment, the local
noise detection module 650 detects local distortions by evaluation of the spectral flatness of the difference between signal powers across the microphones, such as based on the signal power ratio. The spectral flatness measure in channel m for {tilde over (K)} subbands, can be provided as: -
- Energy-based speaker activity detection (SAD) evaluates a signal power ratio (SPR) in each of M≧2 microphone channels. In embodiments, the processing is performed in the discrete Fourier transform domain with the frame index e and the frequency subband index k at a sampling rate of ƒs=16 kHz, for example.
- Temporal smoothing of the spectral flatness with γSF can be provided during speaker activity (SÃDm(l)>0) and decreasing with γdec SF when there is not speaker activity as set forth below:
-
- In one embodiment, the smoothed spectral flatness can be thresholded to determine whether local noise is detected. Local Noise Detection (LND) in channel m with {tilde over (K)}: whole frequency range and threshold ΘLND can be expressed as follows:
-
- In one embodiment, the wind
noise detection module 650 thresholds the smoothed spectral flatness using a selected maximum frequency for wind. Wind noise detection (WND) in channel m with {tilde over (K)} being the number of subbands up to, e.g., 2000 Hz and the threshold ΘWND can be expressed as: -
- It is understood that the maximum frequency, number of subbands, smoothing parameters, etc., can be varied to meet the needs of a particular application. It is further understood that other suitable wind detection techniques known to one of ordinary skill in the art can be used to detect wind noise.
- In an exemplary embodiment, the diffuse sound detection module 354 indicates regions where diffuse sound sources may be active that might harm the speaker activity detection. Diffuse sounds are detected if the power across the microphones is similar. The diffuse sound detection module is based on the speaker activity detection measure χm SAD(l). To detect diffuse events a certain positive threshold has to be exceeded by this measure in all of the available channels, whereas χm SAD(l) has to be always lower than a second higher threshold.
- In one embodiment, the double-talk module 356 estimates the maximum speaker activity detection measure based on the speaker activity detection measure χm SAD(l) set forth above, with an increasing constant γinc χ applied during fullband speaker activity if the current maximum is smaller than the currently observed SAD measure. The decreasing constant γdec χ is applied otherwise, as set forth below.
-
- Temporal smoothing of the speaker activity measure maximum can be provided with γSAD as follows:
-
χ max,m SAD(l)=γSAD·χ max,m SAD(l−1)+(1−γSAD)·{circumflex over (χ)}max,m SAD(l). (10) - Double talk detection (DTD) is indicated if more than one channel shows a smoothed maximum measure of speaker activity larger than a threshold ΘDTD, as follows:
-
- Here the function ƒ(x, Y) performs threshold decision:
-
- With the constant γDTDε{0, . . . , 1} we get a measure for detection of double-talk regions modified by an evaluation of whether double-talk has been detected for one frame:
-
- The detection of double-talk regions is followed by comparison with a threshold:
-
-
FIG. 7 shows an exemplarymicrophone selection system 700 to select a microphone channel using information from aSNR module 702, anevent detection module 704, which can be similar to theevent detection module 604 ofFIG. 6 , and arobust SAD module 706, which can be similar to the robust SAD module 606 ofFIG. 6 , all of which are coupled to achannel selection module 708. A first microphone select/signal mixer 710, which receives input from M driver microphones, for example, is coupled to thechannel selection module 708. Similarly, a second microphone select/signal mixer 712, which receives input from M passenger microphones, for example, is coupled to thechannel selection module 708. As described more fully below, thechannel selection module 708 selects the microphone channel prior to any signal enhancement processing. Alternatively, an intelligent signal mixer combines the input channels to an enhanced output signal. By selecting the microphone channel prior to signal enhancement, significant processing resources are saved in comparison with signal processing of all the microphone channels. - When a speaker is active, the
SNR calculation module 702 can estimate SNRs for related microphones. Thechannel selection module 708 receives information from theevent detection module 704, therobust SAD module 706 and theSNR module 702. If the event of local disturbances is detected locally on a single microphone, that microphone should be excluded from the selection. If there is no local distortion, the signal with the best SNR should be selected. In general, for this decision, the speaker should have been active. - In one embodiment, the two selected signals, one driver microphone and one passenger microphone can be passed to a further signal processing module (not shown), that can include noise suppression, for example. Since not all channels need to be processed by the signal enhancement module, the amount of processing resources required is significantly reduced.
- In one embodiment adapted for a convertible car with two passengers with in-car communication system, speech communication between driver and passenger is supported by picking up the speaker's voice over microphones on the seat belt or other structure, and playing the speaker's voice back over loudspeakers close to the other passenger. If a microphone is hidden or distorted, another microphone on the belt can be selected. For each of the driver and passenger, only the ‘best’ microphone will be further processed.
- Alternative embodiments can use a variety of ways to detect events and speaker activity in environments having multiple microphones per speaker. In one embodiment, signal powers/spectra ΦSS can be compared pairwise, e.g., symmetric microphone arrangements for two speakers in a car with three microphones on each seat belts, for example. The top microphone m for the driver Dr can be compared to the top microphone of the passenger Pa, and similarly for the middle microphones and the lower microphones, as set forth below:
- Events, such as wind noise or body noise, can be detected for each group of speaker-dedicated microphones individually. The speaker activity detection, however, uses both groups of microphones, excluding microphones that are distorted.
- In one embodiment, a signal power ratio (SPR) for the microphones is used:
-
- Equivalently, comparisons using a coupling factor K that maps the power of one microphone to the expected power of another microphone can be used, as set forth below:
- The expected power can be used to detect wind noise, such as if the actual power exceeds the expected power considerably. For speech activity of the passengers, specific coupling factors can be observed and evaluated, such as the coupling factors K above. The power ratios of different microphones are coupled in case of a speaker, where this coupling is not given in case of local distortions, e.g. wind or scratch noise.
-
FIG. 8 shows anexemplary computer 800 that can perform at least part of the processing described herein. Thecomputer 800 includes aprocessor 802, avolatile memory 804, a non-volatile memory 806 (e.g., hard disk), anoutput device 807 and a graphical user interface (GUI) 808 (e.g., a mouse, a keyboard, a display, for example). Thenon-volatile memory 806stores computer instructions 812, anoperating system 816 anddata 818. In one example, thecomputer instructions 812 are executed by theprocessor 802 out ofvolatile memory 804. In one embodiment, anarticle 820 comprises non-transitory computer-readable instructions. - Processing may be implemented in hardware, software, or a combination of the two. Processing may be implemented in computer programs executed on programmable computers/machines that each includes a processor, a storage medium or other article of manufacture that is readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and one or more output devices. Program code may be applied to data entered using an input device to perform processing and to generate output information.
- The system can perform processing, at least in part, via a computer program product, (e.g., in a machine-readable storage device), for execution by, or to control the operation of, data processing apparatus (e.g., a programmable processor, a computer, or multiple computers). Each such program may be implemented in a high level procedural or object-oriented programming language to communicate with a computer system. However, the programs may be implemented in assembly or machine language. The language may be a compiled or an interpreted language and it may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program may be deployed to be executed on one computer or on multiple computers at one site or distributed across multiple sites and interconnected by a communication network. A computer program may be stored on a storage medium or device (e.g., CD-ROM, hard disk, or magnetic diskette) that is readable by a general or special purpose programmable computer for configuring and operating the computer when the storage medium or device is read by the computer. Processing may also be implemented as a machine-readable storage medium, configured with a computer program, where upon execution, instructions in the computer program cause the computer to operate.
- Processing may be performed by one or more programmable processors executing one or more computer programs to perform the functions of the system. All or part of the system may be implemented as, special purpose logic circuitry (e.g., an FPGA (field programmable gate array) and/or an ASIC (application-specific integrated circuit)).
- Having described exemplary embodiments of the invention, it will now become apparent to one of ordinary skill in the art that other embodiments incorporating their concepts may also be used. The embodiments contained herein should not be limited to disclosed embodiments but rather should be limited only by the spirit and scope of the appended claims. All publications and references cited herein are expressly incorporated herein by reference in their entirety.
Claims (20)
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/US2014/018905 WO2015130283A1 (en) | 2014-02-27 | 2014-02-27 | Methods and apparatus for adaptive gain control in a communication system |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2014/018905 A-371-Of-International WO2015130283A1 (en) | 2014-02-27 | 2014-02-27 | Methods and apparatus for adaptive gain control in a communication system |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/671,830 Continuation US11798576B2 (en) | 2014-02-27 | 2019-11-01 | Methods and apparatus for adaptive gain control in a communication system |
Publications (1)
Publication Number | Publication Date |
---|---|
US20170011753A1 true US20170011753A1 (en) | 2017-01-12 |
Family
ID=54009456
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/115,804 Abandoned US20170011753A1 (en) | 2014-02-27 | 2014-02-27 | Methods And Apparatus For Adaptive Gain Control In A Communication System |
US16/671,830 Active 2035-09-04 US11798576B2 (en) | 2014-02-27 | 2019-11-01 | Methods and apparatus for adaptive gain control in a communication system |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/671,830 Active 2035-09-04 US11798576B2 (en) | 2014-02-27 | 2019-11-01 | Methods and apparatus for adaptive gain control in a communication system |
Country Status (3)
Country | Link |
---|---|
US (2) | US20170011753A1 (en) |
EP (1) | EP3103204B1 (en) |
WO (1) | WO2015130283A1 (en) |
Cited By (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160127561A1 (en) * | 2014-10-31 | 2016-05-05 | Imagination Technologies Limited | Automatic Tuning of a Gain Controller |
US20170055078A1 (en) * | 2015-08-20 | 2017-02-23 | Harman Becker Automotive Systems Gmbh | In-car communication |
US20170330565A1 (en) * | 2016-05-13 | 2017-11-16 | Bose Corporation | Handling Responses to Speech Processing |
US20170332168A1 (en) * | 2016-05-13 | 2017-11-16 | Bose Corporation | Processing Speech from Distributed Microphones |
US20190115018A1 (en) * | 2017-10-18 | 2019-04-18 | Motorola Mobility Llc | Detecting audio trigger phrases for a voice recognition session |
US10431237B2 (en) * | 2017-09-13 | 2019-10-01 | Motorola Solutions, Inc. | Device and method for adjusting speech intelligibility at an audio device |
US10600432B1 (en) * | 2017-03-28 | 2020-03-24 | Amazon Technologies, Inc. | Methods for voice enhancement |
US10741195B2 (en) * | 2016-02-15 | 2020-08-11 | Mitsubishi Electric Corporation | Sound signal enhancement device |
CN112544088A (en) * | 2018-08-02 | 2021-03-23 | 日本电信电话株式会社 | Sound pickup and amplification device, method thereof, and program |
US20210092694A1 (en) * | 2019-09-23 | 2021-03-25 | Mediatek Inc. | Wireless communication system using one or more throughput enhancement techniques |
CN112700785A (en) * | 2020-12-21 | 2021-04-23 | 苏州科达特种视讯有限公司 | Voice signal processing method and device and related equipment |
US11164592B1 (en) * | 2019-05-09 | 2021-11-02 | Amazon Technologies, Inc. | Responsive automatic gain control |
US20210343267A1 (en) * | 2020-04-29 | 2021-11-04 | Gulfstream Aerospace Corporation | Phased array speaker and microphone system for cockpit communication |
US11322170B2 (en) | 2017-10-02 | 2022-05-03 | Dolby Laboratories Licensing Corporation | Audio de-esser independent of absolute signal level |
US20220240009A1 (en) * | 2019-05-21 | 2022-07-28 | Nippon Telegraph And Telephone Corporation | Echo cancelation method, apparatus, program and recording medium |
US20230018804A1 (en) * | 2021-07-14 | 2023-01-19 | Alps Alpine Co., Ltd. | In-vehicle communication support system |
Families Citing this family (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10923132B2 (en) | 2016-02-19 | 2021-02-16 | Dolby Laboratories Licensing Corporation | Diffusivity based sound processing method and apparatus |
US10924872B2 (en) | 2016-02-23 | 2021-02-16 | Dolby Laboratories Licensing Corporation | Auxiliary signal for detecting microphone impairment |
US10867620B2 (en) | 2016-06-22 | 2020-12-15 | Dolby Laboratories Licensing Corporation | Sibilance detection and mitigation |
JP6489082B2 (en) * | 2016-08-08 | 2019-03-27 | オンキヨー株式会社 | Equalizer device and equalizer program |
WO2019170874A1 (en) | 2018-03-08 | 2019-09-12 | Sony Corporation | Electronic device, method and computer program |
CN111328008B (en) * | 2020-02-24 | 2021-11-05 | 广州市迪士普音响科技有限公司 | Sound pressure level intelligent control method based on sound amplification system |
CN115335901A (en) * | 2020-03-27 | 2022-11-11 | 杜比实验室特许公司 | Automatic leveling of speech content |
WO2024110035A1 (en) | 2022-11-24 | 2024-05-30 | Sas Bdsk | Appartus for ascending along a cable with back support provided with removable saddle and footrest element |
Citations (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040091123A1 (en) * | 2002-11-08 | 2004-05-13 | Stark Michael W. | Automobile audio system |
US20060262938A1 (en) * | 2005-05-18 | 2006-11-23 | Gauger Daniel M Jr | Adapted audio response |
US7171003B1 (en) * | 2000-10-19 | 2007-01-30 | Lear Corporation | Robust and reliable acoustic echo and noise cancellation system for cabin communication |
US20090028180A1 (en) * | 2007-07-27 | 2009-01-29 | General Instrument Corporation | Method and Apparatus for Mitigating Layer-2 Looping in Home Networking Applications |
US20100001978A1 (en) * | 2008-07-02 | 2010-01-07 | Stephen Brian Lynch | Ambient light interference reduction for optical input devices |
US20130017916A1 (en) * | 2011-07-11 | 2013-01-17 | Toyota Jidosha Kabushiki Kaisha | Vehicle drive device |
US8538763B2 (en) * | 2007-09-12 | 2013-09-17 | Dolby Laboratories Licensing Corporation | Speech enhancement with noise level estimation adjustment |
US8560320B2 (en) * | 2007-03-19 | 2013-10-15 | Dolby Laboratories Licensing Corporation | Speech enhancement employing a perceptual model |
US8583426B2 (en) * | 2007-09-12 | 2013-11-12 | Dolby Laboratories Licensing Corporation | Speech enhancement with voice clarity |
US8989403B2 (en) * | 2010-03-09 | 2015-03-24 | Mitsubishi Electric Corporation | Noise suppression device |
US9318119B2 (en) * | 2005-09-02 | 2016-04-19 | Nec Corporation | Noise suppression using integrated frequency-domain signals |
Family Cites Families (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
FI116643B (en) * | 1999-11-15 | 2006-01-13 | Nokia Corp | Noise reduction |
WO2001097558A2 (en) * | 2000-06-13 | 2001-12-20 | Gn Resound Corporation | Fixed polar-pattern-based adaptive directionality systems |
US7010118B2 (en) * | 2001-09-21 | 2006-03-07 | Agere Systems, Inc. | Noise compensation methods and systems for increasing the clarity of voice communications |
US8280730B2 (en) * | 2005-05-25 | 2012-10-02 | Motorola Mobility Llc | Method and apparatus of increasing speech intelligibility in noisy environments |
EP2120485B1 (en) * | 2008-04-28 | 2014-10-08 | Harman Becker Automotive Systems GmbH | Load detection |
US9336785B2 (en) * | 2008-05-12 | 2016-05-10 | Broadcom Corporation | Compression for speech intelligibility enhancement |
EP2151983B1 (en) * | 2008-08-07 | 2015-11-11 | Nuance Communications, Inc. | Hands-free telephony and in-vehicle communication |
US8229125B2 (en) * | 2009-02-06 | 2012-07-24 | Bose Corporation | Adjusting dynamic range of an audio system |
US20110095875A1 (en) * | 2009-10-23 | 2011-04-28 | Broadcom Corporation | Adjustment of media delivery parameters based on automatically-learned user preferences |
EP2416593A1 (en) | 2010-08-02 | 2012-02-08 | Svox AG | Method for indoor communication |
US20120076320A1 (en) * | 2010-09-28 | 2012-03-29 | Bose Corporation | Fine/Coarse Gain Adjustment |
EP2490459B1 (en) | 2011-02-18 | 2018-04-11 | Svox AG | Method for voice signal blending |
JP2013062609A (en) * | 2011-09-12 | 2013-04-04 | Yawata Electric Industrial Co Ltd | Broadcast sound volume controller and sound volume control method of the same |
US9293151B2 (en) | 2011-10-17 | 2016-03-22 | Nuance Communications, Inc. | Speech signal enhancement using visual information |
US9641934B2 (en) * | 2012-01-10 | 2017-05-02 | Nuance Communications, Inc. | In-car communication system for multiple acoustic zones |
US9407999B2 (en) * | 2013-02-04 | 2016-08-02 | University of Pittsburgh—of the Commonwealth System of Higher Education | System and method for enhancing the binaural representation for hearing-impaired subjects |
US9124234B1 (en) * | 2014-04-11 | 2015-09-01 | Entropic Communications, LLC. | Method and apparatus for adaptive automatic gain control |
-
2014
- 2014-02-27 US US15/115,804 patent/US20170011753A1/en not_active Abandoned
- 2014-02-27 WO PCT/US2014/018905 patent/WO2015130283A1/en active Application Filing
- 2014-02-27 EP EP14883804.8A patent/EP3103204B1/en active Active
-
2019
- 2019-11-01 US US16/671,830 patent/US11798576B2/en active Active
Patent Citations (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7171003B1 (en) * | 2000-10-19 | 2007-01-30 | Lear Corporation | Robust and reliable acoustic echo and noise cancellation system for cabin communication |
US20040091123A1 (en) * | 2002-11-08 | 2004-05-13 | Stark Michael W. | Automobile audio system |
US20060262938A1 (en) * | 2005-05-18 | 2006-11-23 | Gauger Daniel M Jr | Adapted audio response |
US9318119B2 (en) * | 2005-09-02 | 2016-04-19 | Nec Corporation | Noise suppression using integrated frequency-domain signals |
US8560320B2 (en) * | 2007-03-19 | 2013-10-15 | Dolby Laboratories Licensing Corporation | Speech enhancement employing a perceptual model |
US20090028180A1 (en) * | 2007-07-27 | 2009-01-29 | General Instrument Corporation | Method and Apparatus for Mitigating Layer-2 Looping in Home Networking Applications |
US8538763B2 (en) * | 2007-09-12 | 2013-09-17 | Dolby Laboratories Licensing Corporation | Speech enhancement with noise level estimation adjustment |
US8583426B2 (en) * | 2007-09-12 | 2013-11-12 | Dolby Laboratories Licensing Corporation | Speech enhancement with voice clarity |
US20100001978A1 (en) * | 2008-07-02 | 2010-01-07 | Stephen Brian Lynch | Ambient light interference reduction for optical input devices |
US8989403B2 (en) * | 2010-03-09 | 2015-03-24 | Mitsubishi Electric Corporation | Noise suppression device |
US20130017916A1 (en) * | 2011-07-11 | 2013-01-17 | Toyota Jidosha Kabushiki Kaisha | Vehicle drive device |
Cited By (29)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10244121B2 (en) * | 2014-10-31 | 2019-03-26 | Imagination Technologies Limited | Automatic tuning of a gain controller |
US20160127561A1 (en) * | 2014-10-31 | 2016-05-05 | Imagination Technologies Limited | Automatic Tuning of a Gain Controller |
US20170055078A1 (en) * | 2015-08-20 | 2017-02-23 | Harman Becker Automotive Systems Gmbh | In-car communication |
US10741195B2 (en) * | 2016-02-15 | 2020-08-11 | Mitsubishi Electric Corporation | Sound signal enhancement device |
US10149049B2 (en) * | 2016-05-13 | 2018-12-04 | Bose Corporation | Processing speech from distributed microphones |
US20170330563A1 (en) * | 2016-05-13 | 2017-11-16 | Bose Corporation | Processing Speech from Distributed Microphones |
US20170332168A1 (en) * | 2016-05-13 | 2017-11-16 | Bose Corporation | Processing Speech from Distributed Microphones |
US20170330564A1 (en) * | 2016-05-13 | 2017-11-16 | Bose Corporation | Processing Simultaneous Speech from Distributed Microphones |
US20170330565A1 (en) * | 2016-05-13 | 2017-11-16 | Bose Corporation | Handling Responses to Speech Processing |
US10600432B1 (en) * | 2017-03-28 | 2020-03-24 | Amazon Technologies, Inc. | Methods for voice enhancement |
US10431237B2 (en) * | 2017-09-13 | 2019-10-01 | Motorola Solutions, Inc. | Device and method for adjusting speech intelligibility at an audio device |
US11322170B2 (en) | 2017-10-02 | 2022-05-03 | Dolby Laboratories Licensing Corporation | Audio de-esser independent of absolute signal level |
US20190115018A1 (en) * | 2017-10-18 | 2019-04-18 | Motorola Mobility Llc | Detecting audio trigger phrases for a voice recognition session |
US10665234B2 (en) * | 2017-10-18 | 2020-05-26 | Motorola Mobility Llc | Detecting audio trigger phrases for a voice recognition session |
EP3813384A4 (en) * | 2018-08-02 | 2022-07-27 | Nippon Telegraph And Telephone Corporation | Sound collection/loudspeaker device, method therefor, and program |
EP4207802A1 (en) * | 2018-08-02 | 2023-07-05 | Nippon Telegraph And Telephone Corporation | Sound collection loudspeaker apparatus, method and program for the same |
JPWO2020027062A1 (en) * | 2018-08-02 | 2021-08-02 | 日本電信電話株式会社 | Sound collecting loudspeaker, its method, and program |
CN112544088A (en) * | 2018-08-02 | 2021-03-23 | 日本电信电话株式会社 | Sound pickup and amplification device, method thereof, and program |
US11482234B2 (en) * | 2018-08-02 | 2022-10-25 | Nippon Telegraph And Telephone Corporation | Sound collection loudspeaker apparatus, method and program for the same |
US11164592B1 (en) * | 2019-05-09 | 2021-11-02 | Amazon Technologies, Inc. | Responsive automatic gain control |
US11729550B2 (en) * | 2019-05-21 | 2023-08-15 | Nippon Telegraph And Telephone Corporation | Echo cancelation method, apparatus, program and recording medium |
US20220240009A1 (en) * | 2019-05-21 | 2022-07-28 | Nippon Telegraph And Telephone Corporation | Echo cancelation method, apparatus, program and recording medium |
US11889434B2 (en) * | 2019-09-23 | 2024-01-30 | Mediatek Inc. | Wireless communication system using one or more throughput enhancement techniques |
US20210092694A1 (en) * | 2019-09-23 | 2021-03-25 | Mediatek Inc. | Wireless communication system using one or more throughput enhancement techniques |
US20210343267A1 (en) * | 2020-04-29 | 2021-11-04 | Gulfstream Aerospace Corporation | Phased array speaker and microphone system for cockpit communication |
US11170752B1 (en) * | 2020-04-29 | 2021-11-09 | Gulfstream Aerospace Corporation | Phased array speaker and microphone system for cockpit communication |
CN112700785A (en) * | 2020-12-21 | 2021-04-23 | 苏州科达特种视讯有限公司 | Voice signal processing method and device and related equipment |
US20230018804A1 (en) * | 2021-07-14 | 2023-01-19 | Alps Alpine Co., Ltd. | In-vehicle communication support system |
US11956604B2 (en) * | 2021-07-14 | 2024-04-09 | Alps Alpine Co., Ltd. | In-vehicle communication support system |
Also Published As
Publication number | Publication date |
---|---|
WO2015130283A1 (en) | 2015-09-03 |
EP3103204B1 (en) | 2019-11-13 |
US11798576B2 (en) | 2023-10-24 |
EP3103204A4 (en) | 2017-08-23 |
US20200176012A1 (en) | 2020-06-04 |
EP3103204A1 (en) | 2016-12-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11798576B2 (en) | Methods and apparatus for adaptive gain control in a communication system | |
EP3040984B1 (en) | Sound zone arrangment with zonewise speech suppresion | |
EP3053356B1 (en) | Methods and apparatus for selective microphone signal combining | |
US8705753B2 (en) | System for processing sound signals in a vehicle multimedia system | |
US9076456B1 (en) | System and method for providing voice equalization | |
US8886525B2 (en) | System and method for adaptive intelligent noise suppression | |
US6674865B1 (en) | Automatic volume control for communication system | |
US7117145B1 (en) | Adaptive filter for speech enhancement in a noisy environment | |
US8111840B2 (en) | Echo reduction system | |
US8204253B1 (en) | Self calibration of audio device | |
US8165310B2 (en) | Dereverberation and feedback compensation system | |
US8724822B2 (en) | Noisy environment communication enhancement system | |
US9549250B2 (en) | Wind noise detection for in-car communication systems with multiple acoustic zones | |
US7039197B1 (en) | User interface for communication system | |
Schmidt et al. | Signal processing for in-car communication systems | |
US20150127351A1 (en) | Noise Dependent Signal Processing For In-Car Communication Systems With Multiple Acoustic Zones | |
US9626987B2 (en) | Speech enhancement apparatus and speech enhancement method | |
US20180047408A1 (en) | System and method for addressing acoustic signal reverberation | |
US20160073209A1 (en) | Maintaining spatial stability utilizing common gain coefficient | |
US11227622B2 (en) | Speech communication system and method for improving speech intelligibility | |
Schmidt et al. | Evaluation of in-car communication systems | |
US20220199105A1 (en) | Signal component estimation using coherence | |
Yu et al. | A data-driven post-filter design based on spatially and temporally smoothed a priori SNR | |
CN117995211A (en) | Voice communication compensation method and device, automobile, electronic equipment and storage medium | |
Arun | Efficient and Robust Acoustic Feedback Cancellation Algorithm FOR In-car communication system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: NUANCE COMMUNICATIONS, INC., MASSACHUSETTS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HERBIG, TOBIAS;PFEFFINGER, MEIK;ISER, BERND;REEL/FRAME:039538/0114 Effective date: 20140226 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
AS | Assignment |
Owner name: CERENCE INC., MASSACHUSETTS Free format text: INTELLECTUAL PROPERTY AGREEMENT;ASSIGNOR:NUANCE COMMUNICATIONS, INC.;REEL/FRAME:050836/0191 Effective date: 20190930 |
|
AS | Assignment |
Owner name: CERENCE OPERATING COMPANY, MASSACHUSETTS Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE NAME PREVIOUSLY RECORDED AT REEL: 050836 FRAME: 0191. ASSIGNOR(S) HEREBY CONFIRMS THE INTELLECTUAL PROPERTY AGREEMENT;ASSIGNOR:NUANCE COMMUNICATIONS, INC.;REEL/FRAME:050871/0001 Effective date: 20190930 |
|
AS | Assignment |
Owner name: BARCLAYS BANK PLC, NEW YORK Free format text: SECURITY AGREEMENT;ASSIGNOR:CERENCE OPERATING COMPANY;REEL/FRAME:050953/0133 Effective date: 20191001 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
AS | Assignment |
Owner name: CERENCE OPERATING COMPANY, MASSACHUSETTS Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:BARCLAYS BANK PLC;REEL/FRAME:052927/0335 Effective date: 20200612 |
|
AS | Assignment |
Owner name: WELLS FARGO BANK, N.A., NORTH CAROLINA Free format text: SECURITY AGREEMENT;ASSIGNOR:CERENCE OPERATING COMPANY;REEL/FRAME:052935/0584 Effective date: 20200612 |
|
AS | Assignment |
Owner name: CERENCE OPERATING COMPANY, MASSACHUSETTS Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE REPLACE THE CONVEYANCE DOCUMENT WITH THE NEW ASSIGNMENT PREVIOUSLY RECORDED AT REEL: 050836 FRAME: 0191. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT;ASSIGNOR:NUANCE COMMUNICATIONS, INC.;REEL/FRAME:059804/0186 Effective date: 20190930 |