EP1257146A2

EP1257146A2 - Method and system of sound processing

Info

Publication number: EP1257146A2
Application number: EP02009933A
Authority: EP
Inventors: David John Benjamin Pearce; James Alexander Rex
Original assignee: Motorola Inc
Current assignee: Motorola Solutions Inc
Priority date: 2001-05-03
Filing date: 2002-05-03
Publication date: 2002-11-13
Anticipated expiration: 2022-05-03
Also published as: DE60213155D1; PT1257146E; DE60213155T2; GB2375276A; EP1257146B1; GB2375276B; ATE333774T1; EP1257146A3; GB0110861D0

Abstract

A method and a system of enhancing an output sound signal quality, for use in portable sound processing devices, that comprises steps of receiving sound from sound source (412) by a microphone array (402) and detecting a movement of this microphone array. The movement is detected by a platform motion sensor (400). Current location of the sound source (412) relative to the microphone array is estimated in source location estimator (406) on the basis of the sound signal from the microfone array (402) and signals from the platform motion sensor (400). Results of this estimation are used by sound enhancing processor (408) for enhancing output sound signal quality.

Description

Technical Field

The present invention relates to sound signal processing. In particular, the invention relates to enhancing sound signal quality for use in a sound processing device.

Background

In known methods of enhancing sound signal quality, pick-up of noise and reverberation may be reduced by employing an array of remote microphones. The remote microphones' outputs are processed and then combined into a single signal that has an improved signal quality.
The best processing parameters to use in the microphone array depend on the location of the desired sound source relative to the microphones. Many arrays reject noise by having much higher sensitivity to sound coming from the direction (or region) of the desired source than to sounds from elsewhere, where noise sources are located. However, the source's relative location is often unknown initially, and may change over time. For example, people often move their heads whilst speaking, and if an array is in a hand-held device, it is quite likely to move somewhat during use.
The location of a sound source relative to a microphone array, according to methods known in the art, can be estimated by measuring certain characteristics of the signals received from that source by the array of microphones. An example of one such characteristic is cross-power spectral phase.
The other method of location of a sound source relative to a microphone array is estimation by visually imaging the source, with a camera mounted on the same base as the array. The camera image is processed to locate the source within it, using known visual characteristics of the source. For example, a speaker's mouth may be located by looking for his/her face in the image.
Granted US patent US-A-4984087 shows a prior art microphone apparatus for a video camera.

Summary of the Invention

It is an object of the present invention to provide novel methods and a system for enhancing sound signal quality which overcome the disadvantages of the prior art. The methods and system of the invention are for use in portable sound processing devices
The methods and system according to the present invention allow processing of sound signals that result in increased quality. The output signal has decreased content of acoustic noise and reverberation caused by surroundings. The output signal is also less sensitive to changes of location of a sound source relative to the microphones. These changes of location are caused both by movement of the sound source, and movement of the device equiped with these microphones. In a typical example, the sound source would be a person speaking and the device might be held in his/her hand.
In accordance with the present invention, there is thus provided a method for enhancing sound signal quality. The method comprises steps of receiving sound from sound source by a microphone array and detecting a movement of this microphone array. The movement is detected by a platform motion sensor. Current location of the sound source relative to the microphone array is estimated on the basis of the sound signal from the microphone array and signals from the platform motion sensor. Results of this estimation are used for enhancing output sound signal quality.
An advantage of the present invention is that the motion sensor can continously track the motion of the microphone array, and use this to adjust the current estimate of the relative location of the desired sound source. The changes of relative location of the sound source and microphone array can be tracked even when there is no sound signal. In contrast, the sound-based source location estimation can only track changes in source location when sound is present.
There are frequent silences during normal speech, not only between words or sentences, but also within words. If the microphone array was moved during such a silence, and speech then restarted, it would take a relatively long time for a sound-based source location estimator on its own to obtain an accurate estimate of the changed location. The result would be an initial reduction in output sound quality. The present invention, on the other hand, would use information from the motion sensor to continually adjust the source location estimate. Hence, when speech restarts after a silence, there is no initial reduction in output sound quality. This assumes that only the microphone array moves significantly, not the speech source.
In accordance with another aspect of the present invention, there is provided a system for enhancing sound signal quality. The system comprises a microphone array and a platform motion sensor that are connected to a source location estimator. Additionally the microphone array is connected to a sound enhancing processor. The sound enhancing processor is connected to a source location estimator.

Brief description of the drawings

The present invention will be understood and appreciated more fully from the following detailed description taken in conjunction with the drawings in which:
Figure 1 is a flowchart illustrating a method of preparing input signals for use in portable sound processing devices in accordance with an embodiment of the present invention;
Figure 2 is a flowchart illustrating a method of enhancing quality of output sound signals for use in portable sound processing devices in accordance with another embodiment of the present invention;
Figure 3 is a flowchart illustrating a further method of enhancing quality of output sound signals for use in portable sound processing devices in accordance with another embodiment of the present invention;
Figure 4 is a schematic illustration of a system for enhancing quality of output sound signals operative in accordance with the present invention.

Detailed description of the preferred embodiment

The term a "movement signal" herein below refers to a signal generated and provided by a platform motion sensor. A "microphone array" mentioned herein below refers to a set of microphones that are a part of a portable device. Such a portable device is typically in a user's hand, during use, and may be subject to various rotational and translational movements.
Referring to figures 1 and 4, in step 100 a sound from a sound source 412, which may be moving, is received by a microphone array 402. In step 102, a platform motion sensor 400 detects a movement of the microphone array 402. Afterwards, a sound signal is provided to a sound enhancing processor 408 and to a source location estimator 406. At the same time the movement signal is provided to the source location estimator 406.
In step 112 the sound enhancing processor 408 enhances the quality of the sound signal. This enhancement may be performed according to methods known in the art, and is not described further here.
In step 104, on the basis of this sound signal, an estimation of a location of the sound source 412 relative to the microphone array 402 is performed. For the purpose of this estimation, long time intervals of the sound signal are analysed. Simultaneously in step 106, on the basis of the movement signal, an estimation of a change of position of the microphone array 402 is performed. For the purpose of this estimation, short time intervals of the movement signal are analysed.
In step 108, on the basis of results of estimations performed in steps 104 and 106, a current change of source's relative location is estimated.
Finally in step 110, the estimated current location of the sound source 412 relative to the microphone array 402 is obtained, by combining the current change of source's relative location from step 108 with the location of the sound source 412 relative to the microphone array 402.
Results of the estimation obtained in step 110 and the enhanced signal received in step 112 can be used for further processing in other devices, e.g. speech recognition devices.
In Fig. 2, in step 200, a sound from a sound source 412, which may be moving, is received by a microphone array 402. In step 202, platform motion sensor 400 detects a movement of the microphone array 402. Afterwards, a sound signal is provided to a sound enhancing processor 408 and to a source location estimator 406. At the same time the movement signal is provided to the source location estimator 406.
In step 204, on the basis of this sound signal, an estimation of a location of the sound source 412 relative to the microphone array 402 is performed. For the purpose of this estimation, long time intervals of the sound signal are analysed. Results of this estimation are provided to the sound enhancing processor 408.
In step 212 the sound enhancing processor 408 enhances the quality of the sound signal using the results of the estimation performed in step 204.
Simultaneously in step 206, on the basis of the movement signal, an estimation of a change of position of the microphone array 402 is performed. For the purpose of this estimation, short time intervals of the movement signal are analysed.
In step 208, on the basis of results of estimations performed in steps 204 and 206, a current change of the source's relative location is estimated.
Finally, in step 210, an estimation of current location of the sound source 412 relative to the microphone array 402 is performed, by combining the current change of the source's relative location from step 208 with the location of the sound source 412 relative to the microphone array 402.
Results of the estimation obtained in step 210 and the enhanced signal received in step 212 can be used for further processing in other devices, e.g. speech recognition devices.
In fig. 3, in step 300 a sound from a sound source 412, which may be moving, is received by a microphone array 402. In step 302, platform motion sensor 400 detects a movement of the microphone array 402. Afterwards, a sound signal is provided to a sound enhancing processor 408 and to a source location estimator 406. At the same time, the movement signal is provided to the source location estimator 406.
In step 304, on the basis of this sound signal, an estimation of a location of the sound source 412 relative to the microphone array 402 is performed. For the purpose of this estimation, long time intervals of the sound signal are analysed. Simultaneously, in step 306, on the basis of the movement signal an estimation of a change of position of the microphone array 402 is performed. For the purpose of this estimation, short time intervals of the movement signal are analysed.
In step 308, on the basis of results of estimations performed in steps 304 and 306, a current change of the source's relative location is estimated.
Finally, in step 310, by combining the current change of source's relative location from step 308 with the location of the sound source 412 relative to the microphone array 402, the estimated current location of the sound source 412 relative to the microphone array 402 is obtained.
In step 312 the sound enhancing processor 408 enhances the quality of the sound signal using the results of the estimation performed in step 310.
The sound signal of enhanced quality obtained in step 312 can be used for further processing in other devices, e.g. speech recognition devices.
The long time interval referenced in steps 104, 204 and 304 is typically in the range from 10 ms up to 1 s.
The short time interval referenced in steps 106, 206 and 306 is typically in the range from 0.1 µs up to 10 µs.
Reference is now made to Fig. 4, which depicts a system for enhancing quality of an output sound signal, operative in accordance with the embodiment of the present invention as illustrated in Fig. 3.
A system for enhancing sound signal quality, according to the present invention, comprises a microphone array 402, that is connected to a sound enhancing processor 408, and to a source location estimator 406. The system also comprises a platform motion sensor 400 that is connected to the source location estimator 406.
In accordance with some embodiments of the present invention, the source location estimator 406 may be connected to the sound enhancing processor 408.
The microphone array 402 consists of two or more microphones 404. An array of at least two microphones can provide output signals that can be used to estimate both the direction and distance of a source of sound.
The platform motion sensor 400 comprises at least one gyroscope and/or accelerometer. The platform motion sensor 400 is fixed to the microphone array 402, or to any other part of the portable sound processing device 410 that forms one unit with the microphone array 402.
The source location estimator 406 and sound enhancing processor 408 have respectively a first signal output connection and a second signal output connection.
The invention uses the signal from platform motion sensor 400 to provide an indication of movements of the microphone array 402. Therefore it is necessary that sensor 400 and microphone array 402 be linked so that the signal from sensor 400 relates to the motion of microphone array 402. Such a link may simply include sensor 400 being attached to array 402, or both being attached to the same housing. Therefore, although figure 4 shows elements 400, 402, 406 and 408 all in one unit, elements 406 and 408 may in fact be located elsewhere.
A system in accordance with the invention, or the methods of the invention may be used in various portable devices. In particular, the invention is usable in portable radio communication devices. Therefore the system may be used in a mobile telephone or a portable or mobile PMR radio. The invention also may be used in a personal digital assistant (PDA) or laptop computer, linked for example by a radio or infra-red communication link to a cellular network. Such a network may be in a building, or a cellular telephone, or UMTS/3G network.
The invention may form part of a Distributed Speech Recognition (DSR) system. In such a system, some speech processing would be performed remotely, i.e. the processing would be carried out at at least two different locations.

Claims

A method of preparing input signals, for use in a portable sound processing device, the method comprising the steps of:

receiving (100) a sound from a sound source (412) by a microphone array (402);

detecting (102) a movement of said microphone array (402) by a platform motion sensor (400);

providing a sound signal from said microphone array (402) to a sound enhancing processor (408) and to a source location estimator (406);

providing a movement signal from said platform motion sensor (400) to said source location estimator (406);

said sound enhancing processor (408) enhancing (112) a sound signal quality;

said source location estimator (406) estimating a current location of said sound source (412) relative to said microphone array (402);

providing said enhanced sound signal to a sound processing device;

providing to said sound processing device a result of said estimation of said current location of said sound source relative to said microphone array.
A method of enhancing a sound signal quality, for use in a portable sound processing device, the method comprising the steps of:

receiving (300) a sound from a sound source (412) by a microphone array (402);

detecting (302) a movement of said microphone array (402) by a platform motion sensor (400);

providing a sound signal from said microphone array (402) to a sound enhancing processor (408) and to a source location estimator (406);

providing a movement signal from said platform motion sensor (400) to said source location estimator (406);

said source location estimator (406) estimating a current location of said sound source (412) relative to said microphone array (402);

providing a result of said estimation of said current location of said sound source (412) relative to said microphone array (402) to said sound enhancing processor (408);

said sound enhancing processor (408) enhancing (312) said sound signal quality;

providing said enhanced sound signal to a sound processing device.
A method according to claim 1 or claim 2, wherein said estimation of said current location of said sound source relative to said microphone array comprises the steps of:

estimating (104, 304) a location of said sound source relative to said microphone array on the basis of said sound signal;

estimating (106, 306) a change of position of said microphone array on the basis of said movement signal;

estimating (108, 308) a current change of a source's relative location on the basis of a result of said estimation of said location of said sound source relative to said microphone array and said estimation of said change of position of said microphone array;

combining (110, 310) said current change of said source's relative location with said location of said sound source relative to said microphone array.
A method of enhancing a sound signal quality, for use in a portable sound processing device, the method comprising the method of claim 1, and further comprising:

providing (212, 214) a result of said estimation of a location of said sound source (412) relative to said microphone array (402) to said sound enhancing processor (408).
A method according to claim 4 wherein said estimation of said current location of said sound source relative to said microphone array comprises the steps of:

estimating (206) a change of position of said microphone array on the basis of said movement signal from said platform motion sensor;

estimating (208) a current change of a source's relative location on the basis of a result of said estimation of said location of said sound source relative to said microphone array and said estimation of said change of position of said microphone array;

combining (210) said current change of said source's relative location with said location of said sound source relative to said microphone array.
A method according to claim 4 wherein said estimation (204) of a location of said sound source relative to said microphone array is performed on the basis of said sound signal only.
A method according to any previous claim wherein, for the purpose of said estimation of a location of said sound source relative to said microphone array, long time intervals of said sound signal are analysed, the long time intervals being preferably in a range from 10ms to 1s.
A method according to any previous claim wherein, for the purpose of said estimation of said change of position of said microphone array, short time intervals of said movement signal are analysed, the short time intervals being preferably in a range from 0.1µs to 10µs.
A method according to claim 2 wherein, for the purpose of said enhancing (312) of said sound signal quality, said result of said estimation of said current location of said sound source is used.
A system for enhancing sound signal quality, for use in a portable sound processing device (410), said system comprising:

a sound enhancing processor (408);

a source location estimator (406);

a microphone array (402), connected to said sound enhancing processor (408) and to said source location estimator (406);

a platform motion sensor (400), connected to said source location estimator (406).
A system according to claim 10, wherein said source location estimator (406) is connected to said sound enhancing processor.
A system according to claim 11, wherein said platform motion sensor (400) is fixed to said microphone array.
A method according to any of claims 10-12, wherein said sound processing device is a speech recognition module.
A mobile telephone, a portable or mobile PMR radio, a personal digital assistant (PDA), or a laptop computer according to any of claims 10-13, or being adapted to operate in accordance with the method of any of claims 1 - 9.