CN116760442A - Beam forming method, device, electronic equipment and storage medium - Google Patents
Beam forming method, device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN116760442A CN116760442A CN202310615720.8A CN202310615720A CN116760442A CN 116760442 A CN116760442 A CN 116760442A CN 202310615720 A CN202310615720 A CN 202310615720A CN 116760442 A CN116760442 A CN 116760442A
- Authority
- CN
- China
- Prior art keywords
- signals
- beam forming
- signal
- noise
- original array
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 87
- 239000011159 matrix material Substances 0.000 claims abstract description 85
- 238000001914 filtration Methods 0.000 claims abstract description 57
- 238000001228 spectrum Methods 0.000 claims description 78
- 238000012545 processing Methods 0.000 claims description 38
- 230000004044 response Effects 0.000 claims description 35
- 230000003044 adaptive effect Effects 0.000 claims description 19
- 238000000354 decomposition reaction Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 abstract description 23
- 230000001629 suppression Effects 0.000 abstract description 14
- 238000005516 engineering process Methods 0.000 abstract description 12
- 238000010586 diagram Methods 0.000 description 18
- 230000008569 process Effects 0.000 description 9
- 230000008901 benefit Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 5
- 230000005236 sound signal Effects 0.000 description 4
- 238000003491 array Methods 0.000 description 3
- 230000015572 biosynthetic process Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 230000001934 delay Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000008054 signal transmission Effects 0.000 description 2
- FGUUSXIOTUKUDN-IBGZPJMESA-N C1(=CC=CC=C1)N1C2=C(NC([C@H](C1)NC=1OC(=NN=1)C1=CC=CC=C1)=O)C=CC=C2 Chemical compound C1(=CC=CC=C1)N1C2=C(NC([C@H](C1)NC=1OC(=NN=1)C1=CC=CC=C1)=O)C=CC=C2 FGUUSXIOTUKUDN-IBGZPJMESA-N 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000005316 response function Methods 0.000 description 1
- 230000000452 restraining effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B7/00—Radio transmission systems, i.e. using radiation field
- H04B7/02—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas
- H04B7/04—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas
- H04B7/06—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas at the transmitting station
- H04B7/0613—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas at the transmitting station using simultaneous transmission
- H04B7/0615—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas at the transmitting station using simultaneous transmission of weighted versions of same signal
- H04B7/0617—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas at the transmitting station using simultaneous transmission of weighted versions of same signal for beam forming
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L25/00—Baseband systems
- H04L25/02—Details ; arrangements for supplying electrical power along data transmission lines
- H04L25/0202—Channel estimation
- H04L25/021—Estimation of channel covariance
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Power Engineering (AREA)
- Circuit For Audible Band Transducer (AREA)
Abstract
The application discloses a beam forming method, a beam forming device, electronic equipment and a storage medium, wherein the beam forming method comprises the following steps: receiving an original array voice signal, wherein the original array voice signal is a multi-channel signal; performing alignment operation on the original array voice signals to obtain aligned channel signals; and estimating a noise covariance matrix based on the aligned channel signals. The application improves the noise suppression capability by combining the alignment operation and the beam forming technology, thereby improving the spatial filtering effect in the beam forming based on the MVDR algorithm.
Description
Technical Field
The present application relates to the field of communications technologies, and in particular, to a beam forming method, a beam forming device, an electronic device, and a storage medium.
Background
With the continuous development of communication technology, there is an increasing demand for high quality signal transmission. In the existing microphone array processing algorithm, the wave beam formation can adjust the optimal weight coefficient to point to the expected signal according to the incoming wave direction of the sound source, and the null points to the interference and noise signals, so that the purposes of enhancing the signals and suppressing the noise are achieved.
The MVDR (Minimum Variance Distortionless Response, least mean square distortion response) algorithm is used as a mainstream algorithm commonly used in current beam forming, and can obtain an enhanced signal by calculating an optimal weight vector and carrying out weighted summation on multi-channel data. Based on the technical advantages, the method is very significant for the research of MVDR beam forming related technology.
MVDR algorithms require an estimation of the noise covariance and a steering vector, which is typically known information or estimated by a DOA (Direction Of Arrival ) module. In the prior art, the covariance matrix is estimated mainly by modeling a noise field, for example, the scattered noise field is a sine model, and although the method is simple, the noise covariance is difficult to accurately estimate, so that the noise suppression capability is weaker, and the space filtering effect is poorer.
Disclosure of Invention
The invention mainly aims to provide a beam forming method, a beam forming device, electronic equipment and a storage medium, aiming to improve noise suppression capability and further improve the spatial filtering effect in beam forming based on an MVDR algorithm.
To achieve the above object, the present invention provides a beam forming method comprising the steps of:
Receiving an original array voice signal, wherein the original array voice signal is a multi-channel signal;
performing alignment operation on the original array voice signals to obtain aligned channel signals;
and estimating a noise covariance matrix based on the aligned channel signals.
Optionally, the beam forming method further includes:
and performing adaptive beam forming operation on the aligned channel signals based on the estimated noise covariance matrix.
Optionally, the step of performing adaptive beamforming operation on the aligned channel signals based on the estimated noise covariance matrix includes:
performing minimum mean square undistorted response (MVDR) algorithm processing on the aligned channel signals based on the estimated noise covariance matrix to obtain MVDR voice signals;
estimating a noise amplitude spectrum required by post-adaptive filtering based on the estimated noise covariance matrix to obtain an estimated noise amplitude spectrum;
and performing post-adaptive filtering on the MVDR voice signal based on the estimated noise magnitude spectrum, and outputting a target voice signal.
Optionally, the step of estimating the noise magnitude spectrum required by the post-adaptive filtering based on the estimated noise covariance matrix, and obtaining the estimated noise magnitude spectrum further includes:
Calculating a voice signal amplitude spectrum of the MVDR voice signal;
calculating a signal-to-noise ratio of the target speech signal based on the speech signal magnitude spectrum and the estimated noise magnitude spectrum;
and calculating a gain coefficient based on the signal-to-noise ratio of the target voice signal.
Optionally, the step of estimating a noise magnitude spectrum required for post-adaptive filtering based on the estimated noise covariance matrix, and obtaining an estimated noise magnitude spectrum includes:
performing matrix decomposition on the estimated noise covariance matrix to obtain estimated noise covariance of each channel;
and calculating an arithmetic mean value of the estimated noise covariance of each channel to obtain an estimated noise amplitude spectrum required by the post-adaptive filtering.
Optionally, the step of post-adaptively filtering the MVDR voice signal based on the estimated noise magnitude spectrum, and outputting a target voice signal includes:
designing a post-adaptive filter based on the estimated noise magnitude spectrum;
and performing post-adaptive filtering on the MVDR voice signal through the post-adaptive filter to output a target voice signal.
Optionally, the step of performing an alignment operation on the original array speech signal to obtain aligned channel signals includes:
And performing time delay alignment operation on the original array voice signals to obtain aligned channel signals. Optionally, the method is applied to a near field scene, and the step of performing an alignment operation on the original array voice signals to obtain aligned channel signals includes:
and performing time delay alignment operation on the original array voice signals, and performing amplitude-frequency response alignment operation on the original array voice signals to obtain aligned channel signals.
Optionally, the step of performing delay alignment operation on the original array voice signals to obtain aligned channel signals includes:
and performing time delay alignment operation on the original array voice signals through a Finite Impulse Response (FIR) filter to obtain aligned channel signals.
Optionally, the step of performing delay alignment operation on the original array voice signal and performing amplitude-frequency alignment operation on the original array voice signal to obtain aligned channel signals includes:
and performing time delay alignment operation on the original array voice signals through a Finite Impulse Response (FIR) filter, and performing amplitude-frequency alignment operation on the original array voice signals through an Equalization (EQ) filter to obtain aligned channel signals.
Optionally, the step of estimating a noise covariance matrix based on the aligned channel signals includes:
converting the aligned channel signals into time domain signals of all channels through formats;
calculating the average value of the time domain signals of all the channels;
calculating the difference value between the time domain signals of the channels and the average value;
performing short-time Fourier transform on the difference value between the time domain signals of each channel and the mean value to obtain noise frequency domain signals of each channel;
calculating the noise covariance of the noise frequency domain signals of each channel;
and obtaining a noise covariance matrix based on the noise covariance.
In addition, to achieve the above object, the present invention also provides a beam forming apparatus including:
the data receiving module is used for receiving an original array voice signal, wherein the original array voice signal is a multi-channel signal;
the alignment module is used for executing alignment operation processing on the original array voice signals to obtain aligned channel signals;
a parameter processing module, configured to estimate a noise covariance matrix based on the aligned channel signals;
and the adaptive beamforming processing module is used for performing adaptive beamforming operation on the aligned channel signals based on the estimated noise covariance matrix.
Optionally, the alignment module is further configured to:
and performing time delay alignment operation on the original array voice signals to obtain aligned channel signals.
Optionally, if the method of the present invention is applied to a near field scene, the alignment module is further configured to:
and performing time delay alignment operation on the original array voice signals, and performing amplitude-frequency response alignment operation on the original array voice signals to obtain aligned channel signals.
Optionally, the alignment module is further configured to:
and performing time delay alignment operation on the original array voice signals through a Finite Impulse Response (FIR) filter to obtain aligned channel signals.
Optionally, if the method of the present invention is applied to a near field scene, the alignment module is further configured to:
and performing time delay alignment operation on the original array voice signals through a Finite Impulse Response (FIR) filter, and performing amplitude-frequency alignment operation on the original array voice signals through an Equalization (EQ) filter to obtain aligned channel signals.
Optionally, the parameter processing module is further configured to:
converting the aligned channel signals into time domain signals of all channels through formats;
calculating the average value of the time domain signals of all the channels;
Calculating the difference value between the time domain signals of the channels and the average value;
performing short-time Fourier transform on the difference value between the time domain signals of each channel and the mean value to obtain noise frequency domain signals of each channel;
calculating the noise covariance of the noise frequency domain signals of each channel;
and obtaining a noise covariance matrix based on the noise covariance.
Optionally, the adaptive beamforming processing module is further configured to:
performing minimum mean square undistorted response (MVDR) algorithm processing on the aligned channel signals based on the estimated noise covariance matrix to obtain MVDR voice signals;
estimating a noise amplitude spectrum required by post-adaptive filtering based on the estimated noise covariance matrix to obtain an estimated noise amplitude spectrum;
and performing post-adaptive filtering on the MVDR voice signal based on the estimated noise magnitude spectrum, and outputting a target voice signal.
Optionally, the adaptive beamforming processing module is further configured to:
performing matrix decomposition on the estimated noise covariance matrix to obtain estimated noise covariance of each channel;
and calculating an arithmetic mean value of the estimated noise covariance of each channel to obtain an estimated noise amplitude spectrum required by the post-adaptive filtering.
Optionally, the adaptive beamforming processing module is further configured to:
calculating a voice signal amplitude spectrum of the MVDR voice signal;
calculating a signal-to-noise ratio of the target speech signal based on the speech signal magnitude spectrum and the estimated noise magnitude spectrum;
and calculating a gain coefficient based on the signal-to-noise ratio of the target voice signal.
In addition, in order to achieve the above object, the present invention also provides an electronic device including a memory, a processor, and a beam forming program stored on the memory and executable on the processor, the beam forming program implementing the beam forming method as described above when executed by the processor.
In addition, in order to achieve the above object, the present invention also provides a computer-readable storage medium having stored thereon a beam forming program which, when executed by a processor, implements the beam forming method as described above.
The beam forming method, the beam forming device, the electronic equipment and the storage medium provided by the embodiment of the invention are used for receiving an original array voice signal, wherein the original array voice signal is a multichannel signal; performing alignment operation on the original array voice signals to obtain aligned channel signals; and estimating a noise covariance matrix based on the aligned channel signals. Therefore, through the multi-channel alignment processing, the noise covariance matrix can be estimated more accurately, and the noise signal component in the target voice signal is effectively reduced, so that the noise suppression capability in the beam forming is improved, and the spatial filtering effect in the beam forming based on the MVDR algorithm is improved.
Drawings
Fig. 1 is a schematic diagram of functional modules of an electronic device to which a beam forming apparatus of the present invention belongs;
FIG. 2 is a flow chart of an embodiment of a beam forming method according to the present invention;
FIG. 3 is a schematic diagram of waveforms of the original array voice signals according to an embodiment of the beam forming method of the present invention;
FIG. 4 is a schematic diagram of time domain waveforms of two different channel signals according to an embodiment of the beam forming method of the present invention;
fig. 5 is a schematic diagram of frequency response curves of different channel signals in a near field scene in an embodiment of a beam forming method according to the present invention;
FIG. 6 is a schematic diagram of a signal waveform of an average value of time domain signals of each channel in an embodiment of a beam forming method according to the present invention;
FIG. 7 is a schematic diagram of waveforms of a channel noise signal in an embodiment of a beam forming method according to the present invention;
FIG. 8 is a schematic diagram of another embodiment of a beam forming method according to the present invention;
FIG. 9 is a schematic diagram of another embodiment of a beam forming method according to the present invention;
fig. 10 is a flow chart of another embodiment of the beam forming method of the present invention;
FIG. 11 is a schematic diagram illustrating a refinement procedure of step S1040 in the embodiment shown in FIG. 10;
fig. 12 is a waveform diagram of a target signal output by the prior art scheme;
FIG. 13 is a diagram illustrating waveforms of a target signal combining an alignment operation and MVDR beamforming techniques in an embodiment of a beamforming method according to the present application;
fig. 14 is a detailed flowchart of step S1040 in another embodiment of the beam forming method of the present application.
The achievement of the objects, functional features and advantages of the present application will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
The main solutions of the embodiments of the present application are: receiving an original array voice signal, wherein the original array voice signal is a multi-channel signal; performing alignment operation on the original array voice signals to obtain aligned channel signals; and estimating a noise covariance matrix based on the aligned channel signals. Therefore, through the multi-channel alignment processing, the noise covariance matrix can be estimated more accurately, and the noise signal component in the target voice signal is effectively reduced, so that the noise suppression capability in the beam forming is improved, and the spatial filtering effect in the beam forming based on the MVDR algorithm is improved.
Technical terms related to the embodiment of the application:
BeamForming, beamForming; the beam forming can realize focusing and picking up signals, and the non-target direction signals are restrained and the target direction signals are enhanced by combining the signals of multiple microphones (usually multiple omni-directional microphones); further, focusing pickup in a specific direction can be realized, the signal to noise ratio of a received signal can be effectively improved, and the effect of noise reduction is also achieved; the basic principle of beam forming is wave interference, and signals of certain angles are enhanced and signals of other angles are mutually counteracted by adjusting parameters among signals of different array units. As the technology level and performance of devices have matured, the cost has become lower and the application of beamforming has become wider, including but not limited to the following equipment and usage scenarios: the conversation pickup of earphone/mobile phone is noise reduced; a digital hearing aid; positioning a sound source; conference room microphones; vehicle pickup, etc.
MVDR, minimum Variance Distortionless Response, least mean square distortion free response; MVDR algorithm is one of the most widely used adaptive beamforming methods. The MVDR principle is: the appropriate filter coefficients are selected so that the average power of the array output is minimized under the constraint that the desired signal is undistorted.
Wiener filtering; in signal processing, wiener filtering is a common noise reduction method, which can extract an actual signal from an observed quantity with noise, and has important application in both speech signals and image signals. Wiener filtering is a linear minimum mean square error (LMMSE, linear Minimum Mean Square Error) estimate, which means that the form of the estimate is linear, and the minimum variance is the optimization criterion for constructing the filter, that is, the difference between the actual signal and the estimated quantityThere is minimal variance. Wiener filtering is to construct a filter such that the output that can be obtained after the observed signal passes through the filter is the minimum mean square error estimate of the actual signal.
According to the embodiment of the application, the noise covariance matrix in the beam forming based on the MVDR algorithm in the related technical scheme is difficult to accurately estimate, so that the noise suppression capability is weaker, and the spatial filtering effect is poorer.
Based on this, the embodiment of the application provides a solution, by performing an alignment operation on the original array voice signals, a noise covariance matrix is estimated based on the aligned channel signals. The scheme is accurate in estimating the noise covariance. Based on the estimated noise covariance matrix, a target speech signal may be obtained by an adaptive beamforming operation. The scheme provides an estimated noise covariance matrix method combined with alignment operation, so that noise components in target voice signals are reduced more effectively, and the effect of wave beam forming spatial filtering based on an MVDR algorithm is improved.
Specifically, referring to fig. 1, fig. 1 is a schematic diagram of functional modules of an electronic device to which the beam forming apparatus of the present application belongs. The beamforming means may be electronic device independent means capable of beamforming, which may be carried on the electronic device in the form of hardware or software. The electronic device may be an intelligent mobile terminal with a data processing function, for example, an AR (Augmented Reality) or VR (virtual reality) device or an earphone product, or may be a fixed device or a server with a data processing function.
In this embodiment, the electronic device to which the beam forming apparatus belongs includes at least an output module 110, a processor 120, a memory 130, and a communication module 140.
The memory 130 stores an operating system and a beam forming program; the output module 110 may be a display screen or the like. The communication module 140 may include a WIFI module, a mobile communication module, a bluetooth module, and the like, and communicates with an external device or a server through the communication module 140.
Wherein the beamforming program in the memory 130 when executed by the processor performs the steps of:
receiving an original array voice signal, wherein the original array voice signal is a multi-channel signal;
Performing alignment operation on the original array voice signals to obtain aligned channel signals;
and estimating a noise covariance matrix based on the aligned channel signals.
Further, the beamforming program in the memory 130 when executed by the processor also performs the steps of:
and performing adaptive beam forming operation on the aligned channel signals based on the estimated noise covariance matrix.
Further, the beamforming program in the memory 130 when executed by the processor also performs the steps of:
performing minimum mean square undistorted response (MVDR) algorithm processing on the aligned channel signals based on the estimated noise covariance matrix to obtain MVDR voice signals;
estimating a noise amplitude spectrum required by post-adaptive filtering based on the estimated noise covariance matrix to obtain an estimated noise amplitude spectrum;
and performing post-adaptive filtering on the MVDR voice signal based on the estimated noise magnitude spectrum, and outputting a target voice signal.
Further, the beamforming program in the memory 130 when executed by the processor also performs the steps of:
calculating a voice signal amplitude spectrum of the MVDR voice signal;
Calculating a signal-to-noise ratio of the target speech signal based on the speech signal magnitude spectrum and the estimated noise magnitude spectrum;
and calculating a gain coefficient based on the signal-to-noise ratio of the target voice signal.
Further, the beamforming program in the memory 130 when executed by the processor also performs the steps of:
performing matrix decomposition on the estimated noise covariance matrix to obtain estimated noise covariance of each channel;
and calculating an arithmetic mean value of the estimated noise covariance of each channel to obtain an estimated noise amplitude spectrum required by the post-adaptive filtering.
Further, the beamforming program in the memory 130 when executed by the processor also performs the steps of:
designing a post-adaptive filter based on the estimated noise magnitude spectrum;
and performing post-adaptive filtering on the MVDR voice signal through the post-adaptive filter to output a target voice signal.
Further, the beamforming program in the memory 130 when executed by the processor also performs the steps of:
and performing time delay alignment operation on the original array voice signals to obtain aligned channel signals.
Further, the beamforming program in the memory 130 when executed by the processor also performs the steps of:
And performing time delay alignment operation on the original array voice signals, and performing amplitude-frequency response alignment operation on the original array voice signals to obtain aligned channel signals.
Further, the beamforming program in the memory 130 when executed by the processor also performs the steps of:
and performing time delay alignment operation on the original array voice signals through a Finite Impulse Response (FIR) filter to obtain aligned channel signals.
Further, the beamforming program in the memory 130 when executed by the processor also performs the steps of:
and performing time delay alignment operation on the original array voice signals through a Finite Impulse Response (FIR) filter, and performing amplitude-frequency alignment operation on the original array voice signals through an Equalization (EQ) filter to obtain aligned channel signals.
Further, the beamforming program in the memory 130 when executed by the processor also performs the steps of:
converting the aligned channel signals into time domain signals of all channels through formats;
calculating the average value of the time domain signals of all the channels;
calculating the difference value between the time domain signals of the channels and the average value;
performing short-time Fourier transform on the difference value between the time domain signals of each channel and the mean value to obtain noise frequency domain signals of each channel;
Calculating the noise covariance of the noise frequency domain signals of each channel;
and obtaining a noise covariance matrix based on the noise covariance.
According to the scheme, the original array voice signal is received, and the original array voice signal is a multi-channel signal; performing alignment operation on the original array voice signals to obtain aligned channel signals; and estimating a noise covariance matrix based on the aligned channel signals. Therefore, through the multi-channel alignment processing, the noise covariance matrix can be estimated more accurately, and the noise signal component in the target voice signal is effectively reduced, so that the noise suppression capability in the beam forming is improved, and the spatial filtering effect in the beam forming based on the MVDR algorithm is improved.
The method embodiments of the present application are presented based on, but not limited to, the above-described architecture of the electronic device.
Referring to fig. 2, fig. 2 is a schematic flow chart of an exemplary embodiment of a beam forming method according to the present application. The beam forming method comprises the following steps:
step S1010, receiving an original array voice signal, wherein the original array voice signal is a multi-channel signal;
in particular, the beamforming method of the present application may be applied to AR/VR devices or earphone product lines. The main implementation body of the beam forming method of this embodiment may be a beam forming device, which may be integrated on an electronic device such as an AR/VR device or an earphone product, and the beam forming device may be configured with a microphone, where the microphone is used to collect a voice signal.
The embodiment mainly performs beam forming processing on voice signals collected by a microphone so as to solve the problem of serious noise interference in voice signal transmission, improve the noise suppression capability in beam forming and improve the spatial filtering effect in beam forming based on an MVDR algorithm.
The voice signal collected by the microphone is an array voice signal, and the array voice signal is a multi-channel signal.
The principle of the microphone for collecting voice signals is as follows: when a target speaker speaks, acquiring an acoustic wave signal of the speaker; the sound wave signal can make the vibrating diaphragm in the microphone shake so as to generate electric signals with the same amplitude, wherein, because the microphones are distributed according to arrays, and the microphone arrays are arrays formed by arranging a group of omnidirectional microphones positioned at different positions in space according to a certain shape rule, the device is a device for spatially sampling the spatially transmitted sound signals, and the collected electric signals contain the spatial position information of the spatially transmitted sound signals. The electrical signal produced by the microphone is the original array speech signal.
The embodiment can receive the voice signal uttered by the user through the microphone and convert the voice signal from the sound wave signal to the electrical signal to obtain the original array voice signal. And after the microphone finishes acquisition and conversion, transmitting the converted original array voice signals into a beam forming device. The beam forming device receives the original array voice signals and performs subsequent processing.
As shown in fig. 3, fig. 3 is a schematic diagram of the waveform of the original array voice signal in the present embodiment.
After the original array speech signal is generated, the original array speech signal is received by a beam forming device. The original array speech signal may be broken down into multiple channel signals due to the array nature of the original array speech signal, and thus the original array speech signal is also a multi-channel signal. The advantage of using a microphone in this embodiment is that the microphone can achieve the effect of echo cancellation and sound source localization.
Step S1020, performing alignment operation processing on the original array voice signals to obtain aligned channel signals;
in this embodiment, the positions of the target speakers are not fixed, and the number of the target speakers is not limited, so that the distances between the sound source positions of the target speakers and the microphones are different, the phases and delays of the arrival of the sound wave signals at the microphones are also different, and the arrival times of the channel signals of the original array voice signals generated by the microphones are different. The specific situation is shown in fig. 4, and fig. 4 is a time domain waveform diagram of two different channel signals. Meanwhile, in the near-field scene, since the sound wave signals of the target speaker have the characteristics of spherical waves, the amplitude-frequency response of the sound wave signals emitted by the sound sources at different positions is also different.
Therefore, the present embodiment proposes to perform an alignment operation on the received original array voice signal, so that the delay and the amplitude-frequency response of each channel signal in the original array voice signal can be the same.
In one embodiment, the alignment operation is performed on the original array voice signals, and a time delay alignment operation and an amplitude-frequency response alignment operation may be used to obtain aligned channel signals.
Specifically, as an embodiment, in the case that the beamforming method is not applied to the near field scene, the step S1020 may perform an alignment operation process on the original array speech signal, and the obtaining the aligned channel signals may include:
and performing time delay alignment operation on the original array voice signals to obtain aligned channel signals.
Specifically, considering that the time delays of the channel signals are different in the original array voice signals generated by the microphone device, the embodiment proposes to use time delay alignment operation, so that the multi-channel signals with the same time delay can be obtained.
Specifically, the step of performing delay alignment operation on the original array voice signal to obtain aligned channel signals may include:
And performing time delay alignment operation on the original array voice signals through a Finite Impulse Response (FIR) filter to obtain aligned channel signals.
In this step, an FIR filter may be configured in the beamforming device to perform a time-delay alignment operation on the original array speech signal. FIR filters are generally the most basic elements of digital signal processing systems.
The FIR filter is used as the most basic element in the digital signal processing system, can ensure any amplitude-frequency characteristic and has strict linear phase-frequency characteristic, and meanwhile, the unit sampling response is limited in length, so that the stability of the digital signal processing system can be ensured.
In this embodiment, the delay and phase of the original array speech signal need to be operated, and an FIR filter, which is a component capable of ensuring the stability of the system, is required.
Furthermore, a time delay is introduced when any signal passes through the FIR filter, which means that the same signal is shifted in time after being filtered by the FIR filter. By utilizing the characteristics, the embodiment passes the original array voice signals through the FIR filters corresponding to the channels, namely, each channel signal with different time delay in the original array voice signals is added with a corresponding time delay compensation, so that the uniform time delay of all the channel signals is realized, and the aligned channel signals are obtained.
Further, in the case that the beamforming method is applied to the near field scene, the step S1020 of performing an alignment operation on the original array voice signal, the obtaining aligned channel signals may include:
and performing time delay operation on the original array voice signals, and performing amplitude-frequency response alignment operation on the original array voice signals to obtain aligned channel signals.
Specifically, as an implementation manner, the step of performing a time delay alignment operation on the original array voice signal and performing an amplitude-frequency response alignment operation on the original array voice signal to obtain aligned channel signals may include:
and performing time delay alignment operation on the original array voice signals through a Finite Impulse Response (FIR) filter, and performing amplitude-frequency alignment operation on the original array voice signals through an Equalization (EQ) filter to obtain aligned channel signals.
In this embodiment, if the beam forming method is applied to the near field scenario, the time delay alignment operation needs to be performed on the original array voice signal, and the amplitude-frequency alignment operation needs to be performed.
The near-field scene is a scene where a sound source is relatively close to the microphone array, and at this time, sound waves can be regarded as spherical waves, so that after the microphone receives sound signals and converts the sound signals into electrical signals, amplitude differences among the signals received by the microphone array need to be considered, as shown in fig. 5, and fig. 5 is a schematic diagram of frequency response curves of signals of different channels in the near-field scene.
By the scheme, when the original array voice signals are aligned, time delay alignment is needed, and amplitude-frequency response alignment is needed. The EQ filter can gain or attenuate one or more frequency bands of sound, that is, can respectively adjust the amplification of the electric signal of each frequency component, thereby adjusting the signal amplitude. Therefore, the EQ filter performs an amplitude-frequency alignment operation on the original array voice signals, so that aligned channel signals can be obtained.
Step S1030, estimating a noise covariance matrix based on the aligned channel signals.
In this step, the time delays of the respective channel signals are aligned, and a noise covariance matrix can be estimated from the aligned respective channel signals.
The noise covariance matrix is the linear correlation degree between noise elements in each channel signal, and the estimated noise covariance matrix is the linear correlation degree between noise elements in each channel signal.
According to the scheme, specifically, the received original array voice signals are aligned, so that all channel signals with the same time delay and amplitude-frequency response are obtained, and then the noise covariance matrix of all channel signals is estimated. Compared with the prior art for estimating the noise covariance matrix, the method and the device can obtain a more accurate noise covariance matrix, so that better noise suppression capability is obtained in MVDR wave beam forming. And the scheme is widely applicable, and specific assumptions are not needed, for example, the receiving scene of the microphone is assumed to be a scene of quiet transition noise.
More specifically, the step S1030 of estimating the noise covariance matrix based on the aligned channel signals may include:
step A, converting the aligned channel signals into time domain signals of the channels through formats;
in this step, the signal formats of the channels need to be unified, so that the following calculation process is guaranteed to be performed in the time domain.
Step B, calculating the average value of the time domain signals of all the channels;
the specific formula is as follows:
in the formula (1), M is the number of array channels, i is the channel number, and x i For each channel time domain signal, x sum As shown in fig. 6, fig. 6 is a signal waveform schematic diagram of the mean value of the time domain signals of each channel in the beam forming method according to the embodiment of the present invention.
Step C, calculating the difference value between the time domain signals of the channels and the average value;
the specific formula is as follows:
xn i ＝x sum -x i (2)
in the formula (2), xn i The noise signal waveforms of the ith channel are shown in fig. 7, 8 and 9, respectively.
Step D, carrying out short-time Fourier transform on the difference value between the time domain signals of each channel and the mean value to obtain noise frequency domain signals of each channel;
in this step, the difference between the time domain signal of each channel and the mean value is the noise in each channel, and the time domain signal of each channel noise can be transformed into the frequency domain signal of each channel noise by performing short-time fourier transform on the difference between the time domain signal of each channel and the mean value, so as to obtain XN.
E, calculating the noise covariance of the noise frequency domain signals of each channel;
the specific formula is as follows:
nc ij ＝cov(XN i ,XN j ) (3)
in formula (3), nc ij For each channel noise frequency domain signal, XN i Is the ith channel noise frequency domain signal, XN j Is the jth channel noiseAn audio domain signal.
Step F, obtaining a noise covariance matrix based on the noise covariance;
in the step, an M×M matrix can be constructed according to the result obtained in the formula (3), and the value of M is assigned to n, so that the formula for specifically obtaining the noise covariance matrix is as follows:
R n ＝(nc ij ) n×n (4)
according to the method, the noise covariance matrix can be estimated according to the steps, and the method is different from the estimation method of the noise covariance matrix in the prior art, and the estimation operation is performed after the alignment operation of the original array voice signals, so that the noise covariance matrix can be estimated more accurately, the noise suppression capability of MVDR wave beam formation is improved, and the spatial filtering effect of MVDR wave beam formation is improved.
Referring to fig. 10, fig. 10 is a flowchart illustrating another embodiment of the beam forming method according to the present application. Based on the embodiment shown in fig. 2, in this embodiment, after estimating the noise covariance matrix based on the aligned channel signals in step S1030, the method further includes:
Step S1040, performing adaptive beamforming operation on the aligned channel signals based on the estimated noise covariance matrix.
In comparison with the embodiment shown in fig. 2 described above, the present embodiment further includes a processing scheme of adaptive beamforming after estimating the noise covariance matrix.
Referring to fig. 11, fig. 11 is a schematic diagram of a refinement flow of step S1040 in the embodiment shown in fig. 10.
Specifically, as an embodiment, the step S1040 may include:
step a, performing minimum mean square undistorted response (MVDR) algorithm processing on the aligned channel signals based on the estimated noise covariance matrix to obtain MVDR voice signals;
the MVDR algorithm is based on a minimum mean square error criterion, and the beam forming technology based on the MVDR algorithm can realize the effect of spatial filtering by restraining the target signal gain to be unchanged and simultaneously enabling the beam forming device to output the minimum total energy, namely outputting the minimum interference and noise power so as to realize the suppression of the interference and noise signals.
Among these, the MVDR algorithm requires two parameters, namely a steering vector and a noise covariance matrix. The guide vector is known information and can be obtained through a pre-configured DOA estimation module; the noise covariance matrix is estimated by the scheme.
In this embodiment, after the noise covariance matrix is estimated, the noise covariance matrix is brought into an MVDR optimal weight formula, and the weight w is calculated, where the MVDR optimal weight formula is as follows:
/>
in the formula (5), c is a guide vector, R n For the noise covariance matrix, H is the conjugate transpose of c.
From this weight w, an MVDR speech signal may be derived.
Step b, estimating a noise amplitude spectrum required by post-adaptive filtering based on the estimated noise covariance matrix to obtain an estimated noise amplitude spectrum;
in this step, since the MVDR beamforming cannot completely eliminate noise, the present embodiment further needs to perform post-adaptive filtering after the MVDR beamforming to further suppress noise interference.
In this embodiment, wiener filtering is taken as an example, and a post-adaptive filter used in the wiener filtering method needs to track and estimate the noise amplitude spectrum more accurately.
Specifically, as an embodiment, the step of estimating the noise magnitude spectrum required for post-adaptive filtering based on the estimated noise covariance matrix, and obtaining the estimated noise magnitude spectrum may include:
performing matrix decomposition on the estimated noise covariance matrix to obtain estimated noise covariance of each channel; and calculating an arithmetic mean value of the estimated noise covariance of each channel to obtain an estimated noise amplitude spectrum required by the post-adaptive filtering.
In this step, the noise magnitude spectrum is a parameter required for post-adaptive filtering. In the prior art, the noise magnitude spectrum is estimated by adopting a method such as minimum tracking in an OMLSA (Optimally Mdified Log-Spectral Amplitude, optimal improved log-magnitude estimation) algorithm, but the operation amount is large, so the embodiment proposes that the noise magnitude spectrum is estimated by adopting the noise covariance of each channel noise frequency domain signal in the method for estimating the noise covariance matrix, for example, the ith channel noise frequency domain signal XN i The specific calculation formula of the noise amplitude spectrum is as follows:
in the formula (6), Y N The noise magnitude spectrum, M, is the number of array channels.
And c, performing post-adaptive filtering on the MVDR voice signal based on the estimated noise magnitude spectrum, and outputting a target voice signal.
In this step, in order to further suppress noise, the MVDR speech signal needs to be subjected to post-adaptive filtering, and a post-adaptive filter is required for a specific processing operation.
Specifically, the step of post-adaptively filtering the MVDR voice signal based on the estimated noise magnitude spectrum, and outputting a target voice signal may include:
and designing a post-adaptive filter based on the estimated noise amplitude spectrum to obtain the post-adaptive filter based on the noise amplitude spectrum estimation, performing post-adaptive filtering through the post-adaptive filter, and finally outputting a target voice signal.
In this embodiment, the post-adaptive filter is exemplified by a wiener filter.
The wiener filter can also be called a minimum mean square error wiener filter, is an optimal linear filter, can enable a signal with noise passing through the filter to be more approximate to an original signal without noise interference, and finally outputs a target voice signal subjected to wiener filtering.
The basic principle of wiener filters is: a linear estimate of the filtered desired target speech signal is made by using known information of the MDRV speech signal that is to be filtered. When the noise is independent of the noise-filtered target speech signal, the wiener filter impulse response function can be determined from the known information of the MDRV speech signal and the noise amplitude spectrum estimate.
In the embodiment, the noise covariance matrix is skillfully utilized to estimate the noise amplitude spectrum, so that the calculation amount is reduced compared with the traditional OMLSA algorithm, and the method can be rapidly deployed on an embedded real-time platform.
Compared with the prior art, the waveform of the target signal output by the prior art may be as shown in fig. 12, and the waveform of the target voice signal output by the final embodiment of the present invention is as shown in fig. 13. As can be seen from a comparison of the waveform diagrams, the present embodiment can improve the noise suppression capability, thereby improving the spatial filtering effect in MVDR beamforming.
It should be understood that, the technical effect test of the present embodiment may also directly compare the sound quality of the output target voice signal through the human ear listening, so as to compare the technical effect of MVDR beamforming according to the prior art with the technical effect of MVDR beamforming combined with the alignment operation provided in the present embodiment.
The scheme of the embodiment realizes the improvement of the spatial filtering effect in the beam forming based on the MVDR algorithm. In this embodiment, from the above practical application scenario, the MVDR beamforming technology is designed to combine alignment processing to achieve the purpose of improving the tone quality of the received user voice signal by using the advantage that the MVDR algorithm beamforming technology can enhance the useful signal and suppress noise.
Based on the embodiment shown in fig. 11, another embodiment of the beam forming method of the present invention is proposed. In this embodiment, after the step of estimating the noise magnitude spectrum required for post-adaptive filtering based on the estimated noise covariance matrix to obtain the estimated noise magnitude spectrum, the method may further include:
step d, calculating a voice signal amplitude spectrum of the MVDR voice signal;
step e, calculating the signal-to-noise ratio of the target voice signal based on the voice signal amplitude spectrum and the estimated noise amplitude spectrum;
And f, calculating a gain coefficient based on the signal-to-noise ratio of the target voice signal.
Compared with the embodiment shown in fig. 11, in this embodiment, after the step of estimating the noise magnitude spectrum required for post-adaptive filtering based on the estimated noise covariance matrix to obtain the estimated noise magnitude spectrum, the signal-to-noise ratio and the gain coefficient of the target speech signal may also be calculated.
Specifically, in this embodiment, as a method for evaluating the actual effect of the MVDR beamforming scheme, this embodiment may more intuitively embody the scheme of the present invention by calculating the signal-to-noise ratio and the gain coefficient, so as to effectively reduce the noise component in the target speech signal, thereby improving the noise suppression capability and improving the MVDR beamforming spatial filtering effect.
The detailed explanation of the steps in this embodiment is as follows:
step d, calculating a voice signal amplitude spectrum of the MVDR voice signal;
the specific formula is as follows:
Y S ＝wX (7)
in the formula (7), w is the weight obtained by the MVDR algorithm, and X is a matrix constructed by the input aligned voice signals of each channel.
Step e, calculating the signal-to-noise ratio of the target voice signal based on the voice signal amplitude spectrum and the estimated noise amplitude spectrum;
The specific formula is as follows:
snr＝Y S /(Y N +Δ) (8)
in the formula (8), snr is the signal-to-noise ratio of the target voice signal, Y S Is the voice amplitude spectrum of MVDR voice signal, Y N For the noise magnitude spectrum, Δ is a constant value that prevents the denominator from being 0.
And f, calculating a gain coefficient based on the signal-to-noise ratio of the target voice signal.
The specific formula is as follows:
G＝snr/(snr+β) (9)
in the formula (9), G is a gain coefficient, beta is a constant, and the value range is 1 to 2.
By the scheme of the embodiment, the spatial filtering effect in beam forming based on the MVDR algorithm is improved. Starting from the actual application scene, the embodiment designs an MVDR wave beam forming technology combined with alignment processing by utilizing the advantages of enhancing useful signals and suppressing noise of wave beam forming technology based on MVDR algorithm, so as to achieve the purpose of improving the tone quality of received user voice signals.
The present embodiment also calculates the signal-to-noise ratio and gain coefficient of the process after performing the adaptive beamforming operation on the aligned channel signals. The invention can more intuitively show that the noise component in the target voice signal can be effectively reduced through the signal-to-noise ratio and the gain coefficient, thereby improving the noise suppression capability and the MVDR wave beam forming spatial filtering effect.
Based on the embodiment shown in fig. 10, another embodiment of the beam forming method of the present invention is proposed. Referring to fig. 14, fig. 14 is a detailed flowchart of step S1040 in another embodiment of the present invention.
As another embodiment, step S1040 of performing an adaptive beamforming operation on the aligned channel signals based on the estimated noise covariance matrix may include:
step S1041, performing minimum mean square undistorted response MVDR algorithm processing on the aligned channel signals based on the estimated noise covariance matrix to obtain MVDR voice signals;
step S1042, calculating the voice signal amplitude spectrum of the MVDR voice signal;
step S1043, estimating a noise amplitude spectrum required by post-adaptive filtering based on the estimated noise covariance matrix to obtain an estimated noise amplitude spectrum;
step S1044, calculating a signal-to-noise ratio of the target voice signal based on the voice signal amplitude spectrum and the estimated noise amplitude spectrum;
step S1045, calculating a gain factor based on the signal-to-noise ratio of the target voice signal.
In this embodiment, the MVDR algorithm is performed on each channel signal after the alignment based on the noise covariance matrix, then the voice signal amplitude is calculated, the noise amplitude spectrum is estimated, and finally the signal-to-noise ratio and the gain coefficient are calculated based on the voice amplitude spectrum and the noise amplitude spectrum.
It should be understood that the sequence of the step S1042 and the step S1043 is not required, i.e. the step S1042 may be performed before the step S1043, may be performed after the step S1043, or may be performed simultaneously with the step S1043.
In this embodiment, the step S1041 is the same as the step a in the embodiment, the step S1042 is the same as the step d in the embodiment, the step S1043 is the same as the step b in the embodiment, the step S1044 is the same as the step e in the embodiment, and the step S1045 is the same as the step f in the embodiment, and specific reference is made to the embodiments, and the details are not repeated here.
Starting from the above practical application scenario, the present embodiment designs an MVDR beam forming technology combining alignment processing to achieve the purpose of improving the tone quality of the received voice signal of the user by utilizing the advantage that the beam forming technology based on the MVDR algorithm can enhance the useful signal and suppress noise.
In this embodiment, after each channel signal after the alignment processing is processed by the MVDR algorithm, the signal-to-noise ratio and the gain coefficient of the process are calculated. The invention can be more intuitively seen through the signal-to-noise ratio and the gain coefficient, effectively reduce noise interference and improve the MVDR wave beam forming spatial filtering effect.
In addition, an embodiment of the present application further provides a beam forming apparatus, where the beam forming apparatus includes:
the data receiving module is used for receiving an original array voice signal, wherein the original array voice signal is a multi-channel signal;
the alignment module is used for executing alignment operation processing on the original array voice signals to obtain aligned channel signals;
a parameter processing module, configured to estimate a noise covariance matrix based on the aligned channel signals;
and the adaptive beamforming processing module is used for performing adaptive beamforming operation on the aligned channel signals based on the estimated noise covariance matrix.
The present embodiment implements the principle and implementation process of beam forming, please refer to the above embodiments, and the description thereof is omitted herein.
In addition, the embodiment of the application also provides electronic equipment, which can be: an AR/VR device or headset product, etc., the electronic device comprising a memory, a processor, and a beamforming program stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the beamforming method as described above.
Because the beam forming program is executed by the processor and adopts all the technical schemes of all the embodiments, the beam forming program has at least all the beneficial effects brought by all the technical schemes of all the embodiments and is not described in detail herein.
In addition, the embodiment of the application further provides a computer readable storage medium, wherein the computer readable storage medium stores a beam forming program, and the beam forming program realizes the steps of the beam forming method when being executed by a processor.
Because the beam forming program is executed by the processor and adopts all the technical schemes of all the embodiments, the beam forming program has at least all the beneficial effects brought by all the technical schemes of all the embodiments and is not described in detail herein.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above ordering of embodiments of the invention is merely for illustration, and does not represent the advantages or disadvantages of the embodiments.
From the description of the above embodiments, it will be apparent to those skilled in the art that the above embodiment methods may be implemented by means of software plus necessary general hardware platforms. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) as described above, including several instructions for causing an electronic device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
Claims (14)
1. A method of beam forming, the method comprising the steps of:
Receiving an original array voice signal, wherein the original array voice signal is a multi-channel signal;
performing alignment operation on the original array voice signals to obtain aligned channel signals;
and estimating a noise covariance matrix based on the aligned channel signals.
2. The beamforming method of claim 1, wherein the beamforming method further comprises:
and performing adaptive beam forming operation on the aligned channel signals based on the estimated noise covariance matrix.
3. The beamforming method of claim 2 wherein said step of adaptively beamforming said aligned channel signals based on an estimated noise covariance matrix comprises:
performing minimum mean square undistorted response (MVDR) algorithm processing on the aligned channel signals based on the estimated noise covariance matrix to obtain MVDR voice signals;
estimating a noise amplitude spectrum required by post-adaptive filtering based on the estimated noise covariance matrix to obtain an estimated noise amplitude spectrum;
and performing post-adaptive filtering on the MVDR voice signal based on the estimated noise magnitude spectrum, and outputting a target voice signal.
4. The beam forming method as claimed in claim 3, wherein the step of estimating a noise magnitude spectrum required for post-adaptive filtering based on the estimated noise covariance matrix, and obtaining the estimated noise magnitude spectrum further comprises:
calculating a voice signal amplitude spectrum of the MVDR voice signal;
calculating a signal-to-noise ratio of the target speech signal based on the speech signal magnitude spectrum and the estimated noise magnitude spectrum;
and calculating a gain coefficient based on the signal-to-noise ratio of the target voice signal.
5. The beam forming method of claim 3, wherein the step of estimating a noise magnitude spectrum required for post-adaptive filtering based on the estimated noise covariance matrix, and obtaining an estimated noise magnitude spectrum comprises:
performing matrix decomposition on the estimated noise covariance matrix to obtain estimated noise covariance of each channel;
and calculating an arithmetic mean value of the estimated noise covariance of each channel to obtain an estimated noise amplitude spectrum required by the post-adaptive filtering.
6. The beam forming method of claim 3, wherein the step of post-adaptively filtering the MVDR voice signal based on the estimated noise magnitude spectrum, and outputting a target voice signal comprises:
Designing a post-adaptive filter based on the estimated noise magnitude spectrum;
and performing post-adaptive filtering on the MVDR voice signal through the post-adaptive filter to output a target voice signal.
7. The beam forming method as claimed in claim 1, wherein the step of performing an alignment operation on the original array voice signals to obtain aligned channel signals comprises:
and performing time delay alignment operation on the original array voice signals to obtain aligned channel signals.
8. The beam forming method as claimed in claim 1, wherein the method is applied to a near field scene, and the step of performing an alignment operation on the original array voice signals to obtain aligned channel signals comprises:
and performing time delay alignment operation on the original array voice signals, and performing amplitude-frequency response alignment operation on the original array voice signals to obtain aligned channel signals.
9. The beam forming method as claimed in claim 7, wherein the step of performing a time delay alignment operation on the original array voice signals to obtain the aligned channel signals comprises:
And performing time delay alignment operation on the original array voice signals through a Finite Impulse Response (FIR) filter to obtain aligned channel signals.
10. The beam forming method as claimed in claim 8, wherein the step of performing a delay alignment operation on the original array voice signals and performing an amplitude-frequency response alignment operation on the original array voice signals to obtain the aligned channel signals includes:
and performing time delay alignment operation on the original array voice signals through a Finite Impulse Response (FIR) filter, and performing amplitude-frequency alignment operation on the original array voice signals through an Equalization (EQ) filter to obtain aligned channel signals.
11. The beam forming method according to any one of claims 1 to 10, wherein the step of estimating a noise covariance matrix based on the aligned channel signals comprises:
converting the aligned channel signals into time domain signals of all channels through formats;
calculating the average value of the time domain signals of all the channels;
calculating the difference value between the time domain signals of the channels and the average value;
performing short-time Fourier transform on the difference value between the time domain signals of each channel and the mean value to obtain noise frequency domain signals of each channel;
Calculating the noise covariance of the noise frequency domain signals of each channel;
and obtaining a noise covariance matrix based on the noise covariance.
12. A beam forming apparatus, the beam forming apparatus comprising:
the data receiving module is used for receiving an original array voice signal, wherein the original array voice signal is a multi-channel signal;
the alignment module is used for executing alignment operation processing on the original array voice signals to obtain aligned channel signals;
a parameter processing module, configured to estimate a noise covariance matrix based on the aligned channel signals;
and the adaptive beamforming processing module is used for performing adaptive beamforming operation on the aligned channel signals based on the estimated noise covariance matrix.
13. An electronic device comprising a memory, a processor, and a beamforming program stored on the memory and executable on the processor, the beamforming program when executed by the processor implementing the beamforming method of any of claims 1-11.
14. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a beam forming program, which when executed by a processor, implements the beam forming method according to any of claims 1-11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310615720.8A CN116760442A (en) | 2023-05-26 | 2023-05-26 | Beam forming method, device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310615720.8A CN116760442A (en) | 2023-05-26 | 2023-05-26 | Beam forming method, device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116760442A true CN116760442A (en) | 2023-09-15 |
Family
ID=87952393
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310615720.8A Pending CN116760442A (en) | 2023-05-26 | 2023-05-26 | Beam forming method, device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116760442A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117686086A (en) * | 2024-02-02 | 2024-03-12 | 北京谛声科技有限责任公司 | Equipment running state monitoring method, device, equipment and system |
-
2023
- 2023-05-26 CN CN202310615720.8A patent/CN116760442A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117686086A (en) * | 2024-02-02 | 2024-03-12 | 北京谛声科技有限责任公司 | Equipment running state monitoring method, device, equipment and system |
| CN117686086B (en) * | 2024-02-02 | 2024-06-04 | 北京谛声科技有限责任公司 | Equipment running state monitoring method, device, equipment and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106710601B (en) | Noise-reduction and pickup processing method and device for voice signals and refrigerator | |
| US8374358B2 (en) | Method for determining a noise reference signal for noise compensation and/or noise reduction | |
| EP3542547B1 (en) | Adaptive beamforming | |
| US9723422B2 (en) | Multi-microphone method for estimation of target and noise spectral variances for speech degraded by reverberation and optionally additive noise | |
| KR101449433B1 (en) | Noise cancelling method and apparatus from the sound signal through the microphone | |
| CN1809105B (en) | Dual-microphone speech enhancement method and system applicable to mini-type mobile communication devices | |
| US8194880B2 (en) | System and method for utilizing omni-directional microphones for speech enhancement | |
| CN110085248B (en) | Noise estimation at noise reduction and echo cancellation in personal communications | |
| US8204252B1 (en) | System and method for providing close microphone adaptive array processing | |
| DK2433437T3 (en) | Signal Enhancement using wireless streaming | |
| KR20130035990A (en) | Enhanced blind source separation algorithm for highly correlated mixtures | |
| WO2005055644A1 (en) | Method and apparatus for producing adaptive directional signals | |
| Tashev et al. | Microphone array for headset with spatial noise suppressor | |
| CN116760442A (en) | Beam forming method, device, electronic equipment and storage medium | |
| CN111354368B (en) | Method for compensating processed audio signal | |
| JP4409642B2 (en) | Method and apparatus for optimized processing of disturbance signals during sound acquisition | |
| US11153695B2 (en) | Hearing devices and related methods | |
| CN113838472A (en) | Voice noise reduction method and device | |
| Miyahara et al. | A hearing device with an adaptive noise canceller for noise-robust voice input | |
| Reindl et al. | An acoustic front-end for interactive TV incorporating multichannel acoustic echo cancellation and blind signal extraction | |
| CN113744752A (en) | Voice processing method and device | |
| EP3886463A1 (en) | Method at a hearing device | |
| CN115035905A (en) | Method for improving noise elimination effect by using beam forming under strong internal noise | |
| Datla | Implementation and evaluation of spectral subtraction (SS) with minimum statistics and wiener beamformer combination | |
| Schmidt | Part 3: Beamforming |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |