CN109218920B

CN109218920B - Signal processing method and device and terminal

Info

Publication number: CN109218920B
Application number: CN201710524410.XA
Authority: CN
Inventors: 陈日林; 李贤胜; 赵翔宇
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2017-06-30
Filing date: 2017-06-30
Publication date: 2020-09-18
Anticipated expiration: 2037-06-30
Also published as: CN109218920A

Abstract

The application provides a signal processing method, a signal processing device and a signal processing terminal, relates to the technical field of audio, and can compensate low-frequency loss of an obtained heart-shaped directional signal in the process of generating surround sound, so that the signal-to-noise ratio of a low-frequency band of the heart-shaped directional signal is improved. The specific scheme comprises the following steps: acquiring signals of a first microphone, a second microphone, a third microphone and a fourth microphone in a microphone array; carrying out differential processing on signals of all microphones to obtain a first initial directional signal, a second initial directional signal, a third initial directional signal and a fourth initial directional signal; respectively performing low-frequency compensation on each initial directional signal based on a wiener filtering method to obtain a first directional signal, a second directional signal, a third directional signal and a fourth directional signal; and obtaining a first sound field component, a second sound field component, a third sound field component and a fourth sound field component according to the directional signals.

Description

Signal processing method and device and terminal

Technical Field

The embodiment of the invention relates to the technical field of audio, in particular to a signal processing method, a signal processing device and a terminal.

Background

With the rapid increase of broadband (broadband band) due to the development of 4G technology and 5G technology, the surround sound recording and playing technology is increasingly applied to the current mobile terminals. Wherein, the sound field of the surround sound can be a three-dimensional sound field.

Currently, 4 directional microphones can be used to collect 4 cardioid directional signals (referred to as directional signals for short) in different directions respectively to generate sound field components W, X, Y and Z of a three-dimensional sound field. Wherein the sound field components W, X, Y and Z are signals in B-format. The sound field components W, X, Y and Z can then be decoded to obtain speaker excitation signals pointing in different directions, which can be output to generate surround sound. However, the mobile terminal cannot be provided with the directional microphone, and may be provided with an omni-directional microphone. At this time, sound signals may be collected by 4 omni-directional microphones, and the sound signals collected by the omni-directional microphones are processed using a differential method to obtain directional signals in 4 different directions. The 4 directional signals in different directions obtained by the difference method have serious low-frequency loss and the signal-to-noise ratio is low.

In the prior art, in order to improve the signal-to-noise ratio of the 4 directional signals obtained by the difference method, a low-pass filter may be used to perform low-frequency compensation on the 4 directional signals. Although the low-frequency loss of the directional signals after the 4 difference processes is compensated by adopting the low-pass filter, the background noise of the 4 directional signals is simultaneously raised, so that the signal-to-noise ratio of the 4 directional signals in the low-frequency band is low. Thus, the quality of the speaker excitation signal obtained from the 4 directional signals is to be improved.

Disclosure of Invention

The application provides a signal processing method, a signal processing device and a signal processing terminal, which can compensate low-frequency loss of an obtained cardioid directional signal in the process of generating surround sound and improve the signal-to-noise ratio of a low-frequency band of the cardioid directional signal.

In order to achieve the purpose, the technical scheme is as follows:

in a first aspect, a signal processing method is provided, which is applied to a terminal comprising a microphone array consisting of at least 4 omni-directional microphones; the method comprises the following steps: acquiring signals of a first microphone, a second microphone, a third microphone and a fourth microphone in a microphone array, wherein the first microphone, the second microphone and the third microphone are positioned in a first plane, and the fourth microphone is positioned outside the first plane; carrying out differential processing on the signal of the first microphone, the signal of the second microphone, the signal of the third microphone and the signal of the fourth microphone to obtain a first initial directional signal, a second initial directional signal, a third initial directional signal and a fourth initial directional signal; respectively performing low-frequency compensation on the first initial directional signal, the second initial directional signal, the third initial directional signal and the fourth initial directional signal based on a wiener filtering method to obtain a first directional signal, a second directional signal, a third directional signal and a fourth directional signal; and obtaining a first sound field component, a second sound field component, a third sound field component and a fourth sound field component according to the first directional signal, the second directional signal, the third directional signal and the fourth directional signal.

In the signal processing method provided by the present application, the terminal employs a wiener-based filtering method, so that the low-frequency loss of the obtained cardioid directional signal can be compensated, and the signal-to-noise ratio of the cardioid directional signal can be further improved. Thus, the terminal can improve the quality of the sound field components W, X, Y and Z obtained from the compensated cardioid directional signal. That is, the terminal can improve the quality of surround sound obtained from the sound field components W, X, Y and Z.

In addition, the terminal provided by the application comprises a microphone array, wherein the first microphone, the second microphone and the third microphone are positioned in the first plane, and the fourth microphone is positioned outside the first plane; thus, the microphone array is three-dimensional; thus, the terminal can generate three-dimensional surround sound from signals obtained by the microphones in the three-dimensional microphone array.

With reference to the first aspect, in a first possible implementation manner, the performing low-frequency compensation on the first initial directional signal, the second initial directional signal, the third initial directional signal, and the fourth initial directional signal based on the wiener filtering method to obtain the first directional signal, the second directional signal, the third directional signal, and the fourth directional signal may include: calculating an omni-directional microphone signal from the first initial directional signal, the second initial directional signal, the third initial directional signal and the fourth initial directional signal; calculating a self-power spectrum of the omni-directional microphone signal; calculating a first cross-power spectrum, a second cross-power spectrum, a third cross-power spectrum and a fourth cross-power spectrum, wherein the first cross-power spectrum is a cross-power spectrum of the omni-directional microphone signal and the first initial directional signal, the second cross-power spectrum is a cross-power spectrum of the omni-directional microphone signal and the second initial directional signal, the third cross-power spectrum is a cross-power spectrum of the omni-directional microphone signal and the third initial directional signal, and the fourth cross-power spectrum is a cross-power spectrum of the omni-directional microphone signal and the fourth initial directional signal; obtaining a first gain according to the first cross power spectrum and the self-power spectrum, obtaining a second gain according to the second cross power spectrum and the self-power spectrum, obtaining a third gain according to the third cross power spectrum and the self-power spectrum, and obtaining a fourth gain according to the fourth cross power spectrum and the self-power spectrum; and obtaining a first directional signal according to the first gain and the first initial directional signal, obtaining a second directional signal according to the second gain and the second initial directional signal, obtaining a third directional signal according to the third gain and the third initial directional signal, and obtaining a fourth directional signal according to the fourth gain and the fourth initial directional signal.

Since the mean square error between the output of the filter of the wiener filter and the desired output is the minimum, the terminal performs low-frequency compensation on the first initial directional signal, the second initial directional signal, the third initial directional signal and the fourth initial directional signal respectively based on the wiener filter method, and the obtained low-frequency loss of the first directional signal, the second directional signal, the third directional signal and the fourth directional signal can be compensated well. In this way, the signal-to-noise ratio of the first directional signal, the second directional signal, the third directional signal, and the fourth directional signal in the low frequency band is improved, that is, the signal-to-noise ratio of the first directional signal, the second directional signal, the third directional signal, and the fourth directional signal is improved.

With reference to the first possible implementation manner, in a second possible implementation manner, the obtaining a first gain according to the first cross-power spectrum and the self-power spectrum, obtaining a second gain according to the second cross-power spectrum and the self-power spectrum, obtaining a third gain according to the third cross-power spectrum and the self-power spectrum, and obtaining a fourth gain according to the fourth cross-power spectrum and the self-power spectrum may include: obtaining a first gain according to the ratio of the first cross-power spectrum to the self-power spectrum; obtaining a second gain according to the ratio of the second cross-power spectrum to the self-power spectrum; obtaining a third gain according to the ratio of the third cross-power spectrum to the self-power spectrum; and obtaining a fourth gain according to the ratio of the fourth cross-power spectrum to the self-power spectrum.

In the signal processing method provided by the present application, the terminal can obtain the gain of each cardioid directional signal, and the gain can provide directivity to the cardioid directional signal while preserving the frequency domain characteristics of the cardioid directional signal. Therefore, the compensated cardioid directional signal obtained by the terminal has directivity and has full-passband response characteristics, and the effect of low-frequency compensation is achieved.

In combination with the second possible implementation manner, in a third possible implementation manner, since it is time-delay-differentiated that 4 microphones in a microphone array included in the terminal obtain signals emitted by the same sound source, the terminal may obtain 4 cardioid directional signals according to the time-delay differences between the signals of the 4 microphones in combination with the signals of the 4 microphones. Specifically, the obtaining the first initial directivity signal, the second initial directivity signal, the third initial directivity signal, and the fourth initial directivity signal by performing the difference processing on the signal of the first microphone, the signal of the second microphone, the signal of the third microphone, and the signal of the fourth microphone may include: according to the time delay difference between the signal of the first microphone and the signal of the second microphone, carrying out differential processing on the signal of the first microphone and the signal of the second microphone to obtain a first initial directional signal; according to the time delay difference between the signal of the second microphone and the signal of the third microphone, carrying out differential processing on the signal of the second microphone and the signal of the third microphone to obtain a second initial directional signal; according to the time delay difference between the signal of the third microphone and the signal of the first microphone, carrying out differential processing on the signal of the third microphone and the signal of the first microphone to obtain a third initial directional signal; and carrying out differential processing on the signal of the fourth microphone and the signal of any microphone according to the time delay difference between the signal of the fourth microphone and the signal of any microphone of the first microphone, the second microphone and the third microphone to obtain a fourth initial directional signal.

In the process of carrying out differential processing on the signals of each microphone by the terminal, the time delay difference between the signals of each microphone can be eliminated. Therefore, the terminal can obtain the heart-shaped directional signal with better quality, and further obtain the surround sound with better quality.

With reference to the third possible implementation manner, in a fourth possible implementation manner, the obtaining the first sound field component, the second sound field component, the third sound field component, and the fourth sound field component according to the first directional signal, the second directional signal, the third directional signal, and the fourth directional signal may include: and performing inversion processing on the first directional signal, the second directional signal, the third directional signal and the fourth directional signal by adopting a decoding matrix to respectively obtain a first sound field component, a second sound field component, a third sound field component and a fourth sound field component, wherein the decoding matrix is preset.

Specifically, when the vector (denoted by C) and the decoding matrix (denoted by D) of each of the cardioid directional signals after the low frequency compensation are known, where D denotes a decoding matrix corresponding to 4 cardioid directional signals, the terminal may calculate B ═ D^-1C to obtain a matrix (denoted as B) of sound field components W, X, Y and Z. In this case, since the terminal can obtain the optimal solution of the matrix composed of the sound field components W, X, Y and Z by the matrix inversion method, that is, the quality of the sound field components W, X, Y and Z is good, the quality of the surround sound obtained according to the sound field components W, X, Y and Z is good.

With reference to the fourth possible implementation manner, in a fifth possible implementation manner, before the acquiring signals of the first microphone, the second microphone, the third microphone, and the fourth microphone in the microphone array, the method may further include: determining that the working state of the terminal is a first working state, wherein the first working state is any one of a horizontal state, a vertical state and a transverse state; when the terminal is in a first working state, the first microphone, the second microphone and the third microphone are located in a first plane, and the fourth microphone is located outside the first plane.

Wherein the plane parallel to the horizontal plane in the terminal is different in different operating states, i.e. the first plane is different. The terminal determines the first plane when determining the working state of the terminal. In addition, when the terminal is in different working states, the first microphone, the second microphone, the third microphone and the fourth microphone provided in the present application refer to different specific microphones. Therefore, the terminal can determine the first microphone, the second microphone, the third microphone and the fourth microphone according to different working states of the terminal, so that 4 cardioid directional signals obtained according to signals of the 4 microphones can reflect an actual sound field, and the quality of the obtained surround sound is good.

With reference to the first aspect or any one of the foregoing possible implementation manners, in a sixth possible implementation manner, after obtaining the first sound field component, the second sound field component, the third sound field component, and the fourth sound field component, the method may further include: and obtaining at least one path of loudspeaker excitation signal according to the first sound field component, the second sound field component, the third sound field component and the fourth sound field component.

Specifically, in the method provided by the present application, the terminal may include at least 1 speaker for outputting the at least one speaker excitation signal. In the case of known determination of the decoding matrix, e.g. the decoding matrix D for a 5.0 system loudspeaker_5.0The terminal can calculate and obtain an excitation signal C of the loudspeaker_5.0＝D_5.0B. Then, the terminal outputs the speaker excitation signal C through the speaker_5.0Thereafter, surround sound is generated.

With reference to the sixth possible implementation manner, in a seventh possible implementation manner, the acquiring signals of the first microphone, the second microphone, the third microphone, and the fourth microphone in the microphone array may further include: acquiring initial signals of a first microphone, a second microphone, a third microphone and a fourth microphone, wherein the initial signals of the first microphone, the second microphone, the third microphone and the fourth microphone are time domain signals; respectively carrying out Fourier transform on initial signals of the first microphone, the second microphone, the third microphone and the fourth microphone to obtain signals of the first microphone, the second microphone, the third microphone and the fourth microphone, wherein the signals of the first microphone, the second microphone, the third microphone and the fourth microphone are frequency domain signals. The terminal transforms the initial signals collected by each microphone to the frequency domain, so that the signals of each microphone obtained by transformation can be applied to the signal processing method based on the wiener filtering method, and the purpose of the application is achieved.

In a second aspect, the present application provides a signal processing apparatus applied to a terminal including a microphone array composed of at least 4 omni-directional microphones; the device includes: the device comprises an acquisition module, a difference module, a compensation module and an acoustic field generation module. The microphone array comprises a first microphone, a second microphone, a third microphone and a fourth microphone, wherein the acquisition module is used for acquiring signals of the first microphone, the second microphone, the third microphone and the fourth microphone in the microphone array, the first microphone, the second microphone and the third microphone are positioned in a first plane, and the fourth microphone is positioned outside the first plane; the difference module is used for carrying out difference processing on the signal of the first microphone, the signal of the second microphone, the signal of the third microphone and the signal of the fourth microphone acquired by the acquisition module to obtain a first initial directional signal, a second initial directional signal, a third initial directional signal and a fourth initial directional signal; a compensation module, configured to perform low-frequency compensation on the first initial directional signal, the second initial directional signal, the third initial directional signal, and the fourth initial directional signal obtained by the difference module based on a wiener filtering method, respectively, to obtain a first directional signal, a second directional signal, a third directional signal, and a fourth directional signal; and the sound field generation module is used for obtaining a first sound field component, a second sound field component, a third sound field component and a fourth sound field component according to the first directional signal, the second directional signal, the third directional signal and the fourth directional signal obtained by the compensation module.

With reference to the second aspect, in a first possible implementation manner, the compensation module may be specifically configured to calculate an omni-directional microphone signal according to the first initial directional signal, the second initial directional signal, the third initial directional signal, and the fourth initial directional signal obtained by the difference module; calculating a self-power spectrum of the omni-directional microphone signal; calculating a first cross-power spectrum, a second cross-power spectrum, a third cross-power spectrum and a fourth cross-power spectrum, wherein the first cross-power spectrum is a cross-power spectrum of the omni-directional microphone signal and the first initial directional signal, the second cross-power spectrum is a cross-power spectrum of the omni-directional microphone signal and the second initial directional signal, the third cross-power spectrum is a cross-power spectrum of the omni-directional microphone signal and the third initial directional signal, and the fourth cross-power spectrum is a cross-power spectrum of the omni-directional microphone signal and the fourth initial directional signal; obtaining a first gain according to the first cross power spectrum and the self-power spectrum, obtaining a second gain according to the second cross power spectrum and the self-power spectrum, obtaining a third gain according to the third cross power spectrum and the self-power spectrum, and obtaining a fourth gain according to the fourth cross power spectrum and the self-power spectrum; and obtaining a first directional signal according to the first gain and the first initial directional signal, obtaining a second directional signal according to the second gain and the second initial directional signal, obtaining a third directional signal according to the third gain and the third initial directional signal, and obtaining a fourth directional signal according to the fourth gain and the fourth initial directional signal.

With reference to the first possible implementation manner, in a second possible implementation manner, the compensation module may be specifically configured to obtain a first gain according to a ratio of the first cross-power spectrum to the self-power spectrum; obtaining a second gain according to the ratio of the second cross-power spectrum to the self-power spectrum; obtaining a third gain according to the ratio of the third cross-power spectrum to the self-power spectrum; and obtaining a fourth gain according to the ratio of the fourth cross-power spectrum to the self-power spectrum.

With reference to the second possible implementation manner, in a third possible implementation manner, the difference module may be specifically configured to perform difference processing on the signal of the first microphone and the signal of the second microphone according to a delay difference between the signal of the first microphone and the signal of the second microphone obtained by the obtaining module, so as to obtain a first initial directional signal; according to the time delay difference between the signal of the second microphone and the signal of the third microphone obtained by the obtaining module, carrying out differential processing on the signal of the second microphone and the signal of the third microphone to obtain a second initial directional signal; according to the time delay difference between the signal of the third microphone and the signal of the first microphone obtained by the obtaining module, carrying out differential processing on the signal of the third microphone and the signal of the first microphone to obtain a third initial directional signal; and according to the time delay difference between the signal of the fourth microphone and the signal of any one of the first microphone, the second microphone and the third microphone obtained by the obtaining module, carrying out differential processing on the signal of the fourth microphone and the signal of any one microphone to obtain a fourth initial directional signal.

With reference to the third possible implementation manner, in a fourth possible implementation manner, the sound field generating module may be specifically configured to perform inversion processing on the first directional signal, the second directional signal, the third directional signal, and the fourth directional signal obtained by the compensating module by using a decoding matrix, so as to obtain a first sound field component, a second sound field component, a third sound field component, and a fourth sound field component, respectively, where the decoding matrix is preset.

With reference to the fourth possible implementation manner, in a fifth possible implementation manner, the apparatus may further include: and determining a module. The terminal comprises an acquisition module, a determination module and a processing module, wherein the acquisition module is used for acquiring signals of a first microphone, a second microphone, a third microphone and a fourth microphone in a microphone array; when the terminal is in a first working state, the first microphone, the second microphone and the third microphone are located in a first plane, and the fourth microphone is located outside the first plane.

With reference to the sixth possible implementation manner, in a seventh possible implementation manner, the apparatus may further include: and an acquisition module. The acquisition module is used for acquiring initial signals of a first microphone, a second microphone, a third microphone and a fourth microphone before the acquisition module acquires signals of the first microphone, the second microphone, the third microphone and the fourth microphone in the microphone array, wherein the initial signals of the first microphone, the second microphone, the third microphone and the fourth microphone are time-domain signals; the acquisition module is specifically configured to perform fourier transform on the initial signals of the first microphone, the second microphone, the third microphone, and the fourth microphone obtained by the acquisition module, respectively, to obtain signals of the first microphone, the second microphone, the third microphone, and the fourth microphone, where the signals of the first microphone, the second microphone, the third microphone, and the fourth microphone are frequency domain signals.

In a third aspect, the present application provides a terminal, including: one or more processors, memory, audio circuits, and buses; the audio circuit comprises at least 4 microphones; the memory is to store at least one instruction; the one or more processors, the memory, and the audio circuit are connected via the bus, and when the terminal is operating, the one or more processors execute at least one instruction stored in the memory to cause the terminal to perform the signal processing method according to the first aspect or any possible implementation manner thereof via the audio circuit.

In a fourth aspect, the present application provides a computer storage medium comprising: at least one instruction; the at least one instruction, when executed on a computer, causes the computer to perform a method of signal processing as in the first aspect or its various possible implementations.

In a fifth aspect, the present application provides a computer program product comprising: at least one instruction; the at least one instruction, when executed on a computer, causes the computer to perform a method of signal processing as in the first aspect or its various possible implementations.

It should be noted that, in the third aspect of the present application, the one or more processors may be an integration of the functional modules of the second aspect, such as the obtaining module, the difference module, the compensation module, the sound field generating module, and the speaker signal generating module, and the one or more processors may implement the functions of the functional modules of the second aspect. For the detailed description and the beneficial effect analysis of each module in the second aspect and the third aspect, reference may be made to the corresponding description and the technical effect in the first aspect and various possible implementation manners thereof, which are not described herein again.

Drawings

Fig. 1 is a schematic diagram of a possible structure of a terminal according to an embodiment of the present invention;

fig. 2a is a schematic diagram of a microphone array included in a terminal according to an embodiment of the present invention;

fig. 2b is a schematic diagram of a microphone array according to an embodiment of the present invention;

fig. 2c is a schematic diagram of a microphone array according to an embodiment of the invention;

fig. 3 is a first flowchart of a signal processing method according to an embodiment of the present invention;

fig. 4 is a flowchart of a signal processing method according to an embodiment of the present invention;

fig. 5 is a flowchart of a signal processing method according to an embodiment of the present invention;

fig. 6 is a fourth flowchart of a signal processing method according to an embodiment of the present invention;

fig. 7 is a fifth flowchart of a signal processing method according to an embodiment of the present invention;

fig. 8 is a sixth flowchart of a signal processing method according to an embodiment of the present invention;

fig. 9 is a seventh flowchart of a signal processing method according to an embodiment of the present invention;

fig. 10 is a flowchart eight of a signal processing method according to an embodiment of the present invention;

fig. 11 is a first schematic structural diagram of a signal processing apparatus according to an embodiment of the present invention;

fig. 12 is a schematic structural diagram of a signal processing apparatus according to an embodiment of the present invention;

fig. 13 is a schematic structural diagram of a signal processing apparatus according to a third embodiment of the present invention;

fig. 14 is a schematic structural diagram of a signal processing apparatus according to a fourth embodiment of the present invention;

fig. 15 is a schematic diagram of a possible structure of a terminal according to an embodiment of the present invention.

Detailed Description

The embodiment of the invention provides a signal processing method, which can be applied to a process of generating surround sound by a terminal, particularly to a process of processing an obtained cardioid directional signal by the terminal, can compensate low-frequency loss of the cardioid directional signal, and improve the signal-to-noise ratio of a low-frequency band of the cardioid directional signal.

For example, the terminal in the embodiment of the present invention may be a mobile phone, a wearable device, an Augmented Reality (AR) \ Virtual Reality (VR) device, a tablet computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a Personal Digital Assistant (PDA), and the like, which is not limited in this respect.

The following embodiments take a mobile phone as an example to illustrate how a terminal implements a specific technical solution in the embodiments. As shown in fig. 1, the terminal in this embodiment may be a mobile phone 100. The embodiment will be specifically described below by taking the mobile phone 100 as an example.

It should be understood that the illustrated handset 100 is merely one example of a terminal and that the handset 100 may have more or fewer components than shown in the figures, may combine two or more components, or may have a different configuration of components. The various components shown in fig. 1 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

As shown in fig. 1, the cellular phone 100 includes: the audio circuit 110, the memory 120, the input unit 130, the display unit 140, the sensor 150, an RF (Radio Frequency) circuit 160, a Wireless Fidelity (Wi-Fi) module 170, a processor 180, and a power supply 190. Those skilled in the art will appreciate that the handset configuration shown in fig. 1 is not intended to be limiting and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.

The following describes the components of the mobile phone 100 in detail with reference to fig. 1:

included in the audio circuitry 110 are a microphone component 111 and a speaker component 112, the audio circuitry 110 providing an audio interface between a user and the handset 100. The microphone assembly 111 may include at least 4 omni-directional microphones for collecting sound signals. Optionally, the number of microphones in the microphone array is not unique, for example, 4 or 6, and this is not limited in the embodiment of the present invention. The sound signal obtained by the omnidirectional microphone is an omnidirectional signal, and the omnidirectional microphone is a 360-degree recording and is mainly used for chat communication. Wherein, in some embodiments, the one or more omni-directional microphones may be disposed at edge locations on the respective sides of the handset 100. Such as a position on each face that is offset toward either vertex or side. Illustratively, the microphone array of the terminal as shown in fig. 2a comprises 6 microphones mic1, mic2, mic3, mic4, mic5 and mic 6. Wherein the mic1, the mic2, the mic3, the mic4, the mic5 and the mic6 are located at positions biased to the vertex on the corresponding plane of the terminal. The speaker assembly 112 may include at least 1 speaker, and the speaker assembly 112 is configured to output speaker excitation signals to generate surround sound.

The memory 120 may be used to store software programs and data. The processor 180 executes software programs and data stored in the memory 120 to perform various functions and data processing of the cellular phone 100. The memory 120 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound recording function or a sound playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone 100, and the like. Further, the memory 120 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The input unit 130, such as a touch screen, may be used to receive input numeric or character information and generate signal inputs related to user settings and function control of the cell phone 100. Specifically, the input unit 130 may include a touch panel 131, which may collect a touch operation of a user on or near the touch panel 131 (e.g., an operation of the user on or near the touch panel 131 using any suitable object or accessory such as a finger, a stylus pen, etc.), and drive a corresponding connection device according to a preset program. Alternatively, the touch panel 131 may include two parts, a touch detection device and a touch controller (not shown in fig. 1). The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 180, and can receive and execute instructions sent by the processor 180. In addition, the touch panel 131 may be implemented by various types such as a resistive type, a capacitive type, an infrared ray, and a surface acoustic wave.

The display unit 140 (i.e., a display screen) may be used to display information input by or provided to a User and a Graphical User Interface (GUI) of various menus of the mobile phone 100. The display unit 140 may include a display panel 141 disposed on the front surface of the cellular phone 100. The display panel 141 may be configured in the form of a liquid crystal display, a light emitting diode, or the like.

In some embodiments, the bottom of the front surface of the mobile phone 100 may also be provided with an optical touch key; a touch panel 131 and a display panel 141 are also provided, and the touch panel 131 is covered on the display panel 141. When the touch panel 131 detects a touch operation on or near the touch panel, the touch operation is transmitted to the processor 180 to determine a touch event, and then the processor 180 provides a corresponding visual output on the display panel 141 according to the type of the touch event. Although the touch panel 131 and the display panel 141 are shown as two separate components in fig. 1 to implement the input and output functions of the mobile phone 100, in some embodiments, the touch panel 131 and the display panel 141 may be integrated to implement the input and output functions of the mobile phone 100, and the integrated touch panel 131 and the display panel 141 may be referred to as a touch display screen.

In some other embodiments, the touch panel 131 may further include a pressure-sensitive sensor, so that when a user performs a touch operation on the touch panel, the touch panel can detect a pressure of the touch operation, and the mobile phone 100 can detect the touch operation more accurately.

The handset 100 may also include at least one sensor 150, such as a light sensor, a motion sensor, a geomagnetic sensor, and other sensors. The motion sensor may be configured to obtain the orientation information of the mobile phone 100 according to the information collected by the acceleration sensor. For example, the orientation information may be information indicating an angle between the mobile phone 100 and a horizontal plane, such as the mobile phone 100 being in a flat state when the included angle is 0 degrees. The accelerometer sensor can detect the acceleration in all directions (generally three axes), can detect the gravity and the direction when the accelerometer sensor is static, and can be used for identifying the application of the gesture of the mobile phone (such as horizontal and vertical screen conversion, related games and magnetometer gesture calibration), vibration identification related functions (such as pedometer and knocking) and the like; as for other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor, which can be configured on the mobile phone 100, further description is omitted here.

The RF circuit 160 may be used for receiving and transmitting signals during information transmission and reception or during a call, and may receive downlink information from a base station and then process the downlink information to the processor 180; in addition, data relating to uplink is transmitted to the base station. Typically, the RF circuitry includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the RF circuitry 110 may also communicate with networks and other mobile devices via wireless communications. The wireless communication may use any communication standard or protocol including, but not limited to, global system for mobile communications, general packet radio service, code division multiple access, wideband code division multiple access, long term evolution, email, short message service, and the like.

Wi-Fi is a short-range wireless transmission technology, and the mobile phone 100 may help a user to send and receive e-mails, browse webpages, access streaming media, and the like through the Wi-Fi module 170, and provide the user with wireless broadband internet access, such as audio data sending and receiving.

The processor 180 is a control center of the mobile phone 100, connects various parts of the entire mobile phone by using various interfaces and lines, and performs various functions of the mobile phone 100 and processes data by operating or executing software programs stored in the memory 120 and calling data stored in the memory 120, thereby performing overall monitoring of the mobile phone. In some embodiments, processor 180 may include one or more processing units; the processor 180 may also integrate an application processor, which primarily handles operating systems, user interfaces, applications, etc., and a modem processor, which primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 180. In addition, the processor 180 may further process the sound signal collected by the microphone assembly 111 to obtain a heart-shaped directional signal, so as to obtain a speaker excitation signal for generating surround sound. The surround sound provided in the embodiment of the present invention may be a three-dimensional surround sound, and the surround sound may be obtained from a cardioid directional signal based on an x-axis, a cardioid directional signal based on a y-axis, and a cardioid directional signal based on a z-axis, where the x-axis, the y-axis, and the z-axis may be coordinate axes of a natural coordinate system.

The handset 100 also includes a power supply 190 (such as a battery) to power the various components. The power supply may be logically coupled to the processor 180 through a power management system to manage charging, discharging, and power consumption functions through the power management system. It is understood that in the following embodiments, the power supply 190 may be used to power the speaker assembly 111 and the microphone assembly 112.

Although not shown, the mobile phone 100 may further include a bluetooth module, a camera, and other functional modules, which are not described in detail herein. For example, the bluetooth module is used for performing information interaction with other devices through a short-range communication protocol such as bluetooth. The mobile phone 100 can establish a bluetooth connection with a wearable electronic device (e.g., a smart watch) having a bluetooth module through the bluetooth module, so as to perform data interaction.

The methods in the following embodiments can be implemented in the mobile phone 100 having the above hardware structure.

It is to be understood that the method steps in the subsequent embodiments of the present invention may be executed by the terminal, or the execution main body of the signal Processing method provided in the embodiments of the present invention may also be a part of a functional module in the terminal, such as a Central Processing Unit (CPU), which is not limited in this embodiment of the present invention. In the following, the embodiment of the present invention takes a terminal to execute a signal processing method as an example, and details an image processing method provided by the embodiment of the present invention.

Wherein, the at least 4 microphones can be divided into at least one microphone group. Each of the at least one microphone sets may include a first microphone, a second microphone, a third microphone, and a fourth microphone. The first microphone, the second microphone and the third microphone may be located in a first plane, the fourth microphone may be located outside the first plane, the first plane may be any plane on the terminal, and the first plane may be parallel to the horizontal plane. For example, a terminal such as a mobile phone may be regarded as a hexahedron, i.e., the terminal has 6 planes. In this case, the first plane may be any one of the 6 planes.

It should be noted that the first microphone, the second microphone, and the third microphone may form a triangle on the first plane, such as a right triangle or an acute triangle, and the embodiment of the present invention is not limited thereto. A connection line between the fourth microphone and any one of the first microphone, the second microphone and the third microphone may be perpendicular to the first plane, for example, a connection line between the fourth microphone and the second microphone may be perpendicular to the first plane.

In addition, the first microphone, the second microphone and the third microphone may be located at an edge position on a first plane of the terminal, such as a position offset to any vertex or a position offset to any side in the first plane. Likewise, the fourth microphone may be located at an edge position on the corresponding plane.

Further, fig. 2b shows the relationship among the positions of the mic1, mic2, mic3, mic4, mic5 and mic6 in fig. 2a on the terminal shown in fig. 2 a.

Wherein, assuming that the planes of the mic1, the mic2, the mic3 and the mic6 shown in fig. 2b are first planes, the mic4 and the mic5 are located outside the first planes, and the first planes are parallel to the horizontal plane, i.e. the planes of the x axis and the y axis. In this case, if the mic4 is the fourth microphone, the first microphone, the second microphone, and the third microphone may be divided into 3 microphone groups of mic1, mic2, and mic3, or mic6, mic2, and mic3, or mic1, mic2, and mic6, respectively. If the mic5 is the fourth microphone, the first microphone, the second microphone and the third microphone may be respectively mic6, mic1 and mic2, or mic6, mic1 and mic3, or mic2, mic1 and mic3, which are 3 microphone groups in total. That is, when the terminal includes a microphone array of 6 microphones as shown in fig. 2b, the microphone array may be divided into 6 microphone groups.

Since each of the microphone groups in the microphone array included in the terminal in the embodiment of the present invention includes the first microphone, the second microphone, the third microphone, and the fourth microphone, for convenience of describing the signal processing method provided in the embodiment of the present invention, the embodiment of the present invention is described below by taking only the first microphone, the second microphone, the third microphone, and the fourth microphone in one microphone group included in the terminal as an example.

Illustratively, the microphone array as shown in fig. 2c includes a microphone group, the first, second and third microphones in the microphone group are mic1, mic2 and mic3, respectively, and the fourth microphone may be mic 4. In the following, a signal processing method provided by an embodiment of the present invention is described with reference to fig. 2c only to illustrate an example of a microphone array included in a terminal.

An embodiment of the present invention provides a signal processing method, as shown in fig. 3, the signal processing method may include S301 to S304:

s301, the terminal acquires signals of a first microphone, a second microphone, a third microphone and a fourth microphone in the microphone array.

Illustratively, the above step 301 may be performed by the audio circuit 160 included in the handset 100 shown in fig. 1.

Specifically, in conjunction with the microphone array shown in fig. 2c, the signal of the first microphone acquired by the terminal in step 301 may be a signal of mic1, denoted as S₁(jw); the signal of the second microphone may be a signal of mic2, denoted as S₂(jw); the signal of the third microphone may be a signal of mic3, denoted as S₃(jw); the signal of the fourth microphone may be a signal of mic4, denoted as S₄(jw). Wherein, the above-mentioned S₁(jw)、S₂(jw)、S₃(jw) and S₄(jw) is a frequency domain signal.

Further, in the process of generating the surround sound by the terminal, the surround sound may be generated according to the above S₁(jw)、S₂(jw)、S₃(jw) and S₄(jw) obtaining the cardioid directional signal based on the x-axis, the cardioid directional signal based on the y-axis, and the cardioid directional signal based on the z-axis. Specifically, in the signal processing method provided in the embodiment of the present invention, the terminal may execute step 302:

s302, the terminal performs difference processing on the signal of the first microphone, the signal of the second microphone, the signal of the third microphone, and the signal of the fourth microphone to obtain a first initial directional signal, a second initial directional signal, a third initial directional signal, and a fourth initial directional signal.

Specifically, since the first microphone, the second microphone and the third microphone are located in the first plane and the first plane may be parallel to the horizontal plane, the terminal may select signals of the microphones from signals of the first microphone, signals of the second microphone and signals of the third microphone to perform differential processing between each two microphones, so as to obtain a cardioid directional signal based on the x-axis and a cardioid directional signal based on the y-axis. Similarly, in a case where a connection line of the fourth microphone and the second microphone is perpendicular to the first plane, the terminal may perform differential processing on the signal of the second microphone and the signal of the fourth microphone to obtain a cardioid directional signal based on the z-axis. The terminal is the same for the difference processing between the signals of the first microphone, the second microphone and the third microphone.

Wherein, any one of the initial directional signals is obtained by the signals of the two microphones through differential processing, and the differential processing appears as a high-pass filter in frequency response, so that the low frequency of the signals of the two microphones through the differential processing is weakened, and the obtained initial directional signal has low-frequency loss.

In addition, although the initial directivity signal has a low-frequency loss, the initial directivity signal has a certain directivity, for example, in the x-axis direction, because the initial directivity signal is a cardioid directivity signal.

Specifically, the x-axis based cardioid directional signal and the y-axis based cardioid directional signal may include a first initial directional signal (denoted as C)₁(jw)), a second initial directional signal (denoted as C)₂(jw)) and a third initial directivity signal (denoted as C)₃(jw)). The heart-shaped directional signal based on the z-axis may be a fourth initial directional signal (denoted by C)₄(jw))。

Subsequently, the terminal may perform low frequency compensation on the first initial directivity signal, the second initial directivity signal, the third initial directivity signal, and the fourth initial directivity signal. Specifically, in the signal processing method provided in the embodiment of the present invention, the terminal may execute step 303:

and S303, the terminal respectively performs low-frequency compensation on the first initial directional signal, the second initial directional signal, the third initial directional signal and the fourth initial directional signal based on a wiener filtering method to obtain the first directional signal, the second directional signal, the third directional signal and the fourth directional signal.

It should be noted that wiener filtering is an optimal estimator for stationary process based on the minimum mean square error criterion. The mean square error between the output of such a filter and the desired output is minimal and is an optimal filtering system.

Specifically, since the mean square error between the output of the filter of the wiener filter and the desired output is the minimum, the terminal performs low-frequency compensation on the first initial directional signal, the second initial directional signal, the third initial directional signal, and the fourth initial directional signal based on the wiener filter method, and the obtained low-frequency loss of the first directional signal, the second directional signal, the third directional signal, and the fourth directional signal can be compensated well. In this way, the signal-to-noise ratio of the first directional signal, the second directional signal, the third directional signal, and the fourth directional signal in the low frequency band is improved, that is, the signal-to-noise ratio of the first directional signal, the second directional signal, the third directional signal, and the fourth directional signal is improved.

Subsequently, the terminal may obtain sound field components W, X, Y and Z in the three-dimensional sound field generating surround sound from the low-frequency compensated cardioid directional signals, such as the first directional signal, the second directional signal, the third directional signal, and the fourth directional signal. Specifically, in the signal processing method provided in the embodiment of the present invention, the terminal may execute step 304:

s304, the terminal obtains a first sound field component, a second sound field component, a third sound field component and a fourth sound field component according to the first directional signal, the second directional signal, the third directional signal and the fourth directional signal.

Illustratively, the steps 302 and 304 can be performed by the processor 180 included in the handset 100 shown in fig. 1.

Specifically, the first sound field component may be a sound field component X, the second sound field component may be a sound field component Y, the third sound field component may be a sound field component Z, and the fourth sound field component may be a sound field component W. Wherein sound field components X, Y and Z are both directional; the sound field component X is a 1 st order signal based on an X-axis, the sound field component Y is a 1 st order signal based on a Y-axis, and the sound field component Z is a 1 st order signal based on a Z-axis. The sound field component W is a 0-order signal having no directivity.

It should be noted that, in the signal processing method provided in the embodiment of the present invention, the terminal employs a wiener-based filtering method, so that the low-frequency loss of the obtained cardioid directional signal can be compensated, and the signal-to-noise ratio of the cardioid directional signal is further improved. Thus, the terminal can improve the quality of the sound field components W, X, Y and Z obtained from the compensated cardioid directional signal. That is, the terminal can improve the quality of surround sound obtained from the sound field components W, X, Y and Z.

In addition, the terminal provided by the embodiment of the invention comprises a microphone array, wherein the first microphone, the second microphone and the third microphone are positioned in the first plane, and the fourth microphone is positioned outside the first plane; thus, the microphone array is three-dimensional; thus, the terminal can generate three-dimensional surround sound from signals obtained by the microphones in the three-dimensional microphone array.

Optionally, if the microphone array included in the terminal according to the embodiment of the present invention includes a plurality of microphone sets, the terminal may perform the signal processing method on signals of the first microphone, the second microphone, the third microphone, and the fourth microphone in each of the plurality of microphone sets, respectively, to obtain a plurality of sound field components W, X, Y and Z. Subsequently, the terminal may perform an operation such as averaging processing on the above-described plurality of sets of sound field components W, X, Y and Z, resulting in a set of sound field components W, X, Y and Z for generating surround sound. Here, the above-described one set of sound field components W, X, Y and Z for generating surround sound is obtained from the plurality of sets of sound field components W, X, Y and Z, and therefore the quality of the above-described one set of sound field components W, X, Y and Z for generating surround sound is better, that is, the quality of the surround sound is better.

Further, in a possible implementation manner, the step 303 is described in detail below with steps 401 and 404, that is, a process of the terminal performing low frequency compensation on each initial directional signal based on wiener filtering is described in detail. For example, in the signal processing method shown in fig. 4, the step 303 shown in fig. 3 may include steps 401 and 404:

s401, the terminal calculates an omni-directional microphone signal according to the first initial directional signal, the second initial directional signal, the third initial directional signal and the fourth initial directional signal.

Wherein the omni-directional microphone signal is obtained by combining two cardioid directional approximations to form an omni-directional microphone signal

To obtain, the omni-directional microphone signal can be marked as P₁(jw)。

S402, the terminal calculates the self-power spectrum of the omni-directional microphone signal.

The power spectrum of a signal is an abbreviation of a power spectral density function, and is defined as the signal power in a unit frequency band, and represents the variation condition of the signal power along with frequency, namely the distribution condition of the signal power in a frequency domain.

The self-power spectrum of the omni-directional microphone signal may be used to indicate the energy of the overall omni-directional microphone signal, corresponding to the ensemble of the 4 microphone signals.

Specifically, the omni-directional microphone signal P may be obtained by the terminal using E { P (jw) P × (jw) }₁(jw) of the self-power spectrum, i.e. the omni-directional microphone signal P₁The self-power spectrum of (jw) can be denoted as E { P (jw) }.

S403, the terminal calculates a first cross-power spectrum, a second cross-power spectrum, a third cross-power spectrum and a fourth cross-power spectrum.

The first cross-power spectrum is a cross-power spectrum of the omni-directional microphone signal and the first initial directional signal, the second cross-power spectrum is a cross-power spectrum of the omni-directional microphone signal and the second initial directional signal, the third cross-power spectrum is a cross-power spectrum of the omni-directional microphone signal and the third initial directional signal, and the fourth cross-power spectrum is a cross-power spectrum of the omni-directional microphone signal and the fourth initial directional signal.

Specifically, the terminal may adopt E { C }₁(jw) P x (jw) } obtaining said first cross-power spectrum; by using

E{C₂(jw) P x (jw) } obtaining said second cross-power spectrum; using E { C₃(jw) P x (jw) } obtaining said third cross-power spectrum; using E { C₄(jw) P × (jw) } results in the fourth cross-power spectrum described above.

It should be noted that any cross-power spectrum may be the energy of the corresponding initial directional signal obtained from the signal of any omni-directional microphone.

S404, the terminal obtains a first gain according to the first cross-power spectrum and the self-power spectrum, obtains a second gain according to the second cross-power spectrum and the self-power spectrum, obtains a third gain according to the third cross-power spectrum and the self-power spectrum, and obtains a fourth gain according to the fourth cross-power spectrum and the self-power spectrum.

Wherein the first gain can be denoted as G₁(jw), the second gain may be denoted as G₂(jw), third gainCan be recorded as G₃(jw), the fourth gain may be denoted as G₄(jw)。

Optionally, in the method provided in the embodiment of the present invention, the step 404 may be specifically a step 501. For example, in the signal processing method flowchart shown in fig. 5, step 404 shown in fig. 4 may be replaced by step 501:

s501, the terminal obtains a first gain according to the ratio of the first cross-power spectrum to the self-power spectrum; obtaining a second gain according to the ratio of the second cross-power spectrum to the self-power spectrum; obtaining a third gain according to the ratio of the third cross-power spectrum to the self-power spectrum; and obtaining a fourth gain according to the ratio of the fourth cross-power spectrum to the self-power spectrum.

Illustratively, the terminal can adopt

Calculating to obtain a first gain G₁(jw), second gain G₂(jw), third gain G₃(jw) and a fourth gain G₄(jw)。

Then, the terminal may perform low frequency compensation on the corresponding initial directional signals according to the obtained gains. Specifically, in the signal processing method provided in the embodiment of the present invention, as shown in fig. 4 or fig. 5, the terminal may execute step 405:

s405, the terminal obtains a first directional signal according to the first gain and the first initial directional signal, obtains a second directional signal according to the second gain and the second initial directional signal, obtains a third directional signal according to the third gain and the third initial directional signal, and obtains a fourth directional signal according to the fourth gain and the fourth initial directional signal.

Specifically, the terminal can adopt

Calculating to obtain a first directional signal, a second directional signal, a third directional signal and a fourth directional signal;

wherein the first directional signal can be recorded as

The second directional signal can be recorded as

The third directional signal can be recorded as

The fourth directional signal can be recorded as

In the signal processing method according to the embodiment of the present invention, the terminal may obtain the gain of each of the cardioid directional signals, and the gain may provide directivity to the cardioid directional signal while preserving the frequency domain characteristics of the cardioid directional signal. Therefore, the compensated cardioid directional signal obtained by the terminal has directivity and has full-passband response characteristics, and the effect of low-frequency compensation is achieved.

Further, in a possible implementation manner, since it is time-delayed that 4 microphones in a microphone array included in the terminal obtain signals emitted by the same sound source, the terminal may obtain 4 cardioid directional signals according to the time-delayed difference between the signals of the 4 microphones and the signals of the 4 microphones. Specifically, step 302 in the above embodiment may specifically be step 601. For example, in the signal processing method shown in fig. 6, step 302 in fig. 3 may be replaced with step 601:

s601, the terminal carries out difference processing on the signal of the first microphone and the signal of the second microphone according to the time delay difference between the signal of the first microphone and the signal of the second microphone to obtain a first initial directional signal; according to the time delay difference between the signal of the second microphone and the signal of the third microphone, carrying out differential processing on the signal of the second microphone and the signal of the third microphone to obtain a second initial directional signal; according to the time delay difference between the signal of the third microphone and the signal of the first microphone, carrying out differential processing on the signal of the third microphone and the signal of the first microphone to obtain a third initial directional signal; and carrying out differential processing on the signal of the fourth microphone and the signal of any microphone according to the time delay difference between the signal of the third microphone and the signal of any microphone of the first microphone, the second microphone and the third microphone to obtain a fourth initial directional signal.

Specifically, the terminal can adopt

C₁(jw)＝S₁(jw)-S₂(jw)exp(-jwτ₁₂)

C₂(jw)＝S₂(jw)-S₃(jw)exp(-jwτ₂₃)

C₃(jw)＝S₃(jw)-S₁(jw)exp(-jwτ₃₁)

C₄(jw)＝S₄(jw)-S₂(jw)exp(-jwτ₄₂)

Calculating to obtain a first initial directional signal C₁(jw), second initial directivity Signal C₂(jw) third initial directivity Signal C₃(jw) and a fourth initial directivity signal C₄(jw)。

Wherein, the signal of the first microphone can be recorded as S₁(jw), the signal of the second microphone can be denoted as S₂(jw), the signal of the third microphone can be denoted as S₃(jw), the signal of the fourth microphone can be denoted as S₄(jw). The time delay difference between the signal of the first microphone and the signal of the second microphone can be recorded as tau₁₂The time delay difference between the signal of the second microphone and the signal of the third microphone can be recorded as tau₂₃The time delay difference between the signal of the third microphone and the signal of the first microphone can be recorded as tau₃₁(ii) a The time delay difference between the signal of the fourth microphone and the signal of the second microphone can be recorded as tau₄₂。

In the embodiment of the invention, in the process of carrying out differential processing on the signals of each microphone by the terminal, the time delay difference between the signals of each microphone can be eliminated. Therefore, the terminal can obtain the heart-shaped directional signal with better quality, and further obtain the surround sound with better quality.

Further, in a possible implementation manner, in the signal processing method provided in the embodiment of the present invention, the step 304 may specifically be a step 701. For example, as the signal processing method shown in fig. 7, the step 304 shown in fig. 3 may be replaced by the step 701:

and S701, the terminal adopts a decoding matrix to perform inversion processing on the first directional signal, the second directional signal, the third directional signal and the fourth directional signal to respectively obtain a first sound field component, a second sound field component, a third sound field component and a fourth sound field component.

The decoding matrix may be preset for the terminal. Obviously, in the case that the matrix (denoted as B) composed of the sound field components W, X, Y and Z and the decoding matrix (denoted as D) are known, the terminal may obtain a decoding equation C as DB, where C is a vector composed of the above-mentioned each low-frequency compensated cardioid directional signal (e.g. the first directional signal). The decoding matrices for different speakers are different, for example, the decoding matrices for 5.0 system speakers are different from the decoding matrices for 5.1 system speakers.

Similarly, in a case where the vector C and the decoding matrix D of the cardioid directional signal after low frequency compensation are known, where D denotes a decoding matrix corresponding to 4 cardioid directional signals, the terminal may calculate that B ═ D^-1C, sound field components W, X, Y and Z are computed.

The terminal can obtain the optimal solution of the matrix formed by the sound field components W, X, Y and Z by the matrix inversion method, the quality of the sound field components W, X, Y and Z is good, and therefore the quality of surround sound obtained according to the sound field components W, X, Y and Z is good.

Further, the terminal may determine the first plane in the terminal before processing the signals of at least 4 microphones. The method provided by the embodiment of the present invention may further include step 801 before step 301. Illustratively, as the signal processing method shown in fig. 8, step 301 shown in fig. 3 may further include step 801:

s801, the terminal determines that the working state of the terminal is a first working state.

The first working state of the terminal can be any one of a flat state, a vertical state and a transverse state. When the terminal is in a first working state, the microphone array of the terminal comprises the first microphone, the second microphone, the third microphone and the fourth microphone. The plane parallel to the horizontal plane in the terminal is different in different operating states, i.e. the first plane is different. The terminal determines the first plane when determining the working state of the terminal.

Illustratively, the terminal is shown in fig. 2a in a flat position. Obviously, if the terminal shown in fig. 2a is in a vertical state, the first plane of the terminal may be the plane of the mic1, mic5, mic4 and mic 2. If the terminal shown in fig. 2a is in a vertical state, the first plane of the terminal may be the plane of the mic2, mic4, and mic 3.

In the embodiment of the present invention, when the terminal is in different working states, the microphones specifically referred to by the first microphone, the second microphone, the third microphone, and the fourth microphone are different. For example, when the first operating state of the terminal shown in fig. 2a is the vertical operating state, the first microphone, the second microphone, and the third microphone may be 3 microphones of mic1, mic5, mic4, and mic2 shown in fig. 2b, and the fourth microphone may be 1 microphone of mic3 and mic 6. When the first operating state of the terminal is the lateral operating state, the first, second and third microphones may be mic3, mic4 and mic2 shown in fig. 2a, and the fourth microphone may be 1 of mic1, mic5 and mic6 shown in fig. 2 a.

It should be noted that the terminal may obtain the angle (<90 degrees) with the horizontal plane by using a sensor such as an acceleration sensor. When the included angle is less than 30 degrees, the operation state of the terminal is approximately considered to be the square state. When the included angle is larger than 60 degrees, the terminal is approximately considered to be vertical to the horizontal plane, and then whether the terminal is held vertically or horizontally is judged. However, when the included angle is 30 to 60 degrees, the terminal can prompt that sound field recording cannot be accurately performed. Alternatively, the terminal may display a prompt message "recording not supported" or the like on its display screen, such as the display panel 141 of the mobile phone 100 shown in fig. 1.

Optionally, the terminal is determined according to its current system parameters related to the interface of the terminal. For example, if the current system parameter of the terminal is parameter 1, it indicates that the terminal is held vertically, that is, the terminal is in a vertical state; if the current system parameter of the terminal is parameter 2, it indicates that the terminal is held in the horizontal direction, i.e. the terminal is in the horizontal state. In the embodiment of the present invention, a specific implementation manner of the "terminal determines whether to hold the terminal vertically or horizontally" is not limited. For example, the terminal may also perform the above-mentioned "determining whether the terminal holds vertically or horizontally" according to data collected by a sensor such as an accelerometer.

It should be noted that, with the signal processing method provided in the embodiment of the present invention, the terminal may determine the first microphone, the second microphone, the third microphone, and the fourth microphone according to different working states of the terminal, so that the 4 cardioid directional signals obtained according to the signals of the 4 microphones can better reflect an actual sound field, and further, the obtained surround sound has better quality.

Further, the signal processing method provided in the embodiment of the present invention may further include step 901 before step 301, and step 301 may be replaced with step 902. Illustratively, in the signal processing method shown in fig. 9, step 901 may be further included before step 301 in fig. 8, and step 301 may be replaced with step 902:

s901, the terminal collects initial signals of a first microphone, a second microphone, a third microphone and a fourth microphone.

And the initial signals of the first microphone, the second microphone, the third microphone and the fourth microphone are all time-domain signals.

S902, the terminal respectively performs Fourier transform on initial signals of the first microphone, the second microphone, the third microphone and the fourth microphone to obtain signals of the first microphone, the second microphone, the third microphone and the fourth microphone.

And the signals of the first microphone, the second microphone, the third microphone and the fourth microphone are frequency domain signals.

It should be noted that the terminal transforms the initial signals collected by each microphone to the frequency domain, so that the transformed signals of each microphone can be applied to the low-frequency filtering method based on wiener filtering, so as to achieve the purpose of the present application.

Further, the signal processing method provided in the embodiment of the present invention may further include step 1001 after step 304 or step 701. For example, as the signal processing method shown in fig. 10, step 701 in fig. 7 may further include step 1001:

s1001, the terminal obtains at least one path of loudspeaker excitation signals according to the first sound field component, the second sound field component, the third sound field component and the fourth sound field component.

In the method provided by the embodiment of the present invention, the terminal may include at least 1 speaker, and is configured to output the at least one speakerThe acoustic exciter excites the signal. When the decoding matrix is determined, for example, the decoding matrix D corresponding to the 5.0 system speaker is described in the above embodiment with reference to step 701_5.0The terminal can calculate and obtain an excitation signal C of the loudspeaker_5.0＝D_5.0B. Then, the terminal outputs the speaker excitation signal C through the speaker_5.0Then, the surround sound is generated.

The above description mainly introduces the scheme provided by the embodiment of the present invention from the perspective of the terminal. It is understood that the terminal includes corresponding hardware structures and/or software modules for performing the respective functions in order to implement the above-described functions. Those of skill in the art will readily appreciate that the present invention can be implemented in hardware or a combination of hardware and computer software, in conjunction with the exemplary algorithm steps described in connection with the embodiments disclosed herein. Whether a function is performed as hardware or computer software drives hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

The signal processing apparatus applied to the terminal according to the above method example may perform functional module division, for example, each functional module may be divided according to each function, or two or more functions may be integrated into one processing module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. It should be noted that, the division of the modules in the embodiment of the present invention is schematic, and is only a logic function division, and there may be another division manner in actual implementation.

In the case of dividing each functional module by corresponding functions, fig. 11 shows a schematic diagram of a possible composition of the signal processing apparatus provided in the embodiment of the present invention, and as shown in fig. 11, the signal processing apparatus 11 may include: an acquisition module 111, a difference module 112, a compensation module 113 and a sound field generation module 114.

Wherein the obtaining module 111 is configured to support the signal processing apparatus 11 to perform S301 and S902 in the above embodiments, and/or other processes for the technology described herein. A difference module 112 for supporting the signal processing apparatus 11 to perform S302 and S601 in the above embodiments, and/or other processes for the techniques described herein. A compensation module 113 for supporting the signal processing device 11 to perform S303, S401, S402, S403, S404, S405, and S501 in the above embodiments, and/or other processes for the techniques described herein. A sound field generating module 114 for supporting the signal processing apparatus 11 to perform S304 and S701 in the above-described embodiments, and/or other processes for the techniques described herein.

Further, as shown in fig. 12, a schematic diagram of another possible composition of the signal processing apparatus provided in the embodiment of the present invention is shown. In fig. 12, the signal processing apparatus 11 may further include: a determination module 115. Wherein the determining module 115 is configured to support the signal processing apparatus 11 to perform S801 in the above-described embodiment and/or other processes for the techniques described herein.

Further, as shown in fig. 13, a schematic diagram of another possible composition of the signal processing apparatus provided in the embodiment of the present invention is shown. In fig. 13, the signal processing apparatus 11 may further include: an acquisition module 116. Wherein the acquisition module 116 is configured to support the signal processing apparatus 11 to perform S901 in the above embodiments and/or other processes for the techniques described herein.

Further, as shown in fig. 14, another possible composition diagram of the signal processing apparatus provided in the embodiment of the present invention is shown. In fig. 14, the signal processing apparatus 11 may further include: a speaker signal generation module 117. Wherein the speaker signal generating module 117 is configured to support the signal processing apparatus 11 to perform S1001 in the above-mentioned embodiment and/or other processes for the technology described herein.

It should be noted that all relevant contents of each step related to the above method embodiment may be referred to the functional description of the corresponding functional module, and are not described herein again. The signal processing device applied to the terminal is used for executing the signal processing method, so that the same effect as the signal processing method can be achieved.

In case of using an integrated unit, the above-mentioned obtaining module 111, the difference module 112, the compensation module 113, the sound field generating module 114, the determining module 115, the speaker signal generating module 117, and the like may be integrated into one processing module. The processing module may be a Processor or a controller, such as a CPU, a general purpose Processor, a Digital Signal Processor (DSP), an Application-Specific integrated circuit (ASIC), a Field Programmable Gate Array (FPGA), or other Programmable logic device, a transistor logic device, a hardware component, or any combination thereof. Which may implement or execute the various illustrative logical blocks, modules, and circuits described in connection with the disclosure. The processing units described above may also be combinations that perform computing functions, e.g., including one or more microprocessor combinations, DSPs and microprocessors, and the like. The storage module may be a memory. The above-mentioned acquisition module 151 may be implemented by a microphone in an audio circuit of the terminal. Besides, the signal processing apparatus applied to the terminal may further include other functional modules, for example, the signal processing apparatus may further include a speaker signal output module, and the speaker signal output module may be implemented by a speaker in an audio circuit of the terminal.

When the processing module is a processor and the storage module is a memory, the embodiment of the present invention provides a terminal 15 as shown in fig. 15. As shown in fig. 15, the terminal 15 includes: processor 151, memory 152, audio circuitry 153, and bus 154. Wherein the processor 151, the memory 152 and the audio circuit 153 are connected to each other by a bus 154. Illustratively, the audio circuit 153 may include at least 4 microphones. In addition, the audio circuit 153 may also include at least 1 speaker.

The bus 154 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus 154 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 15, but this is not intended to represent only one bus or type of bus.

The detailed description of each module in the signal processing apparatus applied to the terminal 15 and the technical effects brought by each module after executing the related method steps in the foregoing embodiments provided by the embodiments of the present invention may refer to the related description in the embodiments of the method of the present invention, and are not repeated herein.

Through the above description of the embodiments, it is clear to those skilled in the art that, for convenience and simplicity of description, the foregoing division of the functional modules is merely used as an example, and in practical applications, the above function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device may be divided into different functional modules to complete all or part of the above described functions. For the specific working processes of the system, the apparatus and the unit described above, reference may be made to the corresponding processes in the foregoing method embodiments, and details are not described here again.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit. The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone computer program product, may be stored in a computer readable storage medium.

When implemented in software, the technical solutions of the present application may be implemented in whole or in part in the form of a computer program product. The computer program product includes at least one instruction. When loaded and executed on a computer, cause the processes or functions described in accordance with embodiments of the invention to occur, in whole or in part. The computer may be by computer, a special purpose computer, a network of computers, or other programmable device. The instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer may transmit from one website, computer, server, or data center to another website, computer, server, or data center via a wired means such as coaxial cable, fiber optics, Digital Subscriber Line (DSL), or wireless means such as infrared, radio, microwave, etc. The computer readable medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more available media. The usable medium may be a magnetic medium such as a floppy Disk, a hard Disk, or a magnetic tape, or a semiconductor medium such as a Solid State Disk (SSD).

The above description is only an embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions within the technical scope of the present disclosure should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A signal processing method, characterized by being applied to a terminal comprising a microphone array of at least 4 omni-directional microphones;

the method comprises the following steps:

acquiring signals of a first microphone, a second microphone, a third microphone and a fourth microphone in the microphone array, wherein the first microphone, the second microphone and the third microphone are positioned in a first plane, and the fourth microphone is positioned outside the first plane;

performing differential processing on the signal of the first microphone, the signal of the second microphone, the signal of the third microphone and the signal of the fourth microphone to obtain a first initial directivity signal, a second initial directivity signal, a third initial directivity signal and a fourth initial directivity signal;

respectively performing low-frequency compensation on the first initial directional signal, the second initial directional signal, the third initial directional signal and the fourth initial directional signal based on a wiener filtering method to obtain a first directional signal, a second directional signal, a third directional signal and a fourth directional signal;

obtaining a first sound field component, a second sound field component, a third sound field component and a fourth sound field component according to the first directional signal, the second directional signal, the third directional signal and the fourth directional signal;

the obtaining a first directional signal, a second directional signal, a third directional signal and a fourth directional signal by performing low frequency compensation on the first initial directional signal, the second initial directional signal, the third initial directional signal and the fourth initial directional signal respectively based on a wiener filtering method includes:

calculating an omni-directional microphone signal from the first initial directional signal, the second initial directional signal, the third initial directional signal, and the fourth initial directional signal;

calculating a self-power spectrum of the omni-directional microphone signal;

calculating a first cross-power spectrum, a second cross-power spectrum, a third cross-power spectrum and a fourth cross-power spectrum, wherein the first cross-power spectrum is a cross-power spectrum of the omni-directional microphone signal and the first initial directional signal, the second cross-power spectrum is a cross-power spectrum of the omni-directional microphone signal and the second initial directional signal, the third cross-power spectrum is a cross-power spectrum of the omni-directional microphone signal and the third initial directional signal, and the fourth cross-power spectrum is a cross-power spectrum of the omni-directional microphone signal and the fourth initial directional signal;

obtaining a first gain according to the first cross power spectrum and the self-power spectrum, obtaining a second gain according to the second cross power spectrum and the self-power spectrum, obtaining a third gain according to the third cross power spectrum and the self-power spectrum, and obtaining a fourth gain according to the fourth cross power spectrum and the self-power spectrum;

obtaining a first directional signal according to the first gain and the first initial directional signal, obtaining a second directional signal according to the second gain and the second initial directional signal, obtaining a third directional signal according to the third gain and the third initial directional signal, and obtaining a fourth directional signal according to the fourth gain and the fourth initial directional signal.

2. The method of claim 1, wherein deriving the first gain from the first cross-power spectrum and the self-power spectrum comprises:

and obtaining a first gain according to the ratio of the first cross-power spectrum to the self-power spectrum.

3. The method of claim 2, wherein the differentiating the signal of the first microphone to obtain a first initial directional signal comprises:

and carrying out differential processing on the signal of the first microphone and the signal of the second microphone according to the time delay difference between the signal of the first microphone and the signal of the second microphone to obtain a first initial directional signal.

4. The method of claim 3, wherein deriving a first soundfield component from the first directional signal comprises:

and performing inversion processing on the first directional signal by adopting a decoding matrix to obtain a first sound field component, wherein the decoding matrix is preset.

5. The method of claim 4, wherein prior to the obtaining signals of a first microphone, a second microphone, a third microphone, and a fourth microphone of the array of microphones, further comprising:

determining that the working state of the terminal is a first working state, wherein the first working state is any one of a horizontal state, a vertical state and a transverse state; when the terminal is in the first working state, the first microphone, the second microphone and the third microphone are located in a first plane, and the fourth microphone is located outside the first plane.

6. The method according to any of claims 1-5, further comprising, after said obtaining the first, second, third and fourth sound field components:

and obtaining at least one path of loudspeaker excitation signals according to the first sound field component, the second sound field component, the third sound field component and the fourth sound field component.

7. A signal processing apparatus, characterized by being applied to a terminal including a microphone array composed of at least 4 omnidirectional microphones;

the device comprises:

an obtaining module, configured to obtain signals of a first microphone, a second microphone, a third microphone, and a fourth microphone in the microphone array, where the first microphone, the second microphone, and the third microphone are located in a first plane, and the fourth microphone is located outside the first plane;

a difference module, configured to perform difference processing on the signal of the first microphone, the signal of the second microphone, the signal of the third microphone, and the signal of the fourth microphone acquired by the acquisition module to obtain a first initial directivity signal, a second initial directivity signal, a third initial directivity signal, and a fourth initial directivity signal;

a compensation module, configured to perform low-frequency compensation on the first initial directivity signal, the second initial directivity signal, the third initial directivity signal, and the fourth initial directivity signal obtained by the difference module based on a wiener filtering method, respectively, so as to obtain a first directivity signal, a second directivity signal, a third directivity signal, and a fourth directivity signal;

a sound field generating module, configured to obtain a first sound field component, a second sound field component, a third sound field component, and a fourth sound field component according to the first directional signal, the second directional signal, the third directional signal, and the fourth directional signal obtained by the compensating module;

the compensation module is specifically configured to calculate an omni-directional microphone signal according to the first initial directional signal, the second initial directional signal, the third initial directional signal, and the fourth initial directional signal obtained by the difference module; calculating a self-power spectrum of the omni-directional microphone signal; calculating a first cross-power spectrum, a second cross-power spectrum, a third cross-power spectrum and a fourth cross-power spectrum, wherein the first cross-power spectrum is a cross-power spectrum of the omni-directional microphone signal and the first initial directional signal, the second cross-power spectrum is a cross-power spectrum of the omni-directional microphone signal and the second initial directional signal, the third cross-power spectrum is a cross-power spectrum of the omni-directional microphone signal and the third initial directional signal, and the fourth cross-power spectrum is a cross-power spectrum of the omni-directional microphone signal and the fourth initial directional signal; obtaining a first gain according to the first cross power spectrum and the self-power spectrum, obtaining a second gain according to the second cross power spectrum and the self-power spectrum, obtaining a third gain according to the third cross power spectrum and the self-power spectrum, and obtaining a fourth gain according to the fourth cross power spectrum and the self-power spectrum; obtaining a first directional signal according to the first gain and the first initial directional signal, obtaining a second directional signal according to the second gain and the second initial directional signal, obtaining a third directional signal according to the third gain and the third initial directional signal, and obtaining a fourth directional signal according to the fourth gain and the fourth initial directional signal.

8. The apparatus of claim 7, wherein the compensation module is specifically configured to obtain a first gain according to a ratio of the first cross-power spectrum to the self-power spectrum.

9. The apparatus according to claim 8, wherein the differentiating module is specifically configured to perform a differential processing on the signal of the first microphone and the signal of the second microphone according to the time delay difference between the signal of the first microphone and the signal of the second microphone obtained by the obtaining module to obtain a first initial directional signal.

10. The apparatus according to claim 9, wherein the sound field generating module is specifically configured to perform an inversion process on the first directional signal obtained by the compensating module by using a decoding matrix to obtain a first sound field component, and the decoding matrix is preset.

11. The apparatus of claim 10, further comprising:

the determining module is used for determining that the working state of the terminal is a first working state before the acquiring module acquires signals of a first microphone, a second microphone, a third microphone and a fourth microphone in the microphone array, wherein the first working state is any one of a flat state, a vertical state and a transverse state; wherein the first, second, and third microphones lie within a first plane and the fourth microphone lies outside of the first plane.

12. The apparatus of any one of claims 7-11, further comprising:

and the loudspeaker signal generating module is used for obtaining at least one path of loudspeaker excitation signals according to the first sound field component, the second sound field component, the third sound field component and the fourth sound field component obtained by the sound field generating module.

13. A terminal, comprising: one or more processors, memory, audio circuits, and buses;

the audio circuit comprises at least 4 microphones;

the memory is configured to store at least one instruction, the one or more processors, the memory, and the audio circuit are connected via the bus, and when the terminal is operating, the one or more processors execute the at least one instruction stored in the memory to cause the terminal to perform the signal processing method according to any one of claims 1 to 6 via the audio circuit.

14. A computer storage medium, comprising: at least one instruction;

when the at least one instruction is run on a computer, the computer is caused to perform the signal processing method of any one of claims 1-6.