Disclosure of Invention
The invention aims to provide an acoustic and image signal mixing terminal which has a simple structure, high signal synchronism of an acoustic sensor and high-efficiency positioning of a sound source.
The technical scheme of the invention is as follows:
an acoustic and image signal mixing terminal comprising:
the acoustic sensor array is used for sensing and collecting sound signals of a sound source;
the digital sound signal processing module is used for receiving the sound signal and processing the sound signal and converting the sound signal into a digital signal;
an image sensor located at the center of the acoustic sensor array for capturing an optical image of the sound source;
The main control unit is used for receiving the digital signals and the optical images and processing the digital signals and the optical images to obtain acoustic data frames and image data frames;
The acoustic sensor array is provided with 8 branches, each branch is provided with 8 acoustic sensor units, each 8 acoustic sensor units comprise 4 main channel sensors and 4 auxiliary channel sensors, each main channel sensor is electrically connected with the corresponding auxiliary channel sensor, and the acoustic sensor units are uniformly distributed on a curved surface or a plane;
Wherein, the main control unit comprises:
a crystal oscillator for emitting a bit clock signal and a byte clock signal;
a bit clock phase-locked loop for transmitting the bit clock signal to a bit clock distribution unit;
a byte phase-locked loop for transmitting the byte clock signal to a byte clock distribution unit;
A bit clock distribution unit for receiving the bit clock signal and distributing an output bit clock signal corresponding to the converter;
a byte clock distribution unit for receiving the bit clock signal and distributing an output byte clock signal corresponding to the converter;
and the converter is used for receiving the output bit clock signal and the byte clock signal and converting and outputting the two output signals to the main channel sensor.
In the above technical solution, the acoustic and image signal hybrid terminal further includes a power supply unit for supplying power to the main control unit.
In the above technical solution, the power supply unit includes:
the voltage conversion module is used for converting alternating current into direct current and supplying power to the main control unit;
The standby power supply module is used for supplying power to the main control unit under the condition of power failure;
And the control switch is used for controlling the opening and closing of the main control unit.
In the above technical solution, each of the main channel sensors is electrically connected to one of the converters, and the number of the converters is 32.
In the above technical solution, the bit clock distribution unit includes 8 bit clock distributors, and the bit clock distributors adopt one-minute-four clock distributors.
In the above technical solution, the byte clock distribution unit includes 8 byte clock distributors, and the byte clock distributors adopt one-minute-four clock distributors.
In the above technical scheme, the main control unit adopts an FPGA chip.
In the above technical solution, the image sensor is a planar array type visible light image sensor.
In the above technical solution, the acoustic sensor units are distributed in a spiral involute type circular array on a curved surface or a plane.
Another object of the present invention is to provide a processing method based on an acoustic and image signal mixing terminal, comprising the steps of:
(1) Starting an acoustic and image signal mixing terminal, converting a crystal oscillator in a main control unit into a bit clock signal and a byte clock signal through a bit clock phase-locked loop and a byte clock phase-locked loop respectively, and sending the bit clock signal and the byte clock signal to a bit clock distribution unit and a byte clock distribution unit;
(2) The bit clock distribution unit and the byte clock distribution unit respectively distribute the bit clock signal and the byte clock signal into 32 bit clock output signals and 32 byte clock output signals through 8 bit clock distributors and 8 byte clock distributors, and send the 32 bit clock output signals and the 32 byte clock output signals to 32 converters corresponding to the 32 bit clock output signals and the 32 byte clock output signals one by one;
(3) After receiving the 32 bit clock output signals and the 32 byte clock output signals, the 32 converters convert the 32 bit clock output signals and the 32 byte clock output signals into 32 output signals, and send the 32 output signals to 32 main channel sensors corresponding to the 32 output signals one by one, and each main channel sensor is used for transmitting the signals to the corresponding auxiliary channel sensor, so that the synchronization of the bit clock signals and the byte clock signals of the acoustic sensor units is completed;
(4) The acoustic sensor array after clock signal excitation starts to work, collects the sound signal of the sound source, sends the sound signal to the digital sound signal processing module, and the digital sound signal processing module carries out filtering and digital signal conversion, processes and converts the sound signal into a digital signal;
(5) The digital acoustic signal processing module sends the processed digital signal to the main control unit, the main control unit adopts a time division multiplexing and space division multiplexing technology to intercept 20 bits of data in the sound signal acquired in the acoustic sensor unit, intercept 0-15 bits of the 20 bits of data when a sound source is weakest, intercept 1-16 bits of the 20 bits of data when the sound source is slightly strong, and the like, and according to the actual condition of the sound source, each path of output signal of the acoustic sensor unit is intercepted to obtain 16 bits of effective data;
(6) A camera in the image sensor shoots an optical image of a sound source, and the optical image is sent to a main control unit;
(7) The main control unit receives the digital signal and the optical image and processes the digital signal and the optical image into an acoustic data frame and an image data frame according to the digital signal and the optical image.
The invention has the advantages and positive effects that:
1. The main control unit sends out clock source signals, and transmits clock signals from the same clock source to the acoustic sensor units on the acoustic sensor array through multi-layer grading, so that the clock signals of the acoustic sensor units are effectively ensured to be synchronous, the synchronous error of the clock signals is reduced, and the reliability of data sampling of the acoustic sensor is improved.
2. When the main control unit carries out acoustic data framing, the signals output by the main channel sensor and the auxiliary channel sensor are processed by adopting time division multiplexing and space division multiplexing technologies, effective data are intercepted for framing, the data integrity is improved, and the data transmission uniformity is ensured.
3. The standby power supply module can continuously supply power to the acoustic and image signal mixing terminal when power failure or other unexpected situations occur, and the acoustic and image signal mixing terminal is supported to continuously work.
Detailed Description
The present invention will be described in further detail with reference to specific examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the present invention.
Example 1
As shown in the drawings, the acoustic and image signal mixing terminal of the present invention includes:
the acoustic sensor array is used for sensing and collecting sound signals of a sound source;
the digital sound signal processing module is used for receiving the sound signal and processing the sound signal and converting the sound signal into a digital signal;
The image sensor is positioned in the center of the acoustic sensor array and is used for shooting an optical image of the sound source;
the main control unit is used for receiving the digital signal and the optical image and processing the digital signal and the optical image to obtain an acoustic data frame and an image data frame;
the acoustic sensor array is provided with 8 branches, each branch is provided with 8 acoustic sensor units, each 8 acoustic sensor units comprise 4 main channel sensors and 4 auxiliary channel sensors, each main channel sensor is electrically connected with the corresponding auxiliary channel sensor, and the acoustic sensor units are distributed in a spiral involute type circular array on a curved surface or a plane;
Wherein, the main control unit (FPGA chip) comprises:
a crystal oscillator for emitting a bit clock signal and a byte clock signal;
a bit clock phase-locked loop for transmitting a bit clock signal to the bit clock distribution unit;
A byte phase-locked loop for transmitting the byte clock signal to the byte clock distribution unit;
A bit clock distribution unit for receiving the bit clock signal and distributing the bit clock signal corresponding to the output of the converter;
A byte clock distribution unit for receiving the bit clock signal and distributing the output byte clock signal corresponding to the converter;
and the converter is used for receiving the output bit clock signal and the byte clock signal and converting and outputting the two output signals to the main channel sensor.
Further, each main channel sensor is correspondingly and electrically connected with one converter, the number of the converters is 32, the bit clock distribution unit comprises 8 bit clock distributors, the bit clock distributor adopts a one-to-four clock distributor, the byte clock distribution unit comprises 8 byte clock distributors, and the byte clock distributor adopts a one-to-four clock distributor.
Further, the acoustic and image signal mixing terminal further comprises a power supply unit for supplying power to the main control unit.
Further, as shown in fig. 2, the main control unit sends a bit clock signal and a byte clock signal based on the same clock source to the acoustic sensor array, and distributes the bit clock signal and the byte clock signal into 32 output bit clock signals and 32 output byte clock signals through an 8 bit clock distributor (a quarter clock distributor) and an 8 byte clock distributor (a quarter clock distributor) in the bit clock distributor and the byte clock distributor, respectively, where the total of 64 output clock signals correspondingly enter 32 converters, the converters adopt two-in-two converters to synthesize 2 input signals into 1-channel signals, the 64 input signals enter the converters and are converted into 32 output signals and are sent to the main channel sensor corresponding to each converter, after the 32 main channel sensors on the acoustic sensor array receive the 32 output signals, the 32 clock output signals are distributed to the corresponding 32 auxiliary channel sensors, so that signal synchronization of the 64 sensors on the acoustic sensor array is completed, the signal sent by the same clock source reaches all the acoustic sensors, and after the acoustic sensor array is triggered, the acoustic signal is synchronously acquired, and the acoustic sensor array is completely stimulated.
Further, the sampling frequency of the acoustic data is 48kHz, the main channel sensor and the auxiliary channel sensor are transmitted to the main control unit in a group of 20 bits through the same line, and the data transmitted by the main channel sensor and the auxiliary channel sensor are of the same grade; the data acquired by the acoustic sensor are output by the main channel sensor and the auxiliary channel sensor at the high level and the low level of the excitation clock respectively, the output signals are synchronous with the excitation clock, and the output signals are converted into digital signals through 32 digital acoustic signal processing modules which are correspondingly arranged and are sent to the main control unit. The main control unit adopts a time division multiplexing and space division multiplexing technology, when a sound source is weakest, 0-15 bits of 20-bit acoustic data are intercepted, when the sound source is stronger, 1-16 bits of 20-bit data are intercepted, and the like, according to the actual situation of the sound source, each path of output signal is intercepted to obtain 16-bit effective data, namely 2Byte, and 64 paths of output signals are intercepted to obtain 128Byte data; the data are stored in a buffer domain in a complementary form for clock domain conversion, data framing is carried out by taking effective data obtained by 8 times of sampling as a unit, 1024Byte data are obtained as a group for acoustic data framing, and then a frame header is added before the acoustic data framing, so that a complete acoustic data frame is obtained.
After receiving the processed digital signals and the optical images acquired by the image sensors, the main control unit can intercept 64 paths of signals sent by the acoustic sensor arrays to obtain 128Byte data, the data can be stored in a buffer domain in a complementary form to be converted in a clock domain, and the data is framed by taking effective data obtained by 8 times of sampling as a unit, namely 1024Byte data is framed as a group, and the main control unit adopts a time and space two-dimensional framing mode to carry out acoustic data framing; the optical image data sent by the image sensor is sent to the main control unit in 1024Byte units, and an image data frame can be obtained without combination processing.
When the acoustic and image signal hybrid terminal is used, the method specifically comprises the following steps:
(1) Starting an acoustic and image signal mixing terminal, and enabling a crystal oscillator in a main control unit to simultaneously send a bit clock signal and a byte clock signal, wherein the bit clock phase-locked loop and the byte clock phase-locked loop respectively send the bit clock signal and the byte clock signal to a bit clock distribution unit and a byte clock distribution unit;
(2) The bit clock distribution unit and the byte clock distribution unit respectively distribute the bit clock signals and the byte clock signals into 32 output bit clock signals and 32 output byte clock signals through the bit clock distributor and the byte clock distributor, and send the 32 output bit clock signals and the 32 output byte clock signals to the 32 converters corresponding to the 32 output bit clock signals and the 32 output byte clock signals;
(3) After receiving 64 output signals in total, the 32 converters convert the 64 output signals into 32 output signals, send the 32 output signals to the 32 main channel sensors corresponding to the 32 output signals, and transmit the signals to the corresponding auxiliary channel sensors through each main channel sensor, so that the synchronization of the bit clock signals and the byte clock signals of the acoustic sensor units is completed;
(4) The acoustic sensor array after clock signal excitation starts to work, collects the sound signal of the sound source, sends the sound signal to the digital sound signal processing module, processes and converts the sound signal into a digital signal, and intercepts 20-bit acoustic data in the sound signal collected in the acoustic sensor unit;
(5) The digital acoustic signal processing module sends the processed digital signal to the main control unit, the main control unit adopts a time division multiplexing and space division multiplexing technology, when a sound source is weakest, 0-15 bits in 20-bit acoustic data are intercepted, when the sound source is stronger, 1-16 bits in 20-bit data are intercepted, and according to the actual condition of the sound source, each path of output signal can be intercepted to obtain 16-bit effective data;
(6) A camera in the image sensor shoots an optical image of a sound source and sends the optical image to the main control unit;
(7) The main control unit receives the digital signal and the optical image and processes the digital signal and the optical image into an acoustic data frame and an image data frame.
The main control unit sends out clock signals for the same clock source to excite all acoustic sensors on the acoustic sensor array to perform clock signal synchronization, a plurality of clock sources are not required to send out signals to excite different acoustic sensors on the acoustic sensor array, the phenomenon of signal asynchronism is reduced, the spatial distribution characteristics of the acoustic wave transmission channels are considered, in order to ensure the convergence of subsequent beam forming algorithm processing and the like, the excitation clock synchronization error is one hundred times lower than the sampling frequency, and the excitation clock synchronization error is lower than 200ns at the 48kHz sampling frequency, so that the high synchronism and the high reliability of sound source data sampling are ensured.
Example 2
Based on embodiment 1, the image sensor is placed in the center of the acoustic sensor array, the pixels of the visible light image sensor reach 500W, automatic focusing, automatic exposure control and automatic white balance are achieved, the image sensor receives an excitation clock sent by the main control unit, the main control unit configures an operating mode for the image sensor, the image sensor starts image acquisition under the action of the excitation clock, the sampling frequency is set to 30 frames/second, and flicker effects can be eliminated by refreshing image display under the sampling frequency.
In order to match with acoustic data, the length of an image data frame acquired by the image sensor is adjusted to be the same as that of the acoustic data frame, namely, the length of each line is 1024 bytes, so that the data uniformity of external transmission is high.
Example 3
As shown in fig. 3, the power supply unit includes a voltage conversion module for converting alternating current into direct current and supplying power to the main control unit;
the standby power supply module is used for supplying power to the main control unit under the condition of power failure;
and the control switch is used for controlling the opening and closing of the main control unit.
Wherein, the standby power module comprises a rechargeable lithium battery.
Under the condition that the acoustic and image signal mixing terminal is powered off or unexpected, the standby power supply module supplies power to the main control unit, and normal use of the acoustic and image signal mixing terminal can not be interrupted or stopped.
The foregoing describes one embodiment of the present invention in detail, but the description is only a preferred embodiment of the present invention and should not be construed as limiting the scope of the invention. All changes and modifications that come within the meaning and range of equivalency of the claims are to be embraced within their scope.