CN114964175B

CN114964175B - Multi-sensor data synchronous acquisition device and acquisition method

Info

Publication number: CN114964175B
Application number: CN202210322339.8A
Authority: CN
Inventors: 裴海龙; 韩澍
Original assignee: South China University of Technology SCUT
Current assignee: South China University of Technology SCUT
Priority date: 2022-03-30
Filing date: 2022-03-30
Publication date: 2023-05-23
Anticipated expiration: 2042-03-30
Also published as: CN114964175A

Abstract

The invention discloses a ZYNQ-based multi-sensor data synchronous acquisition device and an acquisition method. The invention designs a data synchronous acquisition method of a plurality of sensors by utilizing FPGA and ARM resources on SOC, which is realized by the comprehensive actions of a high-precision time reference module, a camera time service module, a laser radar time service module, a GNSS receiver time service module, an inertial measurement unit time service module and a data receiving and integrating module, and performs time synchronous control on the data acquisition of the plurality of sensors, thereby greatly improving the real-time performance and time accuracy of the data acquisition.

Description

Multi-sensor data synchronous acquisition device and acquisition method

Technical Field

The invention relates to the technical field of multi-sensor data acquisition and mobile mapping, in particular to a multi-sensor data synchronous acquisition device and method based on ZYNQ.

Background

The synchronous acquisition technology of the multi-sensor data is one of key technologies of mobile measurement, synchronous acquisition means that the multi-source data of each sensor are unified with the same time reference, the data synchronization of each sensor is guaranteed, the synchronous acquisition device is a tie of the multi-sensor, and the multi-sensor in the mobile measurement system is integrated on the synchronous acquisition device to perform acquisition work. With the development of image mapping technology, laser scanning technology and inertial navigation technology, the traditional and single spatial data acquisition system has been rapidly replaced by a multi-sensor integrated acquisition system, the traditional manual measurement mode has not been capable of meeting the acquisition requirements of modern geospatial data, the topographic information data has been developed from traditional map data to new data forms such as true three dimensions, street views and the like, and the mobile measurement system integrating multiple sensors has become a trend of data acquisition of the modern geospatial system.

In the past, the main control chip of the multi-sensor data synchronous acquisition device mostly adopts the SOC of x86 or ARM framework, and the SOC is characterized in that the serial computing capacity is stronger, but the concurrent processing capacity is weaker, and meanwhile, the acquisition device using the SOC is generally provided with a non-real-time operating system, so that the time synchronization of the multi-sensor data acquisition is difficult to ensure.

Currently, along with the application and popularization of mapping technology, the acquisition precision and the real-time performance of the multi-sensor data acquisition system are required, and the master control chip of the multi-sensor acquisition device is required to have stronger concurrency and real-time processing capacity. The Xilinx company pushes out that the SOC of the ZYNQ series takes ARM architecture as a leading part, meanwhile, the FPGA resource is integrated inside the SOC, and the FPGA has the characteristics of strong concurrent processing capacity and capability of strictly ensuring the real-time performance of program execution. The multi-sensor data signal interface is integrated at the FPGA end of the ZYNQ, real-time data acquisition and synchronous control are carried out on the multi-sensor by utilizing the concurrency processing capability and the real-time property of the FPGA, and the integration and storage functions of the multi-sensor data are realized at the ARM end. The advantages brought by the combination of ARM and FPGA lead the ZYNQ series SOC to be widely applied in the design of a multi-sensor data synchronous acquisition device.

Disclosure of Invention

The invention aims to solve the problem of insufficient acquisition instantaneity and synchronism of the existing multi-sensor acquisition system and provides a multi-sensor data synchronous acquisition device and an acquisition method. The main control chip 2 of the system adopts ZYNQ series SOC, so that synchronous acquisition control of most commonly used sensors can be realized, the sensors are connected into the FPGA end 3 for data acquisition by utilizing the strong parallel and real-time processing capacity of the FPGA end 3 on the main control chip 2, and the acquired data are integrated and stored by utilizing the ARM end 4 on the main control chip 2 to integrate the existing various hardware resources. The device is highly integrated, can be loaded on various mapping devices moving at high speed, such as a mapping vehicle, a mapping unmanned aerial vehicle and the like, and provides a high-precision data source for a mapping system.

The purpose of the invention is realized by the following hardware platform scheme:

the data synchronous acquisition device comprises a hardware device carrier plate 1 and a plurality of hardware devices, wherein the hardware device carrier plate 1 is composed of a circuit board and is used for integrating the plurality of hardware devices;

the hardware equipment is all connected with the hardware equipment carrier plate 1, includes: the main control chip 2 is divided into an FPGA end 3 and an ARM end 4;

A laser radar 5, a sensor for acquiring three-dimensional distance data;

the inertial measurement unit 6 is used for acquiring the sensor of the angular velocity and acceleration data in the motion process of the device;

a GNSS receiver 7, a sensor for acquiring real-time position data of the device, and for providing PPS second pulse signals;

a camera 8, a sensor for acquiring image data;

the on-board differential crystal oscillator 9 is used for providing high-precision time information for the device;

a wireless data transmission station 10 for remote communication of the device;

the local disk 11 is used for storing the multi-sensor data acquired by the local device;

a high precision time reference module 12 for providing high precision time for the device;

the camera time service module 13 is used for realizing data acquisition and time service of the camera;

the laser radar time service module 14 is used for realizing data acquisition and time service of the laser radar 5;

the GNSS receiver time service module 15 is used for realizing data acquisition and time service of the GNSS receiver 7;

the inertial measurement unit timing module 16 is used for realizing data acquisition and timing of the inertial measurement unit 6;

the data receiving and integrating module 17 is used for receiving and integrating the multi-sensor data after synchronization is completed;

The multi-sensor data synchronous acquisition method based on the multi-sensor data synchronous acquisition device comprises the following steps of:

s1, starting a high-precision time reference module 12, receiving PPS second pulse signals and $GPRMC message data sent by a GNSS receiver 7, simultaneously receiving pulse signals sent by an on-board differential crystal oscillator 9, internally analyzing the $GPRMC message data by the high-precision time reference module 12 to extract UTC time information, simultaneously carrying out pulse counting on the PPS second pulse signals and the pulse signals sent by the on-board differential crystal oscillator 9, and carrying out integration calculation on the pulse count values and the UTC time information to obtain real-time high-precision time for operation of a data synchronous acquisition device;

s2, a camera timing module 13 is started, a real-time high-precision time obtained by calculating a pulse signal sent by the on-board differential crystal oscillator 9 and a high-precision time reference module 12 is received, the pulse signal sent by the on-board differential crystal oscillator 9 is subjected to frequency division and then is output to a camera 8 to trigger the camera 8 to collect image data, meanwhile, the moment of triggering the camera 8 to collect the data is obtained by internal calculation of the camera timing module 13 and is used as time data corresponding to the image data, and the time data corresponding to the image data is stored in an ARM end 4 of the main control chip 2;

s3, starting a laser radar time service module 14, receiving $GPRMC message data and PPS second pulse signals sent by a GNSS receiver 7, simultaneously forwarding the PPS second pulse signals to a laser radar 5, monitoring the PPS second pulse signals in the laser radar time service module 14, simultaneously buffering the $GPRMC message data, and sending the $GPRMC message data to the laser radar 5 at 9600bps baud rate after the laser radar time service module 14 monitors the rising edge of the PPS second pulse and delays for 200 ms;

S4, starting a GNSS receiver time service module 15, receiving $GPRMC message data sent by a GNSS receiver 7, analyzing UTC time information and position information in the message, packaging the UTC time information and the position information, and storing the packaged UTC time information and position information in an ARM end 4 of a main control chip 2;

s5, starting an inertial measurement unit time service module 16, receiving a pulse signal sent by the on-board differential crystal oscillator 9 and real-time high-precision time calculated by a high-precision time reference module 12, dividing the frequency of the pulse signal sent by the on-board differential crystal oscillator 9, outputting the divided frequency to the inertial measurement unit 6 to trigger the inertial measurement unit 6 to acquire data, reading the data by an SPI protocol in the inertial measurement unit time service module 16 after the inertial measurement unit 6 finishes data acquisition, simultaneously, calculating the time for triggering the inertial measurement unit 6 to acquire the data as a time stamp corresponding to the inertial measurement unit data in the internal calculation, and storing the time stamp and the corresponding inertial measurement unit data in an ARM end 4 of the main control chip 2;

s6, starting a data receiving and integrating module 17, receiving sensor data which are subjected to timing by a camera timing module 13, a laser radar timing module 14, a GNSS receiver timing module 15 and an inertial measurement unit timing module 16, compressing the data, and storing the compressed data in a local disk 11.

Further, the main control chip 2 is a ZYNQ SOC series of Xilinx company, the model is XCZU3EG, CPU of ARM Cortex-A53 architecture of 1.3GHz of a dual core and FPGA resources of 154K programmable logic units are integrated, USB3.0, serial ATA and 1000Mbps Ethernet high-speed communication interface controllers are built in, and meanwhile I2C, SPI, UART low-speed Serial communication interface controllers are built in.

Further, the laser radar 5 model is VLP-16, PPS + $ GPRMC time service mode is internally supported, and the scanning frequency is 20Hz; the model number of the GNSS receiver 7 is NovAtel OEM-718D, and the time precision of an internal PPS second pulse signal is 20ns RMS; the model 6 of the inertial measurement unit is ADIS16480, and supports an active triggering data acquisition mode; the model 8 of the camera is a Madweiser MVSUA202GC-T, and an active triggering data acquisition mode is supported; the type of the on-board differential crystal oscillator 9 is SIT9121AI, the frequency is 100MHz, and the frequency stability is +/-10 ppm; the model number of the wireless data transmission radio station 10 is Microhard pMDDL2450.

Further, the wireless data radio station 10 uses 100Mbps ethernet to connect to the ARM end 4 of the main control chip 2 as a remote communication tool of the mapping instrument, and simultaneously sends the differential positioning data received by the wireless data radio station to the GNSS receiver 7 through a serial port of the UART protocol, so as to assist the GNSS receiver 7 to realize the differential positioning function.

Further, the GNSS receiver 7 uses UART protocol to transmit the message data, and because the high-precision time reference module 12 is established and the time service of the plurality of sensors is implemented by the FPGA end 3, the implementation of these functions requires the time message information acquired by the GNSS receiver 7, so that the data channel of the GNSS receiver 7 is accessed to the FPGA end 3, and the UART protocol is driven by the FPGA end 3 using custom IP, so as to receive the time and positioning message data acquired by the GNSS receiver 7 in real time.

Further, the laser radar 5 uses 100Mbps ethernet to transmit data, but at the same time, PPS signals and UTC time message data of UART protocol are needed to be used for time service, so that the data channel of the laser radar 5 is connected to the 100Mbps ethernet interface of the ARM end 4, the time service channel is connected to the FPGA end 3, and the FPGA end 3 sends PPS signals and UTC time message data for the same.

Further, the inertial measurement unit 6 uses the SPI protocol to transmit data, and meanwhile, the inertial measurement unit has an external signal triggering and collecting function, so that a pulse triggering signal with a fixed frequency is sent to the inertial measurement unit 6 at the FPGA end 3 to collect data, and the SPI protocol is driven by the custom IP to receive the data collected by the inertial measurement unit 6.

Further, the camera 8 uses the USB3.0 protocol to transmit image data, and at the same time, it also has an external signal triggering and collecting function, but since the USB3.0 driver is easier to be implemented at the ARM end 4, the triggering signal is given by the FPGA end 3, and the image data receiving is completed by the USB3.0 driver at the ARM end 4.

Further, the FPGA end 3 and the ARM end 4 perform data interaction through a bus of an AXI protocol built in the main control chip 2. Finally, all sensor data acquired by the FPGA end 3 are sent to the ARM end 4 by an AXI bus, and are integrated, stored and downloaded by a data receiving and integrating module 17 of the ARM end 4.

Further, the high-precision time reference module 12 is used for providing a high-precision time reference for time service of various sensors of the device. The module is realized by utilizing the self-defined IP core of the FPGA end 3. The module inputs PPS second pulse signals generated by the GNSS receiver 7 and $GPRMC message data and high-frequency pulse signals generated by the on-board differential crystal oscillator 9, wherein the $GPRMC message data can extract current UTC time information, and the rising edge of PPS second pulse indicates one whole second moment in UTC time. When the GNSS receiver 7 successfully searches 4 satellites and more, the PPS second pulse signal is continuously corrected by the GNSS satellite time, so that the GNSS receiver has very high long-term stability, and the on-board differential crystal oscillator 9 can output a high-frequency pulse signal of 100MHz, so that the on-board differential crystal oscillator has very high time accuracy in a short time. Further, the following more detailed description is given to the step S1 in the process of the multi-sensor data synchronous acquisition method based on the multi-sensor data synchronous acquisition device: the high-precision time reference module 12 calculates the data information according to the input data information to obtain the real-time reference of the operation of the device. Firstly, the module is initialized, which comprises two steps, namely, an on-board differential crystal oscillator pulse counter and a second-level time counter, wherein the on-board differential crystal oscillator pulse counter is used for counting pulses input by an on-board differential crystal oscillator 9, the second-level counter is used for counting PPS second pulses, and the second step is used for receiving a $GPRMC message, analyzing time-division second time information of UTC time and converting the time-division second information into total seconds so as to initialize the second-level time timer. Then the module counts the pulse input by the on-board differential crystal oscillator 9 to obtain a time count value with the unit of 10ns, meanwhile, the module continuously monitors the jump edge of the input PPS second pulse signal, when the PPS second pulse generates a rising edge, the second moment of a complete UTC time is indicated to start, at the moment, the on-board differential crystal oscillator pulse counter is cleared, the second-level time counter is accumulated for one second, and the on-board differential crystal oscillator pulse counter is continuously clocked from zero. And finally, integrating and accumulating the on-board differential crystal oscillator pulse counter and the second-level time counter, wherein the count value of the second-level time counter represents the second-level total time of the current UTC time, and the on-board differential crystal oscillator pulse counter obtains the time from the beginning of the current second time to the beginning of the next second time, and the theoretical unit of the count value is 10ns, so that the UTC total time reference with the accuracy of 10ns is obtained theoretically.

Because the PPS pulse signal has no accumulated error under the condition that the GNSS receiver 7 successfully searches for satellites, the module eliminates the accumulated error of the on-board differential crystal oscillator pulse counter by using the PPS pulse signal to continuously calibrate the on-board differential crystal oscillator 9, and in addition, the module operates at the FPGA end 3, so that the operation of the module has hard instantaneity and parallelism, and the on-board differential crystal oscillator pulse counter and the second time counter simultaneously operate in parallel and calculate the total time in real time, and finally, the UTC total time reference with the accuracy of 10ns is obtained. The time reference is used as a real-time reference for the operation of the data synchronous acquisition device and is used for providing time service for other sensors.

Further, the camera timing module 13 is configured to collect image data of the camera 8 and time the image data of the camera 8. The module inputs high-precision time information provided by the high-precision time reference module 12 and a high-frequency pulse signal output by the on-board differential crystal oscillator 9, generates a pulse signal for triggering the camera 8 to collect data by dividing the frequency of the high-frequency pulse signal, and accurately records the moment for triggering the camera 8 to collect data by the high-precision time information provided by the high-precision time reference module 12 to realize image data timing of the camera 8.

Further, the following more detailed description is given to the step S2 in the process of the multi-sensor data synchronous acquisition method based on the multi-sensor data synchronous acquisition device: after the camera timing module 13 is started, the high-frequency pulse signal output by the on-board differential crystal oscillator 9 is subjected to frequency division to obtain a 10Hz pulse signal, the pulse signal is output to the camera 8, the camera 8 starts to collect image data when receiving the rising edge of the 10Hz pulse signal, meanwhile, the module monitors the rising edge of the 10Hz pulse signal in real time, when the rising edge arrives, the high-precision time information provided by the high-precision time reference module 12 is utilized to record the moment of the rising edge, after the recording is finished, the time data corresponding to the moment is sent to the ARM end 4 through an AXI bus, the ARM end 4 starts USB3.0 driving to receive the image data collected by the camera 8 after receiving the time data, and therefore the acquisition and timing of the data of the camera 8 are completed once.

The camera timing module 13 operates at the FPGA end 3, so that the operation of the module has hard real-time performance and parallelism, and the module can output a pulse trigger signal of 10Hz to stably trigger the camera 8 to collect data, and simultaneously, the module can monitor the moment generated by the rising edge of the pulse in real time, and the moment corresponds to the moment when the camera 8 starts to collect image data. Therefore, the module realizes the functions of collecting the image data of the camera 8 and timing the image data of the camera 8 in terms of technical scheme and working principle.

Further, the laser radar timing module 14 is configured to time three-dimensional distance data of the laser radar 5. The module outputs a time service signal which meets the time service requirement of the laser radar 5 to finish time service by inputting $GPRMC message data and PPS second pulse signals provided by the GNSS receiver 7. According to the data manual of the lidar 5, the PPS second pulse signal and the $gprmc packet data are used simultaneously for time service of the lidar 5, and specific requirements are imposed on the transmission timing and level between the PPS signal and the packet data, the lidar 5 receives the PPS signal and the $gprmc packet data with 5V level logic, the $gprmc packet data needs to be transmitted at least 50ms after the rising edge of the PPS signal, and needs to be transmitted within 300ms before the next rising edge of the PPS, the high level time of the PPS signal is kept between 10 μs and 200ms, in addition, the $gprmc packet data is transmitted through UART protocol with 9600bps baud rate, and it can be known from UART protocol that the 5V level represents logic 0 and the 0V level represents logic 1. Because the FPGA end 3 is 1.8V level logic and the time service signal is 5V level logic, a level logic conversion chip with the MAX13035EETE model is used for carrying out level conversion on the time service signal, and meanwhile, the level inversion is carried out on a signal path for sending the $GPRMC message data in the module so as to adapt to the requirement of the logic level of the time service signal.

Further, the following more detailed description is given to the step S3 in the process of the multi-sensor data synchronous acquisition method based on the multi-sensor data synchronous acquisition device: the laser radar time service module 14 is started, and firstly monitors the rising edge of the PPS pulse signal from the GNSS receiver 7, simultaneously receives $GPRMC message data through the UART protocol and caches the data into a data queue, then when the module monitors the rising edge of the PPS signal, according to the time service requirement of the laser radar 5, in order to ensure the time service success, the module starts to perform time delay operation, and after waiting for 200ms of time delay, takes out the latest and complete $GPRMC message from the cache data queue of the $GPRMC message, sets the baud rate of 9600bps, and sends the same to the laser radar 5 by using the UART protocol to finish one time of time service. After the laser radar 5 finishes time service, the data and the time stamp thereof can be obtained from the UDP data packet, the UDP data packet is received by using the 100Mbps Ethernet interface of the ARM end 4 and stored in the local disk 11.

The laser radar time service module 14 starts to operate at the FPGA end 3, so that the operation of the module has hard real-time performance and parallelism, and the module calculates and processes the PPS pulse signal and $GPRMC message data of the input GNSS receiver 7 and outputs a time service signal which meets the time service requirement of the laser radar 5. Therefore, the module realizes the function of three-dimensional distance data timing of the laser radar 5 in terms of technical scheme and working principle.

Further, the GNSS receiver timing module 15 is configured to collect position data of the GNSS receiver 7 and time the data. The module will input the $gprmc message data sent by the GNSS receiver 7 and parse the location information and time information therein. When the GNSS receiver 7 successfully searches for a star, it will include UTC time information in its $gprmc message data, and the update of the UTC time is synchronized with the PPS second pulse signal, i.e. the rising edge of the PPS second pulse signal represents the start of one second of the UTC time, so that the UTC time information can be used as the timestamp of the position information in its corresponding $gprmc message data. Further, the following more detailed description is given to the step S4 in the process of the multi-sensor data synchronous acquisition method based on the multi-sensor data synchronous acquisition device: the GNSS receiver timing module 15 is started, the module receives the $GPRMC message by using the UART protocol, then analyzes the $GPRMC message in the module, extracts the position information data and the corresponding UTC time information data in the $GPRMC message, integrates the two data, and then sends the two data to the ARM end 4 through the AXI bus, and the ARM end 4 stores the data in the local disk 11 after receiving the data, thus completing the acquisition of the data once.

The GNSS receiver timing module 15 operates on the FPGA 3, so that the operation of the module has hard real-time performance and parallelism, and furthermore, the module will not generate omission and delay for receiving the $gprmc message, and can accurately analyze the position and time information data in the $gprmc message. The module thus realizes, from the technical solution and working principle, the functions of acquiring the position data of the GNSS receiver 7 and timing the data.

Furthermore, the inertial measurement unit timing module 16 is configured to collect and time the angular velocity and acceleration data of the inertial measurement unit 6. The module inputs high-precision time information provided by the high-precision time reference module 12 and a high-frequency pulse signal output by the on-board differential crystal oscillator 9, generates a pulse signal for triggering the inertial measurement unit 6 to acquire data by dividing the frequency of the high-frequency pulse signal, and accurately records the moment of triggering the inertial measurement unit 6 to acquire data by the high-precision time information provided by the high-precision time reference module 12 to realize data timing of the inertial measurement unit 6.

Further, the following more detailed description is given in the step S5 in the process of the multi-sensor data synchronous acquisition method based on the multi-sensor data synchronous acquisition device: the inertial measurement unit timing module 16 is started, firstly, the module divides the frequency of the high-frequency pulse signal output by the on-board differential crystal oscillator 9 to obtain a pulse signal of 2000Hz, the pulse signal is output to the inertial measurement unit 6, the inertial measurement unit 6 starts to collect data when receiving the rising edge of the pulse signal, meanwhile, the module monitors the rising edge of the pulse signal of 2000Hz in real time and accurately records the moment generated by the rising edge by utilizing the high-precision time information provided by the high-precision time reference module 12 when the rising edge arrives, the inertial measurement unit 6 sends the pulse signal representing that the data is completely collected to the module after completing the data collection, the module reads the collected data from a register of the inertial measurement unit 6 through an SPI protocol after receiving the pulse signal, and the data corresponds to the rising edge moment of the pulse signal corresponding to the triggering the data one by one, thus completing the data collection and timing of the data of the inertial measurement unit 6.

The inertial measurement unit timing module 16 operates at the FPGA end 3, so that the operation of the module has hard real-time performance and parallelism, and the module can output a pulse trigger signal of 2000Hz to stably trigger the inertial measurement unit 6 to collect data, and simultaneously, the module can monitor the moment generated by the rising edge of the pulse in real time, and the moment corresponds to the moment when the inertial measurement unit 6 starts to collect data. Therefore, the module realizes the functions of collecting the data of the inertial measurement unit 6 and timing the data from the technical scheme and the working principle.

Compared with the existing multi-sensor acquisition device, the invention has the following advantages and effects:

1. the invention constructs a high-precision time reference module by utilizing the FPGA end 3 self-design of the main control chip 2, and the module constructs a time reference by utilizing the PPS second pulse signal of the GNSS receiver 7 and the on-board differential crystal oscillator 9, so as to time the multiple sensors, and the time reference has the characteristics of high precision, real-time performance and no accumulated error.

2. The invention adopts the mode of hardware active triggering and active recording of triggering time by using the FPGA end for data acquisition of the camera 8 and the inertial measurement unit 6, and compared with the conventional software triggering acquisition, the time delay of hardware triggering acquisition is extremely low and the triggering time can be accurately recorded. For the data of the laser radar 5, the invention designs the IP core of the FPGA end 3 by itself to realize the time service signal meeting the time service requirement of the laser radar 5, and actively time service is carried out on the laser radar 5, thereby ensuring the synchronism and the accuracy of the data time of the laser radar 5.

3. The multi-sensor data synchronous acquisition device combines two computing resources of an FPGA end 3 and an ARM end 4 in a main control chip 2, utilizes a driving system with high concurrency and hard timeliness of the running of the FPGA end 3 software and complete functions of the ARM end 4, and designs a high-precision time reference module 12, a camera time service module 13, a laser radar time service module 14, a GNSS receiver time service module 15, an inertial measurement unit time service module 16 and a data receiving and integrating module 17 by self, wherein the modules are in comprehensive action to synchronously control and acquire the multi-sensor data, the accuracy and the instantaneity of the multi-sensor data acquisition are ensured, the ARM end and the FPGA end are combined, and the advantages are complementary, so that the problem of low time accuracy of the acquired data in a conventional pure software acquisition system is fundamentally solved.

4. The multi-sensor data synchronous acquisition device has the expansibility that the time service and data acquisition modules of a plurality of sensors in the device are connected to the FPGA end 3 of the main control chip 2, and one big characteristic of the FPGA is that hardware can be flexibly programmed, and the sensors which use various different protocols for data transmission can be adapted by self-designing an IP core running on the FPGA. For example, UDP, SPI, UART, IIC, MIPI protocol and the like can be realized by FPGA programming, and the flexible design mode enables the acquisition device to be expanded and integrated with other needed sensors.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiments of the invention and together with the description serve to explain the invention and do not constitute a limitation on the invention. In the drawings:

FIG. 1 is a block diagram of a multi-sensor data synchronous acquisition device based on ZYNQ implementation, disclosed by the invention;

FIG. 2 is a schematic diagram of integration of hardware devices in a multi-sensor data synchronous acquisition device based on ZYNQ implementation;

FIG. 3 is a schematic diagram of various signal and data flow directions when the multi-sensor data synchronous acquisition device based on ZYNQ is operated;

FIG. 4 is a workflow diagram of a high-precision time reference module in a multi-sensor data synchronous acquisition device based on ZYNQ implementation, disclosed by the invention;

FIG. 5 is a workflow diagram of a camera timing module in a multi-sensor data synchronous acquisition device based on ZYNQ implementation;

FIG. 6 is a workflow diagram of an inertial measurement unit timing module in a multi-sensor data synchronous acquisition device based on ZYNQ implementation, disclosed by the invention;

FIG. 7 is a schematic diagram of laser radar timing signals in a multi-sensor data synchronous acquisition device based on ZYNQ;

FIG. 8 is a workflow diagram of a laser radar timing module in a multi-sensor data synchronous acquisition device based on ZYNQ;

FIG. 9 is a diagram of the relationship between the camera data sequence number and the corresponding time stamp in a multi-sensor data synchronous acquisition device based on ZYNQ;

fig. 10 is a time interval distribution diagram of each set of data collected by a camera in a multi-sensor data synchronous collection device based on ZYNQ implementation.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

Example 1

The embodiment discloses a synchronous acquisition device and method of multisensor data based on ZYNQ, and FIG. 1 is a block diagram of the device, as shown in FIG. 1, the synchronous acquisition device of multisensor data based on ZYNQ includes:

The hardware equipment carrier plate 1 is composed of a circuit board and is used for integrating various hardware equipment 2 and ensuring the stable operation of an electrical system of the hardware equipment;

a hardware device 2 comprising: the main control chip 2 is connected with the hardware device carrier plate 1 and used for controlling the multi-sensor data synchronous acquisition device to work, is a ZYNQ SOC series of Xilinx company, is of the model XCZU3EG, integrates CPU of ARM Cortex-A53 architecture of dual-core 1.3GHz and FPGA resources of 154K programmable logic units, is internally provided with high-speed communication interface controllers such as USB3.0, serial ATA and 1000Mbps Ethernet, and is also internally provided with low-speed Serial communication interface controllers such as I2C, SPI, UART; the laser radar 5 is of a VLP-16 type, internally supports a PPS + $ GPRMC time service mode, is connected with the hardware equipment carrier plate 1 and is used for acquiring data; the inertial measurement unit 6 is of the model ADIS16480, supports an active triggering data acquisition mode, is connected with the hardware equipment carrier plate 1 and is used for acquiring data; a GNSS receiver 7, which is of the model NovAtel OEM-718D, is connected with the hardware device carrier plate 1, is used for acquiring data, and can be used for providing PPS second pulse signals; a camera 8, the model is a Maidesic MVSUA202GC-T, supports an active triggering data acquisition mode and is a sensor for acquiring data; the on-board differential crystal oscillator 9 is welded on the hardware equipment carrier plate 1, the model is SIT9121AI, the frequency is 100MHz, the frequency stability is +/-10 ppm, and the on-board differential crystal oscillator is used for providing high-precision time information for the multi-sensor data synchronous acquisition device and assisting in synchronous acquisition of the multi-sensor data; the wireless data transmission radio station 10 is connected with the hardware equipment carrier plate 1, has the model of Microhard pMDDL2450 and is used for the remote communication of the multi-sensor data synchronous acquisition device; the local disk 11 is connected with the hardware device carrier plate 1 and is used for storing the multi-sensor data acquired by the multi-sensor data synchronous acquisition device.

Fig. 2 is an integrated schematic diagram of the hardware device of the present apparatus, as shown in fig. 2, wherein the dashed line represents the time synchronization signal, and the solid line represents the data flow. The hardware device 2 will be integrated into the hardware device carrier plate 1 according to the following manner:

the wireless data transmission radio station 10 is connected with the ARM end 4 of the main control chip 2 by using 100Mbps Ethernet, so as to be used for remotely controlling the device and providing differential positioning data for the GNSS receiver 7, thereby realizing the RTK positioning function of the GNSS receiver 7; the GNSS receiver 7 uses UART protocol to transmit message data, a data channel of the GNSS receiver 7 is accessed to the FPGA end 3, the FPGA end 3 uses self-defined IP to realize the drive of the UART protocol, and the time and the positioning message data acquired by the GNSS receiver 7 are received in real time; the laser radar 5 uses 100Mbps Ethernet UDP protocol to transmit data, and simultaneously, PPS signals and UTC time message data of UART protocol are required to be used for time service, a data channel of the laser radar 5 is accessed to the ARM end 4 through the 100Mbps Ethernet, and a time service channel is accessed to the FPGA end 3; the inertial measurement unit 6 uses SPI protocol to transmit data and has an external pulse signal triggering and collecting function, and the FPGA end 3 self-defines IP to realize SPI protocol driving so as to receive the data of the inertial measurement unit 6; the camera 8 uses the USB3.0 protocol to transmit data, and simultaneously has an external pulse signal triggering and collecting function, wherein a triggering signal is given by the FPGA end 3, and data receiving is completed by the USB3.0 drive of the ARM end 4; the data interaction between the FPGA end 3 and the ARM end 4 on the main control chip 2 is realized through an AXI bus built in the main control chip 2, and after sensor data are acquired by various sensor synchronous control modules realized at the FPGA end 3, the sensor data are transmitted to the ARM end 4 through the AXI bus for storage.

Fig. 3 is a schematic diagram of various signal and data flow directions when the device operates, and the following describes the process of the acquisition method of the multi-sensor data synchronous acquisition device based on the diagram:

firstly, after the device is powered on, the GNSS receiver 7 starts to start and performs a satellite searching process, and after the satellite signals are successfully acquired, the GNSS receiver 7 can output accurate positioning message data and PPS second pulse signals without integral errors, and one rising edge of the PPS second pulse signals represents one whole second moment in UTC time. Next, the high precision time reference module 12 starts to operate, and fig. 4 is a flowchart of a high precision time reference module, and the module performs two-step initialization, wherein the first step is to create an on-board differential crystal oscillator pulse counter and a PPS second pulse counter, the two counters count the pulses of the on-board differential crystal oscillator 9 and the PPS second pulse respectively, and the second step is to receive the $gprmc message of the GNSS receiver 7 and analyze UTC time information therein, and convert the UTC time information into total seconds according to time-division second information of UTC time to initialize the value of the PPS second pulse counter. Then the module counts the pulse input by the on-board differential crystal oscillator 9, monitors the rising edge of the PPS second pulse signal, adds 1 to the count value of the PPS second pulse signal when the rising edge of the PPS second pulse signal arrives, and simultaneously clears the on-board differential crystal oscillator pulse counter to indicate that the current whole second moment arrives. The value of the PPS second pulse counter represents the second-level total time of the current UTC time, the value of the on-board differential crystal oscillator pulse counter represents the time passing between the beginning of the current second and the beginning of the next second, and finally the value of the on-board differential crystal oscillator pulse counter and the value of the PPS second pulse counter are accumulated after unit conversion, so that the current UTC total time reference is obtained.

And then after the high-precision time reference module is built, the various sensor time service modules start to work simultaneously, and the specific working process of the sensor time service modules is as follows.

The specific working process of the camera timing module 13 is as follows: as shown in fig. 5, which is a working flow chart of the camera timing module, the camera timing module 13 inputs the pulse signal generated by the on-board differential crystal oscillator 9, divides the frequency to obtain a pulse signal of 10Hz, outputs the pulse signal to the camera 8 to trigger the camera 8 to collect image data at the rising edge of the pulse signal, and meanwhile, the module accurately records the rising edge time of the pulse signal, which is called time data, by utilizing the time information of the high-precision time reference module 12. After obtaining the time data, the time data is sent to a memory address space appointed by the ARM end 4 through an AXI bus, the ARM end 4 starts a USB3.0 drive to receive the image data acquired by the camera 8 after receiving the time data at the rising edge moment, integrates the image data and the time data and then stores the image data and the time data in the local disk 11, and the acquisition process is completed once.

The specific working process of the inertial measurement unit timing module 16 is as follows: fig. 6 is a flowchart of the inertial measurement unit timing module. The inertial measurement unit time service module 16 firstly configures the inertial measurement unit 6 to work in an external pulse triggering acquisition mode, then divides the frequency of the pulse signal generated by the input on-board differential crystal oscillator 9 to obtain a 2000Hz pulse signal and outputs the 2000Hz pulse signal to the inertial measurement unit 6, and meanwhile, the module accurately records the time of the rising edge of the pulse signal by utilizing the time information of the high-precision time reference module 12. The inertial measurement unit 6 starts to collect data when sensing the rising edge of the pulse signal, after waiting for the data to be collected, the inertial measurement unit 6 returns a pulse signal to the module, the module receives the data collected by the inertial measurement unit 6 through an SPI protocol after sensing the pulse signal, and sends the pulse rising edge moment corresponding to the data to the ARM end 4 through an AXI bus together, and the ARM end 4 stores the received data to finish a collecting process.

The specific working process of the laser radar time service module 14 is as follows: as shown in fig. 7, the time service signal requirement of the lidar 5 indicates that the lidar 5 needs PPS second pulse signals and $gprmc packet data for time service, and the time sequence between the time service signals has clear requirements, and fig. 8 is a workflow diagram of the lidar time service module. The module inputs PPS second pulse signals and $GPRMC messages of the GNSS receiver 7, firstly, the module caches the $GPRMC message data in the module by utilizing a data queue, meanwhile, the module monitors the PPS second pulse signals, starts delaying when rising edges of the second pulse signals come, takes out the $GPRMC message data of the latest frame from the data queue after waiting for 200ms of delaying, sets the baud rate of 9600bps, and sends the data to the laser radar 5 through a UART protocol to finish time service of the laser radar 5.

The specific working process of the GNSS receiver timing module 15 is as follows: the module receives the $GPRMC data message sent by the GNSS receiver 7 in real time through the UART protocol, analyzes the position information and the time information in the message, integrates the position information and the time information, and then sends the integrated position information and the time information to the ARM end for storage, so that the time service process is completed once.

Finally, after each sensor timing module performs timing on the sensor data, the data receiving and integrating module 17 located at the ARM end 4 starts to operate, and the module adopts multithread programming, and meanwhile, the sensor data after timing by the camera timing module 13, the laser radar timing module 14, the GNSS receiver timing module 15 and the inertial measurement unit timing module 16 are received and integrated, and the data are stored in the local disk 11.

In summary, the invention uses the ZYNQ series SOC of Xilinx company as the main control chip, the chip integrates two computing resources of FPGA and ARM, the multi-sensor time synchronization control software is integrated at the FPGA end of ZYNQ, the multi-sensor is time synchronization controlled by utilizing the concurrency processing capability and hard real-time property of the FPGA, and simultaneously, the flexible integration, processing and downloading of data are realized by utilizing the rich hardware resources integrated at the ARM end and a Linux system with complete functions.

Example 2

The embodiment discloses an actual experiment and result analysis of data acquisition and time service of a camera 8 through a multi-sensor data synchronous acquisition device based on ZYNQ, and the actual technical effect of the invention is supplemented and illustrated by the embodiment.

According to embodiment 1, after the high-precision time reference module 12 is waited for, the camera timing module 13 in the device starts to operate, triggers the camera 8 at the theoretical 10Hz pulse frequency and timing the camera data, and fig. 9 is a relationship diagram of the data serial number of the camera 8 and the corresponding timestamp obtained in one actual collection process, wherein the data serial number is obtained by sequencing according to the actual time sequence of the image data collected by the device reaching the ARM end 4, and the timestamp represents the time data corresponding to the image data. As can be seen from fig. 9, as the camera 8 data is continuously collected, the time stamp corresponding to the data is continuously continued, and the linear change is presented, which indicates that the camera timing module 13 stably works and successfully timing the camera data.

Further, in order to verify the validity of the camera timing module 13 for timing camera data, the image data of two adjacent serial numbers is a set of data, the time difference between two frames of data in a set of data is referred to as the time interval of the set of data, then fig. 10 is a time interval distribution diagram of each set of data collected by the camera, and it can also be seen that the time interval of each set of data is stabilized between 100000 μs and 100002 μs, there is no drift trend, which indicates that the on-board differential crystal oscillator 9 with the frequency stability of ±10ppm can work normally and that the camera timing module 13 is valid for timing camera data.

The above examples are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above examples, and any other changes, modifications, substitutions, combinations, and simplifications that do not depart from the spirit and principle of the present invention should be made in the equivalent manner, and the embodiments are included in the protection scope of the present invention.

Claims

1. The ZYNQ-based multi-sensor data synchronous acquisition device is characterized by comprising a hardware device carrier plate (1) and a plurality of hardware devices, wherein the hardware device carrier plate (1) is formed by a circuit board and is used for integrating the plurality of hardware devices;

The hardware devices are connected with the hardware device carrier plate (1), and the hardware device carrier plate comprises:

the main control chip (2) is used for controlling the multi-sensor data synchronous acquisition device to work, and the main control chip (2) is divided into an FPGA end (3) and an ARM end (4);

a laser radar (5), a sensor for acquiring three-dimensional distance data;

the inertial measurement unit (6) is used for acquiring the sensor of the angular velocity and acceleration data in the motion process of the data synchronous acquisition device;

a GNSS receiver (7) for acquiring a sensor of real-time position data of the data synchronous acquisition device and for providing PPS second pulse signals;

a camera (8), a sensor for acquiring image data;

the on-board differential crystal oscillator (9) is used for providing high-precision time information for the data synchronous acquisition device;

the wireless data transmission radio station (10) is used for realizing remote communication of the data synchronous acquisition device;

the local disk (11) is used for storing the multi-sensor data acquired by the data synchronous acquisition device;

a high-precision time reference module (12) for providing high-precision time for the data synchronous acquisition device;

the camera time service module (13) is used for realizing data acquisition and time service of the camera;

the laser radar time service module (14) is used for realizing data acquisition and time service of the laser radar (5);

The GNSS receiver time service module (15) is used for realizing data acquisition and time service of the GNSS receiver (7);

the inertial measurement unit timing module (16) is used for realizing data acquisition and timing of the inertial measurement unit (6);

the data receiving and integrating module (17) is used for receiving and integrating the multi-sensor data after synchronization is completed;

s1, starting a high-precision time reference module (12), receiving PPS second pulse signals and $GPRMC message data sent by a GNSS receiver (7), simultaneously receiving pulse signals sent by an on-board differential crystal oscillator (9), internally analyzing the $GPRMC message data by the high-precision time reference module (12) to extract UTC time information, simultaneously carrying out pulse counting on the PPS second pulse signals and the pulse signals sent by the on-board differential crystal oscillator (9), and integrating and calculating the pulse count value and the UTC time information to obtain real-time high-precision time for operation of a data synchronous acquisition device;

s2, a camera timing module (13) is started, real-time high-precision time obtained by calculation of a pulse signal sent by an on-board differential crystal oscillator (9) and a high-precision time reference module (12) is received, the pulse signal sent by the on-board differential crystal oscillator (9) is output to a camera (8) after being divided in frequency so as to trigger the camera (8) to acquire image data, meanwhile, the moment of triggering the camera (8) to acquire the data is obtained by internal calculation of the camera timing module (13) and is used as time data corresponding to the image data, and the time data corresponding to the image data is stored in an ARM end (4) of a main control chip (2);

S3, starting a laser radar timing module (14), receiving $GPRMC message data and PPS second pulse signals sent by a GNSS receiver (7), simultaneously forwarding the PPS second pulse signals to a laser radar (5), monitoring the PPS second pulse signals in the laser radar timing module (14) and simultaneously buffering the $GPRMC message data, and sending the $GPRMC message data to the laser radar (5) at 9600bps baud rate after the laser radar timing module (14) monitors the rising edge of the PPS second pulse and delays for 200 ms;

s4, starting a GNSS receiver time service module (15), receiving $GPRMC message data sent by a GNSS receiver (7), analyzing UTC time information and positioning information in the message, packaging the UTC time information and the positioning information, and storing the packaged UTC time information and positioning information in an ARM end (4) of a main control chip (2);

s5, starting an inertial measurement unit time service module (16), receiving a real-time high-precision time obtained by calculating a pulse signal sent by an on-board differential crystal oscillator (9) and a high-precision time reference module (12), dividing the frequency of the pulse signal sent by the on-board differential crystal oscillator (9), outputting the frequency to an inertial measurement unit (6) to trigger the inertial measurement unit (6) to acquire data, reading the data through an SPI protocol after the inertial measurement unit (6) finishes data acquisition, simultaneously internally calculating the time for triggering the inertial measurement unit (6) to acquire the data as a time stamp corresponding to the inertial measurement unit data, and storing the time stamp and the corresponding inertial measurement unit data at an ARM end (4) of a main control chip (2);

S6, starting a data receiving and integrating module (17), receiving sensor data which are subjected to time service by a camera time service module (13), a laser radar time service module (14), a GNSS receiver time service module (15) and an inertial measurement unit time service module (16), compressing the data, and storing the compressed data in a local disk (11).

2. The ZYNQ-based multi-sensor data synchronous acquisition device according to claim 1, wherein the master control chip (2) is a ZYNQ SOC series, and is of a type XCZU3EG; the laser radar (5) model is VLP-16; the model of the GNSS receiver (7) is OEM-718D; the model of the inertial measurement unit (6) is ADIS16480; the model of the camera (8) is MVSUA202GC-T; the type of the on-board differential crystal oscillator (9) is SIT9121AI; the model of the wireless data transmission station (10) is pMDDL2450.

3. The ZYNQ-based multi-sensor data synchronous acquisition device of claim 1, wherein the inertial measurement unit (6) is in data communication with the FPGA end (3) of the main control chip (2) through an SPI protocol, the camera (8) is in data communication with the ARM end (4) of the main control chip (2) through a USB3.0 protocol, the lidar (5) is in data communication with the ARM end (4) of the main control chip (2) through a 100Mbps ethernet protocol, the GNSS receiver (7) is in data communication with the FPGA end (3) of the main control chip (2) through a UART protocol, and the wireless data transfer station (10) is in data communication with the ARM end (4) of the main control chip (2) through a 100Mbps ethernet protocol.

4. The ZYNQ-based multi-sensor data synchronous acquisition device according to claim 1, wherein the high-precision time reference module (12) uses the FPGA end (3) of the main control chip (2) to perform self-defining IP verification, the high-precision time reference module (12) takes PPS second pulse signals of the GNSS receiver (7) and pulse signals of the on-board differential crystal oscillator (9) as input, a PPS second pulse counter and an on-board differential crystal oscillator pulse counter are built in the high-precision time reference module, the rising edges of the PPS second pulse and the pulse signals of the on-board differential crystal oscillator (9) are counted in an accumulated manner, and a time reference with the precision of 10ns level is output after internal processing and is used as a real-time reference for operation of the data synchronous acquisition device and a time reference for providing time service for other sensors.

5. The ZYNQ-based multi-sensor data synchronous acquisition device according to claim 1, wherein the camera timing module (13) is realized by using an FPGA end (3) of the main control chip (2) to customize an IP core, the camera timing module (13) takes a pulse signal of the on-board differential crystal oscillator (9) and time information provided by the high-precision time reference module (12) as inputs, actively triggers the camera (8) to acquire data by outputting a 20Hz pulse signal, accurately records the time of triggering the acquired data and transmits the time of triggering the acquired data to an ARM end (4) of the main control chip (2) through an AXI bus for storage, the time of triggering the acquired data is expressed by UTC time in μs, and image data acquired by the camera (8) is directly transmitted to the ARM end (4) of the main control chip (2) through a USB3.0 protocol for storage.

6. The ZYNQ-based multi-sensor data synchronous acquisition device according to claim 1, wherein the inertial measurement unit timing module (16) is implemented by using a custom IP core at the FPGA end (3) of the main control chip (2), the inertial measurement unit timing module (16) takes a pulse signal of the on-board differential crystal oscillator (9) and time information provided by the high-precision time reference module (12) as inputs, actively triggers the pulse signal of the inertial measurement unit (6) to output 2000Hz, receives data output by the inertial measurement unit (6) through an SPI protocol, and accurately records a time of data acquisition, the time is expressed by UTC time in μs, and the data and the acquisition time are transmitted to the ARM end (4) of the main control chip (2) through an AXI bus for storage.

7. The ZYNQ-based multi-sensor data synchronous acquisition device according to claim 1, wherein the laser radar timing module (14) is implemented by using an FPGA end (3) custom IP core of the main control chip (2), the laser radar timing module (14) takes $gprmc message data and PPS second pulse signals acquired by the GNSS receiver (7) as input, generates PPS second pulse signals of RS232 level and $gprmc messages of UART protocol RS232 level according to timing requirements of the laser radar (5) and outputs the signals to the laser radar (5), wherein the $gprmc message data of UART protocol RS232 level is sent 200ms after a rising edge of the PPS signals, and after successful timing of the laser radar (5), the laser radar data is received and stored at an ARM end (4) of the main control chip (2) through 100Mbps ethernet interface UDP protocol.

8. The ZYNQ-based multi-sensor data synchronous acquisition device of claim 1, wherein the GNSS receiver timing module (15) is implemented by using a custom IP core at an FPGA end (3) of the main control chip (2), the GNSS receiver timing module (15) receives $gprmc message data acquired by the GNSS receiver (7) through a UART protocol TTL level, analyzes UTC time and positioning data in the message, correlates the positioning data with UTC time, and sends the positioning data to an ARM end (4) of the main control chip (2) through an AXI bus for storage.

9. The ZYNQ-based multi-sensor data synchronous acquisition device according to claim 1, wherein the data receiving and integrating module (17) is implemented by using ARM (4) programming of the main control chip (2), the data receiving and integrating module (17) adopts a multi-thread operation mode, and simultaneously receives data from a plurality of sensors after time service, integrates the data and stores the integrated data in the local disk (11).