CN111580596B - Method, device and terminal for synchronizing multiple IMUs (inertial measurement units) in time - Google Patents
Method, device and terminal for synchronizing multiple IMUs (inertial measurement units) in time Download PDFInfo
- Publication number
- CN111580596B CN111580596B CN202010422627.1A CN202010422627A CN111580596B CN 111580596 B CN111580596 B CN 111580596B CN 202010422627 A CN202010422627 A CN 202010422627A CN 111580596 B CN111580596 B CN 111580596B
- Authority
- CN
- China
- Prior art keywords
- imu
- time
- imus
- translation
- precision
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/04—Generating or distributing clock signals or signals derived directly therefrom
- G06F1/12—Synchronisation of different clock signals provided by a plurality of clock generators
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Navigation (AREA)
Abstract
The embodiment of the invention discloses a method, a device and a terminal for synchronizing a plurality of IMUs (inertial measurement units), wherein the method comprises the following steps: performing external parameter calibration on a plurality of IMUs based on a camera to obtain external parameters among the IMUs, converting angle data obtained by the IMUs to the same coordinate system, and calculating angle change values of the IMUs at a preselected attitude angle; the method comprises the steps of taking a timestamp of an IMU with the largest sampling frequency as a reference, roughly synchronizing the timestamp according to an initial time difference between respective starting times of the current IMU and the IMU used as the reference, and then performing timestamp translation on the current IMU for a first preset number of times according to a first preset value range corresponding to the initial time difference so as to calculate and obtain an optimal time difference corresponding to first precision. Compared with the prior art, the technical scheme of the invention can solve the problem of hardware cost, greatly improves time synchronization and the like.
Description
Technical Field
The invention relates to the technical field of navigation multi-sensor fusion, in particular to a method, a device and a terminal for synchronizing a plurality of IMUs (inertial measurement units).
Background
With the continuous reduction of the cost of inertial measurement units (hereinafter abbreviated as IMUs) and the advent of integrated inertial sensors, more and more platforms such as robots are equipped with multiple IMUs. Typically, these platforms have in common that the primary IMU is used for positioning and alignment (i.e. mounted close to the center of gravity and aligned with the principal axis of the platform) and the secondary IMU is mounted in a less obstructed location. For most platforms, these two locations are very different, and in order to fuse data of multiple IMUs, it is necessary to perform a time stamp alignment operation on them.
However, there are two main types of time stamp alignment or synchronization for the currently common multiple IMUs:
firstly, a hardware synchronization method is adopted, for example, in the existing patent application with patent publication number CN110217178A, "a driverless sensing system based on hardware synchronization and its working method", that is, a high-precision hardware is used to solve the data synchronization problem of multiple IMUs, and certainly, there are "a distributed POS sub-IMU data synchronization acquisition system" with patent publication number CN102736591A, and so on. Although this type of method is highly accurate, it is also costly and not well suited for most users or businesses.
And secondly, a software synchronization method is adopted, data of two groups of IMUs are aligned in a thread locking mode mainly according to different timestamp information of the two IMUs, and the method is convenient, but cannot meet the requirement of high precision and is not suitable for occasions requiring high precision. In addition, there are other time synchronization methods combining software and hardware, for example, when the problem of time synchronization is solved in a "multi-sensor acquisition timestamp synchronization apparatus" with patent publication No. CN109729277A, timestamp synchronization is mainly realized through serial port transmission, and each sensor can acquire timestamp information, which is not applicable to a scene where a camera itself does not have a timestamp, and has a great limitation; for another example, the method for time synchronization of an underwater sensor network disclosed in patent publication CN109951248A mainly uses the clock time of the sending/receiving of the data packet exchanged between the sensor node to be synchronized and the neighboring node, fits the linear regression equation between the clock time of each neighboring node and the clock time of the own node, and continuously updates the clock time parameters of the own node in an iterative manner, thereby realizing time synchronization between the sensor nodes. Since this method requires communication between the sensors, it is not suitable for IMUs that do not have mutual communication capability and operate independently of each other, i.e. it also has some limitations. Therefore, in consideration of the problem of hardware cost, the invention provides a time synchronization method which adopts software synchronization and can achieve higher precision.
Disclosure of Invention
In view of this, the embodiments of the present invention provide a method, an apparatus, and a terminal for time synchronization of multiple IMUs.
An embodiment of the present invention provides a method for time synchronization of multiple IMUs, including:
performing external parameter calibration on a plurality of IMUs based on a camera to obtain external parameters among the IMUs, converting angle data obtained by the IMUs to the same coordinate system by using the external parameters, and calculating angle change values of the IMUs at a preselected attitude angle;
roughly synchronizing time stamps according to initial time differences between respective starting times of a current IMU and the IMU serving as the reference by taking the time stamp of the IMU with the largest sampling frequency as the reference, and then performing time stamp translation on the current IMU for a first preset number of times according to a first preset value range corresponding to the initial time differences so as to calculate and obtain an optimal time difference corresponding to first precision; after each translation, performing interpolation and data interception according to the angle change value of the current IMU to calculate an angle data accumulated difference value between the current IMU and the IMU serving as a reference, and then obtaining a translation time value corresponding to a minimum accumulated difference value from a plurality of accumulated difference values to serve as an optimal time difference corresponding to the first precision;
and performing time stamp translation on the current IMU for a second preset number of times according to a second preset value range corresponding to the optimal time difference of the first precision to obtain an optimal time difference corresponding to the second precision, and so on until the optimal time difference meeting the preset precision is obtained to be used as final synchronization time between the current IMU and the IMU serving as the reference.
Further, in the above method for time synchronization of multiple IMUs, the "performing external reference calibration on multiple IMUs based on a camera to obtain external parameters between the IMUs, converting angle data obtained by the IMUs into the same coordinate system using the external parameters, and then calculating an angle change value of each IMU at a preselected attitude angle" includes:
performing external reference calibration on a plurality of IMUs by using a calibration plate and based on a camera to obtain external parameters between each IMU and the camera;
calculating external parameters between each IMU according to the external parameters between each IMU and the camera;
selecting a coordinate system of one IMU from the IMUs as a reference coordinate system, and then converting angle data acquired by other IMUs into the reference coordinate system according to the external parameters between the other IMUs and the IMU serving as the reference coordinate system;
and then calculating the angle change value of each IMU at a preselected attitude angle in the reference coordinate system.
Further, in the above-mentioned time synchronization method for multiple IMUs, the IMUs include a first IMU and a second IMU, and if an external parameter between the first IMU and the camera is recorded as
An external parameter between the second IMU and the camera as
An external parameter between the first IMU and the second IMU
The following formula is satisfied:
further, in the above method for time synchronization of multiple IMUs, the "roughly synchronizing timestamps according to an initial time difference between respective starting times of a current IMU and an IMU serving as a reference, and then performing a first preset number of time stamp translations on the current IMU according to a first preset value range corresponding to the initial time difference" with a timestamp of an IMU having a maximum sampling frequency as a reference includes:
selecting an IMU with the largest sampling frequency from the plurality of IMUs as a time reference for angle data interpolation;
integrally translating the time stamp of the current IMU according to the initial time difference between the starting time of the current IMU and the starting time of the IMU serving as a reference so as to perform rough synchronization;
sequentially taking values from the first preset value range corresponding to the first precision and accumulating the initial time difference to obtain a corresponding translation time value; the first preset times are equal to the times of the current value;
and respectively carrying out integral translation on the time stamp of the current IMU according to the translation time value obtained each time.
Further, in the above multiple IMU time synchronization method, the "calculating obtains an optimal time difference corresponding to the first precision; performing interpolation and data interception according to the angle change value of the current IMU after each translation to calculate an angle data cumulative difference between the current IMU and an IMU serving as a reference, and then obtaining a translation time value corresponding to a minimum cumulative difference from a plurality of cumulative differences to serve as an optimal time difference corresponding to the first precision "includes:
for each of the other IMUs, performing angle or angular velocity interpolation at each sampling time of the IMU serving as a reference according to the angle change value corresponding to the current IMU to obtain an angle data set of the current IMU, which contains the interpolation;
carrying out equal-time equal-quantity interception on the angle data set of the IMU serving as the reference and the angle data set of the IMU, calculating and accumulating the difference value of the angle data between the current IMU and the IMU serving as the reference at each sampling moment to obtain the accumulated difference value corresponding to the current translation;
then repeating the steps to obtain all accumulated difference values corresponding to the first preset times of translation;
and selecting a minimum value from all the accumulated difference values, and taking the minimum value as the optimal time difference corresponding to the first precision.
Further, in the above method for time synchronization of multiple IMUs, the second precision is smaller than the first precision, and the "then performing timestamp translation on the current IMU for a second preset number of times according to a second preset value range corresponding to the optimal time difference of the first precision to obtain the optimal time difference corresponding to the second precision" includes:
sequentially taking values from the second preset value range corresponding to the second precision and accumulating the optimal time difference of the first precision to obtain a corresponding translation time value; the second preset times are equal to the times of the current value;
and respectively carrying out integral translation on the time stamp of the current IMU according to the translation time value obtained each time, then calculating the corresponding accumulated difference value after each translation, and selecting the translation time value corresponding to the minimum accumulated difference value from the accumulated difference value to be used as the optimal time difference corresponding to the second precision.
Further, in the above-mentioned multi-IMU time synchronization method, the pre-selected attitude angle is a rotation angle around a Z-axis in an upward direction when the apparatus provided with the plurality of IMUs moves on the ground.
Another embodiment of the present invention provides a multiple IMU time synchronization apparatus, including:
the external parameter calibration module is used for carrying out external parameter calibration on the IMUs based on the camera so as to obtain external parameters among the IMUs, converting angle data obtained by the IMUs to the same coordinate system by using the external parameters, and then calculating angle change values of the IMUs at a preselected attitude angle;
the initial synchronization calculation module is used for roughly synchronizing the time stamps according to initial time differences between respective starting moments of a current IMU and the IMU serving as a reference by taking the time stamp of the IMU with the largest sampling frequency as the reference, and then performing time stamp translation on the current IMU for a first preset number of times according to a first preset value range corresponding to the initial time differences so as to calculate and obtain an optimal time difference corresponding to a first precision; after each translation, performing interpolation and data interception according to the angle change value of the current IMU to calculate an angle data accumulated difference value between the current IMU and the IMU serving as a reference, and then obtaining a translation time value corresponding to a minimum accumulated difference value from a plurality of accumulated difference values to serve as an optimal time difference corresponding to the first precision;
and the accurate synchronization calculation module is used for performing timestamp translation on the current IMU for a second preset number of times according to a second preset value range corresponding to the optimal time difference of the first precision to obtain the optimal time difference corresponding to the second precision, and so on until the optimal time difference corresponding to the preset precision is obtained to serve as the final synchronization time between the current IMU and the IMU serving as the reference.
Another embodiment of the present invention provides a terminal, including: a processor and a memory, the memory storing a computer program for execution by the processor to implement the plurality of IMU time synchronization methods described above.
Yet another embodiment of the present invention provides a computer-readable storage medium storing a computer program which, when executed, implements a plurality of IMU time synchronization methods according to the above.
The method provided by the embodiment of the invention adopts a pure software synchronization method, obtains external parameters among IMUs, and then converts all the IMUs to the same coordinate system according to the external parameter information to carry out angle change analysis of a certain attitude angle; calculating the accumulated difference of angle data among IMUs through a timestamp translation algorithm and based on an interpolation algorithm, and selecting the translation time which enables the difference to be minimum as the optimal time difference of corresponding precision; and then gradually reducing the time difference range according to different precisions to obtain the final synchronization time which can meet the requirement of the needed precision. Compared with the prior art, the method provided by the embodiment of the invention can realize the sensor synchronization with required precision through a computer software technology, does not need to increase other hardware, and can reduce the hardware cost compared with a hardware synchronization mode.
The problem of hardware cost can be solved, and the time synchronization is greatly improved.
Drawings
In order to more clearly illustrate the technical solution of the present invention, the drawings required to be used in the embodiments will be briefly described below, and it should be understood that the following drawings only illustrate some embodiments of the present invention, and therefore should not be considered as limiting the scope of the present invention. Like components are numbered similarly in the various figures.
FIG. 1 illustrates a first flow diagram of a multiple IMU time synchronization method according to an embodiment of the present invention;
FIG. 2 illustrates a second flow diagram of a multiple IMU time synchronization method according to an embodiment of the present invention;
FIG. 3 illustrates a third flow diagram of a multiple IMU time synchronization method according to an embodiment of the present invention;
FIG. 4 illustrates a fourth flowchart of a multiple IMU time synchronization method according to an embodiment of the present invention;
FIG. 5 is a fifth flowchart illustrating a method for multiple IMU time synchronization according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a plurality of IMU time synchronization apparatuses according to an embodiment of the present invention.
Description of the main element symbols:
10-a plurality of IMU time synchronizers; 110-external reference calibration module; 120-initial synchronization calculation module; 130-precise synchronization calculation module.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments.
The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
Hereinafter, the terms "including", "having", and their derivatives, which may be used in various embodiments of the present invention, are only intended to indicate specific features, numbers, steps, operations, elements, components, or combinations of the foregoing, and should not be construed as first excluding the existence of, or adding to, one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing.
Furthermore, the terms "first," "second," "third," and the like are used solely to distinguish one from another and are not to be construed as indicating or implying relative importance.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which various embodiments of the present invention belong. The terms (such as those defined in commonly used dictionaries) should be interpreted as having a meaning that is consistent with their contextual meaning in the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein in various embodiments of the present invention.
Example 1
Referring to fig. 1, the present embodiment provides a method for synchronizing multiple IMU times, which can be applied to occasions requiring multiple sensors to perform information fusion, such as instant positioning and map building (SLAM), autonomous navigation positioning, and the like. The method of the embodiment adopts a software synchronization mode, can greatly improve the precision compared with the existing software synchronization method, and can well solve the problems of high hardware cost and the like caused by hardware synchronization because no hardware is required to be added.
The multiple IMU time synchronization method is explained in detail below, as shown in fig. 1.
Step S100, performing external parameter calibration on a plurality of IMUs based on a camera to obtain external parameters among the IMUs, converting angle data sampled by the IMUs to the same coordinate system by using the external parameters, and calculating angle change values of the IMUs at a preselected attitude angle.
Exemplarily, the step S100 mainly includes the following two parts: firstly, external parameters among the IMUs are obtained to facilitate subsequent conversion of the same coordinate system; and secondly, analyzing the angle change information of the attitude angle with the largest change selected in advance from the three-axis attitude angles, namely the angular velocity information of the attitude angle, according to the angle data acquired by each IMU under the same coordinate system. It can be understood that the angle data acquired by the IMU in this embodiment may be an angle value or an angular velocity value.
As shown in fig. 2, the first part of the above step S100 mainly includes the following sub-steps:
and S101, performing external reference calibration on the IMUs by using a calibration plate and based on the camera to acquire external parameters between each IMU and the camera.
It can be understood that the extrinsic parameters are bridges between the camera and the IMU, and need to be used when processing such as solving camera pose or reprojection errors. In the embodiment, each IMU has its own coordinate system and is measured independently from each other, so that in the process of fusing data of different IMUs, the transformation of the data in different coordinate systems requires the use of external parameters of the coordinate systems between the IMUs. Wherein the extrinsic parameters typically include a rotation matrix and a translation matrix.
Illustratively, a calibration board or other tool may be used to perform external reference calibration on a rigid body on which a plurality of IMUs and cameras are mounted, so as to obtain external parameters before each IMU and camera in the rigid body. For example, during calibration, the rigid body may be oriented to a calibration board for data acquisition, while each IMU has sufficient excitation in the x, y, and z directions to make the corresponding pitch, roll, and yaw (i.e., three-axis attitude) angles change significantly, and then the external parameters between each IMU and the camera may be calculated by means of kalibr or the like from the image data recorded by the camera.
And step S102, calculating external parameters between the IMUs according to the external parameters between each IMU and the camera.
After the external parameters between each IMU and the camera are obtained, the external parameters between the IMUs are further calculated. Where the context between each IMU relative to the same camera is known, the rotation and translation may be used to solve for the context from one IMU to another, i.e., to solve for the corresponding rotation and translation matrices.
Exemplarily, in one embodiment, the plurality of IMUs includes a first IMU and a second IMU, assuming that an external parameter between the first IMU and the camera is
And the above-mentionedThe external parameter between the second IMU and the camera is
An external parameter between the first IMU and the second IMU
The following equation will be satisfied:
it can be understood that the extrinsic solution between two IMUs is performed by the above formula. If there are more IMUs in the device, the calculation can be performed by using the above formula, which is not described herein again.
Step S103, selecting a coordinate system of one IMU from the plurality of IMUs as a reference coordinate system, and then converting the angle data acquired by the other IMUs into the reference coordinate system according to external parameters between the other IMUs and the IMUs serving as the reference coordinate system.
The IMU is used for measuring the three-axis attitude angle and the acceleration of the corresponding rigid body, and the attitude angle information of the rigid body can be obtained through data sampled by a sensor inside the IMU. Generally, the attitude angle information can also be expressed in terms of euler angles, i.e., described by rotation angles respectively rotated about different axes. If the orientation of the Z-axis of the IMU is chosen to be up (or down), the three attitude angles are respectively the roll angle (roll) for rotation about the X-axis, the pitch angle (pitch) for rotation about the Y-axis, and the yaw angle (yaw) for rotation about the Z-axis.
Exemplarily, in step S103, the coordinate system of one IMU may be selected as a reference for the conversion of other IMUs, and the angle data of other IMUs are all converted into the reference coordinate system, so as to facilitate the subsequent comparison of the angle change information of the same attitude angle.
Step S104, then calculating the angle change value of each IMU in the pre-selected attitude angle in the reference coordinate system.
In one embodiment, the pre-selected attitude angle is typically selected as the angle of rotation about the Z-axis, the yaw angle yaw mentioned above, as the device on which the plurality of IMUs and cameras are mounted is moved over the ground. Since the device can move in various shapes such as S-shape, O-shape, or 8-shape during the movement of the ground, the yaw angle is the most variable attitude angle. It is understood that the selection of the attitude angle may be determined according to the motion trajectory of the device in a specific application scene, the placement direction of the IMU, and the like.
In analyzing the angular change of the angle of the yaw of each IMU, angular velocity information of the yaw angle can be calculated by differentiating the angle value of the yaw angle, for example. Of course, if the angular velocity is acquired, the angular velocity change information in the corresponding time may be directly calculated.
Step S200, taking the time stamp of the IMU with the largest sampling frequency as a reference, roughly synchronizing the time stamp according to the initial time difference between the current IMU and the initial time of the IMU as the reference, and then performing time stamp translation on the current IMU for a first preset number of times according to a first preset value range corresponding to the initial time difference so as to calculate and obtain the optimal time difference corresponding to the first precision.
As shown in fig. 3, the rough synchronization and timestamp translation step in step S200 mainly includes the following sub-steps:
in sub-step S201, the time stamp of the IMU with the largest frequency is selected as a reference from the plurality of IMUs.
Because the sampling frequency of each IMU is different, the number of samples of different IMUs at the same time will also vary. To facilitate comparison of the angular variation difference between different IMUs at the same sampling time, the time stamp of the IMU with the largest frequency is exemplarily selected as the time reference.
And a substep S202 of integrally translating the timestamp of the current IMU according to an initial time difference between the starting time of the current IMU and the starting time of the IMU serving as a reference, so as to perform coarse synchronization.
Illustratively, the coarse synchronization is performed based on the recorded initial time difference between the start time of the current IMU and the start time of the reference IMU, i.e., aligning the time stamp of the current IMU towards the time stamp of the reference IMU. For example, if the initial time difference is a negative value, it indicates that the time stamp of the current IMU is ahead, and therefore it needs to be shifted backward by a value corresponding to the initial time difference; if the initial time difference is negative, it indicates that the time stamp of the current IMU is later, and therefore it needs to be moved forward for coarse synchronization.
Step S203, accumulating the values and the initial time difference in sequence from a first preset value range corresponding to the first precision to obtain a corresponding translation time value; wherein the first preset number of times is equal to the number of times of values taken from the first preset value range.
And a substep S204, respectively carrying out integral translation on the time stamp of the current IMU according to the translation time value obtained each time.
Exemplarily, if the initial time difference is T0Then, the values are sequentially recorded as { -t from the first preset value rangem,…,tm}，tm>0, i.e. from [ -tm, tm]The value is sequentially taken, and the translation time value obtained by each calculation is T0+tk，tkIndicating the retrieved kth value, and then moving the timestamp of the current IMU by T in its entirety0+tk. Generally, the first preset value range is a symmetric value, and the specific data can be selected according to actual requirements.
For emutemutexample, if the original sampling timestamp of IMU-a is 1.0, 2.0, 3.0, 4.0.. and the original sampling timestamp of IMU-B is 3.5, 4.0, 4.5, 5.0.. it is known that IMU-B is the reference IMU, if the initial time difference is 2.2, the first preset value range is [ -0.9s, 0.9s ], and if the current value is-0.8 s, the original sampling timestamp of IMU-a needs to be translated backwards by 1.4 as a whole.
As shown in fig. 4, the process of calculating the optimal time difference corresponding to the first accuracy in step S200 mainly includes the following sub-steps:
in the substep S205, for each of the other IMUs, performing angle or angular velocity interpolation according to the angle change value corresponding to the current IMU at each sampling time of the IMU serving as the reference, so as to obtain an angle data set including the interpolation of the current IMU.
It can be understood that the inserted numerical value may be an angle or an angular velocity, and specifically, the numerical value may be calculated according to the type of the acquired angle data, and either of the two may be calculated according to the angular velocity change information of the corresponding IMU. In the interpolation, a linear interpolation method is preferably adopted.
And a substep S206, performing equal-time equal-quantity interception on the angle data set of the reference IMU and the angle data set of the current IMU, and calculating and accumulating the difference of the angle data between the current IMU and the reference IMU at each sampling time to obtain an accumulated difference corresponding to the current translation.
And a substep S207 of repeating the above steps to obtain all accumulated difference values corresponding to the first preset number of translations. It is understood that the number of the accumulated difference values is equal to the first predetermined number.
And a substep S208 of selecting a minimum value from all the accumulated difference values, and taking the translation time value corresponding to the minimum value as the optimal time difference corresponding to the first precision.
Still taking the above emutemutemutexample, after the time stamp of IMU-a is translated for 1.4s, the angular data interpolation needs to be performed on IMU-a at these sampling times, 3.5, 4.0, 4.5, 5.0. The angle data for IMU-A and IMU-B at the same time are then truncated from the respective angle data sets, with equal numbers. Then, the respective angle data of IMU-A and IMU-B are subtracted, and each difference is summed up to obtain the cumulative difference. And then, after the translation is carried out for 1.5s (namely 2.2-0.7), calculating corresponding accumulated difference values according to the method, and so on until 19 groups of accumulated difference values are obtained, and then selecting a minimum value from the accumulated difference values, wherein the translation time corresponding to the minimum value is taken as the optimal time difference corresponding to the first precision (0.1 s).
And step S300, performing timestamp translation on the current IMU for a second preset number of times according to a second preset value range corresponding to the optimal time difference of the first precision to obtain the optimal time difference corresponding to the second precision, and so on until the optimal time difference meeting the preset precision is obtained to serve as the final synchronization time between the current IMU and the IMU serving as the reference.
Illustratively, the second precision is less than the first precision, such as may be 0.05s and 0.1 s. It is understood that the preset precision can be set according to actual requirements, and can be 0.05s, 0.01s and the like. And calculating the optimal time difference corresponding to high precision meeting the requirement when the time difference is calculated, and taking the optimal time difference as the final time difference of the time difference and the optimal time difference so as to carry out time synchronization.
As shown in fig. 5, the calculation of the optimal time difference corresponding to the second accuracy in step S300 mainly includes:
step S301, sequentially adding the optimal time difference between the first precision and the value in a second preset value range corresponding to the second precision to obtain a corresponding translation time value. And the second preset times are equal to the times corresponding to the second preset value range.
Step S302, the whole translation is respectively carried out on the time stamp of the current IMU according to the translation time value obtained each time, then the corresponding accumulated difference value after each translation is calculated, and the translation time value corresponding to the minimum accumulated difference value is selected from the accumulated difference value to be used as the optimal time difference corresponding to the second precision.
The difference from the solution of the optimal time difference of the first accuracy is only that the second accuracy is based on the optimal time difference of the first accuracy and is accumulated with the extracted value as the translation time value. The other steps can be referred to the above-mentioned solving process of the optimal time difference with the first precision, and therefore, the details are not described herein.
The method for synchronizing the plurality of IMUs in time provided by the embodiment comprises the steps of obtaining external parameters among the IMUs, and then converting all the IMUs to the same coordinate system according to the external parameter information to perform angle change analysis of a certain attitude angle; and comparing the difference of the angle data among the IMUs through a timestamp translation algorithm, an interpolation algorithm and the like to obtain an accumulated difference, and selecting the translation time value which enables the accumulated difference to be minimum as final synchronization time. In the translation process, the range of the time difference is gradually reduced according to different precisions until the time difference value meeting the required precision is obtained, so that the precision of time synchronization can be greatly improved. In addition, because the software synchronization mode is adopted, other hardware does not need to be added, and the hardware cost can be reduced compared with the hardware synchronization mode.
Example 2
Referring to fig. 6, based on the multiple IMU time synchronization method of embodiment 1, the present embodiment provides a multiple IMU time synchronization apparatus 10, including:
the external parameter calibration module 110 is configured to perform external parameter calibration on a plurality of IMUs based on a camera to obtain external parameters between the IMUs, convert angle data obtained by each IMU into the same coordinate system by using the external parameters, and calculate an angle change value of each IMU at a preselected attitude angle;
an initial synchronization calculation module 120, configured to use a timestamp of the IMU with the largest sampling frequency as a reference, perform rough timestamp synchronization according to an initial time difference between respective starting times of the current IMU and the IMU serving as the reference, and then perform timestamp translation on the current IMU for a first preset number of times according to a first preset value range corresponding to the initial time difference, so as to calculate an optimal time difference corresponding to a first precision; after each translation, performing interpolation and data interception according to the angle change value of the current IMU to calculate an angle data accumulated difference value between the current IMU and the IMU serving as a reference, and then obtaining a translation time value corresponding to a minimum accumulated difference value from a plurality of accumulated difference values to serve as an optimal time difference corresponding to the first precision;
the precise synchronization calculation module 130 is configured to perform timestamp translation on the current IMU for a second preset number of times according to a second preset value range corresponding to the optimal time difference of the first precision, so as to obtain an optimal time difference corresponding to the second precision, and so on until the optimal time difference meeting the preset precision is obtained, so as to be used as a final synchronization time between the current IMU and the IMU serving as a reference.
It is understood that the plurality of IMU time synchronization apparatuses 10 described above correspond to the on-screen display method of the multi-os system of embodiment 1. Any of the options in embodiment 1 are also applicable to this embodiment, and will not be described in detail here.
The present invention also provides a terminal, such as a computer, a server, etc., which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program, so as to make a terminal device execute the functions of each module in the above-mentioned plurality of IMU time synchronization methods or the above-mentioned plurality of IMU time synchronization apparatuses.
The present invention also provides a computer-readable storage medium for storing the computer program used in the above-mentioned terminal.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention.
Claims (10)
1. A method for time synchronization of a plurality of IMUs, comprising:
performing external parameter calibration on a plurality of IMUs based on a camera to obtain external parameters among the IMUs, converting angle data obtained by the IMUs to the same coordinate system by using the external parameters, and calculating angle change values of the IMUs at a preselected attitude angle; the external parameters comprise a rotation matrix and a translation matrix;
roughly synchronizing time stamps according to initial time differences between respective starting times of a current IMU and the IMU serving as the reference by taking the time stamp of the IMU with the largest sampling frequency as the reference, and then performing time stamp translation on the current IMU for a first preset number of times according to a first preset value range corresponding to the initial time differences so as to calculate and obtain an optimal time difference corresponding to first precision;
after each translation, performing interpolation and data interception according to the angle change value of the current IMU to calculate an angle data accumulated difference value between the current IMU and the IMU serving as a reference, and then obtaining a translation time value corresponding to a minimum accumulated difference value from a plurality of accumulated difference values to serve as an optimal time difference corresponding to the first precision;
and performing time stamp translation on the current IMU for a second preset number of times according to a second preset value range corresponding to the optimal time difference of the first precision to obtain an optimal time difference corresponding to the second precision, and so on until the optimal time difference meeting the preset precision is obtained to be used as final synchronization time between the current IMU and the IMU serving as the reference.
2. The method of claim 1, wherein the step of calculating the angle change value of each IMU at the preselected attitude angle after performing external reference calibration on the IMUs based on the camera to obtain external parameters between the IMUs and converting the angle data obtained by each IMU to the same coordinate system by using the external parameters comprises:
performing external reference calibration on a plurality of IMUs by using a calibration plate and based on a camera to obtain external parameters between each IMU and the camera;
calculating external parameters between each IMU according to the external parameters between each IMU and the camera;
selecting a coordinate system of one IMU from the IMUs as a reference coordinate system, and then converting angle data acquired by other IMUs into the reference coordinate system according to the external parameters between the other IMUs and the IMU serving as the reference coordinate system;
and then calculating the angle change value of each IMU at a preselected attitude angle in the reference coordinate system.
3. The method of claim 2, wherein the plurality of IMUs comprise a first IMU and a second IMU, and wherein the external parameter between the first IMU and the camera is recorded as the external parameter
Extrinsic parameters between the second IMU and the cameraIs marked as
An external parameter between the first IMU and the second IMU
The following formula is satisfied:
4. the method of time synchronizing a plurality of IMUs according to claim 1, wherein the "roughly synchronizing the time stamps according to initial time differences between respective start times of a current IMU and an IMU serving as a reference, with a time stamp of an IMU having a maximum sampling frequency as a reference, and then performing time stamp translation on the current IMU for a first preset number of times according to a first preset value range corresponding to the initial time differences" includes:
selecting an IMU with the largest sampling frequency from the plurality of IMUs as a time reference for angle data interpolation;
integrally translating a time stamp of the current IMU according to an initial time difference between the starting time of the current IMU and the starting time of the IMU serving as a reference so as to perform rough synchronization;
sequentially taking values from the first preset value range corresponding to the first precision and accumulating the initial time difference to obtain a corresponding translation time value; the first preset times are equal to the times corresponding to the first preset value range;
and respectively carrying out integral translation on the time stamp of the current IMU according to the translation time value obtained each time.
5. The multiple IMU time synchronization method of claim 4, wherein the calculating obtains an optimal time difference corresponding to a first precision; performing interpolation and data interception according to the angle change value of the current IMU after each translation to calculate an angle data cumulative difference between the current IMU and an IMU serving as a reference, and then obtaining a translation time value corresponding to a minimum cumulative difference from a plurality of cumulative differences to serve as an optimal time difference corresponding to the first precision "includes:
for each of the other IMUs, performing angle or angular velocity interpolation at each sampling time of the IMU serving as a reference according to the angle change value corresponding to the current IMU to obtain an angle data set of the current IMU, which contains the interpolation;
carrying out equal-time equal-quantity interception on the angle data set of the IMU serving as the reference and the angle data set of the IMU, calculating and accumulating the difference value of the angle data between the current IMU and the IMU serving as the reference at each sampling moment to obtain the accumulated difference value corresponding to the current translation;
then repeating the steps to obtain all accumulated difference values corresponding to the first preset times of translation;
and selecting a minimum value from all accumulated difference values, and taking a translation time value corresponding to the minimum value as an optimal time difference corresponding to the first precision.
6. The method of claim 1, wherein the second precision is less than the first precision, and wherein the step of performing a second predetermined number of time stamp translations on the current IMU according to a second predetermined value range corresponding to the optimal time difference of the first precision to obtain an optimal time difference corresponding to the second precision comprises:
sequentially taking values from a second preset value range corresponding to the second precision and accumulating the optimal time difference of the first precision to obtain a corresponding translation time value; the second preset times are equal to the value times corresponding to the second preset value range;
and respectively carrying out integral translation on the time stamp of the current IMU according to the translation time value obtained each time, then calculating the corresponding accumulated difference value after each translation, and selecting the translation time value corresponding to the minimum accumulated difference value from the accumulated difference value to be used as the optimal time difference corresponding to the second precision.
7. The multiple IMU time synchronization method of claim 1, wherein the preselected attitude angle is a rotation angle about an upwardly directed Z-axis as a device on which the plurality of IMUs are disposed moves over a ground surface.
8. A multiple IMU time synchronization apparatus, comprising:
the external parameter calibration module is used for carrying out external parameter calibration on the IMUs based on the camera so as to obtain external parameters among the IMUs, converting angle data obtained by the IMUs to the same coordinate system by using the external parameters, and then calculating angle change values of the IMUs at a preselected attitude angle; the external parameters comprise a rotation matrix and a translation matrix;
the initial synchronization calculation module is used for roughly synchronizing the time stamps according to initial time differences between respective starting moments of a current IMU and the IMU serving as a reference by taking the time stamp of the IMU with the largest sampling frequency as the reference, and then performing time stamp translation on the current IMU for a first preset number of times according to a first preset value range corresponding to the initial time differences so as to calculate and obtain an optimal time difference corresponding to a first precision; after each translation, performing interpolation and data interception according to the angle change value of the current IMU to calculate an angle data accumulated difference value between the current IMU and the IMU serving as a reference, and then obtaining a translation time value corresponding to a minimum accumulated difference value from a plurality of accumulated difference values to serve as an optimal time difference corresponding to the first precision;
and the accurate synchronization calculation module is used for performing timestamp translation on the current IMU for a second preset number of times according to a second preset value range corresponding to the optimal time difference of the first precision to obtain the optimal time difference corresponding to the second precision, and so on until the optimal time difference corresponding to the preset precision is obtained to serve as the final synchronization time between the current IMU and the IMU serving as the reference.
9. A terminal device, comprising: a processor and a memory, the memory storing a computer program for executing the computer program to implement the plurality of IMU time synchronization methods of any of claims 1-7.
10. A computer-readable storage medium, characterized in that it stores a computer program which, when executed, implements a plurality of IMU time synchronization methods according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010422627.1A CN111580596B (en) | 2020-05-19 | 2020-05-19 | Method, device and terminal for synchronizing multiple IMUs (inertial measurement units) in time |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010422627.1A CN111580596B (en) | 2020-05-19 | 2020-05-19 | Method, device and terminal for synchronizing multiple IMUs (inertial measurement units) in time |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111580596A CN111580596A (en) | 2020-08-25 |
| CN111580596B true CN111580596B (en) | 2022-04-15 |
Family
ID=72126796
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010422627.1A Active CN111580596B (en) | 2020-05-19 | 2020-05-19 | Method, device and terminal for synchronizing multiple IMUs (inertial measurement units) in time |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111580596B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112598757B (en) * | 2021-03-03 | 2021-06-01 | 之江实验室 | Multi-sensor time-space calibration method and device |
| CN115979277B (en) * | 2023-02-22 | 2023-06-02 | 广州导远电子科技有限公司 | Time synchronization method, apparatus, electronic device, and computer-readable storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107850436A (en) * | 2015-05-23 | 2018-03-27 | 深圳市大疆创新科技有限公司 | Merged using the sensor of inertial sensor and imaging sensor |
| CN108629793A (en) * | 2018-03-22 | 2018-10-09 | 中国科学院自动化研究所 | The vision inertia odometry and equipment demarcated using line duration |
| CN109376785A (en) * | 2018-10-31 | 2019-02-22 | 东南大学 | Air navigation aid based on iterative extended Kalman filter fusion inertia and monocular vision |
| CN109544638A (en) * | 2018-10-29 | 2019-03-29 | 浙江工业大学 | A kind of asynchronous online calibration method for Multi-sensor Fusion |
| CN110702107A (en) * | 2019-10-22 | 2020-01-17 | 北京维盛泰科科技有限公司 | Monocular vision inertial combination positioning navigation method |
| CN110987021A (en) * | 2019-12-25 | 2020-04-10 | 湖北航天技术研究院总体设计所 | Inertial vision relative attitude calibration method based on rotary table reference |
-
2020
- 2020-05-19 CN CN202010422627.1A patent/CN111580596B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107850436A (en) * | 2015-05-23 | 2018-03-27 | 深圳市大疆创新科技有限公司 | Merged using the sensor of inertial sensor and imaging sensor |
| CN108629793A (en) * | 2018-03-22 | 2018-10-09 | 中国科学院自动化研究所 | The vision inertia odometry and equipment demarcated using line duration |
| CN109544638A (en) * | 2018-10-29 | 2019-03-29 | 浙江工业大学 | A kind of asynchronous online calibration method for Multi-sensor Fusion |
| CN109376785A (en) * | 2018-10-31 | 2019-02-22 | 东南大学 | Air navigation aid based on iterative extended Kalman filter fusion inertia and monocular vision |
| CN110702107A (en) * | 2019-10-22 | 2020-01-17 | 北京维盛泰科科技有限公司 | Monocular vision inertial combination positioning navigation method |
| CN110987021A (en) * | 2019-12-25 | 2020-04-10 | 湖北航天技术研究院总体设计所 | Inertial vision relative attitude calibration method based on rotary table reference |
Non-Patent Citations (2)
| Title |
|---|
| Sensor Synchronization for AR Applications;T. Kantonen;《2010 IEEE International Symposium on Mixed and Augmented Reality》;20101122;245-246 * |
| 分布式POS时间同步算法及软件设计;贾黎冬 等;《现代电子技术》;20160415;1-5 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111580596A (en) | 2020-08-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105698765B (en) | Object pose method under double IMU monocular visions measurement in a closed series noninertial systems | |
| CN109376785B (en) | Navigation method based on iterative extended Kalman filtering fusion inertia and monocular vision | |
| CN109696183B (en) | Calibration method and device of inertia measurement unit | |
| CN108627153B (en) | Rigid body motion tracking system based on inertial sensor and working method thereof | |
| CN111415387B (en) | Camera pose determining method and device, electronic equipment and storage medium | |
| CN111580596B (en) | Method, device and terminal for synchronizing multiple IMUs (inertial measurement units) in time | |
| CN113311411B (en) | Laser radar point cloud motion distortion correction method for mobile robot | |
| CN109544630B (en) | Pose information determination method and device and visual point cloud construction method and device | |
| KR20210084622A (en) | Time synchronization processing methods, electronic devices and storage media | |
| CN108344413B (en) | Underwater glider navigation system and low-precision and high-precision conversion method thereof | |
| CN107270894B (en) | GNSS/SINS deep integrated navigation system based on dimension reduction | |
| CN114413887B (en) | Sensor external parameter calibration method, device and medium | |
| CN111060138B (en) | Calibration method and device, processor, electronic equipment and storage medium | |
| CN113551665B (en) | High-dynamic motion state sensing system and sensing method for motion carrier | |
| CN114216456B (en) | Attitude measurement method based on fusion of IMU and robot body parameters | |
| CN110044377B (en) | Vicon-based IMU offline calibration method | |
| CN112781586A (en) | Pose data determination method and device, electronic equipment and vehicle | |
| CN110956666A (en) | Motion data calibration method and device, terminal equipment and storage medium | |
| CN113820735A (en) | Method for determining position information, position measuring device, terminal, and storage medium | |
| KR20050097618A (en) | System and method for making digital map using gps and ins | |
| CN111121755B (en) | Multi-sensor fusion positioning method, device, equipment and storage medium | |
| EP3614256B1 (en) | Data processing method, computer device, and storage medium | |
| CN113009816B (en) | Method and device for determining time synchronization error, storage medium and electronic device | |
| CN112154480B (en) | Positioning method and device for movable platform, movable platform and storage medium | |
| CN117073720A (en) | Method and equipment for quick visual inertia calibration and initialization under weak environment and weak action control |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Room 2301-2308, third floor, building 2, incubator, Zhongguancun Software Park, Dongbeiwang, Haidian District, Beijing 100094 Applicant after: Beijing Digital Green Earth Technology Co.,Ltd. Address before: Room 2301-2308, third floor, building 2, incubator, Zhongguancun Software Park, Dongbeiwang, Haidian District, Beijing 100094 Applicant before: BEIJING GREENVALLEY TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |