CN111982148A

CN111982148A - Processing method, device and system for VIO initialization and computer equipment

Info

Publication number: CN111982148A
Application number: CN202010639341.9A
Authority: CN
Inventors: 李东轩; 张高明; 刘海伟; 丛林
Original assignee: Hangzhou Yixian Advanced Technology Co ltd
Current assignee: Hangzhou Yixian Advanced Technology Co ltd
Priority date: 2020-07-06
Filing date: 2020-07-06
Publication date: 2020-11-24
Anticipated expiration: 2040-07-06
Also published as: CN111982148B

Abstract

The application relates to a processing method, a device, a system and computer equipment for VIO initialization, wherein the processing method comprises the following steps: acquiring fusion information of state quantity of the VIO terminal; determining a first device-related state quantity in the state quantity under the condition that the information entropy of the fusion information is greater than a preset threshold value; acquiring the information entropy according to the solving result of the state quantity queue in the preset time period; loading the related state quantity of the first equipment to a filter so as to complete the hot start of the VIO terminal; through the method and the device, the problem of low accuracy in the initialization processing process of the VIO is solved.

Description

Processing method, device and system for VIO initialization and computer equipment

Technical Field

The application relates to the technical field of visual inertial odometers, in particular to a processing method, a device and a system for VIO initialization and computer equipment.

Background

The Visual Inertial Odometer (VIO) algorithm is an algorithm that integrates information of a camera and an Inertial Measurement Unit (IMU), calculates the pose of equipment in space, and is widely applied to the fields of Augmented Reality (AR), unmanned vehicles and the like due to the characteristic of high efficiency and robustness.

The VIO algorithm is realized based on Bayesian estimation algorithm, taking the VIO algorithm based on Extended Kalman Filter (EKF for short) as an example, the EKF can maintain a state quantity queue and a covariance matrix corresponding to the state quantity, and updates the state quantity and the covariance matrix by fusing sensor information; after the VIO algorithm is started, the state quantity and the covariance are initialized once, and the initialization quality directly determines the operation quality of the VIO algorithm, so that the initialization of the state quantity is important in the process of the VIO algorithm; in the related art, a common VIO state quantity initialization method generally provides a rough state quantity initial value by solving nonlinear optimization; then, the nonlinear optimization also has the situation of solving errors or abnormal operation, which introduces risks to the normal operation of the VIO algorithm and leads to lower accuracy in the initialization processing process of the visual inertial odometer.

At present, no effective solution is provided for the problem of low accuracy in the initialization processing process of the VIO in the related technology.

Disclosure of Invention

The embodiment of the application provides a processing method, a device and a system for VIO initialization and computer equipment, which are used for at least solving the problem of low accuracy in the initialization processing process of the VIO in the related art.

In a first aspect, an embodiment of the present application provides a processing method for VIO initialization, where the method includes:

acquiring fusion information of state quantity of the VIO terminal;

determining a first device-related state quantity in the state quantities under the condition that the information entropy of the fusion information is greater than a preset threshold value; acquiring the information entropy according to a solving result of a state quantity queue in a preset time period;

and loading the related state quantity of the first equipment to a filter, and further completing the warm start of the VIO terminal.

In some embodiments, the loading the first device-related state quantity to a filter to complete the warm-boot of the VIO terminal comprises:

acquiring a covariance matrix queue according to the first equipment related state quantity, and determining a second equipment related state quantity at a preset moment under the condition that the data square root of a main diagonal line of the covariance matrix queue at the preset moment is smaller than a preset threshold value;

loading the second device dependent state quantity to the filter.

In some of these embodiments, said loading said second device dependent state quantity to said filter comprises:

determining a matrix block of the second device correlation state quantity matched in the covariance matrix queue by utilizing a covariance matrix marginalization technology;

and pre-reading the related state quantity of the second equipment and the matrix block into a hot start file, and loading the filter according to the hot start file.

In some embodiments, the loading the filter according to the warm boot file comprises:

loading the filter according to the hot start file under the condition that the hot start file is detected; in case the warm boot file is not detected, re-determining the first device related state quantity.

In some embodiments, the determining, in the case that the information entropy is greater than a preset threshold, a first device-related state quantity of the state quantities comprises:

determining the convergence result of the state quantity by using a parameter convergence judgment basis;

and determining the related state quantity of the first equipment under the condition that the convergence result is operation convergence and the information entropy is greater than the preset threshold value.

In a second aspect, an embodiment of the present application provides a processing apparatus for VIO initialization, where the apparatus includes: the system comprises an acquisition module, a first state quantity module and a hot start module;

the acquisition module is used for acquiring the fusion information of the state quantity of the VIO terminal;

the first state quantity module is used for determining the first equipment-related state quantity in the state quantity under the condition that the information entropy of the fusion information is greater than a second preset value; acquiring the information entropy according to a solving result of a state quantity queue in a preset time period;

and the hot start module is used for loading the related state quantity of the first equipment to a filter so as to complete the hot start of the VIO terminal.

In a third aspect, an embodiment of the present application provides a processing system for VIO initialization, where the system includes: a VIO terminal and a processor;

the processor acquires the fusion information of the state quantity of the VIO terminal;

the processor determines a first device-related state quantity in the state quantities under the condition that the information entropy of the fusion information is greater than a preset threshold value; acquiring the information entropy according to a solving result of a state quantity queue in a preset time period;

and the processor loads the first equipment related state quantity to a filter so as to complete the warm start of the VIO terminal.

In some embodiments, the processor is further configured to obtain a covariance matrix queue according to the first device-related state quantity, and determine a second device-related state quantity at a preset time when a square root of data of a main diagonal line of the covariance matrix queue at the preset time is smaller than a preset threshold;

the processor loads the second device-dependent state quantity to the filter.

In a fourth aspect, embodiments of the present application provide a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, the processor implements the processing method for VIO initialization as described in the first aspect.

In a fifth aspect, embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the processing method for VIO initialization as described in the first aspect above.

Compared with the related art, the processing method, the device, the system and the computer equipment for VIO initialization provided by the embodiment of the application acquire the fusion information of the state quantity of the VIO terminal; determining a first device-related state quantity in the state quantity under the condition that the information entropy of the fusion information is greater than a preset threshold value; acquiring the information entropy according to the solving result of the state quantity queue in the preset time period; and loading the related state quantity of the first equipment to a filter to complete the hot start of the VIO terminal, thereby solving the problem of low accuracy in the initialization processing process of the VIO.

The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below to provide a more thorough understanding of the application.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

fig. 1 is a schematic diagram of an application scenario of a VIO initialization processing method according to an embodiment of the present invention;

fig. 2 is a flowchart of a VIO initialization processing method according to an embodiment of the present application;

fig. 3 is a flow chart of another VIO initialization processing method according to an embodiment of the present application;

FIG. 4A is a diagram illustrating covariance matrix marginalization according to an embodiment of the present application;

FIG. 4B is a diagram illustrating another covariance matrix marginalization according to an embodiment of the present application;

fig. 5 is a flowchart of another VIO initialization processing method according to an embodiment of the present application;

fig. 6 is a flowchart of another VIO initialization processing method according to an embodiment of the present application;

fig. 7 is a block diagram of a VIO initialization processing apparatus according to an embodiment of the present application;

fig. 8 is a block diagram of a VIO initialization processing system according to an embodiment of the present application;

fig. 9 is a block diagram of the inside of a computer device according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments provided in the present application without any inventive step are within the scope of protection of the present application. Moreover, it should be appreciated that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which may vary from one implementation to another.

Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the specification. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of ordinary skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments without conflict.

Unless defined otherwise, technical or scientific terms referred to herein shall have the ordinary meaning as understood by those of ordinary skill in the art to which this application belongs. Reference to "a," "an," "the," and similar words throughout this application are not to be construed as limiting in number, and may refer to the singular or the plural. The present application is directed to the use of the terms "including," "comprising," "having," and any variations thereof, which are intended to cover non-exclusive inclusions; for example, a process, method, system, article, or apparatus that comprises a list of steps or modules (elements) is not limited to the listed steps or elements, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. Reference to "connected," "coupled," and the like in this application is not intended to be limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. Reference herein to "a plurality" means greater than or equal to two. "and/or" describes an association relationship of associated objects, meaning that three relationships may exist, for example, "A and/or B" may mean: a exists alone, A and B exist simultaneously, and B exists alone. Reference herein to the terms "first," "second," "third," and the like, are merely to distinguish similar objects and do not denote a particular ordering for the objects.

In this embodiment, an application scenario of a processing method for VIO initialization is provided, and fig. 1 is a schematic diagram of an application scenario of a processing method for VIO initialization according to an embodiment of the present invention, as shown in fig. 1, in this application environment, a VIO terminal 12 communicates with a server 14 through a network; the VIO terminal 12 may include a camera, an IMU sensor, and a processing chip. The server 14 obtains the running time and the fusion information of the state quantity of the VIO terminal 12, and determines a first device-related state quantity in the state quantity according to the running time and the fusion information; the server 14 loads the first device-dependent state quantity to a filter, thereby completing a warm-start of the VIO terminal 12; the VIO terminal 12 may be, but is not limited to, various unmanned aerial vehicles, robots, missiles and AR terminals, and the server 14 may be implemented by an independent server or a server cluster composed of a plurality of servers.

Fig. 2 is a flowchart of a VIO initialization processing method according to an embodiment of the present application, and as shown in fig. 2, the method includes the following steps:

step S202, acquiring the fusion information of the state quantity of the VIO terminal 12; acquiring the information entropy of the fusion information according to a preset time period, namely a solving result of the state quantity queue within a fixed time period; under the condition that the information entropy is larger than a preset threshold value, determining a first device related state quantity in the state quantity; the preset threshold value can be set by a user; the state quantity list of the VIO algorithm generally includes two types of state quantities: a motion-related state quantity and a device-related state quantity; the motion-related state quantity is related to a specific motion mode and a specific track, and is usually different from operation to operation; the device-related state quantities are only related to specific devices on the VIO terminal 12, and in the operation process of the VIO terminal 12, the longer the operation time of each device state quantity on the VIO terminal 12 is, the more the fusion information is, and the more accurate the estimation of the state quantity is;

it should be noted that, after the program starts to run on the VIO terminal 12, the running time can be obtained through the counter; meanwhile, fusion information can be obtained through information entropy calculation; generally, the VIO algorithm is realized based on the EKF, the information entropy can be calculated through the covariance matrix of the EKF, and along with the increase of the running time, the information entropy is larger and larger, which indicates that the information is fused more and more;

therefore, the preset threshold value can be set by a user, and compared with the information entropy, if the information entropy is greater than the preset threshold value, it indicates that the running time is long and the fusion information is more, the estimation of the detected state quantity in the VIO running process is more accurate, and the corresponding state quantity, namely the related state quantity of the first device, can be determined; the first device-related state quantity may comprise parameters such as camera extrinsic parameters, camera-IMU time delay, and camera roll-blind parameters.

Step S204, loading the first device-related state quantity to a filter, thereby completing the warm start of the VIO terminal 12; during the operation of the VIO, the device-related state quantities corresponding to the time points with more accurate estimation are selected and stored, and the stored state quantities are preloaded into the filter to complete initialization at the initial stage of the VIO each time, which is equivalent to performing 'warm start' (Warmstart) of the state quantities of the VIO terminal 12.

The positioning performance of the VIO depends on the precision of visual inertia initialization, the system convergence speed can be reduced by a pathological initialization model, and even the state estimation of the whole system is wrong, so that the reasonably designed VIO state quantity initialization method is particularly critical; in the related art, there is usually a situation of solving a wrong solution or abnormal operation in performing VIO state quantity initialization by solving nonlinear optimization, for example, a linear solver is used to solve an output initial value to a nonlinear optimizer, if the linear solving fails, the initial value error of the state quantity in the VIO initialization is large, and further the nonlinear optimization is wrong;

compared with the nonlinear optimization with very large uncertainty in the related technology, the embodiment of the application selects and determines the first equipment correlation quantity with more accurate estimation according to the fusion information through the steps S202 to S204, so that the preloading mode is more reliable; after the first equipment correlation quantity is determined, the first equipment correlation state quantity is loaded to a filter, so that the state quantity to be calculated by a processor is reduced, the calculation time consumption is also improved, and the processing efficiency of VIO initialization is greatly improved; in summary, since the warm-boot algorithm of the state quantity of the VIO terminal 12 is added in the VIO application, the initialization process of the VIO terminal 12 can be optimized, and the problem of low accuracy in the initialization processing process of the VIO is solved.

In some embodiments, a processing method for VIO initialization is provided, and fig. 3 is a flowchart of another processing method for VIO initialization according to an embodiment of the present application, as shown in fig. 3, the method includes the following steps:

step S302, acquiring a covariance matrix queue according to the first device-related state quantity, and determining a second device-related state quantity at a preset time under the condition that the data square root of the main diagonal line of the covariance matrix queue at the preset time is less than a preset threshold; the preset moment is the operation time point of the state quantity;

in order to ensure that the state quantity is stored at a time point when the state quantity estimation is accurate, the variance of the state quantity and the main diagonal line of a covariance matrix queue are counted by maintaining a state quantity queue and a covariance matrix queue with fixed lengths; vector, namely on the data structure, allocating a vector container with the length of N, wherein the vector container can store N state quantities, and the maintenance rule mainly aims at visual characteristic observation, covariance or observation needing deletion and the like corresponding to the state quantities;

if the square root of the data of each element on the variance of the state quantity at the preset moment and the main diagonal line of the covariance matrix is smaller than a preset threshold, the estimation of the state quantity at the moment tends to be stable, and the state quantity at the preset moment, namely the related state quantity of the second equipment is stored; it should be noted that, assuming that the observed value of the related state quantity of the first device obeys multivariate gaussian distribution, the probability of the gaussian distribution is strongly related to the covariance, so the state quantity vector and the covariance matrix queue are counted to detect the state quantities of a series of time axes; the statistical state vector is shown in formula 1, and the covariance matrix queue is shown in formula 2:

it should be noted that the covariance matrix in formula 2 can be calculated by a recursive expression; for example, the initial value of the covariance matrix is a matrix of all 0 s, and then every time an operation is performed, the covariance matrix is transformed based on the covariance matrix of the previous step, wherein the transformation is mainly realized by multiplying the state transition matrix by the initial value of the covariance matrix and adding the input noise matrix.

Step S304, loading the second device-related state quantity to the filter; in the next VIO starting stage, the hot starting of the state quantity can be completed only by reading the corresponding state quantity and covariance matrix blocks as the stored values in advance.

Through the above steps S302 to S304, the state quantity queue and covariance matrix queue with fixed lengths are maintained, the state quantity variance and the main diagonal line of the covariance matrix are counted, the second device-related state quantity at a time point with a relatively accurate estimation in the first device-related state quantity is determined according to each data on the main diagonal line, and the second device-related state quantity is stored, thereby further improving the accuracy of the VIO state quantity initialization.

In some of these embodiments, loading the second device dependent state quantity to the filter further comprises the steps of:

step S402, determining a matrix block of the second device correlation state quantity matched in the covariance matrix queue by using a covariance matrix marginalization technology; FIG. 4A is a schematic diagram of covariance matrix marginalization according to an embodiment of the present application; as shown in FIG. 4A, the whole square in the graph represents a covariance matrix, and the diagonal overlay portion is the portion to be marginalized by the covariance matrix; the graphic color of each small square represents the value consistency of the corresponding small square; for example, black indicates that there are numbers other than 0 in the matrix block, and white indicates that the numbers in the corresponding matrix block are all 0; it can be understood that the depth of the graphic color can also be used to represent the density of the numerical value, i.e. the darker the color, the higher the density of the numerical value, and will not be described herein again;

FIG. 4B is a schematic diagram of another covariance matrix marginalization according to an embodiment of the application, as shown in FIG. 4B, FIG. 4B identifies the result of one-time marginalization, and it can be seen that the portion to be marginalized in FIG. 4A is deleted compared to FIG. 4A; meanwhile, the deleted information in fig. 4A is absorbed by the retained matrix blocks in fig. 4B, and the density of the values in these matrix blocks is correspondingly improved; the main purpose of the marginalization is therefore to delete certain rows or columns of the matrix, but to ensure that no information is lost as far as possible.

Step S404, pre-reading the related state quantity of the second device and the matrix block as a hot start file, and loading the filter according to the hot start file; reading parameters stored in the hot start file, loading parameters such as state quantity to the filter, and completing the hot start of the VIO state quantity;

in some of these embodiments, in the event that the warm boot file is detected, loading the filter according to the warm boot file; and under the condition that the hot start file is not detected, re-determining the related state quantity of the first equipment, thereby realizing real-time detection of the hot start file and avoiding VIO state quantity initialization errors caused by the fact that parameters are not loaded yet.

Through the steps S402 to S404, the matrix block is obtained through the covariance matrix marginalization technique, so that when the relevant state quantity of the second device is stored, the corresponding block of the relevant state quantity of the second device in the covariance matrix queue is simultaneously stored, information loss is avoided, and accuracy of VIO state quantity initialization is improved.

In some embodiments, a processing method for VIO initialization is provided, and fig. 5 is a flowchart of another processing method for VIO initialization according to an embodiment of the present application, as shown in fig. 5, the method includes the following steps:

step S502, determining the convergence result of the state quantity by using the parameter convergence judgment basis; under an ideal condition, the state quantity of the VIO terminal 12 has a true value, and the estimated value is closer to the true value along with the operation of the algorithm, so that the true value needs to be known in advance, and whether the parameter of the estimated value converges to be close to the true value is judged; alternatively, it is determined whether or not the parameter has converged to a certain extent, that is, whether or not the slope becomes 0, but this case does not necessarily converge to a true value;

step S504, determining the first device-related state quantity when the convergence result is operation convergence and the entropy is greater than a preset threshold.

Through the steps S502 to S504, the estimation of each state quantity of the VIO terminal 12 is comprehensively analyzed to determine the first device related quantity with accurate estimation by using the parameter convergence judgment basis in combination with the judgment result of the running time and the fusion information, thereby avoiding the determination error of the first device related quantity, reducing the initial value error of the state quantity loaded to the filter, and further improving the accuracy of the VIO initialization processing process.

The following describes an embodiment of the present invention in detail with reference to an actual application scenario, and fig. 6 is a flowchart of another VIO initialization processing method according to an embodiment of the present application, where specific implementation steps of the VIO initialization processing method are shown in fig. 6.

Step S602, analyzing the estimation accuracy of each state quantity, wherein the estimation is mainly judged according to the parameter convergence result, and the judgment is carried out according to the running time and the fusion information; then, selecting an accurately estimated state quantity, namely a related state quantity of the first equipment according to the analysis result;

step S604, obtaining the variance and covariance matrix queue of the selected first device state quantity, and judging whether the square root of the element on the main diagonal of the matrix queue is smaller than a preset threshold value; if not, continuously acquiring the variance and covariance matrix of the state quantity at the next moment;

step S606, if yes, saving the state quantity at the moment, namely the related state quantity of the second device, and saving the related state quantity of the second device and a matrix block, which is obtained by utilizing a covariance matrix marginalization technology and is matched with the related state quantity of the second device in the covariance matrix queue, to a hot start file;

step S608, checking whether a hot start file exists, if not, detecting each state quantity in a circulating mode again; and if so, loading the state quantity at the moment, namely the related state quantity of the second equipment to a filter according to the hot start file, and finishing the 'hot start' of the VIO state quantity.

It should be noted that the VIO initialization processing method in this embodiment may be divided into two parts, one is a storage logic part in step S602 to step S606, and the other is a reading logic part in step S608; in a hot start mode, the success rate of VIO state quantity initialization can reach 100%; and only the time consumed by parameter reading in the initialization process is almost ignored compared with the time consumed by calculation.

It should be understood that, although the steps in the flowcharts of fig. 2, 3, 5 and 6 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2, 3, 5, and 6 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or stages is not necessarily sequential, but may be performed alternately or alternatingly with other steps or at least some of the sub-steps or stages of other steps.

In this embodiment, a processing apparatus for VIO initialization is provided, and is used to implement the foregoing embodiments and preferred embodiments, which have already been described and are not described again. As used hereinafter, the terms "module," "unit," "subunit," and the like may implement a combination of software and/or hardware for a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.

Fig. 7 is a block diagram of a configuration of a VIO initialization processing apparatus according to an embodiment of the present application, as shown in fig. 7, the apparatus including: an acquisition module 72, a first state quantity module 74, and a warm start module 76; the acquiring module 72 is configured to acquire fusion information of state quantities of the VIO terminal 12; the first state quantity module 74 is configured to determine a first device-related state quantity in the state quantities when the information entropy of the fusion information is greater than a preset threshold; the warm boot module 76 is configured to load the first device-related state quantity to a filter, thereby completing a warm boot of the VIO terminal 12.

Through the above embodiment, the obtaining module 72 obtains the running time and the fusion time of the state quantity of the VIO terminal 12, and the first state quantity module 74 selects and determines the first device related quantity with more accurate estimation according to the running time and the fusion information, so that the preloading mode is more reliable, and the problem of low accuracy in the initial processing process of the VIO is solved; moreover, after the hot start module 76 determines the first device-related quantity, the first device-related state quantity is loaded to the filter, so that the state quantity to be calculated is reduced, the calculation time is also increased, and the processing efficiency of VIO initialization is greatly improved.

In some of these embodiments, the processing device further comprises a second state quantity module; the second state quantity module acquires a covariance matrix queue according to the first equipment related state quantity, and determines the second equipment related state quantity at a preset moment under the condition that the data square root of the main diagonal line of the covariance matrix queue at the preset moment is smaller than a preset threshold; the warm boot module 76 is also configured to load the second device related state quantity to the filter.

In some embodiments, the second state quantity module is further configured to determine a matrix block in which the second device-related state quantities match in the covariance matrix queue using a covariance matrix marginalization technique; the warm-boot module 76 is further configured to pre-read the second device-related state quantities and the matrix blocks as a warm-boot file, and load the filter according to the warm-boot file.

In some of these embodiments, the warm boot module 76 is further configured to pre-read the second device related state quantities and the matrix blocks as a warm boot file, and load the filter according to the warm boot file.

In some of these embodiments, the processing apparatus further comprises a convergence module; the convergence module is used for determining the convergence result of the state quantity by utilizing the parameter convergence judgment basis; the first state quantity module 74 is further configured to determine the first device-related state quantity if the convergence result is operation convergence and the entropy is greater than a preset threshold.

The above modules may be functional modules or program modules, and may be implemented by software or hardware. For a module implemented by hardware, the modules may be located in the same processor; or the modules can be respectively positioned in different processors in any combination.

In this embodiment, a processing system for VIO initialization is provided, and fig. 8 is a block diagram of a structure of a processing system for VIO initialization according to an embodiment of the present application, and as shown in fig. 8, the system includes: the VIO terminal 12 and the processor 82; the processor 82 obtains the fusion information of the state quantities of the VIO terminal 12; the processor 82 determines a first device-related state quantity in the state quantities when the information entropy of the fusion information is greater than a preset threshold; the processor 82 loads the first device-dependent state quantity into a filter to complete the warm-up of the VIO terminal 12.

Through the above embodiment, the processor 82 obtains the fusion information of the state quantities of the VIO terminal 12, and selects and determines the first device correlation quantity with more accurate estimation according to the fusion information, so that the preloading mode is more reliable, and the problem of low accuracy in the initial processing process of the VIO is solved; moreover, after the processor 82 determines the first device-related quantity, the processor 82 loads the first device-related state quantity to the filter, so that the state quantity to be calculated by the processor 82 is reduced, calculation time consumption is also improved, and the processing efficiency of VIO initialization is greatly improved.

In some embodiments, the processor 82 is further configured to obtain a covariance matrix queue according to the first device-related state quantity, and determine a second device-related state quantity at a preset time when a square root of data of a main diagonal of the covariance matrix queue at the preset time is smaller than a preset threshold; the processor 82 loads the second device dependent state quantity to the filter.

In some of these embodiments, the processor 82 is further configured to determine a matrix block in which the second device-related state quantities match in the covariance matrix queue using a covariance matrix marginalization technique; the processor 82 pre-reads the second device-related state quantities and the matrix blocks as a warm-boot file, and loads the filter according to the warm-boot file.

In some embodiments, the processor 82 is further configured to load the filter according to the warm boot file if the warm boot file is detected; in case the warm boot file is not detected, the first device-related state quantity is re-determined.

In some embodiments, the processor 82 is further configured to determine a convergence result of the state quantity by using a parameter convergence criterion; the processor 82 determines the first device-related state quantity if the convergence result is operation convergence and the entropy is greater than a preset threshold.

In some embodiments, a computer device is provided, and the computer device may be a server, and fig. 9 is a structural diagram of the inside of the computer device according to the embodiment of the present invention, as shown in fig. 9. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing the hot start file data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a processing method for VIO initialization.

Those skilled in the art will appreciate that the architecture shown in fig. 9 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, a computer device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the steps of the VIO initialization processing method provided in the above embodiments.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and the computer program, when executed by a processor, implements the steps in the VIO initialization processing method provided in the above embodiments.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

It should be understood by those skilled in the art that various features of the above-described embodiments can be combined in any combination, and for the sake of brevity, all possible combinations of features in the above-described embodiments are not described in detail, but rather, all combinations of features which are not inconsistent with each other should be construed as being within the scope of the present disclosure.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A method for initializing a visual-inertial odometer VIO, the method comprising:

acquiring fusion information of state quantity of the VIO terminal;

2. The processing method of claim 1, wherein said loading the first device-dependent state quantity to a filter to complete a warm-boot of the VIO terminal comprises:

loading the second device dependent state quantity to the filter.

3. The processing method of claim 2, wherein said loading the second device-dependent state quantity to the filter comprises:

4. The processing method of claim 3, wherein said loading the filter according to the warm boot file comprises:

5. The processing method according to claim 1, wherein the determining a first device-related state quantity of the state quantities in the case that the information entropy of the fusion information is greater than a preset threshold value comprises:

6. A VIO-initialized processing apparatus, the apparatus comprising: the system comprises an acquisition module, a first state quantity module and a hot start module;

the first state quantity module is used for determining the first equipment-related state quantity in the state quantity under the condition that the information entropy of the fusion information is greater than a preset threshold value; acquiring the information entropy according to a solving result of a state quantity queue in a preset time period;

7. A VIO-initialized processing system, the system comprising: a VIO terminal and a processor;

8. The processing system according to claim 7, wherein the processor is further configured to obtain a covariance matrix queue according to the first device-related state quantity, and determine a second device-related state quantity at a preset time in a case that a square root of data of a main diagonal of the covariance matrix queue at the preset time is smaller than a preset threshold;

the processor loads the second device-dependent state quantity to the filter.

9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method of any of claims 1 to 5 are implemented when the computer program is executed by the processor.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 5.