CN111553933A - Optimization-based visual inertia combined measurement method applied to real estate measurement - Google Patents
Optimization-based visual inertia combined measurement method applied to real estate measurement Download PDFInfo
- Publication number
- CN111553933A CN111553933A CN202010304587.0A CN202010304587A CN111553933A CN 111553933 A CN111553933 A CN 111553933A CN 202010304587 A CN202010304587 A CN 202010304587A CN 111553933 A CN111553933 A CN 111553933A
- Authority
- CN
- China
- Prior art keywords
- measurement
- imu
- real estate
- visual
- optimization
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/269—Analysis of motion using gradient-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an optimized vision inertia combination measurement method applied to real estate measurement, and provides a vision inertia combination measurement method aiming at the condition that a GPS signal is shielded or the visibility condition is poor in the real estate measurement process, and mainly relates to the aspects of camera and IMU data processing, system initialization, data fusion algorithm and the like. Firstly, extracting, tracking and matching feature points of an image acquired by a camera, and pre-integrating data acquired by an IMU (inertial measurement Unit); secondly, a good initial value is provided for nonlinear optimization through joint initialization of a camera and an IMU; then, establishing a target function by constructing a visual reprojection error, an IMU measurement error and a priori information error; and finally, minimizing the objective function by a nonlinear optimization method to obtain the optimal position of each moment in the carrier motion process. The invention has the beneficial effects that: the vision inertia combination measurement method improves the measurement positioning precision, the measurement stability and the measurement reliability of the real estate.
Description
Technical Field
The invention belongs to the technical field of surveying and mapping, and particularly relates to an optimization-based visual inertia combination measuring method applied to real estate measurement.
Background
In real estate measurement, a GPS is the most widely applied measurement method, and for an occasion where a GPS signal is lost, a strapdown Inertial measurement system is generally used to complete real estate measurement, and an Inertial measurement unit (IMU, including a gyroscope and an accelerometer) senses angular increment and specific force information of a carrier, and further resolves information such as attitude, speed, position, and the like of the carrier, thereby implementing a real estate measurement positioning function.
The strapdown inertial measurement method is suitable for completing real estate measurement under the condition that GPS signals are easily blocked or the through-view condition is poor, but the positioning error of the strapdown inertial measurement system is dispersed along with the accumulation of time due to the error of the inertial sensor, so that high-precision measurement and positioning for a long time cannot be performed. In order to ensure the smooth operation of real estate measuring work, the invention provides a visual inertia combined measuring mode for carrying out real estate measuring work. The vision measurement system has the advantages of low cost, convenient data acquisition and processing, high precision, capability of capturing rich information in a measurement scene, and large measurement error under the high dynamic condition; under the high dynamic condition, the strapdown inertial measurement system can obtain accurate pose estimation in a short time, and the influence of the dynamic condition on the vision measurement system is reduced. In turn, the vision measurement system can also effectively correct the error accumulation existing in the long-time work of the strapdown inertia measurement system, and the two measurement systems can complement each other to a certain extent and can obtain a better measurement result when being used together.
Disclosure of Invention
The purpose of the invention is as follows: aiming at the problems that information acquired by a single sensor is limited, insufficient in precision and poor in stability, the invention provides a method for completing real estate measurement by adopting a visual-inertial combined measurement mode, and aims to improve the measurement efficiency, the measurement reliability and the stability of a measurement system on the premise of meeting certain measurement precision.
The above purpose is realized by the following technical scheme:
the technical scheme is as follows: an optimization-based visual inertia combination measurement method applied to real estate measurement comprises the following steps:
s1, extracting Harris corners from the image collected by the camera, tracking adjacent frame images by utilizing a pyramid optical flow method, tracking and matching feature points, and then removing mismatching through a RANSAC algorithm;
s2, integrating data acquired by the IMU between two frames of images to obtain the position, the speed and the posture of a carrier at the current moment, and meanwhile, calculating the pre-integration quantity of the IMU data between adjacent frames of images to be used in the following nonlinear optimization;
s3, performing pure vision initialization by using an SFM method, estimating the camera pose when each frame of image is collected in a sliding window, and then aligning with IMU data pre-integration variables to solve initialization parameters;
s4, putting the prior information constraint, the IMU constraint and the visual constraint in a large objective function F (x) for nonlinear optimization, and solving the positions, the speeds and the postures of the carriers at all the moment of the key frame images in the sliding window and the bias of the IMU;
s5, carrying out iterative solution on the nonlinear optimization objective function F (χ) established in S4 by adopting an LM algorithm, and solving χ when the F (χ) obtains a minimum value;
and S6, continuously repeating the steps S4 and S5 for the image and IMU data newly added into the sliding window, outputting the optimal track change of the carrier, and improving the measurement positioning precision of the real estate.
Further, in the method of the present invention, in step S2, the formula for calculating the amount of pre-integration of IMU data between adjacent frames of images is as follows:
wherein i and j represent the time of adjacent frame images, a and omega represent data output by the IMU, and the calculation is carried out by adopting a median method:
further, in the method of the present invention, in step S3, the initialization parameters mainly include a gravity vector direction, a visual scale factor, and an initial velocity and an initial pose of the carrier at each image frame time, and a global reference coordinate system is determined according to the gravity vector direction.
Further, in the method of the present invention, in step S4, the established objective function is:
wherein χ is a state quantity required to be optimized in the sliding window, specifically χ ═ x1,x2,…,xs,λ1,λ2,…,λt],k∈[1,s];{bp,ΛpIs a priori information of the system; a is the set of expected components of the IMU, B is the set of observations of all feature points for all frame images in the sliding window;anderrors in the IMU's predicted components and visual measurements, respectively.
Further, in the method of the present invention, in step S5, the normal equation solved by using the LM algorithm is:
initial value mu0Is to JTJ is subjected to eigenvalue decomposition, i.e. JTJ=VΛVT,μ0And JTThe J max eigenvalues are in the same order of magnitude. Update policy pass ratio of μExample factor ρ, see the following equation:
if ρ < 0, the error function is increased, and rejection is made at this timeThe iteration is updated, the damping factor mu is increased, and the step length of delta chi is reduced; if rho is close to 1 or larger, the damping factor mu can be reduced, the step length of delta chi is increased, and the iteration efficiency is improved; if p is a relatively small positive number, the reduction amplitude of the primitive function is too small, and f (x) is withinThe first order approximation model cannot fit the primitive function well, and at this time, the damping factor μ needs to be increased and the step length of Δ χ needs to be reduced. μ employs the following update strategy:
ifρ>0
else
μ=μ*υ;υ=2*υ
compared with solving methods such as a Gauss-Newton method and the like, the LM algorithm can adaptively adjust the step length of state quantity iterative updating by adding the damping factor, so that the effectiveness of state quantity updating can be ensured, and the iterative efficiency can be timely improved.
Has the advantages that: compared with the prior art, the invention has the following advantages:
compared with the prior art, the optimized visual inertia combination measurement method applied to real estate measurement has the following advantages that:
(1) the IMU adopted in the conventional strapdown inertial measurement system is high in precision and high in cost, and the IMU adopted in the invention is an MEMS (micro electro mechanical system), so that the cost is low, and the measurement positioning precision equivalent to that of the high-precision IMU can be achieved;
(2) the visual inertia combined measurement method provided by the invention can be used for measuring real estate in various motion environments (fast, slow and the like) and various measurement environments (indoor, outdoor, light change and the like), and has the advantages of high program operation robustness, high measurement stability and high reliability.
Drawings
FIG. 1 is a flowchart of a process of the present invention for an optimized visual inertia based combination measurement method for real estate measurement;
FIG. 2 is a schematic diagram showing the comparison of the simulation of the present invention with the estimated trajectory results of the strapdown inertial measurement system;
FIG. 3 is a comparison curve of the position error of the estimated result of the simulation of the present invention and the strapdown inertial measurement system;
FIG. 4 is a schematic diagram illustrating a comparison of the track estimation results with truth values according to the present invention;
FIG. 5 is a graph disclosing data sets for the position error of the estimation results of the present invention.
Detailed Description
The present invention will be further illustrated with reference to the accompanying drawings and specific embodiments, which are to be understood as merely illustrative of the invention and not as limiting the scope of the invention.
As shown in fig. 1, the optimized visual inertia combination measurement method applied to real estate measurement of the present invention specifically includes the following steps:
s1, extracting Harris corners from the image collected by the camera, tracking adjacent frame images by utilizing a pyramid optical flow method, tracking and matching feature points, and then removing mismatching through a RANSAC algorithm;
s2, integrating data acquired by the IMU between two frames of images to obtain the position, the speed and the posture of the carrier at the current moment, and simultaneously calculating the pre-integration quantity of the IMU data between adjacent frames of images to be used in the following nonlinear optimization, wherein the calculation formula is as follows:
wherein i and j represent the time of adjacent frame images, a and omega represent data output by the IMU, and the calculation is carried out by adopting a median method:
s3, performing pure visual initialization by using an SFM method, estimating the camera pose when each frame of image is collected in a sliding window, aligning with IMU data pre-integration quantity to solve initialization parameters, mainly comprising a gravity vector direction, a visual scale factor, and the initial speed and the initial pose of a carrier at each image frame moment, and determining a global reference coordinate system according to the gravity vector direction;
and S4, putting the prior information constraint, the IMU constraint and the visual constraint in a large objective function F (x) for nonlinear optimization, and solving the positions, speeds and postures of the carriers and the bias of the IMU at all the key frame image moments in the sliding window. The established objective function is:
wherein χ is a state quantity required to be optimized in the sliding window, specifically χ ═ x1,x2,…,xs,λ1,λ2,…,λt],k∈[1,s];{bp,ΛpIs a priori information of the system; a is the set of expected components of the IMU, B is the set of observations of all feature points for all frame images in the sliding window;anderrors of the IMU predicted components and the vision measurement values, respectively;
s5, carrying out iterative solution on the nonlinear optimization objective function F (χ) established in S4 by adopting an LM algorithm, and solving to ensure that
F (χ) is the minimum χ; the normal equation solved using the LM algorithm is:initial value mu0Is to JTJ is subjected to eigenvalue decomposition, i.e. JTJ=VΛVT,μ0And JTThe J max eigenvalues are in the same order of magnitude. The update strategy for μ is determined by the scale factor ρ, as follows:
if ρ < 0, the error function is increased, and rejection is made at this timeThe iteration is updated, the damping factor mu is increased, and the step length of delta chi is reduced; if rho is close to 1 or larger, the damping factor mu can be reduced, the step length of delta chi is increased, and the iteration efficiency is improved; if p is a relatively small positive number, the reduction amplitude of the primitive function is too small, and f (x) is withinThe first order approximation model cannot fit the primitive function well, and at this time, the damping factor μ needs to be increased and the step length of Δ χ needs to be reduced. μ employs the following update strategy:
ifρ>0
else
μ=μ*υ;υ=2*υ
compared with solving methods such as a Gauss-Newton method and the like, the LM algorithm can adaptively adjust the step length of state quantity iterative updating by adding the damping factor, so that the effectiveness of state quantity updating can be ensured, and the iterative efficiency can be timely improved.
And S6, continuously repeating the steps S4 and S5 for the image and IMU data newly added into the sliding window, outputting the optimal track change of the carrier, and improving the measurement positioning precision of the real estate.
Specific examples are as follows:
in order to verify the effectiveness of the method, simulation and verification of a semi-physical experiment are respectively carried out. The simulation is set to be that the motion time of the carrier is 39.9993s, and the total length of the motion trail is 58.3886 m; the carrier makes an elliptic motion in an xy plane of a world coordinate system, a long semi-axis and a short semi-axis are respectively 6m and 3m, a sinusoidal motion is made in a z-axis direction of the world coordinate system, and the amplitude and the frequency are respectively 0.3m and 0.2 pi. The experimental results of the method of the invention are compared with the results of the strapdown inertial measurement system as shown in fig. 2 and fig. 3.
The semi-physical experiment verification adopts an open data set for verification, MH _04_ difficult data sequence in the EuRoC data set is used as experiment data, the total track length of the motion of the carrier in the data set is 91.747m, the motion time is 98.76s, the average speed is 0.929m/s, and the average angular speed is 0.24 rad/s. The track and error curve of the carrier motion obtained by the method running on MH _04_ diffcult are shown in figures 4 and 5, and can be obtained from the figures.
The above examples are only preferred embodiments of the present invention, it should be noted that: it will be apparent to those skilled in the art that various modifications and equivalents can be made without departing from the spirit of the invention, and it is intended that all such modifications and equivalents fall within the scope of the invention as defined in the claims.
Claims (5)
1. An optimized visual inertia combination measurement method applied to real estate measurement is characterized by comprising the following steps:
s1, extracting Harris corners from the image collected by the camera, tracking adjacent frame images by utilizing a pyramid optical flow method, tracking and matching feature points, and then removing mismatching through a RANSAC algorithm;
s2, integrating data acquired by the IMU between two frames of images to obtain the position, the speed and the posture of a carrier at the current moment, and meanwhile, calculating the pre-integration quantity of the IMU data between adjacent frames of images to be used in the following nonlinear optimization;
s3, performing pure vision initialization by using an SFM method, estimating the camera pose when each frame of image is collected in a sliding window, and then aligning with IMU data pre-integration variables to solve initialization parameters;
s4, putting the prior information constraint, the IMU constraint and the visual constraint in a large objective function F (x) for nonlinear optimization, and solving the positions, the speeds and the postures of the carriers at all the moment of the key frame images in the sliding window and the bias of the IMU;
s5, carrying out iterative solution on the nonlinear optimization objective function F (χ) established in S4 by adopting an LM algorithm, and solving χ when the F (χ) obtains a minimum value;
and S6, continuously repeating the steps S4 and S5 for the image and IMU data newly added into the sliding window, outputting the optimal track change of the carrier, and improving the measurement positioning precision of the real estate.
2. The method for measuring visual inertia combination based on optimization of real estate measurement as claimed in claim 1, wherein in step S2, the pre-integration amount of IMU data between adjacent frame images is calculated by the formula:
wherein i and j represent the time of adjacent frame images, a and omega represent data output by the IMU, and the calculation is carried out by adopting a median method:
3. the visual inertia combination measurement method applied to real estate measurement based on optimization as claimed in claim 1, wherein in step S3, the initialization parameters mainly include gravity vector direction, visual scale factor, and initial velocity and initial pose of the carrier at each image frame time, and the global reference coordinate system is determined according to the gravity vector direction.
4. The method for optimizing visual-inertial combination measurement for real estate measurement according to claim 1, wherein in step S4, the objective function is established as:
wherein χ is a state quantity required to be optimized in the sliding window, specifically χ ═ x1,x2,…,xs,λ1,λ2,…,λt],{bp,ΛpIs a priori information of the system; a is the set of expected components of the IMU, B is the set of observations of all feature points for all frame images in the sliding window;andthe IMU predicted component and vision, respectivelyError in the measured value.
5. The visual-inertial combination measurement method applied to real estate measurement based on optimization as claimed in claim 1, wherein in step S5, the normal equation solved by using LM algorithm is:initial value mu0Is to JTJ is subjected to eigenvalue decomposition, i.e. JTJ=VΛVT,μ0And JTThe maximum characteristic value of J is in the same order of magnitude; the update strategy for μ is determined by the scale factor ρ, as follows:
if ρ < 0, the error function is increased, and rejection is made at this timeThe iteration is updated, the damping factor mu is increased, and the step length of delta chi is reduced; if rho is close to 1 or larger, the damping factor mu can be reduced, the step length of delta chi is increased, and the iteration efficiency is improved; if p is a relatively small positive number, the reduction amplitude of the primitive function is too small, and f (x) is withinThe first-order approximation model cannot well fit the primitive function, and the damping factor mu needs to be increased and the step length of delta x needs to be reduced; μ employs the following update strategy:
ifρ>0
else
μ=μ*υ;υ=2*υ。
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010304587.0A CN111553933B (en) | 2020-04-17 | 2020-04-17 | Optimization-based visual inertia combined measurement method applied to real estate measurement |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010304587.0A CN111553933B (en) | 2020-04-17 | 2020-04-17 | Optimization-based visual inertia combined measurement method applied to real estate measurement |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111553933A true CN111553933A (en) | 2020-08-18 |
CN111553933B CN111553933B (en) | 2022-11-18 |
Family
ID=72003850
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010304587.0A Active CN111553933B (en) | 2020-04-17 | 2020-04-17 | Optimization-based visual inertia combined measurement method applied to real estate measurement |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111553933B (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112837374A (en) * | 2021-03-09 | 2021-05-25 | 中国矿业大学 | Space positioning method and system |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109993113A (en) * | 2019-03-29 | 2019-07-09 | 东北大学 | A kind of position and orientation estimation method based on the fusion of RGB-D and IMU information |
CN110345944A (en) * | 2019-05-27 | 2019-10-18 | 浙江工业大学 | Merge the robot localization method of visual signature and IMU information |
-
2020
- 2020-04-17 CN CN202010304587.0A patent/CN111553933B/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109993113A (en) * | 2019-03-29 | 2019-07-09 | 东北大学 | A kind of position and orientation estimation method based on the fusion of RGB-D and IMU information |
CN110345944A (en) * | 2019-05-27 | 2019-10-18 | 浙江工业大学 | Merge the robot localization method of visual signature and IMU information |
Non-Patent Citations (1)
Title |
---|
程传奇等: "基于非线性优化的单目视觉/惯性组合导航算法", 《中国惯性技术学报》 * |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112837374A (en) * | 2021-03-09 | 2021-05-25 | 中国矿业大学 | Space positioning method and system |
CN112837374B (en) * | 2021-03-09 | 2023-11-03 | 中国矿业大学 | Space positioning method and system |
Also Published As
Publication number | Publication date |
---|---|
CN111553933B (en) | 2022-11-18 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111024066B (en) | Unmanned aerial vehicle vision-inertia fusion indoor positioning method | |
CN110243358B (en) | Multi-source fusion unmanned vehicle indoor and outdoor positioning method and system | |
CN109307508B (en) | Panoramic inertial navigation SLAM method based on multiple key frames | |
CN108731670B (en) | Inertial/visual odometer integrated navigation positioning method based on measurement model optimization | |
CN111780754B (en) | Visual inertial odometer pose estimation method based on sparse direct method | |
CN110726406A (en) | Improved nonlinear optimization monocular inertial navigation SLAM method | |
Weiss et al. | Real-time onboard visual-inertial state estimation and self-calibration of MAVs in unknown environments | |
CN110702107A (en) | Monocular vision inertial combination positioning navigation method | |
CN110044354A (en) | A kind of binocular vision indoor positioning and build drawing method and device | |
CN112240768A (en) | Visual inertial navigation fusion SLAM method based on Runge-Kutta4 improved pre-integration | |
CN112649016A (en) | Visual inertial odometer method based on point-line initialization | |
US12008173B2 (en) | Multi-sensor handle controller hybrid tracking method and device | |
CN109671105A (en) | A kind of the tracking restoration methods and device of vision navigation system | |
CN110207693B (en) | Robust stereoscopic vision inertial pre-integration SLAM method | |
CN111932674A (en) | Optimization method of line laser vision inertial system | |
CN110986968A (en) | Method and device for real-time global optimization and error loop judgment in three-dimensional reconstruction | |
CN114485640A (en) | Monocular vision inertia synchronous positioning and mapping method and system based on point-line characteristics | |
CN107942090B (en) | A kind of spacecraft Attitude rate estimator method for extracting Optic flow information based on fuzzy star chart | |
Zheng et al. | SE (2)-constrained visual inertial fusion for ground vehicles | |
CN114690229A (en) | GPS-fused mobile robot visual inertial navigation method | |
CN111553933B (en) | Optimization-based visual inertia combined measurement method applied to real estate measurement | |
CN108827287B (en) | Robust visual SLAM system in complex environment | |
CN112945233B (en) | Global drift-free autonomous robot simultaneous positioning and map construction method | |
CN112731503B (en) | Pose estimation method and system based on front end tight coupling | |
CN112284381B (en) | Visual inertia real-time initialization alignment method and system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |