WO2021035669A1 - Procédé de prévision de pose, procédé de construction de carte, plateforme mobile et support de stockage - Google Patents

Procédé de prévision de pose, procédé de construction de carte, plateforme mobile et support de stockage Download PDF

Info

Publication number
WO2021035669A1
WO2021035669A1 PCT/CN2019/103599 CN2019103599W WO2021035669A1 WO 2021035669 A1 WO2021035669 A1 WO 2021035669A1 CN 2019103599 W CN2019103599 W CN 2019103599W WO 2021035669 A1 WO2021035669 A1 WO 2021035669A1
Authority
WO
WIPO (PCT)
Prior art keywords
image frame
current image
encoder
map
frame
Prior art date
Application number
PCT/CN2019/103599
Other languages
English (en)
Chinese (zh)
Inventor
朱张豪
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to CN201980033455.4A priority Critical patent/CN112219087A/zh
Priority to PCT/CN2019/103599 priority patent/WO2021035669A1/fr
Publication of WO2021035669A1 publication Critical patent/WO2021035669A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
    • G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
    • G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
    • G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G01C21/28—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network with correlation of data from several navigational instruments
    • G01C21/30—Map- or contour-matching
    • G01C21/32—Structuring or formatting of map data
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C3/00—Measuring distances in line of sight; Optical rangefinders
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06—Systems determining position data of a target
    • G01S17/08—Systems determining position data of a target for measuring distance only
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/86—Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence

Definitions

  • the embodiment of the present invention relates to the field of machine vision, in particular to a pose prediction method, a map construction method, a movable platform and a storage medium.
  • Visual SLAM Simultaneous Localization and Mapping, real-time positioning and map construction
  • SLAM Simultaneous Localization and Mapping, real-time positioning and map construction
  • the existing visual SLAM method can cause special problems such as sparse feature points in the image (no texture) or too much repetition with the previous image frame, blur or non-overlapping area caused by high-speed motion, and accidental reduction of the camera frame rate.
  • the tracking function of visual SLAM is invalid, that is, the relative pose of the current image frame relative to the previous image frame cannot be obtained, which makes the visual SLAM unable to continue to work, and the vision can be restored by repositioning when the closed loop is triggered.
  • the existing visual SLAM method has poor stability and robustness. When a visual failure occurs, the vision cannot be restored until the closed loop occurs. Therefore, the visual SLAM will be interrupted when the visual failure occurs, and the map points and trajectory during the visual failure process are missing. , Causing the map points and trajectories before and after the visual failure to fail to connect, resulting in the visual SLAM unable to operate stably.
  • the embodiment of the present invention provides a pose prediction method, a map construction method, a movable platform, and a storage medium to improve the accuracy of the pose acquisition of the current image frame, improve the stability and robustness of visual SLAM, and prevent visual failure.
  • Time Vision SLAM can still run stably.
  • the first aspect of the embodiments of the present invention is to provide a pose prediction method, which is suitable for a movable platform.
  • the movable platform includes a first scroll wheel, a second scroll wheel, a camera, a first encoder, and a second encoder.
  • the first scroll wheel and the second scroll wheel are used to displace the movable platform, the axes of the first scroll wheel and the second scroll wheel coincide;
  • the camera is used to collect image frames, the The first encoder is used to detect the speed of the first scroll wheel, and the second encoder is used to detect the speed of the second scroll wheel; the method includes:
  • the second aspect of the embodiments of the present invention is to provide a movable platform, including: a first scroll wheel, a second scroll wheel, a camera, a first encoder, a second encoder, a memory, and a processor;
  • the first rolling wheel and the second rolling wheel are used for displacing the movable platform, and the axes of the first rolling wheel and the second rolling wheel coincide;
  • the camera is used to collect image frames
  • the first encoder is used to detect the speed of the first scroll wheel
  • the second encoder is used to detect the speed of the second scroll wheel
  • the memory is used to store program codes
  • the processor calls the program code, and when the program code is executed, is used to perform the following operations:
  • a third aspect of the embodiments of the present invention is to provide a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to implement the method described in the first aspect.
  • the fourth aspect of the embodiments of the present invention is to provide a map construction method, which is suitable for a movable platform, and the movable platform includes a first scroll wheel, a second scroll wheel, a camera, a first encoder, and a second encoder ,
  • the first scroll wheel and the second scroll wheel are used to displace the movable platform, the axes of the first scroll wheel and the second scroll wheel coincide;
  • the camera is used to collect image frames,
  • the first An encoder is used to detect the speed of the first scroll wheel, and the second encoder is used to detect the speed of the second scroll wheel;
  • the method includes:
  • the constructed map After obtaining the predicted pose of the current image frame, on the basis of the constructed map, the constructed map is updated according to the current image frame; the constructed map is determined before the current image frame The key frames are constructed.
  • the fifth aspect of the embodiments of the present invention is to provide a movable platform, including: a first scroll wheel, a second scroll wheel, a camera, a first encoder, a second encoder, a memory, and a processor;
  • the first rolling wheel and the second rolling wheel are used for displacing the movable platform, and the axes of the first rolling wheel and the second rolling wheel coincide;
  • the camera is used to collect image frames
  • the first encoder is used to detect the speed of the first scroll wheel
  • the second encoder is used to detect the speed of the second scroll wheel
  • the memory is used to store program codes
  • the processor calls the program code, and when the program code is executed, is used to perform the following operations:
  • the constructed map After obtaining the predicted pose of the current image frame, on the basis of the constructed map, the constructed map is updated according to the current image frame; the constructed map is determined before the current image frame The key frames are constructed.
  • the sixth aspect of the embodiments of the present invention is to provide a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to implement the method described in the fourth aspect.
  • the first time period is acquired according to the time stamp of the current image frame and the time stamp of the previous image frame; and the first time is acquired The measurement data of the first encoder and the measurement data of the second encoder in the segment; according to the measurement data of the first encoder and the measurement data of the second encoder in the first time period, the current image frame relative to the upper A predicted transformation matrix of an image frame; obtaining the predicted pose of the current image frame according to the predicted change matrix and the pose information of the previous image frame.
  • the embodiment of the present invention applies the encoder to the visual SLAM and predicts the pose of the current image frame through the encoder measurement data, which can improve the accuracy of the pose acquisition, improve the stability and robustness of the visual SLAM, and prevent visual failure.
  • Time Vision SLAM can still run stably.
  • Fig. 1 is a flowchart of a map construction method provided by an embodiment of the present invention
  • Figure 2 is a flowchart of a pose prediction method provided by an embodiment of the present invention.
  • FIG. 3 is a flowchart of a map construction method provided by another embodiment of the present invention.
  • FIG. 5 is a flowchart of a map construction method provided by another embodiment of the present invention.
  • Fig. 6 is a flowchart of a map construction method provided by another embodiment of the present invention.
  • FIG. 7 is a schematic structural diagram of a movable platform provided by another embodiment of the present invention.
  • a component when referred to as being "fixed to” another component, it can be directly on the other component or a central component may also exist. When a component is considered to be “connected” to another component, it can be directly connected to the other component or there may be a centered component at the same time.
  • the embodiment of the present invention provides a map construction method.
  • the map construction method is applied to a movable platform, the movable platform may be a robot, an unmanned vehicle, etc., as shown in FIG. 7, the movable platform 700 specifically includes a first scroll wheel 701 and a second scroll wheel 702 , The first encoder 703, the second encoder 704, the camera 705, and further may include a processor 706 and a memory 707.
  • the first scroll wheel 701 and the second scroll wheel 702 are used to displace the movable platform, and the axes of the first scroll wheel 701 and the second scroll wheel 702 coincide; the camera 705 is used to capture images Frame, the first encoder 703 is used to detect the rate of the first scroll wheel 701, and the second encoder 704 is used to detect the rate of the second scroll wheel 702.
  • the camera 705 can be a monocular camera, a binocular camera, an RGBD depth camera, etc.
  • the camera 705 adopts a monocular camera it is usually used in close coupling with an IMU (Inertial Measurement Unit).
  • the IMU is a measuring unit integrating two types of sensors, a gyroscope for measuring three-dimensional rotation and an accelerometer for three-dimensional acceleration; of course, when the camera 705 adopts a binocular camera and an RGBD depth camera It can also be used in conjunction with IMU to further improve accuracy, but the use of IMU has higher requirements for sensor calibration.
  • Encoder refers to the angular displacement sensor, which measures the speed of the wheel by detecting the number of radians that the robot wheel has turned in a certain period of time. It is mainly divided into three types: photoelectric, contact, and electromagnetic.
  • the map construction method according to the embodiment of the present invention can be divided into a tracking thread S11, a partial mapping thread S12, and a loop detection thread S13.
  • the tracking thread S11 includes predicting the pose of an image frame, as shown in FIG. As shown, the pose of the predicted image frame specifically includes:
  • Step S101 Obtain a first time period according to the time stamp of the current image frame and the time stamp of the previous image frame.
  • the preprocessing process specifically includes:
  • the images collected include RGB images and depth images. You can first convert the GRB image to a grayscale image, extract feature points from the grayscale image, and then calculate the stereo coordinates of the feature points based on the grayscale image and the depth image. That is, the coordinate of the feature point relative to the false right camera, the image collected by the RGBD camera is equivalent to the image collected by the binocular camera.
  • the collected images are also converted into grayscale images, the feature points are extracted from the grayscale images, and the feature points in the left camera and right camera images are matched to calculate the depth of the feature points to obtain the stereo coordinates of the feature points .
  • the extracted feature points in this embodiment can be ORB (Oriented Brief) feature points.
  • the ORB feature is a rotation-invariant feature, which has high stability, is insensitive to lighting and moving objects, and is calculated The amount is small, and it can be processed in real time using only the CPU of a conventional notebook computer; of course, the feature points can also be Harris corner points, SIFT (Scale-invariant Feature Transform) feature points, SURF (Speeded Up Robust Features) feature points, etc.
  • Step S102 Obtain the measurement data of the first encoder and the measurement data of the second encoder in the first time period.
  • the measurement data of the first encoder or the second encoder is acquired, it can be determined whether the measurement data is available, specifically as follows:
  • the acquisition frequency measured by the encoder is usually high, such as 200 Hz, while the acquisition frequency of the image is usually low, such as 30 Hz.
  • the encoder measurement data collected at the same time as the image Therefore, it is possible to find the encoder measurement data closest to the image acquisition time, that is, the time error between the encoder measurement data acquisition time and the current image frame acquisition time does not exceed the preset time threshold.
  • Step S103 Obtain a prediction transformation matrix of the current image frame relative to the previous image frame according to the measurement data of the first encoder and the measurement data of the second encoder in the first time period.
  • Step S104 Obtain the predicted pose of the current image frame according to the predicted change matrix and the pose information of the previous image frame.
  • the camera pose of the current image frame can be predicted according to the encoder data to obtain the predicted pose of the current image frame.
  • the movement of the movable platform is a two-dimensional movement (plane or curved surface), and the ground has sufficient friction.
  • the curved surface can adopt the idea of Taylor expansion, which is approximated to a flat surface in a short time, that is, the frame rate is high enough. It can also handle curved surface movement.
  • the predictive transformation matrix T 10 may be derived through a plane motion model, such as a motion model of a differential two-wheeled vehicle.
  • x-axis component of the measured speed v v and z-axis EE component of the measured angular velocity ⁇ EE [omega] satisfy the differential motion model of the motorcycle, as shown in formula (2) (wherein E represents a current image frame coding coordinate system, Take the center of the two-wheel track as the origin, and the x-axis points forward):
  • first scroll wheel speed and the second scroll wheel speed measured by the encoder all data between the encoder data closest to the current image frame and the previous image frame acquisition time. It is understandable that in the process of obtaining the predictive transformation matrix T 10 , all the data in the first time period can be obtained with Data, you can also get part of with data.
  • Ei represents the encoder coordinate system of the previous image frame
  • ⁇ EE ⁇ represents the antisymmetric matrix of the measured angular velocity of the current image frame, with Respectively represent the first derivative of the current image frame rotation matrix and position with respect to time.
  • the rotation matrix will be predicted And prediction translation matrix From the encoder coordinate system to the camera coordinate system, the prediction transformation matrix between i and j can be obtained as shown in formula (4):
  • T CjCi is the aforementioned predictive transformation matrix T 10 .
  • the fixed transformation matrix T CE of the camera coordinate system relative to the encoder coordinate system used in the conversion needs to be calibrated in advance.
  • the encoder is applied to the visual SLAM, and the current image frame pose is predicted through the encoder measurement data, which can solve the blur or non-overlapping area caused by sparse feature points (no texture) or too much repetition, high-speed motion, The accidental reduction of the camera frame rate and other visual failure problems, so that the map points and trajectories before and after the failure can be connected, the SLAM system will not be interrupted due to this, and the stability and robustness of the visual SLAM can be improved.
  • the tracking thread S11 further includes optimizing the predicted pose, specifically:
  • the predicted pose of the current image frame is optimized by the preset motion-only BA (motion-only buy adjustment) to obtain the optimized pose of the current image frame;
  • the pure motion beam adjustment includes reprojection error and encoder error.
  • the predicted pose of the current image frame is optimized by pure motion beam adjustment, where BA (bundle adjustment) is to adjust camera parameters (mainly refer to external parameters: the poses of many key frames; it is also possible Including internal parameters: focal length and optical center pixel coordinates) and structure (the position of the map point seen by the camera), so that the reprojection error (that is, the edge connecting the two key frames corresponds to a bunch of matching point pairs, of which key frame 1 is After the point is projected onto key frame 2 through geometric relations, the coordinate difference with the matching point is minimized (that is, the sum of the squares of the two norms of the above-mentioned error about the information matrix is minimized as the objective function of the optimization), a nonlinear optimization method, While Motion-only BA only adjusts camera parameters (mainly refers to the poses of many key frames), but the optimized objective function is still a nonlinear optimization method based on reprojection errors.
  • BA beam adjustment
  • camera parameters mainly refer to external parameters: the poses of many key frames; it is also possible Including internal parameters
  • the reprojection error reprojects the feature points on the predicted pose of the current image frame through the map points of the local map, and calculates the error between the reprojected feature points and the feature points of the current image frame.
  • the optimization formula of the objective function of the pure motion beam adjustment may be as shown in formula (5):
  • the part on the right side min is the objective function.
  • the first term is the prior error, which is generally 0. However, if the poses of the two frames before and after are optimized at the same time when tightly coupled with the IMU, you can Use the Hessian matrix in the optimization process of Motion-only BA of the previous image frame as the information matrix ⁇ 0 -1 to keep the pose of the previous image frame as unchanged as possible; the second term is the reprojection error term corresponding to the visual observation z il ;
  • the last term is the error term formed by the encoder information E ij (that is, the modulus of the error vector is about the square of the covariance matrix).
  • the encoder error term can be obtained by the following formula (6):
  • e R represents the error of the relative rotation matrix (using logarithmic mapping or its Lie algebra, it is a three-dimensional vector)
  • e p represents the error of the relative translation (three-dimensional vector), that is, the encoder error Is a six-dimensional vector corresponding to the covariance matrix It is a 6 ⁇ 6 square matrix, and its inverse matrix, the information matrix, is used here.
  • e R , e p and the covariance matrix All are based on the above-mentioned pre-integration to obtain the rotation matrix And relative translation Calculated.
  • R CiW and p CiW to the world coordinate system, the pose of an image frame, R CE and p CE required i.e. formula (4) in advance involved calibration transformation matrix T CE fixing elements.
  • the aforementioned Motion-only BA uses non-linear optimization to calculate the optimized pose R CjW and p CjW of the current image frame that minimizes the objective function.
  • Jacobian matrix used in the nonlinear optimization process in this embodiment can be numerical, and of course, the analytical solution derived from the Lie algebra related formula can also be used. The latter often makes the optimization more stable.
  • the pure motion beam adjustment further includes an IMU error term.
  • a predictive violent matching method is usually used to project the feature points of the previous image frame onto the current image by using the predictive transformation matrix of the uniform motion model (including the rotation matrix and the translation vector) After the frame, a small rectangular range is selected for fast violent matching.
  • the pose and speed of the previous image frame are used to estimate the initial pose of the current image frame, and then the initial pose is optimized to obtain the optimized pose.
  • the present invention uses the wheel speed measured by the encoder to estimate the initial pose of the current image frame, and then optimizes the initial pose to obtain the optimized pose, which is more robust.
  • the method may further include:
  • Step S202 Judge whether the vision is invalid
  • step S203 If the vision is invalid, go to step S203; if the vision is not invalid, go to step S204;
  • Step S203 Use the predicted pose of the current image frame as the pose of the current image frame
  • Step S204 Use the optimized pose of the current image frame as the pose of the current image frame.
  • the pure motion beam adjustment fails, that is, when the optimized pose cannot be obtained through the objective function of the pure motion beam adjustment described above, it is determined that the vision is invalid. If the vision fails, it means that the Motion-only BA process is invalid, that is, the optimized pose obtained by the final optimization is not accurate, and there is a large deviation from the real pose, so it is not used, and the predicted pose is used as the current image frame. If the vision has not failed, that is, the Motion-only BA process is successful, the optimized pose can be used as the pose of the current image frame.
  • the visual failure can be caused by the following reasons: the feature points in the image are sparse (no texture) or repeated too much with the previous image frame, blur or non-overlapping area caused by high-speed motion, and accidental decrease of the camera frame rate.
  • the optimization result cannot be obtained through the pure motion beam adjustment model, it is determined that the current image frame is visually invalid relative to the previous image frame.
  • the optimized pose obtained by Motion-only BA can be used to re-match the feature points of the current image frame and the frame with the same feature points as the current image frame (or The map points in the map are matched with the feature points of the current image frame). During the matching process, the same feature points in the frames and the current image frame can be projected into the current image frame.
  • a feature point is projected into the current image frame After the distance between the feature points corresponding to the current image frame is less than the threshold, it is determined as an interior point; otherwise, the feature point is an exterior point; judge whether the number of interior points in the current image frame is greater than the preset number threshold, if it is greater than or If it is equal to the preset number threshold, it means that the Motion-only BA is successful, that is, the vision has not failed; if it is less than the preset number threshold, it means that the Motion-only BA has failed, that is, the vision is invalid.
  • the pose information of the current image frame relative to the previous image frame is obtained, and the movement track of the movable platform can be tracked according to the relative pose.
  • the camera when the camera collects the image of the current image frame, the camera can compare the current image frame according to the pose of the camera in the previous image frame and the speed of the first scroll wheel and the speed of the second scroll wheel detected by the encoder. Predict the pose of the current image frame to obtain the predicted pose of the current image frame, and perform feature point matching according to the predicted pose of the current image frame, the image of the current image frame, and the key frame with the same feature point as the current image frame. The predicted pose of the current image frame is optimized, and the optimized pose of the current image frame is obtained.
  • the vision fails temporarily, that is, the current image cannot be passed
  • the predicted pose of the current image frame can be used as the pose information of the current image frame to ensure the subsequent process Continue to avoid the inability to continue working due to the failure of the map point and the track before and after the visual failure.
  • the encoder Since the predicted pose of the current image frame is obtained based on the encoder, and the encoder does not have the characteristic of zero drift, it can ensure that the accumulated error in a short period of time will not be large, which can solve the transition from the short visual failure to the visual recovery period. Problems, thereby improving the stability and robustness of visual SLAM.
  • the local mapping thread S12 can be performed according to the image frame and the corresponding pose information. Specifically, it can first determine whether the current image frame is a key frame. If it is determined to be a key frame, the feature points of the key frame can be matched at the back end to add new map points to realize the construction of a local map and the optimization of the global map. .
  • the visual SLAM method obtaineds camera images and encoder data; obtains camera pose information according to the image frames and the measurement data of the encoder; according to the image frames and the pose information Carry out the construction of the map.
  • the encoder By applying the encoder to the visual SLAM, the stability and robustness of the visual SLAM can be improved, and the failure of the map points and trajectories before and after the visual failure when the visual failure is short-lived can be avoided and the work cannot be continued.
  • the local mapping thread S12 after obtaining the pose of the current image frame, it may further include:
  • Step S301 Determine whether the current image frame is a key frame
  • the current image frame is added to the local mapping thread S12 to construct the map. Specifically, it may include:
  • Step S302 When it is determined that the current image frame is a key frame, on the basis of the constructed map, the constructed map is updated according to the current image frame;
  • the constructed map is constructed by key frames determined before the current image frame.
  • map points that are close to each other can also be directly added. Insert into the map as a new map point.
  • the current image frame will be used as the new key frame; or when the matching number of feature points of the current image frame is greater than the preset matching number threshold and sufficient new map points are provided, the current image frame will also be used as the new key frame.
  • the number of feature point matches refers to the number of matches between the feature points of the current image frame and the map points in the map, or the number of matches between the feature points of the current image frame and the feature points of the key frame that has the same feature points as the current image frame;
  • the time interval between the current image frame and the previous key frame must not be too long, that is, the time interval between the current image frame and the previous key frame exceeds the preset time interval When thresholding, the current image frame must be used as the new key frame.
  • the key frame judgment strategy in the prior art can be used; and when the vision fails, since the predicted pose of the current image frame can be obtained through the encoder data, two frames are allowed It is connected through the encoder (that is, pure encoder edges are allowed), it is not necessary that the distance between the current image frame and the previous key frame must be greater than the minimum value in the preset distance range, and the current image frame is also not required
  • the number of feature points matching is greater than the preset matching number threshold, so the above two restrictions can be removed; but for RGBD cameras or binocular cameras, the minimum number of map points created by the current image frame needs to be increased, that is, the current image frame creates map points
  • the number needs to be greater than the preset number of map points threshold to prevent too few contributed map points and lose the meaning of joining; for the case where the monocular camera is tightly coupled with the IMU, add a minimum time interval limit to tentatively build the map thread In (Partial Mapping), some map points are constructed by triangulation to restore visual tracking, that
  • the key frame judgment strategy when the vision fails can be restored to the key frame judgment strategy in the prior art.
  • updating the constructed map according to the current image frame includes:
  • Step S401 Take out redundant key frames and redundant map points according to a preset strategy, and perform a triangulation operation according to the current image frame to generate new map points;
  • Step S402 After generating a new map point, construct a common view of the current image frame and perform local beam adjustment to adjust other key frames that are in common view with the current image frame, the map points that can be seen in the current image frame, and The map points that can be seen in the other key frames are optimized; wherein the local beam adjustment includes an encoder error term.
  • the redundant map points can be removed first.
  • the redundant map points can include the map points that are not seen near the current key frame and the previous few frames. For map points with a small ratio of times that are continuously viewed, for feature points in the current image frame that can be fused with existing map points, the existing map points can be replaced by fusion.
  • redundant key frames can also be removed.
  • Redundant key frames refer to the existence of at least one other key frame and the number of same map points that it sees exceeds the threshold (for example, 90 %), in this embodiment, it is determined whether those key frames (frame nodes in the common view) that can see the same map point as the current key frame are redundant.
  • the constructing the common view of the current image frame to perform local beam adjustment includes:
  • Step S501 When any key frame and the previous key frame are connected by the pure encoder side, and the key frame and the next key frame are also connected by the pure encoder side, the key frame is removed, according to the The time stamp of the previous key frame and the time stamp of the next key frame acquire the second time period; wherein the pure encoder side is that the visual failure occurs between adjacent key frames;
  • Step S502 Obtain all measurement data of the first encoder and all measurement data of the second encoder in the second time period;
  • Step S503 Obtain a prediction transformation matrix of the next key frame relative to the previous key frame according to all the measurement data measured by the first encoder and all the measurement data of the second encoder in the second time period.
  • the continuous key frames connected by the pure encoder side can be removed, and the key frames connected by the continuous pure encoder side, such as "key frame 1-encoder side 1-key frame 2" -Encoder side 2-key frame 3",
  • the relative pose of key frame 2 to key frame 1 is the predicted pose obtained from the encoder data (ie encoder side 1)
  • key frame 3 is relative to key frame 2
  • the relative pose of is also the predicted pose obtained from the encoder data (that is, encoder side 2).
  • Removing the continuous pure encoder side can reduce the amount of calculation for subsequent optimization and prevent only the encoder data from being used for triangulation.
  • the error is large (the accuracy of the encoder and the friction with the ground cause the encoder to have lower accuracy than the vision).
  • the key frame 2 connected by the continuous pure encoder side can be deleted to obtain "key frame 1-encoder side-key frame 3" , Where the encoder side needs to be recalculated, that is, the predicted pose of key frame 3 relative to key frame 1 is calculated through the pre-integration process in the foregoing embodiment.
  • a new map point can be created by triangulation, where triangulation is based on two points (the relative pose of the key frame) and the ray passing them (the pixel coordinates of the matching point pair, that is, the direction) to obtain the ray The position of the intersection point relative to these two points (the depth or position of the matching point pair or 3D point).
  • the 3D point X is observed by the left and right cameras of the binocular camera, and the imaging position of the 3D point on the left camera is x.
  • the imaging position of the right camera is x'.
  • the images collected by the left and right cameras of the binocular camera can be replaced by two adjacent frames (the relative pose between the two frames is known).
  • the two adjacent frames simultaneously observe a certain fixed in space.
  • the triangulation process of 3D point X is the same as that of a binocular camera, so I won't repeat it here.
  • the image collected by the RGBD camera is equivalent to the image collected by the binocular camera, so the same triangulation process as the binocular camera can be used , I won’t repeat it here.
  • the key frame pose and map points can be optimized by local BA, where the local BA process is similar to the Motion-only BA optimization process in the foregoing embodiment, and the objective function of the local BA also adds an encoder error term.
  • the key frames participating in the local BA are usually the common view of the current key frame (that is, the picture contains two types of vertices, the frame and the map point, and the reprojection between the frame and the map point it can see.
  • these key frames can see the map points seen by the current key frame, and the map points in the figure are all map points that can be seen by these key frames), and the key frame participating in the local BA in this embodiment is changed to the current
  • the image frame is pushed forward by N consecutive key frames (N is greater than or equal to 1), and the objective function of the local BA can be constructed through the N consecutive key frames, so as to optimize the pose and map points of the current key frame, which can reduce Small amount of calculation.
  • the current key frame is sent to the loop detection thread S13 to perform closed loop detection to determine whether the movable platform is located at the previously reached position.
  • Closed-loop detection in the prior art query the map database, calculate the similar transformation (SE3 or Sim3) between the current key frame and the closed-loop key frame through the RANSAC (Random Sample Consensus) algorithm, and merge the current key frame through closed-loop fusion And the map point seen by the key frame near the closed-loop key frame.
  • the RANSAC algorithm uses several pairs of matching points to calculate the transformation randomly, uses the theory of probability statistics to find the most correct transformation, and distinguishes which matching point pairs are correct (the matching point pairs are correct, then called interior points).
  • the bag-of-words (BoW) vector of each key frame can be obtained to obtain the distance between the bag-of-words vector of any key frame and the bag-of-words vector of the current key frame (for example, Euclidean distance), judge whether there is a closed loop based on the distance between two bag-of-words vectors, where the bag-of-words vector is obtained based on the feature points of the key frame and the preset feature point bag of words, and the preset feature point bag of words is obtained by the feature It is constructed by clustering, which contains the feature point information arranged in a predetermined order, and the key frame feature point is matched with each feature point information in the preset feature point word bag.
  • the feature point is set to 1 if it appears in the key frame, and 0 if it does not appear, so that the bag of words vector of the key frame is obtained.
  • the bag-of-words vector of each key frame is stored in the database.
  • the distance between the bag-of-words vector of the current key frame and the bag-of-words vector of each key frame in the database can be used to judge whether the current key frame has a closed loop.
  • the key frame of is the closed-loop key frame of the current image frame.
  • the threshold for detecting the closed loop can be lowered after the visual failure.
  • the threshold for detecting the closed loop includes the number of repeated detections, the threshold for calculating the number of matching points for similar transformations, and the number of internal points.
  • the pose graph can be optimized.
  • the pose graph is a graph in which the vertices do not contain map points, only contain all the key frames, and the edges contain only the relative transformations between the frames on the supporting tree and the similar transformations corresponding to all closed-loop edges.
  • the spanning tree is the connection A loop-free graph of key frames in the entire map.
  • two consecutive key frames in time will become parent and child nodes, but when a parent node is deleted, the parent node of the child node will be modified according to the number of map points seen in common (this time the corresponding removal The operation of the key frame will update the pre-integration of the child nodes), and the root node is the initial key frame when the visual SLAM system is created.
  • the pose graph optimizes the pose of each key frame, and the objective function is relative transformation And vertex pose error About the covariance matrix (Originally a 3 ⁇ 3 identity matrix I) the sum of squares.
  • a modified covariance matrix is generated according to the predictive transformation matrix to modify the covariance matrix of the relative transformation between adjacent key frames.
  • the covariance matrix can be obtained by pre-integration To modify the corresponding relative transformation of the pure encoder side on the support tree Covariance matrix
  • the diagonal element is generally greater than 1, which means that the accuracy of the pure encoder edge is not as good as the encoder and the visually tightly coupled edge, which can reduce the error introduced by the pure encoder edge during nonlinear optimization.
  • the error introduced by the pure encoder side can be dispersed to all sides, for example, on the support tree, key frame 1-key frame 2-key frame 3, when the closed loop of key frame 3 and key 1 occurs , A closed-loop edge is formed between key frame 1 and key frame 3 (similar transformation SE3 or Sim3 of current key frame and closed-loop key frame), which can disperse the cumulative error of key frame 1 and key frame 3 to key frame 1-key
  • the error of each side is relatively small and within an acceptable range.
  • global BA optimization can also be performed.
  • the global BA is similar to the Motion-only BA and local BA optimization process in the above embodiment.
  • the objective function of the global BA also adds the encoder error term, and the difference from the local BA optimization process is that the key frames participating in the global BA are All key frames and map points in the entire map library. After completing the global BA, you can update the map, update all map points, update all key frames in the map library until the latest key frame.
  • the IMU needs to be initialized to obtain the bias of the IMU, including the bias of the gyroscope in the IMU and the bias of the accelerometer. Gravity vector can also be obtained.
  • the map scale needs to be determined through IMU initialization, while binocular cameras and RGBD cameras usually do not need to be initialized to determine the map scale through IMU initialization. When the binocular camera and RGBD camera determine the map scale, the initialization result is close to 1.
  • the first predetermined number of key frames for example, 20
  • the zero offset, gravity vector, and map scale of the IMU can be obtained according to these key frames
  • the map scale Update to all the key frames, including the predetermined number of key frames and the new key frames obtained in the initialization process, and then perform global BA optimization to update the map scale, and then the visual SLAM can run at the updated map scale.
  • this embodiment also provides a lightweight positioning method.
  • the simplest saving method can be adopted, that is, only the key frames and map point information of the necessary map are saved, and binary file storage can be adopted.
  • the main storage content involves sensor type; key frame sequence number, pose, bag-of-words vector, feature point sequence and inter-frame sensor information sequence, sequence number information of map points that can be observed by the frame, and sequence number information of closed-loop key frame connected to the frame; map Point's serial number, position, reference key frame serial number, key frame serial number information that the map point can be observed; supporting tree structure (namely parent node serial number)
  • the sequence number for creating the key frame and the sequence number for the map point may not be the original sequence number.
  • the map information of the long-running visual SLAM system can be stored in a small storage space, which can increase the maximum working time of the visual SLAM system, but it takes more time to reconstruct the map and trajectory when reloading. time.
  • the visual SLAM can only start the tracking thread S11 for positioning, and does not need to start the local mapping thread S12 and the closed loop thread to achieve lightweight positioning. It should be noted that for the case where the monocular camera and the IMU are tightly coupled, the IMU still needs to be initialized (to obtain the initial zero bias and gravitational acceleration).
  • FIG. 7 is a structural diagram of a movable platform provided by an embodiment of the present invention.
  • the movable platform 700 includes a first scroll wheel 701, a second scroll wheel 702, a first encoder 703, and a second encoder 704, camera 705, memory 707, and processor 706;
  • the first rolling wheel 701 and the second rolling wheel 702 are used for displacing the movable platform, and the axes of the first rolling wheel 701 and the second rolling wheel 702 coincide;
  • the camera 705 is used to collect image frames
  • the first encoder 703 is used to detect the speed of the first scroll wheel 701
  • the second encoder 704 is used to detect the speed of the second scroll wheel 702;
  • the memory 707 is used to store program codes
  • the processor 706 calls the program code, and when the program code is executed, is configured to perform the following operations:
  • the processor 706 obtains the prediction of the current image frame relative to the previous image frame according to the measurement data of the first encoder and the measurement data of the second encoder in the first time period.
  • the processor is configured to:
  • the measurement data of the first encoder and the measurement data of the second encoder in the first time period obtain the first derivative of the rotation matrix of the current image frame with respect to time, and the second derivative of the position of the current image frame with respect to time ;
  • the predicted rotation matrix and predicted translation matrix of the current image frame relative to the previous image frame is obtained.
  • the processor 706 After the processor 706 obtains the predicted pose of the current image frame according to the predicted change matrix and the pose information of the previous image frame, the processor 706 is also Configured as:
  • the prediction pose of the current image frame is optimized according to a preset pure motion beam adjustment model; wherein, the pure motion beam adjustment model includes an encoder error term, and the encoder error term passes through the first encoder
  • the pure motion beam adjustment model includes an encoder error term, and the encoder error term passes through the first encoder
  • processor 706 is further configured to:
  • the pose of the current image frame is the predicted pose
  • the pose of the current image frame is the optimization result of the pure motion beam adjustment model.
  • processor 706 is further configured to:
  • processor 706 is further configured to:
  • the constructed map is updated according to the current image frame; the constructed map is determined by the key frame determined before the current image frame Build.
  • the processor 706 when the processor 706 updates the constructed map according to the current image frame, the processor 706 is configured to:
  • the processor 706 constructs the common view of the current image frame to perform local beam adjustment
  • the processor 706 is configured to:
  • the prediction transformation matrix of the next key frame relative to the previous key frame is obtained.
  • the processor 706 when the processor 706 constructs the common view of the current image frame to perform local beam adjustment, the processor 706 is further configured to:
  • N is greater than or equal to 1;
  • the encoder error and the reprojection error optimize the current image frame and the map points it can see, the N consecutive key frames and the map points it can see, and update all the map points in the local optimization map.
  • the pose of the key frame and the position of all map points are the encoder error and the reprojection error.
  • processor 706 is further configured to:
  • the processor 706 is further configured to:
  • the pose map is constructed to optimize all the key frames to update the poses of all the key frames; the pose map includes all the key frames and the relative transformation between the two key frames.
  • the processor 706 when the processor 706 constructs a pose graph to optimize all key frames, the processor 706 is configured to:
  • a modified covariance matrix is generated according to the predictive transformation matrix to modify the covariance matrix of the relative transformation between the adjacent key frames.
  • the processor 706 is further configured to:
  • processor 706 calls the program code, and when the program code is executed, is configured to perform the following operations:
  • the constructed map After obtaining the predicted pose of the current image frame, on the basis of the constructed map, the constructed map is updated according to the current image frame; the constructed map is determined before the current image frame The key frames are constructed.
  • the mobile platform provided in this embodiment obtains the first time period according to the time stamp of the current image frame and the time stamp of the previous image frame; obtains the measurement data and the second code of the first encoder in the first time period Measured measurement data; According to the measurement data of the first encoder and the measurement data of the second encoder in the first time period, obtain the prediction transformation matrix of the current image frame relative to the previous image frame; According to the predicted change The matrix and the pose information of the previous image frame obtain the predicted pose of the current image frame.
  • the encoder is applied to the visual SLAM, and the pose of the current image frame is predicted through the encoder measurement data, which can improve the accuracy of the pose acquisition, improve the stability and robustness of the visual SLAM, and when the vision fails Visual SLAM can still run stably.
  • this embodiment also provides a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to implement the pose prediction method and/or map construction method described in the foregoing embodiments.
  • this embodiment also provides a computer program, including program code.
  • the program code executes the pose prediction method and/or map construction method as described in the foregoing embodiment.
  • the disclosed device and method can be implemented in other ways.
  • the device embodiments described above are merely illustrative, for example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or It can be integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional units in the various embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit may be implemented in the form of hardware, or may be implemented in the form of hardware plus software functional units.
  • the above-mentioned integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium.
  • the above-mentioned software functional unit is stored in a storage medium, and includes several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor execute the method described in each embodiment of the present invention. Part of the steps.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program code .

Landscapes

  • Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Electromagnetism (AREA)
  • Automation & Control Theory (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

L'invention concerne un procédé de prévision de pose, un procédé de construction de carte, une plate-forme mobile, et un support de stockage. Le procédé consiste à : acquérir une première période de temps en fonction d'une estampille temporelle de la trame d'image actuelle et d'une estampille temporelle d'une trame d'image précédente (S101) ; acquérir des données de mesure d'un premier codeur et des données de mesure d'un second codeur dans la première période de temps (S102) ; obtenir une matrice de transformation de prévision de la trame d'image actuelle par rapport à la trame d'image précédente en fonction des données de mesure du premier codeur et des données de mesure du second codeur dans la première période de temps (S103) ; et obtenir une pose prévue de la trame d'image actuelle en fonction de la matrice de transformation de prévision et des informations de pose de la trame d'image précédente (S104). En appliquant des codeurs sur le SLAM visuel, et en prévoyant la pose de la trame d'image actuelle au moyen des données de mesure des codeurs, la précision de l'acquisition de pose peut être améliorée, la stabilité et la robustesse du SLAM visuel sont améliorées, et le SLAM visuel peut toujours fonctionner de façon stable pendant une défaillance visuelle.
PCT/CN2019/103599 2019-08-30 2019-08-30 Procédé de prévision de pose, procédé de construction de carte, plateforme mobile et support de stockage WO2021035669A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201980033455.4A CN112219087A (zh) 2019-08-30 2019-08-30 位姿预测方法、地图构建方法、可移动平台及存储介质
PCT/CN2019/103599 WO2021035669A1 (fr) 2019-08-30 2019-08-30 Procédé de prévision de pose, procédé de construction de carte, plateforme mobile et support de stockage

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/103599 WO2021035669A1 (fr) 2019-08-30 2019-08-30 Procédé de prévision de pose, procédé de construction de carte, plateforme mobile et support de stockage

Publications (1)

Publication Number Publication Date
WO2021035669A1 true WO2021035669A1 (fr) 2021-03-04

Family

ID=74058715

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/103599 WO2021035669A1 (fr) 2019-08-30 2019-08-30 Procédé de prévision de pose, procédé de construction de carte, plateforme mobile et support de stockage

Country Status (2)

Country Link
CN (1) CN112219087A (fr)
WO (1) WO2021035669A1 (fr)

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112819744A (zh) * 2021-02-26 2021-05-18 中国人民解放军93114部队 Gnss和视觉slam融合的轨迹测量方法和装置
CN113379803A (zh) * 2021-07-07 2021-09-10 上海谦尊升网络科技有限公司 一种基于视觉图像的定位方法
CN113516692A (zh) * 2021-05-18 2021-10-19 上海汽车集团股份有限公司 一种多传感器融合的slam方法和装置
CN113587934A (zh) * 2021-07-30 2021-11-02 深圳市普渡科技有限公司 一种机器人、室内定位方法、装置和可读存储介质
CN113763548A (zh) * 2021-08-17 2021-12-07 同济大学 基于视觉-激光雷达耦合的贫纹理隧洞建模方法及***
CN113763560A (zh) * 2021-08-02 2021-12-07 纵目科技（上海）股份有限公司 点云数据的生成方法、***、设备及计算机可读存储介质
CN113781563A (zh) * 2021-09-14 2021-12-10 中国民航大学 一种基于深度学习的移动机器人回环检测方法
CN113850293A (zh) * 2021-08-20 2021-12-28 北京大学 基于多源数据和方向先验联合优化的定位方法
CN114088081A (zh) * 2021-10-10 2022-02-25 北京工业大学 一种基于多段联合优化的用于精确定位的地图构建方法
CN114255323A (zh) * 2021-12-22 2022-03-29 深圳市普渡科技有限公司 机器人、地图构建方法、装置和可读存储介质
CN114279456A (zh) * 2021-12-06 2022-04-05 纵目科技（上海）股份有限公司 图片构建/车辆定位方法、***、终端及计算机存储介质
CN115930971A (zh) * 2023-02-01 2023-04-07 七腾机器人有限公司 一种机器人定位与建图的数据融合处理方法
CN116252581A (zh) * 2023-03-15 2023-06-13 吉林大学 直线行驶工况车身垂向及俯仰运动信息估算***及方法
WO2024002065A1 (fr) * 2022-06-30 2024-01-04 维沃移动通信有限公司 Procédé et appareil de codage vidéo, dispositif électronique et support

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112991515B (zh) * 2021-02-26 2022-08-19 山东英信计算机技术有限公司 一种三维重建方法、装置及相关设备
CN113052855B (zh) * 2021-02-26 2021-11-02 苏州迈思捷智能科技有限公司 一种基于视觉-imu-轮速计融合的语义slam方法
CN113223161B (zh) * 2021-04-07 2022-04-12 武汉大学 一种基于imu和轮速计紧耦合的鲁棒全景slam***和方法
CN113094462B (zh) * 2021-04-30 2023-10-24 腾讯科技（深圳）有限公司 数据处理方法和装置及存储介质
CN113658260A (zh) * 2021-07-12 2021-11-16 南方科技大学 机器人位姿计算方法、***、机器人及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106679648A (zh) * 2016-12-08 2017-05-17 东南大学 一种基于遗传算法的视觉惯性组合的slam方法
CN106997688A (zh) * 2017-06-08 2017-08-01 重庆大学 基于多传感器信息融合的停车场停车位检测方法
US20170337690A1 (en) * 2016-05-20 2017-11-23 Qualcomm Incorporated Predictor-corrector based pose detection
CN109583290A (zh) * 2017-09-28 2019-04-05 百度（美国）有限责任公司 使用运动相关数据改进视觉特征检测的***和方法
CN110160527A (zh) * 2019-05-06 2019-08-23 安徽红蝠智能科技有限公司 一种移动机器人导航方法与装置

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104374395A (zh) * 2014-03-31 2015-02-25 南京邮电大学 基于图的视觉slam方法
CN108256574B (zh) * 2018-01-16 2020-08-11 广东省智能制造研究所 机器人定位方法及装置
CN108921898B (zh) * 2018-06-28 2021-08-10 北京旷视科技有限公司 摄像机位姿确定方法、装置、电子设备和计算机可读介质
CN109671120A (zh) * 2018-11-08 2019-04-23 南京华捷艾米软件科技有限公司 一种基于轮式编码器的单目slam初始化方法及***
CN110044354B (zh) * 2019-03-28 2022-05-20 东南大学 一种双目视觉室内定位与建图方法及装置
CN110706248B (zh) * 2019-08-20 2024-03-12 广东工业大学 一种基于slam的视觉感知建图方法及移动机器人
CN115131420A (zh) * 2022-06-24 2022-09-30 武汉依迅北斗时空技术股份有限公司 基于关键帧优化的视觉slam方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170337690A1 (en) * 2016-05-20 2017-11-23 Qualcomm Incorporated Predictor-corrector based pose detection
CN106679648A (zh) * 2016-12-08 2017-05-17 东南大学 一种基于遗传算法的视觉惯性组合的slam方法
CN106997688A (zh) * 2017-06-08 2017-08-01 重庆大学 基于多传感器信息融合的停车场停车位检测方法
CN109583290A (zh) * 2017-09-28 2019-04-05 百度（美国）有限责任公司 使用运动相关数据改进视觉特征检测的***和方法
CN110160527A (zh) * 2019-05-06 2019-08-23 安徽红蝠智能科技有限公司 一种移动机器人导航方法与装置

Cited By (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112819744A (zh) * 2021-02-26 2021-05-18 中国人民解放军93114部队 Gnss和视觉slam融合的轨迹测量方法和装置
CN112819744B (zh) * 2021-02-26 2024-05-14 中国人民解放军93114部队 Gnss和视觉slam融合的轨迹测量方法和装置
CN113516692A (zh) * 2021-05-18 2021-10-19 上海汽车集团股份有限公司 一种多传感器融合的slam方法和装置
CN113379803B (zh) * 2021-07-07 2024-02-02 上海谦尊升网络科技有限公司 一种基于视觉图像的定位方法
CN113379803A (zh) * 2021-07-07 2021-09-10 上海谦尊升网络科技有限公司 一种基于视觉图像的定位方法
CN113587934A (zh) * 2021-07-30 2021-11-02 深圳市普渡科技有限公司 一种机器人、室内定位方法、装置和可读存储介质
CN113587934B (zh) * 2021-07-30 2024-03-19 深圳市普渡科技有限公司 一种机器人、室内定位方法、装置和可读存储介质
CN113763560A (zh) * 2021-08-02 2021-12-07 纵目科技（上海）股份有限公司 点云数据的生成方法、***、设备及计算机可读存储介质
CN113763560B (zh) * 2021-08-02 2024-02-09 纵目科技（上海）股份有限公司 点云数据的生成方法、***、设备及计算机可读存储介质
CN113763548A (zh) * 2021-08-17 2021-12-07 同济大学 基于视觉-激光雷达耦合的贫纹理隧洞建模方法及***
CN113763548B (zh) * 2021-08-17 2024-02-27 同济大学 基于视觉-激光雷达耦合的贫纹理隧洞建模方法及***
CN113850293A (zh) * 2021-08-20 2021-12-28 北京大学 基于多源数据和方向先验联合优化的定位方法
CN113781563A (zh) * 2021-09-14 2021-12-10 中国民航大学 一种基于深度学习的移动机器人回环检测方法
CN113781563B (zh) * 2021-09-14 2023-10-24 中国民航大学 一种基于深度学习的移动机器人回环检测方法
CN114088081A (zh) * 2021-10-10 2022-02-25 北京工业大学 一种基于多段联合优化的用于精确定位的地图构建方法
CN114088081B (zh) * 2021-10-10 2024-05-28 北京工业大学 一种基于多段联合优化的用于精确定位的地图构建方法
CN114279456A (zh) * 2021-12-06 2022-04-05 纵目科技（上海）股份有限公司 图片构建/车辆定位方法、***、终端及计算机存储介质
CN114279456B (zh) * 2021-12-06 2024-04-30 纵目科技（上海）股份有限公司 图片构建/车辆定位方法、***、终端及计算机存储介质
CN114255323A (zh) * 2021-12-22 2022-03-29 深圳市普渡科技有限公司 机器人、地图构建方法、装置和可读存储介质
WO2024002065A1 (fr) * 2022-06-30 2024-01-04 维沃移动通信有限公司 Procédé et appareil de codage vidéo, dispositif électronique et support
CN115930971B (zh) * 2023-02-01 2023-09-19 七腾机器人有限公司 一种机器人定位与建图的数据融合处理方法
CN115930971A (zh) * 2023-02-01 2023-04-07 七腾机器人有限公司 一种机器人定位与建图的数据融合处理方法
CN116252581B (zh) * 2023-03-15 2024-01-16 吉林大学 直线行驶工况车身垂向及俯仰运动信息估算***及方法
CN116252581A (zh) * 2023-03-15 2023-06-13 吉林大学 直线行驶工况车身垂向及俯仰运动信息估算***及方法

Also Published As

Publication number Publication date
CN112219087A (zh) 2021-01-12

Similar Documents

Publication Publication Date Title
WO2021035669A1 (fr) Procédé de prévision de pose, procédé de construction de carte, plateforme mobile et support de stockage
CN111561923B (zh) 基于多传感器融合的slam制图方法、***
CN109307508B (zh) 一种基于多关键帧的全景惯导slam方法
US11668571B2 (en) Simultaneous localization and mapping (SLAM) using dual event cameras
CN109166149B (zh) 一种融合双目相机与imu的定位与三维线框结构重建方法与***
CN107990899B (zh) 一种基于slam的定位方法和***
CN109993113B (zh) 一种基于rgb-d和imu信息融合的位姿估计方法
CN112634451B (zh) 一种融合多传感器的室外大场景三维建图方法
KR102427921B1 (ko) 실시간 맵핑 및 로컬리제이션을 위한 장치 및 방법
CN109506642B (zh) 一种机器人多相机视觉惯性实时定位方法及装置
CN111210463B (zh) 基于特征点辅助匹配的虚拟宽视角视觉里程计方法及***
Tanskanen et al. Live metric 3D reconstruction on mobile phones
KR101725060B1 (ko) 그래디언트 기반 특징점을 이용한 이동 로봇의 위치를 인식하기 위한 장치 및 그 방법
CN112304307A (zh) 一种基于多传感器融合的定位方法、装置和存储介质
US9858640B1 (en) Device and method for merging 3D point clouds from sparsely distributed viewpoints
CN110726406A (zh) 一种改进的非线性优化单目惯导slam的方法
CN111127524A (zh) 一种轨迹跟踪与三维重建方法、***及装置
WO2020221307A1 (fr) Procédé et dispositif pour suivre un objet mobile
CN111932674A (zh) 一种线激光视觉惯性***的优化方法
CN112802096A (zh) 实时定位和建图的实现装置和方法
Zhang et al. Hand-held monocular SLAM based on line segments
CN114013449A (zh) 针对自动驾驶车辆的数据处理方法、装置和自动驾驶车辆
CN112541423A (zh) 一种同步定位与地图构建方法和***
CN112731503A (zh) 一种基于前端紧耦合的位姿估计方法及***
CN117029802A (zh) 一种基于深度学习的多模态slam方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19942755

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19942755

Country of ref document: EP

Kind code of ref document: A1