CN112381841A - Semantic SLAM method based on GMS feature matching in dynamic scene - Google Patents

Semantic SLAM method based on GMS feature matching in dynamic scene Download PDF

Info

Publication number
CN112381841A
CN112381841A CN202011365138.3A CN202011365138A CN112381841A CN 112381841 A CN112381841 A CN 112381841A CN 202011365138 A CN202011365138 A CN 202011365138A CN 112381841 A CN112381841 A CN 112381841A
Authority
CN
China
Prior art keywords
dynamic
point
matching
feature
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011365138.3A
Other languages
Chinese (zh)
Inventor
陈政
游林辉
胡峰
孙仝
张谨立
宋海龙
黄达文
***
梁铭聪
黄志就
何彧
陈景尚
谭子毅
潘嘉琪
李志鹏
罗鲜林
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhaoqing Power Supply Bureau of Guangdong Power Grid Co Ltd
Original Assignee
Zhaoqing Power Supply Bureau of Guangdong Power Grid Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhaoqing Power Supply Bureau of Guangdong Power Grid Co Ltd filed Critical Zhaoqing Power Supply Bureau of Guangdong Power Grid Co Ltd
Priority to CN202011365138.3A priority Critical patent/CN112381841A/en
Publication of CN112381841A publication Critical patent/CN112381841A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to a semantic SLAM method based on GMS feature matching in a dynamic scene, which comprises the steps of segmenting an input image through a semantic segmentation network to obtain mask codes of all objects, and removing the mask codes of dynamic objects to obtain a primary image with the dynamic objects removed; extracting ORB characteristic points from an input image, and then calculating a descriptor; detecting and eliminating dynamic feature points according to a method combining motion consistency and semantic information; the running precision and robustness of the visual SLAM system in a high-dynamic environment are improved by combining the motion consistency and the semantic information to remove the dynamic mode.

Description

Semantic SLAM method based on GMS feature matching in dynamic scene
Technical Field
The invention relates to the field of positioning and navigation based on vision in autonomous inspection of unmanned aerial vehicles, in particular to a semantic SLAM method based on GMS feature matching in a dynamic scene.
Background
The unmanned aerial vehicle intelligent inspection process requires the unmanned aerial vehicle to autonomously determine the next operation according to the real-time information of the current environment. Therefore, real-time positioning of the unmanned aerial vehicle and construction of a diagram of a working environment are important links in the intelligent inspection process of the unmanned aerial vehicle. Especially in the cooperative work of a plurality of unmanned aerial vehicles in a gridding arrangement, the environment detected by each unmanned aerial vehicle is a dynamic scene (including an occasional moving object), so a special algorithm needs to be developed for the dynamic scene in the positioning and environment mapping process.
Meanwhile, location and Mapping (SLAM) is a technology that can estimate the current position and attitude by a corresponding motion estimation algorithm through a sensor and establish a three-dimensional map of an environment without any environment prior information. With the development of computer vision and deep learning technology and the improvement of hardware computing capability, vision-based SLAM research is continuously deepened and widely applied to the fields of autonomous driving, mobile robots, unmanned aerial vehicles and the like.
Chinese patent application documents with the publication number of "CN 110322511A" and the publication date of 2019, 10 and 11 disclose a semantic SLAM method and a semantic SLAM system based on object and plane characteristics, wherein RGB-D image streams of a scene are obtained, and key frame images are obtained by utilizing the RGB-D image streams to perform frame-by-frame tracking; constructing a local map of a scene by using a key frame image, performing plane segmentation on a depth map of the key frame image to obtain a current plane, constructing a global plane map by using the current plane, performing object detection on the key frame image to obtain a detection frame and confidence, reconstructing point cloud of an object by using the detection frame and the confidence, merging feature points in the detection frame to the object to obtain a global object map; and performing loop detection by using the key frame image to obtain a loop frame, and performing loop correction to optimize plane constraint and object constraint by using the loop frame to obtain a plane map and an object map of the scene. The invention can improve SLAM optimization performance and enhance semantic description of the environment.
However, in the above method, the application scene is basically static, and the dynamic information in the environment is not negligible in the actual application process. The SLAM method lacks a mechanism for processing a dynamic scene, and a series of problems such as initialization failure, overlarge positioning error, map building failure and the like can be caused, so that the problems of low running precision and poor robustness of the SLAM system in the dynamic scene are caused.
Disclosure of Invention
In order to solve the problem that the SLAM method in the prior art is low in detection precision and robustness, the invention provides a semantic SLAM method based on GMS feature matching in a dynamic scene, and the precision and robustness of the running of a visual SLAM system in a dynamic environment are improved by combining motion consistency and semantic information to remove the dynamic.
In order to solve the technical problems, the invention adopts the technical scheme that: a semantic SLAM method based on GMS feature matching in a dynamic scene comprises the following steps:
the method comprises the following steps: calibrating a camera, removing image distortion, acquiring and inputting an environment image;
step two: segmenting the input image through a semantic segmentation network to obtain mask codes of all objects, and removing the mask codes of the dynamic objects to obtain a primary image with the dynamic objects removed;
step three: extracting ORB characteristic points from an input image, and then calculating a descriptor;
step four: detecting and eliminating dynamic feature points according to a method combining motion consistency and semantic information;
step five: performing feature point matching by using a GMS algorithm to remove mismatching;
step six: obtaining a camera pose by tracking the thread;
step seven: performing point cloud processing through a local mapping process to obtain a sparse point cloud map;
step eight: and optimizing the pose by using loop detection and correcting drift errors.
Preferably, in the step one, calibrating the camera, and specifically removing image distortion comprises:
s1.1: first, the internal reference of the camera is obtained, wherein the internal reference comprises fx,fy,cx,cyNormalizing the three-dimensional coordinates (X, Y, Z) to homogeneous coordinates (X, Y);
s1.2: removing the effect of distortion on the image, where k1,k2,k3,p1,p2]For the distortion coefficient of the lens, r is the distance of the point from the origin of the coordinate system:
Figure BDA0002805185600000021
s1.3: and transferring the coordinates in the camera coordinate system to the pixel coordinate system:
Figure BDA0002805185600000022
preferably, the semantic segmentation network is a lightweight semantic segmentation network FcHarDnet. The model size is reduced by convolution of HDB block connection 1x1, and the network has image processing speed about 30% higher than other network structures under the same hardware environment. Potential dynamic regions of the image are segmented by FcHarDnet and a mask is generated.
Preferably, in the third step, a gaussian pyramid is constructed in the process of extracting ORB feature points from the input image, and feature point detection is performed on different layers of the pyramid, so that the characteristic of unchanged scale is achieved.
Preferably, the specific steps of ORB feature point extraction are as follows:
the specific steps of ORB feature point extraction are as follows:
s3.1: when the difference between the gray value i (x) of more than N points around a certain point P and the gray value i (P) of the point P is greater than the threshold epsilon, the point is considered as a target corner point, which is specifically represented as:
Figure BDA0002805185600000031
s3.2: to make the feature points hold the orientation unchanged, the centroid is calculated:
Figure BDA0002805185600000032
in the formula, m00，m10，m01Is the integral of the pixel gray scale of the area in the circle about the key point;
s3.3: by scaling the original image sequence by a certain ratio, the image pyramid is patterned. Extracting corresponding angular points from the images with different sizes at each level of the image pyramid;
s3.4, adopting a quadtree uniform algorithm to continuously subdivide the image downwards into nodes with the same four-division size, combining the nodes without the characteristic points, and increasing the nodes when the number of the characteristic points in the nodes is more than 1; and after the node distribution is finished, deleting redundant feature points in the child nodes.
Preferably, in the fourth step, the motion consistency is detected as: the static point can satisfy epipolar geometric constraint, if the matching point of the object static feature point re-projected on the reference frame is definitely located on the intersection line of the reference frame and the epipolar plane; the method comprises the following specific steps:
S4.1：p1，p2is the homogeneous coordinates of the matching points in the current frame and the reference frame, where u, v are the values in the image frame, specifically:
p1＝[u1,v1,1],p2＝[u2,v2,1]
s4.2: calculating polar lines, wherein F is a basic matrix which can be obtained by calculating eight pairs of characteristic points, and specifically comprises the following steps:
Figure BDA0002805185600000041
wherein [ X Y Z]TRepresenting an epipolar vector;
s4.3: calculating the distance from the matching point to the corresponding epipolar line, if the distance is static D approaching 0, and when D is greater than a threshold epsilon, the characteristic point is dynamic, specifically:
Figure BDA0002805185600000042
the semantic segmentation and the dynamic consistency check are carried out on the image through the steps, but whether the object is dynamic or not cannot be accurately judged from the segmentation result. Whether the characteristic points are dynamic or not can be detected through dynamic consistency, but accurate outline information of the object is not available. The dynamic point determination method comprises the following steps: if there are a sufficient number of dynamic points detected by motion consistency within the mask of the object semantic segmentation, all points of the object are considered dynamic, and all points within the whole object mask are eliminated.
Preferably, in the fifth step, the feature points of the dynamic points removed in the fourth step are subjected to feature matching; the Grid Motion Statistics (GMS) algorithm proposes an assumption based on the smoothness of the motion: the smoothness of the motion causes a similar region to appear around the match, where the position of such region on both graphs moves smoothly in a true match and not smoothly in a false match. That is, there are other correct matches around the correct match that support the match, while a false match is only an occasional case and so there is no or very little other match support around it.
The GMS algorithm eliminates the error matching and comprises the following specific steps:
s5.1: the image is divided into 20 × 20 non-repetitive grids, and in order to solve the problem that the features are positioned at the boundaries of the grids, the length and the width of the grids can be adjusted to perform iterative computation.
S5.2: dividing 3 x 3 pixels around each feature point into a unit, and calculating the sum S of the total matching number of each unit { i, j } and the corresponding neighbor (9 grids) of the reference frame in the neighborij。
Figure BDA0002805185600000043
Wherein the content of the first and second substances,
Figure BDA0002805185600000051
representing the number of matched feature points on the corresponding grid pair;
s5.3: calculating a threshold for dividing each cell correct and mismatch, where niI.e. the average number of feature points in each grid, where α is an empirical value of 5, and the calculation formula is as follows:
Figure BDA0002805185600000052
s5.4: and repeating the steps S5.2 and S5.3 until the image traversal is completed to obtain the correct matching with the reference frame.
Preferably, in the sixth step, the tracking thread specifically includes: and estimating the pose of the current frame according to the motion model in the step five, tracking the map point of the previous frame according to the uniform motion model, and determining the pose.
Preferably, point cloud processing is carried out through a local mapping process, and a sparse point cloud map is obtained through local BA optimization.
Preferably, in the step eight, the bag-of-words model is used to judge whether a loop is generated, and if so, the accumulated error is corrected through the loop to optimize the pose.
Compared with the prior art, the invention has the beneficial effects that:
1) according to the invention, the running precision and robustness of the visual SLAM system in a high-dynamic environment are improved by combining the motion consistency and the semantic information to remove the dynamic mode.
2) The invention mainly aims at the problem of feature matching and provides a method based on grid and with motion statistical characteristics, and the method can quickly eliminate wrong matching so as to improve the stability of matching.
Drawings
Fig. 1 is a flowchart of a semantic SLAM method based on GMS feature matching in a dynamic scenario according to the present invention.
Detailed Description
The drawings are for illustrative purposes only and are not to be construed as limiting the patent; for the purpose of better illustrating the embodiments, certain features of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product; it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted. The positional relationships depicted in the drawings are for illustrative purposes only and are not to be construed as limiting the present patent.
The same or similar reference numerals in the drawings of the embodiments of the present invention correspond to the same or similar components; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "long", "short", etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the drawings, it is only for convenience of description and simplicity of description, but does not indicate or imply that the device or element referred to must have a specific orientation, be constructed in a specific orientation, and be operated, and therefore, the terms describing the positional relationships in the drawings are only used for illustrative purposes and are not to be construed as limitations of the present patent, and specific meanings of the terms may be understood by those skilled in the art according to specific situations.
The technical scheme of the invention is further described in detail by the following specific embodiments in combination with the attached drawings:
examples
Fig. 1 shows an embodiment of a semantic SLAM method based on GMS feature matching in a dynamic scenario, which includes the following steps:
the method comprises the following steps: calibrating a camera and removing image distortion; acquiring and inputting an environment image; calibrating a camera, and specifically removing image distortion by the following steps:
s1.1: first, the internal reference of the camera is obtained, wherein the internal reference comprises fx,fy,cx,cyNormalizing the three-dimensional coordinates (X, Y, Z) to homogeneous coordinates (X, Y);
s1.2: removing the effect of distortion on the image, where k1,k2,k3,p1,p2]For the distortion coefficient of the lens, r is the distance of the point from the origin of the coordinate system:
Figure BDA0002805185600000061
s1.3: and transferring the coordinates in the camera coordinate system to the pixel coordinate system:
Figure BDA0002805185600000062
step two: segmenting the input image through a semantic segmentation network to obtain mask codes of all objects, and realizing preliminary dynamic segmentation; the semantic segmentation network is a lightweight semantic segmentation network FcHarDnet. The model size is reduced by convolution of HDB block connection 1x1, and the network has image processing speed about 30% higher than other network structures under the same hardware environment. Potential dynamic regions of the image are segmented by FcHarDnet and a mask is generated.
Step three: extracting ORB characteristic points from an input image, and then calculating a descriptor; the specific steps of ORB feature point extraction are as follows:
s3.1: when the gray value difference between a certain number of points around a certain point P and the gray value of the point P is large, the point is regarded as a target corner point, and the specific expression is as follows:
Figure BDA0002805185600000071
s3.2: to make the feature points hold the orientation unchanged, the centroid is calculated:
Figure BDA0002805185600000072
s3.3: by scaling the original image sequence by a certain ratio, the image pyramid is patterned. Extracting corresponding angular points from the images with different sizes at each level of the image pyramid;
s3.4, adopting a quadtree uniform algorithm to continuously subdivide the image downwards into nodes with the same four-division size, combining the nodes without the characteristic points, and increasing the nodes when the number of the characteristic points in the nodes is more than 1; and after the node distribution is finished, deleting redundant feature points in the child nodes.
Step four: detecting and eliminating dynamic feature points according to a method combining motion consistency and semantic information; the motion consistency is detected as: the static point can satisfy epipolar geometric constraint, if the matching point of the object static feature point re-projected on the reference frame is definitely located on the intersection line of the reference frame and the epipolar plane; the method comprises the following specific steps:
S4.1：p1，p2is the homogeneous coordinates of the matching points in the current frame and the reference frame, where u, v are the values in the image frame, specifically:
p1＝[u1,v1,1],p2＝[u2,v2,1]
s4.2: calculating polar lines, wherein F is a basic matrix which can be obtained by calculating eight pairs of characteristic points, and specifically comprises the following steps:
Figure BDA0002805185600000073
s4.3: calculating the distance from the matching point to the corresponding epipolar line, if the distance is static D approaching 0, and when D is greater than a threshold epsilon, the characteristic point is dynamic, specifically:
Figure BDA0002805185600000081
the dynamic point determination method comprises the following steps: if there are a sufficient number of dynamic points detected by motion consistency within the mask of the object semantic segmentation, all points of the object are considered dynamic, and all points within the whole object mask are eliminated.
And matching the feature points in the adjacent frames by a fast nearest neighbor method, calculating the Hamming distance between the feature points, matching by the similarity degree between the feature points, and removing the mismatched feature points by adopting a PROSAC algorithm.
Step five: performing feature point matching by using a GMS algorithm to remove mismatching; the GMS algorithm eliminates the error matching and comprises the following specific steps:
s5.1: the image is divided into 20 × 20 non-repetitive grids, and in order to solve the problem that the features are positioned at the boundaries of the grids, the length and the width of the grids can be adjusted to perform iterative computation.
S5.2: dividing 3 x 3 pixels around each feature point into a unit, and calculating the sum S of the total matching number of each unit { i, j } and the corresponding neighbor (9 grids) of the reference frame in the neighborij。
Figure BDA0002805185600000082
S5.3: calculating a threshold value forDivide correct and mismatch, where niI.e. the average number of feature points in each grid, alpha is an empirical value of 5. The calculation formula is as follows:
Figure BDA0002805185600000083
s5.4: and repeating the steps S5.2 and S5.3 until the image traversal is completed to obtain the correct matching with the reference frame.
Step six: and acquiring the pose of the camera through a tracking thread, estimating the pose of the current frame according to the motion model in the step five, tracking the map point of the previous frame according to the uniform motion model, and determining the pose.
Step seven: performing point cloud processing through a local mapping process, and obtaining a sparse point cloud map through local BA optimization;
step eight: and optimizing the pose by using loop detection and correcting drift errors. And judging whether a loop is generated or not by using the bag-of-words model, and if so, correcting the accumulated error by the loop to optimize the pose.
The beneficial effects of this example: 1) according to the invention, the running precision and robustness of the visual SLAM system in a high-dynamic environment are improved by combining the motion consistency and the semantic information to remove the dynamic mode.
2) The invention mainly aims at the problem of feature matching and provides a method based on grid and with motion statistical characteristics, and the method can quickly eliminate wrong matching so as to improve the stability of matching.
It should be understood that the above-described embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the claims of the present invention.

Claims (10)

1. A semantic SLAM method based on GMS feature matching in a dynamic scene is characterized by comprising the following steps:
the method comprises the following steps: calibrating a camera and removing image distortion; acquiring and inputting an environment image;
step two: segmenting the input image through a semantic segmentation network to obtain mask codes of all objects, and removing the mask codes of the dynamic objects to obtain a primary image with the dynamic objects removed;
step three: extracting ORB characteristic points from an input image, and then calculating a descriptor;
step four: detecting and eliminating dynamic feature points according to a method combining motion consistency and semantic information;
step five: performing feature point matching by using a GMS algorithm to remove mismatching;
step six: obtaining a camera pose by tracking the thread;
step seven: performing point cloud processing through a local mapping process to obtain a sparse point cloud map;
step eight: and optimizing the pose by using loop detection and correcting drift errors.
2. The semantic SLAM method based on GMS feature matching in a dynamic scene, as claimed in claim 1, wherein in said step one, calibrating the camera, and removing image distortion specifically comprises the steps of:
s1.1: first, the internal reference of the camera is obtained, wherein the internal reference comprises fx,fy,cx,cyNormalizing the three-dimensional coordinates (X, Y, Z) to homogeneous coordinates (X, Y);
s1.2: removing the effect of distortion on the image, where k1,k2,k3,p1,p2]For the distortion coefficient of the lens, r is the distance of the point from the origin of the coordinate system:
Figure FDA0002805185590000011
s1.3: and transferring the coordinates in the camera coordinate system to the pixel coordinate system:
Figure FDA0002805185590000012
3. the semantic SLAM method based on GMS feature matching in a dynamic scene according to claim 1, wherein the semantic segmentation network is a lightweight semantic segmentation network FcHarDnet.
4. The semantic SLAM method based on GMS feature matching in the dynamic scene as claimed in claim 1, wherein in the third step, a Gaussian pyramid is constructed in the process of extracting ORB feature points from the input image and feature point detection is performed on different layers of the pyramid, so as to achieve the feature of unchanged scale.
5. The semantic SLAM method based on GMS feature matching in a dynamic scene, according to claim 4, wherein the specific steps of ORB feature point extraction are as follows:
s3.1: when the difference between the gray value i (x) of more than N points around a certain point P and the gray value i (P) of the point P is greater than the threshold epsilon, the point is considered as a target corner point, which is specifically represented as:
Figure FDA0002805185590000021
s3.2: to make the feature points hold the orientation unchanged, the centroid is calculated:
Figure FDA0002805185590000022
in the formula, m00，m10，m01Is the integral of the pixel gray scale of the area in the circle about the key point;
s3.3: the image pyramid is constructed by scaling the original image sequence in a certain proportion; extracting corresponding angular points from the images with different sizes at each level of the image pyramid;
s3.4, adopting a quadtree uniform algorithm to continuously subdivide the image downwards into nodes with the same four-division size, combining the nodes without the characteristic points, and increasing the nodes when the number of the characteristic points in the nodes is more than 1; and after the node distribution is finished, deleting redundant feature points in the child nodes.
6. The semantic SLAM method based on GMS feature matching in a dynamic scenario as claimed in claim 5, wherein in the fourth step, the motion consistency detection is: the static point can satisfy epipolar geometric constraint, if the matching point of the object static feature point re-projected on the reference frame is definitely located on the intersection line of the reference frame and the epipolar plane; the method comprises the following specific steps:
S4.1：p1，p2is the homogeneous coordinates of the matching points in the current frame and the reference frame, where u, v are the values in the image frame, specifically:
p1＝[u1,v1,1],p2＝[u2,v2,1]
s4.2: calculating polar lines, wherein F is a basic matrix which can be obtained by calculating eight pairs of characteristic points, and specifically comprises the following steps:
Figure FDA0002805185590000023
wherein [ X Y Z]TRepresenting an epipolar vector;
s4.3: calculating the distance from the matching point to the corresponding epipolar line, if the distance is static D approaching 0, and when D is greater than a threshold epsilon, the characteristic point is dynamic, specifically:
Figure FDA0002805185590000031
the dynamic point determination method comprises the following steps: if there are a sufficient number of dynamic points detected by motion consistency within the mask of the object semantic segmentation, all points of the object are considered dynamic, and all points within the whole object mask are eliminated.
7. The semantic SLAM method based on GMS feature matching in a dynamic scenario as claimed in claim 6, wherein in said step five, GMS algorithm eliminates specific steps of false matching:
s5.1: dividing the image into 20 multiplied by 20 non-repetitive grids, and adjusting the length and width of the grids to perform iterative computation in order to solve the problem that the features are positioned at the boundaries of the grids;
s5.2: dividing 3 x 3 pixels around each feature point into a unit, and calculating the sum S of the total matching number of each unit { i, j } and the corresponding neighbor of the reference frame in the neighborsij，
Figure FDA0002805185590000032
Wherein the content of the first and second substances,
Figure FDA0002805185590000034
representing the number of matched feature points on the corresponding grid pair;
s5.3: calculating a threshold for dividing each cell correct and mismatch, where niI.e. the average number of feature points in each grid, where α is an empirical value, and the calculation formula is as follows:
Figure FDA0002805185590000033
s5.4: and repeating the steps S5.2 and S5.3 until the image traversal is completed to obtain the correct matching with the reference frame.
8. The semantic SLAM method based on GMS feature matching in a dynamic scenario as claimed in claim 1, wherein in said step six, the tracking thread is specifically: and estimating the pose of the current frame according to the motion model in the step five, tracking the map point of the previous frame according to the uniform motion model, and determining the pose.
9. The semantic SLAM method based on GMS feature matching in a dynamic scene as claimed in claim 1, wherein the point cloud processing is performed by a local mapping process, and a sparse point cloud map is obtained by local BA optimization.
10. The semantic SLAM method based on GMS feature matching in a dynamic scene as claimed in claim 1, wherein in the eighth step, a bag of words model is used to determine whether a loop is generated, and if so, the accumulated error is corrected by the loop to optimize the pose.
CN202011365138.3A 2020-11-27 2020-11-27 Semantic SLAM method based on GMS feature matching in dynamic scene Pending CN112381841A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011365138.3A CN112381841A (en) 2020-11-27 2020-11-27 Semantic SLAM method based on GMS feature matching in dynamic scene

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011365138.3A CN112381841A (en) 2020-11-27 2020-11-27 Semantic SLAM method based on GMS feature matching in dynamic scene

Publications (1)

Publication Number Publication Date
CN112381841A true CN112381841A (en) 2021-02-19

Family

ID=74588637

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011365138.3A Pending CN112381841A (en) 2020-11-27 2020-11-27 Semantic SLAM method based on GMS feature matching in dynamic scene

Country Status (1)

Country Link
CN (1) CN112381841A (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113012197A (en) * 2021-03-19 2021-06-22 华南理工大学 Binocular vision odometer positioning method suitable for dynamic traffic scene
CN113159043A (en) * 2021-04-01 2021-07-23 北京大学 Feature point matching method and system based on semantic information
CN113378746A (en) * 2021-06-22 2021-09-10 中国科学技术大学 Positioning method and device
CN113673524A (en) * 2021-07-05 2021-11-19 北京物资学院 Method and device for removing dynamic characteristic points of warehouse semi-structured environment
CN113743413A (en) * 2021-07-30 2021-12-03 的卢技术有限公司 Visual SLAM method and system combining image semantic information
CN113740871A (en) * 2021-07-30 2021-12-03 西安交通大学 Laser SLAM method, system equipment and storage medium in high dynamic environment
CN115388880A (en) * 2022-10-27 2022-11-25 联友智连科技有限公司 Low-cost memory parking map building and positioning method and device and electronic equipment
CN116339336A (en) * 2023-03-29 2023-06-27 北京信息科技大学 Electric agricultural machinery cluster collaborative operation method, device and system
CN117274620A (en) * 2023-11-23 2023-12-22 东华理工大学南昌校区 Visual SLAM method based on self-adaptive uniform division feature point extraction

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108596974A (en) * 2018-04-04 2018-09-28 清华大学 Dynamic scene robot localization builds drawing system and method
CN109631855A (en) * 2019-01-25 2019-04-16 西安电子科技大学 High-precision vehicle positioning method based on ORB-SLAM
CN110675437A (en) * 2019-09-24 2020-01-10 重庆邮电大学 Image matching method based on improved GMS-ORB characteristics and storage medium
CN110738667A (en) * 2019-09-25 2020-01-31 北京影谱科技股份有限公司 RGB-D SLAM method and system based on dynamic scene
CN111161318A (en) * 2019-12-30 2020-05-15 广东工业大学 Dynamic scene SLAM method based on YOLO algorithm and GMS feature matching
CN111402336A (en) * 2020-03-23 2020-07-10 中国科学院自动化研究所 Semantic S L AM-based dynamic environment camera pose estimation and semantic map construction method
CN111724439A (en) * 2019-11-29 2020-09-29 中国科学院上海微***与信息技术研究所 Visual positioning method and device in dynamic scene
CN111784576A (en) * 2020-06-11 2020-10-16 长安大学 Image splicing method based on improved ORB feature algorithm
CN111797688A (en) * 2020-06-02 2020-10-20 武汉大学 Visual SLAM method based on optical flow and semantic segmentation

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108596974A (en) * 2018-04-04 2018-09-28 清华大学 Dynamic scene robot localization builds drawing system and method
CN109631855A (en) * 2019-01-25 2019-04-16 西安电子科技大学 High-precision vehicle positioning method based on ORB-SLAM
CN110675437A (en) * 2019-09-24 2020-01-10 重庆邮电大学 Image matching method based on improved GMS-ORB characteristics and storage medium
CN110738667A (en) * 2019-09-25 2020-01-31 北京影谱科技股份有限公司 RGB-D SLAM method and system based on dynamic scene
CN111724439A (en) * 2019-11-29 2020-09-29 中国科学院上海微***与信息技术研究所 Visual positioning method and device in dynamic scene
CN111161318A (en) * 2019-12-30 2020-05-15 广东工业大学 Dynamic scene SLAM method based on YOLO algorithm and GMS feature matching
CN111402336A (en) * 2020-03-23 2020-07-10 中国科学院自动化研究所 Semantic S L AM-based dynamic environment camera pose estimation and semantic map construction method
CN111797688A (en) * 2020-06-02 2020-10-20 武汉大学 Visual SLAM method based on optical flow and semantic segmentation
CN111784576A (en) * 2020-06-11 2020-10-16 长安大学 Image splicing method based on improved ORB feature algorithm

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113012197A (en) * 2021-03-19 2021-06-22 华南理工大学 Binocular vision odometer positioning method suitable for dynamic traffic scene
CN113159043A (en) * 2021-04-01 2021-07-23 北京大学 Feature point matching method and system based on semantic information
CN113159043B (en) * 2021-04-01 2024-04-30 北京大学 Feature point matching method and system based on semantic information
CN113378746B (en) * 2021-06-22 2022-09-02 中国科学技术大学 Positioning method and device
CN113378746A (en) * 2021-06-22 2021-09-10 中国科学技术大学 Positioning method and device
CN113673524A (en) * 2021-07-05 2021-11-19 北京物资学院 Method and device for removing dynamic characteristic points of warehouse semi-structured environment
CN113740871A (en) * 2021-07-30 2021-12-03 西安交通大学 Laser SLAM method, system equipment and storage medium in high dynamic environment
CN113743413B (en) * 2021-07-30 2023-12-01 的卢技术有限公司 Visual SLAM method and system combining image semantic information
CN113740871B (en) * 2021-07-30 2024-04-02 西安交通大学 Laser SLAM method, system equipment and storage medium under high dynamic environment
CN113743413A (en) * 2021-07-30 2021-12-03 的卢技术有限公司 Visual SLAM method and system combining image semantic information
CN115388880A (en) * 2022-10-27 2022-11-25 联友智连科技有限公司 Low-cost memory parking map building and positioning method and device and electronic equipment
CN116339336A (en) * 2023-03-29 2023-06-27 北京信息科技大学 Electric agricultural machinery cluster collaborative operation method, device and system
CN117274620A (en) * 2023-11-23 2023-12-22 东华理工大学南昌校区 Visual SLAM method based on self-adaptive uniform division feature point extraction
CN117274620B (en) * 2023-11-23 2024-02-06 东华理工大学南昌校区 Visual SLAM method based on self-adaptive uniform division feature point extraction

Similar Documents

Publication Publication Date Title
CN112381841A (en) Semantic SLAM method based on GMS feature matching in dynamic scene
CN112396595B (en) Semantic SLAM method based on point-line characteristics in dynamic environment
CN111462200B (en) Cross-video pedestrian positioning and tracking method, system and equipment
CN112258618B (en) Semantic mapping and positioning method based on fusion of prior laser point cloud and depth map
CN112132893B (en) Visual SLAM method suitable for indoor dynamic environment
CN112991447B (en) Visual positioning and static map construction method and system in dynamic environment
CN112734852B (en) Robot mapping method and device and computing equipment
CN112435262A (en) Dynamic environment information detection method based on semantic segmentation network and multi-view geometry
CN108776989B (en) Low-texture planar scene reconstruction method based on sparse SLAM framework
CN110599522B (en) Method for detecting and removing dynamic target in video sequence
CN111998862B (en) BNN-based dense binocular SLAM method
CN113744315B (en) Semi-direct vision odometer based on binocular vision
CN112446882A (en) Robust visual SLAM method based on deep learning in dynamic scene
CN112652020B (en) Visual SLAM method based on AdaLAM algorithm
CN112802096A (en) Device and method for realizing real-time positioning and mapping
CN111105460A (en) RGB-D camera pose estimation method for indoor scene three-dimensional reconstruction
CN112541423A (en) Synchronous positioning and map construction method and system
CN116128966A (en) Semantic positioning method based on environmental object
CN114140527A (en) Dynamic environment binocular vision SLAM method based on semantic segmentation
CN113487631A (en) Adjustable large-angle detection sensing and control method based on LEGO-LOAM
CN111950599A (en) Dense visual odometer method for fusing edge information in dynamic environment
CN112446885A (en) SLAM method based on improved semantic optical flow method in dynamic environment
CN110570473A (en) weight self-adaptive posture estimation method based on point-line fusion
CN116309817A (en) Tray detection and positioning method based on RGB-D camera
CN111915632B (en) Machine learning-based method for constructing truth database of lean texture target object

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination