CN113763466B - Loop detection method and device, electronic equipment and storage medium - Google Patents
Loop detection method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113763466B CN113763466B CN202011079676.6A CN202011079676A CN113763466B CN 113763466 B CN113763466 B CN 113763466B CN 202011079676 A CN202011079676 A CN 202011079676A CN 113763466 B CN113763466 B CN 113763466B
- Authority
- CN
- China
- Prior art keywords
- image frame
- historical
- loop
- dimensional code
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 70
- 238000000034 method Methods 0.000 claims abstract description 31
- 230000006870 function Effects 0.000 claims description 17
- 239000011159 matrix material Substances 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 4
- 238000000926 separation method Methods 0.000 claims description 3
- 238000005286 illumination Methods 0.000 abstract description 11
- 238000005457 optimization Methods 0.000 description 21
- 230000008859 change Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 230000000694 effects Effects 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000000007 visual effect Effects 0.000 description 5
- 238000013507 mapping Methods 0.000 description 4
- 238000013519 translation Methods 0.000 description 4
- 239000013598 vector Substances 0.000 description 4
- 230000033001 locomotion Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000004807 localization Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06K7/00—Methods or arrangements for sensing record carriers, e.g. for reading patterns
- G06K7/10—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation
- G06K7/14—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
- G06K7/1404—Methods for optical code recognition
- G06K7/1408—Methods for optical code recognition the method being specifically adapted for the type of code
- G06K7/1417—2D bar codes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06K7/00—Methods or arrangements for sensing record carriers, e.g. for reading patterns
- G06K7/10—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation
- G06K7/14—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
- G06K7/1404—Methods for optical code recognition
- G06K7/1439—Methods for optical code recognition including a method step for retrieval of the optical code
- G06K7/1447—Methods for optical code recognition including a method step for retrieval of the optical code extracting optical codes from image or text carrying said optical code
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06K7/00—Methods or arrangements for sensing record carriers, e.g. for reading patterns
- G06K7/10—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation
- G06K7/14—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
- G06K7/1404—Methods for optical code recognition
- G06K7/1439—Methods for optical code recognition including a method step for retrieval of the optical code
- G06K7/1452—Methods for optical code recognition including a method step for retrieval of the optical code detecting bar code edges
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Electromagnetism (AREA)
- General Health & Medical Sciences (AREA)
- Toxicology (AREA)
- Artificial Intelligence (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention discloses a loop detection method, a loop detection device, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring a current image frame and a historical image frame shot by a camera, wherein a plurality of different two-dimensional codes are placed in the shooting environment of the camera in advance; detecting whether a two-dimensional code exists in a photographed current image frame; if the target two-dimensional code exists in the current image frame, determining a historical loop image frame with the target two-dimensional code from the historical image frames. By the technical scheme provided by the embodiment of the invention, the problem of inaccurate detection caused by poor ambient illumination condition in the existing mode can be solved, so that the robustness and accuracy of loop detection are improved, and further the accumulated error is effectively eliminated.
Description
Technical Field
The embodiment of the invention relates to an image processing technology, in particular to a loop detection method, a loop detection device, electronic equipment and a storage medium.
Background
In the field of computer vision research, camera pose is often estimated using a sequence of image frames taken by a camera. SLAM (Simultaneous localization AND MAPPING, instant localization and mapping) is a common technique that constructs a 3D trajectory of a camera by tracking the pose of the camera and maps the shooting environment in which the camera is located. SLAM has a wide range of application scenarios, such as robotic navigation, autopilot, augmented reality, etc.
In the positioning and mapping process based on SLAM, since camera pose estimation is usually a recursive process, that is, the pose of the current image frame is calculated from the pose of the previous image frame, errors are transmitted frame by frame, that is, accumulated errors are larger and larger, and further, the estimated pose and the actual pose have great deviation, so that the constructed environment map can be dislocated and ghost. For this, the cumulative errors in the positioning and mapping process can be eliminated by using a loop detection and global optimization mode.
At present, loop detection is generally performed based on a bag-of-word model, namely, feature information in each image frame is extracted, and a historical loop image frame for loop is determined from historical image frames in a mode of comparing the feature information so as to perform global optimization based on the historical loop image frames.
However, in the process of implementing the present invention, the inventors found that at least the following problems exist in the prior art:
In a loop detection mode based on the word bag model, when the illumination condition in the environment is poor, the extracted texture feature information is sparse, so that the accuracy of loop detection can be reduced, and further the accumulated error cannot be effectively eliminated.
Disclosure of Invention
The embodiment of the invention provides a loop detection method, a loop detection device, electronic equipment and a storage medium, which are used for solving the problem of inaccurate detection caused by poor ambient illumination conditions in the existing mode, thereby improving the robustness and the accuracy of loop detection and further effectively eliminating accumulated errors.
In a first aspect, an embodiment of the present invention provides a loop detection method, including:
Acquiring a current image frame and a historical image frame shot by a camera, wherein a plurality of different two-dimensional codes are placed in the shooting environment of the camera in advance;
detecting whether a two-dimensional code exists in the current shot image frame;
and if the target two-dimensional code exists in the current image frame, determining a historical loop image frame with the target two-dimensional code from the historical image frame.
In a second aspect, an embodiment of the present invention further provides a loop detection apparatus, including:
The image frame acquisition module is used for acquiring a current image frame and a historical image frame shot by the camera, wherein a plurality of different two-dimensional codes are placed in the shooting environment of the camera in advance;
The current image frame detection module is used for detecting whether a two-dimensional code exists in the shot current image frame;
The historical loop image frame determining module is used for determining a historical loop image frame with the target two-dimensional code from the historical image frame if the target two-dimensional code exists in the current image frame.
In a third aspect, an embodiment of the present invention further provides an electronic device, including:
one or more processors;
A memory for storing one or more programs;
The one or more programs, when executed by the one or more processors, cause the one or more processors to implement the loop detection method as provided by any embodiment of the present invention.
In a fourth aspect, embodiments of the present invention further provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a loop detection method as provided by any embodiment of the present invention.
The embodiments of the above invention have the following advantages or benefits:
By pre-placing a plurality of different two-dimensional codes in a camera shooting environment, the two-dimensional codes can exist in image frames shot by a camera, so that the image frames shot under different camera poses can be distinguished by using the different two-dimensional codes, and loop detection can be performed by using the two-dimensional codes. If the target two-dimensional code exists in the shot current video frame, a historical loop image frame with the target two-dimensional code is determined from the shot historical image frame, so that the historical image frame with the same target two-dimensional code existing in the current image frame can be used as a historical loop image frame for loop, the two-dimensional code is in a black-and-white form, the contrast is higher, the two-dimensional code can be accurately detected when the ambient illumination condition is poor, the influence of ambient illumination change is not easy, the robustness and the accuracy of loop detection are improved, and further the accumulated error is effectively eliminated.
Drawings
Fig. 1 is a flowchart of a loop detection method according to a first embodiment of the present invention;
Fig. 2 is a flowchart of a loop detection method according to a second embodiment of the present invention;
Fig. 3 is a schematic structural diagram of a loop detection device according to a third embodiment of the present invention;
Fig. 4 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present invention.
Detailed Description
The invention is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present invention are shown in the drawings.
Example 1
Fig. 1 is a flowchart of a loop detection method according to a first embodiment of the present invention, where the present embodiment is applicable to a case of loop detection on a current image frame, and in particular, the present embodiment may be used for loop detection in a visual positioning process, so as to perform global pose optimization based on a detected historical loop image frame. The method may be performed by a loop detection device, which may be implemented in software and/or hardware, integrated in an electronic device. As shown in fig. 1, the method specifically includes the following steps:
s110, acquiring a current image frame and a historical image frame shot by a camera, wherein a plurality of different two-dimensional codes are placed in the shooting environment of the camera in advance.
The current image frame may refer to an image frame photographed by the camera at the current moment. The historical image frames may refer to image frames captured by the camera prior to the current time. The camera shooting environment may refer to an environment map to be constructed. The two-dimensional code can be a graph which is distributed on a plane according to a certain rule by using a certain specific geometric figure, is black-white and records digital symbol information. The two-dimensional code is in a black-and-white form, so that the contrast is higher, and the two-dimensional code can be accurately detected even if the ambient illumination condition is poor, so that the ambient illumination change can be more robustly dealt with, and the robustness and the accuracy of loop detection are improved. Each two-dimensional code corresponds to one two-dimensional code identifier so as to distinguish different two-dimensional codes. Each two-dimensional code can be placed at different positions which can be shot by the camera, so that the two-dimensional code can exist in an image frame shot by the camera. The placement interval of two adjacent two-dimensional codes can be set based on the pose change speed of the camera, so that the condition that the same two-dimensional code is shot in two adjacent image frames is avoided. For example, a plurality of different two-dimensional codes may be placed on the ground of a camera shooting environment at intervals of several tens of meters.
It should be noted that, because the camera is moving, the camera pose at each shooting time, that is, the camera pose corresponding to each image frame, may also change in real time, so that there may be a history loop image frame similar to the camera pose corresponding to the current image frame in the history image frame, so loop detection needs to be performed in the history image frame so as to establish a pose constraint relationship between the current image frame and the history loop image frame, so that the pose of the current image frame may be calculated based on the pose of the history loop image frame, and compared with the pose of the current image frame calculated based on the pose of the previous image frame, the accumulated error may be effectively eliminated.
Specifically, in a camera shooting environment in which a plurality of different two-dimensional codes are placed, a current image frame and a history image frame shot by the camera are acquired. Each image frame shot by the camera may or may not shoot a two-dimensional code placed in the environment. In this embodiment, loop detection may be performed for each image frame captured by the camera, that is, the operations of steps S110 to S130 are performed based on the current image frame captured at the current moment, so that loop detection is performed in real time, so as to optimize the pose of the camera in real time.
S120, detecting whether a two-dimensional code exists in the current shot image frame.
Specifically, whether a two-dimensional code exists in the current image frame can be detected through an apriltag two-dimensional code algorithm. For example, detecting various edges in the current image frame according to the gradient, detecting the edges, removing the non-linear edges, searching the adjacent edges at the linear edges, determining a quadrilateral image forming a closed loop, and determining a two-dimensional code identifier corresponding to the quadrilateral image by encoding and decoding the quadrilateral image. And if the two-dimensional code identification corresponding to the quadrilateral image can be determined, indicating that the two-dimensional code exists in the current image frame. If the quadrilateral image does not exist or the two-dimensional code identification corresponding to the quadrilateral image cannot be determined, the fact that the two-dimensional code does not exist in the current image frame is indicated, namely, the placed two-dimensional code is not shot when the current image frame is shot.
It should be noted that, if the two-dimensional code does not exist in the current image frame, it indicates that the history loop image frame similar to the current image frame in pose does not exist in the history image frame, and loop detection is not required to be performed on the current image frame, at this time, step S110 may be returned to perform loop detection on the next image frame.
And S130, if the target two-dimensional code exists in the current image frame, determining a historical loop image frame with the target two-dimensional code from the historical image frame.
The target two-dimensional code may be a two-dimensional code existing in the current image frame and used for representing the pose of the camera corresponding to the current image frame. The historical loop-back image frame may refer to a historical image frame that is similar to the camera pose corresponding to the current image frame, so that a pose constraint relationship between the current image frame and the historical loop-back image frame may be established. If the two image frames have the same two-dimensional code, the two image frames indicate that the corresponding camera pose of the two image frames is similar, namely the camera pose changes less, so that the pose of the second image frame can be calculated based on the pose of the first image frame. If different two-dimensional codes exist in the two image frames, the situation that the camera pose difference corresponding to the two image frames is larger, namely the camera pose change is larger, and at the moment, the pose of the second image frame is calculated based on the pose of the first image frame, namely the pose constraint relation between the two image frames cannot be established.
Specifically, if the two-dimensional code exists in the current image frame, namely, the target two-dimensional code is detected, whether the target two-dimensional code exists in each historical image frame or not can be detected, and the historical image frame with the target two-dimensional code is determined to be a historical loop image frame, so that loop detection can be more accurately carried out by utilizing the two-dimensional code, a historical loop image frame similar to the camera pose corresponding to the current image frame is obtained, the pose constraint relation between the current image frame and the historical loop image frame is established later to carry out global pose optimization, and accumulated errors are effectively eliminated.
According to the technical scheme, the plurality of different two-dimensional codes are placed in the camera shooting environment in advance, so that the two-dimensional codes can exist in the image frames shot by the camera, the image frames shot under different camera poses can be distinguished by using the different two-dimensional codes, and loop detection can be performed by using the two-dimensional codes. If the target two-dimensional code exists in the shot current video frame, a historical loop image frame with the target two-dimensional code is determined from the shot historical image frame, so that the historical image frame with the same target two-dimensional code existing in the current image frame can be used as a historical loop image frame for loop, the two-dimensional code is in a black-and-white form, the contrast is higher, the two-dimensional code can be accurately detected when the ambient illumination condition is poor, the influence of ambient illumination change is not easy, the robustness and the accuracy of loop detection are improved, and further the accumulated error is effectively eliminated.
Based on the above technical solution, S120 may further include: if at least two-dimensional codes exist in the current image frame, acquiring an image area of each two-dimensional code in the current image frame; and determining the two-dimensional code with the largest image area as a target two-dimensional code.
Specifically, the current image frame shot by the camera may not have a two-dimensional code, may have one two-dimensional code, and may have a plurality of two-dimensional codes. When at least two-dimensional codes exist in the current image frame, one target two-dimensional code used as a loop detection reference is required to be determined from the at least two-dimensional codes, at the moment, the image area occupied by each two-dimensional code in the current image frame can be obtained, the size of the image area is compared, the two-dimensional code with the largest image area is taken as the target two-dimensional code, and the two-dimensional code with the largest image area can be taken as the target two-dimensional code, so that the two-dimensional code closest to the camera can be taken as the target two-dimensional code, the accuracy of loop detection is further improved, and the pose optimization effect is further improved.
Based on the above technical solution, the "determining, from the historical image frames, the historical loop image frame in which the target two-dimensional code exists" in S130 may include: determining candidate historical image frames with target two-dimension codes based on the corresponding relation between the two-dimension codes and the historical image frames and the target two-dimension codes; if only one candidate historical image frame exists, determining the candidate historical image frame as a historical loop image frame; if at least two candidate historical image frames exist, a historical loop image frame is determined from the candidate historical image frames based on the image area of the target two-dimensional code in each candidate historical image frame.
The corresponding relation between the two-dimensional code and the historical image frames can be used for representing the two-dimensional code existing in each historical image frame, and the corresponding relation can be established based on the two-dimensional code identification and the historical image frame identification.
Specifically, in the loop detection process for each image frame, the target two-dimensional code existing in each image frame detected in real time can be stored, so that a corresponding relationship between the two-dimensional code and the historical image frame is established. Based on the corresponding relation and the target two-dimensional code existing in the current image frame, each history image frame corresponding to the target two-dimensional code can be determined from all history image frames, namely each candidate history image frame with the target two-dimensional code. If there is only one candidate historical image frame, the candidate historical image frame can be directly used as a historical loop-back image frame. If at least two candidate historical image frames exist, the image area of the target two-dimensional code in each candidate historical image frame can be obtained, the size of the image area is compared, the candidate historical image frame with the largest image area is used as a historical loop image frame, namely the candidate historical image frame with the largest image area can be used as a historical loop image frame, so that the candidate historical image frame shot when the target two-dimensional code is closest to the camera can be used as a historical loop image frame, namely the historical loop image frame with the best loop constraint effect can be obtained, the pose optimization effect is further improved, and the accumulated error is effectively eliminated.
Illustratively, determining the candidate historical image frame in which the target two-dimensional code exists based on the correspondence between the two-dimensional code and the historical image frame and the target two-dimensional code may include: determining an image frame loop-back range corresponding to a current image frame, and determining a historical image frame to be selected within the image frame loop-back range from the historical image frames; and determining candidate historical image frames with target two-dimension codes from the historical image frames to be selected based on the corresponding relation between the two-dimension codes and the historical image frames and the target two-dimension codes.
The image frame loop-back range corresponding to the current image frame refers to a historical image frame range capable of performing loop-back constraint with the current image frame, namely a loop-back detection range of the historical image frame. For example, the image frame loop-back range may be set by a time constraint or a distance constraint, for example, a history image frame having a shooting time interval of 20 seconds or more from the current image frame may be set as the history image frame range, or a history image frame having a shooting time interval of 20 frames or more from the current image frame may be set as the history image frame range.
Specifically, an image frame loop-back range corresponding to the current image frame can be determined based on the shooting time or the shooting frame number of the current image frame, each historical image frame in the image frame loop-back range is used as a historical image frame to be selected, so that candidate historical image frames with target two-dimension codes are determined in the historical image frames to be selected, loop-back detection in the historical image frames near the current image frame can be avoided, the detected historical loop-back image frames can be the historical image frames with earlier shooting time, loop-back constraint effects are further improved, and accumulated errors are more effectively eliminated.
Example two
Fig. 2 is a flowchart of a loop detection method according to a second embodiment of the present invention, where, based on the foregoing embodiments, a process of global pose optimization using a historical loop image frame is described in detail after determining the historical loop image frame corresponding to the current image frame. Wherein the explanation of the same or corresponding terms as those of the above embodiments is not repeated herein.
Referring to fig. 2, the loop detection method provided in this embodiment specifically includes the following steps:
S210, acquiring a current image frame and a historical image frame shot by a camera, wherein a plurality of different two-dimensional codes are placed in the shooting environment of the camera in advance.
S220, detecting whether a two-dimensional code exists in the current shot image frame.
And S230, if the target two-dimensional code exists in the current image frame, determining a historical loop image frame with the target two-dimensional code from the historical image frame.
S240, determining the loop-back relative pose between the current image frame and the historical loop-back image frame.
The loop-back relative pose may refer to a relative pose between a current image frame and a historical loop-back image frame. The embodiment can determine the loop-back relative pose between the current image frame and the historical loop-back image frame based on the position relation between the target two-dimensional code and the current image frame and the historical loop-back image frame respectively.
Illustratively, S240 may include: determining a first relative pose between the target two-dimensional code and the current image frame and a second relative pose between the target two-dimensional code and the target historical image frame based on the actual angular point interval distance corresponding to the target two-dimensional code; and determining the loop-back relative pose between the current image frame and the historical loop-back image frame according to the first relative pose and the second relative pose.
The actual angular point interval distance may refer to an actual distance between two adjacent angular points in the target two-dimensional code. The actual corner separation distance in this embodiment is known and fixed so that pose estimation scale information can be provided.
Specifically, a first relative pose between the target two-dimensional code and the current image frame can be determined based on the actual angular point spacing distance corresponding to the target two-dimensional code through an apriltag two-dimensional code algorithmAnd a second relative pose/>, between the target two-dimensional code and the target historical image frameObtaining loop-back relative pose/>, between the current image frame and the historical loop-back image frame, by multiplying the first relative pose and the second relative poseI.e./>Thereby establishing a loop constraint relationship.
Illustratively, determining a first relative pose between the target two-dimensional code and the current image frame based on an actual corner separation distance corresponding to the target two-dimensional code includes: extracting four corner information of a target two-dimensional code in a current image frame; constructing a homography matrix based on the four corner information and the actual corner interval distance corresponding to the target two-dimensional code; and determining a first relative pose between the target two-dimensional code and the current image frame by solving the homography matrix.
Specifically, because the target two-dimensional code is on the plane, a homography matrix can be constructed based on the four corner information of the target two-dimensional code and the actual corner interval distance, and a rotation matrix and a translation vector, namely a first relative pose between the target two-dimensional code and the current image frame, are decomposed by solving the homography matrix. Similarly, the second relative pose between the target two-dimensional code and the target historical image frame can be determined through the mode. In the embodiment, the pose estimation can be performed based on the actual angular point spacing distance, so that the actual scale information can be introduced, and the accuracy and precision of the pose estimation are further improved.
S250, acquiring initial relative pose corresponding to two adjacent image frames between the current image frame and the historical loop image frame.
The SLAM system is typically composed of a front-end visual odometer (Visual Odometry, VO) and a back-end optimization, which are used to estimate camera motion between adjacent image frames and a local map. The back-end optimization is used for optimizing the estimated motion and the loop detection information to obtain a globally consistent track and map. The initial relative pose in this embodiment may be the initial relative pose between two adjacent image frames obtained with a visual odometer. The initial relative pose corresponding to two adjacent image frames between the current image frame and the historical loop-back image frame may include: an initial relative pose between a previous image frame and a current image frame, an initial relative pose between two adjacent intermediate image frames, and an initial relative pose between a historical loop-back image frame and a next image frame. Wherein the intermediate image frame may refer to an image frame located between the current image frame and the historical loop back image frame.
Specifically, the embodiment may determine, for each captured image frame, an initial relative pose between each image frame and a previous image frame based on a preset camera pose estimation manner. For example, when a current image frame photographed by a camera is obtained, a loop detection operation of the current image frame and an operation of estimating an initial relative pose between the current image frame and a previous image frame using a visual odometer may be simultaneously performed with respect to the current image frame. And during global optimization, the initial relative pose corresponding to the two adjacent image frames between the current image frame and the historical loop image frame can be screened from the initial relative pose corresponding to each group of two adjacent image frames obtained in advance.
The preset camera pose estimation mode can comprise a direct method and a characteristic point method. The direct method specifically comprises the following steps: firstly extracting high-gradient pixel points from one image frame, then setting an initial camera pose, constructing a loss function of luminosity errors of the pixel points extracted in the previous step between two adjacent image frames according to the initial camera pose, and solving and determining the optimal solution of the pose in a nonlinear optimization mode. The direct method can save the time for calculating the feature points and the descriptors and can be applied to some application occasions with missing features. The feature point method is to extract feature points in two adjacent image frames and calculate descriptors respectively, and then solve the initial relative pose between the two image frames through feature matching between the two image frames. The feature point method can specifically determine an initial relative pose between a current image frame and a previous image frame by: n feature points in the current image frame and the previous image frame are extracted respectively, and descriptors of each feature point are calculated. And carrying out feature matching on the feature points of the current image frame and the feature points of the previous image frame by using a descriptor to obtain N feature matching pairs. Based on the N pairs of feature matches, N pairs of level constraint equations may be constructed. The N pairs of constraint equations may be solved by singular value decomposition or least squares to obtain a rotation matrix and translation vectors that are transformed from the current image frame to the previous image frame, i.e., an initial relative pose between the current image frame and the previous image frame. Wherein, the epipolar constraint equation corresponding to each feature matching pair may be:
p1＝K(RK-1p2+t)
Where p 1 is the pixel coordinates of the feature point in the previous image frame and p 2 is the pixel coordinates of the feature point in the current image frame; k is a known camera internal reference; r is a rotation matrix for converting the current image frame into the previous image frame; t is the translation vector for the current image frame to be transformed to the previous image frame.
The characteristic point method has strong tolerance to overexposure or rapid movement of a camera because the assumption based on the unchanged gray level is not adopted, loss and failure are not easy to track, and the robustness is strong. The present embodiment can determine an initial relative pose between two adjacent image frames using a feature point method.
And S260, optimizing the camera pose corresponding to each image frame between the current image frame and the historical loop image frame according to the loop relative pose and each initial relative pose, and determining the target camera pose corresponding to each image frame.
The camera pose corresponding to the image frame may refer to a camera pose at a shooting time corresponding to the image frame. The camera pose may be characterized using a rotation matrix and a translation vector of the camera. The target camera pose may refer to the final camera pose obtained after optimization. The target camera pose in this embodiment may refer to a camera pose in the world coordinate system. For example, the target camera pose corresponding to each image frame may refer to the camera pose of each image frame relative to the first image frame captured for the first time.
Specifically, according to the loop-back relative pose between the current image frame and the historical loop-back image frame and the initial relative pose corresponding to each two adjacent image frames between the current image frame and the historical loop-back image frame, global optimization can be performed on the camera poses corresponding to all the image frames between the current image frame and the historical loop-back image frame, and the target camera pose corresponding to each optimized image frame is obtained, so that accumulated errors can be effectively eliminated, the pose optimization effect is improved, and more accurate camera poses are obtained.
Illustratively, S260 may include: establishing an objective function containing a camera pose corresponding to each image frame between the current image frame and the historical loop image frame according to the loop relative pose and each initial relative pose; and minimizing an objective function based on a least square mode to obtain the pose of the objective camera corresponding to each image frame.
The objective function may refer to a loss function to be optimized, and the optimized pose of the objective camera is obtained by minimizing the objective function. The least squares optimization may be, but is not limited to, gauss-newton's method, levenberg-marquardt. Illustratively, the objective function may be:
wherein c represents the current image frame; h represents a historical loop-back image frame; i represents any image frame between the current image frame and the historical loop-back image frame; Representing an initial relative pose between the i+1th image frame and the i-th image frame; Representing the camera pose corresponding to the ith image frame in the world coordinate system; /(I) Representing the camera pose corresponding to the (i+1) th image frame in the world coordinate system; /(I)Representing loop-back relative pose between a current image frame and a historical loop-back image frame; /(I)Representing the camera pose corresponding to the historical loop image frame under the world coordinate system; /(I)And representing the pose of the camera corresponding to the current image frame in the world coordinate system. Specifically, the camera pose/>, corresponding to the ith image frame, in the objective functionCamera pose corresponding to (i+1) th image frame/>Camera pose/>, corresponding to historical loop image frameAnd the camera pose/>, corresponding to the current image frameIs a parameter to be optimized.
Specifically, by using a least squares optimization method, least squares optimization can be performed on the objective function, so as to obtain an optimal estimate of the pose of the objective camera corresponding to each image frame.
According to the technical scheme of the embodiment, global pose optimization is performed on the camera poses corresponding to all the image frames between the current image frame and the historical loop image frame according to the loop relative poses between the current image frame and the historical loop image frame and the initial relative poses corresponding to each two adjacent image frames between the current image frame and the historical loop image frame, so that accumulated errors can be effectively eliminated, the pose optimization effect is improved, and more accurate camera poses are obtained.
The following is an embodiment of a loop detection device provided in the embodiment of the present invention, which belongs to the same inventive concept as the loop detection method of the above embodiments, and details of the embodiment of the loop detection device that are not described in detail may refer to the embodiment of the loop detection method.
Example III
Fig. 3 is a schematic structural diagram of a loop detection device according to a third embodiment of the present invention, where the embodiment is applicable to a case of loop detection on a current image frame, and the device specifically includes: an image frame acquisition module 310, a current image frame detection module 320, and a historical loop-back image frame determination module 330;
The image frame acquiring module 310 is configured to acquire a current image frame and a historical image frame captured by a camera, where a plurality of different two-dimensional codes are placed in the camera capturing environment in advance; the current image frame detection module 320 is configured to detect whether a two-dimensional code exists in a current image frame that is photographed; the historical loop-back image frame determining module 330 is configured to determine, from the historical image frames, a historical loop-back image frame in which the target two-dimensional code exists if the target two-dimensional code exists in the current image frame.
According to the technical scheme, the plurality of different two-dimensional codes are placed in the camera shooting environment in advance, so that the two-dimensional codes can exist in the image frames shot by the camera, the image frames shot under different camera poses can be distinguished by using the different two-dimensional codes, and loop detection can be performed by using the two-dimensional codes. If the target two-dimensional code exists in the shot current video frame, a historical loop image frame with the target two-dimensional code is determined from the shot historical image frame, so that the historical image frame with the same target two-dimensional code existing in the current image frame can be used as a historical loop image frame for loop, the two-dimensional code is in a black-and-white form, the contrast is higher, the two-dimensional code can be accurately detected when the ambient illumination condition is poor, the influence of ambient illumination change is not easy, the robustness and the accuracy of loop detection are improved, and further the accumulated error is effectively eliminated.
Optionally, the historical loop-back image frame determination module 330 includes:
The candidate historical image frame determining submodule is used for determining candidate historical image frames with target two-dimensional codes based on the corresponding relation between the two-dimensional codes and the historical image frames and the target two-dimensional codes;
A historical loop image frame determination submodule, configured to determine a candidate historical image frame as a historical loop image frame if only one candidate historical image frame exists; if at least two candidate historical image frames exist, a historical loop image frame is determined from the candidate historical image frames based on the image area of the target two-dimensional code in each candidate historical image frame.
Optionally, the candidate historical image frame determination submodule is specifically configured to: determining an image frame loop-back range corresponding to a current image frame, and determining a historical image frame to be selected within the image frame loop-back range from the historical image frames; and determining candidate historical image frames with target two-dimension codes from the historical image frames to be selected based on the corresponding relation between the two-dimension codes and the historical image frames and the target two-dimension codes.
Optionally, the apparatus further comprises:
the target two-dimensional code determining module is used for acquiring an image area of each two-dimensional code in the current image frame if at least two-dimensional codes exist in the current image frame after detecting whether the two-dimensional codes exist in the shot current image frame; and determining the two-dimensional code with the largest image area as a target two-dimensional code.
Optionally, the apparatus further comprises:
The loop-back relative pose determining module is used for determining the loop-back relative pose between the current image frame and the historical loop-back image frame after determining the historical loop-back image frame with the target two-dimensional code from the historical image frames;
The initial relative pose acquisition module is used for acquiring initial relative poses corresponding to two adjacent image frames between the current image frame and the historical loop image frame;
And the camera pose optimization module is used for optimizing the camera pose corresponding to each image frame between the current image frame and the historical loop image frame according to the loop relative pose and each initial relative pose, and determining the target camera pose corresponding to each image frame.
Optionally, the loop relative pose determining module includes:
The relative pose determining sub-module is used for determining a first relative pose between the target two-dimensional code and the current image frame and a second relative pose between the target two-dimensional code and the target historical image frame based on the actual angular point interval distance corresponding to the target two-dimensional code;
and the loop relative pose determining submodule is used for determining the loop relative pose between the current image frame and the historical loop image frame according to the first relative pose and the second relative pose.
Optionally, the relative pose determination submodule is specifically configured to:
Extracting four corner information of a target two-dimensional code in a current image frame; constructing a homography matrix based on the four corner information and the actual corner interval distance corresponding to the target two-dimensional code; and determining a first relative pose between the target two-dimensional code and the current image frame by solving the homography matrix.
Optionally, the camera pose optimization module is specifically configured to:
establishing an objective function containing a camera pose corresponding to each image frame between the current image frame and the historical loop image frame according to the loop relative pose and each initial relative pose; and minimizing an objective function based on a least square mode to obtain the pose of the objective camera corresponding to each image frame.
Optionally, the objective function is:
wherein c represents the current image frame; h represents a historical loop-back image frame; i represents any image frame between the current image frame and the historical loop-back image frame; Representing an initial relative pose between the i+1th image frame and the i-th image frame; Representing the camera pose corresponding to the ith image frame in the world coordinate system; /(I) Representing the camera pose corresponding to the (i+1) th image frame in the world coordinate system; /(I)Representing loop-back relative pose between a current image frame and a historical loop-back image frame; /(I)Representing the camera pose corresponding to the historical loop image frame under the world coordinate system; /(I)And representing the pose of the camera corresponding to the current image frame in the world coordinate system.
The loop detection device provided by the embodiment of the invention can execute the loop detection method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of executing the loop detection method.
It should be noted that, in the embodiment of the loop detection apparatus, each unit and module included are only divided according to the functional logic, but not limited to the above-mentioned division, so long as the corresponding functions can be implemented; in addition, the specific names of the functional units are also only for distinguishing from each other, and are not used to limit the protection scope of the present invention.
Example IV
Fig. 4 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present invention. Fig. 4 illustrates a block diagram of an exemplary electronic device 12 suitable for use in implementing embodiments of the present invention. The electronic device 12 shown in fig. 4 is merely an example and should not be construed as limiting the functionality and scope of use of embodiments of the present invention.
As shown in fig. 4, the electronic device 12 is in the form of a general purpose computing device. Components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, a bus 18 that connects the various system components, including the system memory 28 and the processing units 16.
Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, and a local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, micro channel architecture (MAC) bus, enhanced ISA bus, video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.
Electronic device 12 typically includes a variety of computer system readable media. Such media can be any available media that is accessible by electronic device 12 and includes both volatile and nonvolatile media, removable and non-removable media.
The system memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 30 and/or cache memory 32. The electronic device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from or write to non-removable, nonvolatile magnetic media (not shown in FIG. 4, commonly referred to as a "hard disk drive"). Although not shown in fig. 4, a magnetic disk drive for reading from and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from or writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In such cases, each drive may be coupled to bus 18 through one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to carry out the functions of the embodiments of the invention.
A program/utility 40 having a set (at least one) of program modules 42 may be stored in, for example, system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment. Program modules 42 generally perform the functions and/or methods of the embodiments described herein.
The electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), one or more devices that enable a user to interact with the electronic device 12, and/or any devices (e.g., network card, modem, etc.) that enable the electronic device 12 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 22. Also, the electronic device 12 may communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN) and/or a public network, such as the Internet, through a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 over the bus 18. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with electronic device 12, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.
The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, for example, implementing a loop detection method step provided in the present embodiment, the method includes:
Acquiring a current image frame and a historical image frame shot by a camera, wherein a plurality of different two-dimensional codes are placed in the shooting environment of the camera in advance;
Detecting whether a two-dimensional code exists in a photographed current image frame;
if the target two-dimensional code exists in the current image frame, determining a historical loop image frame with the target two-dimensional code from the historical image frames.
Of course, those skilled in the art will understand that the processor may also implement the technical solution of the loop detection method provided in any embodiment of the present invention.
Example five
The present embodiment provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the loop detection method steps as provided by any embodiment of the present invention, the method comprising:
Acquiring a current image frame and a historical image frame shot by a camera, wherein a plurality of different two-dimensional codes are placed in the shooting environment of the camera in advance;
Detecting whether a two-dimensional code exists in a photographed current image frame;
if the target two-dimensional code exists in the current image frame, determining a historical loop image frame with the target two-dimensional code from the historical image frames.
The computer storage media of embodiments of the invention may take the form of any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium may be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations of the present invention may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
It will be appreciated by those of ordinary skill in the art that the modules or steps of the invention described above may be implemented in a general purpose computing device, they may be centralized on a single computing device, or distributed over a network of computing devices, or they may alternatively be implemented in program code executable by a computer device, such that they are stored in a memory device and executed by the computing device, or they may be separately fabricated as individual integrated circuit modules, or multiple modules or steps within them may be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
Note that the above is only a preferred embodiment of the present invention and the technical principle applied. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, while the invention has been described in connection with the above embodiments, the invention is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit or scope of the invention, which is set forth in the following claims.
Claims (11)
1. The loop detection method is characterized by comprising the following steps of:
Acquiring a current image frame and a historical image frame shot by a camera, wherein a plurality of different two-dimensional codes are placed in the shooting environment of the camera in advance;
detecting whether a two-dimensional code exists in the current shot image frame;
If the target two-dimensional code exists in the current image frame, determining a historical loop image frame with the target two-dimensional code from the historical image frame; the target two-dimensional code is a two-dimensional code which exists in the current image frame and is used for representing the pose of the camera corresponding to the current image frame;
the determining, from the historical image frames, that the historical loop image frame of the target two-dimensional code exists includes:
determining candidate historical image frames with the target two-dimensional code based on the corresponding relation between the two-dimensional code and the historical image frames and the target two-dimensional code; if only one candidate historical image frame exists, determining the candidate historical image frame as a historical loop image frame; and if at least two candidate historical image frames exist, determining a historical loop image frame from the candidate historical image frames based on the image area of the target two-dimensional code in each candidate historical image frame.
2. The method of claim 1, wherein determining that a candidate historical image frame for the target two-dimensional code exists based on a correspondence between two-dimensional codes and historical image frames and the target two-dimensional code, comprises:
Determining an image frame loop-back range corresponding to the current image frame, and determining a history image frame to be selected within the image frame loop-back range from the history image frames;
And determining candidate historical image frames with the target two-dimensional codes from the historical image frames to be selected based on the corresponding relation between the two-dimensional codes and the historical image frames and the target two-dimensional codes.
3. The method according to claim 1, further comprising, after detecting whether a two-dimensional code is present in the current image frame photographed:
If at least two-dimensional codes exist in the current image frame, acquiring an image area of each two-dimensional code in the current image frame;
And determining the two-dimensional code with the largest image area as a target two-dimensional code.
4. The method according to any one of claims 1-3, further comprising, after determining from the historical image frames that the historical loop-back image frame of the target two-dimensional code is present:
Determining loop-back relative pose between the current image frame and the historical loop-back image frame;
acquiring initial relative poses corresponding to two adjacent image frames between the current image frame and the historical loop image frame;
and optimizing the camera pose corresponding to each image frame between the current image frame and the historical loop back image frame according to the loop back relative pose and each initial relative pose, and determining the target camera pose corresponding to each image frame.
5. The method of claim 4, wherein determining a loop-back relative pose between the current image frame and the historical loop-back image frame comprises:
Determining a first relative pose between the target two-dimensional code and the current image frame and a second relative pose between the target two-dimensional code and the historical loop-back image frame based on an actual angular point interval distance corresponding to the target two-dimensional code;
and determining loop-back relative pose between the current image frame and the historical loop-back image frame according to the first relative pose and the second relative pose.
6. The method of claim 5, wherein determining a first relative pose between the target two-dimensional code and the current image frame based on an actual corner separation distance corresponding to the target two-dimensional code comprises:
Extracting four corner information of the target two-dimensional code in the current image frame;
Constructing a homography matrix based on the four corner information and the actual corner interval distance corresponding to the target two-dimensional code;
and determining a first relative pose between the target two-dimensional code and the current image frame by solving the homography matrix.
7. The method of claim 4, wherein optimizing the camera pose for each image frame between the current image frame and the historical loop back image frame based on the loop back relative pose and the initial relative poses, determining a target camera pose for each image frame comprises:
Establishing an objective function containing a camera pose corresponding to each image frame between the current image frame and the historical loop-back image frame according to the loop-back relative pose and each initial relative pose;
And minimizing the objective function based on a least square mode to obtain the pose of the objective camera corresponding to each image frame.
8. The method of claim 7, wherein the objective function is:
wherein c represents the current image frame; h represents the historical loop-back image frame; i represents any image frame between the current image frame and the historical loop-back image frame; Representing an initial relative pose between the i+1th image frame and the i-th image frame; /(I) Representing the camera pose corresponding to the ith image frame in the world coordinate system; /(I)Representing the camera pose corresponding to the (i+1) th image frame in the world coordinate system; /(I)Representing loop-back relative pose between the current image frame and the historical loop-back image frame; /(I)Representing the camera pose corresponding to the historical loop image frame under the world coordinate system; /(I)And representing the camera pose corresponding to the current image frame under the world coordinate system.
9. A loop detection device, comprising:
The image frame acquisition module is used for acquiring a current image frame and a historical image frame shot by the camera, wherein a plurality of different two-dimensional codes are placed in the shooting environment of the camera in advance;
The current image frame detection module is used for detecting whether a two-dimensional code exists in the shot current image frame;
The historical loop image frame determining module is used for determining a historical loop image frame with the target two-dimensional code from the historical image frame if the target two-dimensional code exists in the current image frame; the target two-dimensional code is a two-dimensional code which exists in the current image frame and is used for representing the pose of the camera corresponding to the current image frame;
the historical loop image frame determining module comprises:
the candidate historical image frame determining submodule is used for determining candidate historical image frames with the target two-dimensional code based on the corresponding relation between the two-dimensional code and the historical image frames and the target two-dimensional code;
A historical loop-back image frame determining sub-module, configured to determine, if only one candidate historical image frame exists, the candidate historical image frame as a historical loop-back image frame; and if at least two candidate historical image frames exist, determining a historical loop image frame from the candidate historical image frames based on the image area of the target two-dimensional code in each candidate historical image frame.
10. An electronic device, the electronic device comprising:
one or more processors;
A memory for storing one or more programs;
When executed by the one or more processors, causes the one or more processors to implement the loop back detection method of any of claims 1-8.
11. A computer readable storage medium having stored thereon a computer program, which when executed by a processor implements a loop back detection method according to any of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011079676.6A CN113763466B (en) | 2020-10-10 | 2020-10-10 | Loop detection method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011079676.6A CN113763466B (en) | 2020-10-10 | 2020-10-10 | Loop detection method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113763466A CN113763466A (en) | 2021-12-07 |
| CN113763466B true CN113763466B (en) | 2024-06-14 |
Family
ID=78785813
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011079676.6A Active CN113763466B (en) | 2020-10-10 | 2020-10-10 | Loop detection method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113763466B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115982399B (en) * | 2023-03-16 | 2023-05-16 | 北京集度科技有限公司 | Image searching method, mobile device, electronic device and computer program product |
| CN116358533B (en) * | 2023-05-31 | 2023-08-29 | 小米汽车科技有限公司 | Loop detection data processing method and device, storage medium and vehicle |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109556596A (en) * | 2018-10-19 | 2019-04-02 | 北京极智嘉科技有限公司 | Air navigation aid, device, equipment and storage medium based on ground texture image |
| CN110689562A (en) * | 2019-09-26 | 2020-01-14 | 深圳市唯特视科技有限公司 | Trajectory loop detection optimization method based on generation of countermeasure network |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007142227A1 (en) * | 2006-06-07 | 2007-12-13 | Nec Corporation | Image direction judging device, image direction judging method and image direction judging program |
| TWI506563B (en) * | 2013-01-28 | 2015-11-01 | Tencent Tech Shenzhen Co Ltd | A method and apparatus for enhancing reality of two - dimensional code |
| WO2019169540A1 (en) * | 2018-03-06 | 2019-09-12 | 斯坦德机器人（深圳）有限公司 | Method for tightly-coupling visual slam, terminal and computer readable storage medium |
| CN111060101B (en) * | 2018-10-16 | 2022-06-28 | 深圳市优必选科技有限公司 | Vision-assisted distance SLAM method and device and robot |
| CN111598149B (en) * | 2020-05-09 | 2023-10-24 | 鹏城实验室 | Loop detection method based on attention mechanism |
| CN111696118B (en) * | 2020-05-25 | 2022-11-15 | 东南大学 | Visual loopback detection method based on semantic segmentation and image restoration in dynamic scene |
-
2020
- 2020-10-10 CN CN202011079676.6A patent/CN113763466B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109556596A (en) * | 2018-10-19 | 2019-04-02 | 北京极智嘉科技有限公司 | Air navigation aid, device, equipment and storage medium based on ground texture image |
| CN110689562A (en) * | 2019-09-26 | 2020-01-14 | 深圳市唯特视科技有限公司 | Trajectory loop detection optimization method based on generation of countermeasure network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113763466A (en) | 2021-12-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110322500B (en) | Optimization method and device for instant positioning and map construction, medium and electronic equipment | |
| CN109242913B (en) | Method, device, equipment and medium for calibrating relative parameters of collector | |
| CN110555901B (en) | Method, device, equipment and storage medium for positioning and mapping dynamic and static scenes | |
| CN108805917B (en) | Method, medium, apparatus and computing device for spatial localization | |
| CN107292949B (en) | Three-dimensional reconstruction method and device of scene and terminal equipment | |
| CN111445526B (en) | Method, device and storage medium for estimating pose of image frame | |
| CN110866496A (en) | Robot positioning and mapping method and device based on depth image | |
| CN110349212B (en) | Optimization method and device for instant positioning and map construction, medium and electronic equipment | |
| CN109461208B (en) | Three-dimensional map processing method, device, medium and computing equipment | |
| CN110111388B (en) | Three-dimensional object pose parameter estimation method and visual equipment | |
| CN109543680B (en) | Method, apparatus, device, and medium for determining location of point of interest | |
| US20210056715A1 (en) | Object tracking method, object tracking device, electronic device and storage medium | |
| CN107886048A (en) | Method for tracking target and system, storage medium and electric terminal | |
| CN112734852A (en) | Robot mapping method and device and computing equipment | |
| EP3028252A1 (en) | Rolling sequential bundle adjustment | |
| CN113763466B (en) | Loop detection method and device, electronic equipment and storage medium | |
| CN111368657A (en) | Cow face identification method and device | |
| CN112258647B (en) | Map reconstruction method and device, computer readable medium and electronic equipment | |
| CN113902932A (en) | Feature extraction method, visual positioning method and device, medium and electronic equipment | |
| CN112907657A (en) | Robot repositioning method, device, equipment and storage medium | |
| CN110849380B (en) | Map alignment method and system based on collaborative VSLAM | |
| CN109816726B (en) | Visual odometer map updating method and system based on depth filter | |
| US20220405954A1 (en) | Systems and methods for determining environment dimensions based on environment pose | |
| CN114429631B (en) | Three-dimensional object detection method, device, equipment and storage medium | |
| CN113763468B (en) | Positioning method, device, system and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |