CN113284224B - Automatic mapping method and device based on simple codes, positioning method and equipment - Google Patents
Automatic mapping method and device based on simple codes, positioning method and equipment Download PDFInfo
- Publication number
- CN113284224B CN113284224B CN202110421464.XA CN202110421464A CN113284224B CN 113284224 B CN113284224 B CN 113284224B CN 202110421464 A CN202110421464 A CN 202110421464A CN 113284224 B CN113284224 B CN 113284224B
- Authority
- CN
- China
- Prior art keywords
- points
- dimensional
- frame
- characteristic
- key frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 97
- 238000013507 mapping Methods 0.000 title claims abstract description 33
- 238000005457 optimization Methods 0.000 claims abstract description 17
- 238000004422 calculation algorithm Methods 0.000 claims description 16
- 238000006073 displacement reaction Methods 0.000 claims description 16
- 238000013519 translation Methods 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 5
- 238000000605 extraction Methods 0.000 claims description 5
- 238000010276 construction Methods 0.000 abstract description 9
- 230000000007 visual effect Effects 0.000 abstract description 2
- 238000012545 processing Methods 0.000 description 10
- 238000004364 calculation method Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 239000003550 marker Substances 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 6
- 239000011159 matrix material Substances 0.000 description 5
- 230000000875 corresponding effect Effects 0.000 description 4
- 238000000354 decomposition reaction Methods 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 238000005286 illumination Methods 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000005295 random walk Methods 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000010408 sweeping Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
- G06V10/464—Salient features, e.g. scale invariant feature transforms [SIFT] using a plurality of salient features, e.g. bag-of-words [BoW] representations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Geometry (AREA)
- Multimedia (AREA)
- Computer Graphics (AREA)
- Software Systems (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Image Analysis (AREA)
Abstract
The application provides an automatic mapping method and device based on simple codes, a positioning method and device, and relates to the field of visual positioning and navigation of robots. The automatic mapping method based on the simple codes comprises the following steps: acquiring a series of picture frames containing simple code mark points; extracting features of the series of picture frames to obtain feature points; performing feature point association on the series of picture frames; determining a key frame in the series of picture frames; and performing three-dimensional reconstruction based on global optimization on the key frame and the associated feature points to obtain three-dimensional coordinates of the feature points. According to the technical scheme provided by the embodiment of the application, the automatic drawing construction is adopted, so that the cost is low; the simple codes are set, the position and density limitations are avoided, and the usability is high; based on the scheme of center coordinates relative to other based on corner coordinates, the precision can be improved.
Description
Technical Field
The application relates to the field of visual positioning and navigation of robots, in particular to an automatic drawing method and device based on simple codes, a positioning method and device.
Background
Currently, indoor robots are increasingly applied to various scenes: industrial plants, shopping malls, families, etc. These robots are also becoming more and more intelligent, and autonomous positioning and navigation is a fundamental technology that makes robots more intelligent.
However, vision-based robotic autonomous positioning and navigation techniques are still not mature. Vision autonomous positioning and navigation techniques can be categorized into two main categories, artificial and natural, depending on the characteristics they are used with. The artificial feature based solution requires labeling in the scene. The tag is easy to damage and high in maintenance cost, and more importantly, the three-dimensional position of the tag is complicated to measure. The problem of the scheme based on natural features is that the scene requirement is strict, and the robustness for scenes with weak graininess, large illumination intensity change and the like is poor.
Therefore, a convenient and easy-to-use mapping and positioning method is required to be developed.
Disclosure of Invention
The application aims to provide an automatic drawing construction method and device based on simple codes, a positioning method and device, which can realize automatic drawing construction and improve the precision of the existing drawing construction based on mark points.
According to an aspect of the present application, an automatic mapping method based on simple codes is provided, including:
acquiring a series of picture frames containing simple code mark points;
extracting features of the series of picture frames to obtain feature points;
Performing feature point association on the series of picture frames;
determining a key frame in the series of picture frames;
And performing three-dimensional reconstruction based on global optimization on the key frame and the associated feature points to obtain three-dimensional coordinates of the feature points.
According to some embodiments, the foregoing method further comprises: the simple code mark points are randomly arranged on the space surface.
According to some embodiments, the foregoing method further comprises: extracting features of the series of pictures, including: and extracting the center coordinates of the series of picture frames to obtain the center two-dimensional coordinate values of the simple code mark points as characteristic points.
According to some embodiments, the foregoing method further comprises: performing feature point association on the series of picture frames, including:
performing first feature point association on the newly acquired picture frame and the previous picture frame;
performing second characteristic point association on the newly acquired picture frame and the three-dimensional point cloud map, wherein the three-dimensional point cloud map is generated by three-dimensional coordinates of the associated characteristic points;
Classifying the feature points based on the results of the first feature point association and the second feature point association, and forming a feature point association record;
And maintaining a three-dimensional point cloud map of the feature points.
According to some embodiments, the foregoing method further comprises: the feature points are classified into continuous tracking points, lost tracking points, and new incoming points.
According to some embodiments, the foregoing method further comprises: the feature point association record includes:
The pixel coordinate of each feature point in the frame is < t x,ty >, and the frame number and feature point association relation of the frame in which each feature point is.
According to some embodiments, the maintaining a three-dimensional point cloud map includes:
and continuously optimizing and forming a three-dimensional point cloud map according to the feature points of the newly acquired picture frames which are continuously added.
According to some embodiments, the foregoing method further comprises: determining a key frame in the series of picture frames, comprising:
Determining a newly acquired picture frame meeting the following conditions as a key frame:
dt > t1 or dt > t2& dr > d1 or dr > d2& df > N
Wherein:
dt is the displacement of the newly acquired picture frame relative to the last key frame,
T1 is a first displacement threshold, t2 is a second displacement threshold,
Dr is the rotation angle value of the newly acquired picture frame relative to the previous key frame,
D1 is a first threshold value of the rotation angle, d2 is a second threshold value of the rotation angle,
Df is the interval frame number of the newly acquired picture frame relative to the previous key frame, and N is the interval frame number threshold.
According to some embodiments, the foregoing method further comprises: performing three-dimensional reconstruction based on global optimization on the key frames and the associated feature points, wherein the three-dimensional reconstruction comprises the following steps:
and performing three-dimensional reconstruction based on global optimization on the key frames and the associated feature points by using an SFM algorithm.
According to some embodiments, the foregoing method further comprises: performing three-dimensional reconstruction based on global optimization on the key frames and the associated feature points by using an SFM algorithm, wherein the three-dimensional reconstruction comprises the following steps:
calculating the relative rotation amount between any two key frames;
calculating the absolute rotation quantity of each key frame in a coordinate system;
The relative translation amount between any two key frames is calculated;
calculating the absolute translation amount of each key frame in a coordinate system;
Triangularizing each characteristic point to obtain three-dimensional coordinates of the characteristic points;
and optimizing the three-dimensional coordinates of each characteristic point by using a global beam adjustment method.
According to an aspect of the present application, a positioning method includes: acquiring a series of picture frames containing simple code mark points; and positioning by utilizing the three-dimensional coordinates of the feature points obtained by the automatic mapping method.
According to an aspect of the present application, a positioning method includes: simple code mark points are randomly arranged on the space surface.
According to one aspect of the application, an apparatus comprises:
the acquisition module is used for acquiring a series of picture frames containing simple code mark points;
the extraction module is used for carrying out feature extraction on the series of picture frames to obtain feature points;
the characteristic association module is used for carrying out characteristic point association on the series of picture frames;
A key frame module for determining a key frame in the series of picture frames;
And the reconstruction module is used for carrying out three-dimensional reconstruction based on global optimization on the key frames and the associated feature points to obtain three-dimensional coordinates of the feature points.
According to an aspect of the present application, an electronic device includes: a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the method of any of the above when executing the computer program.
According to the embodiment of the application, the simple code mark points are set, and a series of picture frames containing the mark points are obtained, so that the automatic mode is adopted for drawing, and the drawing construction cost is low, the efficiency is high and the precision is high.
According to some embodiments, by arranging simple codes on the space surface, the space surface has no position and density limitation and is high in usability;
According to some embodiments, the accuracy may be improved based on the center coordinates relative to other corner-based schemes.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application as claimed.
Drawings
The above and other objects, features and advantages of the present application will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings. The drawings described below are only some of the embodiments of the present application and are not intended to limit the present application.
Fig. 1 shows a flowchart of a simplex code-based automatic mapping method according to an embodiment of the present application.
Fig. 2 illustrates an original picture taken by a camera according to an exemplary embodiment.
Fig. 3 illustrates a picture of a processed extraction center according to an exemplary embodiment.
Fig. 4 shows a resulting graph of feature point associations between two picture frames according to an example embodiment.
FIG. 5 illustrates a globally optimized three-dimensional reconstruction procedure according to an example embodiment.
Fig. 6 shows a block diagram of a simplex code based automatic mapping apparatus according to an exemplary embodiment.
Fig. 7 shows a block diagram of an electronic device, according to an example embodiment.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments can be embodied in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and thus a repetitive description thereof will be omitted.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the application may be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the application.
The block diagrams depicted in the figures are merely functional entities and do not necessarily correspond to physically separate entities. That is, the functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
The flow diagrams depicted in the figures are exemplary only, and do not necessarily include all of the elements and operations/steps, nor must they be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the order of actual execution may be changed according to actual situations.
It will be understood that, although the terms first, second, third, etc. may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one element from another element. Accordingly, a first component discussed below could be termed a second component without departing from the teachings of the present inventive concept. As used herein, the term "and/or" includes any one of the associated listed items and all combinations of one or more.
Those skilled in the art will appreciate that the drawings are schematic representations of example embodiments and that the modules or flows in the drawings are not necessarily required to practice the application and therefore should not be taken to limit the scope of the application.
In the robot positioning scheme, a mapping positioning scheme based on ArUco codes is commonly used. Such ArUco codes carry code sequence ID information by encoding and decoding of the information. The core construction steps are as follows: (1) Randomly pasting the printed ArUco codes on flat positions such as ceilings, walls and the like of the indoor space; (2) The hand-held or airborne camera moves slowly, so that the camera can traverse all ArUco codes; (3) The mapping algorithm comprises the steps of calculating ID information of each ArUco codes, calculating relative position information of any two ArUco codes and calculating absolute positions of all ArUco codes by using an error average algorithm.
The map construction positioning scheme based on ArUco codes needs to be pasted with ArUco codes in space, is not attractive, and the calculated map accuracy of ArUco codes and the positioning accuracy according to ArUco codes are not quite high, so that the map construction positioning can not be performed in places with dark illumination.
Another scheme for creating and locating a map based on a marker point is to use a simple code, i.e. a marker point without a pattern feature, generally using a circular code or a square code (or other shapes). The circle code generally utilizes a method of fitting an elliptic equation to extract the center of a circle of the circle code as a characteristic point. Square codes typically use a fitted straight line equation and then calculate the intersection of two straight lines as the feature point.
In the existing scheme based on the mark points, if a simple code is adopted, manual measurement is needed for map construction. In practice, the tool used for manual measurement is typically a total station. The total station is a high-tech measuring instrument integrating light, mechanical and electric, and can measure horizontal angle, vertical angle, distance (inclined distance, flat distance) and the like. In actual operation, the marked points are typically measured manually with a total station. Specifically, the center of each marker point is aligned in turn using a total station, and then the spatial coordinates thereof are measured. Therefore, when the number of marking points is large, the manual measurement is time-consuming and labor-consuming.
Therefore, the application provides a method for constructing a map by setting simple code mark points and acquiring a series of picture frames containing the mark points in an automatic mode, which can automatically and quickly construct a high-precision simple code map.
Exemplary embodiments of the present application are described in detail below with reference to the accompanying drawings.
Fig. 1 shows a flowchart of a simplex code-based automatic mapping method according to an embodiment of the present application.
Referring to fig. 1, at S301, a series of picture frames containing simplex code mark points are acquired.
According to an example embodiment, referring to fig. 2, a series of picture frames containing simplex code mark points may be acquired by a camera. For example, a picture frame may be acquired by an industrial camera with a natural light filter on the lens.
According to some embodiments, the simplex code mark points may be disposed (e.g., adhered) on the indoor ceiling for positioning, and according to other embodiments, the simplex code mark points may be adhered to other space surfaces suitable for positioning, such as the floor or wall. The simple code marking points are not limited to indoor positioning, but can also be used for outdoor positioning, such as hovering or landing positioning of an outdoor delivery unmanned aerial vehicle.
According to some embodiments, the simplex code mark points are made of a material cloth, but the present application is not limited thereto. It will be readily appreciated that the simplex code mark points may be made of any suitable material. Alternatively, the plain code marking points may be made of reflective material, such as cut to coin or bottle cap size.
According to an example embodiment, the simplex code mark points are circular. Due to the symmetry of the circle, better accuracy can be achieved in automatic mapping and positioning. However, the present application is not limited thereto, and alternatively, the simple code mark points may be elliptical, regular polygon, etc.
According to an example embodiment, the density of simplex code mark points posted on a spatial surface may be, but is not limited to, three to five per square meter.
According to an exemplary embodiment of the application, for a certain indoor scene, the camera starts to acquire pictures from the outer ring, and moves from the outside to the inside round to the middle of the scene, similar to a spiral picture acquisition path.
Alternatively, according to some embodiments, the cameras may collect in a spiral path from inside to outside, or along a broken line or straight line path, or a random walk.
In S303, feature extraction is performed on the series of picture frames to obtain feature points.
According to an exemplary embodiment, the center coordinates are extracted from a series of picture frames, for example, by blurring, corroding, expanding, etc., to obtain a clearer circular mark point which is approximately circular. For example, the circle center coordinates may be extracted for the circle marker points by fitting based on a circular equation, as described later with reference to fig. 2 and 3.
At S305, feature point association is performed on the series of picture frames.
According to an example embodiment, feature points obtained from the picture frames are classified by feature point association of the series of picture frames. For example, by sequentially processing a series of picture frames, feature points can be classified as: continuous tracking points, lost tracking points, and new coming points. The continuous tracking point is a feature point appearing in the previous frame or the previous frames, and appears in the current frame. The lost tracking point is a feature point that appears in the previous frame and does not appear in the current frame. The new coming point is a feature point that appears in the picture frame for the first time.
According to an example embodiment, associating the series of picture frames with feature points includes associating a newly acquired picture frame with a previous picture frame with a first feature point and associating the newly acquired picture frame with the three-dimensional point cloud map with a second feature point, while maintaining a three-dimensional point cloud map of feature points.
According to an example embodiment, the associating of the newly acquired picture frame with the previous picture frame by the first feature point may employ a triangle matching method, which may include: and then, carrying out triangle matching on the two picture frames according to the intersection ratio and the angle of the triangles, thereby obtaining the matching association relation of the characteristic points between the two frames (see, for example, fig. 4). This is well known in the art and will not be described in detail.
According to an example embodiment, in this step, a three-dimensional point cloud map of feature points is also generated and maintained. The three-dimensional point cloud map of the feature points is not high in accuracy. And according to the obtained picture containing the single code mark points, a three-dimensional point cloud map is constructed and output in real time by a characteristic point triangularization method, and if the new coming point is judged by the continuously added characteristic points of the newly obtained picture frame, the map is added. Further, the three-dimensional coordinates of the optimized new feature points can be obtained through local optimization (local bundle adjustment), so that a three-dimensional point cloud map is formed.
According to an example embodiment, performing the second feature point association on the newly acquired picture frame and the feature point three-dimensional point cloud map may use a projection method, which may include: and estimating the pose of the current frame by using the uniform motion model, projecting the three-dimensional point cloud map of the characteristic points onto the current frame, and determining the association relationship between the characteristic points and the three-dimensional point cloud of the characteristic points according to the rule of similar distances. This is well known in the art and will not be described in detail.
According to an example embodiment, the feature points are classified based on the results of the above-described first feature point association and second feature point association, and a feature point association record is formed. For example, the data structure of the association record may be: three-dimensional coordinates < x, y, z > of the feature points; two-dimensional pixel coordinates of feature points in a picture frame < t x,ty >; and the frame number of the picture frame where each characteristic point is located. The results of these feature associations may be recorded in the storage unit of the electronic device in the form of, but not limited to, text files.
At S307, a key frame in the series of picture frames is determined.
Key frames are common practice in the art and function to: typically, the camera captures too many picture frames per second, so that a certain key frame algorithm can be used to select the picture frames with more beneficial information, so that the subsequent algorithm can process more quickly and the processing result is more robust.
According to an example embodiment of the application, the currently newly acquired picture frame satisfying the following conditions is determined as a key frame:
dt > t1 or dt > t2& dr > d1 or dr > d2& df > N
Wherein:
dt is the displacement of the current frame relative to the previous key frame and is related to the distance unit taken at the time of system initialization. t1 is a first displacement threshold, t2 is a second displacement threshold, typically t1 takes 3.5 units and t2 takes 1.5 units.
Dr is the rotation angle value of the current frame relative to the previous key frame, and the unit is rad. d1 is a first rotation angle threshold, and d2 is a second rotation angle threshold.
Df is the number of frames spaced from the current frame relative to the previous key frame. N is the threshold of the interval frame number, and the value is related to the frame rate of the camera.
At S309, a three-dimensional reconstruction based on global optimization is performed on the keyframes and the associated feature points.
According to an example embodiment, the key frames and the associated feature points are subjected to a three-dimensional reconstruction based on global optimization using an SFM algorithm. The SFM algorithm (Structure from motion, motion restoration structure algorithm) can be classified into delta type (INCREMENTAL SFM) and global type (global Sfm).
According to an example embodiment, the key frames and the associated feature points are three-dimensionally reconstructed using global optimization SFM, as described later with reference to fig. 5.
By three-dimensionally reconstructing the feature points that have been correlated, more accurate three-dimensional coordinates of the feature points can be obtained, so that more accurate positioning results can be obtained when a robot or the like is positioned using these three-dimensional coordinates.
FIG. 2 illustrates an original picture taken by a camera according to an example embodiment; fig. 3 illustrates a picture in which center coordinates are extracted after processing an original picture according to an exemplary embodiment.
Referring to fig. 2, the picture is a binarized black-and-white picture, for example, 8BIT. The picture contains a circle or an ellipse. If not, certain pretreatment such as morphological treatment, space template treatment and the like can be carried out. For example, from the perspective of fig. 2, the circular or elliptical shape is in fact not ideal. At this time, the compasses Fan Huacheng on the figure can be used for standard circles, so that the found circle center has higher credibility.
Referring to fig. 3, according to an example embodiment, a fitting calculation may be performed using a "least squares of radial error preprocessing" as known in the art, thereby extracting the center coordinates.
Specifically, the algorithm may be divided into three parts: (1) Finding out an initial circle center (the circle center is used as a reference for judging radial errors, and a result obtained by a pixel-by-pixel processing method can be directly taken in the method); (2) finding the pixel coordinates for fitting. Firstly, the pixels must be round edge single pixels (namely, at most 3 pixels in eight neighborhoods), and then screening is carried out, wherein invalid boundary points and boundary points with large errors are removed mainly through radial error range judgment; (3) And fitting the circle center by using a least square method according to the processed boundary points.
Of course, the method of extracting the center coordinates is not limited thereto. For example, other methods, such as machine learning methods, may be used to identify the marker points and determine the center.
Fig. 4 shows a resulting graph of feature point associations between two picture frames according to an example embodiment.
Referring to fig. 4, after feature point association is performed on two frames by using a triangle matching method, a result graph of feature point matching between the two frames is obtained according to some embodiments. As can be seen from fig. 4, the two matching associated feature points have the same feature point number. The feature point No. 22 in the left picture frame of fig. 4 does not have an associated feature point in the right picture frame.
FIG. 5 illustrates a globally optimized three-dimensional reconstruction procedure according to an example embodiment.
The three-dimensional reconstruction method according to an example embodiment includes: (1) resolving the relative rotation amount between any two key frames; (2) Calculating the absolute rotation amount of each key frame in the absolute coordinates; (3) resolving the relative amount of translation between any two key frames; (4) Calculating the absolute translation of each key frame in absolute coordinates; (5) Triangularizing each characteristic point to obtain three-dimensional coordinates of the characteristic points; (6) And optimizing the three-dimensional coordinates of each characteristic point by using a global beam adjustment method.
Referring to fig. 5, at S701, the relative rotation amount between any two key frames is calculated.
The calculation of the relative rotation amount refers to the calculation of the rotation between any two of all key frames. According to some embodiments, a five-point essential matrix decomposition based on RANSAC (Random Sample Consensus) robust estimation algorithm may be employed. Specifically, within the RANSAC framework, five pairs of feature points x i are randomly selected from all the matching pairs at a time, satisfyingWherein < x i,xj > is the corresponding feature point in the left and right images, respectively. After the essential matrix E is calculated, since e=t × R, the relative rotation R between two frames can be obtained by QR decomposition.
According to some embodiments, the two image matching feature points are often far from 5 pairs, and a seven-point basic matrix decomposition method based on RANSAC can be selected. Randomly extracting N samples from the characteristic point set, wherein each sample contains 7 pairs of characteristic points, calculating N basic matrixes of the N samples by using a seven-point algorithm, selecting the basic matrix with the minimum error, and then calculating to obtain the relative rotation R.
In S703, the absolute rotation amount of each key frame in the coordinate system is calculated.
According to some embodiments, the absolute rotation R i, i=1..n, of all frames is calculated from the relative rotation R ij between any two frames. Due to the presence ofThe absolute rotation can be calculated very quickly, for example, according to the least squares method.
According to other embodiments, an error metric solution with a L1 or L2 norm of R ij may also be selected.
At S705, the relative amount of translation between any two key frames is resolved.
The calculation of the relative translation amount refers to calculating a displacement value t ij between any two frames. According to some embodiments, the RANSAC framework may be employed. In each RANSAC calculation model, for example, calculation of the displacement value is performed by a trifocal method.
In S707, an absolute translation of each key frame in the coordinate system is calculated.
According to some embodiments, knowing the relative displacement t ij between any two frames in the above steps requires solving for the absolute displacement t i for each frame, i=0. The solution can be performed according to the equation t i-tj＝sijtij, which is written by the least squares method, unlike the above steps, which increases the solution of the scale s ij between two frames.
In S709, each feature point is triangulated to obtain its three-dimensional coordinates.
According to some embodiments, for each feature point, the three-dimensional coordinates of this feature point may be obtained by a trigonometric method calculation, based on the pose of all its associated keyframes.
At S711, the three-dimensional coordinates of each feature point are optimized using the global beam adjustment method.
According to some embodiments, global pose and three-dimensional points are generally optimized for improved accuracy of computation. For example, a global beam adjustment method, also known as beam adjustment method (Bundle Adjustment, BA), may be employed.
According to the beam adjustment method, for any three-dimensional point P in a scene, light rays emitted from the optical center of a camera corresponding to each view and passing through a pixel corresponding to P in an image are intersected with the point P, and for all three-dimensional points, a plurality of beams are formed; in the actual process, because noise and the like exist, each ray is almost impossible to converge with one point, so that in the solving process, the information to be solved needs to be continuously adjusted (adjustment) to enable the final ray to meet the point P.
Specifically: it is conceivable that there are many rays, each of which connects the camera optical center, the three-dimensional object point, the projection of the object point on the photo, this model (for simplicity, without considering the effect of distortion temporarily) is described by a P matrix, i.e. z=px, Z being the object point coordinate, since the presence of error PX is not absolutely equal to Z, when we just have to adjust (R, t, X) such that the cost of Z-PX 2 is minimal. Since the object point can be seen as one beam from each pose, the BA is (R, t, X) where the cost of optimizing all beams overall is minimal.
According to some embodiments, the present application further provides a positioning method, by acquiring a series of picture frames containing simple code mark points, and positioning by using the three-dimensional coordinates of the feature points obtained according to the foregoing automatic mapping method.
For example, the three-dimensional point cloud map of the feature points obtained by the automatic mapping method is stored in a storage module of the electronic device. When robots such as an unmanned plane, a sweeping robot, an unmanned truck and the like are positioned, estimating the pose of a rough camera by using a uniform model according to each picture frame acquired in real time; and matching the picture feature points with the map three-dimensional points by using the pose.
Specifically, three-dimensional points of a map are projected onto a current frame of picture in sequence, two-dimensional feature points closest to the map are searched for as matching results of the three-dimensional points of the map, and after a group of matching results are obtained, a Ransac +P3P method can be utilized to screen possible incorrect matching.
After the matching result between the two-dimensional feature points and the three-dimensional map points is obtained, the accurate real-time camera pose can be obtained through calculation, for example, a PnP algorithm is used, and therefore positioning is completed.
PnP (PERSPECTIVE-n-Point) is a method of solving 3D to 2D Point-to-Point motion. Since the degree of freedom of the pose is 6, a minimum of 3 point pairs are required to estimate camera motion using PnP. The 3D position of the feature points may be determined by triangulation or a depth map of the RGB-D camera.
There are many solutions to the PnP problem, including P3P, direct Linear Transformation (DLT), EPnP (Efficient PnP), UPnP, etc. for estimating pose with 3 pairs of points. In addition, a nonlinear optimization mode can be used for constructing a least square problem and solving the least square problem iteratively, namely a beam adjustment method (Bundle Adjustment, BA).
In popular terms, the PnP problem is how to calculate the pose of a camera by using the real coordinates of N spatial points in a known world coordinate system and the projection of these spatial points on an image. The known quantity is the real coordinates and image coordinates of the spatial points and the one-to-one matching relationship between them, and the unknown quantity (solution quantity) is the pose of the camera.
Thus, the robot and the like can be positioned by using the three-dimensional coordinates of the feature points obtained by the automatic mapping method through the known algorithms such as PnP and the like.
According to some embodiments, the present application further provides a positioning method, by randomly setting simple code mark points on a space surface, so as to automatically construct a map for positioning by using the foregoing method.
For example, the positioning method may be applied in an industrial plant. According to some embodiments, simple codes are posted on the ceiling inside a factory, cameras are installed on the ground robots, and the positioning and navigation functions of the ground robots can be achieved by performing automatic mapping by the mapping method.
As another example, the positioning method may also be applied in a mall or other indoor location. According to some embodiments, simple codes are posted in a market VR experience room and a home VR entertainment room, and a camera is installed on VR equipment, so that positioning and navigation functions can be realized.
According to some embodiments, the positioning method may also be applied in parking lots. For example, a simple code is posted on the ceiling of a parking lot, and a robot or an automobile is provided with a camera, so that the autonomous positioning and navigation functions of the robot or the automobile can be realized.
It should be clearly understood that the present application describes how to make and use particular examples, but the present application is not limited to any details of these examples. Rather, these principles can be applied to many other embodiments based on the teachings of the present disclosure.
Those skilled in the art will appreciate that all or part of the steps implementing the above described embodiments are implemented as a computer program executed by a CPU. When executed by a CPU, performs the functions defined by the above-described method provided by the present application. The program of (a) may be stored in a computer readable storage medium, which may be a read-only memory, a magnetic disk or an optical disk, or the like.
Furthermore, it should be noted that the above-described figures are merely illustrative of the processes involved in the method according to the exemplary embodiment of the present application, and are not intended to be limiting. It will be readily appreciated that the processes shown in the above figures do not indicate or limit the temporal order of these processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, for example, among a plurality of modules.
Those skilled in the art will readily appreciate from the disclosure of example embodiments that the methods, apparatus and devices for simplex code based automatic mapping according to embodiments of the present application may have at least one or more of the following advantages.
According to an example embodiment, the cost is very low by adopting the automatic mapping technical scheme of the application.
According to the embodiment, the technical scheme of the application adopts the simple code which is pasted with the invisible code, is more attractive compared with the code with the pattern characteristic, has no position and density limitation, and has high usability.
According to the embodiment, the technical scheme of the application adopts the center coordinate, and compared with other schemes, the accuracy is improved based on the corner point coordinates.
The following describes apparatus embodiments of the present application that may be used to perform method embodiments of the present application. For details not disclosed in the embodiments of the device according to the application, reference is made to the embodiments of the method according to the application.
Fig. 6 shows a block diagram of a simplex code based automatic mapping apparatus according to an exemplary embodiment.
The apparatus shown in fig. 6 may perform the aforementioned automatic mapping method based on simple codes according to an embodiment of the present application.
As shown in fig. 6, the automatic mapping apparatus based on simple codes may include: the device comprises an acquisition module 810, an extraction module 820, a feature association module 830, a key frame module 840 and a reconstruction module 850.
Referring to fig. 6 and to the foregoing description, an acquisition module 810 is used to acquire a series of pictures containing simple code marker points.
And the extracting module 820 is used for extracting the characteristics of the series of pictures.
The feature association module 830 is configured to perform feature association on the series of pictures to obtain a continuous tracking point, a lost tracking point, a new incoming point, and a feature association record.
A key frame module 840 for determining key frames in the series of pictures.
And a reconstruction module 850, configured to perform a three-dimensional reconstruction based on global optimization on the key frame and the associated feature, so as to obtain a three-dimensional coordinate of the simple code mark point.
The apparatus performs functions similar to those provided above, and other functions may be found in the foregoing description and will not be repeated here.
Fig. 7 shows a block diagram of an electronic device, according to an example embodiment.
An electronic device 200 according to this embodiment of the application is described below with reference to fig. 7. The electronic device 200 shown in fig. 7 is only an example and should not be construed as limiting the functionality and scope of use of embodiments of the present application.
As shown in fig. 7, the electronic device 200 is in the form of a general purpose computing device. The components of the electronic device 200 may include, but are not limited to: at least one processing unit 210, at least one memory unit 220, a bus 230 connecting the different system components (including the memory unit 220 and the processing unit 210), a display unit 240, and the like.
In which a storage unit stores program code that can be executed by the processing unit 210 such that the processing unit 210 performs the methods according to various exemplary embodiments of the present application described in the present specification. For example, the processing unit 210 may perform the methods as shown in fig. 1, 5.
The storage unit 220 may include readable media in the form of volatile storage units, such as Random Access Memory (RAM) 2201 and/or cache memory 2202, and may further include Read Only Memory (ROM) 2203.
The storage unit 220 may also include a program/utility 2204 having a set (at least one) of program modules 2205, such program modules 2205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.
Bus 230 may be a bus representing one or more of several types of bus structures including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
The electronic device 200 may also communicate with one or more external devices 300 (e.g., keyboard, pointing device, bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device 200, and/or any device (e.g., router, modem, etc.) that enables the electronic device 200 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 250. Also, the electronic device 200 may communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet, through a network adapter 260. Network adapter 260 may communicate with other modules of electronic device 200 via bus 230. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with electronic device 200, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.
From the above description of embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or may be implemented in software in combination with the necessary hardware. The technical solution according to the embodiment of the present application may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a usb disk, a mobile hard disk, etc.) or on a network, and includes several instructions to cause a computing device (may be a personal computer, a server, or a network device, etc.) to perform the above-described method according to the embodiment of the present application.
The software product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, with readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A readable storage medium may also be any readable medium that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).
Those skilled in the art will appreciate that the modules may be distributed throughout several devices as described in the embodiments, and that corresponding variations may be implemented in one or more devices that are unique to the embodiments. The modules of the above embodiments may be combined into one module, or may be further split into a plurality of sub-modules.
From the above description of embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the present application may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a U-disk, a mobile hard disk, etc.) or on a network, and includes several instructions to cause a computing device (may be a personal computer, a server, a mobile terminal, or a network device, etc.) to perform the method according to the embodiments of the present application.
The exemplary embodiments of the present application have been particularly shown and described above. It is to be understood that this application is not limited to the precise arrangements, instrumentalities and instrumentalities described herein; on the contrary, the application is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
Claims (11)
1. An automatic mapping method based on simple codes is characterized by comprising the following steps:
acquiring a series of picture frames containing simple code mark points;
Extracting center coordinates of the series of picture frames to obtain center two-dimensional coordinate values of the simple code mark points as characteristic points;
Performing first characteristic point association on a newly acquired picture frame and a previous picture frame, performing second characteristic point association on the newly acquired picture frame and a three-dimensional point cloud map, wherein the three-dimensional point cloud map is generated by three-dimensional coordinates of the associated characteristic points, classifying the characteristic points based on the results of the first characteristic point association and the second characteristic point association, forming a characteristic point association record, and maintaining the three-dimensional point cloud map;
determining a key frame in the series of picture frames;
Performing three-dimensional reconstruction based on global optimization on the key frame and the associated characteristic points to obtain three-dimensional coordinates of the characteristic points,
Wherein, confirm the newly acquired picture frame that satisfies the following condition as the key frame: dt > t1 or dt > t2& dr > d1 or dr > d2& df > N, dt is the displacement of the newly acquired picture frame relative to the previous key frame, t1 is a first displacement threshold, t2 is a second displacement threshold, dr is the rotation angle value of the newly acquired picture frame relative to the previous key frame, d1 is a first rotation angle threshold, d2 is a second rotation angle threshold, df is the number of interval frames of the newly acquired picture frame relative to the previous key frame, and N is an interval frame number threshold.
2. The automatic mapping method according to claim 1, comprising:
the simple code mark points are randomly arranged on the space surface.
3. The automatic mapping method according to claim 1, wherein classifying the feature points comprises: the feature points are classified into continuous tracking points, lost tracking points, and new incoming points.
4. The automatic mapping method according to claim 1, wherein the feature point association record includes:
The pixel coordinate of each feature point in the frame is < t x,ty >, and the frame number and feature point association relation of the frame in which each feature point is.
5. The automated mapping method of claim 1, wherein maintaining the three-dimensional point cloud map comprises:
and continuously optimizing and forming a three-dimensional point cloud map according to the feature points of the newly acquired picture frames which are continuously added.
6. The automatic mapping method according to claim 1, wherein performing a three-dimensional reconstruction based on global optimization of the keyframes and the associated feature points comprises:
and performing three-dimensional reconstruction based on global optimization on the key frames and the associated feature points by using an SFM algorithm.
7. The automatic mapping method of claim 6, wherein performing a global optimization-based three-dimensional reconstruction of the keyframes and associated feature points using an SFM algorithm comprises:
calculating the relative rotation amount between any two key frames;
calculating the absolute rotation quantity of each key frame in a coordinate system;
The relative translation amount between any two key frames is calculated;
calculating the absolute translation amount of each key frame in a coordinate system;
Triangularizing each characteristic point to obtain three-dimensional coordinates of the characteristic points;
and optimizing the three-dimensional coordinates of each characteristic point by using a global beam adjustment method.
8. A positioning method, comprising:
acquiring a series of picture frames containing simple code mark points;
The positioning is performed using the three-dimensional coordinates of the feature points obtained by the automatic mapping method according to any one of claims 1 to 7.
9. A positioning method, comprising:
randomly setting simple code mark points on the space surface;
The positioning is performed by using the three-dimensional coordinates of the feature points obtained by the automatic mapping method according to claim 1.
10. An automatic mapping device based on simple codes, which is characterized by comprising:
the acquisition module is used for acquiring a series of picture frames containing simple code mark points;
The extraction module is used for extracting the central coordinate operation of the series of picture frames to obtain the central two-dimensional coordinate value of the simple code mark point as a characteristic point;
the characteristic association module is used for carrying out first characteristic point association on a newly acquired picture frame and a previous picture frame, carrying out second characteristic point association on the newly acquired picture frame and a three-dimensional point cloud map, wherein the three-dimensional point cloud map is generated by three-dimensional coordinates of the associated characteristic points, classifying the characteristic points based on the results of the first characteristic point association and the second characteristic point association, forming a characteristic point association record, and maintaining the three-dimensional point cloud map;
A key frame module for determining a key frame in the series of picture frames;
a reconstruction module, configured to perform a three-dimensional reconstruction based on global optimization on the key frame and the associated feature points to obtain three-dimensional coordinates of the feature points,
Wherein the newly acquired picture frame satisfying the following condition is a key frame: dt > t1 or dt > t2& dr > d1 or dr > d2& df > N, dt is the displacement of the newly acquired picture frame relative to the previous key frame, t1 is a first displacement threshold, t2 is a second displacement threshold, dr is the rotation angle value of the newly acquired picture frame relative to the previous key frame, d1 is a first rotation angle threshold, d2 is a second rotation angle threshold, df is the number of interval frames of the newly acquired picture frame relative to the previous key frame, and N is an interval frame number threshold.
11. An electronic device, comprising:
Memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the method according to any of the preceding claims 1-9 when executing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110421464.XA CN113284224B (en) | 2021-04-20 | 2021-04-20 | Automatic mapping method and device based on simple codes, positioning method and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110421464.XA CN113284224B (en) | 2021-04-20 | 2021-04-20 | Automatic mapping method and device based on simple codes, positioning method and equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113284224A CN113284224A (en) | 2021-08-20 |
| CN113284224B true CN113284224B (en) | 2024-06-18 |
Family
ID=77276912
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110421464.XA Active CN113284224B (en) | 2021-04-20 | 2021-04-20 | Automatic mapping method and device based on simple codes, positioning method and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113284224B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111784775A (en) * | 2020-07-13 | 2020-10-16 | 中国人民解放军军事科学院国防科技创新研究院 | Identification-assisted visual inertia augmented reality registration method |
Family Cites Families (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20140108828A (en) * | 2013-02-28 | 2014-09-15 | 한국전자통신연구원 | Apparatus and method of camera tracking |
| WO2018214086A1 (en) * | 2017-05-25 | 2018-11-29 | 深圳先进技术研究院 | Method and apparatus for three-dimensional reconstruction of scene, and terminal device |
| CN107909612B (en) * | 2017-12-01 | 2021-01-29 | 驭势科技（北京）有限公司 | Method and system for visual instant positioning and mapping based on 3D point cloud |
| CN107784671B (en) * | 2017-12-01 | 2021-01-29 | 驭势科技（北京）有限公司 | Method and system for visual instant positioning and drawing |
| CN108225327B (en) * | 2017-12-31 | 2021-05-14 | 芜湖哈特机器人产业技术研究院有限公司 | Construction and positioning method of top mark map |
| WO2019169540A1 (en) * | 2018-03-06 | 2019-09-12 | 斯坦德机器人（深圳）有限公司 | Method for tightly-coupling visual slam, terminal and computer readable storage medium |
| CN110360999B (en) * | 2018-03-26 | 2021-08-27 | 京东方科技集团股份有限公司 | Indoor positioning method, indoor positioning system, and computer readable medium |
| CN110319772B (en) * | 2019-07-12 | 2020-12-15 | 上海电力大学 | Visual large-span distance measurement method based on unmanned aerial vehicle |
| CN110827395B (en) * | 2019-09-09 | 2023-01-20 | 广东工业大学 | Instant positioning and map construction method suitable for dynamic environment |
| CN111402412B (en) * | 2020-04-16 | 2023-06-09 | Oppo广东移动通信有限公司 | Data acquisition method and device, equipment and storage medium |
| CN111795704B (en) * | 2020-06-30 | 2022-06-03 | 杭州海康机器人技术有限公司 | Method and device for constructing visual point cloud map |
-
2021
- 2021-04-20 CN CN202110421464.XA patent/CN113284224B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111784775A (en) * | 2020-07-13 | 2020-10-16 | 中国人民解放军军事科学院国防科技创新研究院 | Identification-assisted visual inertia augmented reality registration method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113284224A (en) | 2021-08-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109521403B (en) | Parameter calibration method, device and equipment of multi-line laser radar and readable medium | |
| CN110009727B (en) | Automatic reconstruction method and system for indoor three-dimensional model with structural semantics | |
| US10592765B2 (en) | Systems and methods for generating information about a building from images of the building | |
| US9886774B2 (en) | Photogrammetric methods and devices related thereto | |
| US8896660B2 (en) | Method and apparatus for computing error-bounded position and orientation of panoramic cameras in real-world environments | |
| CN111694903A (en) | Map construction method, map construction device, map construction equipment and readable storage medium | |
| KR102526542B1 (en) | 2d vehicle localizing using geoarcs | |
| CN115345822A (en) | Automatic three-dimensional detection method for surface structure light of aviation complex part | |
| EP3274964B1 (en) | Automatic connection of images using visual features | |
| WO2016065063A1 (en) | Photogrammetric methods and devices related thereto | |
| Cosido et al. | Hybridization of convergent photogrammetry, computer vision, and artificial intelligence for digital documentation of cultural heritage-a case study: the magdalena palace | |
| US10432915B2 (en) | Systems, methods, and devices for generating three-dimensional models | |
| KR102387679B1 (en) | 3d vehicle localizing using geoarcs | |
| US20240013484A1 (en) | Method for generating roof outlines from lateral images | |
| CN110120093A (en) | Three-dimensional plotting method and system in a kind of room RGB-D of diverse characteristics hybrid optimization | |
| Kwiatek et al. | Immersive photogrammetry in 3D modelling | |
| CN114677435A (en) | Point cloud panoramic fusion element extraction method and system | |
| Lehtola et al. | Indoor 3D: Overview on scanning and reconstruction methods | |
| CN115546422A (en) | Building three-dimensional model construction method and system and electronic equipment | |
| CN113822996B (en) | Pose estimation method and device for robot, electronic device and storage medium | |
| Corral-Soto et al. | 3D town: the automatic urban awareness project | |
| CN113284224B (en) | Automatic mapping method and device based on simple codes, positioning method and equipment | |
| CN112381873A (en) | Data labeling method and device | |
| CN116229247A (en) | Indoor scene semantic segmentation method, device, equipment and medium | |
| Bartczak et al. | Extraction of 3D freeform surfaces as visual landmarks for real-time tracking |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |