CN109461208A - Three-dimensional map processing method, device, medium and calculating equipment - Google Patents
Three-dimensional map processing method, device, medium and calculating equipment Download PDFInfo
- Publication number
- CN109461208A CN109461208A CN201811363016.3A CN201811363016A CN109461208A CN 109461208 A CN109461208 A CN 109461208A CN 201811363016 A CN201811363016 A CN 201811363016A CN 109461208 A CN109461208 A CN 109461208A
- Authority
- CN
- China
- Prior art keywords
- data
- dimensional
- dimensional map
- map
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/05—Geographic models
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Remote Sensing (AREA)
- Computer Graphics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Processing Or Creating Images (AREA)
Abstract
Embodiments of the present invention provide a kind of three-dimensional map processing method, the initial map datum including obtaining target scene, wherein estimated state data when initial map datum includes at least raw image data and acquires original image；The three-dimensional map with true scale corresponding with target scene is constructed according to raw image data included in initial map datum and estimated state data；The pose data of position data and the three-dimensional point corresponding image to the three-dimensional point in the three-dimensional map with true scale optimize respectively, the three-dimensional map with true scale after being rebuild.It is optimized respectively by the pose data of position data and the three-dimensional point corresponding image to the three-dimensional point in the three-dimensional map with true scale, allows to improve the precision of three-dimensional map.In addition, embodiments of the present invention additionally provide a kind of three-dimensional map processing unit, a kind of medium and a kind of calculating equipment.
Description
Technical field
Embodiments of the present invention are related to data processing field, more specifically, embodiments of the present invention are related to dimensionally
Figure processing method, device, medium and calculating equipment.
Background technique
Background that this section is intended to provide an explanation of the embodiments of the present invention set forth in the claims or context.Herein
Description recognizes it is the prior art not because not being included in this section.
Currently, positioning and build diagram technology (SLAM, Simultaneous Localization And Mapping) simultaneously is
Refer to according to the camera and the sensors such as Inertial Measurement Unit (IMU, Inertial Measurement Unit) on mobile terminal
The map of the data building mobile terminal ambient enviroment of acquisition, and sterically defined technology is carried out to mobile terminal, that is, pass through handle
Any mobile terminal with sensor is placed in a unknown environment, and mobile terminal trial is allowed incrementally to establish a company
It continuous map and is positioned with the map, which can be applied to virtual reality device, augmented reality equipment and terminal
The scenes such as navigation.
In the related art, the Processing Algorithm for having already appeared some images as key data source constructs map, for example,
For unordered image (such as various network pictures), mainly exercise recovery structure (SFM, structure from motion) is used
Carry out the three-dimensional reconstruction of " without true scale "；For orderly image (such as one section of continuous video), VSLAM is mainly used
(visual-SLAM) three-dimensional reconstruction or VISLAM (visual-inertial-SLAM) progress for carrying out " no true scale " " have
The three-dimensional reconstruction of true scale ".
However, in the limited situation of hardware resource of terminal device, as computing resource is limited, storage resource is limited, interior
It deposits in the limited situation of resource, terminal device is low using the accuracy of map constructed by the relevant technologies, leads to poor user experience.
Summary of the invention
Therefore in the prior art, in the limited situation of hardware resource of terminal device, using constructed by the relevant technologies
Low this of the accuracy of map be very bothersome process.
Thus, it is also very desirable to a kind of improved three-dimensional map processing method, so that providing finer, more reasonably processing
Process guarantees the precision of three-dimensional map.
In the present context, embodiments of the present invention be intended to provide a kind of three-dimensional map processing method, device, medium and
Calculate equipment.
In the first aspect of embodiment of the present invention, a kind of three-dimensional map processing method is provided, comprising: obtain target
The initial map datum of scene, wherein when above-mentioned initial map datum includes at least raw image data and acquisition original image
Estimated state data；According to above-mentioned raw image data included in above-mentioned initial map datum and above-mentioned estimated state number
According to the building three-dimensional map with true scale corresponding with above-mentioned target scene；And to the above-mentioned three-dimensional with true scale
The position data of three-dimensional point in map and the pose data of the three-dimensional point corresponding image optimize respectively, after obtaining reconstruction
The three-dimensional map with true scale.
In one embodiment of the invention, the above-mentioned raw image data according to included in above-mentioned initial map datum
Constructing the three-dimensional map with true scale corresponding with above-mentioned target scene with above-mentioned estimated state data includes: according to multiframe
Characteristic point similarity in above-mentioned original image determine the similar original image of at least two frames go forward side by side line trace association；And it will be upper
Estimated state data are stated as constrained parameters, Linear Triangular is carried out to the similar original image of above-mentioned at least two frames after tracking association
Change processing.
In another embodiment of the present invention, packet is optimized to the pose data of above-mentioned three-dimensional point corresponding image
It includes: determining at least two field pictures for meeting preset condition from above-mentioned raw image data；Calculate the above-mentioned preset condition of meeting
At least transformation relation data of the corresponding pose of two field pictures；And it is based on above-mentioned transformation relation data and preset pose artwork
Type optimizes the pose data of above-mentioned three-dimensional point corresponding image.
In yet another embodiment of the present invention, determination meets at least the two of preset condition from above-mentioned raw image data
Frame image includes determining to meet time domain difference but the identical at least two field pictures in airspace from above-mentioned raw image data.
In yet another embodiment of the present invention, optimizing to the position data of above-mentioned three-dimensional point includes: by above-mentioned three
The character pair point of dimension point corresponding image carries out merging treatment；And the pixel coordinate meter according to the characteristic point after merging treatment
Calculate the position data of this feature point corresponding three-dimensional points.
In yet another embodiment of the present invention, to the position of the three-dimensional point in the above-mentioned three-dimensional map with true scale
It includes: first to the above-mentioned three-dimensional point corresponding image that data and the pose data of the three-dimensional point corresponding image optimize respectively
Pose data optimize, obtain optimization pose data after three-dimensional map；And again to above-mentioned optimization pose data after
The position data of three-dimensional point in three-dimensional map optimizes, the three-dimensional map with true scale after obtaining above-mentioned reconstruction.
In yet another embodiment of the present invention, in the position to the three-dimensional point in the above-mentioned three-dimensional map with true scale
Set data and the three-dimensional point corresponding image pose data optimize respectively after, the above method further include: after optimization
Three-dimensional point position data coupled with the pose data of the three-dimensional point corresponding image, the three-dimensional map after being coupled；
The three-dimensional map after above-mentioned coupling is optimized again using light beam inertial navigation adjustment method, having after obtaining suboptimization again is true
The three-dimensional map of scale.
In yet another embodiment of the present invention, above-mentioned estimated state data when acquiring original image include at least acquisition
Pose data and the motion parameter data of sensor when acquiring above-mentioned original image when above-mentioned original image.
In the second aspect of embodiment of the present invention, a kind of three-dimensional map processing unit is provided, comprising: obtain mould
Block, for obtaining the initial map datum of target scene, wherein above-mentioned initial map datum include at least raw image data and
Acquire estimated state data when original image；Module is constructed, for above-mentioned according to included in above-mentioned initial map datum
Raw image data and above-mentioned estimated state data construct the three-dimensional map with true scale corresponding with above-mentioned target scene；
And optimization module, for the three-dimensional point in the above-mentioned three-dimensional map with true scale position data and the three-dimensional point institute
The pose data of correspondence image optimize respectively, the three-dimensional map with true scale after being rebuild.
In one embodiment of the invention, above-mentioned building module includes: associative cell, for above-mentioned original according to multiframe
Characteristic point similarity in image determine the similar original image of at least two frames go forward side by side line trace association；And processing unit, it uses
In using above-mentioned estimated state data as constrained parameters, line is carried out to the similar original image of above-mentioned at least two frames after tracking association
Property trigonometric ratio processing.
In another embodiment of the present invention, above-mentioned optimization module comprises determining that unit, is used for from above-mentioned original image
At least two field pictures for meeting preset condition are determined in data；First computing unit, for calculating the above-mentioned preset condition that meets
At least transformation relation data of the corresponding pose of two field pictures；And optimization unit, for based on above-mentioned transformation relation data and
Preset pose graph model optimizes the pose data of above-mentioned three-dimensional point corresponding image.
In yet another embodiment of the present invention, above-mentioned determination unit is used to determine from above-mentioned raw image data and meet
Time domain difference but the identical at least two field pictures in airspace.
In yet another embodiment of the present invention, above-mentioned optimization module includes: combining unit, is used for above-mentioned three-dimensional point institute
The character pair point of correspondence image carries out merging treatment；And second computing unit, for according to the characteristic point after merging treatment
Pixel coordinate calculate the position datas of this feature point corresponding three-dimensional points.
In yet another embodiment of the present invention, above-mentioned optimization module is used for first to the above-mentioned three-dimensional point corresponding image
Pose data optimize, the three-dimensional map after obtaining optimization pose data；And again to three after above-mentioned optimization pose data
The position data of three-dimensional point in dimension map optimizes, the three-dimensional map with true scale after obtaining above-mentioned reconstruction.
In yet another embodiment of the present invention, above-mentioned apparatus further include: coupling module, for having really to above-mentioned
The position data of three-dimensional point in the three-dimensional map of scale and the pose data of the three-dimensional point corresponding image optimize respectively
Later, the three-dimensional point position data after optimization is coupled with the pose data of the three-dimensional point corresponding image, is coupled
Three-dimensional map afterwards；Above-mentioned optimization module is used to carry out the three-dimensional map after above-mentioned coupling again using light beam inertial navigation adjustment method
Optimization, the three-dimensional map with true scale after obtaining suboptimization again.
In yet another embodiment of the present invention, above-mentioned estimated state data when acquiring original image include at least acquisition
Pose data and the motion parameter data of sensor when acquiring above-mentioned original image when above-mentioned original image.
In the third aspect of embodiment of the present invention, a kind of medium is provided, computer executable instructions are stored with, on
Instruction is stated when unit processed executes for realizing three-dimensional map processing method as described above.
In the fourth aspect of embodiment of the present invention, a kind of calculating equipment is provided, comprising: processing unit；And it deposits
Storage unit is stored with computer executable instructions, and described instruction by the processing unit when being executed for realizing as described above
Three-dimensional map processing method.
The three-dimensional map processing method of embodiment, device, medium and calculating equipment according to the present invention, are had in building
After the three-dimensional map for having true scale, by the position data to the three-dimensional point in the three-dimensional map with true scale and it is somebody's turn to do
The pose data of three-dimensional point corresponding image optimize respectively, provide finer, more reasonable process flow to guarantee three
The precision for tieing up map, is particularly suitable for constructing the map of extensive real scene, compared in the related technology by increasing hardware
The scheme of Resource Guarantee three-dimensional map precision, the present invention provides a kind of new process flows, can be in the hardware of terminal device
In the limited situation of resource, more reasonably guarantee the precision of three-dimensional map, brings better experience for user.
Detailed description of the invention
The following detailed description is read with reference to the accompanying drawings, above-mentioned and other mesh of exemplary embodiment of the invention
, feature and advantage will become prone to understand.In the accompanying drawings, if showing by way of example rather than limitation of the invention
Dry embodiment, in which:
Fig. 1 schematically shows the pose schematic diagram in the mobile terminal moving process of embodiment according to the present invention；
Multiple Fig. 2 schematically shows embodiment according to the present invention have the characteristic point pair on the image of common visual field
Answer the schematic diagram of three-dimensional point spatially；
Fig. 3 schematically shows the schematic diagram of the pose graph model optimization of embodiment according to the present invention；
Fig. 4 schematically shows the application scenarios schematic diagrams of embodiment according to the present invention；
Fig. 5 schematically shows the flow chart of three-dimensional map processing method according to an embodiment of the present invention；
Fig. 6 diagrammatically illustrates the original image number according to an embodiment of the present invention according to included in initial map datum
According to the flow chart for constructing the three-dimensional map with true scale corresponding with target scene with estimated state data；
It is excellent that Fig. 7 schematically shows the pose data progress according to an embodiment of the present invention to three-dimensional point corresponding image
The flow chart of change；
Fig. 8 schematically shows the process that the position data according to an embodiment of the present invention to three-dimensional point optimizes
Figure；
Fig. 9 schematically shows the three-dimensional point according to an embodiment of the present invention in the three-dimensional map with true scale
Position data and the three-dimensional point corresponding image the flow chart that optimizes respectively of pose data；
Figure 10 schematically shows the flow chart of three-dimensional map processing method according to another embodiment of the present invention；
Figure 11 schematically shows the block diagram of three-dimensional map processing unit according to an embodiment of the present invention；
Figure 12 schematically shows the block diagram of building module according to an embodiment of the present invention；
Figure 13 schematically shows the block diagram of optimization module according to an embodiment of the present invention；
Figure 14 schematically shows the program for realizing three-dimensional map processing method of embodiment according to the present invention
Product schematic diagram；And
Figure 15 schematically shows the calculating according to an embodiment of the present invention for realizing three-dimensional map processing method and sets
Standby block diagram.
In the accompanying drawings, identical or corresponding label indicates identical or corresponding part.
Specific embodiment
The principle and spirit of the invention are described below with reference to several illustrative embodiments.It should be appreciated that providing this
A little embodiments are used for the purpose of making those skilled in the art can better understand that realizing the present invention in turn, and be not with any
Mode limits the scope of the invention.On the contrary, these embodiments are provided so that this disclosure will be more thorough and complete, and energy
It is enough that the scope of the present disclosure is completely communicated to those skilled in the art.
One skilled in the art will appreciate that embodiments of the present invention can be implemented as a kind of system, device, equipment, method
Or computer program product.Therefore, the present disclosure may be embodied in the following forms, it may be assumed that complete hardware, complete software
The form that (including firmware, resident software, microcode etc.) or hardware and software combine.
Embodiment according to the present invention proposes a kind of three-dimensional map processing method, device, medium and calculates equipment.
Herein, it is to be understood that related term can be for realizing the technology hand of a part of the invention
Section or other summing-up technical terms.For example, term may include:
Terrestrial reference (landmark): it with the object of apparent shape and feature in space, such as can use on carrier
The geometry that pertinent instruments are very easy to distinguish and detected, either natural, it is also possible to artificial.Some ground
Mark can also include additional information (such as bar code, two dimensional code).
Position (position): given three-dimensional system of coordinate (cartesian coordinate system), the position of target in the coordinate system, one
As with (x, y, z) indicate.
Pose (pose): position and posture (direction), such as: it is usually (x, y, yaw) to be usually in two dimension in three-dimensional
(x, y, z, aw, pitch, roll), rear three elements describe the posture of object.Fig. 1 schematically shows real according to the present invention
Apply the pose schematic diagram in the mobile terminal moving process of mode.As shown in Figure 1, wherein yaw is course angle, rotate about the z axis,
Pitch is pitch angle, is rotated around Y-axis, and roll is roll angle, is rotated around X-axis.
Augmented reality (Augmented Reality or AR): position and angle actuarial through camera image simultaneously add
Image analysis technology, the technology that the virtual world on screen can be combined and be interacted with real-world scene.
Inertial Measurement Unit (inertial measurement unit or IMU): measurement object triaxial attitude angle (or angle
Rate) and the device of acceleration generally can gyroscope equipped with three axis and three directions in an inertia sensing unit
Accelerometer to measure the angular speed and acceleration of object in three dimensions, and calculates the posture of object with this.
Odometer (odomerry): the pose of time is closed on using motion sensor (such as " Inertial Measurement Unit ") assessment and is become
Change.
It positions simultaneously and builds figure (Simultaneous Localization And Mapping or SLAM): a band
The robot (or any carrier) of sensor is placed in a unknown environment, and robot trial is allowed incrementally to establish a company
It continuous map and is positioned with the map, such a process can be referred to as " while positioning with build figure ".
Trigonometric ratio (triangulate): using the geometrical constraint of two field pictures, to a pair of corresponding pixel, solving should
To the pose of 2 relative image 1 of the corresponding three-dimensional point position of pixel and image.
Bundle adjustment (bundle adjustment or BA): multiple images for having common visual field are given, with a certain frame image
Pose be reference frame, landmark feature is extracted from all images, while optimizing this feature in reference frame
Three-dimensional position and image three-dimensional pose (pose) in reference frame.Fig. 2 schematically shows implement according to the present invention
Multiple of mode have the schematic diagram of the corresponding three-dimensional point spatially of the characteristic point on the image of common visual field.As shown in Fig. 2, in light
Under conditions of line constraint, by optimizing mean error, the process that binocular establishes majorized function is extended to the N width figure of N mesh acquisition
Picture, and majorized function is established in the same way.
Pass through exercise recovery structure (structure from motion or SFM): a kind of photogrammetric range imaging skill
Art estimates the three-dimensional structure in two-dimensional image sequence, these images may be combined with local motion signal, and SFM generally uses " light
Beam adjustment " algorithm solves.
Light beam inertial navigation adjustment (bundle inertial adjustment or BIA): the landmark feature that BA is calculated
The three-dimensional pose of three-dimensional position and image is no true scale, and between every two field pictures, Inertial Measurement Unit is added
(IMU) constraint of data optimizes the pose of N width image, M three-dimensional point respectively, the available knot for having true scale
Fruit (using BIA optimization as a result, being approximatively considered true scale).
Pose graph model optimization (pose graph optimization or PGO): establishing graph model to a series of poses,
It is carried out using gradient descent method, least square, Levenberg-Marquard (LM), gradient descent (GD) scheduling algorithm
Optimization.Fig. 3 schematically shows the schematic diagram of the pose graph model optimization of embodiment according to the present invention.As shown in figure 3, former
Beginning and end in beginning track is not coincided together due to the presence of noise, is established by the pose to a series of images
Graph model, after being optimized using related algorithm, track after being optimized optimizes beginning and end in track almost
It coincides together.
Closed loop (loop closure): by the relevant calculation of sensing data (such as image), whether judge current data
It is the data in the place once acquired；If it is, it is excellent to add binding side (edge) progress in BA or BIA or PGO model
Change.
Positioning (localization): by the relevant calculation of sensing data (such as image), whether judge current data
It is the data in the place once acquired；If it is, providing pose of the sensor in existing three-dimensional map.
Node (node): the unit in graph model refers to pose in PGO.
Side (edge): the unit in graph model is referring to the transformation relation between two poses in PGO.
Image retrieval (image retrieval): given target image and image data base, index out in the database with
The preceding k frame image of target image similar enough (similarity is greater than some threshold value), wherein k is natural number.
Image data base (image database): being stored with the memory space of certain data format data, can carry out
" image retrieval (image retrieval) ", such as BOW (bag of words), IMI (inverted multi index),
Service is provided for image retrieval.
Below with reference to several representative embodiments of the invention, the principle and spirit of the present invention are explained in detail.
Summary of the invention
The inventors discovered that can according to the data of camera and the sensors acquisition such as Inertial Measurement Unit on mobile terminal
To construct the map of mobile terminal ambient enviroment, and space orientation is carried out to mobile terminal, i.e., by appointing with sensor
Meaning mobile terminal is placed in a unknown environment, and mobile terminal trial is allowed incrementally to establish a continuous map and use
The map is positioned, which can be applied to the scenes such as virtual reality device, augmented reality equipment and terminal guidance.?
In the related technology, the Processing Algorithm for having already appeared some images as key data source constructs map, for example, for unordered graph
As (such as various network pictures), the main three-dimensional reconstruction that " without true scale " is carried out using exercise recovery structure；For ordered graph
As (such as one section of continuous video), mainly using VSLAM (visual-SLAM) carry out " without true scale " three-dimensional reconstruction or
VISLAM (visual-inertial-SLAM) carries out the three-dimensional reconstruction of " having true scale ".
However, in the limited situation of hardware resource of terminal device, as computing resource is limited, storage resource is limited, interior
It deposits in the limited situation of resource, terminal device is low using the accuracy of map constructed by the relevant technologies, leads to poor user experience.
Based on the above analysis, inventor is contemplated after building obtains the three-dimensional map with true scale, by right
The position data of three-dimensional point in three-dimensional map with true scale and the pose data of the three-dimensional point corresponding image are distinguished
It optimizes, finer, more reasonable process flow can be provided to guarantee the precision of three-dimensional map, it is big to be particularly suitable for building
The map of scale real scene, compared to the scheme for guaranteeing three-dimensional map precision by increasing hardware resource in the related technology, originally
Invention provides a kind of new process flow, can more reasonably guarantee in the limited situation of hardware resource of terminal device
The precision of three-dimensional map brings better experience for user.
After introduced the basic principles of the present invention, lower mask body introduces various non-limiting embodiment party of the invention
Formula.
Application scenarios overview
The three-dimensional map processing method of the embodiment of the present invention and its application scenarios of device are elaborated referring initially to Fig. 4.
Fig. 4 schematically shows the application scenarios schematic diagrams of embodiment according to the present invention.
As shown in figure 4, the user C on selected building edge is coordinate origin (0,0,0,0,0,0), user A is at one
Before building, user A can be (13,9,0,0,0,0) relative to the coordinate of origin.Due to according to the camera on mobile terminal
Can construct the map of mobile terminal ambient enviroment with the data of the sensors acquisition such as Inertial Measurement Unit, and to mobile terminal into
Row space orientation, user A can construct the map around building by mobile terminal, mobile terminal trial allowed incrementally to build
It founds a continuous map and is positioned with the map, for example, user A takes a round around building, its institute can be passed through
The environment around mobile terminal shooting building and building used, takes in mobile terminal for constructing three-dimensional map
After data, coordinate system is established, then constructs the three-dimensional map of current environment.In the related art, user A is walked around building
Beginning and end in the initial trace of one circle is not coincided together, as shown in Figure 3 due to the presence of noise.Using this hair
Bright three-dimensional map processing method passes through the position data and the three-dimensional to the three-dimensional point in the three-dimensional map with true scale
The pose data of point corresponding image optimize respectively, such as establish graph model by the pose to a series of images, use
Gradient descent method, least square, Levenberg-Marquard (LM), gradient descent (GD) scheduling algorithm carry out excellent
Change, the track after being optimized, the beginning and end optimized in track almost coincides together, and specifically, can refer to Fig. 3
Shown in optimization front and back track compare.
Embodiment through the invention, after building obtains the three-dimensional map with true scale, by true to having
The position data of three-dimensional point in the three-dimensional map of real scale and the pose data of the three-dimensional point corresponding image carry out excellent respectively
Change, finer, more reasonable process flow can be provided to guarantee the precision of three-dimensional map, it is extensive true to be particularly suitable for building
The map of real field scape, compared to the scheme for guaranteeing three-dimensional map precision by increasing hardware resource in the related technology, the present invention is mentioned
A kind of new process flow has been supplied, can more reasonably have been guaranteed dimensionally in the limited situation of hardware resource of terminal device
The precision of figure brings better experience for user.
Illustrative methods
Below with reference to the application scenarios of Fig. 4, it is described with reference to Figure 5 the three-dimensional map of illustrative embodiments according to the present invention
Processing method.It should be noted which is shown only for the purpose of facilitating an understanding of the spirit and principles of the present invention for above-mentioned application scenarios, this
The embodiment of invention is unrestricted in this regard.On the contrary, embodiments of the present invention can be applied to it is applicable any
Scene.
Fig. 5 schematically shows the flow chart of three-dimensional map processing method according to an embodiment of the present invention.
As shown in figure 5, three-dimensional map processing method according to an embodiment of the present invention includes operation S210~operation S230.
In operation S210, the initial map datum of target scene is obtained, wherein initial map datum includes at least original graph
Estimated state data when as data and acquisition original image.
According to an embodiment of the invention, target scene can be " museum ", " artistic museum ", " game museum ", " outdoor
More people AR " of large scene etc..Raw image data can be bring into operation while positioning with when building diagram technology related algorithm by
Camera acquired image data.
According to an embodiment of the invention, when estimated state data when acquisition original image include at least acquisition original image
Pose data and acquisition original image when sensor motion parameter data.It is taken the photograph when for example, it may be shooting each frame image
Pose (pose), velocity (speed) as head, the acceleration of the inertia sensing unit (IMU) between two continuous frames image
The supplemental characteristic etc. of bias, angular speed bias and IMU.
In operation S220, when according to raw image data included in initial map datum and acquiring the original image
Estimated state data construct the three-dimensional map with true scale corresponding with target scene.
According to an embodiment of the invention, when constructing three-dimensional map, suitable three-dimensional reconstruction algorithm can choose, including but
It is not limited to: ORB-SLAM, LSD-SLAM and LOAM etc.；Three-dimensional reconstruction algorithm is related to sensor, and sensor includes but is not limited to:
Monocular camera, binocular camera, depth camera, radar and RGB-D camera etc..Generally, for the equipment of monocular camera+IMU
Configuration first carries out visual-inertial odometry (calculating a series of pose), while trigonometric ratio characteristic point.Then right
Calculated pose establishes pose graph model, if there is closed loop, can optimize to the graph model.
Fig. 6 diagrammatically illustrates the original image number according to an embodiment of the present invention according to included in initial map datum
According to the flow chart for constructing the three-dimensional map with true scale corresponding with target scene with estimated state data.
As shown in fig. 6, when the raw image data according to included in initial map datum and the acquisition original image
It includes operation S221~S222 that estimated state data, which construct the three-dimensional map with true scale corresponding with target scene,.
In operation S221, the similar original image of at least two frames is determined according to the characteristic point similarity in multiframe original image
Line trace of going forward side by side association.
According to an embodiment of the invention, feature extraction can be carried out to multiframe original image, in every frame original image really
Fixed multiple characteristic points, then calculate the similarity between the multiple characteristic points determined, find out similarity greater than d (similarity threshold
Value) characteristic point, based on similarity greater than the characteristic point of d (similarity threshold) can determine between different original images whether
For similar image, so that it is determined that going out the similar original image of at least two frames, and tracking association is carried out to corresponding similar original image.
Then the image difference according to caused by based on characteristic point of the similarity greater than d and because of cam movement calculates camera in turn
Pose and characteristic point three-dimensional coordinate, carry out three-dimensional reconstruction.
In operation S222, using estimated state data when acquiring the original image as constrained parameters, after tracking association
The similar original image of at least two frames carry out Linear Triangular processing.
According to an embodiment of the invention, in order to obtain with the three-dimensional map of true scale, it can be by " the three of two field pictures
Angling " is generalized to multiframe consecutive image, and using estimated state data as constrained parameters, using light beam inertial navigation error compensation method
(BIA) between every two field pictures, the constraint of Inertial Measurement Unit (IMU) data is added, pose, M three-dimensional to N width image
Point optimizes respectively, the available result for having true scale.For example, it is similar original to handle at least two frames in linear trigonometric ratio
The supplemental characteristic of IMU is added in the result that image obtains, what is obtained is three-dimensional point and the position of " the having true scale " of " more acurrate "
Appearance optimization.
In operation S230, the position data and three-dimensional point institute to the three-dimensional point in the three-dimensional map with true scale are right
The pose data of image are answered to optimize respectively, the three-dimensional map with true scale after being rebuild.
According to an embodiment of the invention, for example, being used when being optimized to the position data of the three-dimensional point in three-dimensional map
Loop closure algorithm carry out winding detection, judge current data whether be the place once acquired data, if so,
The data are added in pose graph model as binding side (edge) then and are optimized, for example, using least square,
Levenberg-Marquard (LM), gradient descent (GD) scheduling algorithm optimize.After optimization position data
When the pose data of three-dimensional point corresponding image optimize, light beam inertial navigation adjustment BIA progress error can be used and be averaged, use
Odometry/SLAM algorithm carries out posture tracking, and backstage carries out error check and correction using location algorithm simultaneously.It can be incited somebody to action before check and correction
The error of the corresponding posture information of image is compared with threshold value, and error is more than being proofreaded again for threshold value, and the size of threshold value can
To be to be compared a large amount of pose measurement data with true pose data, calculate what analysis determined later.
According to an embodiment of the invention, can using simultaneously positioning with build diagram technology related algorithm output various states as
Input carries out " three-dimensional reconstruction of true scale " in conjunction with camera and IMU data, the Three-dimensional Gravity of especially extensive true scale
It builds, for example, carrying out three-dimensional map reconstruction for scene around museum, obtains that there is the true scale of degree of precision dimensionally
Figure.
The three-dimensional map processing method of embodiment through the invention obtains the three-dimensional map with true scale in building
Later, pass through the position of position data and the three-dimensional point corresponding image to the three-dimensional point in the three-dimensional map with true scale
Appearance data optimize respectively, provide finer, more reasonable process flow to guarantee the precision of three-dimensional map, especially suitable
In the map for constructing extensive real scene, can more reasonably guarantee in the limited situation of hardware resource of terminal device
The precision of three-dimensional map brings better experience for user.
Below with reference to Fig. 7~Figure 10, method shown in fig. 5 is described further in conjunction with specific embodiments.
It is excellent that Fig. 7 schematically shows the pose data progress according to an embodiment of the present invention to three-dimensional point corresponding image
The flow chart of change.
As shown in fig. 7, optimizing the pose data of three-dimensional point corresponding image including operating S231~operation S233.
In operation S231, at least two field pictures for meeting preset condition are determined from raw image data.
According to an embodiment of the invention, may include multiframe original image in the raw image data acquired by sensor,
For acquiring in different time, but pose corresponding to when position is identical original image collected is likely to occur difference, this is poor
The different possible accuracy for influencing three-dimensional map therefore can be to the pose data of image corresponding when acquiring identical three-dimensional point
It optimizes.Preset condition can be time domain difference but airspace is identical, can determine from raw image data and meet time domain not
The same but identical at least two field pictures in airspace.Specifically, for example, at 3 points in afternoon has clapped a Zhang Zhao in the Tian'anmen Square, then two
After a hour, 5 PM goes to the same position in the Tian'anmen Square to take one or more photograph again.Different time is shot upper
Stating photo is exactly time domain difference but the identical at least two field pictures in airspace.According to an embodiment of the invention, can be examined using closed loop
The mode of survey passes through the relevant calculation of sensing data (such as image), judges whether current data is the place once acquired
Data, and at least two field pictures for meeting preset condition are determined from raw image data.
In operation S232, the transformation relation data for meeting the corresponding pose of at least two field pictures of preset condition are calculated.
According to an embodiment of the invention, the transformation relation data of at least corresponding pose of two field pictures can be between pose
Transformation matrix, or perhaps functional transformation relationship.For example, as shown in figure 4, user A and user B is in same building terrestrial reference
Before, for user A, user A is that coordinate origin is indicated with (0,0,0,0,0,0), but from user in terms of the local coordinate of user A
The origin of C coordinate system is (- 13, -9,0,0,0,0), is (13,9,0,0,0,0) from user A from the point of view of the coordinate system of user C；For
For user B, user's B coordinate origin is indicated with (0,0,0,0,0,0), and the origin from C coordinate system in terms of the local coordinate of user B is
(- 4,9,0,0,0,0) are (4,9,0,0,0,0) from user B from the point of view of the coordinate system of user C.
In operation S233, based on transformation relation data and preset pose graph model to the pose of three-dimensional point corresponding image
Data optimize.
According to an embodiment of the invention, can lead to using transformation relation as in " edge (side) " addition " pose graph model "
The constraint for crossing " edge (side) " that provides, optimizes entire three-dimensional map.Pose graph model can be in advance to a series of
Gradient descent method, most can be used to the pose data of three-dimensional point corresponding image in the pose graph model that the pose of image is established
Small two multiply, Levenberg-Marquard (LM), gradient descent (GD) scheduling algorithm optimize.
According to an embodiment of the invention, allowing to reduce optimization as far as possible based on transformation relation data as priori
Error improves the accuracy of three-dimensional map.
Fig. 8 schematically shows the process that the position data according to an embodiment of the present invention to three-dimensional point optimizes
Figure.
As shown in figure 8, optimizing the position data of three-dimensional point including operating S234~operation S235.
In operation S234, the character pair point of three-dimensional point corresponding image is subjected to merging treatment.
In operation S235, the position of this feature point corresponding three-dimensional points is calculated according to the pixel coordinate of the characteristic point after merging treatment
Set data.
According to an embodiment of the invention, prolonged error accumulation has obtained a degree of amendment, on " airspace " from the point of view of,
Originally in the pose of the image of the same place shooting relatively close to can be by the correspondence of these three-dimensional point corresponding images
Characteristic point merges processing, is optimized with realizing to the position data of three-dimensional point.
According to an embodiment of the invention, since the characteristic point for corresponding to the same three-dimensional point in image is merged, space
In three-dimensional point will obtain the pixel coordinate of unique corresponding characteristic point, can be with by the pixel coordinate of unique corresponding characteristic point
The pose of camera and the three-dimensional coordinate of three-dimensional point are calculated in turn.By the way that the feature of three-dimensional point spatially on the image is clicked through
Row merges, and is optimized with realizing to the position data of three-dimensional point, to achieve the effect that optimize image.
Fig. 9 schematically shows the three-dimensional point according to an embodiment of the present invention in the three-dimensional map with true scale
Position data and the three-dimensional point corresponding image the flow chart that optimizes respectively of pose data.
As shown in figure 9, the position data and three-dimensional point institute to the three-dimensional point in the three-dimensional map with true scale are right
The pose data of image are answered to be optimized respectively including operating S236~operation S237.
In operation S236, first the pose data of the three-dimensional point corresponding image are optimized, obtain optimization pose data
Three-dimensional map afterwards.
It is optimized in operation S237, then to the position data of the three-dimensional point in the three-dimensional map after optimization pose data,
The three-dimensional map with true scale after being rebuild.
According to an embodiment of the invention, first being optimized to the pose data of the three-dimensional point corresponding image, so that when long
Between error accumulation obtained a degree of amendment, on " airspace " from the point of view of, originally it is same place shooting image position
Appearance is relatively close to, can then when optimizing to the position data of three-dimensional point in the three-dimensional map after optimization pose data
The character pair of these three-dimensional point corresponding images point is carried out merging treatment, three-dimensional point is determined according to the characteristic point after merging
Position, with realize the position data of three-dimensional point is optimized, so as to advanced optimize map, further improve structure
The accuracy for the three-dimensional map built.
Figure 10 schematically shows the flow chart of three-dimensional map processing method according to another embodiment of the present invention.
As shown in Figure 10, to the three-dimensional point in the three-dimensional map with true scale position data and the three-dimensional point institute
After the pose data of correspondence image optimize respectively, three-dimensional map processing method according to an embodiment of the present invention further includes behaviour
Make S240~operation S250.
In operation S240, the pose data of three-dimensional point position data and the three-dimensional point corresponding image after optimization are carried out
Coupling, the three-dimensional map after being coupled.
In operation S250, the three-dimensional map after coupling is optimized again using light beam inertial navigation adjustment method, is obtained again
The three-dimensional map with true scale after optimization.
According to an embodiment of the invention, for the three-dimensional map after coupling, it can be by light beam inertial navigation adjustment method every two
Between frame image, the constraint of Inertial Measurement Unit (IMU) data is added, the pose of N width image, M three-dimensional point are carried out respectively
Optimization, the three-dimensional map with true scale after available suboptimization again.Embodiment through the invention, provides one kind
Finer and more reasonably calculating can advanced optimize map, and the accurate of the three-dimensional map of reconstruction can be further improved
Degree.
Exemplary means
After describing the method for exemplary embodiment of the invention, next, exemplary to the present invention with reference to Figure 11
The three-dimensional map processing unit of embodiment is described.
Figure 11 schematically shows the block diagram of three-dimensional map processing unit according to an embodiment of the present invention.
As shown in figure 11, which includes obtaining module 410, building module 420 and optimization module
430。
Obtain the initial map datum that module 410 is used to obtain target scene, wherein initial map datum includes at least original
Estimated state data when beginning image data and acquisition original image.
It constructs module 420 and is used for the raw image data according to included in initial map datum and estimated state data structure
Build the three-dimensional map with true scale corresponding with target scene.
Optimization module 430 is used for position data and the three-dimensional point to the three-dimensional point in the three-dimensional map with true scale
The pose data of corresponding image optimize respectively, the three-dimensional map with true scale after being rebuild.
The three-dimensional map processing method of embodiment through the invention obtains the three-dimensional map with true scale in building
Later, pass through the position of position data and the three-dimensional point corresponding image to the three-dimensional point in the three-dimensional map with true scale
Appearance data optimize respectively, provide finer, more reasonable process flow to guarantee the precision of three-dimensional map, especially suitable
In the map for constructing extensive real scene, compared in the related technology by increasing hardware resource guarantee three-dimensional map precision
Scheme, the present invention provides a kind of new process flows, can be in the limited situation of hardware resource of terminal device, more rationally
Guarantee three-dimensional map precision, bring better experience for user.
Figure 12 schematically shows the block diagram of building module according to an embodiment of the present invention.
As shown in figure 12, building module 420 includes associative cell 421 and processing unit 422.
Associative cell 421 is used to determine that at least two frames are similar original according to the characteristic point similarity in multiframe original image
Image go forward side by side line trace association.
Processing unit 422 is used for using estimated state data as constrained parameters, similar at least two frames after tracking association
Original image carries out Linear Triangular processing.
Figure 13 schematically shows the block diagram of optimization module according to an embodiment of the present invention.
As shown in figure 13, optimization module 430 includes determination unit 431, the first computing unit 432 and optimization unit 433.
Determination unit 431 meets at least two field pictures of preset condition for determining from raw image data.
First computing unit 432 is used to calculate the transformation relation for the corresponding pose of at least two field pictures for meeting preset condition
Data.
Optimize unit 433 to be used for based on transformation relation data and preset pose graph model to three-dimensional point corresponding image
Pose data optimize.
According to an embodiment of the invention, determination unit 431 be used to determine from raw image data meet time domain it is different but
The identical at least two field pictures in airspace.
According to an embodiment of the invention, allowing to reduce optimization as far as possible based on transformation relation data as priori
Error improves the accuracy of three-dimensional map.
According to an embodiment of the invention, as shown in figure 13, optimization module 430 further includes that combining unit 434 and second calculate
Unit 435.
Combining unit 434 is used to the character pair point of three-dimensional point corresponding image carrying out merging treatment.
Second computing unit 435 is used to calculate this feature point corresponding three according to the pixel coordinate of the characteristic point after merging treatment
Tie up the position data of point.
According to an embodiment of the invention, since the characteristic point for corresponding to the same three-dimensional point in image is merged, space
In three-dimensional point will obtain the pixel coordinate of unique corresponding characteristic point, can be with by the pixel coordinate of unique corresponding characteristic point
The pose of camera and the three-dimensional coordinate of three-dimensional point are calculated in turn.By the way that the feature of three-dimensional point spatially on the image is clicked through
Row merges, and is optimized with realizing to the position data of three-dimensional point, to achieve the effect that optimize image.
According to an embodiment of the invention, optimization module 430 for first to the pose data of the three-dimensional point corresponding image into
Row optimization, the three-dimensional map after obtaining optimization pose data；And again to the three-dimensional in the three-dimensional map after optimization pose data
The position data of point optimizes, the three-dimensional map with true scale after being rebuild.
According to an embodiment of the invention, first being optimized to the pose data of the three-dimensional point corresponding image, for a long time
Error accumulation has obtained a degree of amendment, on " airspace " from the point of view of, originally it is same place shooting image pose
Through relatively close to, can will then when being optimized to the position data of three-dimensional point in the three-dimensional map after optimization pose data
The character pair point of these three-dimensional point corresponding images carries out merging treatment, and the position of three-dimensional point is determined according to the characteristic point after merging
It sets, the position data of three-dimensional point is optimized with realizing, map can be advanced optimized, further improve the three-dimensional of building
The accuracy of map.
According to an embodiment of the invention, as shown in figure 11, which further includes coupling module 440.
Coupling module 440 is used in the position data and the three-dimensional to the three-dimensional point in the three-dimensional map with true scale
Point corresponding image pose data optimize respectively after, by after optimization three-dimensional point position data and the three-dimensional point it is right
The pose data of image are answered to be coupled, the three-dimensional map after being coupled.
Optimization module 430 is used to optimize the three-dimensional map after coupling again using light beam inertial navigation adjustment method, obtains
The three-dimensional map with true scale after suboptimization again.
Embodiment through the invention, map can be advanced optimized by providing a kind of finer and more reasonably calculating,
The accuracy of the three-dimensional map of reconstruction can be further improved.
According to an embodiment of the invention, when estimated state data when acquisition original image include at least acquisition original image
Pose data and acquisition original image when sensor motion parameter data.
Exemplary media
After describing the device of exemplary embodiment of the invention, next, exemplary to the present invention with reference to Figure 14
Embodiment, for being stored with computer executable instructions, the instruction when unit processed executes for realizing Fig. 5 to scheme
The medium of three-dimensional map processing method in 10 is described.
In some possible embodiments, various aspects of the invention are also implemented as a kind of shape of program product
Formula comprising program code, when described program product is run on the computing device, said program code is for making the calculating
Equipment executes described in above-mentioned " illustrative methods " part of this specification the three of various illustrative embodiments according to the present invention
The operation in maps processing method is tieed up, for example, the calculating equipment can execute operation S210 as shown in Figure 5, obtains mesh
Mark the initial map datum of scene, wherein when initial map datum includes at least raw image data and acquires original image
Estimated state data.S220 is operated, according to raw image data included in initial map datum and estimated state data structure
Build the three-dimensional map with true scale corresponding with target scene.S230 is operated, in the three-dimensional map with true scale
The position data of three-dimensional point and the pose data of the three-dimensional point corresponding image optimize respectively, having after being rebuild
The three-dimensional map of true scale.
Described program product can be using any combination of one or more readable mediums.Readable medium can be readable letter
Number medium or readable storage medium storing program for executing.Readable storage medium storing program for executing for example may be-but not limited to-electricity, magnetic, optical, electromagnetic, red
The system of outside line or semiconductor, device or device, or any above combination.The more specific example of readable storage medium storing program for executing
(non exhaustive list) includes: the electrical connection with one or more conducting wires, portable disc, hard disk, random access memory
(RAM), read-only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disc
Read memory (CD-ROM), light storage device, magnetic memory device or above-mentioned any appropriate combination.
Figure 14 schematically shows the program for realizing three-dimensional map processing method of embodiment according to the present invention
Product schematic diagram.
As shown in figure 14, the program product 50 of the three-dimensional map processing method of embodiment according to the present invention is described,
It can be using portable compact disc read only memory (CD-ROM) and including program code, and can calculate equipment, such as
It is run on PC.However, program product of the invention is without being limited thereto, in this document, readable storage medium storing program for executing, which can be, appoints
What include or the tangible medium of storage program that the program can be commanded execution system, device or device use or and its
It is used in combination.
Readable signal medium may include in a base band or as the data-signal that carrier wave a part is propagated, wherein carrying
Readable program code.The data-signal of this propagation can take various forms, including --- but being not limited to --- electromagnetism letter
Number, optical signal or above-mentioned any appropriate combination.Readable signal medium can also be other than readable storage medium storing program for executing it is any can
Read medium, the readable medium can send, propagate or transmit for by instruction execution system, device or device use or
Program in connection.
The program code for including on readable medium can transmit with any suitable medium, including --- but being not limited to ---
Wirelessly, wired, optical cable, RF etc. or above-mentioned any appropriate combination.
The program for executing operation of the present invention can be write with any combination of one or more programming languages
Code, described program design language include object oriented program language --- and such as Java, C++ etc. further include routine
Procedural programming language --- such as " C " language or similar programming language.Program code can fully exist
It executes in user calculating equipment, partly execute on a user device, part executes on a remote computing or exists completely
It is executed on remote computing device or server.In the situation for being related to remote computing device, remote computing device can be by appointing
The network --- including local area network (LAN) or wide area network (WAN) --- of meaning type is connected to user calculating equipment, alternatively, can be with
It is connected to external computing device (such as connecting using ISP by internet).
Exemplary computer device
After method, medium and the device for describing exemplary embodiment of the invention, next, with reference to Figure 15 to this
The calculating equipment of invention illustrative embodiments is described, which includes processing unit and storage unit, and storage is single
Member is stored with computer executable instructions, and above-metioned instruction by above-mentioned processing unit when being executed for realizing Fig. 5 into Figure 10
Three-dimensional map processing method is described.
The embodiment of the invention also provides a kind of calculating equipment.Person of ordinary skill in the field is it is understood that this hair
Bright various aspects can be implemented as system, method or program product.Therefore, various aspects of the invention can be implemented as
Following form, it may be assumed that complete hardware embodiment, complete Software Implementation (including firmware, microcode etc.) or hardware and
The embodiment that software aspects combine, may be collectively referred to as circuit, " module " or " system " here.
In some possible embodiments, it is single can to include at least at least one processing for calculating equipment according to the present invention
Member and at least one storage unit.Wherein, the storage unit is stored with program code, when said program code is described
When processing unit executes, so that the processing unit executes described in above-mentioned " illustrative methods " part of this specification according to this
Invent the step in the three-dimensional map processing method of various illustrative embodiments.For example, the processing unit can execute such as
S210 is operated shown in Fig. 5, obtains the initial map datum of target scene, wherein initial map datum includes at least original
Estimated state data when image data and acquisition original image.S220 is operated, according to original included in initial map datum
Beginning image data and estimated state data construct the three-dimensional map with true scale corresponding with target scene.S230 is operated,
The pose data of position data and the three-dimensional point corresponding image to the three-dimensional point in the three-dimensional map with true scale point
It does not optimize, the three-dimensional map with true scale after being rebuild.
Figure 15 schematically shows the calculating according to an embodiment of the present invention for realizing three-dimensional map processing method and sets
Standby block diagram.
The calculating that the three-dimensional map processing method of this embodiment according to the present invention is described referring to Figure 15 is set
Standby 60.Calculating equipment 60 as shown in figure 15 is only an example, should not function and use scope to the embodiment of the present invention
Bring any restrictions.
As shown in figure 15, equipment 60 is calculated to show in the form of universal computing device.The component for calculating equipment 60 can wrap
It includes but is not limited to: at least one above-mentioned processing unit 601, at least one above-mentioned storage unit 602, the different system components of connection
The bus 603 of (including storage unit 602 and processing unit 601).
Bus 603 indicates one of a few class bus structures or a variety of, including memory bus or Memory Controller,
Peripheral bus, graphics acceleration port, processor or the local bus using any bus structures in a variety of bus structures.
Storage unit 602 may include the readable medium of form of volatile memory, such as random access memory (RAM)
6021 and/or cache memory 6022, it can further include read-only memory (ROM) 6023.
Storage unit 602 can also include program/utility with one group of (at least one) program module 6024
6025, such program module 6024 includes but is not limited to: operating system, one or more application program, other program moulds
It may include the realization of network environment in block and program data, each of these examples or certain combination.
Calculating equipment 60 can also be with one or more external equipment 604 (such as keyboard, sensing equipment, bluetooth equipment etc.)
Communication can also be enabled a user to communicate with the equipment that calculating equipment 60 interacts with one or more, and/or be set with to calculate
The standby 60 any equipment (such as router, modem etc.) that can be communicated with one or more of the other calculating equipment are led to
Letter.This communication can be carried out by input/output (I/O) interface 605.Also, calculating equipment 60 can also be suitable by network
Orchestration 606 and one or more network (such as local area network (LAN), wide area network (WAN) and/or public network, such as because of spy
Net) communication.As shown, network adapter 606 is communicated by bus 603 with the other modules for calculating equipment 60.It should be understood that
Although not shown in the drawings, other hardware and/or software module can be used in conjunction with equipment 60 is calculated, including but not limited to: micro- generation
Code, device driver, redundant processing unit, external disk drive array, RAID system, tape drive and data backup are deposited
Storage system etc..
It should be noted that although being referred to several units/modules or subelement/module of device in the above detailed description,
But it is this division be only exemplary it is not enforceable.In fact, embodiment according to the present invention, above-described
The feature and function of two or more units/modules can embody in a units/modules.Conversely, above-described one
The feature and function of a units/modules can be to be embodied by multiple units/modules with further division.
In addition, although describing the operation of the method for the present invention in the accompanying drawings with particular order, this do not require that or
Hint must execute these operations in this particular order, or have to carry out shown in whole operation be just able to achieve it is desired
As a result.Additionally or alternatively, it is convenient to omit multiple steps are merged into a step and executed by certain steps, and/or by one
Step is decomposed into execution of multiple steps.
Although detailed description of the preferred embodimentsthe spirit and principles of the present invention are described by reference to several, it should be appreciated that, this
It is not limited to the specific embodiments disclosed for invention, does not also mean that the feature in these aspects cannot to the division of various aspects
Combination is benefited to carry out, this to divide the convenience merely to statement.The present invention is directed to cover appended claims spirit and
Included various modifications and equivalent arrangements in range.
Claims (10)
1. a kind of three-dimensional map processing method, comprising:
Obtain the initial map datum of target scene, wherein the initial map datum includes at least raw image data and adopts
Collect estimated state data when original image；
According to the raw image data included in the initial map datum and estimated state data building and institute
State the corresponding three-dimensional map with true scale of target scene；And
The position of position data and the three-dimensional point corresponding image to the three-dimensional point in the three-dimensional map with true scale
Appearance data optimize respectively, the three-dimensional map with true scale after being rebuild.
2. according to the method described in claim 1, wherein, according to the original image included in the initial map datum
Data and the estimated state data construct the three-dimensional map with true scale corresponding with the target scene
Characteristic point similarity in the original image according to multiframe determines that the similar original image of at least two frames is gone forward side by side line trace
Association；And
Using the estimated state data as constrained parameters, the similar original image of at least two frames described in after tracking association is carried out
Linear Triangularization processing.
3. according to the method described in claim 1, wherein, optimizing packet to the pose data of the three-dimensional point corresponding image
It includes:
At least two field pictures for meeting preset condition are determined from the raw image data；
Calculate the transformation relation data of the corresponding pose of at least two field pictures for meeting preset condition；And
Based on the transformation relation data and preset pose graph model to the pose data of the three-dimensional point corresponding image into
Row optimization.
4. according to the method described in claim 3, wherein, determination meets preset condition at least from the raw image data
Two field pictures include:
It is determined from the raw image data and meets time domain difference but the identical at least two field pictures in airspace.
5. according to the method described in claim 4, wherein, the position data of the three-dimensional point is optimized includes:
The character pair point of the three-dimensional point corresponding image is subjected to merging treatment；And
The position data of this feature point corresponding three-dimensional points is calculated according to the pixel coordinate of the characteristic point after merging treatment.
6. according to the method described in claim 1, wherein, to the position of the three-dimensional point in the three-dimensional map with true scale
The pose data for setting data and the three-dimensional point corresponding image optimize respectively includes:
First the pose data of the three-dimensional point corresponding image are optimized, obtain optimization pose data after dimensionally
Figure；And
The position data of the three-dimensional point in the three-dimensional map after the optimization pose data is optimized again, obtains the reconstruction
The three-dimensional map with true scale afterwards.
7. according to the method described in claim 1, wherein, to the three-dimensional point in the three-dimensional map with true scale
After position data and the pose data of the three-dimensional point corresponding image optimize respectively, the method also includes:
Three-dimensional point position data after optimization is coupled with the pose data of the three-dimensional point corresponding image, after obtaining coupling
Three-dimensional map；
The three-dimensional map after the coupling is optimized again using light beam inertial navigation adjustment method, having after obtaining suboptimization again
The three-dimensional map of true scale.
8. a kind of three-dimensional map processing unit, comprising:
Module is obtained, for obtaining the initial map datum of target scene, wherein the initial map datum includes at least original
Estimated state data when image data and acquisition original image；
Module is constructed, the raw image data according to included in the initial map datum and the estimated state are used for
Data construct the three-dimensional map with true scale corresponding with the target scene；And
Optimization module, for the three-dimensional point in the three-dimensional map with true scale position data and the three-dimensional point institute
The pose data of correspondence image optimize respectively, the three-dimensional map with true scale after being rebuild.
9. a kind of medium is stored with computer executable instructions, described instruction is when unit processed executes for realizing right
It is required that three-dimensional map processing method described in any one of 1 to 7.
10. a kind of calculating equipment, comprising:
Processing unit；And
Storage unit is stored with computer executable instructions, and described instruction by the processing unit when being executed for realizing power
Benefit require any one of 1 to 7 described in three-dimensional map processing method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811363016.3A CN109461208B (en) | 2018-11-15 | 2018-11-15 | Three-dimensional map processing method, device, medium and computing equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811363016.3A CN109461208B (en) | 2018-11-15 | 2018-11-15 | Three-dimensional map processing method, device, medium and computing equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109461208A true CN109461208A (en) | 2019-03-12 |
| CN109461208B CN109461208B (en) | 2022-12-16 |
Family
ID=65610611
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811363016.3A Active CN109461208B (en) | 2018-11-15 | 2018-11-15 | Three-dimensional map processing method, device, medium and computing equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109461208B (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110060343A (en) * | 2019-04-24 | 2019-07-26 | 百度在线网络技术（北京）有限公司 | Map constructing method and system, server, computer-readable medium |
| CN110298921A (en) * | 2019-07-05 | 2019-10-01 | 青岛中科智保科技有限公司 | The construction method and processing equipment of three-dimensional map with personage's semantic information |
| CN110749308A (en) * | 2019-09-30 | 2020-02-04 | 浙江工业大学 | SLAM-oriented outdoor positioning method using consumer-grade GPS and 2.5D building models |
| CN110956697A (en) * | 2019-12-20 | 2020-04-03 | 上海有个机器人有限公司 | Laser slam-based memory optimization method, medium, terminal and device |
| CN111080784A (en) * | 2019-11-27 | 2020-04-28 | 贵州宽凳智云科技有限公司北京分公司 | Ground three-dimensional reconstruction method and device based on ground image texture |
| CN111698422A (en) * | 2020-06-10 | 2020-09-22 | 百度在线网络技术（北京）有限公司 | Panoramic image acquisition method and device, electronic equipment and storage medium |
| CN111738087A (en) * | 2020-05-25 | 2020-10-02 | 完美世界(北京)软件科技发展有限公司 | Method and device for generating face model of game role |
| CN112184768A (en) * | 2020-09-24 | 2021-01-05 | 杭州易现先进科技有限公司 | SFM reconstruction method and device based on laser radar and computer equipment |
| CN112308904A (en) * | 2019-07-29 | 2021-02-02 | 北京初速度科技有限公司 | Vision-based drawing construction method and device and vehicle-mounted terminal |
| CN112539755A (en) * | 2019-11-22 | 2021-03-23 | 百度（美国）有限责任公司 | Updating point cloud registration pipeline based on ADMM algorithm for autonomous vehicle |
| CN113178000A (en) * | 2021-03-26 | 2021-07-27 | 杭州易现先进科技有限公司 | Three-dimensional reconstruction method and device, electronic equipment and computer storage medium |
| CN112308904B (en) * | 2019-07-29 | 2024-07-02 | 北京初速度科技有限公司 | Vision-based image construction method and device and vehicle-mounted terminal |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8761439B1 (en) * | 2011-08-24 | 2014-06-24 | Sri International | Method and apparatus for generating three-dimensional pose using monocular visual sensor and inertial measurement unit |
| CN105869136A (en) * | 2015-01-22 | 2016-08-17 | 北京雷动云合智能技术有限公司 | Collaborative visual SLAM method based on multiple cameras |
| CN107193279A (en) * | 2017-05-09 | 2017-09-22 | 复旦大学 | Robot localization and map structuring system based on monocular vision and IMU information |
| CN108489482A (en) * | 2018-02-13 | 2018-09-04 | 视辰信息科技(上海)有限公司 | The realization method and system of vision inertia odometer |
| CN108520543A (en) * | 2018-04-09 | 2018-09-11 | 网易（杭州）网络有限公司 | A kind of method that relative accuracy map is optimized, equipment and storage medium |
| US20180285653A1 (en) * | 2017-03-31 | 2018-10-04 | Alarm.Com Incorporated | Supervised delivery techniques |
| CN108665540A (en) * | 2018-03-16 | 2018-10-16 | 浙江工业大学 | Robot localization based on binocular vision feature and IMU information and map structuring system |
-
2018
- 2018-11-15 CN CN201811363016.3A patent/CN109461208B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8761439B1 (en) * | 2011-08-24 | 2014-06-24 | Sri International | Method and apparatus for generating three-dimensional pose using monocular visual sensor and inertial measurement unit |
| CN105869136A (en) * | 2015-01-22 | 2016-08-17 | 北京雷动云合智能技术有限公司 | Collaborative visual SLAM method based on multiple cameras |
| US20180285653A1 (en) * | 2017-03-31 | 2018-10-04 | Alarm.Com Incorporated | Supervised delivery techniques |
| CN107193279A (en) * | 2017-05-09 | 2017-09-22 | 复旦大学 | Robot localization and map structuring system based on monocular vision and IMU information |
| CN108489482A (en) * | 2018-02-13 | 2018-09-04 | 视辰信息科技(上海)有限公司 | The realization method and system of vision inertia odometer |
| CN108665540A (en) * | 2018-03-16 | 2018-10-16 | 浙江工业大学 | Robot localization based on binocular vision feature and IMU information and map structuring system |
| CN108520543A (en) * | 2018-04-09 | 2018-09-11 | 网易（杭州）网络有限公司 | A kind of method that relative accuracy map is optimized, equipment and storage medium |
Non-Patent Citations (3)
| Title |
|---|
| MUR-ARTAL R ET AL: "Visual-Inertial Monocular SLAM With Map Reuse", 《 IEEE ROBOTICS AND AUTOMATION LETTERS》 * |
| TONG QIN ET AL: "《VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator》", 《IEEE TRANSACTIONS ON ROBOTICS》 * |
| 陈慧岩 等: "《智能车辆理论与应用》", 31 July 2018, 北京理工大学出版社 * |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110060343B (en) * | 2019-04-24 | 2023-06-20 | 阿波罗智能技术(北京)有限公司 | Map construction method and system, server and computer readable medium |
| CN110060343A (en) * | 2019-04-24 | 2019-07-26 | 百度在线网络技术（北京）有限公司 | Map constructing method and system, server, computer-readable medium |
| CN110298921A (en) * | 2019-07-05 | 2019-10-01 | 青岛中科智保科技有限公司 | The construction method and processing equipment of three-dimensional map with personage's semantic information |
| CN112308904B (en) * | 2019-07-29 | 2024-07-02 | 北京初速度科技有限公司 | Vision-based image construction method and device and vehicle-mounted terminal |
| CN112308904A (en) * | 2019-07-29 | 2021-02-02 | 北京初速度科技有限公司 | Vision-based drawing construction method and device and vehicle-mounted terminal |
| CN110749308B (en) * | 2019-09-30 | 2021-10-29 | 浙江工业大学 | SLAM-oriented outdoor positioning method using consumer-grade GPS and 2.5D building models |
| CN110749308A (en) * | 2019-09-30 | 2020-02-04 | 浙江工业大学 | SLAM-oriented outdoor positioning method using consumer-grade GPS and 2.5D building models |
| CN112539755B (en) * | 2019-11-22 | 2024-04-09 | 百度（美国）有限责任公司 | Updating point cloud registration pipeline based on ADMM algorithm for autonomous vehicles |
| CN112539755A (en) * | 2019-11-22 | 2021-03-23 | 百度（美国）有限责任公司 | Updating point cloud registration pipeline based on ADMM algorithm for autonomous vehicle |
| CN111080784A (en) * | 2019-11-27 | 2020-04-28 | 贵州宽凳智云科技有限公司北京分公司 | Ground three-dimensional reconstruction method and device based on ground image texture |
| CN111080784B (en) * | 2019-11-27 | 2024-04-19 | 贵州宽凳智云科技有限公司北京分公司 | Ground three-dimensional reconstruction method and device based on ground image texture |
| CN110956697A (en) * | 2019-12-20 | 2020-04-03 | 上海有个机器人有限公司 | Laser slam-based memory optimization method, medium, terminal and device |
| CN110956697B (en) * | 2019-12-20 | 2023-05-05 | 上海有个机器人有限公司 | Memory optimization method, medium, terminal and device based on laser slam |
| CN111738087A (en) * | 2020-05-25 | 2020-10-02 | 完美世界(北京)软件科技发展有限公司 | Method and device for generating face model of game role |
| CN111738087B (en) * | 2020-05-25 | 2023-07-25 | 完美世界(北京)软件科技发展有限公司 | Method and device for generating face model of game character |
| CN111698422A (en) * | 2020-06-10 | 2020-09-22 | 百度在线网络技术（北京）有限公司 | Panoramic image acquisition method and device, electronic equipment and storage medium |
| CN111698422B (en) * | 2020-06-10 | 2022-01-25 | 百度在线网络技术（北京）有限公司 | Panoramic image acquisition method and device, electronic equipment and storage medium |
| CN112184768B (en) * | 2020-09-24 | 2023-10-31 | 杭州易现先进科技有限公司 | SFM reconstruction method and device based on laser radar and computer equipment |
| CN112184768A (en) * | 2020-09-24 | 2021-01-05 | 杭州易现先进科技有限公司 | SFM reconstruction method and device based on laser radar and computer equipment |
| CN113178000B (en) * | 2021-03-26 | 2022-06-24 | 杭州易现先进科技有限公司 | Three-dimensional reconstruction method and device, electronic equipment and computer storage medium |
| CN113178000A (en) * | 2021-03-26 | 2021-07-27 | 杭州易现先进科技有限公司 | Three-dimensional reconstruction method and device, electronic equipment and computer storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109461208B (en) | 2022-12-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109461208A (en) | Three-dimensional map processing method, device, medium and calculating equipment | |
| US11313684B2 (en) | Collaborative navigation and mapping | |
| US11165959B2 (en) | Connecting and using building data acquired from mobile devices | |
| CN109307508B (en) | Panoramic inertial navigation SLAM method based on multiple key frames | |
| CN106840148B (en) | Wearable positioning and path guiding method based on binocular camera under outdoor working environment | |
| CN109087359A (en) | Pose determines method, pose determining device, medium and calculates equipment | |
| US10657714B2 (en) | Method and system for displaying and navigating an optimal multi-dimensional building model | |
| Zollmann et al. | Augmented reality for construction site monitoring and documentation | |
| CN107888828A (en) | Space-location method and device, electronic equipment and storage medium | |
| CN102609942B (en) | Depth map is used to carry out mobile camera location | |
| CN104106262B (en) | Head pose tracking using a depth camera | |
| Chen et al. | Rise of the indoor crowd: Reconstruction of building interior view via mobile crowdsourcing | |
| CN109084732A (en) | Positioning and air navigation aid, device and processing equipment | |
| US11557083B2 (en) | Photography-based 3D modeling system and method, and automatic 3D modeling apparatus and method | |
| CN108682027A (en) | VSLAM realization method and systems based on point, line Fusion Features | |
| CN108051002A (en) | Transport vehicle space-location method and system based on inertia measurement auxiliary vision | |
| CN110084832A (en) | Correcting method, device, system, equipment and the storage medium of camera pose | |
| CN108805917A (en) | Sterically defined method, medium, device and computing device | |
| US20200309534A1 (en) | Systems and methods for robust self-relocalization in a pre-built visual map | |
| CN111127524A (en) | Method, system and device for tracking trajectory and reconstructing three-dimensional image | |
| US20220398775A1 (en) | Localization processing service | |
| US11651569B2 (en) | System and method for mapping | |
| Oskiper et al. | Stable vision-aided navigation for large-area augmented reality | |
| CA3069813C (en) | Capturing, connecting and using building interior data from mobile devices | |
| TW202238449A (en) | Indoor positioning system and indoor positioning method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20190619 Address after: 311200 Room 102, 6 Blocks, C District, Qianjiang Century Park, Xiaoshan District, Hangzhou City, Zhejiang Province Applicant after: Hangzhou Yixian Advanced Technology Co.,Ltd. Address before: 310052 Building No. 599, Changhe Street Network Business Road, Binjiang District, Hangzhou City, Zhejiang Province, 4, 7 stories Applicant before: NETEASE (HANGZHOU) NETWORK Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |